Biological state guidance device, biological state guidance method, and storage medium

Abstract

This invention provides a biological state guidance device. The biological state guidance device includes: a biological state inference unit that infers the biological state of a vehicle driver; a target value determination unit that determines a target value, which is an index for setting the driver's biological state as a target state; an acoustic stimulus generation unit that, based on the inferred biological state and the target value, controls an acoustic stimulus generation device to generate acoustic stimuli to guide the driver's biological state towards the target value; a passenger information acquisition unit that acquires passenger information related to passengers in the vehicle other than the driver; a target state determination unit that, based on the passenger information, determines a target state of the passenger's biological state; and a vibration stimulus generation unit that controls a vibration stimulus generation device to stimulate the passenger through vibration, the vibration stimulus generation unit generating the vibration to guide the passenger's biological state to the target state.

Description

Technical Field

[0001] This invention relates to a biological state guidance device, a biological state guidance method, and a storage medium. Background Technology

[0002] In recent years, efforts to provide access to transportation systems that are sustainable and also cater to vulnerable groups among transportation participants have been increasingly active. To achieve this goal, research and development efforts are focused on further improving the safety and convenience of transportation through technologies related to preventative safety. Against this backdrop, for example, Japanese Patent Application Publication No. 2019-214334 discloses a technology that, when detecting the level of brain activation in a vehicle occupant and determining that the activation level is inappropriate, guides the current heart rate to a target heart rate via sound or vibration to make the activation state appropriate. Summary of the Invention

[0003] Regarding preventative safety technologies, existing technologies also employ similar methods to provide passengers with guiding stimuli aimed at activating the driver's biological state. However, the guiding stimuli provided to the driver may not be appropriate for the passengers. For example, while it may be desirable to provide the driver with guiding stimuli that promote wakefulness, it may also be desirable to provide the child, who is a passenger, with guiding stimuli that promote sleep.

[0004] The present invention was made in consideration of such circumstances, and one of its objectives is to provide a biological state guidance device, biological state guidance method, and storage medium that, in addition to guiding the driver's biological state to an appropriate state, also provides appropriate stimulation for passengers, thereby contributing to the development of a sustainable transportation system.

[0005] To address the aforementioned issues and achieve the aforementioned objectives, the present invention employs the following solution.

[0006] (1): A biological state guidance device according to one aspect of the present invention comprises: a biological state inference unit that infers the biological state of a driver of a vehicle; a target value determination unit that determines a target value, which is an index for setting the driver's biological state as a target state; an acoustic stimulus generation unit that controls an acoustic stimulus generation device based on the inferred biological state and the target value, the acoustic stimulus generation device generating an acoustic stimulus to guide the driver's biological state to approach the target value; a passenger information acquisition unit that acquires passenger information related to passengers of the vehicle other than the driver; a target state determination unit that determines a target state of the passenger's biological state based on the passenger information; and a vibration stimulus generation unit that controls a vibration stimulus generation device that stimulates the passenger by vibration, the vibration stimulus generation unit generating the vibration in order to guide the passenger's biological state to the target state.

[0007] (2): In the above (1) scheme, the vibration stimulation generating unit may change the vibration period based on the BPM (Beats Per Minute) of the sound stimulation.

[0008] (3): In the above (1) scheme, the vibration stimulation generating unit may set the vibration period to an integer multiple or an inverse multiple of the period determined based on the BPM (Beats Per Minute) of the sound stimulation.

[0009] (4): In the above scheme (3), the target state may include at least an awake state and a relaxed state. When the target state is the awake state, the vibration stimulation generating unit sets the vibration period to an integer multiple of the period determined by the BPM of the sound stimulation. When the target state is the relaxed state, the vibration stimulation generating unit sets the vibration period to an integer multiple of the period determined by the BPM of the sound stimulation.

[0010] (5): In the above (4) scheme, the target state determination unit can determine whether the passenger's eyes are open based on the passenger information, and if the eyes are closed, determine the target state as the relaxed state.

[0011] (6): In the above scheme (4), the target state determination unit may determine whether the passenger is active based on the passenger information, and if the passenger is active, determine the target state as the awake state.

[0012] (7): In the above (4) scheme, the target state determination unit determines whether the passenger is talking based on the passenger information. If it is determined that the passenger is talking, the target state is determined to be normal. The vibration stimulation generation unit sets the vibration period to the same degree as the period determined based on the BPM of the sound stimulation.

[0013] (8): In any of the above (1) to (7), the vibration stimulation generating unit may cause the vibration of the vibrating element provided on the seat where the passenger sits to vibrate.

[0014] (9): In any of the above schemes (1) to (8), the target value determination unit may also determine the target value based on the information selected by the driver, and the target state determination unit may also determine the target state based on the information selected by the passenger.

[0015] (10): In a biological state guidance method of one aspect of the present invention, an on-board computer performs the following processing: inferring the biological state of the driver of the vehicle, determining a target value, the target value being an indicator for setting the driver's biological state as a target state, controlling an acoustic stimulus generating device based on the inferred biological state and the target value, the acoustic stimulus generating device generating an acoustic stimulus to guide the driver's biological state in a manner that brings it close to the target value, obtaining passenger information related to other passengers in the vehicle besides the driver, determining a target state of the passenger's biological state based on the passenger information, and controlling a vibration stimulus generating device to stimulate the passenger through vibration, thereby guiding the passenger's biological state to the target state.

[0016] (11): A storage medium of one aspect of the present invention stores a program, wherein the program causes an on-board computer to perform the following processes: inferring the biological state of the driver of the vehicle, determining a target value, the target value being an indicator for setting the driver's biological state as a target state, controlling an acoustic stimulus generating device based on the inferred biological state and the target value, the acoustic stimulus generating device generating an acoustic stimulus in a manner that guides the driver's biological state to approach the target value, obtaining passenger information related to passengers of the vehicle other than the driver, determining a target state of the passenger's biological state based on the passenger information, and controlling a vibration stimulus generating device that stimulates the passenger through vibration, thereby guiding the passenger's biological state to the target state.

[0017] According to the schemes (1), (10) and (11) above, depending on the situation, sound stimulation may sometimes be inappropriate for the passenger. Even in such cases, vibration stimulation can guide the passenger to the appropriate target state.

[0018] According to the above scheme (2), even if the passenger hears the sound stimulation, it is difficult for them to feel discomfort from the vibration.

[0019] According to the above scheme (3), even if the passenger hears the sound stimulation, it is difficult for them to feel discomfort from the vibration.

[0020] According to the above scheme (4), by guiding the heart rhythm of the fellow passenger to the period of vibration, it is easy to guide the target state of the organism.

[0021] According to the above scheme (5), appropriate guidance stimulation can be provided even when the passenger is asleep.

[0022] According to the above scheme (6), the passenger may be in a state of wanting to activate or prone to motion sickness. In such cases, activation can be achieved by guiding the passenger to a state of wakefulness, or by making the sympathetic nervous system dominant, which can help reduce motion sickness.

[0023] According to the above scheme (7), when guiding fellow passengers to a normal state, it is possible to easily become a biological state of the same level as the driver.

[0024] According to the scheme in (8) above, it is possible to effectively stimulate fellow passengers.

[0025] According to the scheme in (9) above, the driver or passenger may sometimes expect a desired biological state, and in such cases, guiding stimuli can be given based on information selected by the driver or passenger. Attached Figure Description

[0026] Figure 1 This is a schematic block diagram illustrating the system structure of a vehicle system equipped with a biological state guidance device.

[0027] Figure 2 This is a diagram showing a specific example of passenger information.

[0028] Figure 3 This is a diagram illustrating a specific example of alertness.

[0029] Figure 4 This is a diagram illustrating a specific example of vibration generated by the vibration stimulation generating part of the seat vibrating component.

[0030] Figure 5This is a flowchart illustrating an example of the process by which a biological state guidance device provides biofeedback based on audio content to the driver.

[0031] Figure 6 This is a flowchart illustrating an example of the process for handling vibrations in the seat of a passenger using a bio-state guidance device.

[0032] Figure 7 This is a flowchart illustrating an example of a processing flow for determining the target state of a passenger corresponding to passenger information in a biological state guidance device. Detailed Implementation

[0033] Hereinafter, with reference to the accompanying drawings, embodiments of the organism state guidance device, organism state guidance method, and storage medium of the present invention will be described.

[0034] [Vehicle System]

[0035] Figure 1 This is a structural diagram of a vehicle system 1 equipped with the organism state guidance device 10 of the embodiment. The vehicle equipped with vehicle system 1 (hereinafter also referred to as the vehicle M) is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. When an electric motor is provided, the electric motor operates using electricity generated by a generator connected to the internal combustion engine or electricity discharged from a secondary battery or fuel cell. The vehicle equipped with vehicle system 1 includes driving control units such as an accelerator pedal, brake pedal, and steering wheel, a driving force output device, a braking device, a steering device, and a control unit (not shown), all typical of vehicles.

[0036] Vehicle system 1 includes, for example, a bio-state guidance device 10, vehicle sensors 20, an in-vehicle camera 22, a heart rate sensor 24, a perspiration sensor 26, a body temperature sensor 28, a seat vibration device 30, an audio system 32, and a navigation device 34. Occupants of vehicles equipped with vehicle system 1 may maintain portable terminals (smartphones) 38 and wearable terminals 40. These devices are interconnected via multiplexed communication lines such as CAN (Controller Area Network) or serial communication lines, wireless communication networks, etc. (hereinafter referred to only as the in-vehicle network 36). Figure 1 The structure shown is just one example; some structures can be omitted, and other structures can be added.

[0037] Vehicle sensors 20 include, for example, a vehicle speed sensor to detect the speed of the vehicle M, an acceleration sensor to detect acceleration, a yaw rate sensor to detect the angular velocity about a vertical axis, and an orientation sensor to detect the direction of the vehicle M. Vehicle sensors 20 can also detect the magnitude of vibrations experienced by the moving vehicle M from the road surface (not shown).

[0038] Vehicle sensors 20 may also include control sensors (not shown) such as throttle opening sensors, brake sensors, steering sensors, and grip sensors found in the driving control unit. The throttle opening sensor detects the amount of throttle pedal operation (throttle opening). The brake sensor detects the amount of brake pedal operation. The brake sensor may detect the amount of brake pedal depressor input, for example, based on changes in the brake pedal or the hydraulic pressure of the master cylinder of the brake system. The steering sensor detects the amount of steering wheel operation. The steering sensor may be located on the steering shaft and detect the amount of steering wheel operation based on the rotation angle of the steering shaft. The steering sensor may also detect steering torque and detect the amount of steering wheel operation based on the detected steering torque. The grip sensor may be an electrostatic capacitance sensor positioned along the circumference of the steering wheel. The grip sensor detects changes in electrostatic capacitance as a result of contact between an object (the occupant's hand) and the area of ​​the detected object. These vehicle sensors 20 transmit the acquired sensor information to the bio-state guidance device 10 via the vehicle network 36.

[0039] The in-vehicle camera 22 is, for example, a digital camera utilizing a solid-state imaging element such as a CCD or CMOS. The in-vehicle camera 22 is installed at a location capable of photographing the occupants of the vehicle M (e.g., the occupant seated in the driver's seat, hereinafter referred to as the driver; or the occupant seated in the front passenger seat or rear seat, hereinafter referred to as the passenger). The in-vehicle camera 22, for example, photographs the area of ​​the subject at predetermined intervals and transmits the captured images to the bio-state guidance device 10 via the vehicle network 36. The in-vehicle camera 22 may also be a stereo camera.

[0040] The heart rate sensor 24 detects the occupant's heart rate as a biological state. The heart rate sensor 24 can be, for example, a heart rate sensor installed on the steering wheel, a millimeter-wave radar installed in the vehicle interior, a pressure sensor installed on the seatbelt, a heart rate sensor installed in a child seat, or an ultrasonic sensor, etc. The heart rate sensor 24 transmits the electrical signal corresponding to the detected occupant's heart rate to the biological state guidance device 10 via the vehicle network 36.

[0041] The sweat sensor 26 detects the amount of sweat produced by the occupant as a biological state. The sweat sensor 26 is, for example, installed on the steering wheel, and detects the amount of sweat produced by the driver holding the steering wheel while driving. The sweat sensor 26 sends an electrical signal corresponding to the detected amount of sweat produced by the occupant to the biological state guidance device 10 via the vehicle network 36.

[0042] The body temperature sensor 28 detects the occupant's body temperature as a biological state. The body temperature sensor 28 may be, for example, a thermal imaging camera installed inside the vehicle. The body temperature sensor 28 transmits an electrical signal corresponding to the detected occupant's body temperature to the biological state guidance device 10 via the vehicle network 36.

[0043] The seat vibration element 30 is installed in part of the seat (seat) of the vehicle M (seat cushion supporting the occupant's buttocks or legs, seat back supporting the occupant's waist or back, headrest supporting the occupant's head or neck, etc.). The seat vibration element 30 causes the seat to vibrate by generating vibrations corresponding to electrical signals received via the vehicle network 36.

[0044] The audio system (car audio) 32 includes, for example, an operation unit, a media storage unit, a playback unit, and a modulation unit, and stores music information in a storage device such as an HDD (Hard Disk Drive) or flash memory. The audio system 32 is connected to external speakers (not shown). The audio system 32 plays and outputs music information stored in the storage device or music information stored in the media storage unit to external speakers via operation of the operation unit by the occupant or control signals from the bio-state guidance device 10 via the vehicle network 36. The audio system can change and play the BPM (Beats Per Minute) or pitch of the music information via the modulation unit. The audio system 32 can also obtain music information from an external server via a communication device installed in the vehicle.

[0045] The navigation device 34 includes, for example, a GNSS (Global Navigation Satellite System) receiver, a navigation HMI (Human Machine Interface), and a route determination unit, and stores map information (not shown) in a storage device such as an HDD (Hard Disk Drive) or flash memory. The GNSS receiver determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M can also be determined or supplemented by an INS (Inertial Navigation System) utilizing the output of the vehicle sensor 20. The navigation HMI includes a display device, a speaker, a touch panel, keys, etc. The route determination unit, for example, refers to map information to determine the path (hereinafter referred to as the map path) from the position of the vehicle M determined by the GNSS receiver (or any input position) to the destination input by the occupant using the navigation HMI. The map information includes, for example, information showing the shape of roads by representing road links and nodes connected by the links. The map information may also include road curvature or POI (Point of Interest) information, etc. The navigation device 34 can also provide route guidance using the navigation HMI based on the map path determined by the route determination unit. The location information of the vehicle M determined by the GNSS receiver, the information related to the destination input by the occupants, and the information of the route on the map determined by the route determination unit are output to the biological status guidance device 10 via the vehicle network 36.

[0046] The navigation device 34 can also be implemented using the functions of a terminal device such as a smartphone or tablet owned by the occupant. The navigation device 34 can also send its current location and destination to a navigation server via a communication device mounted on the vehicle, and obtain the route on the map returned by the navigation server. The vehicle system 1 can also have a GNSS receiver (GPS receiver) in addition to the navigation device 34 to obtain the location information of the vehicle M.

[0047] The portable terminal (smartphone) 38 includes, for example, an auxiliary storage unit consisting of a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), and flash memory, a communication unit, a display unit, an input unit, and various sensors. The portable terminal 38 uses the CPU to read various programs stored in the auxiliary storage unit, etc., into the RAM and execute them to achieve the desired functions. The portable terminal 38 communicates with the bio-state guidance device 10 via wireless communication, for example, sending various information about the owner of the portable terminal 38, i.e., the occupant, and information obtained from various sensors. The portable terminal 38 can also perform the functions of various vehicle sensors 20, audio devices 32, and navigation devices 34 in the vehicle system 1.

[0048] The wearable terminal 40 is, for example, a wristwatch-type portable terminal, equipped with an auxiliary storage unit consisting of a CPU, RAM, ROM, and flash memory, a communication unit, a display unit, an input unit, and various sensors. The wearable terminal 40 uses its CPU to read various programs stored in the auxiliary storage unit, etc., into RAM and execute them to achieve the desired functions. The wearable terminal 40 communicates with the bio-state guidance device 10 via wireless communication, for example, sending various information about the owner (occupant) of the wearable terminal 40 and information obtained from various sensors. The wearable terminal 40 can also function as various vehicle sensors 20, heart rate sensor 24, perspiration sensor 26, body temperature sensor 28, audio device 32, and navigation device 34 in the vehicle system 1.

[0049] [Bio-body state guidance device]

[0050] Next, the organism state guidance device 10 will be described in detail. The organism state guidance device 10 is an information processing device capable of communication, such as an in-vehicle computer. The organism state guidance device 10 includes a communication unit 11, an input unit 13, an output unit 15, a position information detection unit 16, a storage unit 17, and a control unit 19.

[0051] Communication unit 11 is a communication device. Communication unit 11 communicates with other devices or apparatuses via vehicle network 36. In addition, communication unit 11 communicates with portable terminal 38 or wearable terminal 40 via wireless communication.

[0052] The input unit 13 is an input device such as a button or touch panel. The input unit 13 is operated by the occupant when inputting instructions to the bio-state guidance device 10. The input unit 13 may also be an interface for connecting an input device to the bio-state guidance device 10. In this case, the input unit 13 inputs the input signal generated in the input device based on the occupant's input to the bio-state guidance device 10. The bio-state guidance device 10 may also use a portable terminal 38 or a wearable terminal 40 as an input device. In this case, the communication unit 11 (or the input unit 13) inputs the input signal generated in these terminals based on the occupant's input to the bio-state guidance device 10.

[0053] The output unit 15 may be, for example, an image display device such as a liquid crystal display or an organic EL (Electro-Luminescent) display. The output unit 15 may also output to an external speaker connected to the audio device 32; in this case, the output signal is output to the audio device 32 via the communication unit 11 (or the output unit 15). The output unit 15 outputs information to the occupants. The input unit 13 and the output unit 15 may also be configured as an integrated touch panel display.

[0054] The location information detection unit 16 detects (acquires) location information indicating the current location of the bio-state guidance device 10 (the vehicle M). The location information detection unit 16 may, for example, be configured using a GPS (Global Positioning System) receiver. The location information detection unit 16 may also be configured to detect location information based on the received signal strength from multiple base stations used for mobile communication. In this case, some of the functions of the location information detection unit 16 may also be installed in the control unit 19. The bio-state guidance device 10 may also acquire location information indicating its current location from the navigation device 34, the GNSS receiver (GPS receiver) provided in the vehicle system 1, the portable terminal 38, or the wearable terminal 40.

[0055] Storage unit 17 is a storage device such as a magnetic hard disk drive or a semiconductor storage device. Storage unit 17 stores information required for processing by control unit 19. Storage unit 17 includes at least a passenger information storage unit 171 and an identification information storage unit 172.

[0056] Passenger information storage unit 171 stores information related to passengers other than the driver among the occupants (hereinafter also referred to as passenger information). Figure 2This diagram illustrates an example of passenger information 201. Specifically, passenger information 201 includes, for each passenger, identification information (identification ID, passenger's name), the biological state of the target being the passenger (i.e., the target state), and parameters for a recognizer pre-learned based on multiple image data of the passenger. Target states can be categorized as, for example, "relaxed state," "normal state," and "awake state." Passenger information may not be set for each passenger; conversely, even if passenger information is available, a target state may not be set.

[0057] exist Figure 2 In the examples, for Mr. / Ms. "A", the target organism's state is set to "awake state" for reasons such as being prone to motion sickness. For "B", who is a toddler, the target organism's state is set to "relaxed state". For Mr. / Ms. "C", the target organism's state is set to "normal state". For Mr. / Ms. "D", no target organism state is set.

[0058] The identification information storage unit 172 stores recognition parameters learned in advance from the image data of the person to identify the state of the passenger. For example, these are recognition parameters for identifying whether the person is closing their eyes, recognizing whether they are talking, and recognizing whether they are playing games or using a smartphone, reading, or watching videos on a monitor.

[0059] The control unit 19 is a processor such as a CPU and a memory. The control unit 19 executes programs via the processor and functions as a target value determination unit 191, a biological state inference unit 192, an acoustic stimulus generation unit 193, a wakefulness determination unit 194, a passenger information acquisition unit 195, a target state determination unit 196, and a vibration stimulus generation unit 197. All or part of the functions of the control unit 19 can also be implemented using hardware such as ASICs, PLDs, and FPGAs. The aforementioned programs can also be recorded on a computer-readable recording medium. Computer-readable recording media include, for example, removable media such as floppy disks, optical disks, ROMs, CD-ROMs, and semiconductor storage devices (e.g., SSDs), as well as storage devices such as hard disks or semiconductor storage devices built into a computer system. The aforementioned programs can also be transmitted via electrical communication lines.

[0060] The target value determination unit 191 determines the target value of alertness related to the driver's biological state. Figure 3This diagram illustrates an example of alertness level 301. "Awareness" is categorized into six stages: "0," "1," "2," "3," "4," and "5," corresponding to the "organismal state" of "inattentive," "relaxed," "neutral," "focused," "awake," and "tense," respectively. In the example of alertness level 301, the lower the "awareness," the closer the "organismal state" is to "inattentive," and the higher the "awareness," the closer the "organismal state" is to "tense." The target value determination unit 191 can also determine the target value of alertness related to the driver's biological state based on various conditions, such as information related to the destination or information related to the driver, and can set the target value of alertness to, for example, "2," which represents "neutral."

[0061] The bio-state inference unit 192 infers the driver's alertness (hereinafter also referred to as driver alertness) based on information obtained from various sensors (20, 24, 26, 28) or the in-vehicle camera 22. The inferred alertness is used in conjunction with... Figure 3 The alertness level 301 shown is the same index. For example, the bio-state inference unit 192 obtains the driver's heart rate from the heart rate sensor 24, divides the heart rate into six stages, and infers the corresponding alertness level. Additionally, for example, the bio-state inference unit 192 obtains the driver's perspiration amount from the perspiration sensor 26, divides the perspiration amount into six stages, and infers the corresponding alertness level. Additionally, for example, the bio-state inference unit 192 obtains the driver's body temperature from the body temperature sensor 28, divides the body temperature into six stages, and infers the corresponding alertness level. Furthermore, for example, the bio-state inference unit 192 obtains eye movement information from the driver's image obtained by the in-vehicle camera 22, divides the amount of eye movement into six stages, and infers the corresponding alertness level.

[0062] Additionally, for example, the biological state inference unit 192 obtains information such as the speed or acceleration of the vehicle M, yaw rate, throttle opening, braking operation amount, steering wheel operation amount, corrected steering angle, and steering angular velocity from the vehicle sensor 20, divides this information into six stages, and infers the corresponding level of alertness. The information from the vehicle sensor 20 is not information directly related to the driver's biological state, but it is considered to be related to the driver's biological state.

[0063] The biological state inference unit 192 can also infer the driver's alertness based on multiple inferred alertness levels. For example, the biological state inference unit 192 can also take the average or weighted sum of multiple alertness levels to infer the driver's alertness. The biological state inference unit 192 can also obtain information such as heart rate, perspiration, and body temperature from the wearable terminal 40.

[0064] The sound stimulation generation unit 193 generates sound stimuli in a manner that guides the driver's level of alertness, as inferred by the organism state inference unit 192, to approach the target level of alertness determined by the target value determination unit 191. Specifically, when the driver's level of alertness is lower than the target value, the sound stimulation generation unit 193 selects and generates (plays) alertness-enhancing sound content as a stimulus to awaken the driver. When the driver's level of alertness is higher than the target value, the sound stimulation generation unit 193 selects and generates (plays) relaxation-enhancing sound content as a stimulus to relax the driver. When the driver's level of alertness is the same as the target value, the sound stimulation generation unit 193 performs no processing.

[0065] As an audio stimulus, the audio stimulus generating unit 193 can, for example, send a signal to the audio device 32 to play music at an effective BPM (Beats Per Minute), or modulate (change the playback pitch) the currently playing music and make it play. As audio content for alertness, the audio stimulus generating unit 193 can, for example, make the audio device 32 play music at a higher BPM, or increase the BPM or pitch by a predetermined percentage based on the difference between the target value and the current alertness level, and make the music play. As audio content for relaxation, the audio stimulus generating unit 193 can, for example, make the audio device 32 play music at a lower BPM, or decrease the BPM or pitch by a predetermined percentage based on the difference between the target value and the current alertness level, and make the music play. For example, when using heart rate as a measure of alertness, the audio stimulus generating unit 193 can also use the normal or current heart rate as a reference, increasing the BPM or pitch by a predetermined percentage in audio content for alertness, and decreasing the BPM or pitch by a predetermined percentage in audio content for relaxation, and making the audio device 32 play music. The sound stimulation generating unit 193 can also send control signals to the portable terminal 38 or the wearable terminal 40 instead of the sound device 32, and still play music.

[0066] The alertness determination unit 194 generates an audio stimulus (plays audio content) through the audio stimulus generation unit 193, thereby determining whether the driver's alertness is the target alertness value determined by the target value determination unit 191.

[0067] The passenger information acquisition unit 195 acquires information related to the passenger. Specifically, the passenger information acquisition unit 195 controls the in-vehicle camera 22 to acquire image data of passengers other than the driver, and detects whether a passenger is traveling in the vehicle. The passenger information acquisition unit 195 acquires passenger information from the passenger information storage unit 171. The passenger information includes, for each passenger, identification information (identification ID or passenger's name), the biological state of the target passenger (target state), and parameters for the identifier learned in advance based on multiple image data of the passenger. Then, the passenger information acquisition unit 195 uses the acquired image data and the identifier parameters for each passenger to identify (determine) the passenger using the identifier provided by the biological state guidance device 10. Then, the passenger information acquisition unit 195 acquires information about the target state corresponding to the identified passenger. If no target state is set for the identified passenger, the passenger information acquisition unit 195 cannot acquire the target state information. The method for identifying passengers by the passenger information acquisition unit 195 described above is one example, and is not limited to this method. The passenger information acquisition unit 195 may be equipped with an identifier, the target status determination unit 196 described below may be equipped with an identifier, or other components may be equipped with an identifier.

[0068] The target state determination unit 196 determines the target state of the biological organism, i.e., the target state, based on the current state of the passenger. Specifically, when the passenger information acquisition unit 195 can obtain information about the passenger's target state, the target state determination unit 196 determines the obtained target state as the passenger's target state. When the passenger information acquisition unit 195 cannot obtain information about the passenger's target state, the target state determination unit 196 uses the acquired image data of the passenger and the recognition parameters obtained from the recognition information storage unit 172 to infer the passenger's current state using the recognition device provided by the biological state guidance device 10. Then, the target state determination unit 196 sets the target state to "relaxed state" when the passenger has their eyes open, "normal state" when the passenger is talking, "awake state" when the passenger is active, and "relaxed state" in all other cases. The above-described combination of the inferred current state of the passenger and the target state is an example, and is not limited to this. For example, in determining whether passengers are talking, the target state determination unit 196 can also use sound information that can be obtained from the microphone of the portable terminal 38 or the wearable terminal 40.

[0069] The vibration stimulation generating unit 197 causes the seat to vibrate by vibrating the seat vibrating member 30 installed on the seat of the passenger at an effective period. Specifically, when the target biological state of the passenger is "awake", the vibration stimulation generating unit 197 generates vibration by setting the vibration period of the seat vibrating member 30 to a multiple of an integer relative to the period determined by the BPM or pitch of the sound content played by the sound stimulation generating unit 193. This is a short-period vibration, which is a vibration for waking up. On the other hand, when the target biological state of the passenger is "relaxed", the vibration stimulation generating unit 197 generates vibration by setting the vibration period of the seat vibrating member 30 to an integer multiple relative to the period determined by the BPM or pitch of the sound content played by the sound stimulation generating unit 193. This is a long-period vibration, which is a vibration for relaxation. Furthermore, when the target biological state of the passenger is in a "normal state," the vibration stimulation generating unit 197 sets the vibration period of the seat vibrator 30 to be the same as the period determined by the BPM or pitch of the audio content played by the audio stimulation generating unit 193. In this case, the vibration stimulation generating unit 197 may also not cause the seat vibrator 30 to vibrate.

[0070] Figure 4 This diagram illustrates specific examples of vibrations generated by the seat vibrator 30 due to the vibration stimulation generating unit 197. The normal seat vibration 402 is a vibration with the same period as the period of the sound content played by the sound stimulation generating unit 193. The waking seat vibration 401 is an example of a vibration with a period half that of the normal seat vibration 402. The relaxation seat vibration 403 is an example of a vibration with a period twice that of the normal seat vibration 402. The seat vibration is generated once per cycle.

[0071] [Processing Flow]

[0072] Figure 5 , Figure 6 and Figure 7 This is a flowchart illustrating an example of the processing flow of the organism state guidance device 10. Figure 5 This is a flowchart illustrating an example of a process for providing biofeedback based on audio content to a driver. Figure 6 yes Figure 5 The following is a flowchart illustrating an example of the process for handling vibrations in the seat used by a passenger. Figure 7 This is a flowchart illustrating an example of a processing flow for determining the target status of a passenger corresponding to passenger information. Figure 5 and Figure 6 ( Figure 7 The processing is repeated at a prescribed cycle.

[0073] First, the target value determination unit 191 sets the target value of alertness related to the driver's biological state to, for example, "2" (step S101). The target value determination unit 191 can also determine the target value of alertness related to the driver's biological state based on various conditions such as information related to the destination or information related to the driver.

[0074] Next, the organism state inference unit 192 obtains or considers information related to the organism state from the heart rate sensor 24, sweating sensor 26, body temperature sensor 28, in-vehicle camera 22, various vehicle sensors 20, or wearable terminal 40 (step S102).

[0075] Next, the biological state inference unit 192 infers the driver's level of alertness related to the biological state based on the obtained information related to the biological state (step S103). For example... Figure 3 As shown, for example, six stages are used to infer alertness.

[0076] Next, the sound stimulation generating unit 193 determines whether the driver's alertness level is lower than the target value (step S104). If it is lower than the target value (yes in step S104), the sound stimulation generating unit 193 selects alertness sound content (step S105). On the other hand, if it is not lower than the target value, the sound stimulation generating unit 193 determines whether the driver's alertness level is higher than the target value (step S106). If it is higher than the target value (yes in step S106), the sound stimulation generating unit 193 selects relaxation sound content (step S107). On the other hand, if the driver's alertness level is the same as the target value, the control unit 19 of the biological state guidance device 10 ends the process.

[0077] Next, the sound stimulation generating unit 193 plays the selected sound content to stimulate the driver (step S108). As sound content for alertness, the sound stimulation generating unit 193, for example, causes the sound device 32 to play music at a higher BPM, or increases the BPM or pitch by a predetermined percentage based on the difference between the target value and the current alertness level. As sound content for relaxation, the sound stimulation generating unit 193, for example, causes the sound device 32 to play music at a lower BPM, or decreases the BPM or pitch by a predetermined percentage based on the difference between the target value and the current alertness level. Afterwards, the control unit 19 of the bio-state guidance device 10 shifts control to... Figure 6 Step S201.

[0078] Next, the passenger information acquisition unit 195 controls the in-vehicle camera 22 to acquire image data of passengers other than the driver, and detects whether a passenger is riding in the vehicle (step S201). If no passenger is riding in the vehicle, the control unit 19 of the bio-state guidance device 10 ends the process. On the other hand, if a passenger is riding in the vehicle ("Yes" in step S202), the passenger information acquisition unit 195 performs processing to determine the target state corresponding to the passenger information (step S203).

[0079] Move to Figure 7 The passenger information acquisition unit 195 acquires passenger information from the passenger information storage unit 171 (step S301). Figure 2 As illustrated, passenger information for each passenger includes information that identifies the passenger (identification ID or passenger's name), the biological state of the target being the passenger, i.e., the target state, and parameters for the recognizer that have been learned in advance based on multiple image data of the passenger.

[0080] Next, the passenger information acquisition unit 195 uses the acquired image data and the recognition parameters of each passenger to identify (determine) the passenger using the recognition device provided by the biometric state guidance device 10 (step S302). Then, the passenger information acquisition unit 195 acquires information about the target state corresponding to the identified passenger. If no target state is set for the identified passenger, the passenger information acquisition unit 195 cannot acquire the target state information.

[0081] Next, the target status determination unit 196 determines whether the passenger information acquisition unit 195 can acquire the passenger's target status information (step S303). If it can acquire the information ("Yes" in step S303), the target status determination unit 196 determines the acquired target status (the target status set in the passenger information) as the passenger's target status (step S304).

[0082] On the other hand, if the image data of the passenger cannot be obtained, the target state determination unit 196 uses the acquired image data of the passenger and the recognition parameters obtained from the recognition information storage unit 172 to infer whether the passenger is closing their eyes using the recognition device provided by the biometric state guidance device 10 (step S305). If the passenger is closing their eyes ("yes" in step S305), the target state determination unit 196 determines the target state of the passenger as "relaxed state" (step S306).

[0083] On the other hand, if the passenger's eyes are not closed, the target state determination unit 196 uses the acquired image data of the passenger and the recognition parameters obtained from the recognition information storage unit 172 to infer whether the passenger is talking using the recognition device provided by the biometric state guidance device 10 (step S307). In determining whether the passenger is talking, the target state determination unit 196 may also use audio information that can be obtained from the microphone of the portable terminal 38 or the wearable terminal 40. If the passenger is talking ("yes" in step S307), the target state determination unit 196 determines the passenger's target state as "normal state" (step S308).

[0084] On the other hand, if the passenger is not talking, the target state determination unit 196 uses the acquired image data of the passenger and the identifier parameters obtained from the identification information storage unit 172 to infer whether the passenger is active using the identifier provided by the biometric state guidance device 10 (step S309). Being active includes, for example, playing a game or using a smartphone, reading a book, or watching videos on a monitor. If the passenger is active ("yes" in step S309), the target state determination unit 196 determines the passenger's target state as "awake state" (step S310).

[0085] On the other hand, when the passenger is not active, the target state determination unit 196 determines the target state of the passenger as "relaxed state" (step S311). If the target state of the passenger is determined, the control unit 19 of the biological state guidance device 10 returns control to Figure 6 Step S204.

[0086] Next, the vibration stimulation generating unit 197 determines whether the target biological state, i.e., the target state, of the passenger is "awake" (step S204). If the target state is "awake" ("yes" in step S204), the vibration stimulation generating unit 197 sets the vibration period of the seat vibration member 30 installed in the passenger's seat to a multiple of an integer relative to the period determined based on the BPM or pitch of the sound content played by the sound stimulation generating unit 193, and generates vibration (awake seat vibration) (step S205).

[0087] On the other hand, if the target state is not "awake state", the vibration stimulation generating unit 197 determines whether the target biological state of the passenger is "relaxed state" (step S206). If the target state is "relaxed state" ("yes" in step S206), the vibration stimulation generating unit 197 sets the vibration period of the seat vibration member 30 provided on the passenger's seat to an integer multiple relative to the period determined based on the BPM or pitch of the sound content played by the sound stimulation generating unit 193, and generates vibration (relaxation seat vibration) (step S207).

[0088] On the other hand, if the target state is not a "relaxed state," the vibration stimulation generating unit 197 considers the target biological state of the passenger to be a "normal state," and the vibration stimulation generating unit 197 sets the vibration period of the seat vibrator 30 to be the same as the period determined based on the BPM or pitch of the audio content played by the audio stimulation generating unit 193. In this case, the vibration stimulation generating unit 197 may also not cause the seat vibrator 30 to vibrate (step S208).

[0089] above, Figure 5 , Figure 6 and Figure 7 The processing of the bio-state guidance device 10 (processing of bio-feedback based on audio content to the driver, processing of the target state of the passenger corresponding to the passenger information, and processing of the vibration of the passenger seat) ends.

[0090] According to this embodiment, the target value determination unit 191 sets a target value for alertness, the biological state inference unit 192 infers the driver's alertness based on information related to the biological state, the audio stimulus generation unit 193 selects audio content based on the driver's alertness and the target value, and plays the audio content to the driver. The biological state inference unit 192 again infers the driver's alertness based on information related to the biological state, and the alertness determination unit 194 determines whether the driver's alertness is the target value. Then, the passenger information acquisition unit 195 identifies the passenger, the target state determination unit 196 determines the target biological state of the passenger based on the passenger's state, and the vibration stimulus generation unit 197 vibrates the passenger's seat according to the target state, so as to coordinate with the period of the played audio content.

[0091] Therefore, in addition to guiding the driver's biological state to an appropriate state, it is also possible to provide appropriate stimulation for passengers. Even when audio content is inappropriate for passengers, the vibration of the seat can still guide them to an appropriate target state. Passengers are unlikely to feel discomfort from the seat vibration even when listening to audio content. By guiding the passenger's heart rate to the vibration cycle, it is easy to guide them towards the target biological state. Appropriate guidance stimulation can be provided even when passengers are asleep. Passengers may be in a state of wanting to activate or prone to motion sickness. In such cases, activation can be achieved by guiding them towards a state of wakefulness, or motion sickness can be reduced by making the sympathetic nervous system dominant. When passengers are guided to a normal state, they can easily achieve the same level of biological state as the driver. Sometimes, either the driver or passengers desire a specific biological state; in such cases, guidance stimulation can be provided based on information chosen by the driver or passengers.

[0092] The embodiments described above illustrate the methods for implementing the present invention, but the present invention is not limited to these embodiments in any way, and various modifications and substitutions can be made without departing from the spirit of the present invention.

[0093] For example, in the above processing flow, after the control unit 19 of the organism state guidance device 10 plays the audio content in step S108, it proceeds... Figure 6 The vibration of the seat can be used to control the vibration of the passenger's seat, but it could also be done after the audio content has been played. Figure 6 Simultaneously, the biological state inference unit 192 obtains information related to the driver's biological state from various sensors and infers the driver's level of alertness. The alertness determination unit 194 determines whether the driver's level of alertness is the same as the target value. If they are equal, the sound stimulus generation unit 193 stops the playback of the sound content.

[0094] While biofeedback based on audio content is provided to the driver, it can also be provided to passengers based on seat vibrations. To perform this biofeedback, in... Figure 6 After the processing of steps S205, S206 or S207, the biological information of the passenger (e.g., level of consciousness) is obtained. If the target state is not achieved, the same seat vibration is repeated.

[0095] For example, if the driver selects sobering audio content and the passenger also experiences sobering seat vibrations, the passenger may become overly alert. Therefore, seat vibrations can also be applied to the passenger only if the driver's target state of alertness or relaxation differs from the passenger's target state of alertness or relaxation.

[0096] The state of the passenger in steps S305, S307, and S309 is one example. The target state, or biological state, can also be determined based on other states. Of course, different target states can be determined for multiple passengers, resulting in different seat vibrations.

Claims

1. A biological state guidance device, wherein, The organism state guidance device includes: The organism state inference unit infers the organism state of the vehicle's driver; The target value determination unit determines the target value, which is an indicator used to set the driver's biological state as the target state. An audio stimulus generating unit controls an audio stimulus generating device based on the inferred biological state and the target value, the audio stimulus generating device generating audio stimuli in a manner that guides the driver's biological state to approach the target value. The passenger information acquisition unit acquires passenger information related to passengers in the vehicle other than the driver; The target state determination unit determines the target state of the biological state of the passenger based on the passenger information. as well as A vibration stimulation generating unit controls a vibration stimulation generating device that stimulates the passenger through vibration. The vibration stimulation generating unit generates the vibration in order to guide the biological state of the passenger to the target state. The vibration stimulation generating unit changes the vibration period based on the BPM of the sound stimulation, where BPM is the heart rate per minute.

2. The organism state guidance device according to claim 1, wherein, The vibration stimulation generating unit sets the vibration period to an integer multiple or a reciprocal multiple of the period determined based on the BPM of the sound stimulation.

3. The organism state guidance device according to claim 2, wherein, The target state includes at least a conscious state and a relaxed state. When the target state is the conscious state, the vibration stimulation generating unit sets the vibration period to a multiple of an integer multiple of the period determined based on the BPM of the sound stimulation. When the target state is the relaxed state, the vibration stimulation generating unit sets the vibration period to an integer multiple of the period determined based on the BPM of the sound stimulation.

4. The organism state guidance device according to claim 3, wherein, The target state determination unit determines whether the passenger's eyes are open based on the passenger information, and if the eyes are closed, determines the target state as the relaxed state.

5. The organism state guidance device according to claim 3, wherein, The target state determination unit determines whether the passenger is active based on the passenger information, and if the passenger is active, determines the target state as the conscious state.

6. The organism state guidance device according to claim 3, wherein, The target state determination unit determines whether the passenger is engaged in conversation based on the passenger information. If the passenger is engaged in conversation, the target state is determined to be a normal state. The vibration stimulation generating unit sets the vibration period to be the same as the period determined based on the BPM of the acoustic stimulation.

7. The organism state guidance device according to claim 1, wherein, The vibration stimulation generating unit causes the vibrating component installed on the seat where the passenger sits to vibrate, thereby generating the vibration.

8. The organism state guidance device according to any one of claims 1 to 7, wherein, The target value determination unit also determines the target value based on information selected by the driver. The target state determination unit also determines the target state based on information selected by the passenger.

9. A method for guiding the state of an organism, wherein, The onboard computer performs the following processing: Inferring the biological state of the vehicle's driver. A target value is determined, which is an indicator used to set the driver's biological state as the target state. Based on the inferred biological state and the target value, an acoustic stimulus generating device is controlled to generate acoustic stimuli that guide the driver's biological state to approach the target value. Obtain passenger information related to passengers in the vehicle other than the driver. Based on the information of the fellow passengers, the target state of the biological state of the fellow passengers is determined. A vibration stimulation generating device is controlled to stimulate the passenger through vibration, thereby guiding the passenger's biological state to the target state. The period of the vibration is changed based on the BPM of the sound stimulus, where BPM is the heart rate per minute.

10. A storage medium having a stored program, wherein, The program causes the onboard computer to perform the following processes: Inferring the biological state of the vehicle's driver. A target value is determined, which is an indicator used to set the driver's biological state as the target state. Based on the inferred biological state and the target value, an acoustic stimulus generating device is controlled to generate acoustic stimuli that guide the driver's biological state to approach the target value. Obtain passenger information related to passengers in the vehicle other than the driver. Based on the information of the fellow passengers, the target state of the biological state of the fellow passengers is determined. A vibration stimulation generating device is controlled to stimulate the passenger through vibration, thereby guiding the passenger's biological state to the target state. The period of the vibration is changed based on the BPM of the sound stimulus, where BPM is the heart rate per minute.

Images