Information processing systems, information processing methods, and programs

Abstract

To provide an information processing system and the like capable of offering a novel user experience with a highly healing effect.SOLUTION: According to an aspect of the present invention, an information processing system is provided. The information processing system includes at least one processor, and the processor is configured to execute a program so that each of the following steps is performed. In a setting step, theme information indicating at least one of a plurality of pre-managed themes is set. In a first presentation step, multimodal information corresponding to the theme information is presented to a user located in a closed space. The multimodal information is a combination of information comprising at least two of visual information, auditory information, tactile information, and olfactory information.SELECTED DRAWING: Figure 1

Description

【Technical Field】 【0001】 The present invention relates to an information processing system, an information processing method, and a program. 【Background Art】 【0002】 For example, Patent Document 1 discloses an environmental control system for a private room space that can easily relieve stress without acquiring special knowledge and has a low hurdle for system introduction. Such an environmental control system is installed in a private room space, and includes a lighting device capable of changing the illuminance or color temperature in the private room space, a video output device that outputs video in the private room space, and a communication device capable of transmitting control instructions to the lighting device and the video output device. The control device has a refresh control for executing lighting relaxation control for reducing the illuminance or color temperature in the private room space and video relaxation control for outputting a first video in the private room space, and then executing lighting awakening control for increasing the illuminance or color temperature in the private room space and video awakening control for outputting a second video different from the first video in the private room space. 【Prior Art Documents】 【Patent Documents】 【0003】 【Patent Document 1】 Japanese Patent Application Laid-Open No. 2020-053346 【Summary of the Invention】 【Problems to be Solved by the Invention】 【0004】 However, a new user experience having a healing effect, which is different from the technology disclosed in Patent Document 1, is required. 【0005】 In view of the above circumstances, the present invention aims to provide an information processing system or the like that can realize a new user experience with a high healing effect. 【Means for Solving the Problems】 【0006】 According to one aspect of the present invention, an information processing system is provided. This information processing system comprises at least one processor, which is configured to execute a program so that the following steps are performed: In the setting step, theme information is set, which indicates at least one of a plurality of pre-managed themes. In the first presentation step, multimodal information corresponding to the theme information is presented to a user located in a closed space. The multimodal information is a combination of information consisting of at least two of visual information, auditory information, tactile information, and olfactory information. 【0007】 This approach makes it possible to create a novel user experience with a high level of therapeutic effect. [Brief explanation of the drawing] 【0008】 [Figure 1] This is a configuration diagram showing the hardware configuration of the information processing system 1 according to the first embodiment. [Figure 2] This is a block diagram showing the functional configuration of an information processing device 2 according to one embodiment. [Figure 3] This is a schematic diagram showing the specific configuration of Information Processing System 1. [Figure 4] This is an activity diagram illustrating the overview of information processing. [Figure 5] These are examples of screens presented to user U. Figure 5A shows screen 8a corresponding to activity A104, and Figure 5B shows screen 8b corresponding to activity A109. [Figure 6] This is a schematic diagram illustrating an example of a method for presenting visual information (VI). [Figure 7] This document outlines the experimental protocol for the example used to verify the effectiveness of the method. [Figure 8] The experimental results of the example are shown. [Figure 9] The experimental results of the example are shown. [Figure 10] The experimental results of the example are shown. [Modes for carrying out the invention] 【0009】 Embodiments of the present invention will be described below with reference to the drawings. The various features shown in the embodiments below can be combined with each other. 【0010】 Incidentally, the program for implementing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or it may be provided as a downloadable medium from an external server, or it may be provided so that the program is launched on an external computer and its functions are realized on a client terminal (so-called cloud computing). 【0011】 Furthermore, in one embodiment, "part" may include, for example, hardware resources implemented by a circuit in a broad sense, and the information processing of software that can be specifically realized by these hardware resources. Also, in one embodiment, various types of information are handled, and this information can be represented, for example, by the physical values ​​of signal values ​​representing voltage and current, the high or low values ​​of signal values ​​as a set of binary bits composed of 0s or 1s, or by quantum superposition (so-called qubits), and communication and calculations can be performed on a circuit in a broad sense. 【0012】 Furthermore, a circuit in a broad sense is a circuit realized by combining at least a suitable combination of circuits, circuits, processors, and memory. In other words, it includes application-specific integrated circuits (ASICs), programmable logic devices (for example, simple programmable logic devices (SPLDs), complex programmable logic devices (CPLDs), and field programmable gate arrays (FPGAs)), etc. 【0013】 [Embodiment] 1. Hardware Configuration This section describes the hardware configuration of one embodiment. 【0014】 1.1 Information Processing System 1 FIG. 1 is a configuration diagram showing the hardware configuration of an information processing system 1 according to the first embodiment. The information processing system 1 includes, for example, an information processing device 2, an image output device 3, an audio output device 4, a tactile presentation device 5, an olfactory presentation device 6, and a wearable device 7, which are electrically connected. These components will be further described. Here, the system exemplified by the information processing system 1 consists of one or more devices or components. Therefore, note that even the information processing device 2 alone, or a combination of the information processing device 2 and at least two of the image output device 3, the audio output device 4, the tactile presentation device 5, and the olfactory presentation device 6, may be included in the information processing system 1. More specifically, the information processing system 1 may include elements selected from the group consisting of the information processing device 2 and components 3 to 7. Also, a plurality of information processing devices 2 may be used corresponding to each of the components 3 to 7. Even if the unselected elements are not included in the information processing system 1, they are electrically connected to the selected elements as external elements. 【0015】 1.2 Information Processing Device 2 The information processing device 2 has a communication unit 21, a storage unit 22, a control unit 23, and an input unit 24, and these components are electrically connected inside the information processing device 2 via a communication bus 20. These components will be further described. 【0016】 The communication unit 21 preferably uses wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., but may include wireless LAN network communication, mobile communication such as LTE / 5G, Bluetooth (registered trademark) communication, etc. as needed. That is, it is more preferable to implement it as a collection of these multiple communication means. That is, the information processing device 2 may communicate various information from the outside via the communication unit 21 and the network. Specifically, the information processing device 2 is connected to the image output device 3, the audio output device 4, the tactile presentation device 5, and the olfactory presentation device 6 via the communication unit 21 and each transmission path. Each of the components 3 to 6 will be described later. 【0017】 The storage unit 22 stores various information defined by the foregoing description. This can be implemented as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the information processing apparatus 2 executed by the control unit 23, or as a memory such as a random access memory (RAM) that stores temporarily necessary information (arguments, arrays, etc.) related to the operation of the program. The storage unit 22 stores various programs, variables, etc. related to the information processing apparatus 2 executed by the control unit 23. 【0018】 The control unit 23 (processor) performs processing and control of the overall operation related to the information processing apparatus 2. The control unit 23 is, for example, a central processing unit (CPU) not shown in the figure. The control unit 23 realizes various functions related to the information processing apparatus 2 by reading a predetermined program stored in the storage unit 22. That is, the information processing by software stored in the storage unit 22 is specifically realized by the control unit 23 which is an example of hardware, and can be executed as each functional unit included in the control unit 23. These will be described in more detail in the next section. Note that the control unit 23 is not limited to being single, and may be implemented to have a plurality of control units 23 for each function. Or combinations thereof may also be possible. In particular, the control unit 23 can generate a control signal for controlling the operations of the components 3 to 6 described later. In other words, the components 3 to 6 can be controlled by the control unit 23 (processor) of the information processing apparatus 2. 【0019】 The input unit 24 may be included in the housing of the information processing device 2, or it may be an external unit. For example, the input unit 24 may be integrated with a display (which may be the display 31 described later) and implemented as a touch panel. If it is a touch panel, the user can input tap operations, swipe operations, etc. Of course, instead of a touch panel, a switch button, mouse, QWERTY keyboard, microphone for voice input, etc. may be used. In other words, the input unit 24 receives operation input made by the user. This input is transmitted as a command signal to the control unit 23 via the communication bus 20, and the control unit 23 can execute predetermined controls and calculations as needed. 【0020】 1.3 Image Output Device 3 The image output device 3 is a device that can present visual information (hereinafter also referred to as visual information VI) to the user, and may display a graphical user interface (GUI) screen that the user can operate. The image output device 3 may be, for example, the display 31 described later, or a projector 32 (see Figure 3). If the image output device 3 is the display 31, it is preferable to use different display devices such as CRT displays, liquid crystal displays, organic EL displays, and plasma displays. 【0021】 1.4 Audio Output Device 4 The audio output device 4 is a device that can present auditory information (hereinafter also referred to as auditory information AI) to the user, and may be, for example, headphones 41 described later, or a speaker 42 (see Figure 3). The audio output device 4 should preferably be able to output sounds in a frequency range above the human audible range. As headphones 41, an open-air type that does not significantly block out external sounds when worn by the user should be adopted. Also, if the audio output device 4 is a speaker 42, the speaker 42 may be connected to the information processing device 2 via an amplifier (not shown), or it may be an active speaker with an amplifier built into the speaker 42 itself. Furthermore, the audio output device 4 may also be an ultrasonic generator (not shown). 【0022】 1.5 Tactile presentation device 5 The tactile presentation device 5 is a device that can present tactile information (hereinafter also referred to as tactile information TI) to the user. For example, it may be a fan 51 or an air conditioner 52 as described later (see Figure 3). Alternatively, the tactile presentation device 5 may be an ultrasonic generator (not shown) or a glove-type device. 【0023】 1.6 Olfactory presentation device 6 The olfactory presentation device 6 is a device that can present olfactory information (hereinafter also referred to as olfactory information OI) to the user, and may be, for example, the aroma diffuser 61 described later. 【0024】 In summary, the information processing system 1 comprises at least one image output device 3 that provides visual information VI, at least one sound output device 4 that provides auditory information AI, at least one tactile presentation device 5 that provides tactile information TI, and at least one olfactory presentation device 6 that provides olfactory information OI. The processor (e.g., control unit 23) is configured to control at least two devices among the image output device 3, sound output device 4, tactile presentation device 5, and olfactory presentation device 6 that act on different senses, thereby presenting multimodal information IF2. Multimodal information IF2 will be described in more detail later. According to this embodiment, a novel user experience can be realized as a composite device. Furthermore, as mentioned above, the sound output device 4 or the tactile presentation device 5 may include at least an ultrasonic generator. According to this embodiment, an even more novel user experience can be realized. 【0025】 1.7 Wearable Devices 7 The wearable device 7 is worn by user U and is configured to continuously measure user U's biometric information via sensors (not shown). These sensors may include temperature sensors, infrared sensors, light sensors, pressure sensors, potential sensors, chemical sensors, etc. The wearable device 7 may also include, for example, a smartwatch, smart ring, smart glasses, smart earphones, etc. The wearable device 7 can transmit and receive information via wireless communication with the communication unit 21 of the information processing device 2. 【0026】 2. Functional Configuration Next, with reference to Figure 2, the functional configurations of the information processing device 2 in the information processing system 1 will be described. Figure 2 is a block diagram showing the functional configuration of the information processing device 2 according to one embodiment. As shown in Figure 2, the control unit 23 (processor) functions as a reception unit 231, an acquisition unit 232, a setting unit 233, and a presentation unit 234 by executing various programs stored in the storage unit 22. In other words, information processing by software stored in the storage unit 22 is concretely realized by the control unit 23, which is an example of hardware, and can be executed as each functional unit included in the control unit 23. To put it another way, the information processing system 1 comprises at least one processor (for example, the control unit 23), and the processor executes a program to function as the following units. From another perspective, the program causes the computer to execute each step of the information processing system 1. Each step includes, for example, a reception step, an acquisition step, a setting step, and a presentation step. 【0027】 The reception unit 231 is configured to receive various types of information. For example, the reception unit 231 receives various types of information based on operations performed by the user via the input unit 24. 【0028】 The acquisition unit 232 is configured to acquire various types of information as a first or second acquisition step. Preferably, the acquisition unit 232 acquires the biometric information of user U. Further details will be described later. 【0029】 The configuration unit 233 is configured to execute a configuration step. The configuration unit 233 sets theme information IF1, which indicates at least one of several pre-managed themes. Details will be described later. 【0030】 The presentation unit 234 is configured to present various information, including visual information VI, auditory information AI, tactile information TI, and olfactory information OI, as a first or second presentation step. "Presentation" means making various information understandable to the user, and may include, for example, the control unit 23 generating control signals for controlling the image output device 3, sound output device 4, tactile presentation device 5, and olfactory presentation device 6. For example, when controlling the image output device 3, the presentation unit 234 is configured to output display information to the image output device 3. The display information may be the visual information VI itself, generated in a manner that is visible to the user, such as a screen, image, icon, or text, or it may be rendering information for displaying the visual information VI on the image output device 3, for example. 【0031】 3. Overview of Information Processing This section describes the overview of information processing performed by the information processing system 1. Figure 3 is a schematic diagram showing the specific configuration of the information processing system 1. When using the information processing system 1, user U is assumed to be in a closed space 11 while wearing a wearable device 7. The closed space 11 is, for example, a small room with white inner walls. In the closed space 11, a display 31 and a projector 32 are arranged as an example of an image output device 3, headphones 41 and a speaker 42 are arranged as an example of an audio output device 4, a fan 51 and an air conditioner 52 are arranged as an example of a tactile presentation device 5, and an aroma diffuser 61 is arranged as an example of an olfactory presentation device 6. Visual information VI1 is presented by the display 31, and visual information VI2 is presented by the projector 32. Auditory information AI1 is presented by the headphones 41, and auditory information AI2 is presented by the speaker 42. 【0032】 These are merely examples, and at least some of the components may be provided in multiples. Furthermore, the specific arrangement of each component 3 to 6 is not particularly limited, and a preferred arrangement is acceptable in accordance with the physical structure of the closed space 11. User U can experience a novel user experience by receiving the presentation of multimodal information IF2 within the closed space 11. 【0033】 Figure 4 is an activity diagram illustrating the overview of information processing. In a closed space 11 isolated from the outside world, while user U is resting, the wearable device 7 acquires user U's biometric information (Activity A101). This biometric information may include, for example, a sleep score quantifying the quality and quantity of sleep, blood pressure, pulse, heart rate, blood oxygen saturation, respiratory rate, body temperature, electrocardiogram, electroencephalogram, blood glucose, alcohol, lactic acid, level of consciousness, number of steps, walking distance, exercise level, and a biometric score derived from at least one of these pieces of information. 【0034】 Next, biometric information is shared from the wearable device 7 to the information processing device 2 via wireless communication (Activity A102). Subsequently, the acquisition unit 232 acquires (receives) user U's biometric information as the first acquisition step via the communication unit 21 of the information processing device 2 (Activity A103). Thereafter, even as time passes, the wearable device 7 will continue to acquire user U's biometric information and share it with the information processing device 2. 【0035】 Next, the user U's fatigue level, estimated based on biometric information, is displayed on the display 31 (Activity A104). Then, user U operates the GUI displayed on the display 31 to select a theme of their choice (Activity A105). At this time, the reception unit 231 accepts the user's operation. Note that multiple themes are provided in advance, and when selecting a theme, a recommended theme may be visualized to the user based on their current fatigue level. After that, the setting unit 233 sets the selected theme as theme information IF1 (Activity A106). In other words, the setting unit 233 sets theme information IF1 to indicate at least one of the multiple themes that are managed in advance. 【0036】 Next, based on the set theme information IF1, at least two of the image output device 3, sound output device 4, tactile presentation device 5, and olfactory presentation device 6 are controlled (Activity A107). As a result, multimodal information IF2, which is a combination of information consisting of at least two of the visual information VI, auditory information AI, tactile information TI, and olfactory information OI, is presented to the user U (Activity A108). In other words, as the first presentation step, the presentation unit 234 presents multimodal information IF2 corresponding to the theme information IF1 to the user U located in the closed space 11. This configuration makes it possible to realize a novel user experience with a high healing effect. 【0037】 Furthermore, while user U is in the closed space 11, the information processing related to activities A107 and A108 may be repeatedly performed. That is, the presentation unit 234 continuously presents multimodal information IF2. Subsequently, when the repeated information processing related to activities A107 and A108 satisfies the termination condition, the fatigue level of user U estimated according to the biometric information is displayed again on the display 31 (activity A109). Such termination conditions are not particularly limited and may include, for example, the passage of a certain amount of time, the fatigue level falling below or less than a threshold, or user U performing a predetermined operation via the input unit 24. 【0038】 Incidentally, the presented visual information VI, auditory information AI, tactile information TI, and olfactory information OI are presented as unified multimodal information IF2 that makes the user U feel a specific theme. More specifically, multimodal information IF2 is information structured to make the user U feel as if they are in a place that matches the theme information IF1. For example, the theme selected by the user U may be something that evokes the image of a place that can actually be visited, such as "forest," "beach," or "countryside," or it may be something that represents something or a concept that humans cannot actually go to in their physical bodies, such as "fire," "seabed," "moon," "space," or "stars." For example, if the set theme is "forest," an image of lush trees is displayed on the display 31 as visual information VI1, and an image of faint sunlight filtering through the trees from the projector 32 is projected onto the inner wall of the enclosed space 11 as visual information VI2. In addition, a faint sound of wind is played from the headphones 41 as auditory information AI1, and the sound of birds chirping in the distance is played from the speaker 42 as auditory information AI2. Furthermore, an airflow system 51 provides a gentle breeze, an air conditioner 52 is set to a comfortable temperature range, and an aroma diffuser 61 fills the enclosed space 11 with the scent of plants. In other words, depending on the theme, control methods for various components 3-6 are pre-managed in a table and stored in the memory unit 22. Based on the selected theme information IF1, the presentation unit 234 generates control signals for each component 3-6 and controls them, thereby presenting multimodal information IF2 that aligns with the theme. This configuration makes it possible to give the user U an illusion of being in a different location, and ultimately realize a novel user experience. 【0039】 Preferably, the presented multimodal information IF2 is configured to change in content or intensity over time. For example, if the set theme is "forest," the wind direction may change, birdsong may be heard, or the lighting from the projector 32 may change as time passes. This configuration allows for the presentation of multimodal information IF2 that keeps the user U engaged, and ultimately enables a novel user experience. 【0040】 More preferably, the display unit 234 variably displays multimodal information IF2 according to biological information. For example, the content or intensity may change in response to changes in biological information. For instance, if user U is expected to recover from fatigue, the intensity of multimodal information IF2 may be reduced. This configuration makes it possible to present desirable multimodal information IF2 tailored to user U's physical condition, thereby realizing a novel user experience. 【0041】 In summary, the information processing method according to one embodiment comprises the following steps executed by the information processing system 1. In the setting step, theme information IF1 is set, which indicates at least one of a plurality of pre-managed themes. In the first presentation step, multimodal information IF2 corresponding to the theme information IF1 is presented to the user U located in the closed space 11. The multimodal information IF2 is a combination of information consisting of at least two of the following: visual information VI, auditory information AI, tactile information TI, and olfactory information OI. According to this embodiment, a novel user experience with a high healing effect can be realized. 【0042】 4. Details of Information Processing This section will explain the details of the information processing outlined in the previous section, using diagrams and other visual aids. 【0043】 4.1 Screen Example Figure 5 shows examples of screens presented to user U, with Figure 5A illustrating screen 8a corresponding to activity A104, and Figure 5B illustrating screen 8b corresponding to activity A109. Screens 8a and 8b are displayed, for example, on display 31. Screen 8a comprises areas 81-83 and a button 84. Screen 8b also comprises areas 81-83. 【0044】 Focusing on screen 8a, area 81 displays the text "Your energy level," and below it, five star marks are drawn. Of the five stars, only the leftmost star is drawn in black. This indicates that user U is currently not feeling well (high fatigue level). Area 82 displays the text "You seem very tired, so please rest in this space." Area 83 displays an avatar Ab with an expression corresponding to the energy level (fatigue level) presented in area 81. Button 84 is a button with the text "Select a theme," and when user U presses button 84 (click or tap, etc.), the user transitions to an unshown screen for selecting a theme (theme information IF1). For an example of selectable themes, please refer to the description in the previous section. 【0045】 On the other hand, focusing on screen 8b, area 81 displays the text information "Your energy level," and below it, five star marks are drawn. All five star marks are drawn in black. This indicates that user U is currently energetic (low or no fatigue). Area 82 displays the text information "It's great to hear you're full of energy." Area 83 displays an avatar Ab with an expression corresponding to the energy level (fatigue level) presented in area 81. 【0046】 As described above, screens 8a and 8b display support information IF3 that allows user U to understand whether or not fatigue has been recovered and to what extent. For example, the information presented in areas 81 to 83 is an example of support information IF3. In other words, as a second presentation step, the presentation unit 234 further presents support information IF3 that allows user U to understand whether or not fatigue has been recovered and to what extent. With this configuration, user U can understand their own physical condition, etc., through the information processing system 1. 【0047】 Furthermore, before and after using the information processing system 1 (i.e., corresponding to the timing when screens 8a and 8b are displayed), user U's energy level increases (fatigue level decreases or disappears). In other words, the presentation unit 234 presents multimodal information IF2 to user U, who is resting in the closed space 11, thereby promoting user U's fatigue recovery. This configuration can promote user U's fatigue recovery. 【0048】 4.2 Presentation of Visual Information VI Figure 6 is a schematic diagram showing an example of a method for presenting visual information VI. As also shown in Figure 3, the information processing system 1 according to one embodiment has a display 31 and a projector 32 as image output devices 3. The display 31 can directly display visual information VI1. The projector 32 can project visual information VI2 onto the inner wall of the closed space 11. As shown in Figure 3, when using the information processing system 1, the user U mainly looks directly at the display 31, so visual information VI1 is mainly presented to the user U. It is also preferable that visual information VI2 be projected over the entire inner wall of the closed space 11, which is an area not being looked at. In other words, visual information VI includes visual information VI1 (an example of first visual information) and visual information VI2 (an example of second visual information). Visual information VI1 is presented in the closed space 11 at a position where the user U looks directly at it. Visual information VI2 is presented around visual information VI1 so as to cover the user U's field of view. This configuration allows for the addition of contrast to visual information (VI), thereby enabling a novel user experience. 【0049】 Although not shown, contrast may also be added to information other than visual information VI. For example, in one embodiment of the information processing system 1, headphones 41 and a speaker 42 are included as the audio output device 4. The headphones 41 can directly deliver auditory information AI1 to the user U at a position close to their ears. The speaker 42 outputs auditory information AI2 from a distant position in the enclosed space 11, and can, for example, create an effect where only a faint sound is audible. In other words, auditory information AI includes auditory information AI1 (an example of first auditory information) and auditory information AI2 (an example of second auditory information). Auditory information AI1 is presented at a position close to the user U's ears. Auditory information AI2 is presented in the enclosed space 11 in a way that the user U can perceive it as ambient sound. With this configuration, contrast can be added to the auditory information AI, and consequently, a novel user experience can be realized. 【0050】 [others] The information processing system 1 according to one embodiment can also be implemented in the following manner. 【0051】 In one embodiment, the display 31 presents visual information VI1 and the projector 32 presents visual information VI2. However, multiple displays 31 presenting visual information VI2 may be used, or a projector 32 presenting visual information VI1 may be used. 【0052】 In one embodiment, headphones 41 present auditory information AI1 and speakers 42 present auditory information AI2. However, speakers 42 that present auditory information AI1 may be used, or headphones 41 that present auditory information AI2 may be used. When auditory information AI is presented only by headphones 41, it is preferable to use a shielded type with noise-canceling functionality rather than the open-air type exemplified in one embodiment in order to improve immersion. Furthermore, when speakers 42 are used, it is preferable to have a surround sound system. 【0053】 In one embodiment, user U is assumed to be resting in the enclosed space 11, but user U may perform actions other than resting. For example, user U may eat or drink in the enclosed space 11. In such a case, the presentation unit 234 presents multimodal information IF2 to user U who is eating or drinking in the enclosed space 11, thereby causing a change in user U's taste perception. This embodiment can cause a change in user U's taste perception and, consequently, realize a novel user experience. 【0054】 In one embodiment, the enclosed space 11 was described as a small room, but it may also be the interior of a vehicle such as an automobile. In other words, the information processing system 1 is an in-vehicle system, and the enclosed space 11 is the interior of the vehicle. This configuration can contribute to providing a vehicle that can realize a novel user experience. 【0055】 Biometric information may be acquired by other devices instead of the wearable device 7. For example, other devices may include at least a smartphone, a body composition analyzer, a thermometer, medical equipment such as an MRI, CT scanner, and X-ray machine, a thermograph, a blood pressure monitor, a skin age measuring device, and a blood glucose measuring device. On the other hand, an embodiment that does not acquire biometric information may also be adopted. For example, instead of biometric information from the wearable device 7, the degree of fatigue of user U may be determined from facial expressions. Specifically, the information processing system 1 may be equipped with a camera (not shown), and the user U's face may be captured by the camera. By analyzing the facial expressions included in the captured image data, the degree of fatigue of user U can be determined. 【0056】 In one embodiment, a table is used in which theme information IF1 and the control methods for each component 3-6 are related to each other. However, instead of or in addition to the table, a trained model that has undergone deep learning or the like may be used. For example, when the multimodal information IF2 is made variable over time, the preferences and fatigue recovery tendencies of past users U may be learned, and the system may be designed to perform more favorably each time it is used. 【0057】 In one embodiment, multimodal information IF2 was presented variably according to biometric information obtained via the wearable device 7. However, multimodal information IF2 may be presented variably according to specified information desired by user U, either together with or in place of biometric information. More specifically, the specified information may include information about the state after using the information processing system 1. For example, the specified information may include information about states such as feeling energetic, wanting to calm down, or wanting to relax. Furthermore, the specified information may be recommended by the information processing system 1, or it may be selected by user U. In other words, the acquisition unit 232 acquires the specified information designated by user U as a second acquisition step. Then, the presentation unit 234 presents multimodal information IF2 variably according to the specified information as a first presentation step. With this embodiment, it is possible to present the user U's preferred multimodal information IF2, and thereby realize a novel user experience. 【0058】 When implementing the information processing system 1 as a business, a program may be implemented, an information processing method may be implemented, an information processing device 2 may be implemented, the information processing device 2 and at least two of the components 3 to 6 may be implemented, and a wearable device 7 may also be implemented. 【0059】 The information processing device 2 and each component 3 to 6 may be remotely connected via a network. In such a case, for example, a user U who wishes to use it personally may purchase each component 3 to 6 for a small room in their home and access the information processing device 2, which is being used commercially, to perform the aforementioned information processing (a so-called cloud-type service). When provided as such a service, at least one of the devices included in the information processing system 1 may be installed outside of Japan. For example, even if the information processing device 2 is installed outside of Japan, user U may access the information processing device 2 from within Japan using each component 3 to 6 in their own small room. According to this configuration, a more interesting experience can be provided to the user through various management forms. 【0060】 In this embodiment, the reception unit 231, acquisition unit 232, setting unit 233, and presentation unit 234 are described as functional units realized by the control unit 23 of the information processing device 2. However, at least a part of these may be implemented as functional units realized by components other than the information processing device 2. Furthermore, the various types of information described in the above example may be stored in a distributed manner not only in the storage unit 22 of the information processing device 2 but also in other external devices. In such cases, distributed ledger management based on blockchain or the like may be implemented. Alternatively, each of the components 3 to 6 may have a processor and be able to communicate with each other, so that information processing can be performed without going through the information processing device 2. 【0061】 Furthermore, they may be provided in the following embodiments. 【0062】 (1) An information processing system comprising at least one processor, the processor configured to execute a program such that the following steps are performed: in a setting step, theme information indicating at least one of a plurality of pre-managed themes is set; and in a first presentation step, multimodal information corresponding to the theme information is presented to a user located in a closed space, wherein the multimodal information is a combination of information consisting of at least two of visual information, auditory information, tactile information and olfactory information. 【0063】 This approach makes it possible to create a novel user experience with a high level of therapeutic effect. 【0064】 (2) The information processing system described in (1) above, further comprising at least one image output device for providing the visual information, at least one sound output device for providing the auditory information, at least one tactile presentation device for providing the tactile information, and at least one olfactory presentation device for providing the olfactory information, wherein the processor is configured to control at least two devices among the image output device, the sound output device, the tactile presentation device, and the olfactory presentation device that act on different senses, thereby presenting the multimodal information. 【0065】 In this configuration, a novel user experience can be realized as a complex device. 【0066】 (3) In the information processing system described in (2) above, the system includes at least an ultrasonic generator as the voice output device or the tactile presentation device. 【0067】 This approach makes it possible to realize a more novel user experience. 【0068】 (4) An information processing system according to any one of (1) to (3) above, wherein the visual information includes first visual information and second visual information, the first visual information is presented in a closed space at a position where the user can directly look at it, and the second visual information is presented around the first visual information so as to cover the user's field of view. 【0069】 This approach allows for the addition of contrast to visual information, thereby enabling a novel user experience. 【0070】 (5) An information processing system according to any one of (1) to (4) above, wherein the auditory information includes first auditory information and second auditory information, the first auditory information is presented at a position close to the user's ear, and the second auditory information is presented in a closed space so that the user can perceive it as ambient sound. 【0071】 This approach allows for the addition of contrast to auditory information, thereby enabling a novel user experience. 【0072】 (6) An information processing system according to any one of (1) to (5) above, wherein in the first presentation step, the multimodal information is continuously presented, and the presented multimodal information is configured to change in content or intensity over time. 【0073】 This approach allows for the presentation of multimodal information that keeps users engaged, ultimately leading to the realization of a novel user experience. 【0074】 (7) An information processing system described in any one of (1) to (6) above, wherein the multimodal information is information configured to give the user the illusion that they are in a location related to the theme information. 【0075】 This approach makes it possible to create a sense of spatial illusion in the user, and consequently, to realize a novel user experience. 【0076】 (8) An information processing system according to any one of (1) to (7) above, further comprising: acquiring the user's biometric information in the first acquisition step, and presenting the multimodal information in a variable manner according to the biometric information in the first presentation step. 【0077】 This configuration allows for the presentation of desirable multimodal information tailored to the user's physical condition, ultimately enabling a novel user experience. 【0078】 (9) An information processing system as described in any one of (1) to (8) above, wherein in a second acquisition step, specified information designated by the user is acquired, wherein the specified information includes information about the state after using the information processing system as desired by the user, and in the first presentation step, the multimodal information is presented variably according to the specified information. 【0079】 This configuration allows for the presentation of desirable multimodal information desired by the user, thereby enabling a novel user experience. 【0080】 (10) An information processing system described in any one of (1) to (9) above, wherein in the second presentation step, the system further presents support information that allows the user to understand whether or not fatigue has been recovered or to what extent. 【0081】 In this configuration, users can understand their own physical condition and other information through the information processing system. 【0082】 (11) An information processing system according to any one of (1) to (10) above, wherein in the first presentation step, the system presents the multimodal information to the user who is resting in the enclosed space, thereby promoting the user's recovery from fatigue. 【0083】 This configuration can promote user fatigue recovery. 【0084】 (12) An information processing system according to any one of (1) to (11) above, wherein in the first presentation step, the system presents the multimodal information to the user who is eating or drinking in the enclosed space, thereby causing a change in the user's taste perception. 【0085】 This approach can induce changes in the user's sense of taste, thereby enabling a novel user experience. 【0086】 (13) In an information processing system described in any one of (1) to (12) above, the information processing system is an in-vehicle system, and the enclosed space is the interior of a vehicle. 【0087】 This approach can contribute to providing vehicles that enable novel user experiences. 【0088】 (14) An information processing method comprising each step performed on an information processing system described in any one of (1) to (13) above. 【0089】 This approach makes it possible to create a novel user experience with a high level of therapeutic effect. 【0090】 (15) A program that causes a computer to perform each step of the information processing system described in any one of (1) to (13) above. 【0091】 This approach makes it possible to create a novel user experience with a high level of therapeutic effect. Of course, this is not always the case. 【0092】 Finally, various embodiments of the present invention have been described, but these are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Examples] 【0093】 The following describes the verification of the effectiveness of the information processing method described in the embodiment using the example. Figure 7 shows the experimental protocol of the example used for effectiveness verification. Figures 8 to 10 show the experimental results of the example. 【0094】 As an experiment, we quantitatively investigated the extent to which fatigue caused by work of a similar workload to office work could be recovered by stimulation from this device. The experimental protocol for this purpose is shown in Figure 7. This experimental protocol consists of three blocks: an evaluation task to measure information processing ability, a control task to accumulate mental fatigue, and a rest period. First, information processing ability in the initial state at the start of the experiment was measured using the evaluation task. Next, after accumulating mental fatigue using the control task, we confirmed whether a decrease in information processing ability had occurred using the evaluation task. After a rest, the evaluation task was performed again, and the presence or absence of recovery was confirmed by how much the score changed compared to before the rest. In addition, the degree of recovery was evaluated from physiological indicators by recording heart rate variability during the experiment. 【0095】 Three types of assessment tasks were used: a 2-back calculation (medium difficulty), a 2-back calculation (high difficulty), and a cognitive ability test (CogEvo). The cognitive ability test consists of five tasks (such as visual search) and is said to be able to measure four types of cognitive abilities: spatial awareness, attention, memory, and planning ability. However, the five tasks of CogEvo were of low difficulty, and there was a possibility that performance would decline due to mental fatigue or not improve with rest. Therefore, a more challenging task, a medium-to-high difficulty 2-back calculation, was adopted. The stimulus conditions during rest are shown in Table 1. [Table 1] 【0096】 Three samples were prepared: "Full," "No Visual Stimulation," and "Control Condition." The "Full" sample consisted of a composite stimulus of four types—sight, smell, and touch—designed to evoke the image of a forest. The "No Visual Stimulation" sample consisted of a composite stimulus of three types, with the visual stimulus removed from the "Full" sample. The "Control Condition" consisted of a visual stimulus only, presented with an image representing an office computer. The experiment involved 47 participants (25 males, 22 females, age 30.3 ± 4.4 years), and each participant performed all three conditions in a random order. 【0097】 Figure 8A shows a medium-difficulty 2-back problem (N aThe following shows the score changes for each stimulus during the break before and after the break. In the full and no-visual conditions with composite stimuli, the score rate improved statistically significantly before and after the break, whereas in the control condition without composite stimuli, no improvement in score due to the break was observed. No statistically significant differences were observed between conditions in the other tasks. 【0098】 Furthermore, Figure 8B shows a comparison of the workload of daily work (office work) and the task in this experiment using NASA-TLX. Since both show generally similar trends, it can be said that the workload in this experiment is equivalent to office work. Also, as shown in Figure 9A, the average heart rate changes of subjects during the experiment show generally similar trends under all conditions until before the break, but differences in heart rate changes during the break were observed due to differences in stimulus conditions. In particular, when comparing the difference in heart rate of each subject immediately before the start of the break and 2 minutes after the start of the break between conditions, a statistically significant difference in the amount of heart rate reduction was observed compared to the control experiment in both the full and no-visual conditions (see Figure 9B). In other words, the decrease in heart rate was greater with complex stimuli than with the control condition, indicating a higher relaxation effect. These results indicate that, for fatigue caused by a workload equivalent to office work, the environment with complex stimuli has a higher fatigue recovery effect in terms of both information processing ability and physiological indicators. In addition, it is possible to quantify the effect of stimulation from the increase or decrease in parasympathetic nerve activity indicators, including heart rate. 【0099】 Based on the above results, the Japanese version of the Perceived Restorativeness Scale (PRS) was administered under each condition to investigate the factors that led to the high restorative effect of the stimulus. The PRS is a scale that evaluates the restorative properties of an environment as perceived by people, and it has a track record of use indoors. Table 2 shows a list of questions for each sub-item of the PRS. [Table 2] 【0100】 Furthermore, Figure 10A shows the PRS results for each stimulus in this experiment (47 subjects). In the three PRS sub-items, "Avoidance," "Attraction," and "Preference," the scores for "Full" and "No Visual" were higher than in the control condition. However, although familiarity showed a similar trend, it was excluded because it did not contribute to the identification of high-recovery characteristic environments in previous studies. In other words, recovery may be promoted in environments where the scores for the PRS sub-items "Avoidance," "Attraction," and "Preference" are higher than in the control condition. Figure 10B shows the PRS results when one stimulus is removed from "Full" (the legend indicates the removed stimulus). "Main video" in Figure 10B is the closest condition to "No Visual" in Figure 9B, and the average scores for Avoidance, Attraction, and Preference are all the lowest in Figure 10B. Nevertheless, recovery was observed in this experiment, so when three stimuli are presented with one removed from the "Full" set as shown in Figure 10B, a higher recovery effect can be expected in each case compared to when there are no combined stimuli. [Explanation of Symbols] 【0101】 1: Information Processing System 11: Closed space 2: Information Processing Device 20: Communications bus 21: Communications Department 22: Storage section 23: Control Unit 231: Reception Department 232: Acquisition Department 233: Settings Section 234:Presentation part 24: Input section 3: Image output device 31: Display 32: Projector 4: Audio output device 41: Headphones 42: Speaker 5: Tactile presentation device 51: Electric fan 52: Air conditioner 6: Olfactory presentation device 61: Aroma diffuser 7: Wearable devices 8a: Screen 8b: Screen 81 :Area 82 :Area 83 :Area 84: Button AI: Auditory Information Ab: Avatar IF1: Theme Information IF2: Multimodal Information IF3: Support Information OI: Olfactory information TI: Tactile information U: User VI: Visual Information

Claims

[Claim 1] An information processing system, The system comprises at least one processor, the processor configured to execute a program such that the following steps are performed: In the setup step, theme information is set based on user actions, indicating at least one of several pre-managed themes, where the theme represents a location or concept. In the first presentation step, multimodal information corresponding to the theme information is presented to the user who is resting in a closed space, thereby promoting the user's fatigue recovery, where the multimodal information is a combination of at least two of the following: visual information, auditory information, tactile information, and olfactory information. At least one of the aforementioned pieces of information that affect the same sense is presented by multiple devices in such a way that it adds contrast. In the second presentation step, the system further presents support information that allows the user to understand whether or not they have recovered from fatigue, or to what extent, based on the difference in the user's state before and after the first presentation step. [Claim 2] In the information processing system described in claim 1, The image output device that provides the aforementioned visual information, The audio output device that provides the aforementioned auditory information, A tactile presentation device that provides the aforementioned tactile information, The system further comprises at least one olfactory presentation device that provides the aforementioned olfactory information, The processor is configured to control at least two devices that act on different senses among the image output device, the sound output device, the tactile presentation device, and the olfactory presentation device, thereby presenting the multimodal information in a system. [Claim 3] In the information processing system described in claim 2, The system includes at least an ultrasonic generator, wherein the audio output device or the tactile presentation device is a system. [Claim 4] In the information processing system described in claim 1, The aforementioned visual information includes first visual information and second visual information. The first visual information is presented in the enclosed space at a position where the user can directly look at it. A system in which the second visual information is presented so as to cover the user's field of view around the first visual information. [Claim 5] In the information processing system described in claim 1, The aforementioned auditory information includes first auditory information and second auditory information. The aforementioned auditory information is presented in a position close to the user's ear. The second auditory information is presented to the user in the enclosed space in a manner that the user can perceive as ambient sound. [Claim 6] In the information processing system described in claim 1, In the first presentation step, the multimodal information is presented continuously. A system in which the multimodal information presented is configured to change in content or intensity over time. [Claim 7] In the information processing system described in claim 1, The multimodal information is a system in which information is configured to give the user the illusion that they are in a location consistent with the theme information. [Claim 8] In the information processing system described in claim 1, Furthermore, in the first acquisition step, the user's biometric information is acquired, The system, in the first presentation step, presents the multimodal information in a variable manner according to the biological information. [Claim 9] In the information processing system described in claim 1, Furthermore, in the second acquisition step, the specified information designated by the user is acquired, where the specified information includes information about the state after using the information processing system, as desired by the user. In the first presentation step, the system presents the multimodal information variably according to the specified information. [Claim 10] In the information processing system described in claim 1, The system, in the first presentation step, presents the multimodal information to the user who is eating or drinking in the enclosed space, thereby causing a change in the user's taste perception. [Claim 11] In the information processing system described in claim 1, The aforementioned information processing system is an in-vehicle system, The aforementioned enclosed space is the interior of the vehicle, and the system is defined as such. [Claim 12] Information processing method, A method comprising each step performed on the information processing system according to any one of claims 1 to 11. [Claim 13] It is a program, A program that causes a computer to perform each step of the information processing system described in any one of claims 1 to 11.

Images