Control device, information processing system, and control method

Abstract

A control device comprises at least one processor that controls a presentation device for presenting content to a user. The at least one processor outputs, to the presentation device, control information for presenting a first type of content that raises the arousal level of the user and induces a pleasant feeling in the user. After the first type of content is presented by the presentation device, the at least one processor outputs, to the presentation device, control information for presenting a second type of content that induces a pleasant feeling in the user and lowers the arousal level.

Description

Control Device, Information Processing System, and Control Method

[0001] The present disclosure relates to a control device, an information processing system, and a control method.

[0002] In the background of the labor shortage in an aging society with a low birthrate, the government is promoting work-style reforms. Companies are working on creating various mechanisms to improve the intellectual productivity of their employees. The motivation of employees and the office space are closely related, and the development of environmental design or space control that can increase intellectual productivity is underway. In order to increase intellectual productivity, it is important not only whether employees can concentrate on their work (tasks), but also whether they can think more creatively. Creative thinking includes convergent thinking that logically progresses from known information to reach one solution, and divergent thinking that generates new ideas by thinking around known information.

[0003] For example, Patent Document 1 discloses a technique for improving the creativity and productivity of users.

[0004] Japanese Patent Application Laid-Open No. 2021-070959

[0005] However, from the perspective of how to appropriately guide people into a state where they can concentrate or exert their creativity through environmental design or space control, for example, the state at the start of work (tasks) varies among people, and there is room for further consideration.

[0006] The present disclosure provides a technique for constructing a surrounding environment that can appropriately guide people into a state where they can concentrate or exert their creativity.

[0007] A control device according to one aspect of the present disclosure includes at least one processor that controls a presentation device that presents content to a user. The at least one processor outputs control information for causing the presentation device to present a first type of content that enhances the user's arousal level and induces the user's pleasant feeling, and after the presentation device presents the first type of content, outputs control information for causing the presentation device to present a second type of content that induces the user's pleasant feeling and reduces the arousal level.

[0008] According to this disclosure, it is possible to create an ambient environment that can appropriately guide people into a state where they can concentrate or exercise their creativity. The effects described herein are not necessarily limited to those described herein and may include any of the effects described herein.

[0009] Figure 1 is a diagram showing an example of the configuration of the display system according to the embodiment. Figure 2 is a diagram showing an example of the hardware configuration of the information processing device that realizes each device included in the display system according to the embodiment. Figure 3 is a diagram for explaining the measurement conditions of an experiment that yielded the knowledge that formed the basis of the environmental control according to the embodiment. Figure 4 is a diagram for explaining the measurement conditions of an experiment that yielded the knowledge that formed the basis of the environmental control according to the embodiment. Figure 5 is a diagram for explaining the experimental results of an experiment that yielded the knowledge that formed the basis of the environmental control according to the embodiment. Figure 6 is a diagram for explaining the experimental results of an experiment that yielded the knowledge that formed the basis of the environmental control according to the embodiment. Figure 7 is a diagram for explaining the knowledge regarding the relationship between the introduced content and the guidance of the user's state, which formed the basis of the environmental control according to the embodiment. Figure 8 is a flowchart showing an example of the flow of environmental control processing executed in the display system according to the embodiment. Figure 9 is a diagram showing an example of the configuration of the content database according to the embodiment. Figure 10 is a diagram showing an example of the display screen displayed in the display system according to the embodiment. Figure 11 is a diagram showing an example of the display screen displayed in the display system according to the embodiment. Figure 12 is a flowchart showing another example of the flow of environmental control processing executed in the display system according to the embodiment. Figure 13 is a diagram for explaining an application example of the display system according to the embodiment. Figure 14 is a diagram illustrating an example of the application of the display system according to the embodiment.

[0010] Hereinafter, embodiments of the signal processing device (information processing device), optical detection system, information processing system, signal processing method (information processing method), program, and computer-readable non-transient recording medium relating to this disclosure will be described with reference to the drawings.

[0011] In this disclosure, components having the same or substantially the same function as those described above in previously shown drawings are denoted by the same reference numerals, and explanations may be omitted as appropriate. Even when representing the same or substantially the same parts, the dimensions and proportions may differ between drawings. For example, from the viewpoint of ensuring the readability of the drawings, major components are denoted by reference numerals in the explanation of each drawing, and components having the same or substantially the same function as those described above in previously shown drawings may not be denoted by reference numerals.

[0012] In addition, in the descriptions of this disclosure, components having the same or substantially the same function may be distinguished by adding alphanumeric characters to the end of the reference numeral. Alternatively, if multiple components having the same or substantially the same function are not to be distinguished, they may be described together by omitting the alphanumeric characters at the end of the reference numeral.

[0013] (First Embodiment) Figure 1 is a diagram showing an example of the configuration of a display system 1 according to an embodiment. The display system 1 is a system configured to perform information processing related to environmental control that guides the user's state by controlling the presentation of content to the user. Here, the display system 1 according to the embodiment is an example of the information processing system, environmental control system and light detection system according to the present disclosure.

[0014] As shown in Figure 1, the display system 1 includes a sensor 3, a user interface 4, a signal processing device 5, a storage device 6, and a presentation device 7. In the display system 1, the signal processing device 5 is communicated with each of the sensor 3, user interface 4, storage device 6, and presentation device 7 via a network such as the Internet (not shown) or a dedicated communication line.

[0015] Sensor 3 is a biometric measurement device configured to acquire the user's biometric information. This user's biometric information is used to estimate the user's state, as described later. Sensor 3 may also be configured to operate according to control signals from the signal processing device 5.

[0016] In the display system 1 described herein, the user's biometric information is information corresponding to at least one of the following: electrocardiogram, cerebral blood flow, electroencephalogram, heart rate, facial features, and skeletal movement. The user's biometric information can be any information that changes in accordance with the user's state or its transitions, and other biometric information such as respiration and body temperature may also be used. In other words, in the display system 1 described herein, sensor 3 has a detector that detects the user's biometric information or the information used to generate it. As a biometric measurement device, devices such as cameras, smartwatches, electrocardiographs, cerebral blood flow meters, electroencephalographs, heart rate monitors, smartphones, personal computers (PCs), tablet PCs, and server devices can be used as appropriate, depending on the biometric information being measured.

[0017] As an example, in the display system 1 as a light detection system, sensor 3 has a photodetector (not shown) that detects light from the user (reflected light and / or diffused light). This photodetector may be, for example, an electrical signal with a signal level corresponding to the received intensity of light incident on the detection surface, or a photodetector that detects light incident at a received intensity above a threshold and converts it into an electrical signal. The photodetector may also be, for example, an image sensor that captures light incident on the detection surface (e.g., the imaging surface) and generates an image signal (electrical signal) with a signal level corresponding to the received intensity. As the image sensor, solid-state image sensors such as CCD (Charge Coupled Device) and CMOS (Complementary Metal-Oxide Semiconductor) can be used as appropriate. The image sensor performs analog processing such as noise reduction, amplification, and A / D conversion on the generated electrical signal (analog signal) and outputs an electrical signal (digital signal) in digital format. Furthermore, some or all of these analog processes may be implemented in at least one other circuit (for example, at least one processor 81 of the sensor 3 or signal processing device 5) located downstream of the image sensor as an analog front-end (AFE).

[0018] As an example, in a photodetection system, sensor 3 further includes at least one processor 81 (see Figure 2) that generates the user's biological information based on an electrical signal (measurement signal) output from the photodetector. The biological information may also be generated by at least one processor in an external information processing device, such as a signal processing device 5, based on the electrical signal output from the photodetector. In other words, sensor 3 may be configured as a device that outputs an electrical signal based on light from the user detected by the photodetector.

[0019] Furthermore, the photodetector may be sensitive not only to visible light but also to near-infrared (NIR) and infrared (IR) light. In other words, the light from the user detected by sensor 3 may be visible light, near-infrared light, or infrared light. The photodetector may further have a light source (not shown) that illuminates the user with illumination light containing light of a wavelength corresponding to the light-receiving sensitivity of the image sensor, such as infrared light. In that case, the photodetector may detect the reflected light (e.g., the field of view) that returns from the user due to the illumination light.

[0020] For example, information corresponding to an electrocardiogram (ECG) may include the signal intensity of electrical signals transmitted to the heart muscle, measured using electrodes placed on the skin surface of the chest, the amount of change in these signals, or values ​​based on these. For instance, a biomedical measurement device that acquires ECG-related information may be an electrocardiograph, a smartwatch or smartphone that works in conjunction with the electrocardiograph, a PC, a tablet PC, or a server device.

[0021] For example, information corresponding to cerebral blood flow may be cerebral blood flow rate, its change, or a value based on these, measured using near-infrared light at a wavelength easily absorbed by hemoglobin (for example, a wavelength of 800 nm). Specifically, cerebral blood flow rate is the amount or concentration of oxygenated hemoglobin and / or deoxygenated hemoglobin in the brain, generated by near-infrared spectroscopy (NIRS) based on near-infrared light scattered inside the user's head. For example, a biomedical measurement device that acquires information corresponding to cerebral blood flow may be a non-contact NIRS device, or a smartwatch, smartphone, PC, tablet PC, or server device that works in conjunction with the NIRS device.

[0022] For example, information corresponding to brain waves may include voltage fluctuations (scalp electroencephalogram: EEG) based on the electrical activity of nerve cells (neurons) in the cerebral cortex, the amount of change therein, or values ​​based on these. For example, a biomedical measurement device that acquires information corresponding to brain waves may be an electroencephalograph, or a smartwatch, smartphone, PC, tablet PC, or server device that works in conjunction with an electroencephalograph.

[0023] For example, information corresponding to heart rate may include heart rate, heart rate interval, their changes, or values ​​based on these. Here, heart rate includes pulse. Furthermore, information corresponding to heart rate may also be information corresponding to heart rate in response to an input image that includes a part of the body where the effect of changes in arterial blood flow in response to the heartbeat (e.g., pulse) is visualized as, for example, skin chromaticity, or it may be the output from a machine learning model whose parameters have been determined (learned) to output information based on heart rate information, or information based on such output. For example, a biometric measurement device that acquires information corresponding to heart rate may be a heart rate monitor, camera, smartwatch, or smartphone, or it may be a smartwatch, smartphone, PC, tablet PC, or server device that works in conjunction with a heart rate monitor or camera.

[0024] For example, information corresponding to facial features may include a facial image, blinking frequency (eye closure rate), gaze, face orientation, facial expression, the amount of change in these, or values ​​based on these. Here, information regarding facial expressions may include information indicating expressions such as a smile, a worried face, or a frown; information indicating the user's state corresponding to the expression, such as a tense (stressed) state or a relaxed state; or information about the position of the corners of the mouth or whether the mouth is open or closed. Furthermore, information corresponding to facial features may also be the output from a machine learning model whose parameters have been determined (learned) to output facial features or information based on facial features in response to an input image including the user's face, or information based on such output. For example, a biometric measurement device that acquires information corresponding to facial features may be a camera, or a smartwatch, smartphone, PC, tablet PC, or server device that works in conjunction with the camera.

[0025] For example, information corresponding to skeletal movement may include posture such as the orientation of the face and body, the position of the arms, the presence or absence of changes (body movements) in these, the amount of change, or values ​​based on these. Furthermore, information corresponding to skeletal movement may also be the output from a machine learning model whose parameters have been determined (learned) to output information about skeletal movement or information based on skeletal movement in response to an input image including the user's body, or information based on such output. For example, a biometric measurement device that acquires information corresponding to skeletal movement may be a camera, or a smartwatch, smartphone, PC, tablet PC, or server device that works in conjunction with the camera.

[0026] Furthermore, any machine learning model, such as CNN (Convolutional Neural Network), can be used as the machine learning model for generating the user's biometric information as described above, depending on the processing. The structure and parameters of this machine learning model may be stored, for example, in the memory unit 52 of the signal processing device 5, or in an external information processing device 8 or storage device.

[0027] Sensor 3 acquires electrical signals output from detectors such as photodetectors and generates the user's biometric information based on these electrical signals. For example, Sensor 3 performs signal processing on the measurement signals (measured values) output from the detectors to generate measurement data in a predetermined format as the user's biometric information. As an example, Sensor 3 performs signal processing to convert image signals (measurement signals) from a camera into text data such as CSV (Comma-Separated Values) format. For example, Sensor 3 converts an image signal showing an image including the user's face, skeleton, and heart rate into text data showing facial features, skeletal movement, information corresponding to the heart rate, or information used to generate such information. This reduces the amount of data for the user's biometric information and reduces the amount of communication required when transmitting from Sensor 3 to the signal processing device 5. Note that the conversion to text data may also involve, for example, converting the image signal itself into text.

[0028] Sensor 3 transmits the generated user biometric information to signal processing device 5.

[0029] User interface 4 is an information processing device configured to receive input from the user and present various types of information to the user. User interface 4 accepts subjective evaluations from the user regarding their own state. User interface 4 accepts specifications from the user for creative content. User interface 4 presents the user with feedback information regarding the user's state during the task, as described below, after the task is completed. Note that user interface 4 may be implemented by information processing devices such as sensor 3, signal processing device 5, storage device 6, presentation device 7, or a PC used by the user to perform the task.

[0030] The signal processing device 5 is an information processing device configured to perform information processing related to environmental control that guides the user's state by controlling the presentation of content to the user. Here, the signal processing device 5 according to the embodiment is an example of a control device according to the present disclosure. As an example, the signal processing device 5 controls a presentation device 7 that presents content to the user. In other words, the signal processing device 5 presents content to the user by outputting control information to the presentation device 7.

[0031] Here, "user state" refers to a state corresponding to the degree of the user's concentration and / or creativity. The degree of the user's concentration can also be expressed as the degree of concentration the user exhibits, or the user's level of concentration. In the signal processing device 5, "user state" is defined by at least one of "alertness" and "pleasure level." At least one of "alertness" and "pleasure level" may be interpreted as "alertness," "pleasure level," or "alertness" and "pleasure level." This "alertness" is an index indicating either "alertness" or "calmness," or an index indicating a degree between "alertness" and "calmness." "Pleasure level" is an index indicating either "pleasantness" or "unpleasantness," or an index indicating a degree between "pleasantness" and "unpleasantness." Here, a "pleasantness level" of "pleasantness," a high degree of "pleasantness," or a low degree of "unpleasantness" means that the user is experiencing "pleasant feelings," that is, "positive feelings." Similarly, a "pleasure level" of "discomfort," a high degree of "discomfort," or a low degree of "pleasure" indicates that the user is experiencing "unpleasant emotions," that is, "negative emotions."

[0032] In this disclosure, a state in which the user is in a "pleasant" state (out of "pleasant" and "unpleasant") or a state where the degree of "pleasantness" is high may also be described as a "highly pleasurable" state. Similarly, a state in which the user is in an "aware" state (out of "aware" and "calm") or a state where the degree of "awareness" is high may also be described as a "highly aroused" state. A state in which the user is not too highly aroused, for example, a moderate state where the degree of "awareness" is lower than "highly aroused" but higher than "lowly aroused," may also be described as a "moderately aroused" state. A state in which the user is in a "calm" state (out of "aware" and "calm") or a state where the degree of "awareness" is lower than "moderately aroused" may also be described as a "lowly aroused" state.

[0033] As shown in Figure 1, the signal processing device 5 includes a communication unit 51, a storage unit 52, and an arithmetic processing unit 53.

[0034] The communication unit 51 communicates with the outside of the signal processing device 5.

[0035] The storage unit 52 stores programs, parameters, data being processed, and processing result data related to each process executed by the signal processing device 5. For example, the storage unit 52 stores user information 521 about users registered as users of the display system 1. This user information 521 stores identification information to uniquely identify the user, as well as user attributes.

[0036] The arithmetic processing unit 53 implements the functions of the signal processing device 5, including the acquisition unit 531, the determination unit 532, the content selection unit 533, and the presentation control unit 534.

[0037] The acquisition unit 531 acquires the user's biometric information from the sensor 3 via the communication unit 51. The acquisition unit 531 also acquires information from the user interface 4 indicating the user's subjective evaluation and information indicating specified content via the communication unit 51. The acquisition unit 531 may acquire some or all of the user information 521 concerning any user from, for example, the reservation history (usage history) of a predetermined space to which the display system 1 according to embodiments such as a private room (booth) or a vehicle interior is applied.

[0038] The determination unit 532 determines the user's state based on the user's biometric information from the sensor 3 or subjective evaluation information input by the user. For example, the determination unit 532 determines the user's state during the presentation of the high-comfort x high-arousal content, based on the biometric information acquired by the sensor 3 during the presentation of the high-comfort x high-arousal content described later. For example, the determination unit 532 determines the user's state during the presentation of the arousal-suppressing content, based on the biometric information acquired by the sensor 3 during the presentation of the arousal-suppressing content, described later. For example, the determination unit 532 determines the user's state during the presentation of the creative content, i.e., during the task, based on the biometric information acquired by the sensor 3 during the presentation of the creative content described later. In this disclosure, "x" may mean "and". For example, "high-comfort x high-arousal" may mean "high-comfort and high-arousal".

[0039] As an example, the determination unit 532 may input biometric information into a trained machine learning model and quantify the user's state on two axes: "pleasantness" ("pleasantness") and "alertness" ("alertness") ("calmness"). The biometric information used as input to this machine learning model may be, for example, a combination of information corresponding to cerebral blood flow and information corresponding to heart rate. For example, the biometric information may be a combination of information corresponding to cerebral blood flow and information corresponding to facial features. For example, the biometric information may be a combination of information corresponding to heart rate and information corresponding to facial features. For example, the biometric information may be information corresponding to facial features. The trained machine learning model is a model whose parameters have been determined (learned) to output information indicating the "pleasantness" and "alertness" quantified in response to the input of biometric information. The output of the machine learning model may be information indicating the state defined by the "pleasantness" and "alertness".

[0040] For example, the determination unit 532 may determine the user's state by comparing the numerically represented "pleasure level" and "awareness level" with predetermined thresholds stored in the memory unit 52 or the like. For example, if the numerically represented "pleasure level" is higher than the predetermined threshold for "pleasure level" stored in the memory unit 52 or the like, the determination unit 532 determines that the user is in a "pleasant" state. On the other hand, if the numerically represented "pleasure level" is below the threshold, the determination unit 532 determines that the user is in an "unpleasant" state. For example, if the numerically represented "awareness level" is higher than the predetermined threshold for high arousal stored in the memory unit 52 or the like, the determination unit 532 determines that the user is in a "highly aroused" state. If the numerically represented "awareness level" is below the predetermined threshold for high arousal stored in the memory unit 52 or the like, and within a threshold range higher than the threshold for low arousal, the determination unit 532 determines that the user is in a "moderately aroused" state. On the other hand, the determination unit 532 determines that the user is in a "calmed" state if the quantified "level of arousal" is below the threshold for low arousal.

[0041] For example, the determination unit 532 may determine the "level of comfort" based on information corresponding to cerebral blood flow. For instance, the determination unit 532 determines that the user is in a "comfortable" state if the amount of change in cerebral blood flow from a reference point is lower than a predetermined threshold stored in the memory unit 52 or the like. On the other hand, the determination unit 532 determines that the user is in an "uncomfortable" state if the amount of change in cerebral blood flow is greater than or equal to the threshold.

[0042] The threshold for the change in cerebral blood flow is, for example, a negative value, but is not limited to this. This threshold can be set to an appropriate value for each user or user attribute. For example, a value of 0 or greater may be set for some users. The threshold may also be set to an optimal value for each user before measurement, for example, by calibration.

[0043] As an example, the determination unit 532 may determine the "level of arousal" based on the information corresponding to the heartbeat. For example, when the change amount of the heartbeat number from the reference time point is higher than the threshold value of high arousal that is predetermined and stored in the storage unit 52 or the like, the determination unit 532 determines that the user is in the "high arousal" state. When the change amount of the heartbeat number from the reference time point is within the threshold range that is lower than the threshold value of high arousal and higher than the threshold value of low arousal, which are predetermined and stored in the storage unit 52 or the like, the determination unit 532 determines that the user is in the "medium arousal" state. On the other hand, when the change amount of the heartbeat number is lower than the threshold value of low arousal, the determination unit 532 determines that the user is in the "calm" state. Note that, as the heartbeat numbers at the reference time point and the measurement time point, for example, the average value of the heartbeat numbers measured over a certain period of time (for example, about several seconds to several minutes) can be used.

[0044] Note that the threshold value of high arousal for the change amount of the heartbeat number is, for example, a positive value, but is not limited thereto. Similarly, the threshold value of low arousal is, for example, a negative value, but is not limited thereto. These threshold values can be set to appropriate values for each user or user attribute, for example. For example, for some users, a value of 0 or less may be set as the threshold value of high arousal, or a value of 0 or more may be set as the threshold value of low arousal. At least one of these threshold values may be set to an optimal value for each user before measurement, for example, by calibration. "At least one of these threshold values" may be interpreted as the threshold value of high arousal, the threshold value of low arousal, or both the threshold value of high arousal and the threshold value of low arousal.

[0045] Note that the reference time point for the change amount of cerebral blood flow and the change amount of the heartbeat number may be, for example, a time point when the user is in a neutral state, such as a "comfortable" state, an "uncomfortable" state, a "high arousal" state, or a "calm" state, for example, a resting state. The reference time point may be the time point when the environmental control for the user P is started, the time point when the output of the introduction content (for example, welcome content) described later is started, or a time point before these time points.

[0046] As an example, the determination unit 532 may determine whether rest is necessary. For example, after starting the presentation of the creative content described later, if the desired time set by the user or a predetermined time stored in the storage unit 52 or the like has elapsed, the determination unit 532 may determine that rest is necessary. For example, after starting the presentation of the creative content described later, if the user operates a stop button installed in the booth or a stop button provided on the user interface 4, the determination unit 532 may determine that rest is necessary. For example, after starting the presentation of the creative content described later, even if the creative content is changed a predetermined number of times stored in the storage unit 52 or the like in advance, if it is determined that the user's concentration and / or creativity has decreased, the determination unit 532 may determine that rest is necessary.

[0047] Here, the state in which the user's concentration and / or creativity has decreased is, for example, a case where the determination unit 532 determines that it is not in the "high comfort × medium arousal" state in which the user can exert concentration and / or creativity.

[0048] In addition, the determination unit 532 may determine that the user's concentration and / or creativity has decreased, for example, based on an image of the user during the task when the user's movement becomes large or when non-task-related movements such as looking at a smartphone occur or occur frequently.

[0049] The determination unit 532 may determine, for example, based on biometric information whether the user is in a state of divergent thinking, and if the user is not in a state of divergent thinking, the determination unit 532 may determine that the user's creativity has decreased. Here, the user being in a state of divergent thinking means that the user is in a state where the degree of creativity exerted is high. Similarly, the user not being in a state of divergent thinking means that the user is in a state where the degree of creativity exerted is low.

[0050] For example, the determination unit 532 may determine that the user is not in a divergent thinking state if the amount of variation in facial features such as the position and orientation of the user's face and the direction of their gaze, and / or the amount of variation in skeletal movement, or the rate of eye closure, is smaller than the respective thresholds predetermined and stored in the memory unit 52 or the like. Similarly, the determination unit 532 may determine that the smaller each variation or the rate of eye closure is, the further the user is from a divergent thinking state and the more their creativity is reduced.

[0051] For example, the determination unit 532 may determine that the user is not in a divergent thinking state if the index (e.g., difference or ratio) of the increase in cerebral blood flow in the central part of the user's forehead relative to the increase in the edge of the forehead is smaller than the respective thresholds predetermined and stored in the memory unit 52 or the like. Similarly, the determination unit 532 may determine that the smaller the index of the increase, the further the user is from a divergent thinking state and the more their creativity is reduced.

[0052] For example, the determination unit 532 may calculate ccvHF (component confidence of variance High Frequency), which indicates the degree of relaxation, and ccvLF (ccv Low Frequency), which indicates the degree of concentration, by processing the heart rate interval and pulse wave interval using FFT (Fast Fourier Transform) for a certain period of time. If ccvHF and ccvLF are smaller than their respective thresholds, which are predetermined and stored in the memory unit 52 or the like, the determination unit 532 may determine that the user is not in a divergent thinking state. Similarly, the determination unit 532 may determine that the smaller ccvHF and ccvLF are, the further the user is from a divergent thinking state and the more their creativity is reduced. The determination unit 532 may also determine that the user's concentration is reduced if the value of ccvLF is smaller than a threshold.

[0053] The content selection unit 533 refers to the content database 61 and selects content to be presented to the user by the presentation device 7.

[0054] The “Content” in this disclosure refers to the presentation of sensory stimuli to users subject to environmental control. The sensory stimuli presented by this “Content” include at least one of the following: display control, lighting control, sound control, air conditioning control, and scent control. In other words, the presentation of the “Content” in this disclosure means presenting the content to the user using at least one of the following: display, lighting, sound, air conditioning, and scent. In this disclosure, “at least one selected from the group consisting of A1, A2, ..., and An” may be interpreted as “A1, A2, ..., An, or any combination of A1, A2, ..., An,” where n is an integer of 2 or more. For example, at least one of A1 and A2 may be interpreted as “A1,” “A2,” or “A1 and A2.” For example, at least one of A1, A2, and A3 may be interpreted as "A1", "A2", "A3", "A1 and A2", "A1 and A3", "A2 and A3", or "A1, A2, and A3". "A and / or B" in this disclosure may be interpreted as "A", "B", or "A and B".

[0055] In this embodiment, the "content" described will primarily be an example of content that presents images. In this case, the "content" may present audio along with images, or it may present images only. Here, "image" may be a "moving image (video)" containing multiple frames, or it may be at least one "still image".

[0056] As an example, when environmental control for a target user is initiated, the content selection unit 533 selects welcome content (introductory content). This welcome content is content that raises the expectations of the welcomed user, that is, content that enhances the user's sense of exhilaration through the welcome. For this reason, welcome content may also be called exhilarating content. For example, welcome content includes messages that welcome the user, such as "Welcome" or "Username". With this configuration, the user's sense of exhilaration is enhanced by being welcomed, and it is possible to effectively guide them into a more "highly happy and highly aroused" state. Here, the welcome content in this embodiment is an example of the third type of content.

[0057] As an example, the content selection unit 533 presents welcome content and then selects high-excitement and high-awakening content (introductory content).

[0058] High-pleasure x high-arousal content is content that increases the user's "arousal level" and induces "pleasant emotions (positive emotions)" in the user. In other words, high-pleasure x high-arousal content is content that guides the user into a state of "high pleasure x high arousal" by increasing both "pleasure level" and "arousal level". For example, high-pleasure x high-arousal content is content related to magnificent nature or live music performances. As an example, high-pleasure x high-arousal content includes footage of a waterfall, heavy rain, crashing waves, or a powerful river. This configuration can effectively guide the user into a state of "high pleasure x high arousal". Here, the high-pleasure x high-arousal content according to the embodiment is an example of the first type of content. The state of "high pleasure x high arousal" is an example of the first state.

[0059] As an example, if the content selection unit 533 presents high-stimulation x high-awareness content and determines that the user's state while the content is being presented is "high-stimulation x high-awareness," it selects arousal-suppressing content (introductory content). As another example, if the content selection unit 533 presents high-stimulation x high-awareness content and determines that the user's state while the content is being presented is not "high-stimulation x high-awareness," it selects other high-stimulation x high-awareness content. As yet another example, if the content selection unit 533 changes the high-stimulation x high-awareness content a predetermined number of times, which are stored in the memory unit 52 or the like, and still does not determine that the user's state is "high-stimulation x high-awareness," it selects arousal-suppressing content (introductory content). In this way, a configuration that switches content based on the user's state can more reliably guide the user to the desired state.

[0060] Arousal-suppressing content is content that induces a user's "pleasant feelings" and lowers their "arousal level." In other words, arousal-suppressing content is content that guides the user's state to a "highly pleasurable x moderately aroused" state, while maintaining a "highly pleasurable" state and lowering their "arousal level" from a "highly aroused" state. For example, arousal-suppressing content is content related to videos and sounds that encourage deep breathing. As an example, arousal-suppressing content includes videos and / or audio that encourage the user to take deep breaths. This configuration can effectively lower the arousal level. Here, the arousal-suppressing content according to the embodiment is an example of the second type of content. The "highly pleasurable x moderately aroused" state is an example of the second state.

[0061] For example, if the content selection unit 533 presents arousal-suppressing content and determines that the user's state while the arousal-suppressing content is being presented is "highly relaxed x moderately aroused," it selects the creativity content specified by the user. For example, if the content selection unit 533 presents arousal-suppressing content and determines that the user's state while the arousal-suppressing content is being presented is not determined to be "highly relaxed x moderately aroused," it selects other arousal-suppressing content. For another example, if the content selection unit 533 changes the arousal-suppressing content a predetermined number of times, which are stored in the memory unit 52 or the like, and the user's state is not determined to be "highly relaxed x moderately aroused," it selects the creativity content specified by the user. In this way, a configuration that switches content based on the user's state can more reliably guide the user to the desired state.

[0062] This creative content is content designed to guide or maintain a user's state of focus and / or creativity during a task. Creative content, for example, is content that, upon presentation, can evoke (or give) a sense of "awe" in the user. With this configuration, the user can exhibit a high level of creativity and / or focus during a task. Here, the creative content in this embodiment is an example of the fourth type of content. It should also be noted that since creative content is content presented during work (tasks) performed by a user in a state of focus and / or creativity, it can also be called task content.

[0063] "Awe" refers to the feeling of reverence and respect for something sublime or a great person, and describes being overwhelmed by grandeur or magnificence. For example, videos that users (subjects) might feel awe for include natural landscapes, historical buildings, and skyscrapers. On the other hand, there are cultural and individual differences in what evokes awe, so users may feel awe for other types of videos not mentioned above. Such videos could include slow-motion footage, religious ceremonies, or images of artisans at work.

[0064] For example, if the content selection unit 533 determines, after starting to present creative content, that the user's concentration and / or creativity has decreased while the creative content is being presented, or that the user is not in a "highly alert x moderately awake" state in which they can concentrate and / or be creative, it may select other creative content. In this way, a configuration that switches content based on the user's state can more reliably guide the user to the desired state.

[0065] Furthermore, the content selection and modification by the content selection unit 533 may be carried out in accordance with the individual's tastes and preferences; for example, content with the same label (see Figure 9) may be selected.

[0066] The presentation control unit 534 controls the presentation of content to the user by the presentation device 7. Specifically, the presentation control unit 534 generates presentation control information to present the content selected by the content selection unit 533, and transmits the generated presentation control information to the presentation device 7 via the communication unit 51. If the determination unit 532 determines that a rest is necessary while creative content is being presented, i.e., while the user is performing a task, the presentation control unit 534 generates presentation control information to interrupt (stop) the presentation of the creative content, and transmits the generated presentation control information to the presentation device 7 via the communication unit 51.

[0067] The storage device 6 is an information processing device configured to store information about content. As shown in Figure 1, the storage device 6 stores a content database 61. The content database 61 will be described later (see Figure 9).

[0068] The presentation device 7 is a device configured to perform the presentation of content to the user. The presentation device 7 presents content to the user in accordance with the control of the presentation control unit 534 of the signal processing device 5. The presentation device 7 is realized by at least one of a display (display member), a lighting device, an audio device, an air conditioning device, and a fragrance presentation device.

[0069] For example, the display device 7, acting as a display, performs actions such as turning the display on / off, displaying an image, and changing the display layout based on display information (display control information) from the signal processing device 5. For example, the display information from the signal processing device 5 may include data of the image to be displayed. Alternatively, if, for example, the image to be displayed is stored in the internal memory of the display device 7, or if the display device 7 acquires an image from a storage device 6 or the like, the display information may be information that specifies the image to be displayed.

[0070] As an example, the presentation device 7, which acts as a lighting device, performs at least one of the following actions based on lighting information (presentation control information) from the signal processing device 5: turning the lighting on / off, changing the light intensity (illuminance), changing the lighting color, or changing the light distribution (direction).

[0071] As an example, the presentation device 7, as an audio device, performs at least one of the following actions based on the audio information (presentation control information) from the signal processing device 5: turning the sound on / off, changing the volume, changing the timbre, or changing the direction of the sound.

[0072] As an example, the display device 7, acting as an air conditioning system, performs at least one of the following actions based on air conditioning information (display control information) from the signal processing device 5: turning the fan on / off, turning the air conditioning on / off, changing the airflow, changing the temperature, changing the humidity, or changing the airflow direction.

[0073] As an example, the presentation device 7, acting as a fragrance presentation device, performs at least one of the following actions based on fragrance information (presentation control information) from the signal processing device 5: turning fragrance emission on / off, changing the emission amount, or changing the type of fragrance.

[0074] Figure 2 shows an example of the hardware configuration of an information processing device 8 that implements each device included in the display system 1 according to the embodiment.

[0075] The information processing device 8 is a device that comprehensively controls the operation of each of the implemented devices. As shown in Figure 2, the information processing device 8 has a processor 81, a main memory 82, an auxiliary storage device 83, and a device interface 84. The processor 81, main memory 82, auxiliary storage device 83, and device interface 84 are interconnected by a bus or the like, and the hardware configuration is that of a normal computer. Note that each component of the information processing device 8 may be implemented by a combination of two or more components.

[0076] The processor 81 is, for example, at least one CPU (Central Processing Unit). The processor 81 comprehensively controls the operation of the information processing device 8, for example by executing a program, and realizes each of the functions of the information processing device 8. Some or all of the functions of the information processing device 8 may be realized by dedicated hardware circuits.

[0077] Here, the processor 81 according to this embodiment is an example of at least one processor in the information processing device 8. Instead of the CPU, or in addition to the CPU, at least one other processor may be used as this at least one processor. As the other processor, various processors such as a CPU, GPU (Graphics Processing Unit), DSP (Digital Signal Processor), or dedicated arithmetic circuits implemented with ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array) can be used as appropriate.

[0078] As an example, in each device of the display system 1, the processor 81 may load a program stored in the auxiliary storage device 83 into the main storage device 82 and execute it, thereby realizing the functions of each device of the display system 1, including the functions illustrated in Figure 1. In the example in Figure 1, only the functions necessary for explaining the main parts of this embodiment are illustrated, but the functions of each device of the display system 1 are not limited to these.

[0079] In addition, in each device of the display system 1 according to the embodiment, two or more components may be integrated. For example, in the signal processing device 5, two or more functional units of the arithmetic processing device 53 may be integrated and implemented as a single functional unit. Some of the functional configurations and hardware configurations of each device of the display system 1 according to the embodiment may be implemented by an external information processing device 8 for each device. For example, the storage unit 52 and the storage device 6 of the signal processing device 5 may be integrated.

[0080] The main memory 82 is, for example, at least one RAM (Random Access Memory). The main memory 82 temporarily stores data necessary for various processes performed by the processor 81. Here, the main memory 82 in this embodiment is an example of at least one memory in the information processing device 8.

[0081] The auxiliary storage device 83 is, for example, at least one ROM (Read Only Memory). The auxiliary storage device 83 stores programs, parameters, and various data used for processing that realize various processes by the processor 81. Here, the auxiliary storage device 83 according to this embodiment is an example of at least one memory in the information processing device 8. As this at least one memory, at least one other memory may be used instead of the ROM, or in addition to the ROM. As other memories, various storage media and storage devices such as HDD (Hard Disk Drive), SSD (Solid State Drive), and Flash memory can be used as appropriate.

[0082] As an example, in each device of the display system 1, the main memory 82 and the auxiliary storage device 83 may realize the internal memory of each device. For example, in the information processing device 8 that implements the signal processing device 5, the main memory 82 and the auxiliary storage device 83 may realize the storage unit 52.

[0083] The device interface 84 is an interface for various inputs / outputs and / or communications in the information processing device 8. For example, the device interface 84 may include an output interface configured to connect to an external output device that outputs sound, images, or video, or to function as such an output device. As output devices, various displays (display components) such as liquid crystal displays (LCDs), organic EL (Electroluminescence) displays, projectors, and tablet PCs, as well as speakers, can be used as appropriate. For example, the device interface 84 may include an input interface configured to connect to an external input device that acquires user operations, or to function as such an input device. As input devices, keyboards, mice, touch panels, microphones, and tablet PCs can be used as appropriate. For example, the device interface 84 may include a communication interface configured to connect to an external communication device that communicates with the outside of the information processing device 8, or to function as such a communication device. As a communication interface, wired communication circuits such as USB (Universal Serial Bus®) and Ethernet®, or wireless communication circuits compatible with various standards such as 3G, LTE, 4G, 5G, 6G, Wi-Fi®, Bluetooth®, and infrared communication can be used as appropriate.

[0084] As an example, in each device of the display system 1, the device I / F 84 may implement the user interface, input / output interface, and communication interface for each device.

[0085] Each device included in the display system 1 may be implemented by a common information processing device 8 shared with other devices, or each may be implemented by a different information processing device 8.

[0086] (Knowledge forming the basis of this disclosure) Herein, we will explain the knowledge discovered by the inventors of this application.

[0087] Human creative thinking can be divided into two types: "convergent thinking," which involves logically progressing through known information to arrive at a single solution, and "divergent thinking," which involves exploring known information to generate new ideas. Here, "divergent thinking" is what contributes to the user's "creativity." In other words, users in a state of "divergent thinking" are in a state where their creativity is highly expressed. Positive emotions, or in other words, pleasant emotions, have been reported to broaden users' cognition and thinking in a wide range of ways, not just creativity, by broadening attention and enhancing overall cognition and processing (Fredrickson, B.L., & Branigan, C. 2005. Positive emotions broaden the scope of attention and thought-action reporters. Cognition and Emotion, 19, 313-332). "Awe," as an example of a positive emotion, has been reported to contribute to creativity (Kazuki Sawada, Michio Nomura, "The effect of awe on divergent thinking," 18th Annual Meeting of the Japanese Cognitive Psychology Association (2021)).

[0088] In this context, there are known techniques that enhance creativity by presenting content that evokes positive emotions in users, thereby guiding them into a state where they can unleash their creativity. For example, creativity improves in a state of "high pleasure x moderate arousal," which is a state of "high pleasure" but not excessively aroused, or "moderate arousal." However, content that aims to induce a "high pleasure x moderate arousal" state in users has the problem that the stimulation is weak and it is not possible to guide users to the desired state in a short amount of time. On the other hand, content that aims to induce a "high pleasure x high arousal" state in users has the problem that the stimulation is strong and can guide users to the desired state in a short amount of time, but it is not suitable for tasks that require concentration and / or creativity because users do not pay attention to the task at hand.

[0089] Therefore, the inventors of this application conducted the following experiment to clarify the conditions under which content presentation can induce a state in which concentration and / or creativity can be exercised.

[0090] (Experimental Procedure) Figures 3 and 4 are diagrams illustrating the measurement conditions for an experiment that yielded the knowledge that formed the basis for the environmental control described in the embodiment.

[0091] Figure 3 illustrates the overall flow of the experiment. As shown in Figure 3, the experiment was conducted by performing a task requiring creative thinking twice, following explanation and practice for each measurement condition. After each task, participants were given a questionnaire regarding their creativity during the task. The results of this questionnaire were compiled as a creativity score, which is a rating of creativity.

[0092] This experiment was conducted under two measurement conditions: a "selective environment" condition in which participants performed a task under environmental control that presented creative content aimed at improving creativity, and a "no environmental control" condition in which no environmental control was performed. Regarding the "selective environment" condition, the experiment was conducted under two measurement conditions: a "preliminary content" condition in which preliminary content was presented before the start of the task and creative content was presented during the task, and a "no preliminary content" condition in which no preliminary content was presented before the start of the task, and only creative content was presented during the task.

[0093] "Case A" in Figure 4 illustrates the flow of each task in an experiment under the measurement conditions of "with introductory content × selectable environment". "Case B" in Figure 4 illustrates the flow of each task in an experiment under the measurement conditions of "without introductory content × selectable environment".

[0094] In "Case A," before the task begins, introductory content is presented that aims to guide the user into a state where they can concentrate and / or exercise their creativity (S1-S2). Specifically, a preview video, which is introductory content aimed at building the user's expectations, is presented (S1). Following the presentation of the preview video, welcome content (third type of content) is presented that aims to raise the user's expectations, highly stimulating and arousal content (first type of content) aimed at changing the user's mood / refreshing them, and arousal-suppressing content (second type of content) aimed at relaxing the user is presented (S2).

[0095] On the other hand, in "Case A," the presentation of introductory content (S1-S2) is not carried out.

[0096] Subsequently, the user performs a task in a selection environment where creative content (the fourth type of content) is presented (S3-S4). Specifically, the user selects an environment in which they choose creative content (S3). This environment selection aims to give the user a sense of control by allowing them to choose their preferred environment. After that, the creative content is presented, and the task begins in the selection environment where the creative content is presented, that is, amidst the sensory stimulation of the user's preferred environment (S4). After the task is completed, a refresh is performed, presenting content aimed at changing the user's mood / refreshing (S5). After the refresh, the flow of each task is completed.

[0097] In the experiment under the controlled condition of "no environmental control," the flow of each task is as follows: the task is started in an environment where predetermined creative content is presented (corresponding to S4 in Figure 4), a refresh is performed after the task is completed (corresponding to S5 in Figure 4), and the experiment ends after the refresh.

[0098] (Experimental Results) Figures 5 and 6 are diagrams illustrating the experimental results of an experiment that yielded the knowledge that formed the basis for the environmental control according to the embodiment. Figure 5 illustrates the creativity score obtained under the measurement condition of "with introduced content × selected environment" along with the creativity score obtained under the control condition of "no environmental control". Figure 6 illustrates the creativity score obtained under the measurement condition of "no introduced content × selected environment" along with the creativity score obtained under the control condition of "no environmental control". Figure 7 is a diagram illustrating the knowledge regarding the relationship between introduced content and the guidance of the user's state, which formed the basis for the environmental control according to the embodiment.

[0099] As shown in Figures 5 and 6, under the measurement condition of "introductory content present × choice environment," creativity was found to be statistically significantly improved at a 5% significance level (p < 0.05). Here, "p" represents the p-value, which is the realized value of probability in statistics and is an indicator used to evaluate significant differences. On the other hand, under the measurement condition of "no introductory content × choice environment," i.e., when only creative content aimed at improving creativity was presented, no significant difference was obtained compared to the control condition, indicating that creativity did not improve. This is due to the variability in the initial state of the subjects at the start of the task (start of presentation of creative content), and the fact that the creative content with weak stimulation did not induce a state in which creativity could be expressed in a short time, i.e., a "highly pleasurable × moderately aroused" state.

[0100] Therefore, by presenting introductory content, it is possible to appropriately guide the user's state to one in which they can concentrate and / or exercise their creativity. Specifically, as shown in Figure 7, the user's state is guided to a "highly pleasurable x highly aroused" state by presenting the introductory content (Introduction 1), and then to a "highly pleasurable x moderately aroused" state where the level of arousal is not too high (Introduction 2). In this way, by using highly stimulating, highly pleasurable x highly aroused content to guide the user to a uniform state before the start of the task, it is possible to guide the user's state to one in which they can concentrate and / or exercise their creativity in a short time, even if there is variation in the initial state of the subjects.

[0101] Based on these findings, in the environmental control described in this disclosure, the content that guides the user's state is separated into introductory content and creativity content. Specifically, the environmental control described in this disclosure is configured to guide the user in two stages by presenting introductory content before presenting creativity content.

[0102] In the environmental control described in this disclosure, the introductory content is presented before the task begins. However, the introductory content only needs to be presented prior to the presentation of the creative content, and some or all of the introductory content may be presented after the task begins.

[0103] The following describes an example of the operation of the display system 1 according to the embodiment, with reference to the drawings. Note that the process described below is just one example, and it is possible to change the order of processing, delete some processes, or add other processes.

[0104] Figure 8 is a flowchart showing an example of the flow of environmental control processing performed in the display system 1 according to the embodiment.

[0105] First, the signal processing device 5 determines whether it has detected a user entering a predetermined space to which the display system 1, according to an embodiment such as a private room (booth) or a vehicle compartment, is applied (S101).

[0106] For example, the signal processing device 5 detects a user entering a room when a start button located within a predetermined space is operated. For example, the signal processing device 5 is configured to detect a user sitting in a seat located within a predetermined space, for example, by a sensor on the seat or a camera in the space, and detects a user entering the room when sitting in the seat is detected. For example, the signal processing device 5 is configured to detect the opening and closing of a door located at the entrance to a predetermined space, for example, by a sensor that detects the opening and closing of the door or contact and rotation of the doorknob, or by a camera, and detects a user entering the room when the opening and closing of the door is detected.

[0107] If the system does not detect a user entering the room (S101: No), the signal processing device 5 repeats the process of S101, for example, at predetermined intervals. On the other hand, if the system detects a user entering the room (S101: Yes), the signal processing device 5 presents welcome content to the user using the presentation device 7 (S102).

[0108] Subsequently, the signal processing device 5 selects high-speed and high-awareness content by referring to a predetermined content database 61 stored in the storage device 6, and presents the selected high-speed and high-awareness content to the user using the presentation device 7 (S103).

[0109] Figure 9 shows an example of the configuration of a content database 61 according to an embodiment. As shown in Figure 9, the content database 61 has a memory configuration that includes a "content type" indicating whether it is "introductory content" presented before the start of a task or "creativity content" presented during a task, a "content type" indicating the type of content, a "content No." indicating identification information for uniquely identifying the content, an "awareness level" indicating the awareness level rating of each content, a "pleasantness level" indicating the pleasure level rating of each content, a "awe" level indicating the "awe" level rating of each content, and a "label" indicating the overview or attributes of each content. In other words, in the content database 61, the "content No." of each piece of content is associated with the "content type," "content type," "awareness level," "pleasantness level," "awe," and "label." With this configuration, by quantifying the impact of the content on the user, it is possible to effectively guide the user to states of high pleasure × high awareness, decreased awareness, high creativity and / or concentration. When boredom sets in, it becomes possible to switch to similar content, for example, by selecting content with the same label to match the user's personal tastes and preferences. This is highly effective in guiding or maintaining each state. Here, the "Arousal Level" evaluation value is an example of an arousal evaluation index value that shows the effect of increasing arousal. The "Pleasure Level" evaluation value is an example of a pleasure evaluation index value that shows the effect of inducing pleasant feelings.

[0110] The evaluation values ​​stored in the content database 61 may be obtained based on user input, pre-set for each piece of content, or updated based on user input or estimated user status. The content database 61 may be shared among multiple users, or it may be provided for each user or user attribute.

[0111] Furthermore, content is considered "high-comfort x high-awareness content" if, for example, the rating values ​​for "comfort" and "awareness" satisfy predetermined threshold conditions stored in the memory unit 52. In the example in Figure 9, "high-comfort x high-awareness content" is, for example, content where the rating value for "comfort" is "6" or higher, and the rating value for "awareness" is "6" or "7" or higher.

[0112] The signal processing device 5 then acquires the user's biometric information from the sensor 3 (S104) and estimates the user's state based on the acquired biometric information (S105). The user's state may also be acquired based on the user's input to the user interface 4. For example, the user may be notified (questionnaire) with the message "Are you relaxed?" along with "Yes" and "No" operation buttons. After that, the signal processing device 5 determines whether it was able to induce a "highly relaxed x highly aroused" state (S106).

[0113] If the user is unable to be guided into a "highly relaxed and highly aroused" state (S106: No), the signal processing device 5 determines whether the content has been changed within a predetermined number of times (S107). If the content has been changed within a predetermined number of times (S107: Yes), the signal processing device 5 refers to the content database 61 to select another highly relaxed and highly aroused content (S108), and presents the newly selected highly relaxed and highly aroused content to the user using the presentation device 7 (S103).

[0114] On the other hand, if the user is induced into a "highly relaxed and highly aroused" state (S106: Yes), or if the number of content changes exceeds a predetermined number (S107: No), the signal processing device 5 refers to the content database 61 to select arousal-suppressing content and presents the selected arousal-suppressing content to the user using the presentation device 7 (S109).

[0115] Furthermore, content is considered "arousal-suppressing content" if, for example, the rating values ​​for "pleasure" and "arousal" in the content database 61 satisfy predetermined threshold conditions stored in the storage unit 52. In the example in Figure 9, "arousal-suppressing content" is, for example, content where the rating value for "pleasure" is "6" or higher, and the rating value for "arousal" is between "0" and "2".

[0116] Then, the signal processing device 5 acquires the user's biometric information from the sensor 3, for example, in the same manner as in the process of S104 (S110), and estimates the user's state based on the acquired biometric information, for example, in the same manner as in the process of S105 (S111). After that, the signal processing device 5 determines whether it was possible to induce a "highly relaxed x moderately alert" state in which the level of alertness is reduced while maintaining a highly relaxed state (S112).

[0117] If the user is unable to be induced into a "highly relaxed x moderately aroused" state (S112: No), the signal processing device 5 determines, for example, in the same manner as in the process of S107, whether the content has been changed within a predetermined number of times (S113). If the content has been changed within a predetermined number of times (S113: Yes), the signal processing device 5 selects another arousal-suppressing content by referring to the content database 61, for example, in the same manner as in the process of S109 (S114), and presents the newly selected arousal-suppressing content to the user using the presentation device 7 (S109).

[0118] On the other hand, if the user is induced into a "highly relaxed x moderately aroused" state (S112: Yes), or if the content changes exceed a predetermined number of times (S113: No), the signal processing device 5 refers to the content database 61 and selects a candidate for creative content. Based on the user operation result, for example, of specifying creative content for the user interface 4, the signal processing device 5 selects a candidate for creative content specified by the user as the content to present, and presents it to the user by the presentation device 7 (S115).

[0119] Figure 10 is a diagram showing an example of a display screen 701 displayed in the display system 1 according to the embodiment. As shown in Figure 10, the display screen 701 includes, for example, at least one tab 711 that instructs various screen transitions, and an icon 721 that indicates at least one video (creative content) of selection candidates. Tab 711 may include a "Home" tab 711a that displays a predetermined initial screen, such as presenting at least one video of selection candidates based on the user's viewing history, depending on the selection. Tab 711 may include a "Recommended" tab 711b that presents at least one recommended video, depending on the selection. Tab 711 may include a "Favorites" tab 711c that presents at least one video that the user has registered as a favorite, depending on the selection. Tab 711 may include a "Popular" tab 711d that presents at least one video that is popular with other users, depending on the selection. Depending on the selection, tab 711 may also include a "Settings" tab 711e that displays a settings screen for making various settings, such as registering user information 521 and entering ratings to be stored in the content database 61.

[0120] Furthermore, the at least one candidate video (creative content) displayed in the "Favorites" tab 711c may be updated as appropriate according to the content selected in S115 of Figure 9, i.e., according to the target user's specification. The at least one candidate video displayed in the "Popular" tab 711d may also be updated according to the specification results for multiple users.

[0121] In the display system 1 according to this embodiment, the specified result may be linked to user information 521 and stored in the storage unit 52. When saving the specified result, a questionnaire such as "Were you able to concentrate?", "Was your creativity high?", "Did you feel a sense of awe?", or "Would you like to add this to your favorites?" may be displayed by the display device 7 or user interface 4, and the input results to the user interface 4 may be obtained.

[0122] Then, when the task is completed or when it is determined that a rest is needed, the signal processing device 5 terminates the presentation of creative content and presents feedback information to the user via the presentation device 7 or user interface 4 (S116). After that, the flow shown in Figure 8 ends.

[0123] Here, feedback information refers to information indicating the user's creativity and / or concentration state in relation to a task performed by the user while the creative content is being presented by the presentation device 7. The user's creative state during the task may be determined based on the user's biometric information acquired using the sensor 3. For example, as described in WO2024 / 048163, information indicating the user's creative state may be generated based on at least one index selected from an index including the amount of change in the user's face position, an index including the amount of change in the user's face orientation, an index including the amount of change in the user's gaze direction, and an index including the percentage of the user's eyes closed. The information indicating the user's creative state may be information indicating whether or not the user is demonstrating creativity, or information (score) indicating the degree to which the user is demonstrating creativity. The method for determining the user's creative state during the task is not limited to the above example, and known methods may be used. The user's concentration state during the task may be determined based on the user's biometric information acquired using the sensor 3. For example, as described in JP 2023-114859, information indicating the user's concentration state may be generated based on the user's posture and / or facial content. Figure 11 shows an example of a display screen 751 displayed in the display system according to the embodiment. As shown in Figure 11, feedback to the user after task completion may be provided by feedback information 752a indicating the state of concentration and / or creativity during the task, such as "total score," "individual score," and "comments." The "individual score" may be a score indicating the degree of creativity exhibited during the period in which each of the multiple types of tasks is performed, if the task includes multiple types of tasks. The "total score" may be the average value of the individual scores. The "comments" may be stored in memory in advance, corresponding to the value of the total score. Feedback to the user after task completion may also be provided by feedback information 752b indicating the proportion of time spent in a state of concentration and / or creativity during the task.The time percentage may be derived using the above method based on the time during which the user is determined to be focused and the time during which the user is determined to be exhibiting creativity. Feedback to the user after task completion may be provided by feedback information 752c showing the time-series progression of the user's creative state during the task. This time-series progression may be derived based on the time-series change of a score indicating the degree of the user's creativity, determined using the above method. Although not shown in the illustration, feedback information 752c showing the time-series progression may show the time-series progression of the user's focus state during the task. Feedback to the user after task completion may be provided by feedback information such as comments or graphs that show when creativity was high / low during the task, or icons in the graph, or images of the user at the time when creativity was high / low. Feedback to the user after task completion may also link the user's focus and / or creative state during the task with the content presented.

[0124] By presenting this feedback information 752, it is possible to improve the customer's satisfaction, provide insights for future visits, and increase the repeat rate of the service provided through the environmental control of the display system 1.

[0125] As described above, the environmental control realized by the display system 1 according to this embodiment is configured to guide the user in two stages by presenting introductory content before creative content. Specifically, the environmental control according to this embodiment is configured to direct the user's attention to the surrounding environment during the introduction, and then to the task once a decrease in alertness is detected. With the above configuration, which guides the user to a uniform state before the start of the task, even if there is variation in the user's initial state, it is possible to guide the user's state to a state in which they can concentrate and / or exercise their creativity in a short time. In other words, the environmental control according to this embodiment makes it possible to create an surrounding environment that can appropriately guide a person to a state in which they can concentrate and / or exercise their creativity.

[0126] Other embodiments of this disclosure will be described below with reference to the drawings. In the following descriptions of each embodiment, the differences will be explained primarily, and any content that overlaps with the above will be omitted as appropriate.

[0127] (Second Embodiment) The environmental control according to the above embodiment may switch the content presented according to the passage of time, regardless of the user's biometric information.

[0128] Figure 12 is a flowchart showing another example of the flow of environmental control processing performed in the display system according to the embodiment.

[0129] After the signal processing device 5 presents the selected high-refreshment and high-alertness content to the user using the presentation device 7 (S103), it determines whether a predetermined time stored in the memory unit 52 has elapsed (S201). In other words, the signal processing device 5 determines whether the elapsed time during which the high-refreshment and high-alertness content has been presented by the presentation device 7 has exceeded a predetermined threshold time (determined time). If the predetermined time has not elapsed since the start of presentation of the high-refreshment and high-alertness content (S201: No), the signal processing device 5 returns to the process in S103 and continues to present the high-refreshment and high-alertness content.

[0130] On the other hand, if a predetermined time has elapsed since the start of presentation of high-refreshment x high-arousal content (S201: Yes), the signal processing device 5 refers to the content database 61 to select arousal-suppressing content and presents the selected arousal-suppressing content to the user using the presentation device 7 (S109). The signal processing device 5 then determines whether a predetermined time, which has been stored in the memory unit 52 in advance, has elapsed (S202). In other words, the signal processing device 5 determines whether the elapsed time during which the presentation device 7 has presented the arousal-suppressing content has exceeded a predetermined threshold (predetermined time).

[0131] Furthermore, the elapsed time threshold (predetermined time) may be the same or different between high-stimulation / high-arousal content and arousal-suppressing content. The elapsed time threshold may also be defined for each piece of content. For example, the stronger the stimulus of the content, the smaller the elapsed time threshold.

[0132] If a predetermined amount of time has not elapsed since the start of presentation of the arousal-suppressing content (S202: No), the signal processing device 5 returns to the process in S109 and continues to present the arousal-suppressing content. On the other hand, if a predetermined amount of time has elapsed since the start of presentation of the arousal-suppressing content (S202: Yes), the signal processing device 5 refers to the content database 61 and selects a candidate for creative content. For example, based on the user operation result of specifying creative content for the user interface 4, the signal processing device 5 selects the candidate creative content specified by the user as content to present and presents it to the user by the presentation device 7 (S115).

[0133] Thus, the environmental control according to this embodiment is configured to switch the content presented according to the passage of time, regardless of the user's biometric information. With this configuration, there is no need to estimate the user's state based on biometric information, and the cost of processing can be reduced. For example, the display system 1 does not need to be equipped with a sensor 3, which reduces the cost of realizing the display system 1. The user does not need to wear a brain blood flow sensor or heart rate sensor to acquire biometric information, and an environment is created in which the user can concentrate more on the task.

[0134] The following describes an example of the application of the display system 1 related to this disclosure, with reference to the drawings.

[0135] (First Application Example) Figures 13 and 14 are diagrams illustrating an application example of the display system 1 according to the embodiment. This application example illustrates a usage scenario when the display system 1 according to the embodiment is applied to a booth 201 (a predetermined space), which is a private room where user P performs work that requires concentration and / or creativity (creative activity). Figure 13 illustrates a door 211 provided at the entrance to the booth 201. Figure 14 illustrates the interior of the booth 201.

[0136] As shown in Figure 13, the presentation device 7 according to this application example may be provided near the entrance of the booth 201 and may have a display 7d capable of displaying a display screen 771. In the example in Figure 13, a display screen 771a containing welcome content including a message such as "Welcome, Mr. / Ms. **" is shown. This welcome content may be displayed on the displays 7a to 7c in Figure 14, in addition to or instead of the display 7d in Figure 13. When the welcome content is displayed on the displays 7a to 7c in Figure 14, the display 7d near the entrance of the booth 201 may not be provided.

[0137] Figure 13 illustrates a case where the display 7d is installed on the outside of the door 211, but it is not limited to this. The display 7d only needs to be installed so that it is visible from outside the booth 201. For example, the display 7d may be installed on the outside of the door 211 so that it is visible from outside the booth 201 when the door 211 is closed. For example, the display 7d may be installed on the inside (indoor side) of the door 211 so that it is visible from outside the booth 201 when the door 211 is open. For example, the display 7d may be installed on a wall or the like near the door 211 so that it is visible regardless of whether the door is open or closed.

[0138] Furthermore, the doorknob 212 provided on the door 211 may be used to detect when a user enters or leaves the room.

[0139] As shown in Figures 13 and 14, the presentation device 7 according to this application example may further have three displays 7a to 7c that are installed inside the booth 201 and capable of presenting introductory content and creative content. In the example in Figure 14, a display screen 771b including introductory content is shown on display 7a.

[0140] In the examples shown in Figures 13 and 14, each display 7a to 7c is mounted on a wall inside the booth 201. For example, display 7a (first display) is positioned in front of user P (Z-X plane), where user P is directly facing the display when seated. For example, displays 7b and 7c are positioned to the side of display 7a. Here, "to the side of display 7a" means either the left or right side of display 7a. More specifically, "to the side of display 7a" means positioned to the left, upper left, lower left, right, upper right, or lower right of display 7a. Display 7b (second display) is positioned to the left side of user P (Y-Z plane on the -X side), which is horizontal to display 7a (e.g., X direction). Display 7c (third display) is positioned to the right side of user P (Y-Z plane on the +X side), which is horizontal to display 7a (e.g., X direction).

[0141] The heights (positions in the Z-direction) of each display 7a to 7c may all be the same, may differ from each other, or may be partially the same. For example, each of displays 7b and 7c may be positioned to the left, upper left, lower left, right, upper right, or lower right of display 7a. For example, displays 7b and 7c may be positioned to either the left or right side of display 7a. The sizes of each display 7a to 7c may all be the same, may differ from each other, or may be partially the same. Each display 7a to 7c may be configured as a single display whose display surface is bent or curved in two or more directions.

[0142] Figures 13 and 14 illustrate an example where three displays 7a to 7c are arranged in a U-shape, but are not limited to this. The display of the presentation device 7 may be one, two, or four or more surfaces, or each display surface may be arranged in a U-shape or other configuration, or a display area and a non-display area may be provided on any one of the displays.

[0143] As shown in Figures 13 and 14, a camera 301 (sensor 3) is positioned, for example, near the ceiling inside the booth 201, to capture images of the user P. The camera 301 may be configured to acquire sounds emitted by the user P, or a microphone for acquiring the user P's voice may be provided separately inside the booth 201. Inside the booth 201, along with displays 7a to 7c for displaying images, other sensors 3 of the camera 301, a signal processing device 5, and other devices of the presentation device 7 such as speakers (audio equipment) for outputting sound may be arranged.

[0144] The user interface 4 in this application example may be installed as a standalone device inside the booth 201, or it may be implemented by a sensor 3, a signal processing device 5, a storage device 6, a display device 7, or an information processing device such as a PC used by the user to perform tasks.

[0145] Thus, by applying the display system 1 described herein, it is possible to guide users of booth 201 to a desired state through environmental control using sensory stimuli such as images, sounds, airflow, scents, and temperature sensations.

[0146] (Second Application Example) The display system 1 according to the embodiment may be applied to a moving object such as a vehicle. For example, instead of the booth 201 in the first application example, a predetermined space such as the passenger space (cabin) of the moving object may be used. As an example, the passenger space of the moving object may be the passenger compartment of a vehicle.

[0147] Here, the term "mobile entity" can refer to various types of vehicles, including electric vehicles (EVs) powered by a motor, automobiles powered by an engine (internal combustion engine), and hybrid vehicles powered by both an engine and a motor. The mobile entity may be, for example, a passenger car, a truck, or a motorcycle, but it may also be an electric bicycle, an electric scooter, an electric wheelchair, construction machinery, agricultural machinery, a ship, a train, or an airplane. The mobile entity may be configured to move autonomously, or it may be configured to move in response to direct or remote operation by a driver.

[0148] Furthermore, when the display system 1 according to this embodiment is applied to a moving object such as a vehicle, at least one of the devices included in the display system 1, including the sensor 3, signal processing device 5, storage device 6, and presentation device 7, may be integrated with a computer mounted on the moving object. For example, the information processing device 8 that realizes the sensor 3 may be integrated with an in-vehicle sensor such as a drive recorder, or with a computer that controls its operation. For example, the information processing device 8 that realizes the signal processing device 5 may be integrated with various devices mounted on the moving object, such as an HMI (Human Machine Interface) such as a display or speaker, lighting devices in the passenger space, air conditioning devices, and scent presentation devices, or with a computer that controls their operation. For example, the information processing device 8 that realizes the presentation device 7 may be integrated with a car navigation system of the moving object that is configured to realize functions such as route guidance.

[0149] The computers (information processing devices 8) mounted on these mobile devices may be computers such as ECUs (Electronic Control Units) or DCUs (Domain Control Units) such as CDCs (Cockpit Domain Controllers) that integrate multiple ECUs, or OBUs (On Board Units) located inside the mobile device (e.g., a vehicle). Alternatively, the computer mounted on the mobile device may be an external computer installed in the passenger space (cabin) of the mobile device, for example, near the dashboard or in the trunk.

[0150] The computer (information processing device 8) that implements each device included in the display system 1 may be connected to communicate with other computers in other mobile devices via an in-vehicle network including CAN (Controller Area Network), Ethernet®, USB®, etc., within a mobile device (e.g., a vehicle). These other computers in mobile devices include computers configured to communicate with information processing devices located outside the mobile device, such as external servers, via telecommunication lines such as the Internet.

[0151] As an example, the display of the presentation device 7 in this application example is positioned between the front seats (e.g., the driver's seat) and the rear seats in the passenger compartment of a vehicle, so that the image can be viewed by a user P (passenger) seated in the rear seat. As an example, the display of the presentation device 7 in this application example may be positioned in front of the user P when the user P is seated in the rear seat, and may also be positioned to the side of the user P. The display of the presentation device 7 in this application example may be a partition between the front seats or a side or rear window, made using a transparent organic EL display (OLED), or it may be a screen onto which light from a projector placed in the passenger compartment is projected. The outer surface of the door or window, which is the entrance to the passenger space of the mobile vehicle, may be configured to display welcome content.

[0152] Furthermore, the location where the video is displayed may be changed as appropriate to suit the eye level, preferences, seating position, and seating posture of the passenger (user) of the moving vehicle. For example, the user may be able to arbitrarily set the location of the video display, or the vertical and / or horizontal position of each display location.

[0153] Generally speaking, the greater the period, amplitude, and acceleration changes caused by the motion of a vehicle, the more likely one is to experience motion sickness. Motion sickness is most likely to occur in the upward direction, followed by the forward / backward direction, and then the left / right direction. Therefore, from the perspective of suppressing motion sickness, on large-screen displays, content may be displayed within a narrow range at eye level in order to suppress vertical eye and neck movements.

[0154] In each of the embodiments described above, "is A" means at least one of "is A" or "is not A". In other words, in each of the embodiments described above, the determination of "is A" may be achieved by determining whether it is A, by determining whether it is not A, or by determining both of these.

[0155] The programs executed by each device of the display system 1 according to each embodiment described above may be provided as files in an installable or executable format, recorded on a computer-readable recording medium (Computer Program Product) such as a CD-ROM, floppy disk, CD-R, or DVD.

[0156] The programs executed by each device of the display system 1 according to each embodiment described above may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Alternatively, the programs executed by each device of the display system 1 according to each embodiment described above may be provided or distributed via a network such as the Internet.

[0157] The programs executed by each device of the display system 1 according to each of the embodiments described above may be pre-installed and provided in ROM or the like.

[0158] According to at least one embodiment described above, it is possible to create an ambient environment that can appropriately guide a person into a state where they can concentrate and / or exercise their creativity.

[0159] While several embodiments of this disclosure have been described, these embodiments are presented as examples and are not intended to limit the scope of the disclosure. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications are possible without departing from the gist of the disclosure. These embodiments and their variations are included in the scope and gist of the disclosure, as well as in the disclosure described in the claims and its equivalents.

[0160] (Note) The above description of embodiments discloses the following technology. (1) A photodetector that detects light from a user; a display device that presents content to the user; and at least one processor that generates biometric information of the user based on an electrical signal output from the photodetector, determines the state of the user based on the generated biometric information, and controls the display device according to the determined state of the user, wherein the at least one processor outputs control information to the display device for presenting a first type of content that increases the user's level of arousal and induces a pleasant feeling in the user; determines whether the user is in a first state of pleasant and highly aroused state based on the biometric information acquired by the sensor while the display device is presenting the first type of content; if the user is in the first state, outputs control information to the display device for presenting a second type of content that induces a pleasant feeling in the user and decreases the level of arousal after the display device has presented the first type of content; and if the user is not in the first state, outputs control information to the display device for presenting a first type of content different from the first type of content being presented by the display device.(2) A photodetection system comprising: a photodetector that detects light from a user; a display device that presents content to the user; and at least one processor that generates biometric information of the user based on an electrical signal output from the photodetector, determines the state of the user based on the generated biometric information, and controls the display device according to the determined state of the user, wherein the at least one processor outputs control information to the display device for presenting a first type of content that increases the user's level of arousal and induces a pleasant feeling in the user; determines, based on the biometric information acquired by the sensor while the display device is presenting the second type of content, whether the user is in a pleasant state and a second state in which the level of arousal has decreased from a first state in which the level of arousal has decreased; if the user is in the second state, outputs control information to the display device for presenting a fourth type of content, which is content specified by the user, after the second type of content has been presented by the display device; and if the user is not in the second state, outputs control information to the display device for presenting a second type of content different from the second type of content being presented by the display device. (3) A program for causing a computer that controls a display device that presents content to a user to output control information to the display device for presenting a first type of content that increases the user's level of arousal and induces the user's feelings of pleasure, and after the display device has presented the first type of content, to output control information to the display device for presenting a second type of content that induces the user's feelings of pleasure and decreases the user's level of arousal.

[0161] 1 Display System 201 Booth 211 Door 212 Doorknob 3 Sensor 4 User Interface 5 Signal Processing Unit 51 Communication Unit 52 Storage Unit 521 User Information 53 Arithmetic Processing Unit 531 Acquisition Unit 532 Judgment Unit 533 Content Selection Unit 534 Presentation Control Unit 6 Storage Device 61 Content Database 7 Presentation Device 7a, 7b, 7c, 7d Display (Display Component) 701, 751, 771 Display Screen 711 Tab 721 Icon 752 Feedback Information 8 Information Processing Device 81 Processor 82 Main Memory 83 Auxiliary Storage Device 84 Device I / F P User

Claims

1. A control device comprising at least one processor that controls a display device that presents content to a user, wherein the at least one processor outputs control information to the display device for presenting a first type of content that increases the user's level of arousal and induces a pleasant feeling in the user, and after the display device has presented the first type of content, outputs control information to the display device for presenting a second type of content that induces a pleasant feeling in the user and decreases the user's level of arousal.

2. The control device according to claim 1, wherein at least one processor acquires the user's biometric information from a sensor, determines whether the user is in a first state of well-being and high alertness based on the biometric information acquired by the sensor while the presentation device is presenting the first type of content, outputs control information for presenting the second type of content if the user is in the first state, and outputs control information to the presentation device for presenting a first type of content different from the first type of content being presented by the presentation device if the user is not in the first state.

3. The control device according to claim 1, wherein at least one processor determines whether the elapsed time during which the presentation device has presented the first type of content exceeds a predetermined threshold, and outputs the control information for presenting the second type of content if the elapsed time exceeds the threshold.

4. The control device according to claim 1, wherein the first type of content includes video footage of any of the following: a waterfall, heavy rain, crashing waves, and a fast-flowing river.

5. The control device according to claim 1, wherein the second type of content includes video, audio, or the video and the audio, and each of the video and the audio prompts the user to take a deep breath.

6. The control device according to claim 1, wherein at least one processor outputs control information to the presentation device for presenting a third type of content including a message welcoming the user, and the presentation device presents the first type of content after presenting the third type of content.

7. The control device according to claim 1, wherein at least one processor outputs control information to the presentation device to cause the presentation device to present a fourth type of content, which is content specified by the user, after the second type of content has been presented to the presentation device.

8. The control device according to claim 7, wherein the fourth type of content is content that enhances or maintains the user's creativity and / or concentration on the task during the task.

9. The control device according to claim 7, wherein at least one processor acquires the user's biometric information from a sensor, determines, based on the biometric information acquired by the sensor while the presentation device is presenting the second type of content, whether the user is in a second state where the level of arousal has decreased from a first state where the user is in a comfortable and highly aroused state, outputs the control information for causing the presentation of the fourth type of content if the user is in the second state, and outputs the control information for causing the presentation device to present a second type of content different from the second type of content being presented by the presentation device if the user is not in the second state.

10. The control device according to claim 7, wherein at least one processor determines whether the elapsed time during which the presentation device has presented the second type of content exceeds a predetermined threshold, and outputs the control information for presenting the fourth type of content if the elapsed time exceeds the threshold.

11. The control device according to claim 7, wherein at least one processor generates information indicating the user's creativity with respect to a task performed by the user while the presentation device is presenting the fourth type of content, information indicating the user's state of concentration on the task, or feedback information including the information indicating creativity and the information indicating the state of concentration, and outputs control information to the presentation device for presenting the feedback information.

12. The control device according to claim 1, further comprising a memory, the memory storing content associated with a pleasure evaluation index value indicating the effect of inducing the pleasant feeling and an arousal evaluation index value indicating the effect of increasing the level of arousal, wherein the arousal evaluation index value of the second type of content is lower than the arousal evaluation index value of the first type of content.

13. The control device according to claim 2, wherein the biological information corresponds to at least one of cerebral blood flow, electroencephalogram, heart rate, facial feature quantities, and skeletal movement.

14. An information processing system comprising: a control device according to any one of claims 1 to 13; and a presentation device for presenting the content to the user using at least one of a display, lighting, sound, air conditioning, and scent.

15. An information processing method performed by at least one processor controlling a presentation device that presents content to a user, in an information processing device comprising at least one processor, the method comprising: outputting control information to the presentation device for presenting a first type of content that increases the user's level of arousal and induces a pleasant feeling in the user; and outputting control information to the presentation device for presenting a second type of content that induces a pleasant feeling in the user and decreases the user's level of arousal after the presentation device has presented the first type of content.

Images