Information processing systems, information processing methods, and programs

Abstract

We provide technologies that improve the user experience. [Solution] According to one aspect of the present invention, an information processing system is provided, comprising at least one processor, the processor configured to perform the following steps by reading a program: in the acquisition step, an action of an interactive object that interacts with the user is acquired; in the identification step, the content of feedback to be given to the user based on the action is identified, the feedback is given to the user by non-contact tactile feedback, the non-contact tactile feedback is presented by at least the transport of air quality components to the user; and in the output step, control information is output based on the content of the feedback and an event associated with the user, the control information is used to control the transport of air quality components.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] Patent Document 1 discloses a vehicle air conditioner capable of improving the sense of presence when an occupant receives visual information. In Patent Document 1, when it is determined in step 110 that an automatic driving operation is being performed and it is further determined in step 120 that an occupant's intention to seek a sense of presence is shown, step 150 is executed. In step 150, regardless of the air conditioning heat load in the vehicle interior, the air conditioning unit is operated in a sense-of-presence driving mode so as to blow out air conditioning air into the interior corresponding to the visual information from the visual device.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above technology, there is room for improvement in improving the user experience.

[0005] In view of the above circumstances, the present invention aims to provide a technology for improving the user experience.

Means for Solving the Problems

[0006] According to one aspect of the present invention, an information processing system is provided, comprising at least one processor, the processor configured to perform the following steps by reading a program: in the acquisition step, an action of an interactive object that interacts with a user is acquired; in the identification step, the content of feedback to be given to the user based on the action is identified, the feedback is given to the user by non-contact tactile feedback, the non-contact tactile feedback is presented by at least the transport of air quality components to the user; and in the output step, control information is output based on the content of the feedback and an event associated with the user, the control information is used to control the transport of air quality components.

[0007] This configuration can improve the user experience. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram showing the configuration of Information Processing System 1. [Figure 2] This is a block diagram showing the hardware configuration of server device 10. [Figure 3] This is a block diagram showing the hardware configuration of user terminal 20. [Figure 4] This is a block diagram showing the functions implemented by the user terminal 20 (control unit 21). [Figure 5] This is an activity diagram showing the processing flow for outputting control information that provides predetermined feedback to the user at the user terminal 20. [Figure 6] This is an activity diagram showing the processing flow for providing predetermined feedback to the user at the user terminal 20. [Figure 7] This figure shows an example of a schedule management table T for managing schedule information. [Figure 8] This diagram shows the configuration of the feedback output device 800. [Figure 9] This figure shows an example of the airflow discharge device 900. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. The various features shown in the embodiments below can be combined with each other.

[0010] Incidentally, the program for implementing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or it may be provided as a downloadable medium from an external server, or it may be provided so that the program is launched on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] Furthermore, in various information processing according to one embodiment, an input and an output corresponding to the input can be realized. Here, as long as an output is obtained as a result of the input, the form of the information referenced in such information processing (hereinafter referred to as "reference information") is not limited. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a decision formula such as a regression equation constructed by a statistical method), or a pre-trained model that has learned the correlation between input and output in advance, or a large-scale language model that can output a desired result by inputting a prompt.

[0012] Furthermore, in one embodiment, "part" may include, for example, hardware resources implemented by a circuit in a broad sense, and the information processing of software that can be specifically realized by these hardware resources. Also, in one embodiment, various types of information are handled, and this information can be represented, for example, by the physical values ​​of signal values ​​representing voltage and current, the high or low values ​​of signal values ​​as a set of binary bits composed of 0s or 1s, or by quantum superposition (so-called qubits), and communication and calculations can be performed on a circuit in a broad sense.

[0013] Furthermore, a circuit in a broad sense is a circuit realized by combining at least a suitable combination of circuits, circuits, processors, and memory. The processor may be a general-purpose processor or a dedicated circuit. In other words, it includes application-specific integrated circuits (ASICs), programmable logic devices (for example, simple programmable logic devices (SPLDs), complex programmable logic devices (CPLDs), and field programmable gate arrays (FPGAs)), etc.

[0014] 1. Hardware Configuration This section describes the hardware configuration.

[0015] <Information Processing System 1> Figure 1 is a configuration diagram representing information processing system 1. Information processing system 1 comprises a communication line 2, a server device 10 (an example of an information processing device), and a plurality of user terminals 20. The server device 10 and the user terminals 20 are configured to communicate with each other via the communication line 2. The connection between the server device 10 and the user terminals 20 may be wired or wireless.

[0016] In one embodiment of the information processing system 1, the information processing system 1 consists of one or more devices or components. Therefore, even a server device 10 or a user terminal 20 alone can be an example of the information processing system 1. More specifically, the information processing system 1 may include elements selected from the group consisting of the server device 10 and the user terminal 20. Elements that are not selected may not be included in the information processing system 1, but may be electrically connected to the selected elements as external elements. These components will be described below.

[0017] <Server device 10> FIG. 2 is a block diagram showing the hardware configuration of the server device 10. As shown in FIG. 2, the server device 10 includes a control unit 11, a storage unit 12, a communication unit 13, and a communication bus 14. The control unit 11, the storage unit 12, and the communication unit 13 are electrically connected inside the server device 10 via the communication bus 14.

[0018] <Control unit 11> The control unit 11 performs processing and control of the overall operations related to the server device 10. The control unit 11 is, for example, a Central Processing Unit (CPU). The control unit 11 realizes various functions related to the server device 10 by reading a predetermined program stored in the storage unit 12. That is, the information processing by software stored in the storage unit 12 is specifically realized by the control unit 11, which is an example of hardware, and can be executed as each functional unit included in the control unit 11. These will be described in more detail in the next section. Note that the control unit 11 is not limited to being single, and the server device 10 may have a plurality of control units 11 for each function, or a combination thereof.

[0019] <Storage unit 12> The storage unit 12 stores various information defined by the foregoing description. This can be implemented, for example, as a storage device such as a Solid State Drive (SSD) that stores various programs and the like related to the server device 10 executed by the control unit 11, or as a memory such as a Random Access Memory (RAM) that stores temporarily necessary information (arguments, arrays, etc.) related to the calculation of the program. The storage unit 12 stores various programs, variables, etc. related to the server device 10 executed by the control unit 11.

[0020] <Communication unit 13> The communication unit 13 preferably uses wired communication methods such as USB, IEEE1394, Thunderbolt®, and wired LAN network communication, but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, and Bluetooth® communication as needed. In other words, it is more preferable to implement it as a collection of these multiple communication methods. That is, the server device 10 may communicate various information from the outside via the communication unit 13 and the network.

[0021] The server device 10 may be on-premises or in a cloud environment. In the case of a cloud-based server device 10, for example, it may provide the above-mentioned functions and processing in the form of SaaS (Software as a Service) or cloud computing.

[0022] <User terminal 20> Figure 3 is a block diagram showing the hardware configuration of the user terminal 20. As shown in Figure 3, the user terminal 20 comprises a control unit 21, a storage unit 22, a communication unit 23, a communication bus 24, an input unit 25, an acquisition device 26, and an output device 27. The control unit 21, storage unit 22, communication unit 23, input unit 25, acquisition device 26, and output device 27 are electrically connected within the user terminal 20 via the communication bus 24. The acquisition device 26 and output device 27 may also be connected outside the user terminal 20 via the communication unit 23. The descriptions of the control unit 21, storage unit 22, and communication unit 23 are the same as the descriptions of each part in the server device 10, so redundant explanations will not be repeated.

[0023] <Input section 25> The input unit 25 receives operation inputs made by the user. The operation inputs are transmitted as command signals to the control unit 21 via the communication bus 24. The control unit 21 can perform predetermined controls or calculations based on the transmitted command signals as needed. The input unit 25 may be included in the casing of the user terminal 20 or it may be externally attached. For example, the input unit 25 may be configured as a touch panel integrated with the image output device 271. When the input unit 25 is configured as a touch panel, the user can input tap operations, swipe operations, etc. to the input unit 25. Instead of a touch panel, the input unit 25 can be a switch button, mouse, trackpad, QWERTY keyboard, game controller, etc.

[0024] <Acquisition device 26> The acquisition device 26 is configured to acquire information for identifying events associated with the user. The acquisition device 26 is, for example, a camera provided on the user terminal 20 or a wearable device connected to the user terminal 20. The camera is preferably configured to acquire images of the user and to detect the user's three-dimensional position and movement. The wearable device is worn by the user and is configured to continuously measure the user's biometric information via sensors. Sensors may include, for example, temperature sensors, infrared sensors, light sensors, pressure sensors, potential sensors, chemical sensors, etc. Wearable devices may include, for example, smartwatches, smart rings, smart glasses, smart earphones, etc. The wearable device can send and receive information via wireless communication with the communication unit 23 on the user terminal 20.

[0025] An event is an event that occurs in relation to the user and is contextual information used to identify the situation the user is in in the real world. Events are used to determine the mental workload, which is the degree of concentration the user has on a predetermined task other than the interaction with the interaction target, and to determine whether or not to provide feedback to the user based on the actions of the interaction target, or the manner in which the feedback is provided (e.g., strength). Details of the information acquired by the acquisition device 26 will be described later.

[0026] <Output device 27> The output device 27 is configured to provide the user with feedback based on the actions of the object being interacted with. The feedback is a sensory stimulus that affects the user's five senses. The output device 27 includes an image output device 271, an audio output device 272, a non-contact tactile presentation device 273, a contact-type tactile presentation device 274, and an olfactory presentation device 275. In this embodiment, the non-contact tactile presentation device 273 functions as a presentation device that presents feedback to the user as a non-contact stimulus. The contact-type tactile presentation device 274 functions as a contact-type device that presents feedback to the user as a contact stimulus.

[0027] <Image output device 271> The image output device 271 is a device that can present visual information to the user and may display a graphical user interface (GUI) screen that the user can operate. The image output device 271 may be, for example, a stationary display device, a projector that projects onto a wall, or a head-mounted display device (HMD). The HMD can be a device configured for any application, such as VR (Virtual Reality), AR (Augmented Reality), or MR (Mixed Reality). Examples of stationary display devices include CRT displays, liquid crystal displays, organic EL displays, and plasma displays.

[0028] <Audio output device 272> The audio output device 272 is a device that can present auditory information to the user, and is, for example, headphones or a speaker provided on the image output device 271. The audio output device 272 should preferably be able to output sounds in a frequency range beyond the human audible range. The speaker may be connected to the image output device 271 via an amplifier, or it may be an active speaker with a built-in amplifier. Furthermore, the audio output device 272 may also be an ultrasonic generator.

[0029] <Non-contact tactile presentation device 273> The non-contact tactile presentation device 273 is a device that can present tactile information to a user without physical contact, and is, for example, a device capable of delivering at least air quality components to the user. The non-contact tactile presentation device 273 is, for example, a device that generates airflow such as a fan, or a device that generates warm or cold air including an air conditioner or any temperature control mechanism (such as a heater or Peltier element). In this embodiment, it is preferable that the non-contact tactile presentation device 273 is a voice coil type speaker, which will be described later, and is configured to deliver air quality components accompanied by warmth, as well as to simultaneously provide non-contact sensory stimuli such as smell and sound.

[0030] <Contact-type tactile presentation device 274> The contact-type tactile presentation device 274 is a contact-type device that can present tactile information to the user. Contact-type tactile information includes, for example, vibration and pressure sensation, and the contact-type tactile presentation device 274 is a device that includes a vibration generator or an inflatable and deflatable balloon, for example, installed on an object that the user touches, such as a chair or desk. The contact-type tactile presentation device 274 may also be an ultrasonic generator, installed on various controllers, remote controls, keyboards, etc., or installed on gloves, clothing, belts, etc. worn by the user.

[0031] <Olfactory presentation device 275> The olfactory presentation device 275 is a device that can present olfactory information to the user, and may be, for example, an aroma diffuser.

[0032] 2. Functional Configuration This section describes the functional configuration of this embodiment. Information processing by software stored in the memory unit 22 is concretely realized by the control unit 21, which is an example of hardware, and can be executed as each functional unit included in the control unit 21 (the processor provided by the information processing system 1). In other words, the information processing system includes at least one processor. The processor includes the following parts by reading a program. Each part includes, for example, an acquisition unit, a identification unit, a determination unit, and an output unit. From another perspective, the information processing method includes each step executed in the information processing system. The program causes the computer to execute each step of the information processing system. Each step includes, for example, an acquisition step, a identification step, a determination step, and an output step.

[0033] Figure 4 is a block diagram showing the functions implemented by the user terminal 20 (control unit 21).

[0034] The acquisition unit 211 is configured to acquire various information as an acquisition step. Preferably, the acquisition unit 232 acquires the actions of the interaction target that interacts with the user. In this embodiment, the user associated with the user terminal 20 (hereinafter sometimes simply referred to as the user or the first user) interacts with the interaction target displayed on the image output device 271 via the communication line 2. The interaction target is, for example, a user different from the first user (hereinafter sometimes referred to as the second user). Details will be described later.

[0035] The identification unit 212 is configured to identify the content of the feedback to be given to the user based on the action acquired by the acquisition unit 211 as a identification step. Details will be described later.

[0036] The determination unit 213 is configured to determine, as a determination step, which event is associated with the user based on the action acquired by the identification unit 212. Details will be described later.

[0037] The output unit 214 is configured to output information about the object of interaction to the output device 27 as an output step. In particular, it outputs control information to cause the output device 27 to output feedback based on the content of the feedback determined by the identification unit 212 and the event associated with the user determined by the determination unit 213. Further details will be described later.

[0038] The feedback output from the output device 27 includes visual information, auditory information, tactile information, and olfactory information. "Outputting control information" may include, for example, the control unit 21 generating control signals to control the output device 27.

[0039] When the image output device 271 is controlled, the output unit 214 is configured to output display information to the image output device 271. The display information may be the visual information itself that is generated in a manner that can be seen by the user, such as a screen, image, icon, or text, or it may be rendering information for displaying the visual information on the image output device 271, for example.

[0040] When the audio output device 272 is controlled, the output unit 214 is configured to output audio information to the audio output device 272. The audio information may be the audio generated in the user terminal 20 (character voices, sound effects, background music, ambient sounds, etc.), the audio acquired by the second user's user terminal (speech of the second user, ambient sounds, etc.), or it may be instruction information in the user terminal 20 to cause the control unit 21 to generate audio information.

[0041] When the non-contact tactile presentation device 273 is controlled, the output unit 214 is configured to output non-contact tactile information to the non-contact tactile presentation device 273. The output unit 214 outputs control signals, for example, to control the strength and direction of the airflow provided by the non-contact tactile presentation device 273, or to control the temperature of the temperature control mechanism.

[0042] When the tactile feedback device 274 is controlled, the output unit 214 is configured to output tactile information to the tactile feedback device 274. The output unit 214 outputs control signals, for example, to control the vibration waveform of the vibrations provided by the tactile feedback device 274, or to control the magnitude and direction of the force applied to the user's skin.

[0043] When the olfactory presentation device 275 is controlled, the output unit 214 is configured to output olfactory information to the olfactory presentation device 275. When the olfactory presentation device 275 has multiple cartridges that present a predetermined scent, the output unit 214 outputs a control signal to control which cartridge is supplied with the scent.

[0044] 3. Information Processing 3.1 Overview of Information Processing This section outlines the processes performed in the information processing system 1. The following examples primarily illustrate the case where each process is executed on the user terminal 20. Figure 5 is an activity diagram showing the flow of processing for outputting control information that provides predetermined feedback to the user on the user terminal 20.

[0045] In the user terminal 20, the acquisition unit 211 acquires the action of the interaction target that interacts with the user (activity A101). The identification unit 212 identifies the content of the feedback to be given to the user based on the action (activity A102). The output unit 214 outputs control information based on the content of the feedback and the event associated with the user.

[0046] This approach can help facilitate communication between the user and the object of interaction. In particular, by providing feedback to the user based on events associated with the user, it is possible to reduce user discomfort caused by receiving feedback that does not align with the user's intentions.

[0047] 3.2 Specific Examples of Information Processing This section describes specific examples of processes performed in the information processing system 1. These examples may be included within the scope defined in the overview described above. Figure 6 is an activity diagram showing the flow of processes for providing predetermined feedback to a user at user terminal 20. In the following, an example is given of a situation in which a user associated with user terminal 20 (first user) interacts with a second user associated with another user terminal via communication line 2. That is, an example is given of a case where the object of interaction is the second user.

[0048] In the user terminal 20, the acquisition unit 211 acquires information for identifying event associated with the first user based on the information acquired by the acquisition device 26. In this embodiment, the information for identifying event is the first user's physical information, communication information, and schedule information.

[0049] In this section, we will refer to Figure 6 throughout to explain the processing flow related to a specific example. First, at the user terminal 20, the acquisition unit 211 acquires physical information, communication information, and schedule information, which are information for identifying events (activities A201 to A203).

[0050] In the user terminal 20, the acquisition unit 211 acquires physical information relating to the physical state of the first user (Activity A201). The physical information consists of the movements and posture of the first user's body detected by the acquisition device 26. If the acquisition device 26 is a camera installed on the user terminal 20, the physical information includes, for example, the head coordinates within the imaging range, face orientation information, degree of eye opening, gaze direction, coordinates of the centers of the left and right palms, the amount of change from the previous time, and facial expression (degree of sleepiness). This information can be obtained by applying image processing to the acquired image. The movements and posture of the first user's body can be obtained through a device worn on the head, such as a head-mounted display or AR glasses. Such devices may have built-in accelerometers, gyroscopes, etc., and it is preferable to use the measured values ​​of these sensors.

[0051] The physical information may be the biometric information of the first user. The biometric information may be, for example, heart rate, body temperature, facial recognition, voice pattern, electroencephalogram, electromyogram, skin electrical activity, respiratory rate, oxygen saturation, eye movements, etc., which can be obtained by a wearable device.

[0052] In parallel, the acquisition unit 211 in the user terminal 20 acquires communication information input for transmission from the first user to the second user (Activity A202). The communication information is information input for the purpose of transmission to the second user, such as speech, characters, or pictures detected by the acquisition device 26. For example, it may be the first user's speech voice acquired by the microphone on the user terminal 20, or information input by the first user to the input unit 25. The communication information may be any type or content, or a combination thereof. The communication information may also be the input performed by the first user to a pre-specified keyboard, button, or switch on the input unit 25.

[0053] In parallel, the acquisition unit 211 on the user terminal 20 acquires schedule information related to the first user's planned activities (Activity A203). The schedule information is acquired by accessing the server device 10 from the user terminal 20. The schedule information may be information that has been pre-entered by the first user or another user, such as the second user, in the schedule management database linked to the account associated with the first user on the server device 10. The schedule information may be, for example, a calendar linked to the account associated with the first user, or equipment usage schedules and work plans stored for a predetermined device.

[0054] Following the acquisition of information to identify the event described above, the determination unit 213 in the user terminal 20 determines the event based on at least one or a combination thereof of physical information, communication information, and scheduled information acquired by the acquisition unit 211. Specifically, the determination unit 213 determines the mental workload of the first user based on the event (Activity A204). Mental workload is an indicator of the degree to which the first user is concentrating on a predetermined task other than interaction with the second user. Then, the determination unit 213 determines the feedback level based on the event, indicating the degree to which the feedback has an impact on the first user (Activity A205). The feedback level is a level indicating the degree to which the feedback has an impact on the user. The feedback level is set so that it is lower the higher the degree of concentration on the task in question. The impact of feedback on the user is the strength of the stimulus that the feedback gives to at least one of the user's five senses. For example, the determination unit 213 determines the feedback level so that it is lower the higher the degree of concentration of the first user on a predetermined task related to the event.

[0055] Specifically, the determination unit 213 determines an event based on the first user's physical information (Activity A204). For example, multiple groups may be defined for each numerical value included in the physical information, with predetermined numerical ranges set in advance, and the determination unit 213 may be configured to determine whether the physical information belongs to one of these groups. If the physical information is the movement or posture of the first user's body, for example, multiple groups can be defined using parameters such as those shown below, so as not to overlap with each other.

[0056] In the user terminal 20, the storage unit 22 stores parameters that define physical information, such as the following: ·X: Head coordinates (left and right direction) • Y: Face direction angle (yaw direction) • Z: Angular velocity of the face (yaw direction)

[0057] The memory unit 22 stores information that defines each group, such as the following. This allows the determination unit 213 to determine what kind of event it is. Group 1: X1 < |X| ≤ X2 and Y1 < |Y| ≤ Y2 and 0 ≤ |Z| ≤ Z1 In this case, the head is positioned off-center from the second user's view of the screen and is not facing forward. Group 2: 0 ≤ |X| ≤ X1 and 0 ≤ |Y| ≤ Y1 and Z1 < |Z| ≤ Z2 In this case, the head is in the center of the screen, and is facing almost directly forward, but is turning its head to the left and right. Group 3: 0≦|X|≦X1 and 0≦|Y|≦Y1 and 0≦|Z|≦Z1 In this case, the head is in the center of the screen, facing almost directly forward, and not turning its head from side to side.

[0058] Furthermore, the body movements and postures of the first user to be acquired can include those of any part of the body, such as the head, hands, and feet. It is preferable that each group be appropriately defined in advance in correspondence with the respective body part. In addition, each group may be defined to use a combination of the coordinates and orientation of each body part, degree of eye opening, gaze direction, facial expression, and the amount of change from before a predetermined time.

[0059] Furthermore, the memory unit 22 stores event data associated with the first user's mental workload. This allows the determination unit 213 to determine the first user's mental workload based on the event data. The mental workload corresponds to the degree to which the first user concentrates on an event different from the object of their interaction. Group 1: High mental workload The first user is focused on a specific task that is separate from their interaction with the second user. Group 2: Moderate mental workload The first user is concentrating on a specific task that is separate from their interaction with the second user. Group 3: Low mental workload The first user is focused on interacting with the second user, and their level of concentration on other designated tasks is low.

[0060] More specifically, the determination unit 213 determines an event based on the information transmitted by the first user (Activity A204). Then, it determines the feedback level such that the higher the first user's level of concentration on a predetermined task related to the event, the lower the feedback level becomes.

[0061] The determination unit 213 may determine the level of concentration on a predetermined task according to the type of information being transmitted. For example, it may determine that voice transmission indicates a low level of concentration on the predetermined task (high level of concentration on the conversation with the second user), text transmission indicates a moderate level of concentration on the predetermined task, and information transmitted as images or stamps indicates a high level of concentration on the predetermined task. In all of these cases, it is preferable to determine that the level of concentration on the predetermined task is lower than when no transmission information is detected.

[0062] The determination unit 213 may determine the degree of concentration on a predetermined task according to the content of the transmitted information. For example, the agenda of a meeting in which a dialogue between the first user and the target of the dialogue will be conducted may be identified in advance based on the scheduled information, and the degree of concentration may be determined based on the degree to which the content of the transmitted information is in line with that agenda. In this case, it is preferable that the first user who transmitted the transmitted information containing content that is more in line with the agenda is determined to have a low degree of concentration on the predetermined task.

[0063] More specifically, the determination unit 213 determines an event based on the first user's schedule information (Activity A204). Then, it determines the feedback level such that the higher the first user's level of concentration on a predetermined task related to the event, the lower the feedback level becomes.

[0064] Figure 7 shows an example of a schedule management table T for managing schedule information. The schedule management table T is retrieved based on the calendar associated with the account of the first user and stored in the storage unit 22. The schedule management table T includes information that identifies the time, work content, deadline, level of focus, feedback level, and a notice to be sent to others.

[0065] The work content is set based on input from the first user or another user such as the second user, and is linked to the time. The work content is an example of a predetermined task that is different from the dialogue with the aforementioned dialogue target. The deadline is information indicating the number of days remaining until the deadline set for the work content. The level of concentration may be set by input from the first user or another user such as the second user when the task is set, or it may be configured to be set to the value from when the same or similar task was performed in the past. Note that the work content may be the dialogue with the dialogue target itself (such as a web conference).

[0066] The feedback level is a level indicating the degree of impact the feedback has on the user, determined by the determination unit 213 based on the degree of concentration on the task. The feedback level is set to be lower the higher the degree of concentration on the task. If the task is the dialogue with the object of dialogue, the feedback level is set to be higher the higher the degree of concentration on the task. The notification message to others is a message to other users who have looked at the first user's calendar. The notification message to others may be a message entered by the first user, or it may be a message generated by the control unit 21 based on the feedback level, the degree of concentration on the task, the task content, deadlines, etc.

[0067] If the scheduled information is the usage schedule or work plan of a predetermined device stored for that device, it is preferable that the work content and the required level of concentration are pre-set in association with that device. This allows the determination unit 213 to easily determine the level of concentration and feedback level by identifying the work content included in the user's work plan.

[0068] Refer to Figure 6 again. Following the determination of the various mental workloads described above, the determination unit 213 determines the feedback level (Activity A205), which indicates the degree to which the feedback will affect the first user, based on the determined mental workload of the first user. For example, a first user determined to belong to Group 1 will have a high level of concentration on the given task, so the feedback level will be determined to be low. A first user determined to belong to Group 2 will have a moderate level of concentration on the given task, so the feedback level will be determined to be moderate. A first user determined to belong to Group 3 will have a low level of concentration on the given task, so the feedback level will be determined to be high.

[0069] This allows for greater immersion in the conversation by providing more feedback to the first user who is highly engaged in the interaction with the second user. Conversely, providing less feedback or no feedback at all to the first user who is less engaged in the interaction with the second user reduces the discomfort caused by providing unwanted feedback. Therefore, it is possible to increase the first user's satisfaction with the feedback.

[0070] Meanwhile, separate from the processing on the user terminal 20, the server device 10 acquires information about the second user who is the target of the interaction (Activity A206). The information about the second user includes images (still images or videos) of the second user taken by the camera on the user terminal associated with the second user, the second user's speech and surrounding environment sounds acquired by the microphone, and strings of characters and images entered by the second user into the input device. Furthermore, based on this information, it may be configured so that one or more types of actions are identified from among several types of actions. For example, it may be configured so that the type of action "right-hand punch" is identified based on the movement of the user's right hand. The server device 10 then transmits the information about the target of the interaction, such as information about the actions performed by the second user, to the user terminal 20 (Activity A207).

[0071] Next, in the user terminal 20, the acquisition unit 211 acquires the actions of the second user, who is the subject of the dialogue with the first user, and the identification unit 212 identifies the actions of the second user (activity A208).

[0072] Subsequently, the control unit 21 controls the image output device 271 and the sound output device 272 to display the acquired action (Activity A209). For example, an image related to the second user is displayed on the image output device 271, and the second user's speech and sounds from the surrounding environment are output by the sound output device 272. Alternatively, the identification unit 212 may be configured to identify one or more types of actions from among multiple types of actions based on this information, and to output images and sounds related to the identified type of action. Alternatively, the server device 10 may be configured to output images and sounds related to the identified type of action.

[0073] Meanwhile, in parallel with the processing related to activity A209, the processing related to activities A210 to A212 is executed. First, in the user terminal 20, the identification unit 212 identifies the content of the feedback to be given to the user based on the action of the interaction target acquired by the acquisition unit 211 (activity A210). Preferably, the storage unit 22 stores reference information in which a predetermined action is associated with a predetermined feedback, and the identification unit 212 identifies the content of the feedback based on the content of the action and the reference information stored in the storage unit.

[0074] In the reference information, actions associated with predetermined feedback include, for example, the movement of the object being interacted with meeting predetermined conditions, the state of the surrounding environment of the object being interacted with meeting predetermined conditions, and the content of the object being interacted with meeting predetermined conditions. The movement of the object being interacted with may include, for example, the movement speed or angular velocity of a predetermined part such as the head or limbs exceeding a threshold, or the position of a predetermined part being close to the camera of the object being interacted with and appearing large. The state of the surrounding environment of the object being interacted with may include, for example, the volume of ambient noise in the surroundings exceeding a threshold. The content of the object being interacted with may include, for example, the content of the speech indicating agreement, disagreement, or exclamation to the user, or the volume of the spoken voice exceeding a threshold. In addition, the type of action identified by the identification unit 212 or the server device 10 may be associated with the content of the feedback. The predetermined feedback may have levels of intensity and frequency defined, for example, in correspondence with the type of output device 27.

[0075] Next, in the user terminal 20, the output unit 214 outputs control information (activity A211) based on the content of the feedback and the event associated with the user. The output unit outputs control information based on the content and level of the feedback. In this embodiment, the determination unit 213 determines the manner in which the feedback set for a predetermined action is actually provided to the first user based on the feedback level determined based on the user's event. Then, the output unit 214 outputs control information to the output device 27 to cause the output device 27 to output the feedback in the determined manner.

[0076] For example, if the feedback level determined by the determination unit 213 is "high", the output unit 214 outputs control information so as to provide feedback corresponding to a predetermined action to the first user as defined in the reference information. If the feedback level is "medium", the output unit 214 outputs control information so as to provide feedback corresponding to a predetermined action to the first user weaker or less frequently than defined in the reference information. If the feedback level is "low", the output unit 214 outputs control information so as to provide feedback to the first user even weaker or less frequently than defined in the reference information.

[0077] Furthermore, when the feedback level is "medium," the output unit 214 may output control information to provide feedback to the first user corresponding to a predetermined action, as defined in the reference information. When the feedback level is "high," the output unit 214 may output control information to provide feedback to the first user corresponding to a predetermined action, more strongly or more frequently than defined in the reference information. When the feedback level is "low," the output unit 214 may output control information to provide feedback to the first user less strongly or less frequently than defined in the reference information.

[0078] Then, at the user terminal 20, the output device 27 executes a process to provide feedback to the first user (activity A212). The output unit 214 causes the image output device 271 to display an image related to the acquired action. The output unit 214 also outputs control information to cause the audio output device 272 to display audio related to the acquired action, and to cause the presentation devices (e.g., non-contact tactile presentation device 273, contact tactile presentation device 274, olfactory presentation device 275) to display feedback related to the action. Based on the control information, the output device 27 controls the output device 27 to provide the action and feedback to the first user in a synchronized manner.

[0079] The above is the information processing flow related to the specific example.

[0080] 4. Hardware configuration related to specific examples

[0081] Figure 8 shows the configuration of the feedback output device 800. The feedback output device 800 includes a chair 821 on which the first user 810 sits, a display device 820 on which an image of the second user 811, who is the object of interaction, is displayed, and an airflow discharge device 822, all of which are provided within a member M surrounding the user. The first user makes predetermined inputs from a terminal 830 equipped with an input unit 25. In other words, the information processing system 1 in this embodiment comprises a contact-type device and a presentation device. The contact-type device in this embodiment is configured to present tactile sensations and is incorporated into a contact member (chair 821) that comes into contact with the first user who is seated and interacting with the second user, who is the object of interaction. The presentation device (airflow discharge device 822) is configured to transport air quality components and is incorporated into a peripheral member (member M) surrounding the first user, or into a contact member (chair 821). The tactile sensations in the contact-type device and the transport of air quality components in the presentation device are controlled based on control information.

[0082] In this embodiment, each device constituting the output device 27 is preferably incorporated into a member M surrounding the user in a seated work space such as an office, a vehicle driver's seat, or an operation room for various work equipment. For example, a contact-type device that provides tactile feedback is preferably configured to be incorporated into items that the user comes into contact with while seated, such as a chair, desk, input device, or floor. Furthermore, a presentation device that provides non-tactile feedback is preferably incorporated into a member M surrounding the user that does not come into contact with the user, such as a wall, ceiling, window, display device, partition, shelf, air conditioner, refrigerator, or other furniture or home appliance.

[0083] The display device 820 is an example of an image output device 271 and an audio output device 272. The display device 820 displays an image of the second user, who is the conversation target 811, and outputs the second user's spoken voice from the speaker provided in the display device 820.

[0084] The airflow discharge device 822 is an example of a non-contact tactile presentation device 273. In this embodiment, the airflow discharge device 822 discharges an airflow W corresponding to the feedback identified by the identification unit 212 toward the first user, thereby providing the feedback to the first user as non-contact tactile feedback.

[0085] Non-contact tactile sensation may include the sensation of warmth due to temperature changes. The airflow discharge device 822 provides the first user with a sensation of warmth by releasing high-temperature or low-temperature air quality through a temperature control mechanism such as a Peltier element or heater. In other words, feedback is provided to the first user through non-contact tactile sensation. This non-contact tactile sensation is presented by the transport of at least air quality components to the first user. Control information is used to control the transport of air quality components.

[0086] The airflow discharge device 822 may also function as an olfactory presentation device 275. The odor generation mechanism of the airflow discharge device 822 releases air quality containing various aromatic substances to the first user, thereby providing the first user with a sense of smell.

[0087] Feedback may be further provided to the user through sound. The sound is controlled based on control information. For example, sounds corresponding to applause or exclamations by a second user, or ambient sounds, may be used as feedback. The sound as feedback may be provided to the first user by a speaker built into the display device 820, but it is preferable that it be configured together with the airflow discharge device 822. This allows the first user to be provided with sound feedback synchronized with the non-contact tactile feedback described above, and also allows the control information from the output unit 214 to be easily output together with control information for controlling other feedback.

[0088] The chair 821 is an example of a contact-type tactile presentation device 274. In this embodiment, the chair 821 is equipped with a vibration mechanism, a temperature control mechanism such as a Peltier element or heater, and an expansion / contraction mechanism such as a balloon, and is configured to provide the first user with tactile stimuli such as vibration, temperature changes, and pressure sensations in response to feedback identified by the identification unit 212. Furthermore, the output unit 214 generates control information for controlling the airflow discharge device 822 and the chair 821 all at once, thereby efficiently generating various feedbacks synchronized with the actions of the displayed interactive object. In other words, the feedback is further provided to the first user through tactile sensations in a manner different from the transport of air quality components. The tactile sensations and the transport of air quality components are controlled based on the control information.

[0089] Figure 9 shows an example of an airflow discharge device 900. The airflow discharge device 900 is an example of the airflow discharge device 822 shown in Figure 8, and is an example of a presentation device that discharges high-temperature or low-temperature air quality, and air quality containing various aromatic substances to the user, and also presents sounds. In this embodiment, the presentation device includes one or more voice coil type speakers, and the transport of air quality components and sounds are controlled based on control information.

[0090] The airflow discharge device 900 comprises a housing 910, a discharge unit 920, a vibrating unit 930, a drive unit 940, a temperature control unit 950, and an additive unit 960. The housing 910 defines an internal space S1, and the discharge unit 920 defines a discharge port S2. The device is configured such that an airflow W is discharged from the discharge port S2 in response to the volume of the internal space S1 changing due to the vibration of the vibrating unit 930. The housing 910 is cylindrical, and various cross-sectional shapes such as circular, elliptical, and polygonal can be adopted.

[0091] The vibrating section 930 and the drive section 940 constitute the speaker section, and the vibrating section 930 is configured to be movable relative to the drive section 940. The vibrating section 930 is equipped with a voice coil 931, which is fixed to the vibrating section 930 by being wound around a bobbin 933 that is provided to be fixed to the diaphragm 932 of the vibrating section 930. The shape of the diaphragm 932 is, for example, conical or dome-shaped.

[0092] The drive unit 940 has a frame portion capable of housing the vibrating unit 930, and the frame portion is fixed to the inner wall of the housing portion 910. The frame portion is equipped with a magnet inside, and when the bobbin 933 of the vibrating unit 930 is inserted into the frame portion, the magnet is configured to face the voice coil 931. A signal line (not shown) is connected to the voice coil 931, and a current corresponding to the drive signal (control information) output by the output unit 214 is applied to the voice coil 931. As a result, a force corresponding to the magnetic force of the magnet and the current flowing through the voice coil 931 is applied to the voice coil 931, and because the frame of the drive unit 940 is fixed to the inner wall of the housing portion 910, the voice coil 931 vibrates in a direction along the inner wall of the housing portion 910 to which the drive unit 940 is fixed. As a result, the volume of the internal space S1 of the housing portion 910 changes due to the vibration of the diaphragm 932. The arrow D shown in Figure 9 indicates the direction in which the voice coil 931 and the diaphragm 932 vibrate.

[0093] The drive signal (control information) for controlling the vibration of the vibrating unit 930 includes a low-frequency signal, which is a fundamental wave signal for controlling the transport of air quality components, and a high-frequency signal, which is an acoustic signal for controlling sound. The drive signal is generated, for example, by adding an acoustic signal with a frequency greater than 300 Hz to a single-period fundamental wave signal of 300 Hz or less. As a result, the diaphragm 932 vibrates based on the low-frequency fundamental wave signal, and airflow W is released from the outlet S2. At this time, sound based on the acoustic signal is output from the speaker unit, which is composed of the vibrating unit 930 and the drive unit 940. This allows acoustic stimulation to be provided to the user along with non-contact tactile sensation. Furthermore, with the airflow discharge device 900, control of non-contact tactile sensation and acoustic stimulation can be achieved by inputting the drive signal to a general-purpose device such as a speaker.

[0094] The drive signal is generated by combining the fundamental wave signal and the acoustic signal by adding them together with weighted values ​​for the signal strength at each time point. In this process, the drive signal is a composite acoustic signal obtained by adding the value of the acoustic signal multiplied by (1-α) to the value of the fundamental wave signal multiplied by α. α can be set to any value greater than 0 and less than 1. The larger the value of α, the greater the emitted airflow W and the weaker the sound.

[0095] The temperature control unit 950 is provided near the discharge unit 920 to adjust the temperature of the discharged airflow W. The temperature control unit 950 is, for example, a Peltier element or a heater. The temperature control unit 950 is preferably configured to change the temperature of the discharge port S2 via the discharge unit 920, and is preferably fixed in contact with the discharge unit 920. The temperature control unit 950 may be installed at a predetermined distance from the discharge unit 920, or it may be fixed to the discharge unit 920 via other members such as a heat sink.

[0096] The additive unit 960 is configured to impart various fragrances to the airflow W, thereby providing the user with olfactory stimulation through non-contact contact via the airflow W. The additive unit 960 is made of a porous material and is positioned at the outlet S2 of the discharge unit 920. By configuring the components of the additive unit 960 to generate various fragrances, the airflow W becomes infused with these fragrances as it passes from the internal space S1 through the outlet S2 and the additive unit 960. As a result, the airflow discharge device 900 can deliver fragrant air quality to the first user.

[0097] The additive unit 960 may be located near the discharge port S2 and configured to supply various aromatic substances or aromatic gases to the gas discharged from the internal space S1. For example, the additive unit 960 may be equipped with a storage chamber for storing aromatic substances and a nozzle for releasing the aromatic substances, and may be configured to supply the aromatic substances to the airflow W from the nozzle. The aromatic substances may be configured to be released continuously or to be released at predetermined timings. The aromatic substances can be, for example, aromatic components obtained by volatilizing fragrances, or aromatic components obtained from essential oils extracted from plants.

[0098] The airflow discharge device 900 is not limited to being configured such that, as shown in Figure 9, a speaker unit is provided at one end of the housing 910 and a discharge port S2 is provided at the other end, thereby discharging airflow W from the other end. For example, multiple speaker units may be provided at one end and the other end of the housing 910, and the discharge port S2 may be provided between the one end and the other end of the housing 910 (for example, a part of the side). This makes it possible to control the airflow W by controlling multiple speaker units, and to provide a more precisely controlled airflow W to the user.

[0099] 5. Variations Although embodiments of the present invention have been described above, the present invention is not limited thereto and can be modified as appropriate without departing from the technical spirit of the invention.

[0100] In this embodiment, the example given is that the interaction target is a second user, but it is not limited to this. The interaction target may also be a virtual agent operated by computer control. The virtual agent may be controlled to operate based on predetermined rules, or it may be controlled to operate in response to user actions (such as movements of the limbs, head, or gaze) or various inputs (such as text input via the input unit 25 or voice input). In this case, the action of the interaction target is specified in the computer that controls the operation of the virtual agent. For example, the action of the interaction target may be specified in the user terminal 20, or the action specified in the server device 10 (without obtaining information about the action from other user terminals) may be transmitted to the user terminal 20.

[0101] For example, if the virtual agent is an animal virtually kept by the first user, the various actions the animal performs (moving, approaching, running away, making noise, being startled by noise, etc.) are controlled based on event occurrences associated with the first user. For instance, the probability of the animal performing an action may be adjusted according to the user's mental workload. Alternatively, if the virtual agent is a character controlled by a gaming engine that simulates a pre-created and stored 3D model, its actions may be controlled so that the game difficulty changes based on event occurrences associated with the first user. For example, if the user's mental workload is high, the character's actions towards the first user (attacks, support actions, etc.) may be adjusted to be advantageous to the first user.

[0102] In this embodiment, the feedback provided to the user is exemplified as tactile stimuli, smells, or sounds, but is not limited to these. The feedback may also be visual stimuli. For example, it may be configured to cause a predetermined change in the image presented on the image output device 271. When the object of interaction is a virtual agent, the system may be configured to set the probability that the object of interaction will perform a special action based on the feedback level. For example, when a virtual agent, which is an animal, performs an action such as running towards the user, if the user's feedback level is high, the probability of performing that action can be increased, thereby increasing the user's satisfaction with the interaction with the virtual agent.

[0103] In this embodiment, the type of feedback provided to the first user is determined based on event events associated with the first user, but this is not limited to this. The user terminal associated with the second user may be configured to be controlled based on event events associated with the first user. For example, information indicating the first user's mental workload may be displayed on the user terminal associated with the second user. This allows the second user to understand the first user's mental workload and decide what action to take towards the first user. The information indicating the first user's mental workload may also be output in a manner that provides stimuli to the five senses, such as hearing and touch, in addition to visual information.

[0104] 6. Others In the above embodiment, the server device 10 or user terminal 20 performed various storage and control functions, but instead of the server device 10 or user terminal 20, multiple external devices may be used. That is, various information and programs may be stored in a distributed manner across multiple external devices using blockchain technology or the like. Furthermore, at least one of the devices included in the information processing system 1 may be located outside the country in which the functions of the information processing system 1 are performed.

[0105] The control unit 11 or control unit 21 may further include an artificial intelligence unit (not shown), which may have a trained model constructed by machine learning using data stored in the memory unit 12 or memory unit 22 and information stored in each table as training data. The artificial intelligence unit may also have a general-purpose natural language processing training model, such as a Large Language Model (LLM) that has learned from a vast amount of data. For example, it may take data about events associated with a user as input to construct a trained model so that the feedback actually given to the user is identified, or it may generate prompts to be input to the large language model.

[0106] Furthermore, they may be provided in the following embodiments.

[0107] (1) An information processing system comprising at least one processor, the processor configured to perform the following steps by reading a program, the acquisition step of acquiring an action of an interactive object that interacts with a user, the identification step of identifying the content of feedback to be given to the user based on the action, wherein the feedback is given to the user by non-contact touch, the non-contact touch is presented by at least the transport of air quality components to the user, and the output step of outputting control information based on the content of the feedback and an event associated with the user, wherein the control information is used to control the transport of the air quality components.

[0108] This configuration can help facilitate communication between the user and the object of interaction. In particular, by providing feedback in response to the movement of air quality components and user events, user discomfort can be reduced.

[0109] (2) An information processing system as described in (1) above, wherein in the determination step, a level indicating the degree of influence the feedback has on the user is determined based on the event, and in the output step, the control information is output based on the content of the feedback and the level.

[0110] This configuration can effectively reduce user discomfort.

[0111] (3) An information processing system as described in (2) above, wherein in the determination step, the level is determined such that the level becomes lower the higher the degree of concentration of the user on a predetermined task related to the event.

[0112] This configuration can effectively reduce user discomfort.

[0113] (4) An information processing system according to any one of (1) to (3) above, wherein in the acquisition step, physical information relating to the physical state of the user is acquired, and in the determination step, the event is determined based on the physical information.

[0114] This configuration allows for effective determination of whether or not the user is seeking feedback.

[0115] (5) An information processing system according to any one of (1) to (4) above, wherein in the acquisition step, communication information input by the user to be communicated to the object of the dialogue is acquired, and in the determination step, the event is determined based on the communication information.

[0116] This configuration allows for effective determination of whether or not the user is seeking feedback.

[0117] (6) An information processing system according to any one of (1) to (5) above, wherein in the acquisition step, schedule information relating to the user's planned actions is acquired, and in the determination step, the event is determined based on the schedule information.

[0118] This configuration allows for effective determination of whether or not the user is seeking feedback.

[0119] (7) An information processing system according to any one of (1) to (6) above, wherein the feedback is further provided to the user by sound, and the sound is controlled based on the control information.

[0120] In this configuration, effective feedback can be easily provided to the user through a common presentation device.

[0121] (8) An information processing system further comprising a presentation device for presenting the feedback, in the information processing system described in (7) above.

[0122] (9) An information processing system as described in (8) above, wherein the display device includes one or more voice coil type speakers, and the transport of the air quality components and the sound are controlled based on the control information, and the control information includes a low-frequency signal for controlling the transport of the air quality components and a high-frequency signal for controlling the sound.

[0123] In this configuration, effective feedback can be easily provided to the user through a common presentation device.

[0124] (10) An information processing system according to any one of (1) to (9) above, wherein the feedback is further provided to the user by touch sensation in a manner different from the transport of the air quality components, and the touch sensation and the transport of the air quality components are controlled based on the control information.

[0125] (11) An information processing system as described in (10) above, further comprising a contact device and a presentation device, wherein the contact device is configured to present the tactile sensation and is incorporated into a contact member that comes into contact with the user who is seated and interacting with the object of dialogue, and the presentation device is configured to transport the air quality components and is incorporated into a peripheral member surrounding the user or the contact member, and the transport of the tactile sensation in the contact device and the air quality components in the presentation device is controlled based on the control information.

[0126] In this configuration, effective feedback, including tactile feedback, can be easily provided to the user.

[0127] (12) An information processing method comprising each step performed in an information processing system described in any one of (1) to (11) above.

[0128] (13) A program that causes a computer to perform each step of the information processing system described in any one of (1) to (11) above. Of course, this is not always the case.

[0129] Finally, various embodiments of the present invention have been described, but these are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0130] 1: Information Processing System 10: Server device 20: User terminal 26: Acquisition device 27: Output device 271: Image output device 272: Audio output device 273: Non-contact tactile presentation device 274: Contact-type tactile presentation device 275: Olfactory presentation device 800: Feedback output device 822:Airflow release device 900:Airflow release device T: Schedule Management Table

Claims

1. An information processing system, Equipped with at least one processor, The aforementioned processor is configured to perform the following steps by reading a program: In the acquisition step, the interaction target action that the user interacts with is acquired. In a specific step, the content of the feedback to be given to the user based on the action is identified, and here, The aforementioned feedback is provided to the user via non-contact tactile means. The aforementioned non-contact tactile sensation is presented by at least the delivery of air quality components to the user. In the output step, the information processing system outputs control information based on the content of the feedback and the event associated with the user, where the control information is used to control the transport of the air quality components.

2. In the information processing system described in claim 1, In the determination step, based on the event, a level indicating the degree to which the feedback has an impact on the user is determined. An information processing system that, in the output step, outputs the control information based on the content of the feedback and the level.

3. In the information processing system described in claim 2, An information processing system that, in the determination step, determines the level such that the level decreases as the user's level of concentration on a predetermined task related to the event increases.

4. In the information processing system described in claim 1, In the acquisition step, physical information relating to the user's physical condition is acquired. In the determination step, an information processing system determines the event based on the physical information.

5. In the information processing system described in claim 1, In the acquisition step, the communication information entered by the user to be communicated to the object of the dialogue is acquired. In the determination step, the information processing system determines the event based on the transmitted information.

6. In the information processing system described in claim 1, In the acquisition step, schedule information regarding the user's planned activities is acquired. In the determination step, the information processing system determines the event based on the scheduled information.

7. In the information processing system described in claim 1, The aforementioned feedback is further provided to the user via sound, An information processing system in which the sound is controlled based on the control information.

8. In the information processing system described in claim 7, An information processing system further comprising a presentation device for presenting the aforementioned feedback.

9. In the information processing system described in claim 8, The presenting device includes one or more voice coil type speakers, and the transport of the air quality components and the sound are controlled based on the control information. The control information includes a low-frequency signal for controlling the transport of the air quality components and a high-frequency signal for controlling the acoustics, in an information processing system.

10. In the information processing system described in claim 1, The aforementioned feedback is further provided to the user by tactile sensation in a manner different from the transport of the air quality components. An information processing system in which the transport of touch sensation and air quality components is controlled based on the control information.

11. In the information processing system according to claim 10, The device further comprises a contact-type device and a presentation device. The aforementioned contact-type device is configured to present the tactile sensation and is incorporated into a contact member that comes into contact with the user who is seated and interacting with the object of interaction. The presenting device is configured to transport the air quality components and is incorporated into the peripheral member surrounding the user or the contact member. An information processing system in which the tactile sensation in the contact-type device and the transport of air quality components in the presentation device are controlled based on the control information.

12. Information processing method, A method comprising each step performed in an information processing system according to any one of claims 1 to 11.

13. It is a program, A program that causes a computer to perform each step of the information processing system described in any one of claims 1 to 11.

Images