Information display device, information display method, and program

Abstract

An information presentation device according to one embodiment of the present invention comprises a determination unit, a selection unit, and a generation unit. The determination unit determines whether an intention of dialogue information relating to a dialogue between users is positive or negative. The selection unit selects, on the basis of the determined intention, a modality to be superimposed on the dialogue information. The generation unit generates, by using the selected modality, subjective sense formation information for forming subjective sense corresponding to the determined intention.

Description

Technical Field

[0001] Embodiments relate to an information presentation apparatus, an information presentation method, and a program. Background Art

[0002] Against the backdrop of the COVID-19 pandemic, communication modes involving online conversation have come into active use. It is known that compared to face-to-face conversation, online conversation often restricts the transmission of non-verbal elements, and thus tends to make it difficult for a speaker to convey the content they intend to get across to the interlocutor.

[0003] Numerous studies have been conducted using XR (Cross Reality or Extended Reality) technology to enable a speaker to convey intended content to the interlocutor just as effectively in online conversation as in face-to-face conversation. XR technology is a general term for technologies that create new experiences by fusing the real world with the virtual world. Known studies utilizing XR technology include, for example, research that brings information transmitted to users closer to that of a face-to-face environment by using VR (Virtual Reality) spaces and high-definition videos. These studies take the information transmitted to users in a face-to-face environment as the correct information, and aim to faithfully reproduce this correct information based on audio-visual information. Prior Art Literature Non-Patent Literature

[0004] Non-Patent Literature 1 Pakanen, M., et.al., "Nice to see you virtually”: Thoughtful design and evaluation of virtual avatar of the other user in AR and VR based telexistence systems", Entertainment Computing,Vol. 40,2022 Non-Patent Literature 2 Sayaka Unno, Katsunobu Ito, "A 2-channel spatial audio system using sound field simulation and a selectable head-related transfer function system," Proceedings of the 82nd National Conference, pp. 561-562, 2020. [Overview of the project] [Problems that the invention aims to solve]

[0005] However, faithfully reproducing information transmitted face-to-face online inevitably requires higher resolution audiovisual information and more powerful devices. Furthermore, attempting to faithfully reproduce face-to-face information online often results in limitations on use cases. Therefore, there are concerns that methods for faithfully reproducing face-to-face information online have high barriers to implementation in actual online communication systems.

[0006] This invention was made in view of the above circumstances, and its purpose is to provide a means of presenting information online that evokes the same sense of presence as a face-to-face environment, without faithfully reproducing the face-to-face environment. [Means for solving the problem]

[0007] An information display device according to one embodiment comprises a determination unit and a selection unit, output The system comprises a determination unit and a selection unit. The determination unit determines whether the intent of the dialogue information regarding the interaction between users is positive or negative. The selection unit determines whether the intent determined above is positive or negative. Select a subjective sensation, and based on the subjective sensation selected above... Select a modality. (See above) output The unit uses the above selected modality to perform the above Selected Information that shapes subjective sensations Output by superimposing it on the above dialogue information. do. [Effects of the Invention]

[0008] According to the embodiment, it is possible to provide a means of presenting information online that evokes the same sense of presence as a face-to-face environment, without faithfully reproducing the face-to-face environment. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a block diagram showing an example of an information presentation system according to the first embodiment. [Figure 2] Figure 2 is a block diagram showing an example of the hardware configuration of a terminal according to the first embodiment. [Figure 3] Figure 3 is a block diagram showing an example of the hardware configuration of an information display device according to the first embodiment. [Figure 4] Figure 4 is a block diagram showing an example of the functional configuration of an information display device according to the first embodiment. [Figure 5] Figure 5 shows an example of a subjective sensory model according to the first embodiment. [Figure 6] Figure 6 is a flowchart showing an example of information presentation processing in the information presentation device according to the first embodiment. [Figure 7] Figure 7 is a flowchart showing an example of modality selection processing in an information presentation device according to the first embodiment. [Figure 8] Figure 8 is a block diagram showing an example of the functional configuration of an information display device according to the second embodiment. [Figure 9] Figure 9 is a flowchart showing an example of information presentation processing in the information presentation device according to the second embodiment. [Figure 10] Figure 10 is a flowchart showing an example of modality selection processing in an information presentation device according to the second embodiment. [Modes for carrying out the invention]

[0010] Embodiments will be described below with reference to the drawings. In the following description, components having the same function and configuration will be denoted by the same reference numerals.

[0011] 1. First Embodiment 1.1 Information Presentation System Figure 1 is a block diagram showing an example of an information presentation system according to the first embodiment.

[0012] The information presentation system 1 is, for example, a communication system that enables online interaction between a plurality of users. The information presentation system 1 includes terminals 10A and 10B, and an information presentation device 20. The terminals 10A and 10B, and the information presentation device 20 are configured to communicate with each other via a network NW.

[0013] Each of the terminals 10A and 10B is a personal computer, a smartphone, or the like. Terminals 10A and 10B are used by users UA and UB, respectively. Users UA and UB use the terminals 10A and 10B to conduct online interaction via the network NW.

[0014] The information presentation device 20 is, for example, a server that manages online interactions. The information presentation device 20 acquires information (interaction information) transmitted through online interaction by users UA and UB from the terminals 10A and 10B. Based on the interaction information, the information presentation device 20 presents information (subjective sensation forming information) that causes users UA and UB to form the same subjective sensation as that of a face-to-face interaction to users UA and UB via the terminals 10A and 10B. The subjective sensation forming information is, for example, superimposed on the interaction information in a predetermined modality and presented to users UA and UB.

[0015] 1.2 Terminal Next, the configuration of the terminal according to the first embodiment will be described.

[0016] Figure 2 is a block diagram showing an example of the hardware configuration of the terminal according to the first embodiment. As shown in Figure 2, each of the terminals 10A and 10B includes a control circuit 11, a communication module 12, and a user interface 13. Each of the terminals 10A and 10B has the same configuration as each other. Hereinafter, when terminals 10A and 10B are not particularly distinguished, they are referred to as the terminal 10. Similarly, when users UA and UB are not particularly distinguished, they are referred to as the user U.

[0017] The control circuit 11 is a circuit that controls all components of the terminal 10 as a whole. The control circuit 11 includes a CPU (Central Processing Unit), RAM (Random Access Memory), and ROM (Read Only Memory), etc. The ROM of the control circuit 11 stores programs used for various processes in the terminal 10. The CPU of the control circuit 11 controls the entire terminal 10 according to the programs stored in the ROM of the control circuit 11. The RAM of the control circuit 11 is used as a workspace for the CPU of the control circuit 11.

[0018] The communication module 12 is a circuit used for data communication with other terminals 10, and for sending and receiving data between terminal 10 and the information display device 20.

[0019] The user interface 13 is an interface that manages communication between user U and control circuit 11. The user interface 13 includes input devices and output devices. Input devices include, for example, a voice microphone, a touch panel, and operation buttons. Output devices include, for example, a speaker and a display. The user interface 13 converts dialogue information from user U to other user U into electrical signals and transmits them to the control circuit 11. The user interface 13 outputs dialogue information from other user U and subjective sensory formation information from the information presentation device 20 to user U.

[0020] 1.3 Information presentation device Next, the configuration of the information display device according to the first embodiment will be described.

[0021] 1.3.1 Hardware Configuration Figure 3 is a block diagram showing an example of the hardware configuration of an information display device according to the first embodiment. As shown in Figure 3, the information display device 20 includes a control circuit 21, a communication module 22, storage 23, a drive 24, and a storage medium 25.

[0022] The control circuit 21 is a circuit that controls all components of the information display device 20 as a whole. The control circuit 21 includes a CPU, RAM, and ROM. The ROM of the control circuit 21 stores programs used for various processes in the information display device 20. The CPU of the control circuit 21 controls the entire information display device 20 according to the programs stored in the ROM of the control circuit 21. The RAM of the control circuit 21 is used as a workspace for the CPU of the control circuit 21.

[0023] The communication module 22 is a circuit used for sending and receiving data between the information display device 20 and the terminal 10.

[0024] The storage 23 includes, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive). The storage 23 stores information used for various processes in the information presentation device 20.

[0025] Drive 24 is a device for reading software stored on the storage medium 25. Drive 24 includes, for example, a CD (Compact Disk) drive or a DVD (Digital Versatile Disk) drive.

[0026] The storage medium 25 is a medium for storing software by electrical, magnetic, optical, mechanical, or chemical means. The storage medium 25 may also store programs for executing various processes in the information presentation device 20.

[0027] 1.3.2 Functional Configuration Figure 4 is a block diagram showing an example of the functional configuration of an information presentation device according to the first embodiment. The CPU of the control circuit 21 loads the program stored in the ROM or storage medium 25 of the control circuit 21 into the RAM of the control circuit 21. The CPU of the control circuit 21 then interprets and executes the program loaded into the RAM of the control circuit 21. As a result, the information presentation device 20 functions as a computer comprising an environmental information acquisition unit 31, a storage unit 32, an interaction information acquisition unit 33, an intent determination unit 34, a modality selection unit 35, a generation unit 36, and an output unit 37.

[0028] The environmental information acquisition unit 31 acquires environmental information 32a from the receiving terminal 10. The environmental information 32a includes the types of information that the receiving user interface 13 outputs to user U during interaction between users U. Specifically, for example, the environmental information 32a includes information indicating whether or not the receiving user interface 13 provides visual information to user U, and information indicating whether or not it provides auditory information. The environmental information acquisition unit 31 stores the acquired environmental information 32a in the storage unit 32.

[0029] The memory unit 32 is the memory area of ​​the storage unit 23. The memory unit 32 stores environmental information 32a and subjective sensory models 32b.

[0030] Figure 5 shows an example of a subjective sensory model according to the first embodiment. As shown in Figure 5, the subjective sensory model 32b is represented, for example, as a three-layer model of secondary sensation, primary sensation, and presentation modality.

[0031] Presentation modality is a type of subjective sensory formation information superimposed on dialogue information. Presentation modality is classified into, for example, visual modality and auditory modality. Examples of presentation modality classified as visual modality include body parts, facial expressions, gaze, gestures, state text, and state icons. Examples of presentation modality classified as auditory modality include sound effects, ambient sounds, and background music.

[0032] Primary sensations are describable, such as the presence or absence of presence or the sense of distance, and are easily expressed outwardly by a person. Primary sensations can be classified into, for example, a sense of coexistence, a sense of psychological involvement, and a sense of behavioral involvement. Examples of primary sensations classified as coexistence include a sense of self-inclusion and a sense of recognition. Examples of primary sensations classified as psychological involvement include a sense of attention, empathy, and understanding. Examples of primary sensations classified as behavioral involvement include a sense of helping, a sense of functioning, and a sense of subservience.

[0033] Secondary sensations are indescribable (not-describable) and internally formed feelings, such as the feeling of sharing a room with one's conversation partner or the satisfaction of the conversation itself. Secondary sensations can be classified into positive and negative secondary sensations. Examples of positive secondary sensations include feelings of inclusion, being loved, and affection. The degree of positivity is highest for affection, followed by being loved and then inclusion. Examples of negative secondary sensations include feelings of distrust, alienation, and loneliness. The degree of negativity is highest for loneliness, followed by alienation and then distrust.

[0034] The subjective sensory model 32b shows the relationship between secondary sensations and primary sensations involved in the formation of those secondary sensations, and the relationship between primary sensations and presentation modalities involved in the formation of those primary sensations.

[0035] In the example in Figure 5, it is shown that body parts, state text, and environmental sounds are involved as presentation modalities in the formation of a primary sense of self-inclusion. It is shown that body parts, facial expressions, gaze, state text, and sound effects are involved as presentation modalities in the formation of a primary sense of awareness. It is shown that facial expressions, gaze, state icons, and sound effects are involved as presentation modalities in the formation of a primary sense of attention. It is shown that facial expressions, gestures, state icons, sound effects, and environmental sounds are involved as presentation modalities in the formation of a primary sense of empathy. It is shown that gestures, state icons, and background music are involved as presentation modalities in the formation of a primary sense of understanding.

[0036] Furthermore, the example in Figure 5 shows that the formation of the secondary sensation of inclusion involves, as primary sensations, self-inclusion, awareness, attention, and understanding. The formation of the secondary sensation of being loved involves, as primary sensation, self-inclusion. The formation of the secondary sensation of affection involves, as primary sensations, awareness, attention, empathy, and understanding. The formation of the secondary sensation of distrust involves, as primary sensations, awareness and attention in a positive way, while self-inclusion plays a negative role. The formation of the secondary sensations of alienation and loneliness involves, as primary sensation, self-inclusion in a negative way.

[0037] The subjective sensory model 32b described above is pre-designed based on findings obtained from meta-reviews and experiments in CMC (Computer Mediated Communication) research.

[0038] Again, referring to Figure 4, the functional configuration of the information display device 20 will be explained.

[0039] The dialogue information acquisition unit 33 acquires dialogue information from the transmitting terminal 10. The dialogue information includes video and audio. The audio in the dialogue information includes speech rate and sound pressure. Specifically, the dialogue information acquisition unit 33 performs speech recognition of the utterance and calculates the speech rate from the number of utterances per unit time. The dialogue information acquisition unit 33 also records the utterance and measures the sound pressure based on the recorded information. The dialogue information acquisition unit 33 transmits the acquired dialogue information to the intention determination unit 34.

[0040] The intent determination unit 34 determines, based on the dialogue information, the intent behind the dialogue that the transmitting user U intended to convey to the receiving user U. The intent behind the dialogue indicates whether the secondary feeling that the transmitting user U wants the receiving user U to form is positive or negative. In other words, the intent determination unit 34 determines, based on the dialogue information, whether the intent behind the dialogue information is positive or negative.

[0041] Specifically, for example, if the speech rate of the dialogue information is above a first threshold and / or the sound pressure is below a second threshold, the intention determination unit 34 determines that the secondary sensation that the transmitting user U wants the receiving user U to form is negative. If the speech rate of the dialogue information is below the first threshold and / or the sound pressure exceeds the second threshold, the intention determination unit 34 determines that the secondary sensation that the transmitting user U wants the receiving user U to form is positive. The intention determination unit 34 may also include the degree of deviation of the speech rate from the first threshold and the degree of deviation of the sound pressure from the second threshold in the determination result. The intention determination unit 34 transmits the determination result to the modality selection unit 35.

[0042] The modality selection unit 35 selects a modality for subjective sensation formation information to be superimposed on the dialogue information, based on the determination result by the intention determination unit 34 and the subjective sensation model 32b.

[0043] Specifically, the modality selection unit 35 selects a secondary sensation to be formed by the receiving user U from the subjective sensation model 32b, based on the intention determined by the intention determination unit 34. The modality selection unit 35 may select a secondary sensation according to the degree of discrepancy. If the determination result is positive, the modality selection unit 35 may change the selected secondary sensation in the order of inclusion, being loved, and affection as the degree of discrepancy increases. If the determination result is negative, the modality selection unit 35 may change the selected secondary sensation in the order of distrust, estrangement, and loneliness as the degree of discrepancy increases.

[0044] Furthermore, the modality selection unit 35 selects a primary sensation associated with the selected secondary sensation from the subjective sensation model 32b based on the subjective sensation model 32b. Then, the modality selection unit 35 selects a presentation modality associated with the selected primary sensation. When selecting a presentation modality, the modality selection unit 35 may select a presentation modality from among the modalities that the receiving terminal 10 can present to the user U, based on the environmental information 32a. The modality selection unit 35 transmits information indicating the selected presentation modality to the generation unit 36.

[0045] The generation unit 36 ​​generates subjective sensory formation information based on the presentation modality. If the presentation modality is visual, the generation unit 36 ​​superimposes the subjective sensory formation information onto the video of the dialogue information, for example. If the presentation modality is auditory, the generation unit 36 ​​superimposes the subjective sensory formation information onto the audio of the dialogue information, for example. The generation unit 36 ​​transmits the generated subjective sensory formation information to the output unit 37.

[0046] The output unit 37 outputs the generated subjective sensory formation information to the receiving terminal 10 via the network NW.

[0047] With the above configuration, the receiving user U can interact with the transmitting user U based on dialogue information that has subjective sensory formation information superimposed on it.

[0048] 1.2 Operation Next, the operation of the information display device according to the first embodiment will be described.

[0049] 1.2.1 Information Presentation Processing Figure 6 is a flowchart showing an example of information presentation processing in the information processing apparatus according to the first embodiment. Prior to the information presentation processing, the environmental information acquisition unit 31 is assumed to have already acquired environmental information 32a.

[0050] When an online conversation begins between users U (start), the conversation information acquisition unit 33 acquires the conversation information that is transmitted from the transmitting terminal 10 to the receiving terminal 10 via the network NW (S11).

[0051] The intent determination unit 34 determines the intent of the dialogue based on the dialogue information obtained in the processing of S11 (S12). Specifically, for example, if the speaking speed of the dialogue information is above the first threshold and / or the sound pressure is below the second threshold, the intent determination unit 34 determines that the secondary feeling that the transmitting user U wants the receiving user U to form is negative. If the speaking speed of the dialogue information is below the first threshold and / or the sound pressure exceeds the second threshold, the intent determination unit 34 determines that the secondary feeling that the transmitting user U wants the receiving user U to form is positive.

[0052] The modality selection unit 35 executes a modality selection process (S13) based on the intent of the dialogue determined in the S12 process. Details of the modality selection process will be described later.

[0053] The generation unit 36 ​​generates subjective sensation formation information that forms a subjective sensation corresponding to the intention determined in the S12 process, using the presentation modality selected in the S13 process (S14).

[0054] The output unit 37 superimposes the subjective sensory formation information generated in the processing of S14 onto the dialogue information acquired in the processing of S11 and outputs it to the receiving terminal 10 (S15).

[0055] The dialogue information acquisition unit 33 determines whether or not an online dialogue between users U is ongoing (S16).

[0056] If the dialogue is still ongoing (S16; yes), the dialogue information acquisition unit 33 continues to acquire dialogue information (S11). Then, the subsequent processes S12 to S16 are executed. In this way, processes S11 to S16 are repeated until the dialogue ends.

[0057] If the dialogue ends (S16; no), the information presentation process ends (end).

[0058] 1.2.2 Modality Selection Process Next, the modality selection process in the information presentation device according to the first embodiment will be described.

[0059] Figure 7 is a flowchart showing an example of modality selection processing in an information presentation device according to the first embodiment. The processes S21 to S23 shown in Figure 7 are details of the process S13 in Figure 6.

[0060] When the modality selection process begins (start), the modality selection unit 35 selects a secondary sensation associated with the intention determined in the S12 process (S21). Specifically, if the intention is determined to be positive, the modality selection unit 35 selects inclusion, being loved, or affection as the secondary sensation. If the intention is determined to be negative, the modality selection unit 35 selects distrust, alienation, or loneliness as the secondary sensation. The modality selection unit 35 may determine the secondary sensation to select, for example, according to the degree of deviation between the parameters used to determine the intention, such as speech rate and sound pressure, and a threshold.

[0061] The modality selection unit 35 selects a primary sensation associated with the secondary sensation selected in the processing of S21 (S22). If there are multiple primary sensations associated with a secondary sensation, the modality selection unit 35 may select one primary sensation from among the multiple primary sensations according to predetermined rules (for example, preferentially selecting the primary sensation on the left among the multiple primary sensations shown in Figure 5).

[0062] The modality selection unit 35 selects a presentation modality associated with the primary sensation selected in the processing of S22 (S23). If there are multiple presentation modalities associated with the primary sensation, the modality selection unit 35 may select multiple presentation modalities simultaneously. When selecting a presentation modality, the modality selection unit 35 may select a presentation modality that can be output by the receiving terminal 10 based on the environmental information 32a.

[0063] After processing S23, the modality selection process ends (end).

[0064] 1.3 Effects of the First Embodiment According to the first embodiment, the intent determination unit 34 determines whether the intent of the dialogue information regarding the conversation between users U is positive or negative. The modality selection unit 35 selects a presentation modality to superimpose on the dialogue information based on the determined intent. The generation unit 36 ​​generates subjective sensation formation information that forms a subjective sensation corresponding to the determined intent using the selected presentation modality. This makes it possible to present subjective sensation formation information in addition to the dialogue information to the receiving user U. For this reason, even online, where it is more difficult to perceive the true intent of a conversation than in face-to-face interactions, it is possible to form a subjective sensation in line with the intent of the conversation in the receiving user U. Thus, it is possible to provide a means of presenting information that evokes the same kind of sensation as the face-to-face environment online without faithfully reproducing the face-to-face environment.

[0065] Furthermore, the modality selection unit 35 selects a secondary sensation associated with the determined intention. The modality selection unit 35 selects a primary sensation associated with the selected secondary sensation. The modality selection unit 35 selects a presentation modality associated with the selected primary sensation. This allows for the selection of a modality that matches the secondary sensation to be formed, based on the subjective sensation model 32b.

[0066] Furthermore, the intent determination unit 34 determines that the intent is negative if the speech rate in the dialogue information is above a first threshold, and positive if it is below the first threshold. The intent determination unit 34 also determines that the intent is positive if the sound pressure in the dialogue information is above a second threshold, and negative if it is below the second threshold. This makes it possible to determine the intent of the dialogue based on the speech state of the transmitting user U.

[0067] Furthermore, presentation modalities include visual and auditory modalities. Visual modalities include at least one selected from body parts, facial expressions, gaze, gestures, text, and icons. Auditory modalities include at least one selected from sound effects, ambient sounds, and background music. This allows for the selection of an appropriate presentation modality depending on the secondary sense to be evoked.

[0068] 2. Second Embodiment Next, an information presentation device according to the second embodiment will be described. In the first embodiment, the case in which intent is determined based on dialogue information was described. The second embodiment differs from the first embodiment in that the intent of the dialogue is explicitly indicated by the transmitting user U. Below, the configuration and operation that differ from the first embodiment will be mainly described. Configurations and operations equivalent to the first embodiment will be omitted from description as appropriate.

[0069] 2.1 Information Processing and Presentation Device Figure 8 is a block diagram showing an example of the functional configuration of an information display device according to the second embodiment.

[0070] As shown in Figure 8, the intent determination unit 34 may acquire intent information from the transmitting terminal 10. Intent information includes whether the intent of the dialogue is positive or negative. Intent information may also include information indicating specific secondary feelings. During the dialogue, the intent information is transmitted to the information presentation device 20 via the network NW by inputting it into the transmitting terminal 10 by the transmitting user U.

[0071] 2.2 Information Presentation Processing Figure 9 is a flowchart showing an example of information presentation processing in the information presentation device according to the second embodiment.

[0072] When an online conversation begins between users U (start), the conversation information acquisition unit 33 acquires conversation information and intent information from the transmitting terminal 10 via the network NW (S31). The intent information does not need to be sent to the receiving terminal 10.

[0073] The intent determination unit 34 determines the intent of the dialogue based on the intent information obtained in the processing of S31 (S32). Specifically, if the intent information indicates a positive intent, the intent determination unit 34 determines that the intent of the dialogue is positive. If the intent information indicates a negative intent, the intent determination unit 34 determines that the intent of the dialogue is negative.

[0074] The modality selection unit 35 executes a modality selection process (S33) based on the intent of the dialogue determined in the process of S32. Details of the modality selection process will be described later.

[0075] The generation unit 36 ​​generates subjective sensation formation information that forms a subjective sensation corresponding to the intention determined in the processing of S32, using the presentation modality selected in the processing of S33 (S34).

[0076] The output unit 37 superimposes the subjective sensory formation information generated in the processing of S34 onto the dialogue information acquired in the processing of S31 and outputs it to the receiving terminal 10 (S35).

[0077] The dialogue information acquisition unit 33 determines whether or not an online dialogue between users U is ongoing (S36).

[0078] If the dialogue is still ongoing (S36; yes), the dialogue information acquisition unit 33 continues to acquire dialogue information and intent information (S31). Then, the subsequent processes S32 to S36 are executed. In this way, the processes S31 to S36 are repeated until the dialogue ends.

[0079] If the dialogue ends (S36; no), the information presentation process ends (end).

[0080] 2.3 Modality Presentation Process Figure 10 is a flowchart showing an example of modality selection processing in an information presentation device according to the second embodiment.

[0081] When the modality selection process begins (start), the modality selection unit 35 selects a secondary sensation associated with a positive intention based on the intention information obtained in the process of S31 (S41). If the intention information does not include information that specifies a particular secondary sensation, the process of S41 is substantially equivalent to the process of S21 in the first embodiment. If the intention information includes information that specifies a particular secondary sensation, the modality selection unit 35 uniquely selects a secondary sensation based on the information that specifies a secondary sensation included in the intention information.

[0082] The modality selection unit 35 selects a primary sensation associated with the secondary sensation selected in the processing of S41 (S42).

[0083] The modality selection unit 35 selects a presentation modality associated with the primary sensation selected in the processing of S42 (S43).

[0084] After processing S43, the modality selection process ends (end).

[0085] 2.4 Effects according to the second embodiment According to the second embodiment, the intention determination unit 34 can uniquely determine the intention of a conversation without relying on the conversation information. Furthermore, if the intention information includes information indicating a secondary sensation, the modality selection unit 35 can uniquely select the secondary sensation from the subjective sensation model 32b. As a result, even if the content of the determination based on the conversation information differs from the intention that the transmitting user U truly wants to convey, the generation unit 36 ​​can generate subjective sensation formation information based on the intention that the transmitting user U truly wants to convey. For example, even if conversation information that might appear to have a negative intention online is spoken, subjective sensation formation information that aligns with the true intention of the transmitting user U, who wants to increase the feeling of closeness, can be presented to the receiving user U. Therefore, information that evokes the same kind of sensation as in a face-to-face environment can be presented online without faithfully reproducing the face-to-face environment.

[0086] 3. Variations, etc. Furthermore, various modifications can be applied to the first and second embodiments described above.

[0087] In the first and second embodiments described above, the case in which the type of information output by the receiving terminal 10 to the user U is included in the environmental information 32a was explained, but it is not limited to this. For example, the environmental information 32a may include information about the dominant modality for the receiving user U. According to this modified example, for example, if the dominant modality for the receiving user U is the visual modality, the modality selection unit 35 can preferentially select the visual modality. Also, for example, if the receiving user U has a mild visual impairment and communication is easier using hearing, the modality selection unit 35 can preferentially select the auditory modality. This makes it possible to select a modality that is more effective for the receiving user U. For this reason, information that provides the same kind of sensation as the face-to-face environment can be presented online without faithfully reproducing the face-to-face environment.

[0088] In the first and second embodiments described above, subjective sensory formation information is superimposed on one-to-one dialogue information between a user UA and a user UB, but the invention is not limited to this. For example, subjective sensory formation information may be superimposed on one-to-many dialogue information between a user U and multiple users U.

[0089] In the first and second embodiments described above, the information presentation process was described as being applied to dialogue between users U, but it is not limited to this. For example, the information presentation process may be applied to dialogue in a virtual space between VR avatars. In this case, the modality for presenting subjective sensory formation information becomes the gestures and facial expressions of the VR avatars, thus increasing the degree of freedom for superimposing subjective sensory formation information onto dialogue information.

[0090] In the first and second embodiments described above, the information presentation process was described as being applied to a real-time dialogue, but it is not limited to this. For example, the information presentation process may be applied to recorded dialogue information.

[0091] In the first and second embodiments described above, the case in which the program that performs information presentation processing and modality selection processing is executed on the information presentation device 20 was described, but it is not limited to this. For example, the program that performs information presentation processing and modality selection processing may be executed on computing resources on the cloud.

[0092] It should be noted that the present invention is not limited to the embodiments described above, and can be modified in various ways during implementation without departing from its essence. Furthermore, each embodiment may be combined as appropriate, and in that case, the combined effects can be obtained. Moreover, the above embodiments include various inventions, and various inventions can be extracted by selecting combinations from the multiple constituent elements disclosed. For example, if the problem can be solved and effects obtained even if some constituent elements are deleted from all the constituent elements shown in the embodiment, then the configuration with these deleted constituent elements can be extracted as an invention. [Explanation of Symbols]

[0093] 1… Information presentation system 10A, 10B, 10... terminals 11…Control circuits 12…Communication module 13…User Interface 20...Information presentation device 21...Control circuit 22…Communication module 23…Storage 24... Drive 25…Storage medium 31…Environmental information acquisition department 32...Storage section 32a…Environmental information 32b... Subjective sensory model 33...Dialogue Information Acquisition Unit 34...Intention determination section 35…Modality Selection Section 36…Generation part 37…Output section

Claims

1. A determination unit that determines whether the intent of the dialogue information regarding user-to-user interactions is positive or negative, A selection unit that selects a subjective sensation based on the determined intention and selects a modality based on the selected subjective sensation, An output unit that superimposes subjective sensation formation information, which forms the selected subjective sensation using the selected modality, onto the dialogue information and outputs it; An information display device equipped with the necessary features.

2. The aforementioned selection unit is The secondary sensation associated with the determined intention is selected as the subjective sensation. Select the primary sensation associated with the aforementioned selected secondary sensation, Select a modality associated with the aforementioned selected primary sensation. The information display device according to claim 1.

3. The aforementioned dialogue information includes the speaking speed of the dialogue, The determination unit, If the speaking speed is above the first threshold, the intention is determined to be negative. If the speaking speed is below the first threshold, the intention is determined to be positive. The information display device according to claim 1.

4. The aforementioned dialogue information includes the sound pressure of the dialogue. The determination unit, If the sound pressure is above the second threshold, the intention is determined to be positive. If the sound pressure is below the second threshold, the intention is determined to be negative. The information display device according to claim 1.

5. The aforementioned dialogue information includes intent information that clarifies the intent of the dialogue, The determination unit uniquely determines the intent of the dialogue based on the intent information. The information display device according to claim 1.

6. The aforementioned modalities include visual modalities and auditory modalities. The aforementioned visual modality includes at least one selected from body parts, facial expressions, gaze, gestures, text, and icons. The aforementioned auditory modality includes at least one selected from sound effects, ambient sounds, and background music. The information display device according to claim 1.

7. Based on dialogue information regarding user-to-user interactions, determine whether the intent of the dialogue information is positive or negative. Based on the determined intention, a subjective sensation is selected, and based on the selected subjective sensation, a modality is selected. Using the selected modality, subjective sensation formation information that forms the selected subjective sensation is superimposed on the dialogue information and output; A method of presenting information that includes the following features.

8. A program for causing a computer to function as a component of the information presentation device described in any one of claims 1 to 6.

Images