Information processing system and information processing method

Abstract

To facilitate nonverbal communication other than speech to enable smooth dialogues.SOLUTION: An information processing system S includes a plurality of terminal apparatuses each located at a predetermined distance from each of a plurality of users. The plurality of terminal apparatuses includes a first terminal apparatus 1 located at a predetermined distance from a first user, a second terminal apparatus 2 located at a predetermined distance from a second user, and a third terminal apparatus 3 located at a predetermined distance from a third user. Each of the plurality of terminal apparatuses displays an image of each of the other users on a display. The first terminal apparatus 1 detects first distance information between the first user and the second user on the display of the first terminal apparatus 1 and transmits it to the third terminal apparatus 3.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing system and an information processing method. [Background technology]

[0002] Conventionally, a technology is known in which a second terminal displays a virtual space including a first avatar corresponding to a first user and a second avatar corresponding to a second user, moves the first avatar based on a selected movement pattern, and moves the second avatar based on second video data showing the second user (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2020 / 203999 Summary of the Invention [Problem to be solved by the invention]

[0004] When users have a direct conversation, non-verbal communication is also performed in addition to verbal communication. However, there is still room for improvement in terms of sufficient communication in a remote conversation where the conversation partner is fixedly displayed on a screen.

[0005] The purpose of the present disclosure, made in consideration of the above circumstances, is to promote non-verbal communication other than speech and enable smooth dialogue. [Means for solving the problem]

[0006] An information processing system according to an embodiment of the present disclosure includes: An information processing system including a plurality of terminal devices each located at a predetermined distance from a plurality of users, the plurality of terminal devices include a first terminal device located at a predetermined distance from a first user, a second terminal device located at a predetermined distance from a second user, and a third terminal device located at a predetermined distance from a third user; Each of the plurality of terminal devices displays an image of each of the other users on a display unit; The first terminal device detects first distance information between the first user and a second user on a display unit of the first terminal device and transmits the first distance information to the third terminal device.

[0007] An information processing method according to an embodiment of the present disclosure includes: 1. An information processing method for a plurality of terminal devices located at a predetermined distance from each of a plurality of users, comprising: the plurality of terminal devices include a first terminal device located at a predetermined distance from a first user, a second terminal device located at a predetermined distance from a second user, and a third terminal device located at a predetermined distance from a third user; each of the plurality of terminal devices displays an image of each of the other users on a display unit; The first terminal device detects first distance information between the first user and a second user on a display unit of the first terminal device and transmits the first distance information to the third terminal device; Includes. Effect of the Invention

[0008] According to an embodiment of the present disclosure, non-verbal communication other than speech can be promoted, enabling smooth dialogue. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram of an information processing system. [Diagram 2] FIG. 2 is a block diagram showing a configuration of a first terminal device. [Diagram 3] FIG. 2 is a block diagram showing a first scene of a remote dialogue. [Figure 4] FIG. 13 is a block diagram showing a second scene of the remote dialogue. [Diagram 5]FIG. 13 is a block diagram showing a third scene of the remote dialogue. [Figure 6] FIG. 4 is a sequence diagram showing operations executed in the information processing system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] 1 is a schematic diagram of an information processing system S of this embodiment. The information processing system S includes a first terminal device 1, a second terminal device 2, and a third terminal device 3 that can communicate with each other via a network NW. The network NW includes, for example, a mobile communication network, a fixed communication network, or the Internet. The first terminal device 1 is used by a first user P1. The second terminal device 2 is used by a second user P2. The third terminal device 3 is used by a third user P3. The first terminal device 1, the second terminal device 2, and the third terminal device 3 are each located at a predetermined distance from the first user P1, the second user P2, and the third user P3, respectively.

[0011] For ease of explanation, three terminal devices are illustrated in Fig. 1. However, the number of terminal devices is not limited to this.

[0012] The internal configuration of the first terminal device 1 will be described in detail with reference to FIG.

[0013] The first terminal device 1 may be a general-purpose device such as a PC, or a dedicated device. "PC" is an abbreviation for personal computer. Alternatively, the first terminal device 1 may be a mobile device such as a mobile phone, a smartphone, a wearable device, or a tablet.

[0014] The first terminal device 1 includes a control unit 11, a communication unit 12, a storage unit 13, a display unit 14, an input unit 15, an imaging unit 16, and a sensor 17. The components of the first terminal device 1 are connected to each other so as to be able to communicate with each other, for example, via a dedicated line.

[0015] The control unit 11 includes, for example, one or more general-purpose processors including a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit 11 may include one or more dedicated processors specialized for specific processing. Instead of including a processor, the control unit 11 may include one or more dedicated circuits. The dedicated circuits may be, for example, a FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 11 may include an ECU (Electronic Control Unit). The control unit 11 transmits and receives any information via the communication unit 12.

[0016] The communication unit 12 includes a communication module compatible with one or more wired or wireless LAN (Local Area Network) standards for connecting to the network NW. The communication unit 12 may include a module compatible with one or more mobile communication standards including LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation). The communication unit 12 may include a communication module compatible with one or more short-range communication standards or specifications including Bluetooth (registered trademark), AirDrop (registered trademark), IrDA, ZigBee (registered trademark), Felica (registered trademark), or RFID. The communication unit 12 transmits and receives any information via the network NW.

[0017] The memory unit 13 includes, but is not limited to, for example, a semiconductor memory, a magnetic memory, an optical memory, or a combination of at least two of these. The semiconductor memory is, for example, a RAM or a ROM. The RAM is, for example, an SRAM or a DRAM. The ROM is, for example, an EEPROM. The memory unit 13 may function as, for example, a main memory device, an auxiliary memory device, or a cache memory. The memory unit 13 may store information resulting from analysis or processing by the control unit 11. The memory unit 13 may store various information related to the operation or control of the first terminal device 1. The memory unit 13 may store a system program, an application program, embedded software, etc. The memory unit 13 may be provided outside the first terminal device 1 and accessed from the first terminal device 1.

[0018] The display unit 14 is, for example, a display. The display is, for example, an LCD or an organic EL display. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electro luminescence. The display unit 14 may be connected to the first terminal device 1 as an external output device instead of being provided in the first terminal device 1. As a connection method, any method such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) may be used.

[0019] The input unit 15 is, for example, a physical key, a capacitive key, a pointing device, a touch screen integral with a display, or a microphone. The input unit 15 accepts an operation to input information used for the operation of the first terminal device 1. The input unit 15 may be connected to the first terminal device 1 as an external input device instead of being provided in the first terminal device 1. As a connection method, any method such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used. "USB" is an abbreviation for Universal Serial Bus. "HDMI (registered trademark)" is an abbreviation for High-Definition Multimedia Interface.

[0020] The imaging unit 16 includes a camera. The imaging unit 16 can capture images of the surroundings. The imaging unit 16 may record the captured images in the storage unit 13 or transmit the captured images to the control unit 11 for image analysis. The images include still images and video images.

[0021] The sensor 17 may be, for example, a distance measuring sensor. The imaging unit 16 and the sensor 17 may be externally attached. In this case, for example, the imaging unit 16 and the sensor 17 satisfy the requirements of this embodiment as long as the positional relationship with the first terminal device 1 is known.

[0022] The second terminal device 2 includes a control unit 21, a communication unit 22, a storage unit 23, a display unit 24, an input unit 25, an imaging unit 26, and a sensor 27. The third terminal device 3 includes a control unit 31, a communication unit 32, a storage unit 33, a display unit 34, an input unit 35, an imaging unit 36, and a sensor 37. The explanation of the hardware configuration of each of the second terminal device 2 and the third terminal device 3 may be the same as the explanation of the hardware configuration of the first terminal device 1. The explanation here will be omitted.

[0023] The information processing method executed in the information processing system S will be described in detail below. Here, as an example, a first user P1, a second user P2, and a third user P3, each located in a different place, have a conversation using the information processing system S. As an example, the conversation is a remote conversation. The number of people who can have a conversation may be any number as long as it is more than one. The terminal device of each user captures an image of the user using the terminal device by the imaging unit 26, and transmits the captured image to the terminal devices of the other users one by one. The terminal device of each user displays an image (e.g., a 3D model) of each other user on the display unit during the remote conversation. As shown in FIG. 3, the third terminal device 3 of the third user P3 displays the first user P1 and the second user P2 on the display unit 34. In FIG. 3, the second user P2 and the third user P3 are aligned. Therefore, in FIG. 4, the line of sight P2e of the second user P2 and the line of sight P3e of the third user P3 are aligned.

[0024] 4, the first user P1 approaches the second user P2 in the direction of the arrow A1 while looking at the second user P2 along the line of sight P1e. ​​The control unit 11 of the first terminal device 1 sequentially detects, using the sensor 17, first distance information indicating a first distance between the position of the first user P1 (e.g., the position of the face of the first user P1) and the position of the 3D model of the second user P2 on the display unit 14. The control unit 11 of the first terminal device 1 sequentially transmits the detected first distance information to the third terminal device 3 of the third user P3.

[0025] As an alternative example, the first terminal device 1 may detect and transmit the first distance information between the position of the first user P1 and the position of the second user P2 on the display unit 14 only while the first user P1 pays attention to the second user P2 on the display unit 14 (i.e., while the line of sight or face of the first user P1 is directed toward the second user P2 on the display unit 14). In other words, the first terminal device 1 does not detect and transmit the first distance information while the first user P1 is not paying attention to the second user P2.

[0026] The control unit 31 of the third terminal device 3 of the third user P3 adjusts distance information indicating the distance on the display unit 34 between the 3D model of the first user P1 and the 3D model of the second user P2 based on the first distance information sequentially transmitted from the first terminal device 1. The adjustment means shortening the distance or lengthening the distance. Therefore, as the first user P1 approaches the second user P2 in the direction of the arrow A1 as shown in FIG. 4, the first user P1 also approaches the second user P2 in the direction of the arrow A1 in the third terminal device 3 in FIG. 5. For example, the control unit 31 may increase or decrease (e.g., decrease by 10%) the distance information between the 3D model of the first user P1 and the 3D model of the second user P2 on the display unit 34 according to the rate at which the first distance information between the position of the first user P1 and the 3D model of the second user P2 increases or decreases (e.g., decrease by 10%). The control unit 31 of the third terminal device 3 may adjust the distance information between the first user P1 and the second user P2 on the display unit 34 to be shorter as the first distance information transmitted from the first terminal device 1 becomes shorter. As an alternative example, the control unit 31 may increase or decrease the distance information between the 3D model of the first user P1 and the 3D model of the second user P2 on the display unit 34 according to the value by which the first distance information is increased or decreased.

[0027] As an additional example, the second terminal device 2 may sequentially detect second distance information indicating a second distance between the position of the second user P2 and the position of the first user P1 on the display unit 24 of the second terminal device 2, and transmit the second distance information to the third terminal device 3. In this case, the third terminal device 3 may adjust the distance information between the first user P1 and the second terminal device 2 on the display unit 34 based on the total distance of the first distance information sequentially transmitted from the first terminal device 1 and the second distance information sequentially transmitted from the second terminal device 2. For example, the control unit 31 may increase or decrease (e.g., decrease by 10%) the distance information between the 3D model of the first user P1 and the 3D model of the second user P2 on the display unit 34 in accordance with the rate at which the total distance has increased or decreased (e.g., decrease by 10%).

[0028] In the above embodiment, a case is described in which the first user P1 approaches the second user P2, and the distance information is adjusted by the third terminal device 3 of the third user P3. However, the first user P1, the second user P2, and the third user P3 can be arbitrarily interchanged.

[0029] With reference to FIG. 6, an information processing method executed at any point in time by the information processing system S will be described.

[0030] In step S1, the first terminal apparatus 1 captures an image of the first user P1. In step S2, the first terminal apparatus 1 transmits the captured image to the third terminal apparatus 3.

[0031] In step S3, the second terminal apparatus 2 captures an image of the second user P2. In step S4, the second terminal apparatus 2 transmits the captured image to the third terminal apparatus 3. Steps S3 and S4 may be executed before steps S1 and S2.

[0032] In step S5, the third terminal device 3 displays a 3D model of the first user P1 and a 3D model of the second user P2 on the display unit 34 in accordance with the received captured image.

[0033] In step S6, the first terminal apparatus 1 detects first distance information between the first user P1 and the second user P2 on the display unit 14. The first terminal apparatus 1 transmits the first distance information to the third terminal apparatus 3 in step S7.

[0034] In step S8, the third terminal device 3 adjusts distance information indicating the distance between the first user P1 and the second user P2 on the display unit 34 of the third terminal device 3 based on the first distance information transmitted from the first terminal device 1.

[0035] As described above, according to this embodiment, the third terminal device 3 adjusts the distance information between the first user P1 and the second user P2 on the display unit 14 of the third terminal device 3 based on the first distance information transmitted from the first terminal device 1. With this configuration, the information processing system S enables the third user P3 to visually recognize the change (fluctuation) in the distance between the first user P1 and the second user P2. Thus, the information processing system S promotes non-verbal communication other than speech and enables smooth dialogue.

[0036] According to the present embodiment, the shorter the first distance information transmitted from the first terminal device 1, the shorter the distance information between the first user P1 and the second user P2 is adjusted by the third terminal device 3 on the display unit 14 of the third terminal device 3. With this configuration, for example, when the first user P1 approaches or moves away from the second user P2 on the display unit 14, the third user can visually recognize that the distance between the first user P1 and the second user P2 displayed on the display unit 34 becomes shorter or longer. Thus, the information processing system S can make it easier for the third user P3 to understand the sense of distance between the first user P1 and the second user P2.

[0037] Furthermore, according to this embodiment, the third terminal device 3 adjusts the distance information between the first user P1 and the second user P2 on the display unit 34 of the third terminal device 3 based on the total distance of the first distance information transmitted from the first terminal device 1 and the second distance information transmitted from the second terminal device 2. With this configuration, the information processing system S can make it easier for the third user P3 to understand the sense of distance between the first user P1 and the second user P2.

[0038] Furthermore, according to this embodiment, the first terminal device 1 detects and transmits the first distance information only while the first user P1 is paying attention to the second user P2 on the display unit 14. With this configuration, the information processing system S can reduce the possibility that the distance between the first user P1 and the second user P2 will change on the display unit 34 of the third terminal device 3, for example, when the first user P1 changes his / her posture independently of the second user P2.

[0039] Although the present disclosure will be described based on various drawings and examples, it should be noted that those skilled in the art may make various modifications and alterations based on the present disclosure. Other modifications are possible within the scope of the present disclosure. For example, the functions included in each means or step can be rearranged so as not to cause logical inconsistencies, and multiple means or steps can be combined into one or divided.

[0040] For example, in the above embodiment, a program for executing all or part of the functions or processes of the first terminal device 1, the second terminal device 2, or the third terminal device 3 can be recorded in a computer-readable recording medium. The computer-readable recording medium includes a non-transitory computer-readable medium, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, or a semiconductor memory. The program is distributed, for example, by selling, transferring, or lending a portable recording medium such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory) on which the program is recorded. The program may also be distributed by storing the program in the storage of an arbitrary server and transmitting the program from the arbitrary server to another computer. The program may also be provided as a program product. The present disclosure can also be realized as a program executable by a processor.

[0041] A computer temporarily stores a program recorded on a portable recording medium or a program transferred from a server in a main storage device. Then, the computer reads the program stored in the main storage device with a processor and executes processing according to the read program with the processor. The computer may read the program directly from a portable recording medium and execute processing according to the program. The computer may execute processing according to the received program each time a program is transferred from the server to the computer. Processing may be executed by a so-called ASP-type service that realizes functions only by execution instructions and result acquisition without transferring a program from the server to the computer. "ASP" is an abbreviation for application service provider. Programs include information used for processing by a computer and equivalent to a program. For example, data that is not a direct instruction to a computer but has the property of defining computer processing falls under " equivalent to a program. [Explanation of symbols]

[0042] S Information Processing System

Claims

1. An information processing system including a plurality of terminal devices each located at a predetermined distance from a plurality of users, the plurality of terminal devices include a first terminal device located at a predetermined distance from a first user, a second terminal device located at a predetermined distance from a second user, and a third terminal device located at a predetermined distance from a third user; Each of the plurality of terminal devices displays an image of each of the other users on a display unit; The first terminal device detects first distance information between the first user and a second user on a display unit of the first terminal device and transmits the first distance information to the third terminal device; The third terminal device adjusts distance information between the first user and the second user on a display unit of the third terminal device based on the first distance information transmitted from the first terminal device; The third terminal device adjusts the distance information between the first user and the second user on a display unit of the third terminal device so that the shorter the first distance information transmitted from the first terminal device is, the shorter the distance information between the first user and the second user is. Information processing system.

2. 2. The information processing system according to claim 1, the second terminal device detects second distance information between the second user and the first user on a display unit of the second terminal device and transmits the second distance information to the third terminal device; The third terminal device adjusts the distance between the first user and the second user on a display unit of the third terminal device based on a total distance of the first distance information and the second distance information transmitted from the first terminal device. Information processing system.

3. 2. The information processing system according to claim 1, An information processing system, wherein the first terminal device detects and transmits the first distance information only while the first user is paying attention to the second user on a display unit.

4. 1. An information processing method for a plurality of terminal devices located at a predetermined distance from each of a plurality of users, comprising: the plurality of terminal devices include a first terminal device located at a predetermined distance from a first user, a second terminal device located at a predetermined distance from a second user, and a third terminal device located at a predetermined distance from a third user; each of the plurality of terminal devices displays an image of each of the other users on a display unit; The first terminal device detects first distance information between the first user and a second user on a display unit of the first terminal device and transmits the first distance information to the third terminal device; The third terminal device adjusts distance information between the first user and the second user on a display unit of the third terminal device based on the first distance information transmitted from the first terminal device; the third terminal device adjusts, on a display unit of the third terminal device, distance information between the first user and the second user to be shorter as the first distance information transmitted from the first terminal device becomes shorter; An information processing method comprising:

Images