Communication System

Abstract

To provide a communication device allowing a plurality of persons to make communication with less discomfort in both directions via vision, auditory sense, and tactile sense.SOLUTION: Each of Sync Sofas 1A and 1B includes a camera, a microphone, an acceleration sensor, a speaker, a vibration actuator, and a large display 10. Users of each of Sync Sofas 1A and 1B can make communication in both directions in vision, auditory sense, and tactile sense via a control system. Each of Sync Sofas 1A and 1B has a sofa shape having a seating face and a back rest. The large display 10 is arranged so that the seating face side is directed to at least one side of both of left and right sides of the seating face.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a communication system in which two or more people communicate with each other through sight, hearing, and touch. system In particular, sofa-style interactive communication By device communication system Regarding. [Background technology]

[0002] Recently, remote communication devices and software that enable online interpersonal communication between distant locations have been widely used, and the use of embodied communication devices and software that involve the five senses has also been promoted.

[0003] Non-Patent Document 1 discloses a sofa-type physical communication device that utilizes haptic technology in addition to audiovisual technology to realize physical communication involving the whole body, such as sitting next to someone or patting their back, online between remote locations. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] "Development of Sync Sofa, a sofa-type communication system that uses haptic technology to connect people even when they're apart," KDDI Research, Inc. Press Release 2021112501 Summary of the Invention [Problem to be solved by the invention]

[0005] The physical communication devices and audio-tactile technology adjustment programs disclosed in the prior art are designed for one-way five-sense presentation and are unable to provide the desired two-way five-sense presentation.

[0006] Specifically, in two-way five-sensory presentation, which simultaneously acquires and presents information from the five senses, including visual, auditory, and tactile information, there was a problem that even if one-way five-sensory presentation was performed on both sides, howling would occur due to physical interference between the devices involved in acquisition and presentation and the respective pieces of information, and the desired results would not be achieved.

[0007] The object of the present invention is to provide an embodied communication system that solves the above technical problems and realizes two-way remote communication through visual, auditory, and tactile sensations in adjacent communication modes. system The purpose is to provide [Means for solving the problem]

[0008] In order to achieve the above object, the present invention is characterized in that a communication device for remote communication is provided with the following configuration.

[0009] (1) The device has a chair shape and is equipped with at least a visual information acquisition means and presentation means among a tactile information acquisition means and presentation means, an auditory information acquisition means and presentation means, and a visual information acquisition means and presentation means, and the visual information presentation means is provided on at least one of one end side and the other end side of the chair shape.

[0010] (2) The information presented by each acquisition means is transmitted to other communication devices via a network, and each presentation means receives and presents the information acquired by each acquisition means of the other communication devices via the network.

[0011] (3) The microphone used to acquire the auditory information to be transmitted is determined based on the distance between the speaker that presents the received auditory information and each microphone. B The first loop suppression means is provided for switching the phone.

[0012] (4) The microphones used to acquire the auditory information to be transmitted are determined based on the distance between the speaker that presents the received auditory information and each microphone and the frequency characteristics of the received auditory information. B The first loop suppression means is provided for switching the phone.

[0013] (5) The first loop suppression means further comprises means for suppressing the presentation of auditory information.

[0014] (6) Switching the acceleration sensor used to acquire the tactile information to be transmitted based on the distance between the vibration actuator that presents the received tactile information and each acceleration sensor. Second loop prevention measure Equipped with:

[0015] (7) Switching the acceleration sensor used to acquire the tactile information to be transmitted based on the distance between the vibration actuator that presents the received tactile information and each acceleration sensor and the frequency characteristics of the received tactile information. Second loop prevention measure Equipped with:

[0016] (8) The second loop suppression means further includes means for suppressing the presentation of tactile information. [Effects of the Invention]

[0017] According to the present invention, it is possible to realize remote communication involving two-way presentation of the five senses using adjacent communication modes.

[0018] Furthermore, according to the present invention, it is possible to prevent howling that can occur when auditory information or tactile information is looped between communication devices, thereby realizing high-quality two-way communication with less discomfort. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a diagram illustrating a sofa-type communication system according to an embodiment of the present invention. [Figure 2]FIG. 1 is a diagram showing the configuration of the main parts of Sync Sofa. [Figure 3] FIG. 10 is a diagram showing an example in which the user of one Sync Sofa is displayed on the large display of the other Sync Sofa. [Figure 4] FIG. 1 is a functional block diagram showing the configuration of a control system that realizes two-way communication between Sync Sofas. [Figure 5] This is a diagram showing a schematic example of two-way communication using three or more Sync Sofas. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Figure 1 is a diagram showing a sofa-type communication system 1 according to an embodiment of the present invention, which is configured by connecting a pair of remotely located two-way communication sofas (hereinafter referred to as Sync Sofa) 1A, 1B via a network.

[0021] Each Sync Sofa 1A, 1B is equipped with a camera as a visual information acquisition means, a microphone as an auditory information acquisition means, an acceleration sensor as a tactile information acquisition means, a speaker as an auditory information presentation means, and a vibration actuator as a tactile information presentation means (all of which are omitted in Figure 1), as well as a large display 10 as a visual information presentation means, and users of each Sync Sofa 1A, 1B can communicate two-way visually, auditorily, and tactilely via the control system.

[0022] Each of the Sync Sofas 1A and 1B has a sofa shape with a seat and a backrest, and the large display 10 is placed on at least one of the left and right sides of the seat so as to face the seat. The position of the large display 10 on one Sync Sofa is notified in advance to the other Sync Sofa.

[0023] Figure 2 shows the main components of each Sync Sofa 1A, 1B, with the various cushioning materials and surface coverings removed. The Sync Sofa is equipped with at least one large display 10, multiple cameras 11, multiple microphones 12, multiple acceleration sensors 13, multiple speakers 14, and multiple vibration actuators 15, as well as a control system that controls each component to enable two-way communication.

[0024] The camera 11 mainly captures images of users seated on the seats of each Sync Sofa. It is desirable, but not essential, for the camera 11 to be equipped with a depth sensor. The microphone 12 mainly detects the voice and movement sounds, such as the rustling of clothes, of the users seated on the seats of each Sync Sofa. In this embodiment, an audio microphone that mainly detects voice and a movement sound microphone that mainly detects movement sounds are provided separately.

[0025] The acceleration sensor 13 mainly detects the behavior of a user seated on the seat of each Sync Sofa. Various sensors capable of detecting the behavior of a user, such as a vibration sensor, can be used as the acceleration sensor 13. In the description of this embodiment, the various sensors are collectively referred to as the acceleration sensor 13.

[0026] The speaker 14 reproduces the voice and gesture sounds detected by the microphone 12 on the Sync Sofa of the communication partner. The vibration actuator 15 presents the vibrations detected by the acceleration sensor 13 on the Sync Sofa of the communication partner. An image captured by the camera 11 on one Sync Sofa is displayed on the large display 10 on the other Sync Sofa.

[0027] 3 shows an example in which the user of Sync Sofa 1A is displayed on the large display 10 of Sync Sofa 1B. Note that the user of Sync Sofa 1B is also displayed on the large display 10 of Sync Sofa 1A, realizing two-way communication based on visual information.

[0028] In this embodiment, vibration actuators 15 are provided in a total of ten locations: four locations on the seat surface 8 and six locations on the backrest 9. Each vibration actuator 15 is positioned so as to come into contact with the left, right and center of the back, and the left and right buttocks, respectively, based on the width of a person's shoulders and buttocks. This is to stimulate the tactile sensation on each side of the person's back and buttocks.

[0029] Vibration actuator 15 does not come into direct contact with the human body, but is covered with cushioning material about 2 cm thick and sturdy fabric about 1 mm thick, and the vibrations it presents are transmitted through the cushioning material and fabric. Note that the vibrations presented to one Sync Sofa are signals synthesized from vibrations detected by acceleration sensors 13 located on the seat surface 8 and backrest 9 of the other Sync Sofa and the sound of the movement detected by microphone 12.

[0030] FIG. 4 is a functional block diagram showing the configuration of a control system 20 that realizes two-way communication between the Sync Sofas 1A and 1B.

[0031] The control system 20 has well-known control functions for controlling the large display 10, camera 11, microphone 12, acceleration sensor 13, speaker 14, vibration actuator 15, and communication control unit 16, as well as a motion parallax image generation unit 201, a presentation position control unit 202, and a howling suppression unit 203, and each Sync Sofa 1A, 1B is connected to each other via the communication control unit 16 and a network. The control system 20 may be provided for each Sync Sofa, or may be provided on the cloud so that each Sync Sofa can share the control system.

[0032] The motion parallax image generation unit 201 generates a motion parallax image based on images from multiple cameras with different viewpoints and the estimated position of the subject person. The presentation position control unit 202 localizes the auditory information and tactile information at predetermined positions based on the estimated position of the subject person. The howling suppression unit 203 includes a first loop suppression unit 2031 and a second loop suppression unit 2032, and suppresses howling of the auditory information and tactile information.

[0033] Such a control system 20 can be configured by installing applications (programs) that realize the functions detailed below on a general-purpose computer or server equipped with a CPU, ROM, RAM, bus, interface, etc. Alternatively, it can be configured as a dedicated machine or a single-function machine in which part of the application is implemented as hardware or software.

[0034] In this embodiment, two-way communication is achieved by Sync Sofa 1A and Sync Sofa 1B, which have substantially the same configuration, operating cooperatively via control system 20, so the operation of this embodiment will be explained here focusing mainly on the operation of Sync Sofa 1A. To make the explanation easier to understand, the components on the Sync Sofa 1A side are distinguished by adding the suffix A to each component on the Sync Sofa 1B side, and the suffix B to each component on the Sync Sofa 1B side.

[0035] The motion parallax video generation unit 201A of Sync Sofa 1A edits the visual information received from Sync Sofa 1B into a video that reflects the motion parallax of the user UA of Sync Sofa 1A. In this embodiment, the motion parallax video is generated by rendering foreground and background polygon models arranged in 3D space with a virtual camera.

[0036] The foreground is a person object of user UB, whose person area is extracted from the visual information transmitted by Sync Sofa 1B and mapped onto a single rectangular polygon. From the viewpoint of providing a sense of proximity and realism, it is desirable that the background be an object resembling a sofa, wall, floor, etc. The person object is placed in 3D space so as to be displayed life-size on large display 10A.

[0037] The position of the virtual camera can be obtained as a result of tracking the position information of the head of the user UA obtained from the depth sensor-equipped camera 11A. This allows the user UA on Sync Sofa 1A to realize eye contact with the user UB on Sync Sofa 1B while reflecting his or her own motion parallax. Note that motion parallax images can also be generated by switching between foreground images captured simultaneously using multiple cameras according to the viewpoint position or by synthesizing intermediate images.

[0038] The presentation position control unit 202A localizes at predetermined positions the audio information and gesture sounds received from Sync Sofa 1 B. The presentation position control unit 202A can also localize at predetermined positions the tactile information of vibrations and gesture sounds received from Sync Sofa 1 B.

[0039] Regarding auditory information, in this embodiment, the Sync Sofa 1A is equipped with full-range speakers 14 in a total of six locations: four on the left and right of the seat surface 8 and two on the left and right of the backrest 9. Each speaker 14 is positioned at a distance sufficient to localize a sound image.

[0040] In this embodiment, assumed gesture sounds include the sound of the user's clothes rubbing against the surface fabric of the Sync Sofa 1 when changing posture, the friction sound emitted when crossing or fidgeting their legs, and the collision sound emitted when the user sits on the Sync Sofa 1. The gesture sounds of each user are localized to the seat position of the user, and the voice is localized to the mouth of each user displayed on the large display 10.

[0041] Although tactile information has a lower frequency than auditory information, the tactile presentation position may be varied depending on the position of each user, as in the case of auditory information.

[0042] For example, if the received visual information detects a movement of the user's hands, back, buttocks, etc., the vibration presentation position may be changed, or the presentation position may be continuously changed in accordance with the movement. Therefore, if a movement such as stroking the back is detected from the visual information, the vibration actuator 15 corresponding to that position is selectively driven.

[0043] Howling suppression section 203A includes first loop suppression section 2031A that suppresses the loop of auditory information and second loop suppression section 2032A that suppresses the loop of tactile information in order to prevent howling during two-way communication.

[0044] The howling of auditory information in Sync Sofa 1A occurs when the gesture sounds and voices received from Sync Sofa 1B are output from speaker 14A, picked up by microphone 12A, and then transmitted to Sync Sofa 1B for presentation, creating an unnecessary loop in the transmission path of the auditory information, causing this loop to oscillate.

[0045] It is known that the frequency of the auditory information loop depends on the distance between the speaker 14A and the microphone 12A and the impulse response. Therefore, the occurrence of the loop can be suppressed by increasing the distance between the presentation position of the auditory information controlled by the presentation position control unit 202A and the position of the microphone 12A, or by switching the microphone 12A depending on the frequency characteristics of the auditory information.

[0046] In this embodiment, the first loop suppression unit 2031A monitors the position where the gesture sounds and voice received from Sync Sofa 1B are localized (the position where the auditory information is presented), and can switch between the microphone 12A that acquires the gesture sounds and the microphone 12A that acquires the voice based on the distance between the localization position and each microphone 12A.

[0047] Alternatively, based on the frequency characteristics (e.g., high, mid, and low frequencies) within a short-time frame of the auditory information and the distance between the position where the gesture sounds and voice are localized and each microphone 12A, a microphone 12A located in a position where loops are unlikely to occur may be selected for each frequency band of the auditory information, such as microphone 12A_1 for the high frequency range, microphone 12A_2 for the mid frequency range, and microphone 12A_3 for the low frequency range.

[0048] Alternatively, if the power of the auditory information within a short-time frame exceeds a set threshold, more specifically, if the ratio of the average power of all frequencies within the short-time frame to the power of a certain frequency exceeds a preset threshold, it may be determined that a sound has occurred and the output of speaker 14A may be suppressed.

[0049] The howling of tactile information on Sync Sofa 1A occurs when the tactile information received from Sync Sofa 1B is output from vibration actuator 15A, acquired by acceleration sensor 13A, and then transmitted to Sync Sofa 1B for presentation, creating an unnecessary loop in the transmission path of the tactile information, causing this loop to oscillate.

[0050] It is known that the frequency of the haptic information loop depends on the distance and impulse response between vibration actuator 15A and acceleration sensor 13A. Therefore, the occurrence of the loop can be suppressed by increasing the distance between the presentation position of the haptic information controlled by presentation position control unit 202A and the position of acceleration sensor 13A, or by switching acceleration sensor 13A depending on the frequency characteristics of the haptic information.

[0051] In this embodiment, the second loop suppression unit 2032A monitors the position where the vibrations and gesture sounds received from Sync Sofa 1B are localized (the position where the tactile information is presented), and can switch the acceleration sensor 13A that acquires the vibrations based on the distance between the localization position and each acceleration sensor 13A.

[0052] Alternatively, based on the frequency characteristics (e.g., high, mid, and low frequencies) within the short-time frame of the tactile information and the distance between the position where the vibration and movement sound are localized and each acceleration sensor 13A, an acceleration sensor 13A at a position where a loop is unlikely to occur may be selected for each frequency band of the tactile information, such as acceleration sensor 13A_1 for the high frequency range, acceleration sensor 13A_2 for the mid frequency range, and acceleration sensor 13A_3 for the low frequency range.

[0053] Alternatively, if the power within a short-time frame of tactile information exceeds a set threshold, more specifically, if the ratio of the average power of all frequencies within the short-time frame to the power of a certain frequency exceeds a preset threshold, it may be determined that a sound has occurred and the output of vibration actuator 15A may be suppressed.

[0054] In the above embodiment, two-way communication is achieved through visual, auditory, and tactile information, but the present invention is not limited to this and can be similarly applied to any communication system that achieves two-way communication through at least visual information.

[0055] In the above embodiment, the present invention has been described as being applied to a sofa-shaped Sync Sofa, but the present invention is not limited to this and can be similarly applied to any structure that has at least a seat, such as a bench or chair. Also, while it is desirable for the seat to have a backrest, this is not an essential component of the present invention.

[0056] Furthermore, in the above embodiment, the large display 10 is described as being provided on at least one of the left and right sides of the seat, but the present invention is not limited to this. For chairs with circular seats or chairs without a concept of left and right, such as balance balls, the large display 10 can be provided on at least one of the opposing ends of the seating area.

[0057] Furthermore, in the above embodiment, the present invention has been described using two Sync Sofas as an example of two-way communication, but the present invention is not limited to this and can also be applied to two-way communication using three or more Sync Sofas, as shown in Figure 5.

[0058] Furthermore, according to the above embodiment, two-way remote communication can be realized through adjacent communication modes of sight, hearing, and touch, making it possible to contribute to Goal 9 "Build resilient infrastructure and promote inclusive and sustainable industrialization" and Goal 11 "Make cities inclusive, safe, resilient and sustainable" of the Sustainable Development Goals (SDGs) led by the United Nations. [Explanation of symbols]

[0059] 1...sofa-type communication system, 1A, 1B, 1C...Sync Sofa, 8...seat, 9...backrest, 10...large display, 11...camera, 12...microphone, 13...acceleration sensor, 14...speaker, 15...vibration actuator, 16...communication control unit, 20...control system, 201...motion parallax image generation unit, 202...presentation position control unit, 203...howling suppression unit

Claims

1. In a communication system configured by connecting a plurality of communication devices for remote communication with each other via a network, Each communication device a chair-shaped device including at least one of a tactile information acquisition means and a presentation means, an auditory information acquisition means and a presentation means, and a visual information acquisition means and a presentation means; the information acquired by each of the acquisition means is transmitted to another communication device via a network, and each of the presentation means receives and presents the information acquired by each of the acquisition means of the other communication device via the network; the visual information presenting means is provided on at least one of one end side and the other end side of the chair shape, a means for localizing at least one of the auditory information and the tactile information to be presented at a predetermined position based on position information of a person area estimated based on visual information received from the other communication device; and loop suppression means for suppressing looping of at least one of the auditory information and the tactile information based on a position at which at least one of the auditory information and the tactile information is localized.

2. A display is provided as the means for presenting the visual information, The communication system according to claim 1, further comprising a presentation position control means for controlling the image to be displayed on the display based on the relationship between the visual information received from the other communication device and the visual information transmitted to the other communication device.

3. A device comprising a plurality of microphones as means for acquiring auditory information, The communication system described in claim 1 or 2, characterized in that the loop suppression means includes a first loop suppression means that switches the microphone used to acquire the auditory information to be transmitted based on the distance between the position where the auditory information is localized and each microphone.

4. A device comprising a plurality of microphones as means for acquiring auditory information, The communication system described in claim 1 or 2, characterized in that the loop suppression means includes a first loop suppression means that switches the microphone used to acquire the auditory information to be transmitted based on the distance between the position where the auditory information is localized and each microphone and the frequency characteristics of the auditory information.

5. 4. The communication system according to claim 3, wherein the first loop suppression means further comprises means for suppressing presentation of the auditory information.

6. 5. The communication system according to claim 4, wherein the first loop suppression means further comprises means for suppressing presentation of the auditory information.

7. A device including a plurality of acceleration sensors as means for acquiring tactile information, The communication system described in claim 1 or 2, characterized in that the loop prevention means includes a second loop prevention means that switches the acceleration sensor used to acquire the tactile information to be transmitted based on the distance between the position where the tactile information is localized and each acceleration sensor.

8. A device including a plurality of acceleration sensors as means for acquiring tactile information, The communication system described in claim 1 or 2, characterized in that the loop prevention means includes a second loop prevention means that switches the acceleration sensor used to acquire the tactile information to be transmitted based on the distance between the position where the tactile information is localized and each acceleration sensor and the frequency characteristics of the tactile information.

9. 8. The communication system according to claim 7, wherein the second loop suppression means further comprises means for suppressing presentation of the tactile information.

10. 9. The communication system according to claim 8, wherein the second loop suppression means further comprises means for suppressing presentation of the tactile information.

Images