Information processing device and information processing method
The information processing device enhances remote communication by using selective audibility and spatiality adjustments to improve sound source positioning and audibility, addressing the challenges of multiple speakers in remote communication systems.
Patent Information
- Application Number
- PCT/JP2024/015211
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-10
- Filing Date
- 2024-04-17
- Publication Date
- 2024-11-14
AI Technical Summary
Existing remote communication systems struggle with maintaining accurate sound source positioning and selective audibility when multiple speakers are present, leading to muddled voices and limitations in the number of speakers that can be effectively heard.
An information processing device and method that includes a selective audibility speaker selection unit, which determines the positions of speakers in both parties' spaces to prioritize spatiality and selective audibility, using techniques like audio-person position mapping and speaker selection algorithms to optimize sound output based on speaker positions and numbers.
Improves the selective audibility of multiple speakers by accurately positioning sound sources and allowing for clearer distinction between voices, even when multiple people speak simultaneously, while maintaining a high sense of presence and flexibility in audio output.
Smart Images

Figure JP2024015211_14112024_PF_FP_ABST
Abstract
Description
Information processing device and information processing method
[0001] The present disclosure relates to an information processing device and an information processing method.
[0002] There are known remote communication systems that enable remote communication between multiple people in remote areas. In such remote communication systems, a person in one remote area communicates with a person in another remote area using the microphone, speaker, camera, etc. of the remote communication system.
[0003] Japanese Patent Application Publication No. 11-136656
[0004] The remote communication system uses a plurality of microphones in one remote area to pick up the voice of a speaker, and reproduces the picked-up voice from a speaker in another remote area.
[0005] However, when a speaker is located between two microphones in the remote area of the other party, the speaker's voice is heard from both speakers corresponding to the microphone positions in the remote area of the first party, resulting in a problem that the accuracy of the sound source position (spatiality) is not maintained. Furthermore, when multiple people's voices are played from one speaker, the voices become muddied, making it difficult to hear what is being said when multiple people are talking. Furthermore, if the number of speakers could be increased, it would be easier to selectively listen to the speaker's voice and maintain spatiality, but there is a limit to the number of speakers that can be used.
[0006] The present disclosure has been made in consideration of the above-described circumstances, and provides an information processing device and an information processing method that can improve selective audibility of voices of multiple speakers.
[0007] The information processing device of the present disclosure includes a selective audibility speaker selector that selects each of a plurality of speakers in the other person's space so that the speakers have selective audibility for the plurality of speakers in the own space.
[0008] The information processing method disclosed herein is an information processing method for an information processing device having a selective audibility speaker selection unit that selects multiple speakers in the other person's space so that each speaker has selective audibility for multiple speakers in the one's own space, wherein the selective audibility speaker selection unit determines whether the number of speakers in the other person's space is less than or equal to the number of speakers in the one's own space, and if the number of speakers in the other person's space is less than or equal to the number of speakers in the one's own space, selects the multiple speakers in the one's own space for the multiple speakers in the other person's space based on the positions of the multiple speakers in the other person's space and the positions of the multiple speakers in the one's own space.
[0009] FIG. 1 is a functional block diagram of a remote communication device according to a first embodiment; FIG. 2 is a functional block diagram for explaining the flow of data between a transmitting unit of a remote communication device on the user's side and a receiving unit of a remote communication device on the other side according to the first embodiment; FIG. 3 is a diagram for explaining spatiality; FIG. 4 is a diagram showing an example of mapping between a speaker and a speaker when selective audibility is prioritized; p = 3, n s 10 is a diagram showing an example of the arrangement of speakers and speakers when n = 3. p = 3, n s FIG. 10 is a diagram showing a tree diagram of combinations of speakers and speakers when n = 3. p = 5, n s1 is a diagram illustrating an example of the placement of speakers and speakers when n = 3. A diagram for explaining the speaker position-speaker distance. A diagram for explaining the sound output from speakers when spatiality is prioritized. A diagram illustrating a GUI of a changeover switch for selecting spatiality and selective audibility. A flowchart for explaining the operation of a voice-person position mapping unit of a remote communication device according to a first embodiment. A flowchart for explaining a method for calculating the speaker position-speaker distance in a speaker selection unit of a remote communication device according to a first embodiment. A flowchart for explaining the operation of individual sound-speaker mapping in a speaker selection unit of a remote communication device according to a first embodiment. A flowchart for explaining the overall operation of a speaker selection unit of a remote communication device according to a first embodiment. A flowchart for explaining the operation of a spatiality and selective audibility adjustment unit of a remote communication device according to a first embodiment. A functional block diagram of a remote communication device according to a second embodiment. p = 5, n s = 3, n k 10 is a diagram showing an example of the arrangement of speakers, environmental sound speakers, and speakers when .gtoreq.2. is a diagram showing a slider for continuously switching between spatiality and selective visibility according to the third embodiment. is a diagram showing a method of volume fading in the spatiality and selective audibility adjustment unit according to the third embodiment. is a diagram showing the arrangement of devices between site X and site a according to the fourth embodiment. is a diagram showing the arrangement of devices between site X and site b according to the fourth embodiment. is a diagram showing the arrangement of devices between site X and site c according to the fourth embodiment. is a diagram showing the arrangement of cameras, virtual screens, and speakers at a broadcast site according to the fifth embodiment. is a diagram showing the arrangement of origins of speakers and screens at sites 1 to N according to the fifth embodiment. is a hardware configuration diagram showing an example of a computer that realizes the arithmetic unit of the remote communication device that is the information processing device according to the first to fifth embodiments.
[0010] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted. The description will be given in the following order.
[0011] 1. Remote communication device 1 according to the first embodiment 1.1. Person position detection unit 11 1.2. Individual sound generation unit 12 1.3. Transmission unit 13 1.4. Reception unit 21 1.5. Voice-person position mapping unit 22 1.6. Speaker selection unit 23 1.7. Spatiality and selective audibility adjustment unit 24 2. Data flow between remote communication device 1 3. Definition of terms 3.1. Spatiality 3.2. Selective audibility 3.3. Individual sound 4. Speaker selection algorithm 4.1. (i)n p ≦n s In the case of 4.2. (ii) n p >n s 4.3. Calculation of distance between speaker 51 position and speaker 43 5. Adjustment of spatiality and selective audibility 6. Operation 6.1. Mapping of voice to speaker 51 position 6.2. Speaker selection algorithm 6.2.1. Calculation of distance between speaker 51 position and speaker 43 6.2.2. Mapping of individual sound to speaker 43 6.2.3. Overall operation of speaker 43 selection 6.3. Adjustment of spatiality and selective audibility 7. Effects 8. Remote communication device 101 according to second embodiment 8.1. Individual sound generation unit 112 8.2. Transmission unit 113 8.3. Reception unit 121 8.4. Speaker selection unit 123 8.5. Adjustment of spatiality and selective audibility 8.6. Description of signals 8.7. Selection of Environmental Sound Speaker 8.8. Effects 9. Remote Communication Device 201 According to Third Embodiment 9.1. Effects 10. Fourth Embodiment 10.1. Effects 11. Fifth Embodiment 11.1. Effects 12. Hardware Configuration
[0012] 1. Remote Communication Device 1 According to First Embodiment FIG. 1 is a functional block diagram of a remote communication device 1 according to a first embodiment.
[0013] The remote communication device 1 according to the first embodiment has a transmitting unit 2 and a receiving unit 3. The remote communication device 1 shown in FIG. 1 performs two-way communication with a remote communication device 1 on the other side, and the remote communication device 1 on the other side also has a transmitting unit 2 and a receiving unit 3.
[0014] The remote communication device 1 has a transmitting unit 2 and a receiving unit 3, but if two-way communication is not performed, the transmitting remote communication device 1 may only have the transmitting unit 2, and the receiving remote communication device 1 may only have the receiving unit 3.
[0015] The remote communication device 1 is connected to an audio interface 44. The audio interface 44 is connected to a plurality of microphones 42 and a plurality of speakers 43. The audio interface 44 inputs sounds picked up from the plurality of microphones 42 to the remote communication device 1. The audio interface 44 also outputs sounds output from the remote communication device 1 to the speakers 43.
[0016] The microphone 42 may be a pin microphone worn by each person or an omnidirectional microphone (spatial sound collection). The multiple microphones 42 may be arranged in a predetermined array.
[0017] The camera 41 inputs an image of the space in which the transmission unit 2 is located to the remote communication device 1 .
[0018] In Figure 1, signal S_1 indicates input from microphone 42 (MicIn), S_2 indicates one's own video (VideoIn), S_3 indicates one's own position information (OSCIn), S_4 indicates processed audio for the number of channels (SendOut), S_5 indicates audio for the number of channels of the other party (RefIn), S_6 indicates the other party's video (VideoOut), S_7 indicates the other party's position information (OSCOut), and S_8 indicates output to speaker 43 (SpkOut).
[0019] The remote communication device 1 has an input / output control unit (see the input unit 10 and output unit 25 in FIG. 2 ) (not shown) that controls data input to and output from the remote communication device 1. Data input to and output from the remote communication device 1 is input or output via the input / output control unit.
[0020] The transmitting unit 2 includes a person position detecting unit 11, an individual sound generating unit 12, and a transmitting unit 13. The receiving unit 3 includes a receiving unit 21, a sound-to-person position mapping unit 22, a speaker selecting unit 23, and a spatiality and selective audibility adjusting unit 24.
[0021] <1.1. Person position detection unit 11> Based on the image of the space in which the transmission unit 2 is placed input from the camera 41, the person position detection unit 11 detects the position (3D coordinates) of the speaker relative to the origin of the space in which the transmission unit 2 is placed.
[0022] The human position detection unit 11, for example, assumes that the origin of the coordinates is at the center of the screen in the space in which the transmitting unit 2 is placed, and obtains the speaker's position in 3D coordinates using a certain part of the person's body (e.g., the neck) as a representative point.
[0023] The camera 41 may acquire images using Kinect, LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), or UWB (Ultra Wide Band).
[0024] <1.2. Individual sound generation unit 12> The individual sound generation unit 12 acquires only the individual sounds of the speaker who is in the position of the speaker detected by the person position detection unit 11. This makes the number of coordinates of the speaker's position equal to the number of channels of the individual sounds.
[0025] If the microphone 42 is a pin microphone, the individual sound generation unit 12 does not need to process the collected sounds. If multiple microphones 42 are arranged in a predetermined array, the individual sound generation unit 12 performs beamforming processing on the collected sounds. If the microphone 42 is an omnidirectional microphone (spatial sound collection), the individual sound generation unit 12 performs sound source separation processing on the collected sounds.
[0026] <1.3. Transmitting Unit 13> The transmitting unit 13 transmits the position of the speaker in the space where the transmitting unit 2 is located, detected by the person position detecting unit 11, the individual sound of the speaker at the speaker's position acquired by the individual sound generating unit 12, and an image of the space where the transmitting unit 2 is located, captured by the camera 41. The transmitting unit 13 transmits the position of the speaker, the individual sound, and the image of the speaker to the receiving unit 3 of the remote communication device 1 of the other party via the network 45. The position of the speaker, the individual sound, and the image of the speaker are associated with each other in the audio-person position mapping unit 22 in the receiving unit 3 of the remote communication device 1 of the other party. The network 45 is, for example, the Internet, a LAN (Local Area Network), etc.
[0027] 1.4 Receiving unit 21 The receiving unit 21 receives the speaker's position, individual sound, and video of the speaker transmitted from the transmitting unit 2 of the remote communication device 1 on the other side. The receiving unit 21 outputs the received video to a display 46 (display unit). The display 46 displays the video output from the receiving unit 21.
[0028] <1.5. Audio-person position mapping unit 22> The audio-person position mapping unit 22 associates (maps) the position of the speaker in the other person's space with the individual sound of the speaker, based on the position of the speaker in the other person's space, the individual sound of the speaker, and the image of the speaker received by the receiving unit 21.
[0029] The voice-person position mapping unit 22 may, for example, learn and associate the movement of a speaker's mouth with the content of the speech, and the features of the speaker's face with the features of their voice.
[0030] <1.6. Speaker selection unit 23> The speaker selection unit 23 outputs an individual sound associated with the position of the speaker to a speaker 43 in the user's own space that corresponds to the position of the speaker in the other person's space associated by the voice-person position mapping unit 22.
[0031] The speaker selection unit 23 has a selective audibility speaker selection unit that selects each of a plurality of speakers in the other person's space so that the speakers have selective audibility for the plurality of speakers 43 in the own space. The plurality of speakers 43 selected by the selective audibility speaker selection unit have spatiality.
[0032] When the number of speakers in the other person's space is equal to or less than the number of speakers in the own space, the selective audibility speaker selection unit selects speakers 43 in the own space for speakers in the other person's space based on the positions of the speakers in the other person's space and the positions of the speakers 43 in the own space.
[0033] The selective audibility speaker selection unit has a coordinate conversion unit that converts the positions of multiple speakers in the other person's space into the positions of multiple speaking stations in the user's own space, and the selective audibility speaker selection unit selects multiple speakers 43 in the user's own space for the multiple speakers in the other person's space based on the positions of the multiple speakers in the user's own space converted by the coordinate conversion unit and the positions of the multiple speakers 43 in the user's own space.
[0034] When the number of multiple speakers in the other party's space is greater than the number of multiple speakers 43 in the own space, the selective audibility speaker selection unit selects multiple speakers in the own space for multiple speakers who, among the multiple speakers in the other party's space, have a high priority over the multiple speakers 43 in the own space, based on the positions of the multiple speakers in the other party's space and the positions of the multiple speakers 43 in the own space. The selective audibility speaker selection unit selects a speaker other than the multiple speakers with a high priority among the multiple speakers in the other party's space as one of the multiple speakers in the own space.
[0035] 1.7. Spatiality and selective audibility adjustment unit 24 The spatiality and selective audibility adjustment unit 24 adjusts the audio to be output to the speaker 43 selected by the speaker selection unit 23, and outputs the audio to the speaker 43. Audio adjustment is performed when spatiality is selected and when selective audibility is selected. The speaker 43 outputs the audio output from the spatiality and selective audibility adjustment unit 24.
[0036] The spatiality and selective audibility adjustment unit 24 selects a plurality of speakers 43 for a plurality of speakers so that the spatiality of the selected speakers 43 is higher than that of the selected speakers 43 .
[0037] 2. Data flow between remote communication devices 1> Figure 2 is a functional block diagram for explaining the data flow between the transmitting unit 2 of the local remote communication device 1 according to the first embodiment and the receiving unit 3 of the remote communication device 1 of the remote communication device 1. Note that the same parts as those in Figure 1 will be described using the same reference numerals.
[0038] As shown in FIG. 2, the transmission unit 2 of the remote communication device 1 on the user's side includes an input unit 10 , a person position detection unit 11 , an individual sound generation unit 12 , and a transmission unit 13 .
[0039] The input unit 10 receives the speaker's voice from the microphone 42. The input unit 10 outputs the input speaker's voice to the human position detection unit 11.
[0040] Furthermore, an image of the space in which the transmission unit 2 is located is input to the input unit 10 from the camera 41. The input unit 10 outputs the image input from the camera 41 to the human position detection unit 11.
[0041] The person position detection unit 11 detects the position (3D coordinates) of the speaker in the space in which the transmission unit 2 is placed relative to the origin of the space in which the transmission unit 2 is placed, based on the image of the space in which the transmission unit 2 is placed input from the input unit 10.
[0042] The individual sound generation unit 12 acquires only the individual sounds of speakers who are at the speaker positions in the space detected by the person position detection unit 11. For example, in the case of the video shown on the left side of Fig. 2, the individual sounds of two "speakers" are acquired, not three "speakers."
[0043] The transmitting unit 13 transmits the position of the speaker in the space where the transmitting unit 2 is placed, detected by the person position detection unit 11, the individual sound of the speaker at the speaker's position in the space obtained by the individual sound generation unit 12, and the image of the space where the transmitting unit 2 is located, captured by the camera 41, to the receiving unit 3 of the other party's remote communication device 1 via the network 45.
[0044] The receiving unit 3 of the remote communication device 1 on the other side has a receiving section 21 , a voice-person position mapping section 22 , a speaker selecting section 23 , a spatiality and selective audibility adjusting section 24 , and an output section 25 .
[0045] The receiving unit 21 receives the speaker's position, individual sound and video of the speaker transmitted from the transmitting unit 2 of the remote communication device 1 on the other side.
[0046] The voice-person position mapping unit 22 associates (maps) the speaker's position with the speaker's individual sound based on the speaker's position, individual sound, and video received by the receiving unit 21 .
[0047] The speaker selection unit 23 outputs the individual sound associated with the speaker's position to the speaker 43 in the speaker's own space that corresponds to the speaker's position associated by the voice-person position mapping unit 22 .
[0048] The spatiality and selective audibility adjustment unit 24 adjusts the sound to be output to the speaker 43 selected by the speaker selection unit 23, and outputs the sound to the output unit 25. The sound is adjusted when spatiality is selected and when selective audibility is selected.
[0049] The output section 25 is an input / output control section of the receiving unit 3 of the remote communication device 1 on the other side, and outputs the adjusted sound input from the spatiality and selective audibility adjustment section 24 to the speaker 43 .
[0050] 3. Definitions of Terms Next, definitions of terms used in the embodiments will be described. 3.1. Spatiality "Spatiality" refers to the actual positioning accuracy of a sound source relative to the position of a speaker.
[0051] For example, when spatiality is high, the voice can be heard accurately from the speaker's position. When "spatiality" is prioritized, processing is performed to make it seem as if the sound source is located at the speaker's position in the connected space, and the sound is output from speaker 43. When spatiality is high, the sense of realism is enhanced by reproducing the speaker's position, and audiovisual cognitive dissonance is reduced by matching the speaker's position with the position of the person on the video.
[0052] 3 is a diagram for explaining spatiality, showing an example in which spatiality is high and the closer the position of speaker 51 in one's own space, which corresponds to the position of the speaker in the other person's space, is to speaker 43, the more sound is output.
[0053] Specifically, when the spatiality is high, the position of speaker 51_1 in his / her own space is close to speaker 43_1, at a medium distance to speaker 43_2, and far from speaker 43_3. Therefore, the volume of the speech of speaker 51_1 is output at a high volume from speaker 43_1, at a medium volume from speaker 43_2, and at a low volume from speaker 43_3.
[0054] Similarly, the position of speaker 51_2 in his / her own space is close to speaker 43_3, at a medium distance from speaker 43_2, and far from speaker 43_1. Therefore, the volume of the speech of speaker 51_2 is output from speaker 43_3 at a high volume, from speaker 43_2 at a medium volume, and from speaker 43_1 at a low volume.
[0055] The speaker 43 outputs audio in the direction toward the speaker 51. For example, as shown in FIG. 3 , the speaker 43_1 outputs the speech of the speaker 51_1 at a high volume toward the speaker 51_1, and outputs the speech of the speaker 51_2 at a low volume toward the speaker 51_2. The speaker 43_2 outputs the speech of the speaker 51_1 at a medium volume toward the speaker 51_1, and outputs the speech of the speaker 51_2 at a medium volume toward the speaker 51_2. The speaker 43_3 outputs the speech of the speaker 51_1 at a low volume toward the speaker 51_1, and outputs the speech of the speaker 51_2 at a high volume toward the speaker 51_2.
[0056] <3.2. Selective audibility> "Selective audibility" refers to the ease with which each speech can be distinguished when multiple speeches are present.
[0057] For example, when selective audibility is high, the speech of each person can be clearly distinguished. When "selective audibility" is prioritized, individual sound mapping processing to the speaker 43 is performed, and then sound is output from the speaker 43. When selective audibility is high, in a situation where multiple people are speaking at the same time, the speech of each person can be more clearly heard by dividing the output speaker 43 into individual sounds.
[0058] Fig. 4 is a diagram showing an example of mapping between the speaker 51 and the speaker 43 when selective audibility is prioritized. Fig. 4 shows an example in which the speaker 51_1 is mapped to the speaker 43_1 because the distance between the speaker 51_1 and the speaker 43_1 is closer than the distance between the speaker 51_1 and the speaker 43_2 or the speaker 43_3. The speech of the speaker 51_1 is output from the speaker 43_1.
[0059] In this example, the speaker 51_2 is mapped to the speaker 43_3 because the distance between the speaker 51_2 and the speaker 43_3 is closer than the distance between the speaker 51_2 and the speaker 43_1 or speaker 43_2. The speaker 43_3 outputs the speech of the speaker 51_2.
[0060] In FIG. 4, the speaker 43_2 is not mapped to the speaker 51, and therefore the speech of the speaker 51 is not output from the speaker 43_2.
[0061] 3.3. Individual Sounds The term "individual sounds" refers to the individual voices of the speakers 51.
[0062] 4. Speaker Selection Algorithm Next, the operation of the speaker selection unit 23 will be described. The speaker selection operation is performed when the number of speakers 51, n p and the number n of speakers 43 in the self-space. s The following are different in the relationship between (i) n p ≦n s , (ii) n p >n s The following explanation will be given separately for the following cases.
[0063] <4.1. (i) n p ≦n s In the case of (1), the speaker selector 23 selects the speaker 43 while prioritizing selective audibility while maintaining spatiality to a minimum.
[0064] "Selecting a speaker 43 with priority given to selective audibility while maintaining a minimum level of spatiality" assumes that when selective audibility is prioritized, only one person's voice is played from one speaker. In this case, the speaker 43 from which a person's voice is output is not selected randomly, but the speakers 43 are selected while maintaining their relative left-right positional relationship. For example, suppose there is a speaker 43_1 on the left and a speaker 43_2 on the right, and that speaker A, who is a speaker 51, is on the left and speaker B, who is also a speaker 51, is on the right. In this case, speaker A's voice is played from speaker 43_1, and speaker B's voice is played from speaker 43_2. A generalized method of determining this is shown below.
[0065] The selection of the speaker 43 is made under the following constraints: One speaker 43 outputs the voice of one person. The positions and number of speakers 43 are fixed in advance.
[0066] FIG. 5 shows p = 3, n s 6 is a diagram showing an example of the arrangement of the speaker 43 and the speaker 51 when .times. ... 3 P 3 6, the three speakers 43 are indicated as "1", "2", and "3".
[0067] To generalize, It becomes a street.
[0068] For all these combinations, xi The sum of the above is calculated, and the position of the speaker 51 and the speaker 43 are mapped using the combination that produces the smallest sum. xi is the distance between the speaker 51's position in the speaker's space and the speaker 43. xi The calculation method will be described later.
[0069] To generalize, This becomes:
[0070] where: is the sum of the distance between the speaker 51's position and the speaker 43 in the m-th combination, represents the distance between the xth speaker 51 and the speaker 43 corresponding to the xth speaker 51 .
[0071] <4.2. (ii)n p >n s In the case of (i) n p ≦n s As explained in the case of , the speaker selection algorithm prioritizes selective audibility. s It is desirable to have each speaker 43 emit a voice from one person, but if the number of speakers is n p If the number of speakers 43 is greater than the number of speakers 43, it is necessary to play the voices of multiple people from any of the speakers 43.
[0072] In this situation, the procedure for selecting the speaker 43 is as follows: 1. Among the speakers 51, n speakers who are closest to the speaker 43 in the y direction are selected. s The speaker 51 whose name has the highest priority is selected as the speaker 43 by the same algorithm as in (i).
[0073] 2. The speaker 43 has not yet been selected (n p -n s ) The phonetic order of the name is n s The voice of the 1st speaker 51 is output from the speaker 43 (the 1st speaker 43 shown in FIG. 7).
[0074] By doing this, (n p -n s -1) The voices of the people need to be played from one speaker 43, but s -1) Selective audibility of human voices is maintained.
[0075] FIG. 7 shows p = 5, n s 7 is a diagram showing an example of the arrangement of speakers and speakers when y = 3. In Fig. 7, speakers 51 A to C, who are the first to third speakers with the shortest distance to the speaker 43 in the y direction, are assigned to the first to third speakers 43. Speakers 51 D and E are assigned to the first speaker 43.
[0076] The speaker selection unit 23 selects the number of speakers n p The change in (i) is determined based on the number of channels of the individual sounds received by the receiving unit 21. The speaker selecting unit 23 switches between the speaker selection algorithms (i) and (ii) based on the determination result.
[0077] At the time of the "speaker selection algorithm," selective audibility is prioritized, so sound attenuation according to distance is not taken into consideration. Sound attenuation according to distance is taken into consideration when spatiality is prioritized in the spatiality and selective audibility adjustment unit 24.
[0078] 4.3. Calculation of the distance between the position of the speaker 51 and the speaker 43> Figure 8 is a diagram for explaining the distance between the position of the speaker 51 and the speaker 43. It is assumed that the positions of the camera 41, the speaker 43, and the screen 47, as well as the camera angle of view, have been determined in advance. Furthermore, the screen of the other party's space, which is placed in the user's own space, is assumed to be a virtual screen 48. The remote communication device 1 is premised on a two-way connection, and the user is basically assumed to use the remote communication device 1 by facing the camera 41 (not the screen) in the user's own space.
[0079] Distance d between the position of the speaker 51 and the speaker 43 xi can be obtained by the following procedure: 1. A virtual screen 48 is placed on a plane perpendicular to the light beam of the camera 41. The distance d between the camera 41 and the virtual screen 48 is calculated from the angle of view of the camera 41 and the size of the screen 47. c-scr is derived.
[0080] 2. Since the origins of the self space and the other party space are different, the position (x', y', z') of the speaker 51 in the other party space received by the receiving unit 21 is converted into coordinates (x, y, z) in the self space, using the self space as a reference. 3. The position (x, y, z) of the speaker 51 and the position (x i , y i , z i ) from the speaker 51 position to the speaker 43 distance d xi is obtained, and the distance d between the position of the speaker 51 and the speaker 43 is calculated. xi is used in the "speaker selection algorithm".
[0081] 5. Adjustment of Spatiality and Selective Audibility The "speaker selection algorithm" prioritizes selective audibility when outputting sound. Here, a method for switching to spatiality priority will be described. Figure 9 is a diagram illustrating the sound output from the speaker 43 when spatiality priority is set.
[0082] In the case of prioritizing spatiality, the number of speakers 43 is n s Regardless of the sound pickup method, the individual sounds recorded are all assumed to have the same gain (gain = G org[dB])
[0083] 1. For the target speaker 51, the distance d to each voice speaker 43 xi Calculate the distance d [m]. xi The calculation is made using the "speaker selection algorithm".
[0084] 2. Taking into account distance attenuation, the gain of the voice to be output from each voice speaker 43 for the target speaker 51 is determined. G xi = G org -f(d xi )
[0085] Any attenuation function can be used for f. For example, sound pressure generally decays logarithmically over distance (see https: / / www.patlite.co.jp / support / faq / detail00538.html). The function in this case is f(d xi ) = 20 log 10 d xi This becomes:
[0086] 3. The same calculation is performed for all speakers 51, and the voices are superimposed and played from the speaker 43.
[0087] 10 is a diagram showing a GUI (Graphical User Interface) of a changeover switch 60 for selecting spatiality and selective audibility. As shown in FIG. 10, the changeover switch 60 has a spatiality selection button 61 and a selective audibility button 62.
[0088] The spatiality selection button 61 is used to switch the audio output from the speaker 43 to spatiality priority and adjust the audio output from the speaker 43 as described above in <5. Adjusting spatiality and selective audibility>.
[0089] The selective audibility button 62 switches the audio output from the speaker 43 to selective audibility, and as described above in <4. Speaker Selection Algorithm>, is used to select the speaker 43 and switch the audio output from the speaker 43.
[0090] <6. Operation> <6.1. Mapping of voice-speaker 51 position> Next, the operation of the voice-person position mapping unit 22 will be described. Fig. 11 is a flowchart for explaining the operation of the voice-person position mapping unit 22 of the remote communication device 1 according to the first embodiment. In Fig. 11, the information in [ ] indicates information received by the receiving unit 21 and used for processing.
[0091] 11, the voice-person position mapping unit 22 identifies the position (2D coordinates) of the speaker 51 in the received video (step S1). Next, the voice-person position mapping unit 22 maps the position (2D coordinates) of the speaker 51 in the video identified in step S1 to the position (3D coordinates) of the speaker 51 (step S2). Here, the position (3D coordinates) of the speaker 51 corresponds to the position (x', y', z') of the speaker 51 in the other person's space.
[0092] Next, the voice-person position mapping unit 22 maps the individual sounds to the 2D coordinates of the speaker 51 (step S3), and maps the individual sounds to the positions (3D coordinates) of the speaker 51 at the sound sources (step S4).
[0093] Although FIG. 11 shows a case where the voice and the position of the speaker 51 are mapped using the video as an intermediary, other methods may be used as long as they can link the sound source position with the voice information.
[0094] <6.2. Speaker Selection Algorithm> Next, the operation of the speaker selection unit 23 will be described.
[0095] 12 is a flowchart for explaining a method for calculating the distance between the position of the speaker 51 and the speaker 43 by the speaker selection unit 23 of the remote communication device 1 according to the first embodiment. As shown in FIG. 12, the speaker selection unit 23 calculates the distance d between the camera 41 and the virtual screen 48. c-scr (Step S11). Next, the speaker selection unit 23 performs coordinate transformation of the position of the speaker 51 (Step S12).
[0096] Next, the speaker selection unit 23 selects the distance d between the speaker 51 and the speaker 43. ij is the number of speakers 51 (i=0; i<n p ; i++) and the number of speakers 43 (j=0; j<n s ;j++) (steps S13 to S15), and d[n p ][n s ] is obtained as a two-dimensional array (step S16).
[0097] <6.2.2. Individual Sound-Speaker 43 Mapping> The following describes individual sound-speaker 43 mapping by the speaker selection unit 23. Fig. 13 is a flowchart for explaining the operation of individual sound-speaker 43 mapping by the speaker selection unit 23 of the remote communication device 1 according to the first embodiment.
[0098] As shown in FIG. 13, the speaker selection unit 23 selects a combination of a speaker 51 and a speaker 43. The street is determined (step S21).
[0099] The speaker selection unit 23 selects the input d[n p ][n s ] (step S22), the sum of the distances between m different speakers 51 and the corresponding speakers 43 (step S23, step S24), The sum of the distance between the street speaker 51 and the corresponding speaker 43 is calculated (step S25).
[0100] Then, the minimum distance between the speaker 51 and the corresponding speaker 43 is calculated. (step S26), individual sound-speaker 43 mapping is performed (step S27), and the process ends.
[0101] <6.2.3. Overall Operation of Selecting the Speaker 43> Next, a description will be given of the overall operation of the speaker selection unit 23 to select the speaker 43. Fig. 14 is a flowchart for explaining the overall operation of the speaker selection unit 23 of the remote communication device 1 according to the first embodiment.
[0102] 14, the speaker selection unit 23 calculates the distance between the position of the speaker 51 and the speaker 43 (step S31). The calculation of the distance between the position of the speaker 51 and the speaker 43 in step S31 is performed as described in FIG.
[0103] Then, n p ≦n s It is determined whether n p ≦n s If (Yes in step S32), then x=n p is input (step S33), individual sound-speaker 43 mapping processing is performed (step S34), and the processing ends. The individual sound-speaker 43 mapping processing is performed as described with reference to FIG.
[0104] In step S32, n p ≦n s If not (No in step S32), the individual sounds are ranked based on the distance in the y direction in the self space (step S35), and x=n s is input (step S36), and the individual sound-speaker 43 mapping process is performed (step S37). The individual sound-speaker 43 mapping process is performed as described with reference to FIG.
[0105] Then, (n p -n s The individual sounds of the name are assigned to the speaker 43 (step S38), and the process ends.
[0106] 6.3. Spatiality and Selective Audibility Adjustment FIG. 15 is a flowchart for explaining the operation of the spatiality and selective audibility adjustment unit 24 of the remote communication device 1 according to the first embodiment.
[0107] 15, the spatiality and selective audibility adjustment unit 24 receives a selection of the priority of spatiality and selective audibility (step S31). This selection of spatiality and selective audibility is performed using, for example, a changeover switch 60 displayed on a GUI such as that shown in FIG.
[0108] In step S32, it is determined whether spatiality priority was selected in step S31 (step S32), and if spatiality priority was not selected in step S32 (No in step S32), selective audibility is selected (step S33), and the speaker 43 and audio selected by the speaker selection unit 23 are selected.
[0109] If spatial priority is selected in step S32 (Yes in step S32), the gain attenuation process G xi = G org -f(d xi ) is used to determine the voice to be played from each speaker 43 for the target speaker 51 (steps S34 to S36).
[0110] 7. Effects According to the remote communication device 1 according to the first embodiment, selective audibility of the voices of the multiple speakers 51 can be improved.
[0111] Furthermore, according to the remote communication device 1 of the first embodiment, by collecting individual sounds rather than collecting sounds spatially, it is possible to process the voice of each speaker 51 .
[0112] According to the remote communication device 1 of the first embodiment, when "spatiality" is prioritized, the position from which each individual sound is emitted is identified from the video and associated with the video, thereby matching the image and sound and providing a high sense of realism. When "selective audibility" is prioritized, the voices of multiple speakers 51 that are particularly desired to be heard are played back individually from the speaker 43, making it easier to distinguish between multiple people speaking simultaneously.
[0113] Furthermore, by providing the spatiality and selective audibility adjustment unit 24, it is possible to adjust "spatiality" so that the direction of the sound source of the speaker 51 does not contradict the video, and "selective audibility" so that the voices of multiple speakers 51 can be easily heard. This allows the user to change how the voices are heard to suit their needs. Furthermore, even if the placement of cameras 41 and the like differs at each connection point, it is possible to select an appropriate speaker 43 according to the speaker's position in the other party's space.
[0114] 8. Remote communication device 101 according to a second embodiment Next, a remote communication device 101 according to a second embodiment will be described. In the second embodiment, an environmental sound pickup microphone 71 and an environmental sound speaker 72 are connected to the remote communication device 101 via an audio interface 44. In the second embodiment, the voice of the speaker 51 is also allocated to the environmental sound speaker 72.
[0115] Fig. 16 is a functional block diagram of a remote communication device 101 according to the second embodiment. Note that the same parts as in Fig. 1 are given the same reference numerals, and Fig. 16 will mainly describe parts that are different from Fig. 1.
[0116] The environmental sound pickup microphone 71 picks up environmental sounds in the user's own space and outputs the picked up environmental sounds to the individual sound generation unit 12. The environmental sound speaker 72 receives the environmental sounds at the receiving unit 21 and outputs the environmental sounds from the spatiality and selective audibility adjustment unit 24.
[0117] <8.1. Individual sound generation unit 112> The individual sound generation unit 112 acquires only the individual sounds of the speaker who is at the speaker's position in the space detected by the person position detection unit 11. This makes the number of coordinates of the speaker's position equal to the number of channels of the individual sounds.
[0118] If the microphone 42 is a pin microphone, the individual sound generation unit 112 does not need to process the collected sounds. If multiple microphones 42 are arranged in a predetermined array, the individual sound generation unit 112 performs beamforming processing on the collected sounds. If the microphone 42 is an omnidirectional microphone (spatial sound collection), the individual sound generation unit 112 performs sound source separation processing on the collected sounds.
[0119] The individual sound generation unit 112 transmits the environmental sound collected by the environmental sound collection microphone 71 to the transmission unit 13 .
[0120] <8.2. Transmitting Unit 113> The transmitting unit 113 transmits the position of the speaker in the space where the transmitting unit 2 is located, detected by the person position detecting unit 11, the individual sound of the speaker who is at the speaker's position in the space acquired by the individual sound generating unit 12, and an image of the space where the transmitting unit 2 is located, captured by the camera 41. The transmitting unit 113 transmits the position of the speaker, the individual sound, and the image of the speaker to the receiving unit 3 of the remote communication device 1 of the other party via the network 45. The position of the speaker, the individual sound, and the image of the speaker are associated with each other in the audio-person position mapping unit 22 in the receiving unit 3 of the remote communication device 1 of the other party. The network 45 is, for example, the Internet, a LAN, or the like.
[0121] Furthermore, the transmitting unit 113 transmits the environmental sound transmitted from the individual sound generating unit 112 to the receiving unit 3 of the remote communication device 1 on the other side via the network 45 .
[0122] 8.3 Receiving unit 121 The receiving unit 121 receives the speaker's position, individual sound, and video of the speaker transmitted from the transmitting unit 2 of the remote communication device 1 on the other side. The receiving unit 121 outputs the received video to the display 46 (display unit). The display 46 displays the video output from the receiving unit 121.
[0123] The receiving section 121 also receives the environmental sound transmitted from the transmitting unit 2 of the remote communication device 1 on the other side.
[0124] 8.4. Speaker Selector 123 The speaker selector 123 outputs an individual sound associated with the speaker's position to the speaker 43 in the speaker's own space that corresponds to the speaker's position associated by the voice-person position mapping unit 22.
[0125] Furthermore, the speaker selection unit 123 outputs the environmental sound received by the receiving unit 121 to the environmental sound speaker 72 .
[0126] 8.5. Spatiality and selective audibility adjustment unit 124 The spatiality and selective audibility adjustment unit 124 adjusts the audio to be output to the speaker 43 selected by the speaker selection unit 123, and outputs the audio to the speaker 43. Audio adjustment is performed when spatiality is selected and when selective audibility is selected. The speaker 43 outputs the audio output from the spatiality and selective audibility adjustment unit 124.
[0127] Furthermore, the spatiality and selective audibility adjustment unit 124 outputs the environmental sound output from the speaker selection unit 123 to the environmental sound speaker 72 .
[0128] 8.6. Description of Signals In FIG. 16, a signal S_11 indicates the input of environmental sound from the environmental sound pickup microphone 71, and a signal S_12 indicates the output to the environmental sound speaker 72.
[0129] 8.7. Selection of Speakers for Environmental Sounds FIG. p = 5, n s = 3, n k 17 is a diagram showing an example of the arrangement of speakers 43, environmental sound speakers 72, and speakers 51 when (environmental sound speakers 72)=2. As shown in Fig. 17, speakers A, B, and C of speakers 51 are each assigned to speaker 43. Speakers D and E of speakers 51 are assigned to the environmental sound speaker 73.
[0130] The operation of the speaker selection unit 123 according to the second embodiment will be described. p ≦n s In the first embodiment, in the case of <4.1. (i) n p ≦n s This is the same as the case described in >.
[0131] (ii) n p >n s 1. Among the speakers 51, n speakers who are closest to the speaker 43 in the y direction are s The speaker 51 whose name has the highest priority is selected as the speaker 43 by the same algorithm as in (i).
[0132] 2. The speaker 43 has not yet been selected (n p -n s The voice of the name is played from the environmental sound speaker 72. That is, the speaker selection unit 123 selects, from among the plurality of speakers in the other space, a speaker other than the plurality of speakers with the highest priority, for the environmental sound speaker.
[0133] This results in n s The selective audibility of the voices of the people is maintained. p -n s ) speech is given a low priority and therefore selective audibility is not taken into account.
[0134] 8.8 Effects Therefore, according to the remote communication device 101 of the second embodiment, it is possible to transmit not only the voice of the speaker 51 but also the sound of the space, thereby increasing the sense of realism. Furthermore, when the number of speakers is greater than the number of speakers 43 with selective audibility prioritized, by allocating the sounds of a greater number of speakers to be output from the environmental sound speaker 72, it is possible to reproduce the sounds of all speakers 51 without sacrificing any of the voice speakers 43.
[0135] 9. Remote communication device 201 according to the third embodiment In the first embodiment, as shown in Fig. 10, a case has been described in which spatiality and selective visibility are switched by a changeover switch 60. In the remote communication device 201 according to the third embodiment, priority between "spatiality" and "selective audibility" is continuously switched.
[0136] 18 is a diagram showing a slider 81 for continuously switching between the priority of spatiality and the priority of selective visibility according to the third embodiment. The slider 81 may be displayed by a GUI or may be provided mechanically.
[0137] As shown in Figure 18, when switching from "spatiality" priority to "selective visibility" priority, the voice of each speaker 51 for each speaker 43 when "spatiality" priority is set fades out as the slider 81 is moved to "selective visibility," and eventually becomes zero.
[0138] On the other hand, as the "selective audibility" priority increases, the voice of each speaker 51 for each speaker 43 fades in, and finally each individual sound is mapped to the speaker 43. In this way, by using volume fade, it is possible to continuously transition between the two states. Fig. 19 is a diagram showing a volume fade method in the spatiality and selective audibility adjustment unit 24 according to the third embodiment. As shown in Fig. 19, the volume fade method can be, for example, constant gain, constant power, or exponential fade.
[0139] The spatiality and selective audibility adjustment unit 224 according to the third embodiment has a slider display unit that displays a slider 81 that continuously switches the priority between selective visibility and spatiality, and a volume adjustment unit that adjusts the volume of the selective visibility and the volume of the spatiality of the multiple speakers 43 in the user's own space in accordance with the priority switched by the slider 81.
[0140] 9.1 Effects The remote communication device 201 according to the third embodiment allows for a smoother transition than when switching between spatiality and selective audibility discretely. In addition, by setting a priority level between the two values, the audibility can be changed more flexibly according to the situation.
[0141] 10. Fourth Embodiment In the fourth embodiment, it is assumed that the location of devices (the relative positions of the camera 41, screen 47, and speaker 43) differs between the site X and the sites a to c.
[0142] FIG. 20 is a diagram showing the arrangement of devices at bases X and a in the fourth embodiment, FIG. 21 is a diagram showing the arrangement of devices at bases X and b in the fourth embodiment, and FIG. 22 is a diagram showing the arrangement of devices at bases X and c in the fourth embodiment.
[0143] As shown in FIGS. 20 to 22, the positions of the plurality of speakers 51, the position of the screen 47, and the positions of the plurality of speakers 43 in the plurality of partner spaces (locations a to c) are all different.
[0144] Even in such a situation, the distance d between the position of the speaker 51 and the speaker 43 appropriate for each location can be calculated by the method described in <4.3. Calculation of the distance between the position of the speaker 51 and the speaker 43>. xi It is possible to obtain
[0145] 10.1. Effects According to the remote communication device 301 according to the fourth embodiment, it is possible to flexibly arrange devices according to restrictions on the installation location.
[0146] 11. Fifth Embodiment In the fifth embodiment, a case where a remote communication device 401 is used in a one-to-N one-way connection will be described.
[0147] In a one-to-N connection, assume that video and audio from one location are being broadcast to N locations. In this case, the broadcasting party has only the transmitting unit 2 (i.e., it does not have a screen 47 or an origin), and the viewing party has only the receiving unit 3 (i.e., it does not have a camera 41 in its own space).
[0148] The origin of the broadcast location is made to coincide with the origin of the virtual screen 48 of locations 1 to N. In "Calculation of the distance between the position of the speaker 51 and the speaker 43", only "3." in <4.3. Calculation of the distance between the position of the speaker 51 and the speaker 43> is calculated, and the distance d between the position of the speaker 51 and the speaker 43 is calculated. xi are required respectively.
[0149] Fig. 23 is a diagram showing the arrangement of a camera 41, a virtual screen 48, and a speaker 51 at a broadcast location according to the fifth embodiment. Fig. 24 is a diagram showing the arrangement of origins 50 of speakers 43 and screens 47 at locations 1 to N according to the fifth embodiment.
[0150] In FIG. 23, the origin of the broadcast location coincides with the origin 49 on the virtual screen 48 of locations 1 to N.
[0151] <11.1. Effects> According to the remote communication device 401 of the fifth embodiment, in addition to one-to-one unidirectional / bidirectional connections, one-to-N unidirectional connections can also be realized using the same method of calculating the distance between the position of the speaker 51 and the speaker 43 as in the first embodiment.
[0152] Furthermore, the remote communication device 401 according to the fifth embodiment can simultaneously distribute information from broadcast locations to multiple locations. When spatial priority is given, the information can be distributed in a highly realistic manner, and when selective audibility priority is given, the information can be distributed in a manner that is easier to hear.
[0153] 12. Hardware Configuration FIG. 25 is a hardware configuration diagram showing an example of a computer 1000 that realizes the arithmetic unit of the remote communication devices 1, 101, 201, 301, and 401 that are information processing devices according to the first to fifth embodiments.
[0154] The computer 1000 includes a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected by a bus 1050.
[0155] The CPU 1100 operates and controls each component based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.
[0156] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 is started, as well as programs that depend on the hardware of the computer 1000 .
[0157] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records an application program according to the present disclosure, which is an example of program data 1450.
[0158] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.
[0159] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs and the like recorded on a predetermined recording medium. Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Discs), magneto-optical recording media such as MOs (Magneto-Optical Discs), tape media, magnetic recording media, and semiconductor memories.
[0160] Although the CPU 1100 reads and executes the program data 1450 from the HDD 1400, as another example, the CPU 1100 may obtain these programs from other devices via an external network 1550.
[0161] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical idea described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.
[0162] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.
[0163] The present technology can also be configured as follows: (1) An information processing device having a selective audibility speaker selection unit that selects multiple speakers in a remote space so that each speaker in the remote space has selective audibility. (2) The information processing device according to (1), having a spatiality speaker selection unit that selects the multiple speakers for the multiple speakers so that the spatiality is higher than that of the selected speakers. (3) The information processing device according to (2), having a speaker switching unit that switches between the speaker selected by the selective audibility speaker selection unit and the speaker selected by the spatiality speaker selection unit. (4) The information processing device according to (3), in which the speaker switching unit is a GUI. (5) The information processing device according to (1), in which the multiple speakers selected by the selective audibility speaker selection unit have spatiality. (6) The information processing device according to (1), wherein the selective audibility speaker selection unit selects the plurality of speakers in the one's own space for the plurality of speakers in the other party space based on positions of the plurality of speakers in the other party space and positions of the plurality of speakers in the one's own space when the number of the plurality of speakers in the other party space is equal to or less than the number of the plurality of speakers in the one's own space. (7) The information processing device according to (6), further comprising a coordinate transformation unit that transforms the positions of the plurality of speakers in the other party space into positions of the plurality of speakers in the one's own space, wherein the selective audibility speaker selection unit selects the plurality of speakers in the one's own space for the plurality of speakers in the other party space based on the positions of the plurality of speakers in the one's own space transformed by the coordinate transformation unit. (8) The information processing device according to (6), wherein the selective audibility speaker selection unit calculates dx m The information processing apparatus according to (7), further comprising: selecting a combination of the plurality of speakers and the speaker that minimizes where n p : Number of speakers n s : Number of speakers m: Combination of speaker's position in self-space and speakers in self-space : The sum of the speaker position and the distance between the speakers in the m-th combination dx m: represents the distance between the xth speaker and the corresponding speaker (9) The information processing device according to (1), wherein the selective audibility speaker selection unit, when the number of the plurality of speakers in the partner space is greater than the number of the plurality of speakers in the own space, selects the plurality of speakers in the own space for the plurality of speakers from among the plurality of speakers in the partner space who have a high priority over the plurality of speakers in the own space based on the positions of the plurality of speakers in the partner space and the positions of the plurality of speakers in the own space. (10) The information processing device according to (9), wherein the selective audibility speaker selection unit selects a speaker from among the plurality of speakers in the partner space other than the plurality of speakers with the high priority as one of the plurality of speakers in the own space. (11) The information processing device according to (1), further comprising a determination unit that determines whether the number of the plurality of speakers in the partner space is equal to or less than the number of the plurality of speakers in the own space. (12) The information processing device according to (1), wherein the selective audibility is the ease with which speeches of the multiple speakers can be distinguished. (13) The information processing device according to (2), wherein the spatiality is the accuracy of arrangement of the multiple speakers in the self space relative to the multiple speakers in the self space. (14) The information processing device according to (9), wherein the multiple speakers in the self space include an environmental sound speaker that plays environmental sound in the other person's space, and the selective audibility speaker selector selects, as the environmental sound speaker, a speaker from among the multiple speakers in the other person's space other than the multiple speakers having the highest priority. (15) The information processing device according to (1), comprising: a slider display unit that displays a slider that continuously switches the priority between the selective visibility and the spatiality; and a volume adjustment unit that adjusts a volume of the selective visibility of the multiple speakers in the self space and a volume of the spatiality in accordance with the priority switched by the slider. (16) The information processing device according to (7), wherein the partner space is a plurality of spaces, and the positions of the plurality of speakers and the positions of the plurality of speakers in the plurality of partner spaces are different from each other.(17) The information processing device according to (1), wherein there are multiple partner spaces, and the origins of the multiple partner spaces are the origin in the self space. (18) An information processing method for an information processing device having a selective audibility speaker selection unit that selects multiple speakers in the partner space so that each of the multiple speakers has selective audibility for a multiple speaker in the self space, wherein the selective audibility speaker selection unit determines whether the number of the multiple speakers in the partner space is equal to or less than the number of the multiple speakers in the self space, and if the number of the multiple speakers in the partner space is equal to or less than the number of the multiple speakers in the self space, selects the multiple speakers in the self space for the multiple speakers in the partner space based on the positions of the multiple speakers in the partner space and the positions of the multiple speakers in the self space.
[0164] 1, 101, 201, 301, 401 Remote communication device 2 Transmission unit 3 Receiving unit 10 Input unit 11 Person position detection unit 12, 112 Individual sound generation unit 13, 113 Transmission unit 21, 121 Receiving unit 22 Voice-person position mapping unit 23, 123 Speaker selection unit 24, 124, 224 Spatiality and selective audibility adjustment unit 25 Output unit 41 Camera 42 Microphone 43 Speaker 44 Audio interface 45 Network 46 Display 47 Screen 48 Virtual screen 49, 50 Origin 51 Speaker 60 Changeover switch 61 Spatiality selection button 62 Selective audibility button 71 Environmental sound pickup microphone 72 Environmental sound speaker 81 Slider
Claims
1. An information processing device having a selective audibility speaker selection unit that selects a plurality of speakers in a partner's space so as to have selective audibility for a plurality of speakers in a self-space.
2. The information processing device according to claim 1, further comprising: a spatiality speaker selection unit that selects the plurality of speakers for the plurality of speakers so as to achieve a spatiality higher than that of the selected speakers.
3. An information processing device according to claim 2, further comprising a speaker switching unit that switches between the speaker selected by the selective audibility speaker selection unit and the speaker selected by the spatiality speaker selection unit.
4. The information processing device according to claim 3, wherein the speaker switching unit is a GUI.
5. The information processing device according to claim 1, wherein the plurality of speakers selected by the selective audibility speaker selection unit have spatiality.
6. The information processing device of claim 1, wherein the selective audibility speaker selection unit selects the multiple speakers in the self space for the multiple speakers in the other space based on the positions of the multiple speakers in the other space and the positions of the multiple speakers in the self space when the number of the multiple speakers in the other space is equal to or less than the number of the multiple speakers in the self space.
7. An information processing device as described in claim 6, further comprising a coordinate transformation unit that transforms the positions of the multiple speakers in the other person's space into the positions of the multiple speakers in the self space, and the selective audibility speaker selection unit selects the multiple speakers in the self space for the multiple speakers in the other person's space based on the positions of the multiple speakers in the self space transformed by the coordinate transformation unit and the positions of the multiple speakers in the self space.
8. The selective audibility speaker selection unit calculates dx based on the following formula: m The information processing apparatus according to claim 7 , further comprising: selecting a combination of the plurality of speakers and the speaker that minimizes Here, n p : Number of speakers n s : Number of speakers m: Combination of speaker's position in self space and speakers in self space : The sum of the speaker position and the distance between the speakers in the m-th combination dx m : represents the distance between the xth speaker and the corresponding speaker 9. The information processing device of claim 1, wherein the selective audibility speaker selection unit, when the number of the multiple speakers in the other space is greater than the number of the multiple speakers in the self space, selects the multiple speakers in the self space for the multiple speakers among the multiple speakers in the other space who have a higher priority than the multiple speakers in the self space based on the positions of the multiple speakers in the other space and the positions of the multiple speakers in the self space.
10. The information processing device according to claim 9, wherein the selective audibility speaker selection unit selects a speaker among the plurality of speakers in the other space other than the plurality of speakers having the highest priority as one of the plurality of speakers in the self space.
11. The information processing device according to claim 1, further comprising a determination unit that determines whether the number of the plurality of speakers in the other person's space is equal to or less than the number of the plurality of speakers in the self space.
12. The information processing device according to claim 1, wherein the selective audibility is the ease of distinguishing speech from the plurality of speakers.
13. The information processing device according to claim 2, wherein the spatiality is a positioning accuracy of the multiple speakers in the self-space relative to the multiple speakers in the self-space.
14. An information processing device as described in claim 9, wherein the multiple speakers in the self space include an environmental sound speaker that plays environmental sound in the other space, and the selective audibility speaker selection unit selects a speaker from among the multiple speakers in the other space other than the multiple speakers with the highest priority as the environmental sound speaker.
15. An information processing device as described in claim 2, further comprising: a slider display unit that displays a slider that continuously switches the priority between the selective audibility and the spatiality; and a volume adjustment unit that adjusts the volume of the selective audibility and the volume of the spatiality of the multiple speakers in the personal space according to the priority switched by the slider.
16. The information processing device according to claim 7, wherein the target space is a plurality of spaces, and the positions of the plurality of speakers and the positions of the plurality of speakers in the plurality of target spaces are different from each other.
17. The information processing device according to claim 1, wherein the remote space is multiple, and the origins of the multiple remote spaces are the origin of the self space.
18. An information processing method in an information processing device having a selective audibility speaker selection unit that selects multiple speakers in the other space so that each of the multiple speakers has selective audibility for multiple speakers in the own space, wherein the selective audibility speaker selection unit determines whether the number of the multiple speakers in the other space is less than or equal to the number of the multiple speakers in the own space, and if the number of the multiple speakers in the other space is less than or equal to the number of the multiple speakers in the own space, selects the multiple speakers in the own space for the multiple speakers in the other space based on the positions of the multiple speakers in the other space and the positions of the multiple speakers in the own space.
Citation Information
Patent Citations
Video display device
JP2010157906A
Information processing apparatus, conference system, and method for controlling information processing apparatus
JP2017158134A
Data processing apparatus, data processing method and program
JP2019220848A
Controlling focus of audio signals on speaker during videoconference
US20190222804A1
Cited By
Multi-monitor sound mapping system
US20260304040A1
