Sound processing server

Abstract

To provide a sound processing server capable of performing a sound processing so that a sound from a plurality of devices is not overlapped.SOLUTION: In a sound processing server 100, a grouping processing part 102 selects some AR terminal on the basis of a predetermined condition from an AR terminal as a plurality of user terminals, and forms a group. Then, a selection part 103 selects one representative AR terminal from some AR terminal in the group to be formed. A sound processing part 103 performs a processing for sharing a sound from the AR terminal of the representative speaker that becomes a representation to the other AR terminal and a VR terminal. That is, the sound processing part 103 performs a speaker mute processing to the sound from the terminal other than the AR terminal selected as a representative speaker.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a voice processing server. [Background technology]

[0002] Patent Document 1 describes a device that displays an image on the display of a head mounted display (HMD) worn by a user based on augmented reality space information and outputs sound from a speaker. This device collects the voice emitted by the user and surrounding sounds with a microphone and transmits them to a server. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-1847986 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the device described in Patent Document 1, if there is another device nearby, it is possible that each device will pick up the voice of the other user. In that case, the voice will be transmitted to the other user via the server, causing the voices to overlap and making it difficult to hear the conversation.

[0005] Therefore, an object of the present invention is to provide a voice processing server capable of performing voice processing so that voices from a plurality of devices do not overlap. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention comprises a grouping processing unit that selects several user terminals from a plurality of user terminals based on predetermined conditions to form a group, a selection unit that selects a representative user terminal from several user terminals in the formed group, and an audio processing unit that shares audio from the representative user terminal with other user terminals. Effect of the Invention

[0007] According to the present invention, overlapping voices are not transmitted to the terminals, and easy-to-listen voices can be shared. [Brief description of the drawings]

[0008] [Figure 1] 1A to 1C are diagrams showing images of a game as viewed from an AR terminal and a VR terminal in the present disclosure. [Diagram 2] FIG. 2 is a diagram illustrating an overview of processing by a voice processing server according to the present disclosure. [Diagram 3] 1 is a diagram showing a system configuration including a voice processing server 100 according to the present disclosure. [Figure 4] 1 is a block diagram showing a functional configuration of a speech processing server 100 according to the present disclosure. [Diagram 5] 4 is a flowchart showing the operation of the voice processing server 100 of the present disclosure. [Figure 6] FIG. 13 is a schematic diagram of grouping. [Figure 7] 1 is a diagram illustrating an example of a hardware configuration of a voice processing server 100 according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The embodiments of the present disclosure will be described with reference to the accompanying drawings. Whenever possible, the same parts are designated by the same reference numerals and redundant description will be omitted.

[0010] In the present disclosure, the same spatial situation is provided for both the AR terminal and the VR terminal. For example, the same game space is provided for the AR terminal and the VR terminal. This game space is a space based on an augmented reality space for the AR terminal, and a space based on a virtual space for the VR terminal. Users of each terminal can play games in the same game space.

[0011] FIG. 1 is a diagram showing images of a game as viewed from an AR terminal and a VR terminal in the present disclosure. FIG. 1(a) shows an image as viewed from an AR terminal. An AR terminal is a terminal for providing a user with a space based on augmented reality (AR). In general, an AR terminal provides a space in which an image is synthesized with an image of the real world. In general, a VR terminal is a goggle-type terminal that provides a virtual space to a user, but is not limited thereto, and an image of the virtual space may be provided by a smartphone, a personal computer, or the like.

[0012] FIG. 1(a) shows a screen G in the user's field of view. In this screen, an image G1 is synthesized with an image of the real world. Users U1 and U2 indicate other users who are actually present. User U3 indicates a user of a VR terminal. In this disclosure, a game is played in an attraction at an amusement park, and an image G1 resembling a monster is synthesized with the attraction in the real world. The user uses the terminal he or she owns to play a game in which he or she fights the monster.

[0013] FIG. 1(b) shows the screen of a VR terminal. As shown in the figure, the screen G2 representing the virtual space is prepared with an image of the attraction in the real space. That is, the attraction in the real space is actually photographed, and the screen G2 is created based on the image. In the screen G2, the users U1 and U2 are represented by icons, and their positional relationship also conforms to the positions of the actual users. The image G1 in FIG. 1(b) is the same content as the image G1 shown in FIG. 1(a), and is superimposed on the screen G2.

[0014] Although a game space is shown as an example in Fig. 1, the present invention is not limited to this. For example, the present invention can be applied to cases where a user of an AR terminal (avatar) and a user of a VR terminal (avatar) are in the same virtual space, such as in an office, field work, or extracurricular activities.

[0015] Also, in FIG. 1, the avatar is shown to have a non-human shape that resembles the user, but of course this is not limited to this and it may be an animal shape, a shape without arms or legs, or another inorganic shape (such as a sphere or cube).

[0016] As shown in Figure 1, there are cases where users of AR devices are close to each other. In such cases, the voices of each user may be picked up by the other user's AR device. This will be explained below with reference to the figure.

[0017] FIG. 2 is a diagram showing an overview of processing by a voice processing server in the present disclosure. In FIG. 2(a), there are AR user A and AR user B on the AR side. Also, there is a VR user on the VR side. In the present disclosure, the AR side refers to the range in which a user acts in augmented reality (AR). The AR user is a user who operates an AR terminal. The VR side refers to the range in which a user acts in a virtual space configured by virtual reality. The VR user refers to a user who operates a VR terminal.

[0018] 2(a), the AR terminal 12 of the user U2 picks up what the user U1 says (voice) to the AR terminal 11. The AR terminals 11 and 12 transmit this voice to the voice processing server 100.

[0019] The voice processing server 100 acquires voice from each of the AR terminals 11 and 12 , and transmits it to the VR terminal 13 .

[0020] At the VR terminal 13, the audio from the AR terminal 11 and the AR terminal 12 is played back, but at this time, the audio picked up by each terminal is also being transmitted, making the audio playback difficult to hear.

[0021] Therefore, the audio processing server 100 performs audio muting processing.

[0022] 2(b) is a diagram showing an overview of the process. In the diagram, the voice processing server 100 mutes voice from an arbitrary terminal among voices acquired from the AR terminal 11 and the AR terminal 12. In other words, the voice processing server 100 transmits only the voice from one of the AR terminals 11 and 12 to the VR terminal 13. As a result, the VR terminal 13 can play back voice that is easy to hear.

[0023] In addition, in FIG. 2, an example in which the voice processing server 100 transmits voice to the VR terminal 13 is shown, but this is not limiting, and the voice processing server 100 may also transmit voice to other AR terminals such as the AR terminal 12.

[0024] 3 is a diagram showing a system configuration including the voice processing server 100 of the present disclosure. As shown in the figure, this system includes the voice processing server 100, AR terminals 11 and 12, and a VR terminal 13. In the present disclosure, the AR terminals 11 and 12 are located in the same space K in the real world (for example, a predetermined area in an attraction), and are in a positional relationship in which the AR terminals 11 and 12 pick up each other's voices.

[0025] Then, the voice acquired by the AR terminal 11 or the like is transmitted to the VR terminal 13 or the like via the voice processing server 100.

[0026] Similarly, the voice acquired by the VR terminal 13 is transmitted to the AR terminal 11 and the like via the voice processing server 100.

[0027] As described above, the voice processing server 100 controls so that the voice of either one of the AR terminals 11 and 12 is muted and not transmitted.

[0028] 4 is a block diagram showing a functional configuration of the voice processing server 100 of the present disclosure. As shown in the figure, the voice processing server 100 includes a communication unit 101, a grouping processing unit 102, a selection unit 103, a voice processing unit 104, and a virtual space processing unit 105.

[0029] The communication unit 101 is a part that communicates with the AR terminals 11 and 12 and the VR terminal 13. In the present disclosure, the communication unit 101 is a part that acquires audio from the AR terminals 11 and 12.

[0030] The grouping processing unit 102 is a part that classifies the AR terminal into one or several groups based on the position of the AR terminal. The position of the AR terminal may be grasped by, for example, GPS, or if it is a specific location such as an amusement park, it may be grasped by a position specified by short-range wireless communication such as a wireless tag. The position of the AR terminal may also be specified by an estimated distance from a base station of a mobile communication network, LPWAAR. This position information is acquired from the AR terminal via the communication unit 101. Note that the grouping processing unit 102 may also classify the AR terminal into groups including VR terminals, as described later.

[0031] The selection unit 103 is a part that selects an AR terminal as a representative speaker from among the grouped AR terminals that satisfy a predetermined condition, such as a terminal that is speaking.

[0032] The audio processing unit 104 is a part that processes audio to be shared among terminals in the same group, and also processes audio not to be output from AR terminals in a specific positional relationship within the group. For example, it is a part that processes only audio from a selected AR terminal (e.g., AR terminal 11) to the VR terminal 13, and does not transmit audio from other AR terminals (e.g., AR terminal 12) to the VR terminal 13.

[0033] The virtual space processing unit 105 is a part that provides a virtual space to the AR terminals 11 and 12 and the VR terminal 13. In the present disclosure, an augmented reality space (a space in which an image is synthesized with the real world) is provided to the AR terminals 11 and 12, and a virtual space is provided to the VR space.

[0034] In the present disclosure, the virtual space processing unit 105 identifies the position of the AR terminal by a Visual Positioning Service (VPS) (not shown) and overlays content on the real space of the AR terminal. Known services for this overlay include Immersal and Google Geospatial.

[0035] In addition, in the present disclosure, for example, the virtual space processing unit 105 uses Immersal to identify AR terminals, and the virtual space processing unit 105 uses a synchronization engine (such as Photon) to share position information between the AR terminals and constantly express them as avatars. Since the virtual space processing unit 105 manages position information received from a VPS (not shown) using the synchronization engine, the virtual space processing unit 105 acquires position information via the API of the synchronization engine, and the grouping processing unit 102 performs grouping processing. Note that the same position synchronization processing is also performed for VR terminals.

[0036] FIG. 5 is a flowchart showing the operation of the voice processing server 100 of the present disclosure.

[0037] In the voice processing server 100, the communication unit 101 establishes a communication connection between a plurality of terminals including an AR terminal (S101). In the present disclosure, as shown in Fig. 1 and Fig. 2, a space based on a virtual space and an augmented reality space (e.g., a game space) is shared between the AR terminals 11 and 12 and the VR terminal 13. The communication unit 101 identifies the terminal type of the AR terminal or the VR terminal when establishing a communication connection, and the virtual space processing unit 105 provides each terminal with an image of the virtual space or the augmented reality space according to the respective terminal types.

[0038] In the present disclosure, when each user connects to (logs in to) a game, the user (or avatar) is placed in a game space (virtual space or augmented reality space). At this time, as described above, the user or avatar is given a position in the virtual space and the augmented reality space based on position information obtained by the VPS or the like. The position in the augmented reality space may indicate a position in the actual attraction area (obtained by GPS or the like), and the position in the virtual space may indicate a position in a space simulating the attraction area.

[0039] FIG. 6(a) shows grouping at the time of login. The grouping processing unit 102 performs grouping for sharing audio based on position information in the virtual space, separately from the group that is the target of processing for not outputting audio, which will be described later. Range H indicates the group that shares audio. This range H indicates a range larger than group G that is the target of processing for not outputting audio, which will be described later, but it may be the same size. This range H is, for example, a circle with a radius of about 2 m, and indicates the range in which users can talk to each other. In FIG. 6(a), AR user U4 and VR user U5 are outside range H, so the audio of users U1 to U3 is not shared.

[0040] The grouping processing unit 102 may set groups based on the position information of the VR user or based on the AR user who is closest to the position information of the VR user. This is because, when grouping is performed in consideration of the range in which users can communicate with each other by voice (to achieve more realistic communication), the VR users should be grouped based on the microphone of the AR user who is closest to the VR user's position in order to share audio situations according to their positions.

[0041] The grouping processing unit 102 judges whether there are two or more AR terminals in the range H based on the information identified at the time of communication connection (S102). Then, the grouping processing unit 102 lists all AR users (S103) and selects an arbitrary AR terminal (for example, the top of the list) (S104). Then, the grouping processing unit 102 extracts and groups AR terminals within a radius xm around the selected AR terminal (S105). This xm is, for example, 1m. It is preferable that the range is such that conversations can be heard. Here, the grouping is based on position information in the actual real world, but may also be based on positions in the virtual space. The positional relationship is the same in the virtual space.

[0042] 6(b) shows grouping of AR terminals. As described above, group G is a group for preventing a voice from being output from one AR terminal to the other terminals when the AR terminals pick up each other's voices.

[0043] The grouping processing unit 102 determines whether there are two or more users in the group (S106). If it is determined that there are two or more users, the grouping processing unit 102 determines who is speaking among the users in the group by detecting silence (S107). Then, the selection unit 103 selects a representative speaker (speaker) and mutes the speech of the other users (S108). Here, the selection unit 103 selects the AR terminal of the representative speaker, and the audio processing unit 104 performs processing to mute audio from the other AR terminals.

[0044] In process S106, when the grouping processing unit 102 determines that there are not two or more users, the audio processing unit 104 cancels the function of not outputting audio to users in the group if the audio processing unit 104 has already executed the function of not outputting audio from a specific terminal (S109). If the audio processing unit 104 has not executed the function, it does nothing.

[0045] Then, the grouping processing unit 102 removes the users in the group from the list (S110). This is to exclude the users in the group who have executed the function that does not output audio from the processing targets.

[0046] The grouping processing unit 102 and the selection unit 103 repeatedly execute the processes S104 to S110 until the AR user list is empty, that is, until a representative speaker is selected for each group and a user who executes a function that does not output sound is selected (S111). Then, when the AR user list becomes empty, the selection process of a target user who executes a function that does not output sound is temporarily suspended (S112). When a predetermined time has elapsed (or when a predetermined condition is satisfied), the grouping processing unit 102 and the selection unit 103 perform the grouping process again, regroups, and performs processes such as the selection of a representative speaker, etc. An example of the predetermined condition here is a case where a user is replaced, such as a new user logging in or an existing user logging out.

[0047] In the above description, in process S101, grouping is performed for sharing audio, but this is not necessary.

[0048] Next, the effect of the voice processing server 100 of the present disclosure will be described. In the voice processing server 100, the grouping processing unit 102 selects some AR terminals based on a predetermined condition from AR terminals, which are multiple user terminals, to form a group. Then, the selection unit 103 selects one representative AR terminal from some AR terminals in the formed group. The voice processing unit 104 processes the voice from the AR terminal of the representative speaker to be shared with other AR terminals and VR terminals. That is, the voice processing unit 104 executes a function of not outputting voice for voices from terminals other than the AR terminal selected as the representative speaker.

[0049] According to this configuration, overlapping audio is not transmitted to the terminals, and easy-to-listen audio can be shared.

[0050] Furthermore, at the time of login, the grouping processing unit 102 performs grouping based on the range H for sharing audio for AR terminals and VR terminals, and the audio processing unit 104 performs audio sharing based on that range H.

[0051] Furthermore, the grouping processing unit 102 selects several AR terminals based on the position information of the multiple AR terminals. This position information is information acquired from the AR terminals when a communication connection is established.

[0052] Furthermore, in the speech processing server 100 of the present disclosure, the selection unit 103 selects, from among several AR terminals, a user terminal that has uttered a speech as the AR terminal of the representative speaker.

[0053] In the speech processing server 100 according to the present disclosure, the selection unit 103 may update the user terminals belonging to the group for each predetermined condition. The predetermined condition is determined based on the passage of a predetermined time or other conditions.

[0054] This allows the AR terminal of the user who is speaking to be given priority, thereby improving voice quality.

[0055] In addition, in the present disclosure, the grouping processing unit 102 of the audio processing server 100 selects VR terminals corresponding to other user terminals based on the virtual position in the virtual space of the representative AR terminal, and groups the VR terminals in addition to the AR terminals that are other user terminals based on the virtual position in the virtual space. Then, the audio processing unit 104 shares audio with the AR terminals and VR terminals in the same group. The audio processing unit 104 may not share audio with terminals that are not in the same group.

[0056] This prevents voice from being shared with user terminals that are too far away to have a conversation even though they are in the same area, and therefore allows conversation to be held at an appropriate distance.

[0057] The voice processing server 100 of the present invention has the following configuration.

[0058] [1] a grouping processing unit that selects some user terminals from a plurality of user terminals based on a predetermined condition and forms a group; A selection unit that selects one representative user terminal from among several user terminals in the formed group; a voice processing unit that shares voice from the representative user terminal with other user terminals; A voice processing server comprising:

[0059] [2] The audio processing unit includes: among user terminals other than the representative user terminal, not outputting audio from user terminals that are in the same real space as the representative user terminal; [1] A voice processing server according to the present invention.

[0060] [3] The grouping processing unit selects several user terminals based on location information of the plurality of user terminals. [1] A voice processing server according to the present invention.

[0061] [4] The selection unit is selecting a user terminal that has uttered a voice as a representative user terminal from among the several user terminals; [1] A voice processing server according to the present invention.

[0062] [5] The selection unit updates the user terminals belonging to the group for each predetermined condition. A voice processing server according to any one of [1] to [4].

[0063] [6] The plurality of user terminals include an AR terminal that synthesizes a virtual image in a real world and provides the user with the virtual image, and a VR terminal that provides the user with a virtual space including the virtual image, the grouping processing unit forms a group with the AR terminal as the plurality of user terminals; The audio processing unit performs a process of sharing audio from one AR terminal that is the representative user terminal with the AR terminals and / or the VR terminals that correspond to the other user terminals. A voice processing server according to any one of [1] to [5].

[0064] [7] The grouping processing unit: Group AR and VR devices based on the range for sharing audio, Based on the range, audio sharing is performed based on the representative user terminal. [6] A voice processing server according to the present invention.

[0065] The block diagrams used in the description of the above embodiments show functional blocks. These functional blocks (components) are realized by any combination of at least one of hardware and software. The method of realizing each functional block is not particularly limited. That is, each functional block may be realized by using one device that is physically or logically combined, or may be realized by using two or more devices that are physically or logically separated and directly or indirectly connected (for example, by wire, wireless, etc.). The functional block may be realized by combining the one device or the multiple devices with software.

[0066] Functions include, but are not limited to, judgment, determination, judgment, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, consideration, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocation, mapping, assignment, etc. For example, a functional block (component) that performs the function of transmission is called a transmitting unit or a transmitter. In either case, as described above, there is no particular limitation on the method of realization.

[0067] For example, the voice processing server 100 according to an embodiment of the present disclosure may function as a computer that performs processing of the voice processing method according to the present disclosure. Fig. 7 is a diagram showing an example of a hardware configuration of the voice processing server 100 according to an embodiment of the present disclosure. The above-mentioned voice processing server 100 may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

[0068] In the following description, the term "apparatus" may be replaced with a circuit, a device, a unit, etc. The hardware configuration of the voice processing server 100 may be configured to include one or more of the apparatuses shown in the drawings, or may be configured to exclude some of the apparatuses.

[0069] Each function in the voice processing server 100 is realized by loading a specific software (program) onto hardware such as a processor 1001 and a memory 1002, causing the processor 1001 to perform calculations, control communications via a communication device 1004, and control at least one of the reading and writing of data in the memory 1002 and the storage 1003.

[0070] The processor 1001 controls the entire computer by running, for example, an operating system. The processor 1001 may be configured with a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, etc. For example, the above-mentioned grouping processing unit 102, selection unit 103, audio processing unit 104, etc. may be realized by the processor 1001.

[0071] Moreover, the processor 1001 reads out a program (program code), a software module, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002, and executes various processes according to the read out programs. As the program, a program that causes a computer to execute at least a part of the operations described in the above-mentioned embodiment is used. For example, the grouping processing unit 102 may be realized by a control program stored in the memory 1002 and running on the processor 1001, and other functional blocks may be similarly realized. Although the above-mentioned various processes have been described as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may be transmitted from a network via a telecommunication line.

[0072] The memory 1002 is a computer-readable recording medium, and may be configured by at least one of, for example, a Read Only Memory (ROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), a Random Access Memory (RAM), etc. The memory 1002 may be called a register, a cache, a main memory (primary storage device), etc. The memory 1002 can store a program (program code), software modules, etc. that are executable to implement a voice processing method according to one embodiment of the present disclosure.

[0073] Storage 1003 is a computer-readable recording medium, and may be composed of at least one of, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, and the like. Storage 1003 may be referred to as an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, a server, or other suitable medium including at least one of memory 1002 and storage 1003.

[0074] The communication device 1004 is hardware (transmission / reception device) for performing communication between computers via at least one of a wired network and a wireless network, and is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 1004 may be configured to include a high-frequency switch, a duplexer, a filter, a frequency synthesizer, etc., in order to realize at least one of, for example, Frequency Division Duplex (FDD) and Time Division Duplex (TDD). For example, the above-mentioned communication unit 101, etc. may be realized by the communication device 1004. The communication unit 101 may be implemented as a transmission unit and a reception unit that are physically or logically separated.

[0075] The input device 1005 is an input device (e.g., a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that accepts input from the outside. The output device 1006 is an output device (e.g., a display, a speaker, an LED lamp, etc.) that performs output to the outside. The input device 1005 and the output device 1006 may be integrated into one device (e.g., a touch panel).

[0076] In addition, each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or may be configured using different buses between each device.

[0077] The voice processing server 100 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc., and some or all of the functional blocks may be realized by the hardware. For example, the processor 1001 may be implemented using at least one of these pieces of hardware.

[0078] The notification of information is not limited to the aspects / embodiments described in the present disclosure, and may be performed using other methods. For example, the notification of information may be performed by physical layer signaling (e.g., Downlink Control Information (DCI), Uplink Control Information (UCI)), higher layer signaling (e.g., Radio Resource Control (RRC) signaling, Medium Access Control (MAC) signaling, broadcast information (Master Information Block (MIB), System Information Block (SIB))), other signals, or a combination thereof. In addition, the RRC signaling may be called an RRC message, and may be, for example, an RRC Connection Setup message, an RRC Connection Reconfiguration message, or the like.

[0079] The order of the steps, sequences, flow charts, etc. of each aspect / embodiment described in this disclosure may be changed unless inconsistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0080] The input and output information may be stored in a specific location (e.g., memory) or may be managed using a management table. The input and output information may be overwritten, updated, or added to. The output information may be deleted. The input information may be transmitted to another device.

[0081] The determination may be based on a value represented by a single bit (0 or 1), a Boolean (true or false) value, or a numerical comparison (e.g., comparison with a predetermined value).

[0082] Each aspect / embodiment described in this disclosure may be used alone, in combination, or switched according to execution. In addition, notification of specific information (e.g., notification that "X is the case") is not limited to being done explicitly, but may be done implicitly (e.g., not notifying the specific information).

[0083] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described in the present disclosure. The present disclosure can be implemented as modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure.

[0084] Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0085] Software, instructions, information, etc. may also be transmitted or received over a transmission medium. For example, if the software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave, etc.), then these wired and / or wireless technologies are included within the definition of transmission media.

[0086] The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0087] In addition, the terms described in this disclosure and the terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings. For example, at least one of the channel and the symbol may be a signal (signaling). Also, the signal may be a message. Also, the component carrier (CC) may be called a carrier frequency, a cell, a frequency carrier, etc.

[0088] In addition, the information, parameters, etc. described in the present disclosure may be represented using absolute values, may be represented using relative values ​​from a predetermined value, or may be represented using other corresponding information. For example, a radio resource may be indicated by an index.

[0089] The names used for the above-mentioned parameters are not limiting in any way. Moreover, the formulas etc. using these parameters may differ from those explicitly disclosed in this disclosure. The various channels (e.g., PUCCH, PDCCH, etc.) and information elements may be identified by any suitable names, and therefore the various names assigned to these various channels and information elements are not limiting in any way.

[0090] In this disclosure, the terms "Mobile Station (MS)", "user terminal", "User Equipment (UE)", "terminal", etc. may be used interchangeably.

[0091] A mobile station may also be referred to by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable terminology.

[0092] As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, search, inquiry (e.g., searching in a table, database, or other data structure), ascertaining, and the like. "Determining" and "determining" may also include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in a memory), and the like. In addition, "judgment" and "decision" can include considering resolving, selecting, choosing, establishing, comparing, etc., to be a "judgment" or "decision." In other words, "judgment" and "decision" can include considering some action to be a "judgment" or "decision." In addition, "judgment" can be interpreted as "assuming," "expecting," "considering," etc.

[0093] The terms "connected," "coupled," or any variation thereof, refer to any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, "connected" may be read as "access." As used in this disclosure, two elements may be considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and optical (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0094] As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0095] Any reference to an element using a designation such as "first", "second" etc. used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed or that the first element must precede the second element in some way.

[0096] When "include," "including," and variations thereof are used in this disclosure, these terms are intended to be inclusive, similar to the term "comprising." Further, when used in this disclosure, the term "or" is not intended to be an exclusive or.

[0097] In this disclosure, where articles have been added due to translation, such as, for example, a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural form.

[0098] In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "coupled" may also be interpreted in the same way as "different." [Explanation of symbols]

[0099] 11: AR terminal, 12: AR terminal, 13: VR terminal, 100: audio processing server, 101: communication unit, 102: grouping processing unit, 103: selection unit, 104: audio processing unit, 105: virtual space processing unit.

Claims

1. a grouping processing unit that selects some user terminals from a plurality of user terminals based on a predetermined condition and forms a group; A selection unit that selects one representative user terminal from among several user terminals in the formed group; a voice processing unit that shares voice from the representative user terminal with other user terminals; A voice processing server comprising:

2. The audio processing unit includes: among user terminals other than the representative user terminal, not outputting audio from user terminals that are in the same real space as the representative user terminal; The voice processing server according to claim 1 .

3. The grouping processing unit selects several user terminals based on location information of the plurality of user terminals. The voice processing server according to claim 1 .

4. The selection unit is selecting a user terminal that has uttered a voice as a representative user terminal from among the several user terminals; The voice processing server according to claim 1 .

5. The selection unit updates the user terminals belonging to the group for each predetermined condition. The voice processing server according to claim 1 .

6. The plurality of user terminals include an AR terminal that synthesizes a virtual image into a real world and provides the user with the virtual image, and a VR terminal that provides the user with a virtual space including the virtual image, the grouping processing unit forms a group with the AR terminal as the plurality of user terminals; the audio processing unit performs a process of sharing audio from one AR terminal that is the representative user terminal with the AR terminals and / or the VR terminals that correspond to the other user terminals; The voice processing server according to claim 1 .

7. The grouping processing unit: Grouping the AR and VR devices based on the range for sharing audio; Based on the range, audio sharing is performed based on the representative user terminal. The voice processing server according to claim 6.

Images