Analog chorus type audio noisy sound

Abstract

Systems, methods, and computer-readable storage devices for simulating chorus audio noisy in a communication system are disclosed. A method includes receiving audio data from each of a plurality of users participating in a first group of a plurality of groups for an event using a communication system; generating a first simulated chorus audio noisy sound based on the audio data received from each of the plurality of users in the first group; and providing the generated first simulated chorus audio data to at least one user of the plurality of users of the event.

Description

Technical Field

[0001] The present disclosure is particularly related to a communication system that provides audio and / or video that provides feedback to a group such as a conference with breakout rooms, including audio feedback, laughter, applause, or other aggregated feedback. Specifically, the present disclosure is related to a communication system that provides simulated choral audio noise in a conference breakout application. Background Art

[0002] A communication system may allow multiple users to interact and / or collaborate during a conference. For example, some communication systems allow people to collaborate using real-time video streams, real-time audio streams, and / or other forms of text-based or image-based media. Users of a communication session of a communication system may share a video stream and / or audio stream provided to multiple users.

[0003] During a conference, an administrator or moderator hosting a communication session of the communication system can dynamically create additional groups to form a network of groups for the event or conference. These additional groups can be multiple side conferences and breakout sessions.

[0004] The ability to easily create group rooms and assign individuals to group rooms provides a seamless experience for joining group rooms. An administrator or host can bring group room users back to the main room. However, when users are in side conferences and / or group rooms, the administrator or host may have restrictive feedback from the side conference, group conference, and / or users. In addition, the noise suppression component used by the communication system may suppress audio feedback for administrators, hosts, and / or users.

[0005] These shortcomings result in less than optimal interactions between administrators or moderators and users. In addition, such shortcomings of existing communication systems may result in a loss of user engagement. The loss of user engagement may result in production losses and inefficiencies with respect to multiple computing resources. For example, when a user becomes tired or unengaged, the user may need to refer to a record or other resource. When a viewer misses a significant point or clue during a live conference, the missed content may need to be resent. Viewers may also have to re-watch content when they miss a significant point or non-verbal social clue during a viewing record's presentation. This behavior may result in inefficient use of human resources, networks, processors, memory, or other computing resources. Therefore, there is a continued need to develop improvements to help make the user experience more like being in a meeting and more engaging. Summary of the invention

[0006] According to certain embodiments, systems, methods, and computer-readable media are disclosed for simulating choral audio noise.

[0007] According to certain embodiments, a computer-implemented method for simulating a choral audio noise in a communication system is disclosed. A method includes: receiving audio data from each of a plurality of users of a first group of a plurality of groups participating in an event using the communication system; generating a first simulated choral audio noise based on the audio data received from each of the plurality of users in the first group; and providing the generated first simulated choral audio data to at least one user of the plurality of users of the event.

[0008] According to certain embodiments, a system for simulating choral audio noise in a communication system is disclosed. A system includes: a data storage device storing instructions for simulating choral audio noise in a communication system; and a processor configured to execute the instructions to perform a method including: receiving audio data from each of a plurality of users of a first group of a plurality of groups participating in an event using the communication system; generating a first simulated choral audio noise based on the audio data received from each of the plurality of users in the first group; and providing the generated first simulated choral audio data to at least one of the plurality of users of the event.

[0009] According to a specific embodiment, a computer-readable storage device is disclosed, which stores instructions that, when executed by a computer, cause the computer to perform a method for simulating choral audio noise in a communication system. A method of the computer-readable storage device includes: receiving audio data from each of a plurality of users of a first group of a plurality of groups participating in an event using the communication system; generating a first simulated choral audio noise based on the audio data received from each of the plurality of users in the first group; and providing the generated first simulated choral audio data to at least one user of the plurality of users of the event.

[0010] Additional objects and advantages of the disclosed embodiments will be set forth in part in the following description, and in part will be apparent from the description, or may be learned through the practice of the disclosed embodiments. The objects and advantages of the disclosed embodiments will be realized and obtained by means of the elements and combinations particularly pointed out in the appended claims.

[0011] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In the course of the following detailed description, reference will be made to the accompanying drawings. The accompanying drawings illustrate different aspects of the present disclosure, and where appropriate, reference numerals illustrating similar structures, components, materials, and / or elements in different drawings are similarly labeled. It should be understood that various combinations of structures, components, and / or elements other than those specifically illustrated are contemplated and within the scope of the present disclosure.

[0013] In addition, many embodiments of the present disclosure are described and shown herein. The present disclosure is neither limited to any single aspect or its embodiment, nor to any combination and / or arrangement of these aspects and / or embodiments. In addition, each aspect and / or its embodiment of the present disclosure may be adopted alone or in combination with one or more other aspects and / or its embodiments of the present disclosure. For the sake of brevity, specific arrangements and combinations are not discussed and / or shown separately herein.

[0014] Figure 1 An exemplary architecture of a speech communication system pipeline for devices connected to groups, conference rooms, and / or breakout rooms according to embodiments of the present disclosure is depicted.

[0015] Figure 2 Depicted is an exemplary architecture for one or more devices participating in a group and / or grouped room of an event using a communication system according to an embodiment of the present disclosure.

[0016] Figure 3 A method for simulating choral audio noise in a communication system according to an embodiment of the present disclosure is depicted.

[0017] Figure 4 A high-level diagram of an exemplary computing device that may be used in accordance with the systems, methods, and computer-readable media disclosed herein is depicted in accordance with an embodiment of the present disclosure.

[0018] Figure 5 A high-level diagram of an exemplary computing system that may be used in accordance with the systems, methods, and computer-readable media disclosed herein is depicted in accordance with an embodiment of the present disclosure.

[0019] Likewise, many embodiments are described and shown herein. The present disclosure is neither limited to any single aspect or its embodiment, nor to any combination and / or arrangement of these aspects and / or embodiments. Each aspect and / or its embodiment of the present disclosure may be adopted alone or in combination with one or more other aspects and / or its embodiments of the present disclosure. For the sake of brevity, those combinations and arrangements are not discussed separately herein. DETAILED DESCRIPTION

[0020] Those skilled in the art will recognize that various embodiments and examples of the present disclosure can be practiced according to the description. All such embodiments and examples are intended to be included within the scope of the present disclosure.

[0021] As used herein, the terms "comprises," "including," "has," "containing," "having," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. The term "exemplary" is used in the sense of "example" rather than "ideal." Additionally, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless specified otherwise, or clear from the context, the phrase "X employs A or B" is intended to mean any of the natural inclusive permutations. For example, the phrase "X employs A or B" is satisfied by any of the following instances: X employs A; X employs B; or X employs both A and B. Moreover, the articles "a" and "an" as used in this application and the appended claims should generally be construed to mean "one or more" unless specified otherwise or clear from the context to dictate a singular form.

[0022] For the sake of brevity, conventional techniques related to other functional aspects of the system and server used to perform the method and the system and server (and the various operating components of the system) may not be described in detail herein. In addition, the connecting lines shown in the various figures included herein are intended to represent exemplary functional relationships and / or physical couplings between the various elements. It should be noted that many alternative and / or additional functional relationships or physical connections may be present in the embodiments of the subject matter.

[0023] Reference will now be made in detail to exemplary embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numerals will be used throughout the drawings to refer to the same or like parts.

[0024] The present disclosure relates generally and more particularly to a method of generating a simulated choral audio noise in a communication system that provides feedback to an administrator of an event, a host of an event, a supervising user of an event, and / or a plurality of users attending an event. The event may be of any size and may include a conference, an exhibition hall, a school, and / or other large events where users are divided into groups or clusters, such as grouped rooms, classrooms, lectures, side meetings, etc., where a communication system is used to facilitate virtual attendance at an event.

[0025] Embodiments of the present disclosure provide a method that can be used to enhance the experience of users, administrators, hosts, and / or supervisors who are using a communication system to attend / participate in an event. Specifically, embodiments of the present disclosure can use a communication system to monitor the group activity of an event (such as a meeting), and can generate a spatial audio signal representing the choral activity of the group, which is to be conveyed to other people who may be listening at a certain distance (virtual distance) or in another channel. For example, when a group room is created, the presenter can hear a choral audio prompt from each sub-channel grouping instead of the presenter being kept in a quiet state. The audio can be a simulation of the generation of the amount of noise, the audio can be an actual pass-through audio of the audio communication, and / or a mixed audio of the two based on privacy or a desired mixing ratio. This method can provide a better group / group room experience. In addition, audio can be provided to the group / group room, which allows other groups / group rooms to hear each other, for example, at a given distance from each other.

[0026] As a non-limiting example, the simulated choral audio noise may include at least one or more of the following: the audio may be unfiltered pass-through audio of the user's conversation, the audio may be a filtered version of the user's conversation (using frequency, volume and / or other audio filters), the audio may be a collection of sounds / sounds representing at least one or more measured features of the user's conversation (e.g., speech rate, pitch, volume, quantity, etc.), the audio may be a collection of sounds / sounds representing a measured quantity of other communication activity of the user obtained from a transcription of video data, text data, gesture data, and / or audio data. In addition, embodiments of the present disclosure may not be limited to spatial audio. Stereo or mono signals may be used to provide feedback. Additionally, non-audio signals, such as visual representations or text.

[0027] Therefore, embodiments of the present disclosure provide feedback to presenters, speakers, hosts, administrators, supervisory users, and / or users, where many users are meeting together, rather than the uncomfortable silence surrounding the meeting when grouping is performed. In addition, because the noise suppression component used by the communication system may suppress audio feedback for administrators, hosts, and / or users, embodiments of the present disclosure improve the experience of the event by adding simulated noise that may be lost inadvertently due to noise suppression or from events that are essentially remote / virtual. The simulated noise can be generated from the audio data of the group, and can represent the volume and energy of the group's conversation, but the simulated noise can be filtered (silenced) to protect the privacy of the speech in the audio data. For example, the silenced noise can sound like there is sound but it is difficult to understand, and / or it can be like having a silent soundtrack in a bell, just like the adult voice in the Peanuts cartoon by Charles M. Schulz. Additionally, the simulated noise can also include applause, laughter, and / or other collected feedback.

[0028] Figure 1 An exemplary architecture of a speech communication system pipeline for devices connected to a group, conference room, and / or breakout room according to an embodiment of the present disclosure is depicted. Specifically, Figure 1 A speech communication system pipeline having at least one speech enhancement component (such as noise suppression) that may cause the event to lose audio feedback is depicted. Figure 1 As shown in , the microphone 102 of the device 140 can capture audio data, which includes, among other things, speech of a user in the communication system 100. The audio data captured by the microphone 102 can be processed by one or more speech enhancement components of the communication system 100. Non-limiting examples of speech enhancement components include music detection, acoustic echo cancellation, noise suppression, dereverberation, echo detection, automatic gain control, voice activity detection, jitter buffer management, packet loss concealment, and the like.

[0029] Figure 1 Depicted is audio data received by a noise suppression component 104 that can suppress noise in the audio data. In addition to receiving audio data captured by microphone 102, noise suppression component 104 can also receive speaker data played by speaker 110 to provide microphone-to-speaker alignment, such as microphone 102 and speaker 110 of device 140. Noise suppression component 104 can process audio data and speaker data to isolate speech from other sounds and music during playback. For example, when microphone 102 is turned on, background noise around the user (e.g., shuffling papers, slamming doors, barking dogs, etc.) may distract other users. Noise suppression component 104 can remove such noise around the user in the communication system.

[0030] The audio data may become speech enhanced audio data after being processed by one or more speech enhancement components (e.g., noise suppression component 104), and further processed by one or more other speech enhancement components. The audio data may then be received by encoder 106. Encoder 106 may be an audio codec, such as an AI-driven audio codec, such as a SATIN encoder, which is a digital signal processor with machine learning. Encoder 106 may encode (i.e., compress) the audio data for transmission over network 120. After encoding, encoder 106 may transmit the encoded audio data to network 120, where other components of the communication system (e.g., communication server 130) are provided. The other components of the communication system that are speaking may then transmit the audio data of the user and / or other users of the communication system over network 120.

[0031] Decoder 108 can receive audio data transmitted through network 120 and process audio data. Decoder 108 can be an audio codec, such as an AI-driven audio codec, such as a SATIN decoder, which is a digital signal processor with machine learning. Decoder 108 can decode (i.e., decompress) audio data received through network 120. After decoding, decoder 108 can provide decoded audio data to speaker 110. Speaker 110 can play decoded audio data as speaker data. Speaker data can also be provided to noise suppression component 104. Device 140 can include one or both of microphone 102 and / or speaker 110, for example, device 140 can be a combined microphone and speaker, such as a headset, a handheld phone, a conference call device, a smart speaker, etc., and / or device 140 can be a separate and independent microphone from the speaker. Alternatively, the components of communication system 100 can reside through network 120 and / or in the cloud, and communicate with one or more devices 140 through the network.

[0032] Figure 2 An exemplary architecture 200 of one or more devices (such as device 140) participating in a group and / or breakout room of an event or conference using a communication system according to an embodiment of the present disclosure is depicted. For example, multiple users using associated devices (such as device 140) can connect to a communication system hosted by, for example, conference server 230 via a network (such as network 120). Conference server 230 can host an event, receive audio data, create and / or manage a main room 270 for the event, create and / or manage groups, admit users to the event and / or groups, detect the energy of a group, and / or generate simulated choral audio noises, etc. As described above, the event can be one or more of a conference, an exhibition hall, a school, other large event, etc., and the group can be a breakout room, a classroom, a lecture, a side meeting, etc.

[0033] like Figure 2 As shown in, the conference server 230 can create and / or manage multiple groups, such as group rooms 202A, 202B, ... 202N, where N is an integer. Each group room may include multiple users who provide audio data to the conference server 230 and / or receive audio data from the conference server 230. The conference server 230 can provide the audio data of each user in the group room to other users in the same group room. The conference server 230 can also be connected to at least one user including one or more of the administrator of the event, the host of the event, and the supervising user of the event through, for example, the management / modification system 260. The administrator of the event, the host of the event, and / or the supervising user of the event can host the event from the main room 270 of the event and / or can host the event from one or more of the group rooms 202A, 202B, ... 202N. In addition, the conference server 230 can create and / or manage the admission room 250 for the event. The admission room 250 can be an entry point to the main room 270 and / or the group rooms 202A, 202B, ... 202N. For example, an administrator of an event, a host of an event, and / or a supervising user of an event may admit one or more users waiting in the admission room 250 to the main room 270 and / or the group rooms 202A, 202B, ... 202N. Similar to each group / group room, the admission room may include a plurality of users providing audio data to and / or receiving audio data from the conference server 230, which may provide the audio data to the administrator of the event, the host of the event, and / or the supervising user of the event, and / or may provide the audio data to the main room 270. Additionally and / or alternatively, when a user of the plurality of users rejoins the main room 270 from the group rooms 202A, 202B, ... 202N, a generated simulated choral audio noise for the main room 270 may be provided to one or more of the group rooms 202A, 202B, 202N, so that it is time for the users remaining in their respective group rooms to rejoin the main room 270.

[0034] Each group room 202A-202N may also include a spatial location associated with the group. Based on the spatial location of each group room 202A-202N, the distance 208 between each group room 202A-202N may be determined. Although the group rooms 202A-202N are shown as linear, any spatial topology is possible, such as circular, spherical, etc. In addition, each group room 202A-202N may initially be equidistant from each other, and based on one or more factors, the spatial location of each group room 202A-202N may be adjusted. In addition, each management / host system 260 may include a spatial location associated with a corresponding administrator, host, supervisory user, presenter, etc. Based on the spatial location of each management / host system 260, the distance between each group room 202A-202N and each management / host system 260 may be determined.

[0035] Each group room 202A-202N may have an associated energy detection module 204A, 204B, ... 204N, and the admission room 250 may have an associated energy detection module 204AR. Each of the energy detection modules 204 may detect the energy of the corresponding group room 202. For example, the energy detection module 204 may determine the number of speeches in the corresponding group room 202 based on the audio data received from each of the plurality of users in the corresponding group room 202. Additionally and / or alternatively, the energy detection module 204 may determine the activity level in the corresponding group room 202 based on the audio data received from each of the plurality of users in the corresponding group room 202. The activity level may also be determined based on one or more of video data, text data, gesture data, and transcription of the audio data from each of the plurality of users in the corresponding group room 202. Additionally and / or alternatively, the energy detection module 204 may analyze the audio data of each group room 202 for keywords and / or phrases. Then, the energy detection module 204 may determine whether a keyword or phrase has appeared in the audio data of each group room 202, and when the keyword or phrase is determined to have appeared in the audio data of the group room 202, the spatial position of each group room 202 may be adjusted. For example, if a specific name is spoken in a specific group room, the spatial position of the specific group room may be adjusted to reduce the distance of the specific group room from another group room and / or the management / moderator system 260.

[0036] Additionally and / or alternatively, the energy detection module 204 can determine the number of speeches in each group room 202 based on the audio data received from each of the multiple users in the corresponding group group 202. Then, when the number of determined speeches is below a predetermined threshold, the energy detection module 204 can generate a signal using the management / moderator system 260 and send the signal to one or more of the administrator of the event, the host of the event, and the supervising user of the event. Therefore, the administrator of the event, the host of the event, and the supervising user can determine whether to end the group session, enter the group room, and / or send information to the multiple users in the corresponding group group 202 to induce collaboration and / or interaction.

[0037] Additionally, each group room 202A-202N can have an associated chorus-like generative model module 206A, 206B, 206N. The admission room 250 can also have an associated chorus-like generative model module (not shown). Each chorus-like generative model module 206 can generate a simulated chorus-like audio noise for the corresponding group room 202, and provide the corresponding generated simulated chorus-like audio noise to at least one user who is not in the corresponding group room and / or each management / host system 260. The generated simulated chorus-like audio noise can be based on audio data received from each of the multiple users in the corresponding group room 202 and / or based on the energy detected by the corresponding energy detection module 204 of the corresponding group room 202, such as based on the determined activity level of the group room 202 and / or the number of speeches in the group room 202.

[0038] For example, each choral generative model module 206 can generate a simulated choral audio noise based on i) actual audio captured by a microphone of a user in a corresponding group room 202, ii) derived data obtained from the audio, or iii) a mixture of actual audio data and derived data. Depending on the configuration of the system, the generated simulated choral audio noise can remove recognizable speech of the audio data, and the generated simulated choral audio noise can be muted to protect the privacy of the speech in the audio data. Additionally and / or alternatively, the generated simulated choral audio noise can correspond to the speech rate, pitch, volume, and frequency of the audio data received from each of the multiple users in the group room, but is not a recognizable speech. In other words, the generated simulated choral audio noise may sound audible but difficult to understand, and / or may sound like a soundtrack with silence in a bell, like the adult voices in the Peanuts cartoon by Charles M. Schulz. Furthermore, simulated choral audio may also be generated by applying some filters (e.g., cancellation) (which may be a modified version of the original audio data) and / or may be transformed into some other audio data (e.g., the sound of a Charlie Brown soundtrack).

[0039] For example, the choral generation model module 206A may receive audio data from each of the multiple users participating in the group room 202A, and may generate a simulated choral audio noise based on the audio data received from each of the multiple users in the group room 202A. The generated simulated choral audio data may then be provided to at least one of the multiple users of the event (such as a user in another group room (such as group rooms 202B-202N)) and / or an administrator of the event, a host of the event, a supervising user of the event, a presenter at the event, etc. Simultaneously and / or continuously, the choral generation model module 206B may receive audio data from each of the multiple users participating in the group room 202B, and may generate a simulated choral audio noise based on the audio data received from each of the multiple users in the group room 202B. The generated simulated choral audio data may then be provided to at least one of the plurality of users of the event, such as a user in another grouping room, such as grouping rooms 202A and 202C-202N, and / or an administrator of the event, a host of the event, a supervising user of the event, a presenter of the event, etc. As described above, the conference server 230 may manage an admission room 250 for the event. The admission room 250 may have an associated energy detection module 204AR that may detect the energy of the admission room 250. Based on the detected energy of the admission room 250, the corresponding administrator, host, supervising user, speaker, etc. of the management / moderator system 260 may admit one or more users waiting in the admission room 250 to the event and / or to at least one grouping room 202. The admission room 250 may also have an associated choral generation model module (not shown) that generates an admission simulated choral audio buzz based on audio data received from each of the plurality of users in the admission room 250. The generated admission simulation choral audio data can be provided to the administrator of the event, the host of the event, and the supervising user of the event. Therefore, the administrator of the event, the host of the event, and the supervising user can know the user who hopes to join the event and / or the main room 270, and can act accordingly.

[0040] In addition, as described above, based on the spatial position of each group room 202A-202N, the distance 208 between each group room 202A-202N can be determined. The chorus generation model module 206 can generate a simulated chorus audio noise based on the audio data received from each of the multiple users in the corresponding group room 202 and based on the corresponding distance 208. For example, when the determined distance for a specific group room among the multiple group rooms 202 is less than a first predetermined threshold or less than or equal to the first predetermined threshold, the generated simulated chorus audio data can be provided to the specific group room. Otherwise, when the distance is greater than or greater than or equal to the first predetermined threshold, the generated simulated chorus audio data will not be provided to the specific group group. Alternatively, when the determined distance for the specific group room is less than the first predetermined threshold or less than or equal to the first predetermined threshold, the chorus generation model module 206 can only generate a simulated chorus audio noise.

[0041] Additionally, the choral generation model module 206 can generate a simulated choral audio noise based on the audio data received from each of the plurality of users in the corresponding grouped room 202 and based on the corresponding distance 208. For example, the choral generation model module 206 can adjust the generated simulated choral audio data for each grouped room based on the determined distance between each grouped room. Thus, as the determined distance increases, the volume of the generated simulated choral audio data can decrease.

[0042] In addition, when the determined distance for a specific group room is less than or equal to a second predetermined threshold, the chorus generation model module 206 may determine that the group room 202 is too close so that the simulated chorus audio is no longer provided, but may provide the corresponding group room with audio data of the group room 202 whose distance is less than or equal to the second predetermined threshold. When this situation occurs, when the audio data of the specific group room is provided to the corresponding group room, a warning that their audio is being provided to at least one other user may be provided to one or more of the administrator of the event, the host of the event, the supervising user of the event, and the user of the specific group.

[0043] Figure 3A method 300 for simulating choral audio noise in a communication system according to an embodiment of the present disclosure is depicted. The method 300 may begin at 302, where audio data from each of a plurality of users of a first group of a plurality of groups participating in an event using a communication system may be received. The plurality of groups may be a plurality of group rooms, such as group rooms 202A, 202B, ... 202N, and the event may be a meeting with a main room, such as main room 270. In addition to receiving the audio data, one or more of video data, text data, gesture data, and a transcription of the audio data from each of the plurality of users participating in the first group may be received. Prior to, concurrently with, after, and / or continuing to 302, at 304, audio data from each of a plurality of users of a second group of a plurality of groups participating in an event using a communication system may be received. Each of the plurality of groups may include a spatial location associated with the group. Before, simultaneously with, after, and / or continuing to 302 and / or 304 , at 306 , audio data from each of a plurality of users waiting in an admission room for the event may be received.

[0044] After receiving the audio data, at 308, based on the audio data received from each of the plurality of users in the first group, a first simulated chorus audio noise may be generated. The generated first simulated chorus audio noise may remove the identifiable speech of the audio data, and / or the generated first simulated chorus audio noise may be muted and the privacy of the speech in the audio data may be protected. Additionally, the generation of the first simulated chorus audio noise based on the audio data received from each of the plurality of users in the first group may also be based on one or more of the video data, the text data, the gesture data, and the transcription of the audio data. Alternatively, in order to generate the first simulated chorus audio noise directly based on the audio data received from each of the plurality of users in the first group, the activity level of the first group may be determined based on the audio data received from each of the plurality of users in the first group, and the first simulated chorus audio noise may be determined based on the determined activity level of the first group. The activity level of the first group may be determined based on one or more of the speech rate, pitch, volume, and frequency of the audio data received from each of the plurality of users in the first group.

[0045] At 310, based on the audio data received from each of the plurality of users in the second group, a second simulated chorus audio noise may be generated. The generated second simulated chorus audio noise may remove identifiable speech of the audio data, and / or the generated second simulated chorus audio noise may be muted and privacy of speech in the audio data may be protected. The generated second simulated chorus audio noise may be generated in a manner similar to the generated first simulated chorus audio noise discussed above.

[0046] At 312, an admission simulated chorus audio buzz may be generated based on audio data received from each of the plurality of users in the admission room. The generated admission simulated chorus audio buzz may remove identifiable speech from the audio data, and / or the generated admission simulated chorus audio buzz may be muffled and privacy of speech in the audio data may be protected. The generated admission simulated chorus audio buzz may be generated in a manner similar to the generated first simulated chorus audio buzz discussed above.

[0047] Additionally and / or alternatively, at 314, based on the spatial position of each group, the distance between each group in the plurality of groups and another group can be determined. Then, at 316, based on the determined distance between each group and another group, the generated simulated choral audio data for each group can be adjusted. For example, as the determined distance increases, the volume of the generated simulated choral audio data can be reduced. Alternatively, based on the determined distance between each group and another group, the total gain, left / right channel gain, and / or directionality (left, center, right, top, bottom) can be adjusted.

[0048] Additionally and / or alternatively, at 318, the audio data of each of the plurality of groups may be analyzed for at least one keyword or phrase. Then, at 320, it may be determined whether the at least one keyword or phrase has appeared in the audio data of each group. Next, at 322, when the at least one keyword or phrase is determined to have appeared in the audio data of each group, the spatial position of each of the plurality of groups may be adjusted (e.g., reduced).

[0049] Additionally and / or alternatively, at 324, an activity level for the first group can be determined based on the audio data received from each of the plurality of users in the first group. Then, at 326, the generated first simulated chorus-like audio data for the first group can be adjusted based on the determined activity level for the first group. Additionally, the activity level can also be determined based on one or more of the video data, the text data, the gesture data, and the transcription of the audio data.

[0050] At 328, the generated first simulated choral audio data can be provided to at least one of the multiple users of the event. The at least one user can be one or more of the administrator of the event, the host of the event, the supervising user of the event, and / or the presenter of the event. Optionally, when the determined distance of a specific group among the multiple groups is less than a first predetermined threshold, the generated simulated choral audio data can be provided to the specific group. In addition, when the determined distance is less than a second predetermined threshold, wherein the second predetermined distance is less than the first predetermined distance, the actual audio data can be provided to the specific group, and when the determined distance is less than the second predetermined threshold, the generated simulated choral audio data can be stopped from being provided to the specific group. In this case, when the actual audio data of the specific group is provided to another group among the multiple groups, one or more of the administrator of the event, the host of the event, the supervising user of the event, and the users of the specific group can be warned.

[0051] In addition, at step 330, the generated second simulated choral audio data may be provided to the plurality of users in the first group. The generated first simulated choral audio data provided to the at least one user among the plurality of users may include providing the generated first simulated choral audio data to the plurality of users in the second group. At 332, the generated admission simulated choral audio data may be provided to an administrator of the event, a host of the event, and a supervising user of the event. As described above, the administrator of the event, the host of the event, and / or the supervising user of the event may host the event from the main room 270 of the event and / or may host the event from one or more grouped rooms.

[0052] Finally, optionally, based on the audio data received from each of the multiple users in the corresponding groups in the multiple groups, the number of speeches in each group can be determined. Then, when the number of determined speeches is below a predetermined threshold, a signal can be generated and sent to an administrator of the event, a host of the event, a supervising user of the event, the main room 270 of the event, and / or one or more of the group rooms 202A, 202B, ... 202N.

[0053] The use of detecting simulated choral audio noise can be accomplished by providing a moderator breakout room and detecting audio received from the breakout room without a moderator entering the breakout room. Additionally, the audio detected without a moderator entering the breakout room can be compared to the audio captured by the microphone of the user speaking in the breakout room to show that the audio noise is generated by the user audio data in the breakout room.

[0054] Figure 4A high-level diagram of an exemplary computing device 400 that can be used in accordance with the systems, methods, modules, and computer-readable media disclosed herein, according to an embodiment of the present disclosure, is depicted. For example, according to an embodiment of the present disclosure, the computing device 400 can be used in a system that processes data (such as audio data) using a communication system. The computing device 400 may include at least one processor 402 that executes instructions stored in a memory 404. The instructions may be, for example, instructions for implementing the functions described as performed by one or more of the above components or instructions for implementing one or more of the above methods. The processor 402 can access the memory 404 via a system bus 406. In addition to storing executable instructions, the memory 404 can also store data, audio, etc.

[0055] The computing device 400 may additionally include a data store (also referred to as a database) 408, which is accessible by the processor 402 via the system bus 406. The data store 408 may include executable instructions, data, examples, features, etc. The computing device 400 may also include an input interface 410 that allows external devices to communicate with the computing device 400. For example, the input interface 410 may be used to receive instructions from an external computer device, from a user, etc. The computing device 400 may also include an output interface 412 that interfaces the computing device 400 with one or more external devices. For example, the computing device 400 may display text, images, etc. via the output interface 412.

[0056] It is contemplated that external devices that communicate with computing device 400 via input interface 410 and output interface 412 may be included in an environment that provides substantially any type of user interface with which a user may interact. Examples of user interface types include graphical user interfaces, natural user interfaces, and the like. For example, a graphical user interface may accept input from a user using an input device such as a keyboard, a mouse, a remote control, and may provide output on an output device such as a display. In addition, a natural user interface may enable a user to interact with computing device 400 in a manner that is free from the constraints imposed by input devices such as a keyboard, a mouse, a remote control, and the like. In contrast, a natural user interface may rely on speech recognition, touch and pen recognition, gesture recognition on and near the screen, air gestures, head and eye tracking, voice and speech, vision, touch, gestures, machine intelligence, and the like.

[0057] Additionally, although shown as a single system, it should be understood that computing device 400 may be a distributed system. Thus, for example, several devices may communicate via a network connection and may jointly perform the tasks described as being performed by computing device 400.

[0058] Go to Figure 5 , Figure 5A high-level diagram of an exemplary computing system 500 that can be used in accordance with the systems, methods, modules, and computer-readable media disclosed herein, according to an embodiment of the present disclosure is depicted. For example, computing system 500 can be or can include computing device 400. Additionally and / or alternatively, computing device 400 can be or can include computing system 500.

[0059] Computing system 500 may include multiple server computing devices, such as server computing device 502 and server computing device 504 (collectively referred to as server computing devices 502-504). Server computing device 502 may include at least one processor and a memory; the at least one processor executes instructions stored in the memory. The instructions may be, for example, instructions for implementing the functions described as being performed by the above one or more components or instructions for implementing one or more of the above methods. Similar to server computing device 502, at least one subset of server computing devices 502-504 other than server computing device 502 may each include at least one processor and a memory, respectively. In addition, at least one subset of server computing devices 502-504 may include corresponding data storage.

[0060] The processor(s) of one or more server computing devices 502-504 may be or may include a processor, such as processor 402. Additionally, the memory(s) of one or more server computing devices 502-504 may be or may include a memory, such as memory 404. Additionally, the data storage(s) of one or more server computing devices 502-504 may be or may include a data storage, such as data storage 408.

[0061] The computing system 500 may also include various network nodes 506 for transmitting data between the server computing devices 502-504. In addition, the network nodes 506 may transmit data from the server computing devices 502-504 to external nodes (e.g., external to the computing system 500) via a network 508. The network nodes 502 may also transmit data from the external nodes to the server computing devices 502-504 via the network 508. The network 508 may be, for example, the Internet, a cellular network, etc. The network nodes 506 may include switches, routers, load balancers, etc.

[0062] The fabric controller 510 of the computing system 500 can manage the hardware resources of the server computing devices 502-504 (e.g., the processors, memory, data storage, etc. of the server computing devices 502-504). The fabric controller 510 can also manage the network nodes 506. In addition, the fabric controller 510 can manage the creation, provisioning, de-provisioning, and supervision of the hosted runtime environments instantiated on the server computing devices 502-504.

[0063] As used herein, the terms "component" and "system" are intended to encompass a computer-readable data storage configured with computer-executable instructions that, when executed by a processor, cause specific functions to be performed. Computer-executable instructions may include routines, functions, etc. It should also be understood that a component or system may be located on a single device or distributed across several devices.

[0064] The various functions described herein can be implemented in hardware, software, or any combination thereof. If implemented in software, the function can be stored on a computer-readable medium and / or transmitted on a computer-readable medium as one or more instructions or codes. Computer-readable media may include computer-readable storage media. The computer-readable storage medium may be any available storage medium accessible by a computer. As an example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other medium that can be used to store the desired program code in the form of an instruction or data structure and can be accessed by a computer. As used herein, disks and optical disks may include compact disks ("CD"), laser optical disks, optical disks, digital versatile disks ("DVD"), floppy disks, and blue-ray disks ("BD"), wherein disks typically copy data magnetically, and optical disks typically copy data optically using lasers. In addition, propagation signals are not included in the scope of computer-readable storage media. Computer-readable media may also include communication media, which includes any media that facilitates the transfer of computer programs from one place to another. For example, a connection may be a communication medium. For example, if the software is transmitted from a website, server or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line ("DSL") or wireless technologies such as infrared, radio and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of communication media. Combinations of the above may also be included within the scope of computer-readable media.

[0065] Alternatively and / or additionally, the functions described herein may be performed, at least in part, by one or more hardware logic components. For example, but not limited to, illustrative types of hardware logic components that may be used include field programmable gate arrays ("FPGAs"), application specific integrated circuits ("ASICs"), application specific standard products ("ASSPs"), systems on chip ("SOCs"), complex programmable logic devices ("CPLDs"), etc.

[0066] The above description includes examples of one or more embodiments. Of course, for the purpose of describing the above aspects, it is not possible to describe every conceivable modification and change of the above devices or methods, but those of ordinary skill in the art will recognize that many further modifications and permutations of various aspects are possible. Therefore, the described aspects are intended to cover all such changes, modifications and variations that fall within the scope of the appended claims.

Claims

1. A computer-implemented method for simulating choral audio noise in a communication system, the method comprising: receiving audio data from each of a plurality of users participating in a first group of a plurality of groups for an event using a communication system; generating a first simulated chorus-like audio noise based on the audio data received from each of the plurality of users in the first group; as well as The generated first simulated choral audio data is provided to at least one user among a plurality of users of the event. 2 . The method of claim 1 , wherein the plurality of groups are a plurality of breakout rooms, and the event is a meeting having a main room.

3. The method of claim 1, wherein the at least one user is one or more of an administrator of the event, a host of the event, a supervising user of the event, and a presenter of the event.

4. The method according to claim 3, further comprising: determining, based on the audio data received from each of the plurality of users in a corresponding one of the plurality of groups, a number of speeches in each group; as well as When the determined number of contributions is below a predetermined threshold, a signal is generated and sent to one or more of the administrator of the event, the moderator of the event, and the supervising user of the event.

5. The method according to claim 3, further comprising: receiving audio data from each of a plurality of users waiting in an admission room for the event; generating an admitted simulated choral audio buzz based on the audio data received from each of the plurality of users in the admitted room; as well as The generated admission simulation choral audio data is provided to the administrator of the event, the host of the event, and the supervising user of the event.

6. The method according to claim 1, further comprising: receiving audio data from each of a plurality of users participating in a second group of the plurality of groups; generating a second simulated chorus-like audio noise based on the audio data received from each of the plurality of users in the second group; as well as providing the generated second simulated chorus-like audio data to the plurality of users in the first group, Wherein providing the generated first simulated chorus audio data to the at least one user among the plurality of users comprises: The generated first simulated chorus-like audio data is provided to the plurality of users in the second group.

7. The method of claim 1 , wherein each group of the plurality of groups comprises a spatial location associated with the group, The method further comprises: Determining a distance between each group in the plurality of groups and another group based on the spatial position of each group; as well as When the determined distance for a specific group among the plurality of groups is less than a first predetermined threshold, the generated simulated chorus-like audio data is provided to the specific group.

8. The method according to claim 7, further comprising: The generated simulated chorus-like audio data for each group is adjusted based on the determined distance between each group and another group, wherein the volume of the generated simulated chorus-like audio data decreases as the determined distance increases.

9. The method according to claim 7, further comprising: providing the audio data of the specific group when the determined distance is less than a second predetermined threshold, the second predetermined distance being less than the first predetermined distance; as well as When the determined distance is smaller than the second predetermined threshold, the provision of the generated simulated chorus audio data to the specific group is stopped.

10. The method according to claim 9, further comprising: When the audio data of the specific group is provided to another group of the plurality of groups, one or more of an administrator of the event, a host of the event, a supervising user of the event, and a user of the specific group are alerted.

11. The method according to claim 7, further comprising: analyzing the audio data of each of the plurality of groups for at least one keyword or phrase; determining whether the at least one keyword or phrase has appeared in the audio data for each group; as well as The spatial position of each of the plurality of groups is adjusted when the at least one keyword or phrase is determined to have appeared in the audio data of the group.

12. The method according to claim 1, further comprising: determining an activity level of the first group based on the audio data received from each of the plurality of users in the first group; The first simulated chorus-like audio data generated for the first group is adjusted based on the determined activity level of the first group.

13. The method of claim 1 , wherein generating the first simulated choral audio noise based on the audio data received from each of the plurality of users in the first group comprises: The first simulated chorus audio noise is generated based on speech in the audio data received from each of the plurality of users in the first group, wherein the generated first simulated chorus audio noise removes recognizable speech from the audio data.

14. A system for simulating choral audio noise in a communication system, the system comprising: a data storage device storing instructions for simulating a choral audio noise in a communication system; as well as A processor configured to execute the instructions to perform a method comprising: receiving audio data from each of a plurality of users participating in a first group of a plurality of groups for an event using a communication system; generating a first simulated chorus-like audio noise based on the audio data received from each of the plurality of users in the first group; as well as The generated first simulated choral audio data is provided to at least one user among a plurality of users of the event.

15. A computer readable storage device storing instructions which, when executed by a computer, cause the computer to perform a method for simulating choral audio noise in a communication system, the method comprising: receiving audio data from each of a plurality of users participating in a first group of a plurality of groups for an event using a communication system; generating a first simulated chorus-like audio noise based on the audio data received from each of the plurality of users in the first group; as well as The generated first simulated choral audio data is provided to at least one user among a plurality of users of the event.

Images