Voice flow mixing method, device and equipment and readable medium
By performing mixed streaming on the voice streams of multiple second users in a multi-person online meeting, generating and providing client playback to the first user, the problem of increased downlink bandwidth and resource consumption of terminals is solved, and the voice quality and fluency are improved , improves the user experience.
Patent Information
- Application Number
- CN202411248493.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-11-26
AI Technical Summary
In the multi-person online meeting scenario, as the number of participants increases, the consumption of terminal downlink bandwidth and decoding resources increases exponentially, resulting in increased costs and affected user audio-visual experience.
Provide a voice mixing method, by receiving a mixing request from a target user group, determining voice parameters, and performing mixing on the voice stream of a plurality of second users based on the mixing stream configuration, generating mixed streaming voices, and providing them to customers of the first user playback to reduce downlink bandwidth and resource consumption.
It reduces the downlink bandwidth and resource consumption of the terminal, improves the voice quality and fluency, reduces the number of voice streams pulled by the client, and improves the user experience.
Smart Images

Figure CN119031077A_ABST
Abstract
Description
Technical Field
[0001] Example embodiments of the present disclosure generally relate to the field of computer technology, and more particularly, to a method, apparatus, device, and computer-readable storage medium for voice mixing. Background Art
[0002] In a multi-person online (audio or video) conference scenario, it is usually necessary to pull multiple voice streams in parallel and perform separate decoding on each voice stream. However, as the number of participants increases, the cost of terminal downlink bandwidth, decoding, and resource consumption will increase exponentially, which not only increases costs but also affects the user's audio-visual experience. Summary of the invention
[0003] In a first aspect of the present disclosure, a method for voice mixing is provided. The method comprises: receiving a mixing request of a first user in a target user group associated with a target online conference, the mixing request comprising at least a mixing configuration, the mixing configuration indicating that mixing is performed on the voice streams of a plurality of second users in the target user group except the first user; determining voice parameters for the target online conference; based on the mixing configuration and the voice parameters, mixing is performed on the voice streams of the plurality of second users in the target user group to obtain mixed voice; and providing the mixed voice to at least a client corresponding to the first user, so as to play the mixed voice on the client.
[0004] In a second aspect of the present disclosure, a device for voice mixing is provided. The device includes: a receiving module configured to receive a mixing request of a first user in a target user group associated with a target online conference, the mixing request at least including a mixing configuration, the mixing configuration indicating that mixing is performed on the voice streams of multiple second users in the target user group except the first user; a determining module configured to determine the voice parameters for the target online conference; an executing module configured to perform mixing on the voice streams of multiple second users in the target user group based on the mixing request and the voice parameters to obtain mixed voice; and a providing module configured to provide the mixed voice at least to a client corresponding to the first user, so as to play the mixed voice on the client.
[0005] In a third aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, and when the instructions are executed by the at least one processing unit, the electronic device executes the method of the first aspect of the present disclosure.
[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to perform the method according to the first aspect of the present disclosure.
[0007] In a fifth aspect of the present disclosure, a computer program product is provided, comprising computer executable instructions, wherein the computer executable instructions can be executed by a processor to perform the method according to the first aspect of the present disclosure.
[0008] It should be understood that the contents described in the summary of the invention are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] In the following, in conjunction with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages and aspects of various implementations of the present disclosure will become more apparent. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:
[0010] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;
[0011] Figure 2 A flowchart showing a process of voice mixing according to some embodiments of the present disclosure is shown;
[0012] Figure 3 A schematic diagram showing a process of voice mixing according to some embodiments of the present disclosure is shown;
[0013] Figure 4 A schematic diagram showing a process of backend scheduling according to some embodiments of the present disclosure is shown;
[0014] Figure 5 A schematic structural block diagram of an apparatus for voice mixing according to some embodiments of the present disclosure is shown; and
[0015] Figure 6 A block diagram of an electronic device is shown in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION
[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0017] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.
[0018] It should be noted that the acquisition, storage and application of user personal information involved in the technical solution of the present disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0019] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.
[0020] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information, so that the user can independently choose whether to provide personal information to software or hardware such as electronic devices, applications, servers or storage media that execute operations of the technical solution of the present disclosure based on the prompt message.
[0021] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information is sent to the user in a manner such as a pop-up window, in which the prompt information can be presented in text form. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0022] Traditionally, in a multi-person online conference scenario, multiple voice streams are pulled in parallel and each voice stream is decoded separately. For example, in an online conference with twenty people, each client entering the online conference requests the media server separately and pulls the voice stream of each participant in real time. In this way, twenty voice streams are obtained on the terminal downlink, and twenty decoders are created to play them separately. However, in a multi-person online conference scenario (especially large-scale conferences with dozens or hundreds of people), as the number of participants increases, the cost and resource consumption of terminal downlink bandwidth, decoding, etc. will increase exponentially, increasing enterprise costs and affecting the user's audio-visual experience.
[0023] In view of this, an embodiment of the present disclosure provides a solution for voice mixing. The solution includes: receiving a mixing request from a first user in a target user group associated with a target online conference, the mixing request at least including a mixing configuration, the mixing configuration indicating that mixing is performed on the voice streams of multiple second users in the target user group except the first user; determining the voice parameters for the target online conference; based on the mixing configuration and the voice parameters, mixing is performed on the voice streams of multiple second users in the target user group to obtain mixed voice; and the mixed voice is provided to at least the client corresponding to the first user to play the mixed voice on the client. In this way, the client only needs to pull one mixed voice instead of multiple voices, which can reduce the downlink bandwidth, reduce terminal resource consumption, and improve voice quality and fluency.
[0024] Figure 1 A schematic diagram of an example environment 100 is shown in which embodiments of the present disclosure can be implemented.
[0025] like Figure 1 As shown, the mixing system 110 may include a mixing server 120, which may be used to perform voice mixing. In some embodiments, the mixing system 110 may provide mixed voice 132 (e.g., mixed voice 132-1, mixed voice 132-2, ... mixed voice 132-n) to one or more clients 101 based on mixed voice requests 102 (e.g., mixed voice request 102-1, mixed voice 102-2, ... mixed voice 132-n) received from one or more clients 101 (e.g., client 101-1, client 101-2, ... client 101-n).
[0026] Specifically, after receiving the mixing request 102-1 sent by the client 101-1, the mixing system 110 can perform mixing through the mixing server 120 and provide the mixed voice 132-1 to the client 101-1. Further, after receiving the mixing request 102-2 sent by the client 101-2, the mixing system 110 can perform mixing through the mixing server 120 and provide the mixed voice 132-2 to the client 101-2. In addition, after receiving the mixing request 102-n sent by the client 101-n, the mixing system 110 can perform mixing through the mixing server 120 and provide the mixed voice 132-n to the client 101-n. In some examples, the value of n can be a positive integer greater than or equal to 3. It can be understood that this is merely exemplary and not restrictive.
[0027] The hybrid streaming system 110 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms. The server 120 may include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, and the like.
[0028] It should be understood that the structure and function of the various elements in the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.
[0029] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.
[0030] Figure 2 1 is a flow chart showing a process 200 of voice mixing according to some embodiments of the present disclosure. The process 200 may be implemented at the mixing system 110. For ease of discussion, reference will be made to Figure 1 The process 200 is described with reference to the environment 100 of FIG.
[0031] In block 210, the streaming system 110 receives a streaming request 102 from a first user in a target user group associated with a target online conference. The target online conference may be a video conference or a voice conference. The target user group includes members participating in the target online conference.
[0032] In some embodiments, the mixing request 102 includes at least a mixing configuration, which may indicate that the voice streams of multiple second users in the target user group other than the first user are mixed. In this way, the downlink bandwidth and the terminal resource consumption may be reduced.
[0033] In some embodiments, the mixed stream configuration is determined based on at least the role of the first user in the target online conference and / or the role of multiple second users in the target online conference. In some examples, assuming that the role of the first user in the target user group in the target online conference is the current speaker, and the roles of other users in the target user group except the first user in the target online conference are all listeners, then the mixed stream configuration can indicate that the voice streams of all other users except the first user are mixed, that is, the voice streams of all users whose roles are listeners in the target online conference are mixed. In other examples, assuming that the roles of user A and user B in the target user group are the current speakers in the target online conference, and the roles of other users in the target user group except user A and user B in the target online conference are all listeners, then the mixed stream configuration can indicate that the voice streams of all other users except user A and user B are mixed, that is, the voice streams of all users whose roles are listeners in the target online conference are mixed. It can be understood that this is merely exemplary and not restrictive.
[0034] In some examples, it is assumed that there are 100 users in the target user group, among which user E, user F and user G are speakers, and the other 97 users in the target user group are listeners. Specifically, it is assumed that the first user is any user among the other 97 users, then the mixing configuration corresponding to the first user can indicate to perform mixing on the voice streams of user E, user F and user G. It can be understood that this is merely exemplary and not restrictive.
[0035] In other examples, the voice streams of multiple second users in the target user group except the first user can be mixed. In more examples, the voice streams of some second users in the target user group except the first user can be mixed. It can be understood that this is merely exemplary and not restrictive. More, in the scenario where the bandwidth of the client 101 is less than a predetermined threshold or the performance of the client 101 cannot smoothly pull multiple voice streams, the voice streams of multiple second users in the target user group can be mixed.
[0036] In block 220 , the streaming system 110 determines voice parameters for the target online conference.
[0037] In some embodiments, the mixing system 110 can determine the voice parameters for the target online conference based on the voice quality requirements related to the target online conference. In some examples, the target online conference can include a music online conference, and the music online conference can indicate some media scenarios of regular online conferences that have high requirements for voice quality, such as online concerts, online award ceremonies, and online parties. In other examples, the voice quality requirements for music online conferences are higher than the voice quality requirements for regular multi-person online conferences. In this way, the voice quality and fluency can be improved.
[0038] In some embodiments, in response to indicating a first voice quality requirement with the voice quality requirement, the mixing system 110 may determine that the voice parameter for the target online conference indicates a first voice parameter. In other embodiments, in response to indicating a second voice quality requirement with the voice quality requirement, the mixing system 110 may determine that the voice parameter for the target online conference indicates a second voice parameter, and the second voice quality requirement is higher than the first voice quality requirement. In some examples, the first voice parameter may indicate that the bit rate is in a first bit rate range and the sampling rate is in a first sampling rate or a first sampling rate range. In other examples, the second voice parameter may indicate that the bit rate is in a second bit rate range and the sampling rate is in a second sampling rate or a second sampling rate range.
[0039] In some embodiments, the first voice parameter belongs to a first parameter range, the second voice parameter belongs to a second parameter range, and the minimum value in the second parameter range is greater than the maximum value in the first parameter range. In some examples, the minimum value in the second rate range of the bit rate in the second voice parameter is greater than the maximum value in the first rate range of the bit rate in the first voice parameter. In other examples, the second sampling rate of the sampling rate in the second voice parameter or the minimum value in the second sampling rate range is greater than the first sampling rate of the bit rate in the first voice parameter or the maximum value in the first sampling rate range.
[0040] In some embodiments, the voice parameter includes at least one of a bit rate and a sampling rate. In some examples, in a conventional online conference, the bit rate in the first voice parameter may indicate 8k to 16kpbs, and the sampling rate may indicate 8kHz. In other examples, in a music online conference, the bit rate in the second voice parameter may indicate 128k to 256kpbs, and the sampling rate may indicate 44.1k / 48kHz. It is understood that this is merely exemplary and not restrictive.
[0041] In block 230, based on the mixing configuration and the voice parameters, the voice streams of the plurality of second users in the target user group are mixed to obtain mixed voice 132. In this way, the downlink bandwidth can be reduced and the terminal resource consumption can be reduced.
[0042] After the mixed stream system 110 obtains the mixed stream voice 132 , in block 240 , the mixed stream voice 132 may be provided to at least the client 101 corresponding to the first user, so that the mixed stream voice 132 is played on the client 101 .
[0043] In some examples, when the voice streams of multiple second users in the target user group are mixed, the first user can just pull and parse the mixed voice 132 to play the mixed voice 132 on the client 101 .
[0044] Continue to refer to Figure 3 The example process of voice mixing is further described. It should be understood that this is merely exemplary and not restrictive.
[0045] Figure 3 FIG. 3 is a schematic diagram of a voice mixing process 300 according to some embodiments of the present disclosure. The process 300 may be implemented at the mixing system 110. For ease of discussion, reference will be made to FIG. Figure 1 The process 300 is described with reference to the environment 100 of FIG.
[0046] like Figure 3 As shown, the mixing system 110 may include a mixing server 120 and a media server 310. Further, the mixing server 120 may be used to perform mixing and provide the mixed voice 132 to the media server 310. Further, the media server 310 may be used to receive the mixing request 102 and parse the mixing configuration from the mixing request 102. More, the media server 310 may also provide the mixing configuration to the mixing server 120 so that the mixing server 120 performs mixing based on the mixing configuration and voice parameters. In some examples, the media server 310 may also include a signaling server. In this case, the mixing request 102 may also be received by the signaling server, and the mixing configuration may also be parsed from the mixing request 102 by the signaling server. It can be understood that this is merely exemplary and not restrictive.
[0047] In some embodiments, the mixing system 110 may provide mixed stream voice 132 to one or more clients 101 respectively based on the mixed stream request 102 received from one or more clients 101 (eg, client 101 - 1 , client 101 - 2 . . . client 101 - n).
[0048] Specifically, after receiving a mixing request 102 sent by a certain client 101, the mixing system 110 can determine the mixing configuration and voice parameters through the media server 310, and provide the mixing configuration and voice parameters to the mixing server 120 to perform mixing on the voice streams of multiple second users in the target user group to obtain mixed voice 132. Specifically, assuming that the client 101-1 corresponds to the first user, then the client 101-2, the client 101-3... the client 101-n can correspond to multiple second users. In other words, assuming that the client 101-2 corresponds to the first user, then the client 101-1, the client 101-3... the client 101-n can correspond to multiple second users. It can be understood that this is merely exemplary and not restrictive.
[0049] Further, after the mixing server 120 obtains the mixed stream voice 132, the mixed stream voice 132 can be provided to the client 101 via the media server 310, so as to play the mixed stream voice 132 on the client 101. In some examples, the mixing configuration can indicate the combination of mixed stream users in the target user group. For example, the mixing configuration can indicate that the voice streams of user B and user C in the target user group are mixed.
[0050] Continue to refer to Figure 4 The process of backend scheduling is further described. It is understood that this is merely exemplary and not restrictive.
[0051] Figure 4 FIG. 4 is a schematic diagram of a backend scheduling process 400 according to some embodiments of the present disclosure. The process 400 may be implemented at the stream mixing system 110. For ease of discussion, reference will be made to FIG. Figure 1 The process 400 is described with reference to the environment 100 of FIG.
[0052] like Figure 4 As shown, the background service agent 430 to execute the newly added mixed flow process can be assigned through the foreground interface 410, and the number of mixed flow processes being executed by each background agent service can also be viewed through the foreground interface 410. In some examples, other configuration information related to the scheduling interface layer 420, the background service agent 430, and the mixed flow process can also be adjusted through the foreground interface 410.
[0053] Further, the scheduling interface layer 420 can control one or more background service agents 430 (e.g., background service agent 430-1, background service agent 430-2, and background service agent 430-3), and each background service agent 430 can execute one or more mixing processes at the same time. In some examples, the background service agent 430-1 can execute three mixing processes at the same time, the background service agent 430-2 can execute two mixing processes at the same time, and the background service agent 430-3 can execute multiple mixing processes at the same time. It can be understood that this is merely exemplary and not restrictive.
[0054] Furthermore, each background service agent 430 may be provided with a process number threshold. In some examples, if the number of mixed-stream processes being executed by the background service agent A is equal to the process number threshold, then in the case of a newly added mixed-stream process to be executed, the background service agent A may be used as an unselectable background service agent 430. In other examples, if the number of mixed-stream processes being executed by the background service agent A is less than the process number threshold, then in the case of a newly added mixed-stream process to be executed, the background service agent A may be used as an selectable background service agent 430. It is understood that this is merely exemplary and not restrictive.
[0055] In summary, the embodiments of the present disclosure can determine the mixed stream voice through the mixed stream configuration and voice parameters, and provide the mixed stream voice to the client so that the mixed stream voice can be played on the client. In this way, resource consumption can be reduced, voice quality and fluency can be improved, and the efficiency of the voice decoding process can be improved.
[0056] The embodiments of the present disclosure also provide corresponding devices for implementing the above methods or processes.
[0057] Figure 5 A schematic structural block diagram of an apparatus 500 for voice stream mixing according to some embodiments of the present disclosure is shown. The apparatus 500 may be implemented as or included in a stream mixing system 110. Each module / component in the apparatus 500 may be implemented by hardware, software, firmware, or any combination thereof.
[0058] like Figure 5As shown, the device 500 includes a receiving module 510, which is configured to receive a mixing request of a first user in a target user group associated with a target online conference, the mixing request includes at least a mixing configuration, and the mixing configuration indicates that the voice streams of multiple second users in the target user group other than the first user are mixed. The device 500 also includes a determination module 520, which is configured to determine the voice parameters for the target online conference. The device 500 also includes an execution module 530, which is configured to perform mixing on the voice streams of multiple second users in the target user group based on the mixing request and the voice parameters to obtain mixed voice. The device 500 also includes a providing module 540, which is configured to provide the mixed voice at least to a client corresponding to the first user, so as to play the mixed voice on the client.
[0059] In some embodiments, the determination module 520 is further configured to determine a voice parameter for the target online conference based on a voice quality requirement associated with the target online conference.
[0060] In some embodiments, the determination module 520 is also configured to determine that the voice parameter indication for the target online conference is a first voice parameter in response to the voice quality requirement indicating a first voice quality requirement; and to determine that the voice parameter indication for the target online conference is a second voice parameter in response to the voice quality requirement indicating a second voice quality requirement, and the second voice quality requirement is higher than the first voice quality requirement.
[0061] In some embodiments, the first speech parameter belongs to a first parameter range, the second speech parameter belongs to a second parameter range, and the minimum value in the second parameter range is greater than the maximum value in the first parameter range.
[0062] In some embodiments, the speech parameter includes at least one of a bit rate and a sampling rate.
[0063] In some embodiments, the mixed stream configuration is determined based at least on a role of the first user in the target online conference and / or roles of multiple second users in the target online conference.
[0064] The units and / or modules included in the device 500 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the units and / or modules in the device 500 can be implemented at least in part by one or more hardware logic components. As an example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0065] It should be understood that one or more steps in the above method can be performed by a suitable electronic device or a combination of electronic devices. Such an electronic device or a combination of electronic devices may include, for example, Figure 1 A mixed flow system 110 in FIG.
[0066] Figure 6 1 shows a block diagram of an electronic device 600 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 6 The electronic device 600 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 6 The electronic device 600 shown can be used to implement Figure 1 Mixed flow system 110 or Figure 5 Device 500.
[0067] like Figure 6 As shown, the electronic device 600 is in the form of a general electronic device. The components of the electronic device 600 may include, but are not limited to, one or more processors or processing units 610, a memory 620, a storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processing unit 610 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 620. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 600.
[0068] The electronic device 600 typically includes a plurality of computer storage media. Such media can be any available media accessible to the electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 620 can be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory) or some combination thereof. The storage device 630 can be a removable or non-removable medium, and can include a machine-readable medium, such as a flash drive, a disk, or any other medium, which can be used to store information and / or data and can be accessed within the electronic device 600.
[0069] The electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 6As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 620 may include a computer program product 625 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.
[0070] The communication unit 640 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 600 can be implemented with a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 600 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.
[0071] The input device 650 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 660 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 600 may also communicate with one or more external devices (not shown) through the communication unit 640 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 600, or communicate with any device that allows the electronic device 600 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).
[0072] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.
[0073] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0074] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0075] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0076] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.
[0077] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.
Claims
1. A method for voice mixing, comprising: Receiving a mixing request from a first user in a target user group associated with a target online conference, the mixing request at least including a mixing configuration, the mixing configuration indicating that mixing is performed on voice streams of a plurality of second users in the target user group except the first user; Determining voice parameters for the target online conference; Based on the mixing configuration and the voice parameters, performing mixing on the voice streams of the plurality of second users in the target user group to obtain mixed voice; as well as The mixed stream voice is provided at least to a client corresponding to the first user, so that the mixed stream voice is played on the client.
2. The method according to claim 1, wherein determining the voice parameters for the target online conference comprises: Based on the voice quality requirement related to the target online conference, a voice parameter for the target online conference is determined.
3. The method according to claim 2, wherein determining the voice parameters for the target online conference based on the voice quality requirement comprises: In response to the voice quality requirement indicating a first voice quality requirement, determining the voice parameter indicating a first voice parameter for the target online conference; as well as In response to the voice quality requirement indicating a second voice quality requirement, determining that the voice parameter for the target online conference indicates a second voice parameter, the second voice quality requirement being higher than the first voice quality requirement.
4. The method according to claim 3, wherein the first speech parameter belongs to a first parameter range, the second speech parameter belongs to a second parameter range, and the minimum value in the second parameter range is greater than the maximum value in the first parameter range. The method according to claim 1 , wherein the speech parameter comprises at least one of a bit rate and a sampling rate.
6. The method according to claim 1, wherein the mixed flow configuration is determined based at least on a role of the first user in the target online conference and / or roles of the plurality of second users in the target online conference.
7. A device for voice mixing, comprising: A receiving module is configured to receive a mixing request of a first user in a target user group associated with a target online conference, wherein the mixing request at least includes a mixing configuration, and the mixing configuration indicates that mixing is performed on voice streams of a plurality of second users in the target user group except the first user; A determination module, configured to determine a voice parameter for the target online conference; an execution module, configured to perform mixing on the voice streams of the plurality of second users in the target user group based on the mixing request and the voice parameter to obtain mixed voice; as well as The providing module is configured to provide the mixed stream voice at least to a client corresponding to the first user, so as to play the mixed stream voice on the client.
8. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 6.
10. A computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Voice processing method, device and equipment and storage medium
CN112019488A
Audio and video quality adjusting method and device, electronic equipment and storage medium
CN113347472A
Data processing method and device, equipment and storage medium
CN116546262A
Method, device and system for extensive live broadcast voice talkback
CN117560358A
Communication method and device for conference and conference system
CN118101885A
