Multi-person voice communication methods, devices, equipment, and media in live streaming rooms

Abstract

This application discloses a method, apparatus, device, and medium for multi-person voice communication in a live streaming room. The method includes: establishing a direct communication link with viewer nodes corresponding to the direct connection node information broadcast by a media server; when the number of forwards of the broadcaster's voice audio data and the uplink bandwidth meet corresponding preset conditions, comparing the direct connection latency of one or more voice viewer nodes that have established direct communication links with the media server latency, and determining the voice viewer nodes whose direct connection latency does not exceed the media server latency; determining the direct connection transmission capability of each voice viewer node based on the direct connection transmission characteristic information obtained through the direct connection communication links of each voice viewer node, and transmitting the broadcaster's voice audio data with the voice viewer node with the better direct connection transmission capability through the direct communication link. This application pushes the broadcaster's voice audio through a direct communication network, reducing voice latency in multi-person voice live streaming rooms and improving the user's live streaming voice experience.

Description

Technical Field

[0001] This application relates to the field of live streaming, and more particularly to a method for multi-person voice communication in a live streaming room. It also relates to the corresponding apparatus, equipment, and non-volatile storage medium. Background Technology

[0002] Existing internet live streaming platforms typically offer live audio services in their live streaming rooms, allowing broadcasters and viewers to communicate via voice chat. This enhances the interactive experience between broadcasters and viewers. These live audio services generally receive the broadcaster's audio data from a server cluster and then push it to viewers via a media server. However, because the audio data needs to be forwarded multiple times before reaching the viewers' devices for playback, significant audio latency is a common problem. This substantial latency prevents effective communication between users, severely impacting the live audio experience provided by the service.

[0003] In view of the problem of audio data transmission delay in the live voice online service of existing live streaming platforms, the applicant has made corresponding explorations in order to solve this problem. Summary of the Invention

[0004] The purpose of this application is to provide a method for multi-person voice communication in a live streaming room to meet user needs. It also relates to the corresponding apparatus, equipment, non-volatile storage medium, and computer program products.

[0005] To achieve the objectives of this application, the following technical solution is adopted:

[0006] A method for multi-person voice communication in a live streaming room, proposed for the purposes of this application, includes the following steps:

[0007] Based on the direct connection node information broadcast by the media server, a direct communication link is established with the audience node corresponding to the direct connection node information. The direct connection node information is pushed to the media server by the voice audience node that newly enters the current multi-person voice live broadcast room.

[0008] When the number of times the anchor's audio data is forwarded and the uplink bandwidth meet their respective preset conditions, the direct connection latency of one or more audio audience nodes that have established the direct connection communication link is compared with the media server latency to determine the target audio audience node whose direct connection latency does not exceed the media server latency.

[0009] Based on the direct transmission characteristic information obtained through the direct communication links of each target audio audience node, the direct transmission capability of each target audio audience node is determined, and the broadcaster's audio data is transmitted to the target audio audience node with the better direct transmission capability through the established direct communication link.

[0010] In a further embodiment, after the step of transmitting the broadcaster's voice audio data to the target voice audience node with superior direct connection transmission capability through the established direct communication link, the following steps are included:

[0011] In response to a direct connection packet loss event, obtain the event trigger time of the direct connection packet loss event;

[0012] If the time difference between the time trigger time and the current time exceeds a preset duration, and if it does, and no retransmission packet of the anchor voice audio direct transmission node corresponding to the direct connection packet loss event is obtained, then the data packet lost by the direct connection packet loss event is obtained from the media server.

[0013] In a further embodiment, before the step of determining the direct connection node information broadcast by the media server, the following steps are included:

[0014] Respond to the voice start event applied to the current multi-person voice live streaming room, and obtain the voice audio data pushed by the host node in the current multi-person voice live streaming room;

[0015] The broadcaster's audio data is pushed to one or more audio viewer nodes in the current multi-person audio live broadcast room;

[0016] In response to a broadcaster audio direct connection subscription event of a certain audio viewer node, stop pushing the broadcaster's audio data to the audio viewer node;

[0017] In response to a broadcaster's audio direct connection unsubscribe event at a certain audio viewer node, the broadcaster's audio data is pushed to the audio viewer node.

[0018] In a further embodiment, the step of comparing the direct connection latency of one or more voice audience nodes that have established the direct connection communication link with the media server when the direct connection node information is pushed to the media server by newly entering the current multi-person voice live broadcast room or when the number of forwardings of the anchor's voice audio data and the uplink bandwidth meet their respective preset conditions includes the following steps:

[0019] In response to the voice join event in the current multi-person voice live broadcast room, push its own direct connection node information to the media server;

[0020] Based on one or more other direct connection node information pushed by the media server, establish corresponding direct communication links with the voice audience node or voice broadcaster node corresponding to each of the other direct connection node information.

[0021] Obtain the anchor audio subscription status pushed by the anchor audio direct transmission node through the established direct communication link with it, and determine the anchor audio subscription status as a target anchor audio direct transmission node that can be subscribed to;

[0022] The direct connection latency between the target broadcaster's voice and audio transmission node is determined, and the corresponding direct connection latency and media server latency are pushed through the direct communication link established with the target broadcaster's voice and audio transmission node.

[0023] In a further embodiment, the step of determining when the number of times the broadcaster's audio data is forwarded and the uplink bandwidth meet their respective preset conditions includes the following steps:

[0024] Determine the number of times the broadcaster's voice audio data is forwarded, and determine whether the number of forwarded broadcaster's voice audio data is lower than a preset limit for forwarding.

[0025] Determine whether the uplink bandwidth of the current node exceeds the preset minimum uplink bandwidth;

[0026] When the number of times the broadcaster's voice audio data is forwarded is lower than the limit number of forwardings, and the uplink bandwidth exceeds the minimum uplink bandwidth, the broadcaster audio subscription status, which is indicated as subscribing, is pushed through the direct communication link already established with the voice audience node that does not have the broadcaster's voice audio data.

[0027] When the number of times the broadcaster's voice audio data is forwarded is not less than the limit number of forwards, or the uplink bandwidth does not exceed the minimum uplink bandwidth, the broadcaster's audio subscription status, which is characterized as unsubscribeable, is pushed through the direct communication link established with the voice audience node that does not have the broadcaster's voice audio data.

[0028] In a further embodiment, the step of determining the direct transmission capability of each target voice audience node based on the direct transmission feature information obtained through the direct communication link of each target voice audience node includes the following steps:

[0029] By establishing direct communication links with each of the target voice audience nodes, the direct transmission feature information pushed by each of the target voice audience nodes is obtained, and the direct transmission feature information includes the latest node packet loss rate and the latest direct connection latency.

[0030] Based on the preset transmission capability algorithm, the latest node packet loss rate and the latest direct connection delay contained in each of the direct connection transmission feature information are weighted and calculated to determine the direct connection transmission capability of each of the target voice audience nodes.

[0031] In a further embodiment, the step of transmitting the broadcaster's voice audio data to the target voice audience node with superior direct transmission capability through the established direct communication link includes the following steps:

[0032] Based on the direct transmission capability of each target voice audience node, the target voice audience nodes are ordered sequentially.

[0033] Determine the difference between the number of times the broadcaster's audio data is forwarded and the limit on the number of forwards, and establish a direct broadcaster audio subscription with one or more target audio audience nodes that are ranked high and whose ranking is within the range of the difference.

[0034] The currently available broadcaster audio data is pushed through the direct communication link between the target audio viewer node and the established broadcaster audio direct connection subscription.

[0035] A live streaming multi-person voice device for the purposes of this application includes:

[0036] The direct connection communication establishment module is used to establish a direct connection communication link with the audience node corresponding to the direct connection node information broadcast by the media server. The direct connection node information is pushed to the media server by the voice audience node that newly enters the current multi-person voice live broadcast room.

[0037] The target node determination module is used to determine the target voice audience node whose direct connection latency does not exceed the media server latency by comparing the direct connection latency of one or more voice audience nodes that have established the direct connection communication link with the media server latency when the number of forwarding times of the broadcaster's voice audio data and the uplink bandwidth meet their respective preset conditions.

[0038] The audio direct connection push module is used to determine the direct connection transmission capability of each target audio audience node based on the direct connection transmission feature information obtained through the direct connection communication link of each target audio audience node, and to transmit the broadcaster's audio data with the target audio audience node with better direct connection transmission capability through the established direct connection communication link.

[0039] In a further embodiment, the direct communication establishment module includes:

[0040] The direct connection node information push submodule is used to respond to the voice join event in the current multi-person voice live broadcast room and push its own direct connection node information to the media server;

[0041] The direct communication link establishment submodule is used to establish a corresponding direct communication link with the voice audience node or voice broadcaster node corresponding to each of the other direct connection node information pushed by the media server.

[0042] The audio subscription status acquisition submodule is used to acquire the anchor audio subscription status pushed by the anchor audio direct transmission node through the direct communication link established with it, and determine the anchor audio subscription status as a target anchor audio direct transmission node that can be subscribed to;

[0043] The latency data push submodule is used to determine the direct connection latency between the target broadcaster's voice and audio direct transmission node and push the corresponding direct connection latency and media server latency through the direct communication link established with the target broadcaster's voice and audio direct transmission node.

[0044] In a further embodiment, the target node determination module includes:

[0045] The audio forwarding count verification submodule is used to determine the number of times the broadcaster's audio data has been forwarded and to determine whether the number of times the broadcaster's audio data has been forwarded is lower than the preset limit number of forwardings.

[0046] The uplink bandwidth verification submodule is used to determine whether the uplink bandwidth of the current node exceeds the preset minimum uplink bandwidth.

[0047] The subscribing status determination submodule is used to push the subscribing status of the broadcaster's audio data, which is characterized as subscribing, through a direct communication link established with a voice audience node that does not have the broadcaster's audio data when the number of forwardings of the broadcaster's audio data is lower than the limit number of forwardings and the uplink bandwidth exceeds the minimum uplink bandwidth.

[0048] The unsubscribeable state determination submodule is used to push the unsubscribeable broadcaster audio subscription status through a direct communication link established with a voice audience node that does not have the broadcaster's voice audio data when the number of forwards of the broadcaster's voice audio data is not less than the limit number of forwards, or the uplink bandwidth does not exceed the minimum uplink bandwidth.

[0049] In a further embodiment, the audio direct push module includes:

[0050] The direct transmission feature acquisition submodule is used to acquire the direct transmission feature information pushed by each of the target voice audience nodes through the established direct communication links with each of the target voice audience nodes. The direct transmission feature information includes the latest node packet loss rate and the latest direct connection delay.

[0051] The direct transmission capability determination submodule is used to perform weighted calculations on the latest node packet loss rate and the latest direct transmission delay contained in each of the direct transmission feature information according to a preset transmission capability algorithm, and to determine the direct transmission capability of each of the target voice audience nodes.

[0052] In a preferred embodiment, the audio direct push module further includes:

[0053] The target node sorting submodule is used to sort the target voice audience nodes in order based on their direct transmission capabilities.

[0054] The audio direct subscription establishment submodule is used to determine the difference between the number of times the broadcaster's audio data is forwarded and the limit number of forwardings, and to establish a broadcaster audio direct subscription with one or more target audio audience nodes that are ranked first and whose ranking is within the range of the difference.

[0055] The anchor audio direct connection push submodule is used to push the currently available anchor audio data through the direct communication link of the target voice audience node that has established an anchor audio direct connection subscription.

[0056] To address the aforementioned technical problems, this application also provides a computer device, including a memory and a processor. The memory stores computer-readable instructions, which, when executed by the processor, cause the processor to perform the steps of the aforementioned live-streaming multi-person voice method.

[0057] To address the aforementioned technical problems, this application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the aforementioned live-streaming multi-person voice method.

[0058] To address the aforementioned technical problems, this application also provides a computer program product, including a computer program and computer instructions. When the computer program and computer instructions are executed by a processor, the processor performs the steps of the aforementioned live-streaming multi-person voice method.

[0059] Compared with existing technologies, the advantages of this application are as follows:

[0060] This application constructs a direct-connection communication network architecture for directly transmitting broadcaster audio data. Applied to the live audio service of a live streaming platform, this architecture improves the data transmission rate of broadcaster audio data, ensuring that viewers can quickly obtain broadcaster audio data via direct connection. It also reduces audio latency between broadcasters and viewers, enhancing the user experience. Users participating in the live audio service forward their direct-connection node information to other participating nodes via a media server. This allows each node in the live audio service to build a direct-connection communication network. Broadcaster nodes or viewers obtaining broadcaster audio data directly through this network act as direct-transmission nodes, which can be used by other viewer nodes in the network. The host audio direct connection subscription node, and in the direct communication network, the host audio direct transmission node and other viewer audio nodes apply for the host audio direct connection subscription through the established direct communication link between them. The host audio direct transmission node obtains the latency data and direct transmission characteristic information of the viewer audio nodes that have applied for subscription through the direct communication link to screen the nodes, so as to select the viewer audio nodes whose direct connection latency with itself is lower than the latency with the media server. Based on the direct transmission characteristic information of each viewer audio node, the node further determines the viewer audio nodes with strong direct transmission capabilities, establishes a host audio direct connection subscription with these viewer audio nodes, and transmits its own host audio data through the established direct communication link with these viewer audio nodes. In this way, the latency of these viewer audio nodes to obtain host audio data is reduced by direct transmission, thereby improving the live audio experience of the viewer users participating in the live audio online service in the live room. Attached Figure Description

[0061] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0062] Figure 1 A typical network deployment architecture diagram related to the implementation of the technical solution of this application;

[0063] Figure 2 This is a flowchart illustrating a typical embodiment of the live streaming multi-person voice communication method of this application;

[0064] Figure 3 This is a schematic diagram of the direct communication network for transmitting broadcast voice and audio data as described in this application;

[0065] Figure 4This is a flowchart illustrating the specific implementation method of the direct transmission of broadcaster voice and audio data in this application, in which one party obtains the broadcaster's voice and audio data from the media server after the direct retransmission fails following a direct packet loss event between the two nodes.

[0066] Figure 5 This is a flowchart illustrating the specific implementation method of the live audio service opened in a multi-person voice live streaming room as described in this application.

[0067] Figure 6 This is a flowchart illustrating the specific implementation method executed after a viewer node participates in a live audio service launched in a multi-person voice live streaming room, as described in this application.

[0068] Figure 7 This is a flowchart illustrating the specific implementation method of the present application regarding the broadcaster's audio direct transmission node determining whether it has the ability to establish a broadcaster's audio direct connection subscription with other audio viewer nodes.

[0069] Figure 8 This is a flowchart illustrating the specific implementation method of determining the direct transmission capability of each audio viewer node in the direct audio transmission node of the broadcaster's voice in this application.

[0070] Figure 9 This is a flowchart illustrating the specific implementation method for establishing a direct connection subscription for broadcaster audio data based on the direct transmission capability of the broadcaster's audio data transmission node and the number of broadcaster audio data forwardings in this application.

[0071] Figure 10 This is a schematic block diagram of a typical embodiment of the live streaming multi-person voice device of this application;

[0072] Figure 11 This is a basic structural block diagram of a computer device according to an embodiment of this application. Detailed Implementation

[0073] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0074] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0075] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0076] Those skilled in the art will understand that the terms "user terminal," "terminal," and "terminal device" as used herein include both devices that receive wireless signals, devices that only possess wireless signal receiver capabilities without transmission capabilities, and devices with receiving and transmitting hardware, devices that have receiving and transmitting hardware capable of bidirectional communication over a bidirectional communication link. Such devices may include: cellular or other communication devices such as personal computers or tablets, having single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant) that may include a radio frequency receiver, pager, internet / intranet access, web browser, notepad, calendar, and / or GPS (Global Positioning System) receiver; and traditional laptop and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "user terminal," "terminal," and "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally and / or in a distributed manner, operating in any other location on Earth and / or in space. "User terminal," "terminal," and "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.

[0077] The hardware referred to by the names "server," "client," and "working node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann architecture, such as a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in the secondary storage into the main memory to run it, execute the instructions in the program, and interact with the input and output devices to complete specific functions.

[0078] It should be noted that the concept of "server" used in this application can also be extended to the case of server clusters. Based on the network deployment principles understood by those skilled in the art, the servers should be logically divided. Physically, these servers can be independent of each other but accessible through interfaces, or they can be integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method in this application.

[0079] Please see Figure 1 The hardware infrastructure required for implementing the technical solutions of this application can be deployed according to the architecture shown in the figure. The server 80 mentioned in this application is deployed in the cloud and acts as an online server. It can further connect to relevant data servers and other servers providing related support, thereby forming a logically related service cluster to provide services to relevant terminal devices such as the smartphone 81 and personal computer 82 shown in the figure, or third-party servers (not shown). Both the smartphone and personal computer can access the Internet through known network access methods and establish a data communication link with the cloud server 80 to run terminal applications related to the services provided by the server.

[0080] For servers, the application is usually built as a service process, with corresponding program interfaces exposed for remote calls by applications running on various terminal devices. The relevant technical solutions in this application that are suitable for running on servers can be implemented in servers in this way.

[0081] The application mentioned refers to an application running on a server or terminal device. This application implements the relevant technical solutions of this application in a programmed manner. Its program code can be stored in a non-volatile storage medium that can be recognized by a computer in the form of computer-executable instructions, and is loaded into memory by the central processing unit for execution. The relevant device of this application is constructed by the operation of the application on the computer.

[0082] For servers, the application is usually built as a service process, with corresponding program interfaces exposed for remote calls by applications running on various terminal devices. The relevant technical solutions in this application that are suitable for running on servers can be implemented in servers in this way.

[0083] Those skilled in the art will understand that although the various methods in this application are described based on the same concept and thus present commonality among them, they can be performed independently unless otherwise specified. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept; therefore, concepts expressed in the same way, as well as concepts that are appropriately changed for convenience but are expressed differently, should be understood equivalently.

[0084] Please see Figure 2 The present application discloses a method for multi-person voice communication in a live streaming room, which, in a typical embodiment, includes the following steps:

[0085] Step S11: Based on the direct connection node information broadcast by the media server, establish a direct communication link with the viewer node corresponding to the direct connection node information. The direct connection node information is pushed to the media server by the newly entered voice viewer node in the current multi-person voice live broadcast room.

[0086] The aforementioned multi-person voice live streaming room refers to a live streaming room on an internet live streaming platform that has enabled live voice online service. After the host node in the current live streaming room enables the live voice online service, a voice activation event will be triggered, and the current live streaming room will switch to the current multi-person voice live streaming room with the aforementioned live voice online service. The live voice online service enabled by the multi-person voice live streaming room generally opens multiple voice qualifications that can be used to conduct live voice communication with the host node. Viewer nodes in the multi-person voice live streaming room can apply for the opened voice qualifications to participate in the enabled live voice online service, thereby transforming themselves from viewer nodes into voice viewer nodes, and then conducting live voice communication with the host node and other voice viewer nodes in the multi-person voice live streaming room.

[0087] The broadcaster node pushes its broadcaster voice audio data to the media server, so that the media server can push the broadcaster voice audio data to the voice audience nodes participating in the current multi-person voice live broadcast service for playback.

[0088] The media server is used to forward the broadcaster's voice and audio data in the current live broadcast room. After responding to the voice start event in the current multi-person voice live broadcast room, the media server will obtain the broadcaster's voice and audio data pushed by the broadcaster node in the current multi-person voice live broadcast room, and forward the broadcaster's voice and audio data to one or more of the voice audience nodes participating in the live broadcast voice online service in the current live broadcast room. Of course, in addition to forwarding the broadcaster's voice and audio data, the media server will also forward the audience voice and audio data of each of the voice audience nodes. The media server receives the audience voice and audio data pushed by each of the voice audience nodes in the current live broadcast room, and forwards each audience voice and audio data to each of the voice audience nodes and broadcaster nodes participating in the live broadcast voice online service, so that each of the voice audience nodes and broadcaster nodes can play the received audience voice and audio data.

[0089] The direct communication link mentioned above refers to the direct communication link established between the voice nodes (host nodes and voice audience nodes) participating in the live voice service in the current multi-person voice live room. As understood in the prior art, it generally refers to the voice nodes participating in the live voice service communicating in a P2P (peer-to-peer) manner. In this method, the voice nodes participating in the live voice service can transmit the host's voice audio data through the direct communication link established between them. However, the voice nodes transmitting the host's voice audio data through the direct communication link need to undergo corresponding node screening to ensure that the acquisition latency of the voice nodes that obtain the host's voice audio data through the direct communication link is less than the acquisition latency of obtaining the host's voice audio data from the media server. The specific implementation method of node screening will be described later.

[0090] The direct connection node information in the media server is pushed to the media server by the voice viewer nodes and broadcaster nodes participating in the live audio online service in the current live room. The voice viewer nodes and broadcaster nodes participating in the live audio online service in the current live room will receive all the direct connection node information in the current live room pushed by the media server, and establish the direct communication link with the voice viewer nodes and broadcaster nodes corresponding to each direct connection node information according to each direct connection node information. For new voice viewer nodes newly participating in the live audio online service in the current live room, the new voice viewer node pushes its own direct connection node information to the media server so that the media server forwards the direct connection node information to other voice viewer nodes and broadcaster nodes that have participated in the live audio online service.

[0091] Step S12: When the number of times the broadcaster's audio data is forwarded and the uplink bandwidth meet their respective preset conditions, compare the direct connection latency of one or more audio viewer nodes that have established the direct communication link with the media server latency to determine the target audio viewer node whose direct connection latency does not exceed the media server latency.

[0092] The executor of this step is generally the host node in the current multi-person voice live broadcast room, or a voice audience node that obtains the host's voice and audio data through the direct communication link established with the host node. The host node and the voice audience node can establish a direct communication network with other voice audience nodes in the current multi-person voice live broadcast room to transmit the host's voice and audio data through the direct communication link, so as to reduce the latency for other voice audience nodes to obtain the host's voice and audio data, that is, to reduce the voice communication latency between other voice audience nodes and the host node, and optimize the voice experience of the live voice online service. For the convenience of the following description, the executor of this step, that is, the host node in the current multi-person voice live broadcast room, or the voice audience node that obtains the host's voice and audio data through the direct communication link established with the host node, will be referred to as the host voice and audio direct transmission node.

[0093] The broadcaster audio direct transmission node needs to detect whether the number of times the broadcaster audio data is forwarded and the node's own uplink bandwidth meet their respective preset conditions. This is to determine whether the node can be subscribed to by other audio viewer nodes to transmit the broadcaster audio data through the established direct communication link. Specifically, the broadcaster audio direct transmission node determines the number of times the broadcaster audio data is forwarded and whether the number of forwarded broadcaster audio data is lower than a preset limit. It also determines whether the current node's uplink bandwidth exceeds a preset minimum uplink bandwidth. When the number of forwarded broadcaster audio data is lower than the limit... When the number of forwards exceeds the limit and the uplink bandwidth exceeds the minimum uplink bandwidth, a subscribed status for the broadcaster's audio data is pushed through a direct communication link established with a voice audience node that does not have the broadcaster's audio data. This allows the voice audience node to push a broadcaster's audio subscription request to the broadcaster's audio direct transmission node and request the broadcaster's audio data to be obtained through the established direct communication link. When the number of forwards of the broadcaster's audio data is not less than the limit number of forwards, or the uplink bandwidth does not exceed the minimum uplink bandwidth, a subscribed status for the broadcaster's audio data is pushed through a direct communication link established with a voice audience node that does not have the broadcaster's audio data.

[0094] Please refer to Figure 3 The aforementioned number of times the broadcaster's audio data is forwarded generally refers to the number of times the broadcaster's audio data in a multi-person audio live broadcast room is forwarded from the broadcaster node to other audio viewer nodes. The number of times the broadcaster's audio data is forwarded will be counted starting from the broadcaster node. Figure 3 As shown, the number of times the broadcaster's voice and audio data is forwarded in the direct communication network 301 is 2, and the number of times the broadcaster's voice and audio data is forwarded in the direct communication network 302 is 3. It can be understood that the number of times the broadcaster's voice and audio data is forwarded is counted starting from the broadcaster node, not from the broadcaster's voice and audio data direct transmission node that is currently performing the detection of whether the number of times the broadcaster's voice and audio data is forwarded is lower than the preset limit number of forwarding times. Of course, this does not apply when the broadcaster's voice and audio data direct transmission node is a broadcaster node.

[0095] The uplink bandwidth refers to the upload speed of the broadcaster's audio transmission node, that is, the rate at which the broadcaster node and the audio viewer node in the current live broadcast room obtain data from the broadcaster's audio transmission node.

[0096] The voice audience nodes participating in the current multi-person voice live broadcast room establish the direct communication link with one or more of the anchor voice and audio direct transmission nodes in the current live broadcast room. After obtaining the anchor audio subscription status through the direct communication link established with each anchor voice and audio direct transmission node, the target anchor voice and audio direct transmission node represented by the anchor audio subscription status is determined. Then, the direct connection latency between the node itself and the target anchor voice and audio direct transmission node is determined. The direct connection latency and media server latency are pushed through the direct communication link established with the target anchor voice and audio direct transmission node.

[0097] As described above, when the anchor audio subscription status of the anchor audio direct transmission node is subscribed, that is, when the anchor audio direct transmission node detects that the number of times the anchor audio data is forwarded and the uplink bandwidth meet their respective preset conditions, the anchor audio direct transmission node will receive the direct connection latency and media server latency pushed by other audio viewer nodes in the current live room through the direct connection communication link. The direct connection latency refers to the data acquisition latency of the audio viewer node and the anchor audio direct transmission node through the established direct connection communication link, and the media server latency refers to the acquisition latency of the audio viewer node from the media server. After obtaining the direct connection latency and media server latency corresponding to each audio viewer node, the anchor audio direct transmission node will compare the direct connection latency and media server latency of each audio viewer node to determine the target audio viewer node whose direct connection latency does not exceed its own media server latency.

[0098] In one embodiment, the entity performing the latency comparison between the direct connection latency and the media server latency can be the voice audience node itself, which obtains the broadcaster's voice and audio data without establishing a direct communication link with the broadcaster node, rather than the broadcaster's voice and audio direct transmission node in a multi-person voice live broadcast room. The voice audience node compares its own direct connection latency with each subscribed broadcaster's voice and audio direct transmission node with its own media server latency, determines one or more broadcaster's voice and audio direct transmission nodes whose direct connection latency does not exceed the media server latency, and then pushes broadcaster audio subscription requests to these broadcaster's voice and audio direct transmission nodes.

[0099] Step S13: Based on the direct transmission feature information obtained through the direct communication links of each target audio audience node, determine the direct transmission capability of each target audio audience node, and transmit the broadcaster's audio data with the target audio audience node with the better direct transmission capability through the established direct communication link.

[0100] The broadcaster's audio direct transmission node obtains the direct transmission characteristic information of each target audio viewer node through the direct communication link established with one or more target audio viewer nodes whose direct connection latency does not exceed the media server latency. The direct connection characteristic information includes the latest node packet loss rate and direct connection latency. The node packet loss rate refers to the packet loss rate of data transmitted through the direct communication link established between the target audio viewer node and the broadcaster's audio direct transmission node.

[0101] After obtaining the direct transmission feature information of each target audio viewer node, the broadcaster audio direct transmission node will compete for subscription nodes based on the direct transmission feature information of each target audio viewer node. Specifically, the broadcaster audio direct transmission node will perform a weighted calculation on the latest node packet loss rate and the latest direct connection delay contained in each direct transmission feature information according to a preset transmission capability algorithm, and determine the direct transmission capability of each target audio viewer node accordingly. The transmission capability algorithm generally has weight values ​​that act on the node packet loss rate and the direct connection delay respectively. Based on the weight values ​​corresponding to the node packet loss rate and the direct connection delay, the direct transmission capability of the corresponding target audio viewer is determined.

[0102] After the anchor audio direct transmission node determines the direct transmission capability corresponding to each of the target audio audience nodes, it will generally determine the target audio audience node with the largest value represented by the direct transmission capability among the target audio audience nodes. The direct audio transmission node will establish an anchor audio direct subscription with the target audio audience node to transmit the anchor audio data through the direct communication link established with the target audio audience node.

[0103] The number of nodes established for direct audio subscription through the direct transmission capabilities of each target audio viewer node can be correlated with the number of times the broadcaster's audio data is forwarded. That is, each direct audio transmission node can establish a direct audio subscription with multiple target audio viewer nodes that have strong direct transmission capabilities. Figure 3 The broadcast node in the direct communication network 302 shown establishes a direct broadcast audio subscription with voice audience node A and voice audience node B respectively. Of course, this will be related to the number of times the broadcast voice audio data is forwarded. Specifically, the broadcast voice audio direct transmission node sorts the target voice audience nodes according to their direct transmission capabilities and determines the difference between the number of times the broadcast voice audio data is forwarded and the limit number of forwards. Then, it establishes a direct broadcast audio subscription with one or more target voice audience nodes that are ranked higher and whose ranking is within the range of the difference, so as to push the broadcast voice audio data currently possessed by the broadcast voice audio direct transmission node through the direct communication link established with these target voice audience nodes.

[0104] For a voice viewer node that has established a direct audio subscription with the broadcaster's audio transmission node to obtain broadcaster audio data through the direct communication link established by both parties, the voice viewer node generally still maintains a data communication link connection with the media server. When the voice viewer node is unable to obtain the latest broadcaster audio data through the direct communication link established with the broadcaster's audio transmission node due to packet loss or excessive latency, it switches to the data communication link maintained with the media server to re-obtain the latest broadcaster audio data, ensuring that the voice viewer node does not experience interruption in obtaining broadcaster audio data due to problems with the direct communication link used to obtain broadcaster audio data.

[0105] As can be seen from the typical implementation of this method, this method constructs a direct communication network architecture for direct transmission of broadcaster voice and audio data. Applied to the live audio service of a live streaming platform, this improves the data transmission rate of broadcaster voice and audio data, ensuring that viewers can quickly obtain broadcaster voice and audio data via direct connection. This reduces voice latency between broadcasters and viewers, enhancing the user experience. Users participating in the live audio service forward their direct connection node information to other participating nodes through a media server. This allows each node in the live audio service to build a direct communication network. Broadcasters or viewers who obtain broadcaster voice and audio data directly through this network act as direct transmission nodes, which can be accessed by other nodes in the network. Viewer voice nodes apply for direct audio subscription to the broadcaster. In the direct communication network, the broadcaster voice and audio direct transmission node and other viewer voice nodes apply for direct audio subscription through the established direct communication link between them. The broadcaster voice and audio direct transmission node obtains the latency data and direct transmission characteristic information of the voice viewer nodes that have applied for subscription through the direct communication link to screen the nodes. This is to select viewer voice nodes whose direct connection latency is lower than that with the media server. Based on the direct transmission characteristic information of each viewer voice node, the node further identifies viewer voice nodes with strong direct transmission capabilities and establishes direct audio subscription with these viewer voice nodes. The node then transmits its own broadcaster voice and audio data through the established direct communication link with these viewer voice nodes. This reduces the latency for these viewer voice nodes to obtain broadcaster voice and audio data through direct transmission, thereby improving the live audio experience for viewers participating in the live audio online service in the live room.

[0106] The above typical embodiments and their variations fully disclose the implementation scheme of the live streaming multi-person voice method of this application. However, various variations of the method can still be derived by changing and amplifying some technical means. Other embodiments are briefly described below:

[0107] In one embodiment, please refer to Figure 4 After the step of transmitting the broadcaster's voice audio data to the target voice audience node with superior direct transmission capability through the established direct communication link, the following steps are included:

[0108] Step S14: Respond to the direct connection packet loss event and obtain the event trigger time of the direct connection packet loss event.

[0109] The aforementioned direct connection packet loss event refers to a packet loss event that occurs when the voice viewer node and the broadcaster's voice and audio direct transmission node transmit broadcaster voice and audio data through a direct communication link established by both parties. When the voice viewer node triggers the direct connection packet loss event by obtaining broadcaster voice and audio data through the direct communication link established with the broadcaster's voice and audio direct transmission node, the voice viewer node will respond to the direct connection packet loss event by re-pushing the retransmission packet corresponding to the direct connection packet loss event through the direct communication link. In addition, the voice viewer node responds to the direct connection packet loss event by obtaining the event trigger time of the direct connection packet loss event.

[0110] Step S15: Detect whether the time difference between the time trigger time and the current time exceeds a preset duration. If it does, and no retransmission packet of the anchor's voice audio direct transmission node corresponding to the direct connection packet loss event is obtained, then retrieve the data packet lost by the direct connection packet loss event from the media server.

[0111] When the audio viewer node detects that the time difference between the time trigger time and the current time exceeds a preset duration, and the audio viewer node still fails to obtain the retransmission packet through the direct communication link established with the broadcaster's audio-video direct transmission node, the audio viewer node will obtain the data packet lost in the direct connection packet loss event from the media server. The data packet contains the broadcaster's audio-video data that was lost in the direct connection packet loss event and could not be directly pushed to the audio viewer node.

[0112] In this embodiment, when packet loss occurs between the two parties transmitting the broadcaster's voice and audio data via a direct communication link, the voice audience node, which does not have the broadcaster's voice and audio data, can obtain the broadcaster's voice and audio data lost due to packet loss from the media server because it still maintains a connection with the media server, thus ensuring the voice stability of the voice audience node.

[0113] In one embodiment, please refer to Figure 5Before the step of obtaining the direct connection node information broadcast by the media server, the following steps are included:

[0114] Step S07: Respond to the voice start event acting on the current multi-person voice live streaming room, and obtain the host's voice audio data pushed by the host node in the current multi-person voice live streaming room:

[0115] The voice activation event refers to the event triggered when the live broadcast room starts the live voice online service. The media server responds to the voice activation event of the current multi-person voice live broadcast room and then obtains the voice audio data of the host pushed by the host node in the current multi-person voice live broadcast room.

[0116] Step S08: Push the anchor's voice audio data to one or more voice viewer nodes in the current multi-person voice live broadcast room:

[0117] After receiving the broadcaster's voice and audio data pushed by the broadcaster node in the current multi-person voice live broadcast room, the media server broadcasts the broadcaster's voice and audio data to the voice audience nodes participating in the currently activated live voice online service in the current multi-person voice live broadcast room, so that these voice audience nodes can obtain the live voice and audio data for playback.

[0118] Step S09: In response to a broadcaster audio direct subscription event of a certain audio viewer node, stop pushing the broadcaster's audio data to the audio viewer node.

[0119] The aforementioned direct audio subscription event refers to the event triggered when a voice viewer node in the current multi-person voice live broadcast room establishes a direct audio subscription with the broadcaster's voice and audio direct transmission node. When the media server responds to any of the voice viewer nodes in the current multi-person voice live broadcast room triggering the direct audio subscription event, the media server will stop pushing the broadcaster's voice and audio data to the voice viewer node, so that the voice viewer node can obtain the broadcaster's voice and audio data through the direct communication link with the broadcaster's voice and audio direct transmission node that has established the direct audio subscription.

[0120] Step S10: In response to a broadcaster audio direct connection unsubscribe event at a certain audio viewer node, push the broadcaster's audio data to the audio viewer node.

[0121] The aforementioned direct connection reconnection failure event refers to an event triggered when packet loss occurs during the transmission of the broadcaster's voice and audio data between the voice viewer node and the broadcaster's voice and audio direct transmission node via the direct communication link established by both parties, and the reconnection between the two parties fails. In response to the direct connection reconnection failure event triggered by the voice viewer node in the current live broadcast room, the media server will push the broadcaster's voice and audio data to the voice viewer node.

[0122] In this embodiment, the media server is responsible for pushing the broadcaster's audio data to the audio viewer nodes in the multi-person audio live broadcast room that have not subscribed to the broadcaster's audio direct transmission node. Once the audio viewer node in the multi-person audio live broadcast room establishes a direct audio connection subscription to the broadcaster, the push of the broadcaster's audio data to that audio viewer node will stop, so as to save the transmission pressure on the media server. Furthermore, if the audio viewer node fails to reconnect, it can switch to re-obtaining the broadcaster's audio data from the media server, ensuring that the audio viewer node can continuously obtain the broadcaster's audio data.

[0123] In one embodiment, please refer to Figure 6 The steps of comparing the direct connection latency of one or more voice viewer nodes that have established direct communication links with the media server, where the direct connection node information is pushed to the media server by newly entering the current multi-person voice live broadcast room, or when the number of forwardings of the anchor's voice audio data and the uplink bandwidth meet their respective preset conditions, include the following steps:

[0124] Step S111: In response to the voice join event in the current multi-person voice live broadcast room, push its own direct connection node information to the media server:

[0125] The aforementioned voice joining event refers to the event triggered when a regular viewer node in a multi-person voice live streaming room participates in the live voice online service. When a regular viewer node participates in the live voice online service that has been started in the current live streaming room and triggers the voice joining event, the regular viewer node will switch to a voice viewer node and push its own direct connection node information to the media server.

[0126] Step S112: Based on one or more other direct connection node information pushed by the media server, establish corresponding direct communication links with the voice audience node or voice broadcaster node corresponding to each of the other direct connection node information:

[0127] After the voice viewer node pushes its own direct connection node information to the media server, it can obtain other direct connection node information of one or more other voice viewer nodes and broadcaster nodes participating in the current live voice online service from the media server, and then establish corresponding direct communication links with each of the other voice viewer nodes and broadcaster nodes according to the other direct connection node information.

[0128] Step S113: Obtain the anchor audio subscription status pushed by the anchor audio direct transmission node through the established direct communication link, and determine the anchor audio subscription status as a target anchor audio direct transmission node that can be subscribed to.

[0129] Once the voice viewer node establishes a direct communication link with one or more of the anchor voice and audio direct transmission nodes in the current live broadcast room, the voice viewer node will obtain the anchor audio subscription status pushed by each of the anchor voice and audio direct transmission nodes through each direct communication link, determine the anchor audio subscription status that is subscribed to from each anchor audio subscription status, and determine the anchor voice and audio direct transmission node corresponding to these anchor audio subscription statuses as the target anchor voice and audio direct transmission node.

[0130] Step S114: Determine the direct connection latency between the target broadcaster's voice and audio direct transmission node, and push the corresponding direct connection latency and media server latency through the direct communication link established with the target broadcaster's voice and audio direct transmission node.

[0131] After determining that the target broadcaster's audio direct transmission node is available for subscription, the audio viewer node will determine the direct connection latency between itself and the target broadcaster's audio direct transmission node, as well as the media server latency between itself and the media server. Then, it will push the direct connection latency and the media server latency to the target broadcaster's audio direct transmission node through the direct connection communication link.

[0132] In this embodiment, after a viewer node participates in the live audio service launched in a multi-person voice live streaming room, it will push its own direct connection node information to the media server and obtain the direct connection node information of other nodes participating in the live audio service in the multi-person voice live streaming room. This allows the viewer node to establish a direct communication link with each node in the live audio service. After participating in the live audio service, the viewer node will obtain the subscription status of nodes that have obtained the broadcaster's audio data through direct connection, push its own direct connection latency and media server latency to subscribed nodes, and participate in the competition for nodes that obtain the broadcaster's audio data through direct connection. This allows viewer nodes participating in the live audio service to obtain the broadcaster's audio data through direct connection, reducing the latency caused by the broadcaster's audio data being relayed by the server.

[0133] In one embodiment, please refer to Figure 7 The step of determining when the number of times the broadcaster's audio data is forwarded and the uplink bandwidth meet their respective preset conditions includes the following steps:

[0134] Step S121: Determine the number of times the broadcaster's voice audio data has been forwarded, and determine whether the number of forwardings is lower than a preset limit.

[0135] The aforementioned limit on the number of forwardings is generally set within a value range of 2 or less to prevent excessive data transmission pressure on nodes that push the broadcaster's audio data via direct connection due to multiple forwardings, and to prevent significant data acquisition delays for subsequent audio audience nodes that receive the broadcaster's audio data via direct connection due to multiple forwardings. Of course, with technological advancements, those skilled in the art can flexibly design the aforementioned limit on the number of forwardings, and this step will not be elaborated upon here.

[0136] Step S122: Determine whether the uplink bandwidth of the current node exceeds the preset minimum uplink bandwidth.

[0137] The minimum uplink bandwidth is generally set in the range of 1Mbps to 3Mbps. When the uplink bandwidth of the current node exceeds the minimum uplink bandwidth, it indicates that the current node has the ability to transmit its own broadcast voice audio data through a direct communication link, so as to ensure the data transmission stability and transmission rate when the current node transmits broadcast voice audio data through a direct communication link established with other voice audience nodes. Of course, those skilled in the art can flexibly design the minimum uplink bandwidth, and this step will not be elaborated here.

[0138] Step S123: When the number of forwards of the broadcaster's voice audio data is lower than the limit number of forwards, and the uplink bandwidth exceeds the minimum uplink bandwidth, the broadcaster audio subscription status, which indicates that it is available for subscription, is pushed through the established direct communication link with the voice audience node that does not have the broadcaster's voice audio data.

[0139] When the current node detects that the number of times the broadcaster's audio data is forwarded is lower than the limit number of forwardings, and its own uplink bandwidth exceeds the minimum uplink bandwidth, it indicates that the current node, as a direct transmission node for broadcaster's audio data, has the ability to establish direct broadcaster audio subscription with other audio audience nodes. At this time, the current node will push the broadcaster audio subscription status, which indicates that it is available for subscription, to one or more audio audience nodes in the multi-person audio live broadcast room through the direct communication link established with them.

[0140] Step S124: When the number of forwards of the broadcaster's voice audio data is not less than the limit number of forwards, or the uplink bandwidth does not exceed the minimum uplink bandwidth, push the broadcaster audio subscription status, which is characterized as unsubscribing, through the established direct communication link with the voice audience node that does not have the broadcaster's voice audio data.

[0141] When the current node detects that the number of times the broadcaster's audio data is forwarded is not less than the limit number of forwards, and its own uplink bandwidth does not exceed the minimum uplink bandwidth, it indicates that the current node, as a direct transmission node for broadcaster's audio data, does not have the ability to establish a direct subscription to broadcaster's audio data with other audio audience nodes. At this time, the current node will push the broadcaster's audio subscription status, which indicates that it is not subscribed to, to one or more audio audience nodes in the multi-person audio live broadcast room through the direct communication link established with them.

[0142] In this embodiment, the node acting as the direct transmission node for the broadcaster's voice audio data determines whether it has the ability to establish a direct subscription to the broadcaster's audio data with other voice audience nodes by detecting whether the number of forwardings of the broadcaster's voice audio data and its own uplink bandwidth meet their respective preset conditions, so as to ensure that the broadcaster's voice audio data can be transmitted stably in the direct communication network.

[0143] In one embodiment, please refer to Figure 8 The step of determining the direct transmission capability of each target voice audience node based on the direct transmission feature information obtained through the direct communication link of each target voice audience node includes the following steps:

[0144] Step S131: Obtain the direct transmission feature information pushed by each of the target voice audience nodes through the established direct communication links with each of the target voice audience nodes. The direct transmission feature information includes the latest node packet loss rate and the latest direct connection latency.

[0145] Once the broadcaster's audio direct transmission node identifies one or more target audio viewer nodes whose direct connection latency is lower than that of the media server, it will obtain the direct transmission characteristic information of these target audio viewer nodes through the established direct communication links. The direct transmission characteristic information includes the latest node packet loss rate of the data transmitted through the direct communication link established between the target audio viewer node and the broadcaster's audio direct transmission node, and the latest direct connection latency of the data transmitted through the direct communication link established between the target audio viewer node and the broadcaster's audio direct transmission node.

[0146] Step S132: Based on a preset transmission capability algorithm, perform a weighted calculation on the latest node packet loss rate and the latest direct connection delay contained in each of the direct connection transmission feature information to determine the direct connection transmission capability of each of the target voice audience nodes.

[0147] The transmission capability algorithm generally has weight values ​​that act on the packet loss rate and direct connection delay of the node respectively. Based on the weight values ​​corresponding to the packet loss rate and direct connection delay of the node, the direct connection transmission capability remembered by the target voice audience is determined.

[0148] In this embodiment, after the broadcaster's audio direct transmission node performs a preliminary latency-based screening of the audio audience nodes that apply to it to establish a direct broadcaster audio subscription, it determines the audio audience nodes that need to be transmitted directly. Then, it performs further node screening on the initially screened audio audience nodes to ensure the efficiency of direct transmission of broadcaster's audio data in the direct communication network.

[0149] In one embodiment, please refer to Figure 9 The step of transmitting the broadcaster's voice audio data to the target voice audience node with superior direct transmission capability through the established direct communication link includes the following steps:

[0150] Step S131': Based on the direct transmission capability of each target voice audience node, sort the target voice audience nodes in order:

[0151] Once the current anchor audio direct transmission node determines the direct transmission capability of each target audio audience node, it will sort these target audio audience nodes in descending order of their direct transmission capabilities.

[0152] Step S132': Determine the difference between the number of times the broadcaster's audio data is forwarded and the limit on the number of forwards, and establish a direct broadcaster audio subscription with one or more target audio viewer nodes that are ranked high and within the range of the difference.

[0153] The current broadcaster audio direct transmission node determines the difference between the number of times the broadcaster audio data is forwarded and the limit on the number of forwardings. The difference represents the number of times the broadcaster audio data can be directly transmitted. Then, from the target audio audience nodes sorted according to the direct connection capability, one or more target audio audience nodes whose order position is within the range of the difference are determined to establish the broadcaster audio direct connection subscription. For example, when the difference is 2, the broadcaster audio direct connection subscription will be established with the target audio audience nodes whose order positions are 1 and 2 respectively.

[0154] Step S133': Push the currently available broadcaster audio data through the direct communication link established with the target audio viewer node that has already subscribed to the broadcaster's audio.

[0155] The target audio audience node establishes a direct audio connection subscription with the current broadcaster's audio transmission node. The current broadcaster's audio transmission node will transmit the broadcaster's audio data through the direct communication link established with the target audio audience node.

[0156] In this embodiment, the number of nodes that establish direct audio subscription for broadcasters through the direct transmission capability of each voice audience node can be correlated with the number of times the broadcaster's voice audio data is forwarded. This enables the direct transmission node of broadcaster's voice audio to quickly establish direct audio subscription with multiple voice audience nodes that have strong direct transmission capabilities, thus ensuring the efficiency of direct transmission of broadcaster's voice audio data in the direct communication network.

[0157] Furthermore, by functionalizing the various steps in the methods disclosed in the above embodiments, a multi-person voice communication device for live streaming rooms can be constructed according to this application. Following this approach, please refer to... Figure 10 In one typical embodiment, the device includes: a direct communication establishment module 11, used to establish a direct communication link with the audience node corresponding to the direct communication node information broadcast by the media server, wherein the direct communication node information is pushed to the media server by the newly entered audio audience node in the current multi-person audio live broadcast room; a target node determination module 12, used to determine the target audio audience node whose direct communication latency does not exceed the media server latency by comparing the direct connection latency of one or more audio audience nodes that have established the direct communication link with the media server latency when the number of forwardings of the anchor's audio data and the uplink bandwidth meet their respective preset conditions; and an audio direct connection push module 13, used to determine the direct connection transmission capability of each target audio audience node based on the direct connection transmission feature information obtained through the direct connection communication link of each target audio audience node, and transmit the anchor's audio data with the target audio audience node with better direct connection transmission capability through the established direct communication link.

[0158] In one embodiment, the direct communication establishment module 11 includes: a direct node information push submodule, used to push its own direct node information to the media server in response to a voice joining event acting on the current multi-person voice live broadcast room; a direct communication link establishment submodule, used to establish corresponding direct communication links with the voice audience node or voice broadcaster node corresponding to each of the other direct node information pushed by the media server; an audio subscription status acquisition submodule, used to acquire the broadcaster audio subscription status pushed by the broadcaster audio direct transmission node through the direct communication link established with it, and determine the broadcaster audio subscription status as a target broadcaster audio direct transmission node that is available for subscription; and a latency data push submodule, used to determine the direct connection latency with the target broadcaster audio direct transmission node, and push each of the direct connection latency and media server latency through the direct communication link established with the target broadcaster audio direct transmission node.

[0159] In one embodiment, the target node determination module 12 includes: an audio forwarding count verification submodule, used to determine the forwarding count of the currently available broadcaster audio data and to determine whether the forwarding count is lower than a preset limit; an uplink bandwidth verification submodule, used to determine whether the uplink bandwidth of the current node exceeds a preset minimum uplink bandwidth; a subscribing status determination submodule, used to push a subscribing status indicating subscribing to broadcaster audio data through a direct communication link established with a voice audience node that does not have the broadcaster audio data when the forwarding count of the broadcaster audio data is lower than the limit and the uplink bandwidth exceeds the minimum uplink bandwidth; and an unsubscribing status determination submodule, used to push a subscribing status indicating unsubscribing to broadcaster audio data through a direct communication link established with a voice audience node that does not have the broadcaster audio data when the forwarding count of the broadcaster audio data is not lower than the limit or the uplink bandwidth does not exceed the minimum uplink bandwidth.

[0160] In one embodiment, the audio direct connection push module 13 includes: a direct connection transmission feature acquisition submodule, used to acquire direct connection transmission feature information pushed by each of the target audio audience nodes through a direct connection communication link established with each of the target audio audience nodes, wherein the direct connection transmission feature information includes the latest node packet loss rate and the latest direct connection latency; and a direct connection transmission capability determination submodule, used to perform a weighted calculation on the latest node packet loss rate and the latest direct connection latency included in each of the direct connection transmission feature information according to a preset transmission capability algorithm, and correspondingly determine the direct connection transmission capability of each of the target audio audience nodes.

[0161] In another embodiment, the audio direct connection push module 13 further includes: a target node sorting submodule, used to sort the target audio audience nodes in order based on their direct connection transmission capabilities; an audio direct connection subscription establishment submodule, used to determine the difference between the number of times the broadcaster's audio data is forwarded and the limit number of forwardings, and establish a broadcaster audio direct connection subscription with one or more target audio audience nodes that are ranked higher and whose ranking is within the range of the difference; and a broadcaster audio direct connection push submodule, used to push the currently available broadcaster audio data through the direct connection communication link of the target audio audience nodes that have established broadcaster audio direct connection subscriptions.

[0162] To address the aforementioned technical problems, this application also provides a computer device for running a computer program implemented according to the live-streaming multi-person voice method. Please refer to the following for details. Figure 11 , Figure 11 This is a basic structural block diagram of the computer device in this embodiment.

[0163] like Figure 11The diagram shows the internal structure of a computer device. This computer device includes a processor, non-volatile storage medium, memory, and a network interface connected via a system bus. The non-volatile storage medium stores an operating system, a database, and computer-readable instructions. The database may store control information sequences. When executed by the processor, the computer-readable instructions enable the processor to implement a live-streaming multi-person voice communication method. The processor provides computational and control capabilities, supporting the operation of the entire computer device. The memory stores computer-readable instructions, which, when executed by the processor, enable the processor to implement a live-streaming multi-person voice communication method. The network interface of the computer device is used for communication with terminals. Those skilled in the art will understand that… Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0164] In this embodiment, the processor executes the specific functions of each module / submodule in the live streaming multi-person voice device of this application, and the memory stores the program code and various types of data required to execute the above modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules / submodules in the live streaming multi-person voice device, and the server can call the server's program code and data to execute the functions of all submodules.

[0165] This application also provides a non-volatile storage medium in which the live streaming multi-person voice method is programmed into a computer program and stored in the storage medium in the form of computer-readable instructions. When the computer-readable instructions are executed by one or more processors, it means that the program is running in the computer, thereby causing one or more processors to perform the steps of the live streaming multi-person voice method of any of the above embodiments.

[0166] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0167] In summary, this application reduces audio latency in multi-person voice live streaming rooms and improves the user's live streaming audio experience by pushing the host's voice audio through a direct connection communication network.

[0168] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0169] Those skilled in the art will understand that the steps, measures, and solutions in the various operations, methods, and processes discussed in this application can be alternated, modified, combined, or deleted. Furthermore, other steps, measures, and solutions in the various operations, methods, and processes discussed in this application can also be alternated, modified, rearranged, decomposed, combined, or deleted. Furthermore, steps, measures, and solutions in the prior art that are similar to those disclosed in this application can also be alternated, modified, rearranged, decomposed, combined, or deleted.

[0170] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A live room multi-person voice method, characterized in that, Includes the following steps: Based on the direct connection node information broadcast by the media server, a direct communication link is established with the audience node corresponding to the direct connection node information. The direct connection node information is pushed to the media server by the voice audience node that newly enters the current multi-person voice live broadcast room. When the number of times the anchor's audio data is forwarded and the uplink bandwidth meet their respective preset conditions, the direct connection latency of one or more audio audience nodes that have established the direct connection communication link is compared with the media server latency to determine the target audio audience node whose direct connection latency does not exceed the media server latency. Based on the direct transmission characteristic information obtained through the direct communication links of each target audio audience node, the direct transmission capability of each target audio audience node is determined, and the broadcaster's audio data is transmitted to the target audio audience node with the better direct transmission capability through the established direct communication link.

2. The method according to claim 1, characterized in that, After the step of transmitting the broadcaster's voice audio data to the target voice audience node with superior direct connection transmission capability through the established direct communication link, the following steps are included: In response to a direct connection packet loss event, obtain the event trigger time of the direct connection packet loss event; If the time difference between the event trigger time and the current time exceeds a preset duration, and if it does, and no retransmission packet of the anchor voice audio direct connection node corresponding to the direct connection packet loss event is obtained, then the data packet lost by the direct connection packet loss event is obtained from the media server.

3. The method according to claim 1, characterized in that, Before the step of receiving the direct connection node information broadcast by the media server, the following steps are included: Respond to the voice start event applied to the current multi-person voice live streaming room, and obtain the voice audio data pushed by the host node in the current multi-person voice live streaming room; The broadcaster's audio data is pushed to one or more audio viewer nodes in the current multi-person audio live broadcast room; In response to a broadcaster audio direct connection subscription event of a certain audio viewer node, stop pushing the broadcaster's audio data to the audio viewer node; In response to a broadcaster's audio direct connection unsubscribe event at a certain audio viewer node, the broadcaster's audio data is pushed to the audio viewer node.

4. The method according to claim 1, characterized in that, The steps involving comparing the direct connection latency of one or more established direct connection communication links between the direct connection node information pushed to the media server by newly entering the current multi-person voice live broadcast room, or when the number of forwardings of the anchor's voice audio data and the uplink bandwidth meet their respective preset conditions, include the following steps: In response to the voice join event in the current multi-person voice live broadcast room, push its own direct connection node information to the media server; Based on one or more other direct connection node information pushed by the media server, establish corresponding direct communication links with the voice audience node or voice broadcaster node corresponding to each of the other direct connection node information. Obtain the anchor audio subscription status pushed by the anchor audio direct connection node through the established direct communication link with it, and determine the anchor audio subscription status as a target anchor audio direct connection node that can be subscribed to; The direct connection latency between the target broadcaster's voice and audio direct connection node is determined, and the corresponding direct connection latency and media server latency are pushed through the direct communication link established with the target broadcaster's voice and audio direct connection node.

5. The method according to claim 1, characterized in that, The step of determining when the number of times the broadcaster's audio data is forwarded and the uplink bandwidth meet their respective preset conditions includes the following steps: Determine the number of times the broadcaster's voice audio data is forwarded, and determine whether the number of forwarded broadcaster's voice audio data is lower than a preset limit for forwarding. Determine whether the uplink bandwidth of the current node exceeds the preset minimum uplink bandwidth; When the number of times the broadcaster's voice audio data is forwarded is lower than the limit number of forwardings, and the uplink bandwidth exceeds the minimum uplink bandwidth, the broadcaster audio subscription status, which is indicated as subscribing, is pushed through the direct communication link already established with the voice audience node that does not have the broadcaster's voice audio data. When the number of times the broadcaster's voice audio data is forwarded is not less than the limit number of forwards, or the uplink bandwidth does not exceed the minimum uplink bandwidth, the broadcaster's audio subscription status, which is characterized as unsubscribeable, is pushed through the direct communication link established with the voice audience node that does not have the broadcaster's voice audio data.

6. The method according to claim 1, characterized in that, The step of determining the direct transmission capability of each target voice audience node based on the direct transmission feature information obtained through the direct communication link of each target voice audience node includes the following steps: By establishing direct communication links with each of the target voice audience nodes, the direct transmission feature information pushed by each of the target voice audience nodes is obtained. The direct transmission feature information includes the latest node packet loss rate and the latest direct connection latency. Based on the preset transmission capability algorithm, the latest node packet loss rate and the latest direct connection delay contained in each of the direct connection transmission feature information are weighted and calculated to determine the direct connection transmission capability of each of the target voice audience nodes.

7. The method according to claim 1, characterized in that, The step of transmitting the broadcaster's voice audio data to the target voice audience node with superior direct connection transmission capability through the established direct communication link includes the following steps: Based on the direct transmission capability of each target voice audience node, the target voice audience nodes are ordered sequentially. Determine the difference between the number of times the broadcaster's audio data is forwarded and the limit on the number of forwards, and establish a direct broadcaster audio subscription with one or more target audio audience nodes that are ranked high and whose order position is within the range of the difference; The currently available broadcaster audio data is pushed through the direct communication link between the target audio viewer node and the established broadcaster audio direct connection subscription.

8. A multi-person voice communication device for live streaming, characterized in that, include: The direct connection communication establishment module is used to establish a direct connection communication link with the audience node corresponding to the direct connection node information broadcast by the media server. The direct connection node information is pushed to the media server by the voice audience node that newly enters the current multi-person voice live broadcast room. The target node determination module is used to determine the target voice audience node whose direct connection latency does not exceed the media server latency by comparing the direct connection latency of one or more voice audience nodes that have established the direct connection communication link with the media server latency when the number of forwarding times of the broadcaster's voice audio data and the uplink bandwidth meet their respective preset conditions. The audio direct connection push module is used to determine the direct connection transmission capability of each target audio audience node based on the direct connection transmission feature information obtained through the direct connection communication link of each target audio audience node, and to transmit the broadcaster's audio data with the target audio audience node with better direct connection transmission capability through the established direct connection communication link.

9. An electronic device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 7.

10. A non-volatile storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 7, which, when invoked by a computer, performs the steps included in the method.

Images