A transmission control method and apparatus for a media stream, a storage medium, and an electronic device

Abstract

The specification provides a media stream transmission control method and device, a storage medium and an electronic device, which are applied to a local device. The method comprises: negotiating with a peer device about a media session description of a first media stream; configuring a local side transmission control module according to a first group of media session descriptions obtained through the negotiation, and establishing a peer-to-peer transmission channel with the peer device through the configured local side transmission control module; negotiating with the peer device about a media session description of a second media stream through the peer-to-peer transmission channel, and updating the configuration of the local side transmission control module according to a second group of media session descriptions obtained through the negotiation, so as to realize the transmission of the second media stream through the updated local side transmission control module and the peer device.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of communication, and particularly relates to a media stream transmission control method and device, a storage medium and an electronic device. BACKGROUND

[0002] With the popularity of the Internet, real-time communication technology has made great progress. Communication parties can conveniently interact with audio and video based on terminal devices, such as watching live broadcast and participating in video conference. Taking WebRTC (Web Real-Time Communication) as an example, WebRTC is a technology for real-time video and audio communication within a browser. Based on the WebRTC technology, direct communication between different terminals becomes possible. In order to have the ability of one-to-many communication, a SFU (Selective Forwarding Unit) forwarding node cluster can be configured on this basis. Through the forwarding function of the SFU forwarding node cluster, the data of one client can be forwarded to multiple clients.

[0003] In the related art, after a client establishes a media connection through the WebRTC technology, if it needs to switch media streams, for example, in a live broadcast scenario, switching a live broadcast room; in a video conference scenario, switching to the media stream corresponding to the current speaker when the speaker changes, etc., the client needs to re-establish a signaling channel to transmit the corresponding control signaling and create a new peer connection object (PeerConnection), so as to realize the switching of media streams. However, the above-mentioned method of re-establishing a signaling channel has the problems of cumbersome operation, large control signaling delay and low reliability, and the creation of a peer connection object again also increases the burden of the client. SUMMARY

[0004] In order to overcome the problems in the related art, the present specification provides a media stream transmission control method and device, a storage medium and an electronic device.

[0005] According to a first aspect of an embodiment of the present specification, a media stream transmission control method is provided, applied to a local device, and the method comprises:

[0006] negotiating with a peer device about a first media session description of a media stream; configuring a local side transmission control module according to a first set of media session descriptions obtained through negotiation, and establishing a peer transmission channel with the peer device through the configured local side transmission control module;

[0007] The updating unit is configured to negotiate, through the peer-to-peer transmission channel, a media session description of a second media stream with the peer device, and update the local-side transmission control module according to a second set of media session descriptions obtained through the negotiation, so as to realize transmission of the second media stream through the updated local-side transmission control module and the peer device.

[0008] According to a second aspect of the embodiments of the present specification, a transmission control method of a media stream is provided, and the method comprises the steps of:

[0009] The configuration unit is configured to negotiate, with the peer device, a media session description of a first media stream, and configure the local-side transmission control module according to a first set of media session descriptions obtained through the negotiation, and establish a peer-to-peer transmission channel through the configured local-side transmission control module and the peer device.

[0010] The updating unit is configured to negotiate, through the peer-to-peer transmission channel, a media session description of a second media stream with the peer device, and update the local-side transmission control module according to a second set of media session descriptions obtained through the negotiation, so as to realize transmission of the second media stream through the updated local-side transmission control module and the peer device.

[0011] According to a third aspect of the embodiments of the present specification, an electronic device is provided, and the device comprises:

[0012] A processor, a memory for storing processor-executable instructions, wherein the processor is configured to implement the steps of the method of the first aspect.

[0013] According to a fourth aspect of the embodiments of the present specification, a computer-readable storage medium is provided, and the medium stores executable instructions; wherein the instructions are executed by a processor to implement the steps of the method of the first aspect.

[0014] Based on the above embodiments of the present specification, it can be known that, when negotiating a media session description of a second media stream, the local device no longer needs to re-establish a signaling channel with the peer device, but can reuse the peer-to-peer transmission channel that has been established with the peer device, so as to perform a related negotiation process based on the established peer-to-peer transmission channel, and finally update the local-side transmission control module and transmit the second media stream, thereby reducing the steps of re-establishing a signaling channel between the local device and the peer device, accelerating the efficiency of signaling transmission, effectively reducing signaling delay, and improving the switching efficiency of the media stream. Without the need to create a new transmission control module, the pressure on the local device is reduced.

[0015] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application, as claimed. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and serve to explain the principles of the application, in which:

[0017] Figure 1 is a schematic diagram of an interaction of a WebRTC for establishing real-time communication according to an example embodiment of the present specification;

[0018] Figure 2 is a schematic diagram of a network architecture of a media stream transmission control system according to an example embodiment of the present specification;

[0019] Figure 3 is a flowchart of a media stream transmission control method according to an example embodiment of the present specification;

[0020] Figure 4 is an interaction flowchart of a forwarding task carried by a cluster of forwarding nodes according to an example embodiment of the present specification;

[0021] Figure 5 is a schematic diagram of a subscription signaling interaction according to an example embodiment of the present specification;

[0022] Figure 6 is a schematic diagram of an activation signaling interaction according to an example embodiment of the present specification;

[0023] Figure 7 is a schematic diagram of a push stream signaling interaction according to an example embodiment of the present specification;

[0024] Figure 8 is a schematic diagram of a cancel subscription signaling interaction according to an example embodiment of the present specification;

[0025] Figure 9 is a schematic diagram of a cancel push stream interaction according to an example embodiment of the present specification;

[0026] Figure 10 is a schematic diagram of an interaction of DataChannel notification information issuing according to an example embodiment of the present specification;

[0027] Figure 11 is a schematic structural diagram of an electronic device according to an example embodiment of the present specification;

[0028] Figure 12 is a block diagram of a media stream transmission control apparatus according to an example embodiment of the present specification. DETAILED DESCRIPTION

[0029] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The description herein relates to the drawings, in which the same numbers represent the same or similar elements, unless otherwise represented. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of the description. Instead, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of the description.

[0030] It is noted that the steps of the methods described in other embodiments need not necessarily be performed in the order described in this description. In some other embodiments, the methods described therein can include more or fewer steps than those described in this description. Also, the individual steps described in this description can in some other embodiments be broken down into multiple steps for description purposes, or multiple steps described in this description can be combined into a single step in some other embodiments. It is to be understood that although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a particular order. These terms are used only to distinguish one piece of information from another. For example, a first piece of information could be termed a second piece of information without departing from the scope of this description. As used herein, the word "if' can be interpreted to mean "when" or "upon" or "in response to determining." Terms such as "first", "second", and the like can be understood as having relative meanings and are used herein for the ease of describing particular embodiments unless otherwise indicated.

[0031] With the popularity of the Internet, real-time communication technology has made great progress. The two parties of communication can carry out convenient audio and video interaction based on terminal equipment, such as watching live broadcast and participating in video conference. WebRTC (Web Real-Time Communication) is widely used in the field of real-time communication because of its low delay. Although WebRTC indicates real-time communication between browsers, it is not limited to browsers in actual application, but can also be used in mobile applications and other platforms. WebRTC provides various standard API (Application Programming Interface) interfaces for third-party developers, including Media API, RTCPeerConnection (Peer-to-peer Connection object) and Peer-to-peer Data API. Media API is used to transmit corresponding media stream data, RTC PeerConnection object allows users to communicate directly between two browsers. Peer-to-peer Data API, namely DataChannel interface, represents a bidirectional data channel between two nodes. The channel can be set to reliable transmission or unreliable transmission. In order to better understand the interaction process of WebRTC in the related technology, the present specification takes a forwarding server as an example, and the accompanying drawings are used to illustrate the detailed description. Figure 1

[0032] Figure 1 Fig. 1 is an interaction schematic diagram of WebRTC for establishing real-time communication provided by an exemplary embodiment of the present specification.

[0033] Among them, SFU (Selective Forwarding Unit) is used for routing and forwarding audio and video data streams from WebRTC clients. Through SFU, audio and video data streams can be forwarded to multiple clients, thereby realizing one-to-many real-time communication. Specifically, it can include the following steps:

[0034] Step 101, the terminal sends HTTP signaling carrying SDP Offer (Session Description Proposal).

[0035] In order to establish WebRTC connection, the terminal needs to establish signaling channel to transmit corresponding signaling to the signaling server. One way is as follows: Figure 1 ​As shown, it can send an HTTP (Hyper Text Transfer Protocol) request to the signaling server and carry the corresponding SDP Offer, of course, in some embodiments, the above-mentioned SDP Offer can also be sent based on Websocket (a full-duplex communication protocol based on TCP), and the present specification does not limit this.

[0036] SDP is a Session Description Protocol, mainly used for media negotiation between two session entities, so that the audio and video codecs, codec-related parameters (such as the number of audio channels, sampling rate, etc.), transmission protocols and other information recognized by both parties can be negotiated. SDP Offer (Session Description Proposal) is used to represent the various parameters as described above that the initiator itself expects.

[0037] Step 102, the signaling server forwards the HTTP signaling carrying the SDP Offer (Session Description Proposal).

[0038] Because the communication parties do not know each other's address, a signaling server needs to be set up, which is mainly used for exchanging media information, and SDP can communicate normally through the signaling server exchanging media information. Therefore, the signaling server can forward the received HTTP signaling and SDP Offer to the corresponding SFU.

[0039] Step 103, the SFU establishes a PeerConnection and generates an SDP Answer (Session Description Answer).

[0040] After the SFU receives the SDP Offer sent by the terminal, it can establish a PeerConnection (PeerConnection object) on the opposite side, PeerConnection is the name of a standard interface object, and the establishment of a media channel needs to be completed under this PeerConnection object. Further, the SFU (answering party) will generate an SDP message according to the received proposal (SDP offer) and its own capabilities to describe the sessions it can accept, which is called Answer (SDP Answer). At this point, the local device (requesting party) and the SFU (answering party) complete media negotiation and determine the audio and video codecs and codec-related parameters recognized by both parties.

[0041] Step 104, the SFU returns the HTTP signaling carrying the SDP Answer (Session Description Answer).

[0042] As mentioned above, the transmission of signaling needs to rely on the signaling server for transit, so the SFU can send the generated SDP Answer to the signaling server.

[0043] Step 105, the signaling server returns HTTP signaling carrying the SDP Answer (Session Description Answer).

[0044] The signaling server can forward the HTTP signaling and the SDP Answer sent by the SFU in step 104 to the terminal, so that the terminal can receive the SDP Answer.

[0045] Step 106, the terminal establishes a PeerConnection.

[0046] As mentioned above, both parties of communication need to establish corresponding PeerConnection objects respectively, so the terminal can establish a PeerConnection object based on the received SDP Answer.

[0047] Step 107, the two parties of communication perform ICE negotiation to establish a media channel.

[0048] After completing the media negotiation, the two parties of communication also need to further perform ICE (Interactive Connectivity Establishment) negotiation, because in the real-time communication scenario, both parties of communication can be behind a NAT (Network Address Translation) network, and the network interface addresses directly negotiated can not be directly communicated. Therefore, the purpose of the ICE negotiation is to find out which candidate address pairing can work, and to negotiate the hole punching penetration of the NAT to complete the establishment of the transmission channel. This process can still rely on the signaling server for transit.

[0049] Step 108, the two parties of communication perform ICE negotiation to establish a media channel.

[0050] Step 109, P2P media transmission of the two parties of communication.

[0051] After completing the ICE negotiation as mentioned above, the corresponding media transmission channel can be established, and the P2P (Peer to Peer) media transmission between the terminal and the peer device SFU can be realized. Of course, in addition to the establishment of the media transmission between the terminal and the SFU as described above, WebRTC can also be used to establish P2P media transmission between browsers based on the signaling server.

[0052] However, in related technologies, if the local device needs to switch media streams, it must re-establish a separate signaling channel. This involves sending an HTTP request to the signaling server again, carrying the corresponding SDPOffer, to negotiate the second media stream with the peer device. Furthermore, the existing peer connection object needs to be deleted, and a new peer connection object (PeerConnection) needs to be created. This process of re-establishing the signaling channel is cumbersome, cannot deliver signaling to both communicating parties in a timely manner, and has low reliability. Re-establishing the peer connection object (PeerConnection) also increases the load on the local device.

[0053] Therefore, to solve the above problems, this specification proposes an improved media stream transmission control scheme, which makes corresponding improvements to the WebRTC standard connection. The following is a detailed description with reference to the embodiments.

[0054] Figure 2 This is a schematic diagram of a network architecture for a media stream transmission control system provided in an exemplary embodiment of this specification. For the sake of brevity, in Figure 2 The text does not show the actual content. Figure 1 The signaling server shown is, but it is understandable that, in Figure 2 In the network architecture shown, the terminal and the media network can also be configured as follows: Figure 1 The signaling server shown may specifically include terminal 21, terminal 22, and a media network, wherein the media network may include load balancing node 23 and forwarding node cluster 24.

[0055] Terminals 21 and 22 can be PCs, mobile phones, and wearable devices (such as smartwatches and smart glasses), etc., and this specification states that there are no limitations on this. The terminal devices can negotiate a media session description regarding the first media stream with forwarding node 241 in the forwarding node cluster 24, which is the interactive negotiation process of SDP Offer and SDP Answer as described above, because... Figure 2 The forwarding server shown is hosted by a cluster of forwarding nodes. Therefore, the corresponding load balancing node 23 can be used to determine the forwarding nodes and forward information. The terminal 21 can configure the local transmission control module according to the first set of media session descriptions obtained through negotiation. In the WebRTC-based real-time communication scenario, the transmission control module can refer to, for example, Figure 1The peer connection object of the embodiment, i.e. the PeerConnection configured at the terminal 21 side, negotiates with the peer device through the transmission control module at the local side to establish a peer transmission channel, and then the terminal 21 can transmit the first media stream with the peer device. However, unlike the related art, in the case where the terminal 21 needs to negotiate about the second media stream, the terminal 21 no longer needs to resend the HTTP request, and it can directly negotiate with the peer device about the media session description of the second media stream through the peer transmission channel as described above, and update the configuration of the transmission control module at the local side according to the second set of media session descriptions obtained by the negotiation, so as to realize the transmission of the second media stream with the peer device through the updated transmission control module at the local side.

[0056] The load balancing node 23 can be deployed in a physical server containing a standalone host, or it can be a virtual server (such as a cloud server) carried by a host cluster, and the present specification does not limit this. In an embodiment, the load balancing node 23 can be an LVS (Linux Virtual Server, Linux virtual server) to achieve the purpose of load balancing, which can also be called a dispatcher. The web request of the terminal 21 is sent to the load balancing node 23, which can determine to send the request to a certain web server (for example, the forwarding node 241) of the backend forwarding node cluster according to its own preset load balancing strategy. Of course, the specific load balancing strategy is not limited in the present specification, for example, it can be round-robin scheduling, weighted round-robin scheduling, and minimum connection scheduling, etc.

[0057] The forwarding node 241 can be deployed in a physical server containing a standalone host, or it can be a virtual server (such as a cloud server) carried by a host cluster, and the present specification does not limit this. In an embodiment, the forwarding node 241 can run a server program (SFU) for routing and forwarding WebRTC client audio and video data streams as described above, thereby realizing the routing and forwarding of media stream data. In the case where the terminal 21 needs to perform media negotiation with the forwarding node 241 about the second media stream, the forwarding node 241 described in the present specification can directly receive the SDP Offer sent by the terminal 21 through the established peer transmission channel, and send the corresponding SDP Answer. Of course, for the terminal, it can be unaware of which forwarding node performs the forwarding.

[0058] It should be noted that, because the media negotiation and ICE negotiation described in the specification are negotiation processes performed by both communication parties, the local device and the peer device described in the specification are relative concepts. For example, assuming that the terminal 22 is the initiator of the media negotiation, the terminal 22 can be the local device; assuming that the forwarding node is the initiator of the media negotiation, the forwarding node can be the local device, and the specification does not limit this.

[0059] It should be emphasized that, because the media stream transmission control scheme described in the specification is to multiplex the already established peer-to-peer transmission channel to perform the second media stream related negotiation process, so as to avoid the reconstruction of the signaling channel, and does not involve the adjustment of the network architecture, in addition to the network architecture shown in Figure 2 It should be emphasized that, because the media stream transmission control scheme described in the specification is to multiplex the already established peer-to-peer transmission channel to perform the second media stream related negotiation process, so as to avoid the reconstruction of the signaling channel, and does not involve the adjustment of the network architecture, in addition to the network architecture shown in

[0060] Based on the above network architecture, it can be known that, when performing the negotiation of the media session description of the second media stream, the local device no longer needs to reestablish a signaling channel with the peer device, but can multiplex the already established peer-to-peer transmission channel with the peer device, thereby performing the related negotiation process based on the established peer-to-peer transmission channel, and finally realizing the update of the local side transmission control module and the transmission of the second media stream, reducing the step of reestablishing the signaling channel by the local device and the peer device, accelerating the efficiency of signaling transmission, effectively reducing the signaling delay, and improving the switching efficiency of the media stream. There is no need to re-create (update) a new transmission control module, thereby reducing the pressure on the local device.

[0061] Figure 3 FIG. 1 is a flowchart of a media stream transmission control method provided by an exemplary embodiment of the specification, which can be applied to a local device and can specifically include the following steps:

[0062] Step 302, performing negotiation with a peer device about the media session description of a first media stream; configuring a local side transmission control module according to a first set of media session descriptions obtained through the negotiation, and establishing a peer-to-peer transmission channel with the peer device through the configured local side transmission control module.

[0063] In the context of real-time communication based on WebRTC, the negotiation of the media session description described in this specification may refer to, for example: Figure 1 In the aforementioned embodiment, media negotiation refers to the interaction of SDP Offer and SDP Answer between the communicating parties based on the SDP Session Description Protocol. The local end-side transmission control module can refer to the PeerConnection (peer-to-peer connection object) as described above. Therefore, for details on how the communicating parties initially establish a peer-to-peer transmission channel, please refer to relevant technical documents or other sources. Figure 1 The details of the embodiments shown will not be repeated here. Of course, in some embodiments, the transmission control module may vary depending on the communication protocol. For example, in a WebRTC-based real-time communication scenario, the transmission control module described in this specification may refer to a peer connection object (PeerConnection), but it is not limited to peer connection objects. In scenarios based on other media streaming protocols, the transmission control module may also refer to a functional module responsible for media stream transmission. This specification does not impose any limitations on this. Specifically, the software transmission module can be implemented in software, in hardware, or a combination of both.

[0064] Local device and peer device are relative concepts. In one embodiment, the terminal can act as the initiator of media negotiation, and the terminal can be considered as the local device. In another embodiment, the forwarding node can act as the initiator of media negotiation, and the forwarding node can be considered as the local device. This specification does not impose any restrictions on this.

[0065] Step 304: Negotiate a media session description of the second media stream with the peer device through the peer transmission channel, and update the configuration of the local transmission control module according to the second set of media session descriptions obtained through negotiation, so as to realize the transmission of the second media stream with the peer device through the updated local transmission control module.

[0066] If the local device needs to switch media streams, it needs to negotiate a media session description for the second media stream with the peer device. Therefore, this specification proposes that the relevant signaling can be transmitted based on the established peer transmission channel to complete the negotiation process for the second media stream, eliminating the need to establish a new signaling channel separately.

[0067] The peer-to-peer transmission channel established in WebRTC based on the first set of media session descriptions can include a media transmission channel and a message transmission channel (DataChannel), wherein the DataChannel can be used for sending chat messages, point-to-point file transfer, etc., as another communication mode in addition to audio and video communication. The DataChannel uses SCTP (Stream Control Transmission Protocol) over DTLS (Datagram Transport Layer Security) to transmit messages, ensuring the reliability and security of message transmission. Therefore, in order to ensure the safe and reliable negotiation process of the media session description of the second media stream, the relevant signaling such as SDP Offer or SDP Answer can be transmitted through the message transmission channel (DataChannel) in the peer-to-peer transmission channel, that is, the negotiation of the media session description of the second media stream is performed with the peer device through the message transmission channel.

[0068] As described above, the media stream control scheme described in the present specification is to multiplex the already established peer-to-peer transmission channel to realize the negotiation of the second media stream, which does not exist limit to the network architecture, so the present specification can be applied not only to the network architecture as shown in Figure 2 Fig. 1, but also to other network architectures, such as any one of the local device and the peer device is a terminal for pushing or collecting the media stream, and the other is a forwarding server; or any one of the local device and the peer device is a terminal for pushing the media stream, and the other is a terminal for collecting the media stream, for example, any one can be a push browser, and the other can be a collection browser, so as to approximately consider it as a mesh architecture.

[0069] In order to improve the working efficiency of the forwarding server, a cluster can be used to carry the task of routing and forwarding WebRTC client audio and video data streams, that is, a corresponding forwarding node cluster is established, and the forwarding server can include the forwarding node cluster and a load balancing node. The local device can perform negotiation with the first forwarding node allocated by the load balancing node according to a load balancing strategy about the media session description of the first media stream to obtain the first set of media session descriptions. Because of the existence of the load balancing node, although the negotiation about the media session description is completed with the first forwarding node, the load balancing node can allocate a second forwarding node to perform ICE negotiation with the local device when performing ICE negotiation. In this case, the local device can establish a peer-to-peer transmission channel with the second forwarding node, that is, the local device performs negotiation of candidate pairs with the transmission control module configured on the second forwarding node allocated by the load balancing node according to the load balancing strategy through the configured local-side transmission control module, and establishes a peer-to-peer transmission channel between the negotiated candidate pairs and the transmission control module configured on the second forwarding node; wherein the transmission control module on the second forwarding node is configured by the first set of media session descriptions. In the WebRTC-based real-time communication scenario, the negotiation of candidate pairs can be considered as the above-mentioned ICE negotiation, that is, for multiple candidate IPs and ports, the IP and port that both parties can communicate with are negotiated and determined, so as to realize the hole punching penetration of NAT. On this basis, the local device also completes the establishment of the peer-to-peer transmission channel between the first media stream and the second forwarding node. Assuming that the local device has the demand of switching media streams, the local device needs to perform negotiation about the media session description of the second media stream with the second forwarding node again, so the local device can perform negotiation about the media session description of the second media stream with the second forwarding node through the peer-to-peer transmission channel, that is, the SDP Offer and the SDP Answer are sent through the peer-to-peer transmission channel, and a new signaling channel does not need to be created again.

[0070] When the load balancing node assigns the first forwarding node to perform the negotiation of the media session description, and assigns the second forwarding node to perform the ICE negotiation, the second forwarding node needs to obtain the first set of media session descriptions determined by the first forwarding node when performing the negotiation of the media session description, and establish its own transmission control module (such as the PeerConnection in WebRTC). That is, the transmission control module on the second forwarding node can be configured by the first set of media session descriptions. Further, in order for the second forwarding node to obtain the first set of media session descriptions and configure its own transmission control module, the first set of media session descriptions can be forwarded to the second forwarding node by the first forwarding node through the bypass signaling channel established between the first forwarding node and the second forwarding node, or the second forwarding node reads the first set of media session descriptions from the shared cache space in which the first forwarding node and the second forwarding node both participate. For example, assuming that the first forwarding node negotiates with the local device and determines the first set of media session descriptions, the first forwarding node can forward the first set of media session descriptions to the second forwarding node through the bypass signaling channel established between the first forwarding node and the second forwarding node, or the second forwarding node directly reads the first set of media session descriptions from the shared cache space in which the first forwarding node and the second forwarding node both participate, that is, the first forwarding node needs to store the first set of media session descriptions in the shared cache space, so that the second forwarding node can configure its own transmission control module based on the obtained first set of media session descriptions.

[0071] In order to better understand the media stream control scheme described in the present specification, the following takes the first forwarding node (SFU1), the second forwarding node (SFU2), the shared cache space and the browser as the local device as an example, and describes in detail the interaction process of each party in the WebRTC-based real-time communication scenario. Figure 4 The interaction process of each party is described in detail. Figure 4 An interaction flowchart of a forwarding task carried by a forwarding node cluster according to an example embodiment of the present specification can include the following steps:

[0072] Step 401, the browser sends an HTTP signaling carrying an SDP Offer.

[0073] In order to perform the push stream or the pull stream, the browser terminal can send the corresponding HTTP signaling to the LVS (load balancing node), and carry the SDP Offer about itself, so as to perform the media negotiation with the forwarding node. Of course, in some embodiments, the above-mentioned SDP Offer can also be sent based on Websocket (a full-duplex communication protocol based on TCP), and the present specification does not limit this.

[0074] Step 402, the load balancing node forwards the HTTP signaling carrying the SDP Offer.

[0075] After receiving the HTTP signaling and the corresponding SDP Offer sent by the terminal, the load balancing node can determine the corresponding forwarding node according to the load balancing strategy, for example, determine the first forwarding node (SFU1). Thus, the local device can perform negotiation on the media session description of the first media stream with the first forwarding node allocated by the load balancing node according to the load balancing strategy.

[0076] Step 403, the first forwarding node caches the signaling information.

[0077] After receiving the SDP Offer sent by the terminal, the first forwarding node can save the context information required for establishing a connection into a global cache. Thus, even if the ICE negotiation is performed later, the second forwarding node allocated by the load balancing node can directly obtain the context information based on the global cache, and no longer needs to make the first forwarding node forward the information to the second forwarding node through the bypass signaling channel.

[0078] Step 404, the first forwarding node returns the HTTP signaling carrying the SDP Answer.

[0079] The first forwarding node (SFU1) can generate an SDP message to describe the session it can accept according to the received proposal (SDP offer) and its own capability, that is, the SDP Answer. The first forwarding node can send the SDP Answer to the load balancing node.

[0080] Step 405, the load balancing node returns the HTTP signaling carrying the SDP Answer.

[0081] After receiving the SDP Answer sent by the first forwarding node, the load balancing node can feed it back to the browser.

[0082] Step 406, the browser establishes the PeerConnection.

[0083] After receiving the SDP Answer fed back by the first forwarding node, the browser can configure its PeerConnection object.

[0084] Step 407, the communication parties perform ICE negotiation to establish a media channel.

[0085] After completing step 406, the browser needs to further perform ICE negotiation, that is, determine the corresponding candidate pair, so as to realize the penetration of NAT and ensure that the networks of the interaction parties can be connected to each other.

[0086] Step 408, the two parties of communication carry out ICE negotiation to establish media channel.

[0087] At this time, the load balancing node can determine the second forwarding node to carry out ICE negotiation with the browser according to the load balancing strategy.

[0088] Step 409, the second forwarding node acquires signaling information and establishes media connection.

[0089] Because the forwarding nodes distributed by the load balancing node are inconsistent, the second forwarding node needs to acquire the first group of media session descriptions determined by the first forwarding node from the shared cache space, so as to create and configure the peer connection object (PeerConnection) of itself.

[0090] Step 410, media transmission.

[0091] Step 411, media transmission.

[0092] After the above two negotiations are completed, the peer connection channel between the browser and the second forwarding node is established, so that media transmission can be implemented. If the browser has the demand of new stream pulling, such as watching other live rooms in the live scene, the corresponding signaling can be sent through the already established peer transmission channel, and it is no longer necessary to reestablish a new signaling channel.

[0093] Step 412, the browser sends DataChannel signaling carrying SDP Offer to add new stream pulling.

[0094] As described above, the peer transmission channel includes the media transmission channel and the message transmission channel (DataChannel). Because the message transmission channel has reliability and security, the browser can select to transmit the corresponding signaling to the load balancing node through the message transmission channel.

[0095] Step 413, the load balancing node forwards the DataChannel signaling carrying SDP Offer to add new stream pulling.

[0096] After receiving the DataChannel signaling sent by the browser, the load balancing node will not forward it to other forwarding nodes because the browser and the second forwarding node are the two parties of real-time communication and there is a peer transmission channel.

[0097] Step 414, the second forwarding node returns the DataChannel signaling carrying SDP Answer to add new stream pulling.

[0098] The second forwarding node can send the generated corresponding SDP Answer to the load balancing node through a DataChannel.

[0099] At step 415, the load balancing node returns a DataChannel signaling carrying an SDP Answer and adds a new pull stream.

[0100] The load balancing node can send the SDP Answer to the browser through a DataChannel.

[0101] At step 416, the browser updates a PeerConnection.

[0102] After receiving the SDP Answer fed back by the second forwarding node, the browser can directly perform a corresponding update operation under the same PeerConnection, without the need to create a new peer-to-peer connection object. Moreover, because the communication parties are still the browser and the second forwarding node, the ICE negotiation can be omitted.

[0103] At step 417, media transmission.

[0104] After the update is completed, the browser can receive the media stream data corresponding to the new pull stream. Steps 412 to 415 are all based on the originally established peer-to-peer transmission channel to perform signaling transmission, without the need to create a new signaling channel, thereby ensuring the timeliness of signaling transmission and having high security.

[0105] In addition to the above-mentioned pull new stream, various signals such as subscription signaling, unsubscription signaling, push stream signaling, and unpush stream signaling can also be transmitted through a DataChannel. Therefore, the interaction processes for transmitting the above-mentioned signals are described respectively in the present specification.

[0106] In the negotiation process about the media session description of the second media stream with the peer device, the local device can send a subscription signaling to the peer device through the peer-to-peer transmission channel, so that the peer device: pushes the data of the second media stream to the local device in response to the subscription signaling after the negotiation is completed; or, in the negotiation process about the media session description of the second media stream with the peer device, the local device sends a push stream signaling to the peer device through the peer-to-peer transmission channel, so that the peer device: creates a receiving media context and receives the data of the second media stream pushed by the local device in response to the push stream signaling after the negotiation is completed. As shown in Figure 5 Figure 5 is a schematic diagram of a subscription signaling interaction provided by an example embodiment of the present specification. It should be noted that, because the load balancing node is equivalent to forwarding the corresponding signaling, in order to simplify the description, the subscription signaling interaction process in the present specification is described by taking the browser as an example. Figure 5 ​The load balancing node is not marked in the middle. It can be understood that, Figure 5 The load balancing node can also be included in the middle as described before. As Figure 5 As shown, the following steps can be included:

[0107] Step 501, the browser sends a subscription signaling (SubAdd) and an SDP Offer through a DataChannel.

[0108] The browser can send the corresponding subscription signaling (SubAdd) and SDP Offer to the forwarding node through a DataChannel.

[0109] Step 502, the forwarding node completes negotiation and returns an SDP Answer.

[0110] The forwarding node (SFU) can perform media negotiation with the browser and return the negotiated SDP Answer to the browser, so that the browser can update its peer connection object (PeerConnection) based on the received SDP Answer, and does not need to create a new peer connection object.

[0111] Step 503, the browser updates the PeerConnection and prepares to receive media data.

[0112] After receiving the SDP Answer sent by the SFU, the browser can directly update its peer connection object without creating a new peer connection object, so that the browser is ready to receive the media data sent by the forwarding node.

[0113] Step 504, the forwarding node starts sending media data.

[0114] The forwarding node can send the corresponding media stream subscribed by the browser to the browser through the media transmission channel in the peer-to-peer transmission channel.

[0115] Step 505, send media.

[0116] Based on the subscription signaling sent by the browser, the browser receives the corresponding media stream data sent by the forwarding node.

[0117] So far, the corresponding signaling is sent and received through the originally established peer-to-peer transmission channel (DataChannel), and there is no need to create a new signaling channel, which ensures the timeliness of signaling transmission.

[0118] However, if the forwarding node starts transmitting the media stream corresponding to the subscription after the negotiation is completed (the SDP Answer has been sent), but before the browser receives the SDP Answer, it can cause the browser to fail to receive part of the media data. In view of this, the present specification specifically sets an activation signaling (SubActive signaling) that can be sent by the browser to the forwarding node through the DataChannel, so as to instruct the forwarding node to send the media stream data corresponding to the subscription. Thus, it is ensured that the browser will not lose the key frame or part of the media data. As shown in Figure 6 Figure 6 is an activation signaling interaction diagram provided by an exemplary embodiment of the present specification, which can include the following steps:

[0119] Step 601, the browser sends a subscription signaling (SubAdd) and an SDP Offer through the DataChannel.

[0120] The browser can send the corresponding subscription signaling (SubAdd) and the SDP Offer to the forwarding node through the DataChannel.

[0121] Step 602, the forwarding node completes the negotiation and returns the SDP Answer.

[0122] The forwarding node (SFU) can perform media negotiation with the browser and return the negotiated SDP Answer to the browser.

[0123] Step 603, the browser updates the PeerConnection and prepares to receive media data.

[0124] The browser can update its PeerConnection based on the received SDP Answer sent by the forwarding node SFU, and the browser does not need to create a new PeerConnection.

[0125] Step 604, the browser sends a SubActive (activation) signaling through the DataChannel.

[0126] After the PeerConnection object on the browser side is updated, the browser can send an activation signaling to the forwarding node through the PeerTransport channel, so that the peer device starts pushing the data of the second media stream to the browser in response to the activation signaling.

[0127] Step 605, the forwarding node starts sending media data.

[0128] The forwarding node can start sending media data to the browser in response to the activation signaling.​

[0129] Step 606, the forwarding node returns a SubActive result.

[0130] Meanwhile, the forwarding node can also return the result of the SubActive signaling to the browser, so that the browser knows the corresponding execution result of the forwarding node for the activation signaling.

[0131] Step 607, the forwarding node sends media.

[0132] Under the action of the activation signaling, it can be ensured that the browser completely receives the media stream data pushed by the forwarding node, and key frames or part of the media data are not lost.

[0133] In addition to the above-mentioned subscription signaling, the browser can also act as a provider of media stream data, that is, a streaming end, so that the opposite device can receive the data of the second media stream pushed by the local device, and thus the forwarding node can push the received second media stream to the browser with a subscription relationship. As shown in Figure 7 Figure 7 is a schematic diagram of a push signaling interaction provided by an exemplary embodiment of the present specification. It can include the following steps:

[0134] Step 701, the browser sends PubAdd (push) signaling and SDP Offer through a DataChannel.

[0135] In the negotiation process with the opposite device about the media session description of the second media stream, the browser can send push signaling (PubAdd signaling) and its corresponding SDP Offer to the forwarding node through a peer-to-peer transmission channel in the case of needing to push.

[0136] Step 702, the forwarding node creates a receiving media context.

[0137] After receiving the PubAdd signaling and SDP Offer sent by the browser, the forwarding node (SFU) can create a receiving media context, which is used to represent a subscription relationship, so that the forwarding node can subsequently forward the received media stream data sent by the browser to other terminals with a subscription relationship.

[0138] Step 703, the forwarding node completes negotiation and returns an SDP Answer.

[0139] The forwarding node (SFU) can perform media negotiation with the browser and return the negotiated SDP Answer to the browser.

[0140] Step 704, the browser updates the PeerConnection and starts sending media data. ​

[0141] The browser can update its PeerConnection based on the received SDP Answer sent by the forwarding node SFU, and the browser does not need to recreate a new PeerConnection. After the update is completed, the browser can start sending media data corresponding to the push stream to the forwarding node.

[0142] Step 705, sending media.

[0143] There can also be a cancel subscription signaling and a cancel push stream signaling corresponding to the subscription signaling and the push stream signaling. The local device can send a cancel subscription signaling to the peer device through the peer-to-peer transmission channel, so that the peer device stops pushing data of the second media stream to the local device in response to the cancel subscription signaling; or the local device sends a cancel push stream signaling to the peer device through the peer-to-peer transmission channel, so that the peer device destroys the created receiving media context and stops receiving the data of the second media stream pushed by the local device in response to the cancel push stream signaling. Because the cancel signaling and the cancel push stream are interruptions to the media stream transmission, and are based on the subscription and the push stream, at this time, the SDP Offer and the SDP Answer as described above do not need to be sent or received. See Figure 8 Figure 8 Fig. 8 is a schematic diagram of a cancel subscription signaling interaction provided by an exemplary embodiment of the present specification. It can include the following steps:

[0144] Step 801, the browser sends a SubDelete / SubStop (cancel subscription) signaling through a DataChannel.

[0145] The browser can send a cancel subscription signaling (SubDelete / SubStop signaling) to the forwarding node through a DataChannel.

[0146] Step 802, the forwarding node stops sending media data.

[0147] After receiving the signaling (cancel subscription signaling), the forwarding node can stop sending media stream data to the browser, and return a corresponding signaling result to the browser.

[0148] Step 803, the forwarding node returns a SubDelete / SubStop (cancel subscription) signaling result.

[0149] The forwarding node can return a corresponding execution result of the SubDelete / SubStop (cancel subscription) signaling to the browser.

[0150] ​Step 804, if the PeerConnection of the forwarding node is continuously without subscription or push, disconnect by timeout.

[0151] In order to save the system resources of the forwarding node, if the PeerConnection object of the forwarding node is updated due to continuous lack of subscription or push, the forwarding node can disconnect due to timeout.

[0152] Correspondingly, the cancel push signaling can refer to Figure 9 , Figure 9 Fig. 1 is a schematic diagram of cancel push interaction provided by an exemplary embodiment of the present specification. It can include the following steps:

[0153] Step 901, the browser sends PubDelete / PubStop signaling through the DataChannel.

[0154] The browser can send the cancel push signaling (PubDelete / PubStop signaling) to the forwarding node through the DataChannel.

[0155] Step 902, the forwarding node destroys the receiving media context.

[0156] After receiving the cancel push signaling, the forwarding node can destroy the receiving media context and return the corresponding signaling execution result to the browser.

[0157] Step 903, the forwarding node returns PubDelete / PubStop result.

[0158] The forwarding node can return the corresponding PubDelete / PubStop execution result to the browser.

[0159] Step 904, the browser stops sending media data.

[0160] The browser can stop sending media data to the forwarding node to achieve the purpose of canceling push.

[0161] The above subscription signaling, activation signaling, cancel subscription signaling, push signaling and cancel push signaling can be transmitted by multiplexing the already established peer-to-peer transmission channel, without the need to re-establish the signaling channel, ensuring the timeliness of the signaling, and only updating the corresponding PeerConnection is needed, and there is no need to create a new PeerConnection object.

[0162] The original function of the DataChannel is to send chat messages, point-to-point file transfer, etc., which can be used as another communication method outside the audio and video communication, so although the channel can be multiplexed to transmit the corresponding signaling, the original function of the DataChannel can still be retained. Please refer to Figure 10 ,Figure 10 is an interaction diagram for notifying information of a DataChannel provided by an exemplary embodiment of the present specification. It can include the following steps:

[0163] Step 1001, the browser sends a Notify message through a DataChannel.

[0164] The browser can send a notification message (Notify message) to SFU1 through the DataChannel, which is the original function of the DataChannel.

[0165] Step 1002, the forwarding node delivers the message to all subscribers / pushers.

[0166] After receiving the notification information, the forwarding node (SFU1) can send the notification message to other forwarding nodes, so that the other forwarding nodes broadcast the notification message to all subscribers / pushers of themselves.

[0167] Step 1003, the forwarding node delivers the message to all subscribers / pushers.

[0168] After receiving the notification information, the forwarding node (SFU1) can broadcast the notification message to all subscribers / pushers of itself.

[0169] Step 1004, the forwarding node returns the Notify result.

[0170] After completing steps 1002 and 1003, the forwarding node (SFU1) can return the corresponding execution result to the browser.

[0171] As can be seen from the above embodiment, when the media session description of the second media stream is negotiated, the local device no longer needs to reestablish a signaling channel with the opposite device, but can reuse the peer-to-peer transmission channel that has been established with the opposite device, so as to perform the related negotiation process based on the established peer-to-peer transmission channel, and finally realize the update of the local transmission control module and the transmission of the second media stream, thereby reducing the step of reestablishing the signaling channel between the local device and the opposite device, accelerating the efficiency of signaling transmission, effectively reducing the signaling delay, and improving the switching efficiency of the media stream. Moreover, it is not necessary to create a new transmission control module, thereby reducing the pressure on the local device.

[0172] Corresponding to the embodiments of the foregoing method, the present specification also provides embodiments of an apparatus, an electronic device, and a storage medium.

[0173] Figure 11 is a schematic structural diagram of an electronic device provided by an exemplary embodiment. Please refer to Figure 11At the hardware level, the device includes a processor 1101, a network interface 1102, a memory 1103, a non-volatile memory 1104, and an internal bus 1105, and of course can also include other functionally required hardware. One or more embodiments of the present specification can be implemented in a software manner, such as reading a corresponding computer program from the non-volatile memory 1104 into the memory 1103 by the processor 1101 and then running. Of course, in addition to the software implementation, one or more embodiments of the present specification do not exclude other implementation manners, such as a logic device or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or a logic device.

[0174] Figure 12 The device is a media stream transmission control device provided by an exemplary embodiment of the present specification, applied to a local device, and the device includes:

[0175] A configuration unit 1202 is configured to negotiate a media session description of a first media stream with a peer device, configure a local side transmission control module according to a first set of media session descriptions obtained by negotiation, and negotiate and establish a peer-to-peer transmission channel with the peer device through the configured local side transmission control module.

[0176] An updating unit 1204 is configured to negotiate a media session description of a second media stream with the peer device through the peer-to-peer transmission channel, and configure and update the local side transmission control module according to a second set of media session descriptions obtained by negotiation, so as to realize transmission of the second media stream through the updated local side transmission control module and the peer device.

[0177] Optionally, the peer-to-peer transmission channel includes a media transmission channel and a message passing channel, and the updating unit 1204 is specifically configured to negotiate the media session description of the second media stream with the peer device through the message passing channel.

[0178] Optionally, any one of the local device and the peer device is a terminal for pushing or collecting the media stream, and the other is a forwarding server; or any one of the local device and the peer device is a terminal for pushing the media stream, and the other is a terminal for collecting the media stream.

[0179] Optionally, the forwarding server includes a forwarding node cluster and a load balancing node.

[0180] The configuration unit 1202 is specifically configured to negotiate the media session description of the first media stream with the load balancing node according to a first forwarding node distributed by a load balancing strategy, so as to obtain the first set of media session descriptions.

[0181] The configuration unit 1202 is specifically configured to: the local device performs candidate pair negotiation with the transmission control module configured on the second forwarding node distributed by the load balancing node according to the load balancing strategy through the configured local side transmission control module, and establishes a peer-to-peer transmission channel between the negotiated candidate pair and the transmission control module configured on the second forwarding node; wherein the transmission control module on the second forwarding node is configured by the first group of media session descriptions.

[0182] The updating unit 1204 is specifically configured to: the local device performs negotiation on the media session description of the second media stream with the second forwarding node through the peer-to-peer transmission channel.

[0183] Optionally, the first group of media session descriptions is forwarded to the second forwarding node by the first forwarding node through a bypass signaling channel established between itself and the second forwarding node; or, the first group of media session descriptions is read by the second forwarding node from a shared cache space in which both the second forwarding node and the first forwarding node participate.

[0184] Optionally, the apparatus further comprises:

[0185] The subscription signaling sending unit 1206 is configured to, in the negotiation process on the media session description of the second media stream with the peer device, send a subscription signaling to the peer device through the peer-to-peer transmission channel, so that the peer device: after the negotiation is completed, pushes the data of the second media stream to the local device in response to the subscription signaling; or,

[0186] The push stream signaling sending unit 1208 is configured to, in the negotiation process on the media session description of the second media stream with the peer device, send a push stream signaling to the peer device through the peer-to-peer transmission channel, so that the peer device: creates a receiving media context and after the negotiation is completed, receives the data of the second media stream pushed by the local device in response to the push stream signaling.

[0187] Optionally, the apparatus further comprises:

[0188] The activation signaling sending unit 1210 is configured to, after the local side transmission control module completes the configuration update, send an activation signaling to the peer device through the peer-to-peer transmission channel, so that the peer device starts to push the data of the second media stream to the local device in response to the activation signaling.

[0189] Optionally, the apparatus further comprises:

[0190] The cancel subscription sending unit 1212 is configured to send cancel subscription signaling to the peer device through the peer-to-peer transmission channel, so that the peer device stops pushing data of the second media stream to the local device in response to the cancel subscription signaling; or

[0191] The cancel push stream sending unit 1214 is configured to send cancel push stream signaling to the peer device through the peer-to-peer transmission channel, so that the peer device destroys the created receiving media context and stops receiving data of the second media stream pushed by the local device in response to the cancel push stream signaling.

[0192] For the device embodiment, since it basically corresponds to the method embodiment, the relevant part can be referred to the part of the method embodiment. The device embodiment described above is merely illustrative, wherein the modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, i.e., can be located in one place or distributed on multiple network modules. Part or all of the modules can be selected to achieve the purpose of the scheme of the present specification according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0193] In a typical configuration, the computer device includes one or more processors (CPU), input / output interface, network interface and memory.

[0194] The memory can include non-permanent memory in the computer readable medium, random access memory (RAM) and / or non-volatile memory such as read only memory (ROM) or flash memory (flash RAM). The memory is an example of the computer readable medium.

[0195] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0196] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but also other elements not explicitly listed, or other elements inherent to such a process, method, article or apparatus. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0197] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The description below refers to the accompanying drawings, which show by way of example the exemplary embodiments. In the following description, same reference numerals in different drawings refer to same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments are not meant to represent all implementations consistent with the present specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present specification as detailed in the appended claims.

[0198] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in this specification, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "and / or", as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0199] It should be understood that, although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a particular order or hierarchy among the information. The terms are used only to distinguish one category of information from another. For example, a first information can be termed a second information, and, similarly, a second information can be termed a first information, without departing from the scope of the present description. As used herein, the term "if' can be construed to mean "when" or "upon" or "in response to determining" terms that indicate some manner of correlation or relation.

[0200] The above description is merely illustrative of the application, and is not to be taken in a limiting sense. Any modification, equivalent replacement, improvement, and the like made to the description of the application in the spirit and principle of the application shall be included in the scope of the application.

Claims

1. A method for controlling the transmission of a media stream, characterized in that, Applied to this device, the method includes: Negotiate a media session description for the first media stream with the peer device; configure the local transmission control module according to the first set of media session descriptions obtained through negotiation, and negotiate with the peer device to establish a peer transmission channel through the configured local transmission control module; The local end-side transmission control module is configured and updated based on the negotiated second set of media session descriptions to achieve transmission of the second media stream with the peer device through the updated local end-side transmission control module.

2. The method according to claim 1, characterized in that, The peer-to-peer transmission channel includes a media transmission channel and a message passing channel; the negotiation of a media session description regarding the second media stream with the peer device through the peer transmission channel includes: The media session description of the second media stream is negotiated with the peer device through the messaging channel.

3. The method according to claim 1, characterized in that, Either the local device or the peer device is a terminal that pushes or receives the media stream, and the other is a forwarding server; or, Either the local device or the peer device is a terminal that pushes the media stream, and the other is a terminal that receives the media stream.

4. The method according to claim 3, characterized in that, The forwarding server includes a cluster of forwarding nodes and a load balancing node; The negotiation with the peer device regarding the media session description of the first media stream includes: the local device negotiating with the first forwarding node allocated by the load balancing node according to the load balancing strategy regarding the media session description of the first media stream to obtain the first set of media session descriptions; The step of negotiating and establishing a peer-to-peer transmission channel with the peer device through the configured local-side transmission control module includes: the local device, through the configured local-side transmission control module, negotiating candidate pairs with the transmission control module configured on the second forwarding node allocated by the load balancing node according to the load balancing strategy, and establishing a peer-to-peer transmission channel between the negotiated candidate pairs and the transmission control module configured on the second forwarding node; wherein, the transmission control module on the second forwarding node is configured by the first group of media session descriptions; The negotiation of a media session description of the second media stream with the peer device through the peer transmission channel includes: the local device negotiating a media session description of the second media stream with the second forwarding node through the peer transmission channel.

5. The method according to claim 4, characterized in that, The first group of media session descriptions is forwarded by the first forwarding node to the second forwarding node through the bypass signaling channel established between the first forwarding node and the second forwarding node; or, The first set of media session descriptions is read by the second forwarding node from a shared cache space in which both the second forwarding node and the first forwarding node participate.

6. The method according to claim 1, characterized in that, The method further includes: During the negotiation process with the peer device regarding a media session description of the second media stream, a subscription signaling message is sent to the peer device through the peer transmission channel, causing the peer device to: upon completion of the negotiation, push the data of the second media stream to the local device in response to the subscription signaling message; or, During the negotiation of a media session description of the second media stream with the peer device, a push stream signaling is sent to the peer device through the peer transmission channel, so that the peer device: creates a receiving media context and, after the negotiation is completed, receives the data of the second media stream pushed by the local device in response to the push stream signaling.

7. The method according to claim 6, characterized in that, The method further includes: After the local transmission control module completes the configuration update, it sends an activation signaling message to the peer device through the peer transmission channel, so that the peer device responds to the activation signaling message and begins to push the data of the second media stream to the local device.

8. The method according to claim 6, characterized in that, The method further includes: Sending an unsubscribe signal to the peer device via the peer transmission channel, causing the peer device to stop pushing the second media stream data to the local device in response to the unsubscribe signal; or... The peer-to-peer transmission channel sends a cancel push signaling message to the peer device, causing the peer device to: destroy the created receive media context and, in response to the cancel push signaling message, stop receiving data from the second media stream pushed by the local device.

9. A media stream transmission control device, characterized in that, Applied to this end device, the device includes: The configuration unit is used to negotiate a media session description of the first media stream with the peer device; configure the local transmission control module according to the first set of media session descriptions obtained through negotiation; and negotiate with the peer device to establish a peer transmission channel through the configured local transmission control module. The update unit is used to negotiate a media session description of the second media stream with the peer device through the peer transmission channel, and to update the configuration of the local transmission control module according to the negotiated second set of media session descriptions, so as to realize the transmission of the second media stream with the peer device through the updated local transmission control module.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method as claimed in any one of claims 1 to 8.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the method as claimed in any one of claims 1 to 8.

Images