Method and apparatus for transmitting streaming data

Abstract

The application discloses a streaming media data transmission method and device. A specific embodiment of the method comprises: obtaining streaming media data to be pushed based on a push request of a push end of a first service, wherein the first service is one of a live streaming service and a video conference service; determining whether a received pull request is from a pull end of a second service, wherein the second service is the other of the live streaming service and the video conference service; and in response to a determination that it is, transmitting the streaming media data to the pull end of the second service based on a connection between the first service and the second service. The application achieves the effects of live streaming to RTC and RTC to live streaming based on the connection between the live streaming service and the video conference service.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to the field of real-time interactive technology, and in particular to a method, apparatus, computer-readable medium, and electronic device for transmitting streaming media data. Background Technology

[0002] Live streaming services distribute real-time streaming media data based on real-time transmission protocols. Video conferencing services utilize internet-based communication methods to enable face-to-face communication, collaboration, or interaction between people in different locations. In reality, there are situations where a single live streaming or video conferencing service cannot meet users' diverse operational needs for streaming media data. Summary of the Invention

[0003] This application provides a method, apparatus, computer-readable medium, and electronic device for transmitting streaming media data.

[0004] In a first aspect, embodiments of this application provide a method for transmitting streaming media data, comprising: obtaining streaming media data to be pushed based on a push request from a push end of a first service, wherein the first service is one of a live streaming service and a video conferencing service; determining whether a received pull request comes from a pull end of a second service, wherein the second service is another of a live streaming service and a video conferencing service; and in response to determining yes, transmitting the streaming media data to the pull end of the second service based on the connection between the first service and the second service.

[0005] In some examples, before the streaming request from the streaming end based on the first service is completed and the streaming media data to be streamed is obtained, the method further includes: determining a first relay transmission module in the first service for relaying streaming media data, and a second relay transmission module in the second service for relaying streaming media data; and establishing a connection between the first service and the second service based on the first relay transmission module and the second relay transmission module.

[0006] In some examples, the above-described determination of a first relay transmission module in a first service for relaying streaming media data and a second relay transmission module in a second service for relaying streaming media data includes: determining the first relay transmission module in a first service for relaying streaming media data and the second relay transmission module in a second service for relaying streaming media data based on a received custom operation.

[0007] In some examples, the first service mentioned above is a live streaming service, the second service is a video conferencing service, and the above-mentioned streaming request based on the streaming end of the first service to obtain streaming media data to be streamed includes: obtaining live streaming media data using a preset initial transmission protocol and a preset initial audio and video encoding format based on the streaming request of the streaming end of the first service; and converting the live streaming media data by transmission protocol and encoding format to obtain live streaming media data using a preset target transmission protocol and a preset target audio and video encoding format.

[0008] In some examples, the above-mentioned conversion of the transmission protocol and encoding format of live streaming media data to obtain live streaming media data using a preset target transmission protocol and a preset target audio and video encoding format includes: in response to determining that the preset initial transmission protocol is a first transmission protocol and the preset initial audio and video encoding format is a first audio and video encoding format, performing transmission protocol conversion on the acquired live streaming media data to obtain live streaming media data using a second transmission protocol and a first audio and video encoding format, and relaying the converted live streaming media data to a first data source; performing format conversion and encoding format conversion on the live streaming media data in the first data source to obtain live streaming media data using a third transmission protocol and a second audio and video encoding format, and relaying the converted live streaming media data to a second data source; performing format conversion on the live streaming media data in the second data source to obtain live streaming media data using a custom fourth transmission protocol and a second audio and video encoding format, and relaying the converted live streaming media data to a first relay transmission module, wherein the fourth transmission protocol is the preset target transmission protocol and the second audio and video encoding format is the preset target audio and video encoding format.

[0009] In some examples, the first service mentioned above is a live streaming service, the second service is a video conferencing service, and the above-mentioned transmission of streaming media data to the pull end of the second service based on the connection between the first service and the second service includes: determining the live streaming media data in the first relay transmission module of the first service through the second relay transmission module in the second service based on the pull request and connection; converting the acquired live streaming media data according to the transmission protocol to obtain live streaming media data using the fifth transmission protocol; and distributing the converted live streaming media data to the pull end of the second service.

[0010] In some examples, the first service mentioned above is a video conferencing service, the second service is a live streaming service, and the above-mentioned streaming request based on the streaming end of the first service to obtain streaming media data to be streamed includes: generating a streaming media data producer object corresponding to the streaming end of the first service based on the streaming request of the streaming end of the first service; and obtaining the video conferencing streaming media data of the streaming end of the first service based on the streaming media data producer object.

[0011] In some examples, the first service mentioned above is a video conferencing service, the second service is a live streaming service, and the above-mentioned transmission of streaming media data to the pull end of the second service based on the connection between the first service and the second service includes: according to the pull request received from the pull end of the second service, sequentially performing source return in the first data source, the second data source, and the first relay transmission module, and based on the connection, sending the pull request in the first relay transmission module back to the second relay transmission module; based on the connection, sequentially performing callbacks on the video conferencing streaming media data in the first relay transmission module, the second data source, and the first data source through the second relay module, and distributing the callback-received video conferencing streaming media data to the pull end of the second service.

[0012] Secondly, embodiments of this application provide a streaming media data transmission apparatus, comprising: an acquisition unit configured to acquire streaming media data to be pushed based on a push request from a push end of a first service, wherein the first service is one of a live streaming service and a video conferencing service; a determination unit configured to determine whether a received pull request originates from a pull end of a second service, wherein the second service is another of a live streaming service and a video conferencing service; and a transmission unit configured to, in response to the determination, transmit the streaming media data to the pull end of the second service based on a connection between the first service and the second service.

[0013] In some examples, the above apparatus further includes: a connection establishment unit configured to: determine a first relay transmission module in the first service for relaying streaming media data, and a second relay transmission module in the second service for relaying streaming media data; and establish a connection between the first service and the second service based on the first relay transmission module and the second relay transmission module.

[0014] In some examples, the aforementioned connection unit is further configured to: determine, based on the received custom operation, a first relay transmission module in the first service for relaying streaming media data, and a second relay transmission module in the second service for relaying streaming media data.

[0015] In some examples, the first service mentioned above is a live streaming service, the second service is a video conferencing service, and the acquisition unit mentioned above is further configured to: acquire live streaming media data using a preset initial transmission protocol and a preset initial audio and video encoding format based on the streaming request of the streaming end of the first service; and convert the live streaming media data by transmission protocol and encoding format to obtain live streaming media data using a preset target transmission protocol and a preset target audio and video encoding format.

[0016] In some examples, the aforementioned acquisition unit is further configured to: in response to determining that the preset initial transmission protocol is the first transmission protocol and the preset initial audio and video encoding format is the first audio and video encoding format, perform transmission protocol conversion on the acquired live streaming media data to obtain live streaming media data using the second transmission protocol and the first audio and video encoding format, and relay the converted live streaming media data to the first data source; perform format conversion and encoding format conversion on the live streaming media data in the first data source to obtain live streaming media data using the third transmission protocol and the second audio and video encoding format, and relay the converted live streaming media data to the second data source; perform format conversion on the live streaming media data in the second data source to obtain live streaming media data using a custom fourth transmission protocol and the second audio and video encoding format, and relay the converted live streaming media data to the first relay transmission module, wherein the fourth transmission protocol is a preset target transmission protocol and the second audio and video encoding format is a preset target audio and video encoding format.

[0017] In some examples, the first service mentioned above is a live streaming service, the second service is a video conferencing service, and the transmission unit mentioned above is further configured to: determine the live streaming media data in the first relay transmission module of the first service through the second relay transmission module in the second service based on the pull request and connection; convert the acquired live streaming media data according to the transmission protocol to obtain live streaming media data using the fifth transmission protocol; and distribute the converted live streaming media data to the pull end of the second service.

[0018] In some examples, the first service is a video conferencing service, the second service is a live streaming service, and the acquisition unit is further configured to: generate a streaming media data producer object corresponding to the streaming end of the first service based on the streaming request of the streaming end of the first service; and acquire the video conferencing streaming media data of the streaming end of the first service based on the streaming media data producer object.

[0019] In some examples, the first service mentioned above is a video conferencing service, the second service is a live streaming service, and the transmission unit mentioned above is further configured to: based on the received pull request from the pull end of the second service, sequentially perform a back-to-origin operation in the first data source, the second data source, and the first relay transmission module, and based on the connection, send the pull request in the first relay transmission module back to the second relay transmission module; based on the connection, sequentially perform a callback operation in the first relay transmission module, the second data source, and the first data source through the second relay module, and distribute the callback-received video conferencing streaming media data to the pull end of the second service.

[0020] Thirdly, embodiments of this application provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, it implements the method as described in any implementation of the first aspect.

[0021] Fourthly, embodiments of this application provide an electronic device, including: one or more processors; and a storage device storing one or more programs thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect.

[0022] The streaming media data transmission method and apparatus provided in this application embodiment obtain streaming media data to be pushed by a push request from a push end of a first service, wherein the first service is one of a live streaming service and a video conferencing service; determine whether the received pull request comes from a pull end of a second service, wherein the second service is another of a live streaming service and a video conferencing service; in response to the determination, transmit the streaming media data to the pull end of the second service based on the connection between the first service and the second service, thereby realizing the effect of live streaming to RTC (Real-Time Communication) and RTC to live streaming based on the connection between the live streaming service and the video conferencing service; on the one hand, by cascading the live streaming service with the video conferencing service, the latency introduced by the relay line node can be reduced, and the effect of live streaming to the meeting can also be realized; on the other hand, by cascading the video conferencing service with the live streaming service, the needs of bypass live streaming and cloud recording can be met, which can further optimize and save distribution bandwidth. Moreover, the video conferencing service cluster can support large-scale cascading deployment, which greatly improves concurrency capabilities. Attached Figure Description

[0023] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0024] Figure 1 This is an exemplary system architecture diagram in which one embodiment of this application can be applied;

[0025] Figure 2 This is a flowchart of an embodiment of the streaming media data transmission method according to this application;

[0026] Figure 3 This is a schematic diagram illustrating an application scenario of the streaming media data transmission method according to this embodiment;

[0027] Figure 4 This is a framework flowchart of an embodiment of the streaming media data transmission method according to this application;

[0028] Figure 5This is a flowchart of yet another embodiment of the streaming media data transmission method according to this application;

[0029] Figure 6 This is a flowchart of yet another embodiment of the streaming media data transmission method according to this application;

[0030] Figure 7 This is a structural diagram of one embodiment of a streaming media data transmission apparatus according to this application;

[0031] Figure 8 This is a schematic diagram of the structure of a computer system suitable for implementing the embodiments of this application. Detailed Implementation

[0032] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0033] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0034] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0035] Figure 1 An exemplary architecture 100 is shown that can be applied to the streaming media data transmission method and apparatus of this application.

[0036] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, 108, 109, and 110, networks 104 and 107, and servers 105 and 106. Communication connections between terminal devices 101, 102, and 103 form a topology network, and network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Communication connections between terminal devices 108, 109, and 110 form a topology network, and network 107 serves as the medium for providing communication links between terminal devices 108, 109, and 110 and server 106. Networks 104 and 107 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0037] Users can use terminal devices 101, 102, and 103 to interact with server 105, which deploys live streaming services, via network 104. They can also use terminal devices 108, 109, and 110 to interact with server 106, which deploys video conferencing services, via network 107, to receive or send streaming media data. Terminal devices can be hardware or software that supports network connectivity for data interaction and processing. When a terminal device is hardware, it can be various electronic devices that support network connectivity, information acquisition, interaction, display, and processing functions, including but not limited to smartphones, tablets, e-book readers, laptops, and desktop computers. When a terminal device is software, it can be installed in the aforementioned electronic devices. It can be implemented as multiple software programs or software modules, for example, to provide distributed services, or as a single software program or software module. No specific limitations are imposed here.

[0038] Servers 105 and 106 can be servers providing live streaming and video conferencing services. For example, they can be cascaded to act as backend processing servers for streaming media data transmission to the push and pull ends (specifically, terminal devices 101, 102, 103, 108, 109, and 110) within the live streaming or video conferencing service. As an example, servers 105 and 106 could be cloud servers.

[0039] It should be noted that a server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules (such as software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are made here.

[0040] It should also be noted that the streaming media data transmission method provided in the embodiments of this application can be executed by a server, by a terminal device, or by a combination of both. Accordingly, the various parts (e.g., various units) of the streaming media data transmission device can be all located in the server, all located in the terminal device, or located separately in the server and the terminal device.

[0041] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included. When the electronic devices on which the streaming media data transmission method runs do not need to transmit data with other electronic devices, the system architecture may only include the electronic devices (e.g., servers or terminal devices) on which the streaming media data transmission method runs.

[0042] Continue to refer to Figure 2 The flowchart 200 illustrates an embodiment of a method for transmitting streaming media data, including the following steps:

[0043] Step 201: Based on the streaming request from the streaming end of the first service, obtain the streaming media data to be streamed.

[0044] In this embodiment, the execution entity of the streaming media data transmission method (e.g.) Figure 1 The terminal device or server in the process can obtain the streaming request of the first service from a remote location or from a local location through a wired network connection or a wireless network connection, and obtain the streaming media data to be streamed based on the streaming request of the first service.

[0045] The first service is either a live streaming service or a video conferencing service. For example, when the first service is a live streaming service, after receiving a streaming request from the streaming client in the live streaming service, the aforementioned executing entity will obtain the live streaming media data, such as audio and video, recorded by the streaming client in real time. Similarly, when the first service is a video conferencing service, after receiving a streaming request from the streaming client in the video conferencing service, the aforementioned executing entity will obtain the video conferencing media data, such as audio and video, being recorded by the streaming client in real time.

[0046] Step 202: Determine whether the received streaming request comes from the streaming client of the second service.

[0047] In this embodiment, the aforementioned execution entity can determine whether the received streaming request originates from the streaming client of the second service. The second service is either a live streaming service or a video conferencing service.

[0048] As an example, when the first service is a live streaming service, the second service is a video conferencing service; when the first service is a video conferencing service, the second service is a live streaming service.

[0049] In this embodiment, the aforementioned execution entity can parse the received streaming request from the streaming client, and then determine whether the streaming request comes from the live streaming service or the video conferencing service based on the parsed request parameters.

[0050] Step 203: In response to the determination, based on the connection between the first service and the second service, transmit the streaming media data to the pull end of the second service.

[0051] In this embodiment, in response to determining that the received streaming request comes from the streaming end of the second service, the aforementioned execution entity can transmit streaming media data to the streaming end of the second service based on the connection between the first service and the second service.

[0052] In this embodiment, a connection between the first service and the second service is established beforehand. As an example, the aforementioned executing entity can utilize an API (Application Programming Interface) to facilitate interaction between the first and second services. Specifically, by utilizing the relevant API interfaces provided by the live streaming service provider and the video conferencing service provider, data exchange and information sharing between the two services can be achieved, thereby enabling interoperability in certain scenarios.

[0053] As another example, the aforementioned implementing entity can utilize a third-party platform to coordinate the interaction between the first and second services. Specifically, some third-party online interactive platforms and messaging middleware can integrate live streaming and video conferencing services, and provide embedded components or plugins to facilitate linkage or cross-platform interoperability between the two services.

[0054] Based on the connection between the live streaming service and the video conferencing service, the aforementioned execution entity can forward the streaming media data pushed by the push end of one service to the pull end of the other service, thereby realizing information interaction between the two services.

[0055] It should be noted that, in response to the determination that the received streaming request originates from the streaming end of the first service, that is, both the streaming and streaming ends belong to the live streaming service or both belong to the video conferencing service, the live streaming service or the video conferencing service will perform its own internal video transmission and reception according to its own business logic. At this time, the live streaming service and the video conferencing service will no longer exchange streaming media data.

[0056] Taking a live streaming service where both the push and pull ends are live streaming services as an example, on the push end, firstly, a camera or other audio-visual device is used to acquire streaming media data of the target sound and image; then, the audio and video streaming media data is encoded and compressed for subsequent transmission, storage, and playback; then, the audio and video data is encapsulated and packaged to generate transmission data packets that conform to common network protocols; then, the encapsulated and packaged audio and video data stream is uploaded to the streaming media server via a network connection, and the corresponding Stream ID is requested; finally, the Stream ID and the push URL (UniformResource Locator) are sent to the pull end to start the remote publishing process of the push stream.

[0057] On the streaming client, firstly, a streaming request is used to obtain the valid playback address of the corresponding audio and video stream via the streaming address or media ID. Then, a connection is initiated to the streaming media server using network protocols to establish a transmission link channel such as TCP (Transmission Control Protocol) or UDP (User Datagram Protocol). Next, the audio and video data stream from the server is downloaded via network transmission. The obtained audio and video data is then decapsulated, decrypted, and decoded to restore it to a playable data format. Finally, the restored audio and video data is rendered and played back, and displayed on the streaming client's screen.

[0058] Taking a video conferencing service where both the push and pull ends are involved as an example, at the push end, firstly, the push end acquires audio and video data through a media acquisition device. Then, the push end uses WebRTC technology to establish a connection with the signaling server and sends relevant information, such as SDP (Session Description Protocol) information, through the signaling channel. Next, the signaling server forwards the SDP to the pull end or the MCU (Multipoint Control Unit) for processing. Then, if it's a point-to-point connection, the pull or push end responds with a matching SDP to establish a bidirectional media transmission channel. If it's a multi-party video conference, the signaling server forwards the SDP to the MCU, which coordinates and establishes the corresponding media channel. Finally, the push end transmits the audio and video data to the receiving client or MCU via WebRTC.

[0059] On the streaming end, the streaming end first requests to join the specified room from the server. Then, the server returns descriptive metadata about the media stream, including available quality, codec information, and other parameters. Next, the streaming end uses WebRTC technology to establish a connection with the server and exchanges SDP (the streaming end's appropriate Answer Message) with the server via a signaling channel. The server then forwards the SDP back to the streaming end. The streaming end then calls the WebRTC API to begin pulling media stream data from the server. Finally, the streaming end decodes the acquired audio and video data and plays it.

[0060] See also Figure 3 , Figure 3 This is a schematic diagram 300 illustrating an application scenario of the streaming media data transmission method according to this embodiment. Figure 3In this application scenario, users 301 and 302 are conducting a video conference through server 303, which deploys video conferencing services; user 304 is conducting a live stream through server 305, which deploys live streaming services. During the discussion in the video conference, users 301 and 302 develop a need to watch user 304's live stream content. User 301 sends a streaming request to server 303, and server 303, through its connection with server 305, obtains user 304's live streaming media data and forwards the live streaming media data to the video conference interfaces of users 301 and 302.

[0061] The method provided in the above embodiments of this application obtains streaming media data to be pushed through a push request from a push end of a first service, wherein the first service is one of a live streaming service or a video conferencing service; determines whether the received pull request comes from a pull end of a second service, wherein the second service is another of a live streaming service or a video conferencing service; in response to the determination, transmits the streaming media data to the pull end of the second service based on the connection between the first service and the second service, thereby realizing the effect of live streaming to RTC (Real-Time Communication) and RTC to live streaming based on the connection between the live streaming service and the video conferencing service; on the one hand, by cascading the live streaming service through the video conferencing service, the latency introduced by the relay line node can be reduced, while also achieving the effect of live streaming to the meeting; on the other hand, by cascading the video conferencing service through the live streaming service, the needs of bypass live streaming and cloud recording can be met, further optimizing and saving distribution bandwidth, and the video conferencing service cluster can support large-scale cascading deployment, greatly improving concurrency capabilities.

[0062] Continue to refer to Figure 4 The diagram shows a framework flowchart of yet another embodiment of the streaming media data transmission method according to this application.

[0063] In some optional implementations of this embodiment, before performing step 201 above, the execution entity may also perform the following operations:

[0064] First, identify the first relay transmission module in the first service used for relaying streaming media data, and the second relay transmission module in the second service used for relaying streaming media data.

[0065] As an example, the aforementioned implementing entity can add a first relay transmission module RelayioSource to the live streaming service (first service), which supports the conversion and forwarding of streaming media data in the live streaming service; and add a second relay transmission module RelayioTransport to the video conferencing service (second service), which supports the conversion and forwarding of streaming media data in the video conferencing service.

[0066] Specifically, the RelayioSource and RelayioTransport modules can be implemented based on UDT (UDP-based Data Transfer Protocol), including functions such as rate control, acknowledgment, negative acknowledgment, retransmission, and connection maintenance.

[0067] In terms of rate control, the transmission rate is dynamically adjusted based on the receiving capacity of the streaming end to avoid network congestion and data loss. Regarding acknowledgment, after receiving streaming media data, the streaming end sends an acknowledgment message (ACK) to the streaming end via the relay transmission module to indicate successful reception. For negative acknowledgment, if the streaming end detects an error or loss of a data packet, it sends a negative acknowledgment message (NAK) to the streaming end via the relay transmission module to indicate the need to retransmit the streaming media data. For retransmission, if the streaming end receives a NAK message from the streaming end, or does not receive an ACK message in a timely manner, it will retransmit the data packet. Regarding connection maintenance, the relay transmission module also needs to maintain the connection status between the streaming end and the streaming end to ensure that data packets are sent and received correctly.

[0068] Second, based on the first relay transmission module and the second relay transmission module, a connection is established between the first service and the second service.

[0069] In this implementation, a connection is established between the first relay transmission module and the second relay transmission module to exchange streaming media data based on the UDT protocol, thereby realizing the connection between the first service and the second service.

[0070] This implementation provides a specific method for establishing a connection between a first service and a second service, which improves the information exchange capability between the first service and the second service based on the first relay module and the second relay module.

[0071] In some optional implementations of this embodiment, the execution entity can perform the second step as follows: based on the received custom operation, determine the first relay transmission module in the first service for relaying streaming media data, and the second relay transmission module in the second service for relaying streaming media data.

[0072] As an example, the developers of the first and second relay transmission modules can customize the forwarding logic, transmission protocol, and functions such as rate control, acknowledgment, negative acknowledgment, retransmission, and connection maintenance of streaming media data to obtain the first relay transmission module used for relaying streaming media data in the first service and the second relay transmission module used for relaying streaming media data in the second service.

[0073] In this implementation, based on a custom development approach, the compatibility of the first and second relay transmission modules with the first and second services is improved, as well as the flexibility of the development process, which helps to improve the data transmission efficiency and accuracy between the first and second services after connection.

[0074] In some optional implementations of this embodiment, the first service is a live streaming service and the second service is a video conferencing service. In this implementation, the execution entity can perform step 201 as follows:

[0075] First, based on the streaming request from the streaming end of the first service, obtain live streaming media data using a preset initial transmission protocol and a preset initial audio and video encoding format.

[0076] As an example, the default initial transmission protocol can be RTMP (Real-Time Messaging Protocol), SRT (Secure Reliable Transport), RTC (Real-Time Communication), etc.; the default initial audio and video encoding format can be AAC (Advanced Audio Coding), VAC (Advanced Video Coding), etc.

[0077] This implementation supports multiple preset initial transmission protocols and multiple preset initial audio and video encoding formats.

[0078] Second, the live streaming media data undergoes transmission protocol and encoding format conversion to obtain live streaming media data using a preset target transmission protocol and preset target audio and video encoding format.

[0079] Among them, the preset target transmission protocol represents the transmission protocol that the live streaming media data is ultimately expected to use, and the preset target audio and video encoding format represents the encoding format that the live streaming media data is ultimately expected to use.

[0080] For the various preset initial transmission protocols and preset target audio and video encoding formats supported by the live streaming service, when the preset initial transmission protocol is different from the preset target transmission protocol, or the preset initial audio and video encoding format is different from the preset target audio and video encoding format, one or more transmission protocol conversions and encoding format conversions can be performed to finally obtain live streaming media data using the preset target transmission protocol and preset target audio and video encoding format.

[0081] In this implementation, based on the transmission protocol conversion and encoding format conversion operations, the aforementioned execution entity can convert streaming media data suitable for data transmission, based on supporting the use of multiple preset initial transmission protocols and multiple preset initial audio and video encoding formats for streaming requests. This helps to improve the efficiency of the information transmission process and the adaptability between streaming media data and services.

[0082] In some optional implementations of this embodiment, the execution entity can perform the second step as follows:

[0083] First, in response to determining that the preset initial transmission protocol is the first transmission protocol and the preset initial audio and video encoding format is the first audio and video encoding format, the acquired live streaming media data is converted to a transmission protocol to obtain live streaming media data using the second transmission protocol and the first audio and video encoding format, and the converted live streaming media data is then transferred to the first data source.

[0084] As an example, for streaming media data using the SRT transmission protocol and AAC encoding format, it is generally transferred to SRTSource. The aforementioned execution entity can use the SRT2RTMPBridge module to convert the streaming media data in SRTSource into streaming media data using the RTMP transmission protocol and AAC encoding format, and then transfer the converted data to the first data source RTMPSource.

[0085] Then, the live streaming media data in the first data source is converted in format and encoding format to obtain live streaming media data using the third transmission protocol and the second audio and video encoding format, and the converted live streaming media data is transferred to the second data source.

[0086] As an example, the streaming media data in the first data source RTMPSource is converted into streaming media data using the RTC transport protocol and Opus encoding format through the RTMP2RTCBridge module, and the converted data is then transferred to the second data source RTCSource.

[0087] Finally, the live streaming data from the second data source is converted to a format that uses a custom fourth transmission protocol and a second audio / video encoding format. This converted live streaming data is then relayed to the first relay transmission module. The fourth transmission protocol is a preset target transmission protocol, and the second audio / video encoding format is a preset target audio / video encoding format.

[0088] As an example, the live streaming data in the second data source RTCSource is converted into streaming data using a custom relayIO transmission protocol and Opus encoding format by the RTC2RelayioBridge module, and the converted data is then relayed to the first relay transmission module RelayioSource.

[0089] As described above, the push request in this implementation supports multiple preset initial transmission protocols. For example, when the live streaming media data corresponding to the push request uses the RTMP protocol and AAC encoding format, the aforementioned execution entity can convert the streaming media data relayed to RTMPSource into streaming media data using the RTC protocol and AAC encoding format, and then relay the converted streaming media data to RTCSource. Furthermore, for the live streaming media data in the second data source RTCSource, it is converted into streaming media data using a custom relayIO transmission protocol and Opus encoding format, and the converted data is relayed to the first relay transmission module RelayioSource.

[0090] For example, when the live streaming media data corresponding to the push request uses the RTC protocol, it is only necessary to convert the live streaming media data in the second data source RTCSource into streaming media data using the custom relayIO transmission protocol and Opus encoding format, and then transfer the converted data to the first relay transmission module RelayioSource.

[0091] This implementation provides a specific conversion method for streaming media data in live streaming services, which further improves the efficiency of data transmission and the compatibility between streaming media data and services.

[0092] In some optional implementations of this embodiment, when the first service is a live streaming service and the second service is a video conferencing service, the execution entity can perform step 203 as follows:

[0093] First, based on the pull request and connection, the live streaming media data in the first relay transmission module of the first service is determined through the second relay transmission module in the second service.

[0094] Second, the acquired live streaming media data is converted to a transmission protocol to obtain live streaming media data using the fifth transmission protocol.

[0095] In this implementation, the aforementioned execution entity can perform transmission protocol conversion on the live streaming media data in the first relay transmission module to obtain live streaming media data using the UDT transmission protocol.

[0096] Third, the converted live streaming data is distributed to the pull-streaming end of the second service.

[0097] As an example, the distribution process is as follows:

[0098] 1. ConsumerClient creates WebRtcTransport to establish a connection with SignalServer.

[0099] ConsumerClient is a client application that allows consumers to pull and watch live streams. It establishes a connection with SignalServer by creating a WebRtcTransport object to request the stream, set receiving parameters (such as media type, bitrate, etc.), and specify how to sample and send data in the desired format.

[0100] 2. After receiving the pull request, SignalServer creates CreateSubScriber.

[0101] The SignalServer is the signaling server for the entire system. When it receives a pull request from a ConsumerClient, it requests the corresponding Subscriber from the Worker (worker node) based on the request content.

[0102] 3. After receiving the request, the Worker establishes a path between AsyncBuildRelayPath and RtpRelayIO.

[0103] The Worker component is an intermediary component located between SignalServer and RtpRelayIO. It is responsible for processing signaling information and controlling the behavior of multiple RtpRelayIO components, enabling the establishment and maintenance of RelayTransport connections. In this step, the Worker processes subscription requests and notifies the RtpRelayIO module to establish a path between AsyncBuildRelayPath and RtpRelayIO.

[0104] 4. After receiving the heartbeat, RtpRelayIO relays it through the RelayServer module to the corresponding RtpRelayIO module of the publishing stream.

[0105] When the RtpRelayIO module receives a Subscription Request, it sends a heartbeat signal and status information to the upstream RtpRelayIO module and outputs some key data of this transmission to the log or console.

[0106] 5. After receiving the routing request, the Worker corresponding to the upstream Producer creates SDP information and sends it to the RtpRelayIO relay.

[0107] At this stage, the Producer's Worker receives the Subscriber request and generates SDP information. This information describes the media stream related to the Producer, including detailed information such as encoding format, media type, and bandwidth. The Worker sends this information to the RtpRelayIO component, which then relays the information from the push end to the pull end.

[0108] 6. After receiving the corresponding SDP information through the relay Subscriber Worker, a RelayTransport is created.

[0109] When the Subscriber's Worker receives the Producer's SDP information, it begins parsing the information and establishing a RelayTransport connection. Through the RelayTransport connection, real-time audio and video transmission can occur between the streaming and push ends. This process also includes negotiation-related steps, such as ICE (Interactive Connectivity Establishment) candidate exchange and NAT (Network Address Translation) traversal.

[0110] 7. After the road is successfully built, the Producer worker will distribute rtpData to the downstream subscription Consumer Worker.

[0111] After establishing a RelayTransport connection, a channel is established between the Producer and Consumer Workers, and real-time transmission of audio and video data begins. During this process, the Producer worker sends RTP (Real-time Transport Protocol) data packets to the RtpRelayIO component, which then distributes these packets to the subscriber Consumer workers.

[0112] 8. The Consumer Client receives the distribution stream from the corresponding Worker through WebRTCTransport.

[0113] Video conferencing services can be implemented based on MediaSoup, which involves the following three Transport types:

[0114] WebRtcTransport: Used for media transport over WebRTC connections. This transport supports data encryption and provides sophisticated network topology tracing and congestion control features.

[0115] PlainRtpTransport: Used for media streaming between non-WebRTC applications and devices. This transport does not support data encryption or network congestion control, but it is inexpensive, simple, and easy to use.

[0116] PipeTransport can be seen as an upgraded version of PlainRTpTransport. It offers the same features as PlainRTpTransport, while also providing more advanced functions such as symmetric encryption, robustness, dynamic routing, and slow start. PipeTransport is typically used in media gateways or other advanced application environments that require handling large amounts of media streaming data.

[0117] This implementation provides an operation process for the pull request corresponding to the pull end in a video conferencing service. Based on the above operation process, the pull end in the video conferencing service can conveniently view the live streaming media data in the live streaming service, thereby improving the interactivity between the video conferencing service and the live streaming service.

[0118] In some optional implementations of this embodiment, the first service is a video conferencing service, and the second service is a live streaming service. In this implementation, the executing entity can perform step 201 as follows:

[0119] First, based on the streaming request from the streaming end of the first service, generate the streaming media data producer object corresponding to the streaming end of the first service.

[0120] Second, based on the streaming media data producer object, obtain the video conferencing streaming media data from the first service's push end.

[0121] As an example, first, the ProducerClient creates a WebRtcTransport connection and establishes the connection using the signaling channel provided by the SignalServer. Then, upon receiving the streaming request, the SignalServer sends a CreateProducer request to the MediasoupWorker to create a streaming media data producer object. Next, upon receiving the CreateProducer request, the MediasoupWorker asynchronously calls AsyncBuildRouter and RtpProducer to build routes and register the RTP producer. During this process, the MediasoupWorker generates an SDP offer containing the media transport protocol port to which the client can push media data. Finally, after receiving the SDP offer request, the ProducerClient selects an appropriate encoding format and network parameters for encoding and sends the RTP data to the port provided by the MediasoupWorker. Simultaneously, it also uses the signaling channel provided by the SignalServer to send an Answer message responding to the SDP change back to the SignalServer.

[0122] This implementation provides an operation process for a push request corresponding to the push end in a video conferencing service.

[0123] In some optional implementations of this embodiment, where the first service is a video conferencing service and the second service is a live streaming service, the execution entity can perform step 203 as follows:

[0124] First, based on the received streaming request from the second service's streaming client, the streaming requests are sequentially routed back to the source in the first data source, the second data source, and the first relay transmission module. Then, based on the connection, the streaming requests in the first relay transmission module are routed back to the source in the second relay transmission module.

[0125] As an example, firstly, the streaming player sends the streaming request to the LiveServer; then, the LiveServer sends the streaming request back to the RtmpSource module; then, the RtmpSource sends the streaming request back to the RtcSource module; then, the RtcSource module sends the streaming request back to the RelayioSource module; finally, the RelayioSource module triggers a routing operation, sending the streaming request back to the live streaming service.

[0126] Second, based on the connection, the video conferencing streaming media data is sequentially callbackd through the first relay transmission module, the second data source, and the first data source via the second relay module, and the callback-received video conferencing streaming media data is distributed to the pull end of the second service.

[0127] As an example, after the RelayioSource module receives the video conferencing streaming media data, it calls back to the RtcSource module through the Relayio2RtcBridge module. The RtcSource then calls back the RTP data to the RtmpSource through the Rtc2RtmpBridge. The Player receives the video conferencing streaming media data distributed by the LiveServer.

[0128] This implementation provides an operation process for the pull request corresponding to the pull end in the live streaming service. Based on the above operation process, the pull end in the live streaming service can conveniently view the streaming media data in the video conferencing service, thereby improving the interactivity between the video conferencing service and the live streaming service.

[0129] Continue to refer to Figure 5 The illustration shows a schematic flow 500 of another embodiment of the streaming media data transmission method according to this application, including the following steps:

[0130] Step 501: Based on the streaming request from the streaming end of the live streaming service, obtain live streaming media data using a preset initial transmission protocol and a preset initial audio and video encoding format.

[0131] Step 502: Convert the transmission protocol and encoding format of the live streaming media data to obtain live streaming media data using a preset target transmission protocol and a preset target audio and video encoding format.

[0132] Step 503: Determine whether the received streaming request comes from the streaming client of the video conferencing service.

[0133] Step 504: In response to the determination, based on the pull request and connection, determine the live streaming media data in the first relay transmission module of the live streaming service through the second relay transmission module in the video conferencing service.

[0134] Step 505: Convert the acquired live streaming media data to a transmission protocol to obtain live streaming media data using the fifth transmission protocol.

[0135] Step 506: Distribute the converted live streaming data to the pull-streaming end of the video conferencing service.

[0136] As can be seen from this embodiment, with Figure 2 Compared to the corresponding embodiments, the process 500 of the streaming media data transmission method in this embodiment specifically describes the process by which the pull end of the video conferencing service obtains the live streaming media data from the push end of the live streaming service. This can reduce the latency introduced by the relay line nodes, while also achieving the effect of live streaming into the meeting. Furthermore, by cascading the live streaming service with the video conferencing service, it can meet the needs of bypass live streaming, cloud recording, etc., and can further optimize and save distribution bandwidth. Moreover, the video conferencing service cluster can support large-scale cascading deployment, greatly improving concurrency capabilities.

[0137] Continue to refer to Figure 6 The illustration shows a schematic flow 600 of another embodiment of the streaming media data transmission method according to this application, including the following steps:

[0138] Step 601: Based on the streaming request from the streaming end of the video conferencing service, generate a streaming media data producer object corresponding to the streaming end of the video conferencing service.

[0139] Step 602: Based on the streaming media data producer object, obtain the video conferencing streaming media data from the push end of the video conferencing service.

[0140] Step 603: Determine whether the received streaming request comes from the streaming client of the live streaming service.

[0141] Step 604: In response to the determination, based on the received streaming request from the streaming client of the live streaming service, the streaming request is sequentially routed back to the source in the first data source, the second data source, and the first relay transmission module, and based on the connection, the streaming request in the first relay transmission module is routed back to the source in the second relay transmission module.

[0142] Step 605: Based on the connection, the video conferencing streaming media data is sequentially sent back through the first relay transmission module, the second data source, and the first data source via the second relay module, and then the video conferencing streaming media data after the callback is distributed to the streaming end of the live streaming service.

[0143] As can be seen from this embodiment, with Figure 2Compared to the corresponding embodiments, the process 600 of the streaming media data transmission method in this embodiment specifically describes the process by which the pull end of the live streaming service obtains the video conferencing streaming media data from the push end of the video conferencing service. This can meet the demand for obtaining streaming media data from the video conferencing service during the live streaming process and improve the interactivity between the live streaming service and the video conferencing service.

[0144] Continue to refer to Figure 7 As an implementation of the methods shown in the above figures, this application provides an embodiment of a streaming media data transmission device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0145] like Figure 7 As shown, the streaming media data transmission device 700 includes: an acquisition unit 701 configured to acquire streaming media data to be pushed based on a push request from a push end of a first service, wherein the first service is one of a live streaming service and a video conferencing service; a determination unit 702 configured to determine whether a received pull request comes from a pull end of a second service, wherein the second service is another of a live streaming service and a video conferencing service; and a transmission unit 703 configured to, in response to the determination, transmit the streaming media data to the pull end of the second service based on the connection between the first service and the second service.

[0146] In some optional implementations of this embodiment, the above apparatus further includes: a connection establishment unit (not shown in the figure), configured to: determine a first relay transmission module in the first service for relaying streaming media data, and a second relay transmission module in the second service for relaying streaming media data; and establish a connection between the first service and the second service based on the first relay transmission module and the second relay transmission module.

[0147] In some optional implementations of this embodiment, the connection unit (not shown in the figure) is further configured to: determine, based on the received custom operation, a first relay transmission module in the first service for relaying streaming media data, and a second relay transmission module in the second service for relaying streaming media data.

[0148] In some optional implementations of this embodiment, the first service is a live streaming service, the second service is a video conferencing service, and the acquisition unit 701 is further configured to: acquire live streaming media data using a preset initial transmission protocol and a preset initial audio and video encoding format based on the streaming request from the streaming end of the first service; and convert the live streaming media data by transmission protocol and encoding format to obtain live streaming media data using a preset target transmission protocol and a preset target audio and video encoding format.

[0149] In some optional implementations of this embodiment, the acquisition unit 701 is further configured to: in response to determining that the preset initial transmission protocol is the first transmission protocol and the preset initial audio and video encoding format is the first audio and video encoding format, perform transmission protocol conversion on the acquired live streaming media data to obtain live streaming media data using the second transmission protocol and the first audio and video encoding format, and relay the converted live streaming media data to the first data source; perform format conversion and encoding format conversion on the live streaming media data in the first data source to obtain live streaming media data using the third transmission protocol and the second audio and video encoding format, and relay the converted live streaming media data to the second data source; perform format conversion on the live streaming media data in the second data source to obtain live streaming media data using a custom fourth transmission protocol and the second audio and video encoding format, and relay the converted live streaming media data to the first relay transmission module, wherein the fourth transmission protocol is a preset target transmission protocol and the second audio and video encoding format is a preset target audio and video encoding format.

[0150] In some optional implementations of this embodiment, the first service is a live streaming service, the second service is a video conferencing service, and the transmission unit 703 is further configured to: determine the live streaming media data in the first relay transmission module of the first service through the second relay transmission module in the second service based on the pull request and connection; convert the acquired live streaming media data according to the transmission protocol to obtain live streaming media data using the fifth transmission protocol; and distribute the converted live streaming media data to the pull end of the second service.

[0151] In some optional implementations of this embodiment, the first service is a video conferencing service, the second service is a live streaming service, and the acquisition unit 701 is further configured to: generate a streaming media data producer object corresponding to the streaming end of the first service based on the streaming request of the streaming end of the first service; and acquire the video conferencing streaming media data of the streaming end of the first service based on the streaming media data producer object.

[0152] In some optional implementations of this embodiment, the first service is a video conferencing service, the second service is a live streaming service, and the transmission unit 703 is further configured to: according to the received pull request from the pull end of the second service, sequentially perform source return in the first data source, the second data source, and the first relay transmission module, and based on the connection, send the pull request in the first relay transmission module back to the second relay transmission module; based on the connection, sequentially perform callback of the video conferencing streaming media data in the first relay transmission module, the second data source, and the first data source through the second relay module, and distribute the callback-received video conferencing streaming media data to the pull end of the second service.

[0153] In this embodiment, the acquisition unit in the streaming media data transmission device acquires the streaming media data to be pushed based on the push request of the push end of the first service, wherein the first service is one of a live streaming service and a video conferencing service; the determination unit determines whether the received pull request comes from the pull end of the second service, wherein the second service is another of a live streaming service and a video conferencing service; in response to the determination, the transmission unit transmits the streaming media data to the pull end of the second service based on the connection between the first service and the second service, thereby realizing the effect of live streaming to RTC (Real-Time Communication) and RTC to live streaming based on the connection between the live streaming service and the video conferencing service; on the one hand, by cascading the live streaming service with the video conferencing service, the latency introduced by the relay line node can be reduced, and the effect of live streaming to the meeting can also be realized; on the other hand, by cascading the video conferencing service with the live streaming service, the needs of bypass live streaming and cloud recording can be met, which can further optimize and save distribution bandwidth. Moreover, the video conferencing service cluster can support large-scale cascading deployment, which greatly improves the concurrency capability.

[0154] The following is for reference. Figure 8 It illustrates a device suitable for implementing embodiments of this application (e.g., Figure 1 The diagram shows the structure of the computer system 800 of the devices 101, 102, 103, and 105. Figure 8 The device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0155] like Figure 8 As shown, the computer system 800 includes a processor (e.g., CPU, Central Processing Unit) 801, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 802 or a program loaded from storage section 808 into random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the system 800. The processor 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0156] The following components are connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to I / O interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 810 as needed so that computer programs read from it can be installed into storage section 808 as needed.

[0157] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by processor 801, it performs the functions defined in the methods of this application.

[0158] It should be noted that the computer-readable medium of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0159] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the client computer, partially on the client computer, as a standalone software package, partially on the client computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the client computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0161] The units described in the embodiments of this application can be implemented in software or hardware. The described units can also be housed in a processor; for example, it can be described as: a processor including an acquisition unit, a determination unit, and a transmission unit. The names of these units do not necessarily limit the unit itself; for example, the transmission unit can also be described as "a unit that, in response to a determination, transmits streaming media data to the streaming end of the second service based on the connection between the first service and the second service."

[0162] In another aspect, this application also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the computer device to: acquire streaming media data to be streamed based on a streaming request from a streaming end of a first service, wherein the first service is one of a live streaming service or a video conferencing service; determine whether the received streaming request originates from a streaming end of a second service, wherein the second service is another of a live streaming service or a video conferencing service; and, in response to the determination that it does, transmit the streaming media data to the streaming end of the second service based on the connection between the first service and the second service.

[0163] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for transmitting streaming media data, comprising: Identify the first relay transmission module in the first service used for relaying streaming media data, and the second relay transmission module in the second service used for relaying streaming media data. Based on the first relay transmission module and the second relay transmission module, a connection is established between the first service and the second service; Based on the streaming request from the streaming end of the first service, the following steps are taken: First, live streaming media data to be streamed is obtained using a preset initial transmission protocol and a preset initial audio / video encoding format, wherein the first service is a live streaming service. Second, in response to determining that the preset initial transmission protocol is a first transmission protocol and the preset initial audio / video encoding format is a first audio / video encoding format, the obtained live streaming media data undergoes transmission protocol conversion to obtain live streaming media data using a second transmission protocol and a first audio / video encoding format, and the converted live streaming media data is relayed to a first data source. Third, the live streaming media data in the first data source undergoes format conversion and encoding format conversion to obtain live streaming media data using a third transmission protocol and a preset target audio / video encoding format, and the converted live streaming media data is relayed to a second data source. Fourth, the live streaming media data in the second data source undergoes format conversion to obtain live streaming media data using a custom preset target transmission protocol and the preset target audio / video encoding format, and the converted live streaming media data is relayed to the first relay transmission module. Determine whether the received streaming request originates from the streaming client of the second service, wherein the second service is a video conferencing service; In response to the determination, the streaming media data is transmitted to the pull end of the second service based on the connection between the first service and the second service.

2. The method according to claim 1, wherein, The determination of the first relay transmission module in the first service for relaying streaming media data and the second relay transmission module in the second service for relaying streaming media data includes: Based on the received custom operation, a first relay transmission module in the first service for relaying streaming media data and a second relay transmission module in the second service for relaying streaming media data are determined.

3. The method according to claim 1, wherein, The first service is a live streaming service, the second service is a video conferencing service, and The step of transmitting the streaming media data to the pull-streaming end of the second service based on the connection between the first service and the second service includes: Based on the streaming request and the connection, the live streaming media data in the first relay transmission module of the first service is determined through the second relay transmission module in the second service; The acquired live streaming media data is converted to a transmission protocol to obtain live streaming media data using the fifth transmission protocol; The converted live streaming data is distributed to the pull stream end of the second service.

4. The method according to claim 1, wherein, The first service is a video conferencing service, the second service is a live streaming service, and The streaming request based on the streaming end of the first service, obtaining the streaming media data to be streamed, includes: Based on the streaming request from the streaming end of the first service, a streaming media data producer object corresponding to the streaming end of the first service is generated. Based on the streaming media data producer object, obtain the video conferencing streaming media data from the push end of the first service.

5. The method according to claim 4, wherein, The first service is a video conferencing service, the second service is a live streaming service, and The step of transmitting the streaming media data to the pull-streaming end of the second service based on the connection between the first service and the second service includes: Based on the received streaming request from the second service's streaming client, the streaming requests are sequentially routed back to the source in the first data source, the second data source, and the first relay transmission module. Based on the connection, the streaming requests in the first relay transmission module are routed back to the source in the second relay transmission module. Based on the connection, the video conferencing streaming media data is sequentially routed through the first relay transmission module, the second data source, and the first data source via the second relay transmission module, and then distributed to the pull end of the second service after the callback.

6. A streaming media data transmission device, comprising: The connection unit is configured to: determine a first relay transmission module in the first service for relaying streaming media data, and a second relay transmission module in the second service for relaying streaming media data. Based on the first relay transmission module and the second relay transmission module, a connection is established between the first service and the second service; The acquisition unit is configured to acquire streaming media data to be streamed based on a streaming request from the streaming end of the first service, including: acquiring live streaming media data using a preset initial transmission protocol and a preset initial audio / video encoding format based on a streaming request from the streaming end of the first service, wherein the first service is a live streaming service; in response to determining that the preset initial transmission protocol is a first transmission protocol and the preset initial audio / video encoding format is a first audio / video encoding format, performing transmission protocol conversion on the acquired live streaming media data to obtain live streaming media data using a second transmission protocol and a first audio / video encoding format, and relaying the converted live streaming media data to a first data source; performing format conversion and encoding format conversion on the live streaming media data in the first data source to obtain live streaming media data using a third transmission protocol and a preset target audio / video encoding format, and relaying the converted live streaming media data to a second data source; performing format conversion on the live streaming media data in the second data source to obtain live streaming media data using a custom preset target transmission protocol and the preset target audio / video encoding format, and relaying the converted live streaming media data to the first relay transmission module; The determining unit is configured to determine whether the received streaming request comes from the streaming end of the second service, wherein the second service is a video conferencing service; The transmission unit is configured to, in response to a determination, transmit the streaming media data to the pull end of the second service based on the connection between the first service and the second service.

7. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

8. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

Images