Audio and video transmission method and system

Abstract

The invention discloses an audio and video transmission method and system, and relates to the technical field of audio and video transmission, and the method comprises the following steps: 1, building a communication link, specifically, initializing connection; adding a local media track; the sending end creates a session invitation and sends the session invitation to the receiving end, and after receiving the session invitation, the receiving end sets remote description, creates a session response and returns the session response to the sending end; the sending end sets far-end description; the two parties start to exchange interactive connection to establish candidate addresses; and after the channel connection is successful, monitoring data or media streams. And step 2, data transmission in an actual channel is carried out: when point-to-point connection is successfully established (the connection supports remote peer-to-peer connection, maintenance and monitoring, and the connection is closed when no connection is needed), data transmission is carried out through a data construction transmission mode model. And step 3, confirming the basic flow of audio and video transmission. According to the method, the content is packaged into the pull stream interface and the push stream interface, aiming at the asynchronous execution characteristic of webpage real-time communication, all the content is packaged into success and failure callback with an asynchronous result object, and each step has state return.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of audio and video transmission, in particular to an audio and video transmission method and system. BACKGROUND

[0002] Audio and video transmission is a complex technical process of efficiently, stably and real-timely transmitting sound and video data from one end to another end and playing it out, the core challenge of which lies in huge data volume, unstable network and strict real-time requirement. Currently, the original interactive connection establishment has no awareness or processing complexity problem, and cannot be aware in the first time when the channel is disconnected. The original point-to-point connection has a clear push-pull distinction, the original media stream cannot process a single media type independently, the original event-driven mode is not intuitive, the chain call is difficult to maintain, and there are problems of debugging difficulty and log dispersion. SUMMARY

[0003] The present application aims to provide an audio and video transmission method and system to solve the problems in the background.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical scheme: an audio and video transmission method, comprising the following steps: Step one: establishing a communication link, specifically including initializing a connection; adding a local media track; a sending end creating a session invitation and sending it to a receiving end; the receiving end receiving the session invitation, setting a far-end description and creating a session response, and returning it to the sending end; the sending end setting the far-end description; both sides start exchanging interactive connection establishment candidate addresses; after the channel connection is successful, data or media stream is listened to; Step two: data transmission process in the actual channel: when the point-to-point connection is successfully established, a transmission mode model is constructed through data, and a plurality of transmission modes are configured inside the transmission mode model; Step three: confirming the basic process of audio and video transmission, formulating a protocol model after confirmation, formulating a scene transmission model after the completion of the protocol model, and realizing the corresponding transmission scheme in different scenes.

[0005] Preferably, in step one, the connection process of establishing a communication link specifically includes creating an instance through a real-time communication point-to-point connection; using a create session invitation and a create response to exchange a session description protocol.

[0006] Preferably, in step one, the process specifically includes one end calling a create session invitation to generate a session description protocol session invitation; the other end replies with a create response after setting a far-end description; and finally both sides call a set local description and a set far-end description to complete session parameter negotiation.

[0007] Preferably, in the step one, the interactive connection is used to establish the exchange candidate address to select the optimal transmission path; the interactive connection establishment helps the client to pass through the NAT / firewall and find a feasible connection path, and when the interactive connection establishment state becomes connected or completed, the channel is established, which is mainly used for capability negotiation and path finding.

[0008] Preferably, in the step two, the transmission mode of the transmission mode model includes passing through the media channel and the data channel; the encryption and integrity protection are completed through the datagram transport layer security protocol; the encrypted transmission of the audio and video is completed through the secure real-time transport protocol; and the transmission mode is based on the protocol and the channel after the successful negotiation to carry out the actual data transmission.

[0009] Preferably, in the step three, the audio and video transmission process includes a sending end, a network transmission link and a receiving end. The sending end includes audio and video data collection, encoding, encapsulation and transmission; the network transmission link is an unstable network environment, and there is a risk of packet loss, out-of-order, delay in the transmission process; and the receiving end specifically includes receiving, buffering, decapsulation, decoding, synchronization and rendering steps.

[0010] Preferably, in the step three, the protocol model specifically includes the real-time message protocol based on the transmission control, the video stream in the Flash video format transmitted through the hypertext transfer protocol, the hypertext transfer protocol live streaming, the real-time transport protocol / real-time transport control protocol, the secure reliable transport protocol and the quick user datagram internet connection protocol.

[0011] Preferably, in the step three, the scene transmission model specifically includes the video conference / online education, the low-delay live streaming, the pull streaming and the video on demand.

[0012] The communication link unit establishes a communication link, specifically including initializing connection; adding a local media track; the sending end creates a session invitation and sends it to the receiving end; the receiving end receives the session invitation, sets a far-end description and creates a session response, and returns it to the sending end; the sending end sets the far-end description; both sides start to exchange the interactive connection establishment candidate address; and after the channel connection is successful, the data or the media stream is listened to; The data transmission unit actually transmits data in the channel: when the point-to-point connection is successfully established, a transmission mode model is built through data, and a plurality of transmission modes are configured inside the transmission mode model. The comprehensive processing unit confirms the basic process of the audio and video transmission, formulates a protocol model after the confirmation is completed, formulates a scene transmission model after the protocol model is formulated, and realizes the use of the corresponding transmission scheme in different scenes.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: the content is encapsulated into two independent interfaces, a pull stream interface and a push stream interface; in view of the asynchronous execution characteristics of real-time web page communication, all content is encapsulated into success / failure callbacks with asynchronous result objects, and each execution step returns the corresponding status; In a one-sided scenario, users can pull streams while pushing them. In this method, pulling and pushing streams rely on a real-time, bidirectional communication address between the webpage and the server. This address is returned as a wrapper object through a success callback function.

[0014] Events include mandatory events for real-time web communication and custom events, such as joining a room and obtaining the current client support status. Attached Figure Description

[0015] Figure 1 A schematic diagram of the overall structure is provided for an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1 Please see Figure 1 This invention provides a technical solution: an audio and video transmission method, comprising the following steps: Step 1: Establish a communication link, which includes initializing the connection; adding local media tracks; the sender creating a session invitation and sending it to the receiver; the receiver receiving the session invitation, setting the remote description and creating a session response, and returning it to the sender; the sender setting the remote description; both parties starting to exchange interactive connection establishment candidate addresses; after the channel connection is successfully established, listening for data or media streams; Step 2: Data transmission process in the actual channel: Once the point-to-point connection is successfully established, a transmission mode model is constructed using data, and several transmission modes are configured within the transmission mode model; Step 3: Confirm the basic process of audio and video transmission. After confirmation, formulate a protocol model. After the protocol model is formulated, formulate a scenario transmission model to enable the use of corresponding transmission schemes in different scenarios.

[0018] In step one, the process of establishing a communication link specifically includes creating an instance through a real-time communication point-to-point connection; and using the session description protocol to create a session invitation and create a response exchange. In step one, the process specifically includes one end calling to create a session invitation to generate a session description protocol session invitation; the other end using to set a remote description and then using to create a response; finally, both parties calling to set a local description and set a remote description to complete the negotiation of session parameters.

[0019] In step one, an interactive connection is used to establish exchange candidate addresses and select the optimal transmission path. The purpose of establishing the interactive connection is to help the client traverse NAT / firewalls and find feasible connection paths. The specific process of the connection path includes collecting candidate addresses locally, sending the candidate addresses to the other end via signaling, both parties attempting to connect and punch holes based on the candidate addresses, and selecting the optimal path. When the interactive connection establishment status changes to "connected" or "completed", the channel is established. The core of this process lies in protocol capability negotiation and reachable path finding.

[0020] In step two, the transmission mode of the transmission mode model includes transmission through media channels and data channels; encryption and integrity protection are accomplished through datagram transport layer security protocols; encrypted audio and video transmission is accomplished through secure real-time transmission protocols; the transmission mode carries out actual data transmission based on the successfully negotiated protocol and channel, and has high optimization and security.

[0021] In step three, the audio and video transmission process includes a sending end, a network transmission link, and a receiving end. The sending end includes the acquisition, encoding, encapsulation, and transmission of audio and video data; the network transmission link is an unstable network environment, and data packets are susceptible to loss, out-of-order delivery, and delay during transmission; the receiving end specifically includes receiving, buffering, decapsulation, decoding, synchronization, and rendering steps.

[0022] In step three, the specific components of the protocol model include a real-time message transmission protocol, a video stream in Flash video format transmitted via Hypertext Transfer Protocol, a real-time streaming protocol based on Hypertext Transfer Protocol, a real-time transmission protocol / real-time transmission control protocol, a secure and reliable transmission protocol, and a Fast User Datagram Internet Connection Protocol.

[0023] In step three, the specific application scenarios of the scene transmission model include: video conferencing, online education, low-latency live streaming push and pull, and video on demand.

[0024] Example 2 An audio and video transmission system includes a communication link unit, a data transmission unit, and a comprehensive processing unit; The communication link unit establishes a communication link, specifically including initializing the connection; adding a local media track; the sender creating a session invitation and sending it to the receiver; the receiver receiving the session invitation, setting a remote description and creating a session response, and returning it to the sender; the sender setting a remote description; both parties starting to exchange interactive connection establishment candidate addresses; and monitoring data or media streams after the channel connection is successful. The data transmission process in the actual channel of the data transmission unit is as follows: when the point-to-point connection is successfully established, a transmission mode model is constructed through the data, and several transmission modes are configured within the transmission mode model; The integrated processing unit confirms the basic process of audio and video transmission. After confirmation, it formulates a protocol model. After the protocol model is formulated, it formulates a scenario transmission model to enable the use of corresponding transmission schemes in different scenarios.

[0025] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0026] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An audio-video transmission method, characterized in that: The method comprises the following steps: Step 1: establishing a communication link, specifically comprising the following steps: initializing connection; Adding local media track; the sending end creates a session invitation and sends it to the receiving end; the receiving end receives the session invitation, sets the remote description and creates a session response, and returns it to the sending end; the sending end sets the remote description; both parties start exchanging interactive connection establishment candidate addresses; after the channel connection is successful, data or media stream is listened to; Step 2: data transmission process in the actual channel: when the point-to-point connection is successfully established, a transmission mode model is constructed through data, and several transmission modes are configured inside the transmission mode model; Step 3: confirming the basic flow of audio and video transmission, confirming the protocol model after confirmation, and then formulating a scene transmission model to realize the corresponding transmission scheme in different scenes.

2. The audio-video transmission method of claim 1, wherein: In step 1, the connection process of establishing a communication link specifically comprises creating an instance through a real-time communication point-to-point connection; the session description protocol is exchanged by using the created session invitation and response.

3. The audio-video transmission method of claim 2, wherein: In step 1, the process specifically comprises one end calling the created session invitation to generate a session description protocol session invitation; the other end replies with the created response after setting the remote description; finally, both parties call the local description and set the remote description.

4. The audio-video transmission method of claim 3, wherein: In step 1, the interactive connection establishment exchanges candidate addresses to select the optimal transmission path; the interactive connection establishment is used to help the client to pass through the NAT / firewall and find a feasible connection path, when the interactive connection establishment state becomes connected or completed, the channel is established, and it is mainly used for negotiating capabilities and finding reachable paths.

5. The audio-video transmission method of claim 4, wherein: In step 2, the transmission mode of the transmission mode model includes through the media channel and the data channel; wherein the encryption and integrity protection are completed through the datagram transport layer security protocol; the encrypted transmission of audio and video is completed through the secure real-time transport protocol; the transmission mode is based on the protocol and channel after successful negotiation to carry out actual data transmission.

6. The audio-video transmission method of claim 5, wherein: In step 3, the audio and video transmission flow includes a sending end, a network transmission link and a receiving end; wherein the sending end includes audio and video data collection, encoding, encapsulation and transmission; the network transmission link is an unstable network environment, and there is a risk of packet loss, out-of-order and delay in the transmission process.

7. The audio-video transmission method of claim 6, wherein: In step 3, the receiving end specifically includes the steps of receiving, buffering, decapsulating, decoding, synchronizing and rendering.

8. The audio-video transmission method of claim 7, wherein: In step 3, the protocol model specifically includes real-time message protocol, video stream in flash video format transmitted through hypertext transfer protocol, hypertext transfer protocol live stream, real-time transport protocol / real-time transport control protocol, Secure and reliable transmission protocol, fast user datagram internet connection protocol.

9. The audio-video transmission method of claim 8, wherein: In step 3, the scene transmission model specifically includes video conference / online education, low-delay live streaming, pull streaming and video on demand.

10. An audio / video transmission system characterized by: The audio and video transmission system is suitable for the audio and video transmission method and system of any one of claims 1-9, comprising a communication link unit, a data transmission unit and a comprehensive processing unit; The communication link unit establishes a communication link, specifically including initializing connection; adding a local media track; a sending end creating a session invitation and sending it to a receiving end; the receiving end receiving the session invitation, setting a far-end description and creating a session response and returning it to the sending end; the sending end setting the far-end description; both sides starting to exchange interactive connection establishment candidate addresses; and listening to data or media flow after a channel connection is successful; The data transmission unit is configured to build a transmission mode model through data when a point-to-point connection is successfully established, and configure several transmission modes inside the transmission mode model; The comprehensive processing unit confirms a basic flow of audio and video transmission, formulates a protocol model after confirmation is completed, formulates a scene transmission model after the protocol model is formulated, and realizes corresponding transmission schemes in different scenes.

Images