A video stream transmission method and apparatus

Abstract

The present invention discloses a video stream transmission method and device, which includes: establishing a SIP communication channel based on the WebSocket protocol; using the SIP communication channel to forward a SIP server registration request containing user information to a SIP server via a GB access platform; obtaining the video code of a target video acquisition device; selecting a video stream channel type to establish with a streaming media server based on the video code, browser version, and decoder support status of the target video acquisition device; creating a first video stream channel with the streaming media server through a first SDP negotiation with the streaming media server, or creating a second video stream channel with the streaming media server through a second SDP negotiation with the streaming media server, based on the selected video stream channel type; and receiving a video stream from the target video acquisition device sent by the streaming media server. Based on the WebRTC compatible GB / T28181 protocol framework, the method implements video stream transmission and playback in both H.264 and H.265 encoding formats.

Description

Technical Field

[0001] The present invention relates to the technical field of streaming media transmission, and in particular to a video stream transmission method and device. Background Art

[0002] At present, there is an increasing demand in the security industry for watching videos in browser plug-in-free mode. The existing mainstream plug-in-free video playback method generally uses WebRTC (Web Real-Time Communication) technology to ensure low-latency video playback.

[0003] The video streaming data source for video playback in the security industry needs to rely on surveillance video sources based on the GB / T 28181 protocol, including GB / T 28181 video acquisition equipment and GB / T 28181 video surveillance networking platforms. GB / T 28181 video surveillance networking platforms generally include streaming media servers and SIP servers.

[0004] While video playback solutions using WebRTC technology offer low latency, they are incompatible with video data based on the GB / T 28181 protocol. Chinese invention patent application number 201911203239.8 discloses a method for previewing GB / T 28181-compliant surveillance videos across browsers based on the WebRTC protocol. By integrating WebRTC technology within the framework of the GB / T 28181 standard, this approach expands the adaptability of GB / T 28181-compliant software applications to multiple operating system platforms, including Windows and Android, and multiple browser environments, including Chrome and Firefox. This enables browser-side playback of GB / T 28181-compliant surveillance video streams based on the WebRTC protocol.

[0005] However, this technology is not compatible with the transmission and playback of the two video encoding formats H.264 and H.265 specified by the video data provider (server) based on the GB / T 28181 protocol, nor can it dynamically select the implementation method of WebRTC communication before each video stream is received, establish a suitable video stream channel, and achieve the optimal playback effect. Summary of the Invention

[0006] The purpose of the present invention is to provide a video stream transmission method and device to solve the technical problems of being unable to play video streams in the H.264 and H.265 encoding formats specified by the video data provider (server) based on the GB / T28181 protocol, and being unable to dynamically select the implementation mode of WebRTC communication before each video stream is received, establish a suitable video stream channel, and achieve the optimal playback effect.

[0007] In one aspect, the present invention provides a video stream transmission method, which is applied to a browser and includes the following steps:

[0008] Send a WebSocket connection request to the GB access platform to establish a WebSocket connection with the GB access platform. At the same time, the GB access platform establishes a WebSocket connection with the SIP server to complete the establishment of a SIP communication channel based on the WebSocket protocol.

[0009] Using the SIP communication channel, forwarding the registration SIP server request containing user information to the SIP server through the GB access platform to establish a playback permission mapping table between the user and the video capture device;

[0010] Get the video code of the target video acquisition device;

[0011] Select the type of video stream channel to be established with the streaming media server based on the video encoding, browser version, and decoder support of the target video capture device;

[0012] According to the type of video stream channel selected for establishment, using the SIP communication channel, a first video stream channel between the streaming media server and the streaming media server is established through a first SDP negotiation between the streaming media server and the streaming media server, or a second video stream channel between the streaming media server and the streaming media server is established through a second SDP negotiation between the streaming media server and the streaming media server;

[0013] After receiving the response from the streaming media server that the first video stream channel or the second video stream channel is successfully established, the video stream of the target video acquisition device is sent through the established first video stream channel or the second video stream channel, and rendered and played.

[0014] In some embodiments, obtaining the video code of the target video acquisition device specifically includes:

[0015] In response to a user clicking on a target video capture device, a SIP communication channel based on the WebSocket protocol is used to send a request for obtaining the video code of the target video capture device to the SIP server through the GB access platform. The request for obtaining the video code of the target video capture device includes the target video capture device number, so that the SIP server obtains the video code based on the code of the target video capture device. If the SIP server determines that the user does not have the playback permission for the target video capture device, the SIP server is unable to obtain the video code of the target video capture device.

[0016] Receive the video code of the target video acquisition device sent by the SIP server via the SIP communication channel.

[0017] In some embodiments, the type of video stream channel established with the streaming server is selected based on the video encoding, browser version, and decoder construction support of the target device to be played, specifically including: first determining whether the browser version supports the webrtc function, if so, continuing to determine whether the video encoding of the target video acquisition device is H.265 or H.264, if it is H.264, selecting to create the first video stream channel with the streaming server, if it is H.265, determining whether it supports building a decoder for decoding H.265 video, if so, selecting to create the second video stream channel with the streaming server.

[0018] In some embodiments, the determination of whether to support the construction of a decoder for decoding H.265 video further includes: determining whether hardware support or software support is used to construct a decoder for decoding H.265 video, and saving the determination result; after the browser receives the video stream from the target video capture device, the determination result is used to construct a hardware decoder or a software decoder for rendering and playback.

[0019] In some embodiments, the step of establishing a first video stream channel with the streaming server through a first SDP negotiation with the streaming server specifically includes:

[0020] The SIP communication channel based on the WebSocket protocol sends the first SDP offer to the SIP server through the GB access platform;

[0021] The SIP server parses the target video acquisition device number in the first SDP offer and determines whether it has the playback permission for the target video acquisition device;

[0022] If yes, send a first SDP offer to the streaming server, so that the streaming server generates a first SDP answer according to the first SDP offer, where the first SDP answer at least includes connection information of the streaming server, a media type identifier for establishing a session, and a first SSRC;

[0023] receiving a first SDP answer sent by the streaming server via the SIP communication channel, completing the first SDP negotiation, calling the browser's WEBRTC protocol API (application programming interface) to construct a session media type identifier based on the streaming server's connection information, and completing the WEBRTC session with the streaming server. With the completion of the WEBRTC session, the first video streaming channel is established;

[0024] The step of creating a second video stream channel with the streaming media server through a second SDP negotiation with the streaming media server specifically includes:

[0025] The SIP communication channel based on the WebSocket protocol sends the second SDP offer to the SIP server through the GB access platform;

[0026] The SIP server parses the target video acquisition device number in the second SDP offer and determines whether it has the playback permission for the target video acquisition device;

[0027] If yes, send a second SDP offer to the streaming server, so that the streaming server generates a second SDP answer according to the second SDP offer, where the second SDP answer at least includes the connection information of the streaming server, the media type identifier of the established session, and the second SSRC;

[0028] The second SDP answer is received from the streaming server via the SIP communication channel, the second SDP negotiation is completed, and the browser's WEBRTC protocol API (application programming interface) is called to construct a session media type identifier based on the streaming server's connection information, thereby completing the WEBRTC session with the streaming server. With the completion of the WEBRTC session, the second video streaming channel is established.

[0029] In some embodiments, the video stream of the target video acquisition device is obtained by the streaming server according to the target video acquisition device number after the streaming server responds to the successful establishment of the first video stream channel or the second video stream channel.

[0030] In some embodiments, the video stream of the target video capture device carries an SSRC, which is determined by the streaming media server when generating an SDP answer according to an SDP offer, and is included in the SDP answer and sent to the browser.

[0031] On the other hand, the present invention also provides a video stream transmission device, which is applied to a browser and includes:

[0032] The communication channel establishment module is used to send a WebSocket connection request to the GB access platform, establish a WebSocket connection with the GB access platform, and simultaneously enable the GB access platform to establish a WebSocket connection with the SIP server, thus completing the establishment of a SIP communication channel based on the WebSocket protocol;

[0033] The registration management module uses the SIP communication channel to forward the registration SIP server request containing user information to the SIP server through the GB access platform to establish a playback permission mapping table between the user and the video capture device;

[0034] Video code acquisition module, used to obtain the video code of the target video acquisition device;

[0035] The video stream channel type selection module selects the type of video stream channel to be established with the streaming media server based on the video encoding, browser version, and decoder support of the target video acquisition device;

[0036] A video stream channel establishment module uses the SIP communication channel to create a first video stream channel with the streaming server through a first SDP negotiation with the streaming server, or to create a second video stream channel with the streaming server through a second SDP negotiation with the streaming server;

[0037] The rendering and playback module is used to receive the video stream of the target video acquisition device sent through the established first video stream channel or second video stream channel after the streaming media server responds to the successful establishment of the first video stream channel or the second video stream channel, and render and play it.

[0038] The beneficial effects of the present invention are as follows:

[0039] Based on the advantages of the real-time, low latency, and low bandwidth consumption of the WEBRTC protocol, the present invention is compatible with the GB / T28181 protocol specification communication. It can enable the browser to play video streams in the H.264 and H.265 encoding formats specified by the video data provider (server) based on the GB / T28181 protocol. Moreover, the WEBRTC construction method is selected before the video stream is transmitted. Compared with conventional implementation methods (first establishing a video stream channel, and then creating another video stream channel if the video stream in the H.264 or H.265 encoding format cannot be played), it can reduce one WEBRTC link construction, further saving resource consumption and reducing construction duplication.

[0040] In order to ensure the playback effect, the present invention combines the video encoding, browser version, decoder construction support of the target video acquisition device and multiple factors to select the type of video stream channel established with the streaming media server;

[0041] After the video stream channel between the browser and the streaming media server is established, the browser no longer needs to send a video stream request. The streaming media server automatically sends the video stream of the target video acquisition device to the browser, simplifying the process. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 A flowchart of a video stream transmission method provided by one embodiment of the present invention;

[0044] Figure 2 A block diagram of the principle of a video stream transmission device provided by an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of a video stream transmission process provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0047] The video stream transmission method of the present invention involves a browser, a GB access platform, a SIP server, and a streaming media server, wherein:

[0048] Browser: refers to various types of browsers such as Chrome, Edge, and Firefox

[0049] GB access platform: complies with the GB / T 28181 protocol standard, has protocol parsing, construction, and extended access functions, and can achieve bidirectional conversion and forwarding between the GB / T 28181 national standard protocol and the WEBRTC protocol;

[0050] SIP server: complies with GB / T 28181 protocol standard and has functions such as registration management, routing forwarding, and session management.

[0051] Streaming media server: supports GB / T 28181 protocol video device access, video stream parsing and sending functions, and supports WEBRTC protocol session management and status maintenance functions.

[0052] Please refer to Figure 1 An embodiment of the present invention provides a video stream transmission method, which is applied to a browser and includes the following steps:

[0053] S1. Send a WebSocket connection request to the GB access platform to establish a WebSocket connection with the GB access platform. At the same time, the GB access platform establishes a WebSocket connection with the SIP server, completing the establishment of a SIP communication channel based on the WebSocket protocol.

[0054] After entering the user information and clicking the login button, the establishment of the SIP communication channel is triggered. Specifically, the browser sends a WebSocket connection request to the GB access platform to establish a WebSocket connection between the browser and the GB access platform. When the connection with the browser is successfully established, the GB access platform sends a websocket connection request to the SIP server through the SIP over WebSocket protocol, and a WebSocket connection is established between the GB access platform and the SIP server.

[0055] S2. Using the SIP communication channel, forwarding the registration SIP server request containing the user information to the SIP server through the GB access platform to establish a playback permission mapping table between the user and the video capture device;

[0056] After the SIP communication channel is established in step S1, the registration task will automatically begin:

[0057] S201. Using a SIP communication channel based on the WebSocket protocol, the browser sends a SIP server registration request to the GB access platform. The request information includes user information of the current browser web application.

[0058] S202: The GB access platform receives the SIP server registration request from the browser, stores the user information carried in the request, and then assembles the request into the SIP data format of the GB / T28181 protocol based on the user information. It then uses the SIP over WebSocket protocol to initiate a SIP server registration request to the SIP server.

[0059] S203: The SIP server receives the registration request from the GB access platform, parses and stores the request information in SIP data format, and then generates a registration request result in SIP data format. The result includes a registration result response (success / failure), a unique registration session number, and other information, and sends it to the GB access platform using the SIP over WebSocket protocol.

[0060] S204. The GB access platform receives the registration request result from the SIP server, parses it to obtain the registration result response, and if successful, stores and records the registration request result and sends the registration result to the browser through the SIP communication channel. If unsuccessful, the registration failure result is sent to the browser through the SIP communication channel. After that, the browser disconnects the SIP communication channel among the browser, the GB access platform, and the SIP server.

[0061] Registering a SIP server is used by the SIP server to manage user permissions. It can determine the user's playback permissions for the registered video capture device, and can also push alarms and push notifications when the device is online or offline.

[0062] S3. Get the video code of the target video acquisition device:

[0063] In some embodiments, in response to a user clicking on a target video capture device, the browser uses a SIP communication channel based on the WebSocket protocol to send a request for obtaining the video code of the target video capture device to the SIP server through the GB access platform. The request for obtaining the video code of the target video capture device includes the target video capture device number, so that the SIP server obtains the video code based on the code of the target video capture device. If the SIP server determines that the user does not have playback permission for the target video capture device, the SIP server is unable to obtain the video code of the target video capture device.

[0064] The SIP server sends the video code of the target video acquisition device to the browser via the SIP communication channel.

[0065] S4. Select the type of video stream channel to be established with the streaming media server based on the video encoding, browser version, and decoder support of the target video acquisition device;

[0066] After obtaining the video code, the method of the present invention needs to first determine the method of building the webrtc protocol communication. The present invention mainly adopts two methods, one is to use the PeerConnection technology to build the webrtc protocol communication and establish the first video stream channel, and the other is to use the DataChannel technology to build the webrtc protocol communication and establish the second video stream channel.

[0067] In some embodiments, it is first determined whether the browser version supports the webrtc function. If not, a pop-up message will be displayed indicating that it does not support the function. If it supports the function, it is further determined whether the video encoding of the target video acquisition device is H.265 or H.264. If it is H.264, the first video stream channel between the target video acquisition device and the streaming media server is created. If it is H.265, it is determined whether it supports building a decoder for decoding H.265 video. If it supports the function, the second video stream channel between the target video acquisition device and the streaming media server is created. If it does not support the function, a pop-up message indicating that it does not support the function is displayed.

[0068] In some embodiments, the determination of whether to support the construction of a decoder for decoding H.265 video further includes: determining whether hardware supports or software supports the construction of a decoder for decoding H.265 video, and saving the determination result; after the browser receives the video stream from the target video capture device, based on the determination result, constructing a hardware decoder or a software decoder for rendering and playback; if both hardware and software support it, then choosing to construct a hardware decoder.

[0069] S5. Based on the selected video stream channel type, using the SIP communication channel, establish a first video stream channel with the streaming media server through a first SDP negotiation with the streaming media server, or establish a second video stream channel with the streaming media server through a second SDP negotiation with the streaming media server;

[0070] The process of establishing the first video stream channel:

[0071] The browser sends the first SDP offer to the GB access platform based on the SIP communication channel of the WebSocket protocol. The GB access platform assembles the first SDP offer into SIP format data and sends it to the SIP server using the SIP over WebSocket protocol. The first SDP offer not only contains the target video capture device number, but also includes the offer user name, offer authentication code, browser IP address, port and other connection information, and constructs the session media type identifier (peer connection or data channel) and signaling information (request ID, device ID, etc.).

[0072] The SIP server parses the target video capture device number in the first SDP offer and determines whether it has the playback permission for the target video capture device;

[0073] If yes, send a first SDP offer to the streaming server, so that the streaming server generates a first SDP answer according to the first SDPoffer, wherein the first SDP answer includes at least the connection information of the streaming server, the establishment session media type identifier and the first SSRC; the connection information of the streaming server includes the IP address and port of the streaming server;

[0074] receiving a first SDP answer sent by the streaming server via the SIP communication channel, completing the first SDP negotiation, calling the browser's WEBRTC protocol API (application programming interface) to construct a session media type identifier based on the streaming server's connection information, and completing the WEBRTC session with the streaming server. With the completion of the WEBRTC session, the first video streaming channel is established;

[0075] The process of establishing the second video stream channel:

[0076] The SIP communication channel based on the WebSocket protocol sends the second SDP offer to the SIP server through the GB access platform;

[0077] The SIP server parses the target video acquisition device number in the second SDP offer and determines whether it has the playback permission for the target video acquisition device;

[0078] If yes, send a second SDP offer to the streaming server, so that the streaming server generates a second SDP answer according to the second SDP offer, where the second SDP answer at least includes the connection information of the streaming server, the media type identifier of the established session, and the second SSRC;

[0079] The second SDP answer is received from the streaming server via the SIP communication channel, the second SDP negotiation is completed, and the browser's WEBRTC protocol API (application programming interface) is called to construct a session media type identifier based on the streaming server's connection information, thereby completing the WEBRTC session with the streaming server. With the completion of the WEBRTC session, the second video streaming channel is established.

[0080] S6. After receiving a response from the streaming media server that the first video stream channel or the second video stream channel is successfully established, the video stream of the target video acquisition device sent through the established first video stream channel or the second video stream channel is rendered and played.

[0081] The present invention combines the establishment of a video stream channel and the sending of a video stream application. After the video stream channel between the browser and the streaming media server is established, the browser no longer needs to send a video stream request. The streaming media server automatically sends the video stream of the target video acquisition device to the browser. The video stream of the target video acquisition device is obtained by the streaming media server according to the target video acquisition device number and carries the SSRC. The SSRC is determined by the streaming media server when generating an SDP answer based on the SDP offer, and is included in the SDP answer and sent to the browser.

[0082] The present invention can be compatible with the GB / T28181 protocol specification communication based on the advantages of real-time, low latency and low bandwidth consumption of the WEBRTC protocol. It can realize the browser-side playback of video streams in the H.264 and H.265 encoding formats specified by the video data provider (server) based on the GB / T28181 protocol. Moreover, before the video stream is transmitted, the WEBRTC construction method is judged and selected. Compared with the conventional implementation method (first establishing a video stream channel, and if the video stream in the H.264 or H.265 encoding format cannot be played, then creating another video stream channel), it can reduce one WEBRTC link construction, further save resource consumption, and reduce construction duplication.

[0083] like Figure 2 As shown, another aspect of the present invention further provides a video stream transmission device, which is applied to a browser and includes:

[0084] The communication channel establishment module is used to send a WebSocket connection request to the GB access platform, establish a WebSocket connection with the GB access platform, and simultaneously enable the GB access platform to establish a WebSocket connection with the SIP server, thus completing the establishment of a SIP communication channel based on the WebSocket protocol;

[0085] A registration management module is used to use the SIP communication channel to forward a registration SIP server request containing user information to the SIP server through the GB access platform to establish a playback permission mapping table between the user and the video capture device;

[0086] Video code acquisition module, used to obtain the video code of the target video acquisition device;

[0087] The video stream channel type selection module selects the type of video stream channel to be established with the streaming media server based on the video encoding, browser version, and decoder support of the target video acquisition device;

[0088] A video stream channel establishment module is used to use the SIP communication channel to establish a first video stream channel with the streaming media server through a first SDP negotiation with the streaming media server, or to establish a second video stream channel with the streaming media server through a second SDP negotiation with the streaming media server according to the type of video stream channel selected for establishment;

[0089] The rendering and playback module is used to receive the video stream of the target video acquisition device sent through the established first video stream channel or second video stream channel after the streaming media server responds to the successful establishment of the first video stream channel or the second video stream channel, and render and play it.

[0090] The specific methods that can be implemented by the functional modules in the device of the present invention are described in the above-mentioned method and technical solutions, which will not be described in detail here.

[0091] Let's take a specific example as follows Figure 3 , introduce the video stream transmission process of the present invention.

[0092] 301. The user side (browser) prepares the WebSocket connection environment, and the browser sends a WebSocket connection establishment request to the GB access platform to establish a WebSocket connection between the browser and the GB access platform.

[0093] 302. The GB access platform receives the WebSocket connection request from the browser. When the connection with the browser is successfully established, the GB access platform sends a WebSocket connection request to the SIP server to establish a WebSocket connection between the GB access platform and the SIP server.

[0094] 303. The SIP server receives the SIP over WebSocket connection request from the GB access platform and maintains the connection with the GB access platform. The SIP server stores the connection information to ensure the uniqueness of the connection. The connection information includes the URL of the current connection session, the object of the current connection session, and the format of the subsequent transmitted data content.

[0095] At this point, the SIP communication channel is established and a long connection is maintained, enabling information exchange between the browser and the GB access platform, and between the GB access platform and the SIP server.

[0096] 304. The user side (browser) uses the SIP communication channel based on the WebSocket protocol to send a registration SIP server request to the GB access platform: the request information includes the user information of the current browser Web application, which is used by the SIP server to perform registration management and determine the user's playback authority for the registered video capture device.

[0097] 305. The GB access platform receives the SIP server registration request from the browser, stores the user information carried in the request, and then assembles the request into the SIP data format of the GB / T28181 protocol according to the user information, and uses the SIP over WebSocket protocol to initiate a SIP server registration request to the SIP server.

[0098] 306. The SIP server receives the registration request from the GB access platform, parses and stores the request information in SIP data format, and then generates a registration request result in SIP data format. The SIP server sends the registration request result to the GB access platform using the SIP over WebSocket protocol. The registration request result includes a registration result response (success / failure), a unique registration session number, and other information.

[0099] 307. The GB access platform receives the registration request result from the SIP server, parses it to obtain the registration result response, and if successful, stores and records the registration request result and sends the registration result to the browser via the SIP communication channel. If unsuccessful, the browser receives the registration failure result via the SIP communication channel, and then the browser disconnects the SIP communication channel among the browser, the GB access platform, and the SIP server.

[0100] After the user successfully logs in using the account and password, 301-307 is automatically completed.

[0101] 308. When the user clicks on the target video acquisition device for the video to be played, a video streaming service request is triggered. First, a request to obtain the device video encoding needs to be sent through the SIP communication channel.

[0102] 309. The GB access platform receives the device video encoding request sent by the browser through the SIP communication channel, converts the request into SIP data format, and sends it to the SIP server through the SIP over WebSocket protocol.

[0103] 310. The SIP server receives the device video code acquisition request sent by the browser through the SIP communication channel, parses the SIP data format, obtains the device number, obtains the corresponding video code based on the device number, and then assembles the video code result into the SIP data format and responds to the GB access platform through the SIP communication channel.

[0104] 311. The GB access platform receives the video encoding result of the device to be played back from the SIP server, parses it into the actual video encoding result of the device to be played back, and sends the result to the video stream channel type selection module of the browser.

[0105] 312. Determine the actual WEBRTC link construction method to be adopted, that is, the type of video stream channel to be established.

[0106] The detailed judgment method is: first determine whether the browser version supports the webrtc function. If not, a "Not Supported" pop-up will appear. If supported, continue to determine whether the video encoding of the target video capture device is H.265 or H.264. If it is H.264, choose to use the WEBRTC protocol PeerConnection technology to create the first video stream channel between the streaming server. If it is H.265, determine whether it supports building a decoder for decoding H.265 video. If supported, choose to use the WEBRTC protocol DataChannel technology to create the second video stream channel between the streaming server. If not supported, a "Not Supported" pop-up will appear.

[0107] This step is to prepare for the establishment of a WEBRTC session. The purpose is to more accurately determine which WEBRTC session establishment method should be used. This will avoid improperly establishing a WEBRTC session in the early stages, which may result in video playback failure and waste of resources.

[0108] If the WEBRTC protocol PeerConnection technology is used, execute steps 313-320; if the WEBRTC protocol DataChannel technology is used, execute steps 321-328.

[0109] 313. Determine to use PeerConnection technology to build a WEBRTC session. The browser sends a first SDP offer message to the GB access platform through the SIP communication channel.

[0110] 314. After receiving the first SDP offer sent by the browser, the GB access platform assembles the data into SIP data format and sends it to the SIP server using the SIP over Websocket protocol.

[0111] 315. The SIP server receives the SIP data sent by the GB access platform, parses and obtains the target video acquisition device number in the first SDP offer, determines whether it has the playback permission for the target video acquisition device, and transmits the SIP data format that actually carries the first SDP offer to the streaming media server via the SIP over TCP protocol.

[0112] 316. The streaming media server receives the SIP data sent by the SIP server, parses the data format to obtain the first SDP offer information carried therein, generates a first SDP answer, reassembles the SDP answer into SIP data, and responds to the SIP server via the SIP over TCP protocol.

[0113] The first SDP answer is used to respond to the browser's session negotiation, which carries the video transmission session SSRC of the current device to be played, and uses the SSRC as the unique synchronization source identifier of the current session.

[0114] 317. The SIP server receives the SIP data responded by the streaming server and sends the data to the GB access platform using the SIP over WebSocket protocol.

[0115] 318. The GB access platform receives the SIP data responded by the SIP server, parses the current SIP data, obtains the first SDP answer data actually carried, and sends the data to the browser.

[0116] 319. When the browser receives the first SDP answer data responded by the GB access platform, parses the connection information of the streaming server from the first SDP answer message, constructs the session media type identifier, and then initiates the establishment of a WEBRTC session with the streaming server.

[0117] At this point, the first video stream channel is successfully established.

[0118] 320. Relying on the first video stream channel, the streaming media server transmits the video stream data of the device to be played to the browser. The browser calls the internal API to perform rendering and playback operations on the video stream data, thereby completing the playback function of the video stream data.

[0119] The process of 321-328 is similar to that of 313-320 and can be combined Figure 3 understand.

[0120] The above describes in detail the video stream transmission method and apparatus provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only intended to help understand the method and core concept of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from the principles of the present invention, and such improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0121] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the partial description of the method embodiments.

Claims

1. A video stream transmission method, characterized in that: The method is applied to a browser and includes the following steps: Send a WebSocket connection request to the national standard (GB) access platform to establish a WebSocket connection with the GB access platform. At the same time, the GB access platform establishes a WebSocket connection with the SIP server, completing the establishment of a SIP communication channel based on the WebSocket protocol. Using the SIP communication channel, forwarding the registration SIP server request containing user information to the SIP server through the GB access platform to establish a playback permission mapping table between the user and the video capture device; Obtaining the video code of a target video capture device: In response to a user clicking on a target video capture device, a SIP communication channel based on the WebSocket protocol is used to send a request for obtaining the video code of the target video capture device to a SIP server via the GB access platform. The request for obtaining the video code of the target video capture device includes the target video capture device number, so that the SIP server obtains the video code according to the target video capture device number. If the SIP server determines that the user does not have playback permission for the target video capture device, the SIP server is unable to obtain the video code of the target video capture device. Receiving the video code of the target video acquisition device sent by the SIP server via the SIP communication channel; Select the type of video stream channel to be established with the streaming media server based on the video encoding, browser version, and decoder support of the target video capture device; According to the type of video stream channel selected for establishment, using the SIP communication channel, a first video stream channel between the streaming media server and the streaming media server is established through a first SDP negotiation between the streaming media server and the streaming media server, or a second video stream channel between the streaming media server and the streaming media server is established through a second SDP negotiation between the streaming media server and the streaming media server; After receiving the response from the streaming media server that the first video stream channel or the second video stream channel is successfully established, the video stream of the target video acquisition device is sent through the established first video stream channel or the second video stream channel, and rendered and played.

2. The video stream transmission method according to claim 1, wherein: Based on the video encoding, browser version, and decoder construction support of the target device to be played, choose the type of video stream channel to be established with the streaming server, specifically including: first determine whether the browser version supports the web instant messaging (WEBRTC) function. If it does, continue to determine whether the video encoding of the target video acquisition device is H.265 or H.

264. If it is H.264, choose to create the first video stream channel with the streaming server. If it is H.265, determine whether it supports building a decoder for decoding H.265 video. If it does, choose to create the second video stream channel with the streaming server.

3. The video stream transmission method according to claim 2, wherein: The determination of whether a decoder for decoding H.265 video is supported may further include: determining whether hardware support or software support is provided for constructing a decoder for decoding H.265 video, and saving the determination result; after the browser receives the video stream from the target video capture device, constructing a hardware decoder or a software decoder for rendering and playback according to the determination result.

4. The video stream transmission method according to claim 1, wherein: The step of establishing a first video stream channel with the streaming media server through a first SDP negotiation with the streaming media server specifically includes: The SIP communication channel based on the WebSocket protocol sends the first SDP offer to the SIP server through the GB access platform; The SIP server parses the target video acquisition device number in the first SDP offer and determines whether it has the playback permission for the target video acquisition device; If yes, sending a first SDP offer to the streaming server, so that the streaming server generates a first SDP answer according to the first SDP offer, where the first SDP answer at least includes connection information of the streaming server, a media type identifier of the established session, and a first synchronization source identifier (SSRC); receiving a first SDP answer sent by the streaming server via the SIP communication channel, completing the first SDP negotiation, calling the browser's WEBRTC protocol application program interface (API) to construct a session media type identifier based on the streaming server's connection information, and completing the WEBRTC session with the streaming server. With the completion of the WEBRTC session, the first video streaming channel is established; The step of creating a second video stream channel with the streaming media server through a second SDP negotiation with the streaming media server specifically includes: The SIP communication channel based on the WebSocket protocol sends the second SDP offer to the SIP server through the GB access platform; The SIP server parses the target video acquisition device number in the second SDP offer and determines whether it has the playback permission for the target video acquisition device; If yes, send a second SDP offer to the streaming server, so that the streaming server generates a second SDP answer according to the second SDP offer, where the second SDP answer at least includes the connection information of the streaming server, the media type identifier of the established session, and the second SSRC; The second SDP answer is received from the streaming server via the SIP communication channel, the second SDP negotiation is completed, the browser's WEBRTC protocol API is called to construct a session media type identifier based on the streaming server's connection information, and the WEBRTC session with the streaming server is completed. With the completion of the WEBRTC session, the second video streaming channel is established.

5. The video stream transmission method according to claim 1, wherein: The video stream of the target video acquisition device is obtained by the streaming media server according to the target video acquisition device number after the streaming media server responds to the successful establishment of the first video stream channel or the second video stream channel.

6. The video stream transmission method according to claim 1 or 5, characterized in that: The video stream of the target video capture device carries SSRC, which is determined by the streaming media server when generating an SDP answer according to the SDP offer, and is included in the SDP answer and sent to the browser.

7. A video stream transmission device, applied to a browser, characterized in that: include: The communication channel establishment module is used to send a WebSocket connection request to the national standard (GB) access platform, establish a WebSocket connection with the GB access platform, and simultaneously enable the GB access platform to establish a WebSocket connection with the SIP server, thus completing the establishment of a SIP communication channel based on the WebSocket protocol; A registration management module is used to use the SIP communication channel to forward a registration SIP server request containing user information to the SIP server through the GB access platform to establish a playback permission mapping table between the user and the video capture device; The video code acquisition module is configured to acquire the video code of a target video acquisition device. In response to a user clicking on a target video acquisition device, the module uses a WebSocket-based SIP communication channel to send a request for acquiring the video code of the target video acquisition device to a SIP server via a GB access platform. The request for acquiring the video code of the target video acquisition device includes the target video acquisition device number, so that the SIP server acquires the video code based on the target video acquisition device number. If the SIP server determines that the user does not have playback permission for the target video acquisition device, the SIP server is prevented from acquiring the video code of the target video acquisition device. Receiving the video code of the target video acquisition device sent by the SIP server via the SIP communication channel; The video stream channel type selection module selects the type of video stream channel to be established with the streaming media server based on the video encoding, browser version, and decoder support of the target video acquisition device; A video stream channel establishment module is used to use the SIP communication channel to establish a first video stream channel with the streaming media server through a first SDP negotiation with the streaming media server, or to establish a second video stream channel with the streaming media server through a second SDP negotiation with the streaming media server according to the type of video stream channel selected for establishment; The rendering and playback module is used to receive the video stream of the target video acquisition device sent through the established first video stream channel or second video stream channel after the streaming media server responds to the successful establishment of the first video stream channel or the second video stream channel, and render and play it.

Images