Browser end video scheduling method, system and device of converged communication platform and medium

Abstract

The invention discloses a browser video scheduling method, system and device of a converged communication platform and a medium, belongs to the technical field of video coding and decoding, a video streaming protocol RTSP and a front end, and aims to solve the technical problem of how to gradually transfer a playing tool of a converged communication platform conference video from desktop end software to a browser webpage. According to the technical scheme, video streams decoded by a client side are transmitted to a browser side on the basis of a two-way real-time protocol Websocket, a webpage drawing container Canvas is used for rendering pictures, a rendering assembly at the webpage side is packaged into a front-end video stream playing assembly, and the front-end video stream playing assembly is used for playing the front-end video stream. The cross-platform, cross-browser and cross-terminal browser conference picture playing is realized; the method specifically comprises the following steps: transmitting conference picture video streams in real time; the desktop end plug-in decodes the video stream; and carrying out picture rendering by using a webpage drawing container Canvas.

Description

Technical Field

[0001] The invention relates to the fields of video encoding and decoding, video stream transmission protocol RTSP and front-end technology, and in particular to a browser-side video scheduling method, system, equipment and medium of a fusion communication platform. Background Art

[0002] The converged communication platform uses digital and networked technologies to integrate traditional voice communication, data communication, image transmission, video conferencing and other communication methods, achieving a fully interconnected, cross-platform, smooth and seamless communication experience. This helps break down information barriers between city departments, promote information sharing and collaborative work between city departments, and thus improve the overall efficiency of city management.

[0003] At present, most of the converged communication platforms on the market use the method of central server decoding and distributing video streams to the conference terminal devices of the participants of the scheduling meeting. This method first increases the operating pressure of the central service, and at the same time, the volume of the decoded video stream increases, which increases the bandwidth cost of the system operation. Most of the converged communication platforms on the market only support desktop software for scheduling conference related operations. As modern browsers become more and more powerful, modern browsers already support the playback of videos in a variety of mainstream formats. At the same time, web applications have the advantages of no need to install additional plug-ins, cross-platform, and easy to update. Therefore, how to gradually transfer the playback tool of the converged communication platform conference video from the desktop software to the browser web page, reduce the operating broadband cost, and realize cross-platform, cross-browser, and cross-terminal browser playback is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The technical task of the present invention is to provide a browser-side video scheduling method, system, device and medium for a converged communication platform to solve the problem of how to gradually transfer the playback tool of the converged communication platform conference video from the desktop software to the browser web page, reduce the operating broadband cost, and realize cross-platform, cross-browser and cross-terminal browser playback.

[0005] The technical task of the present invention is achieved in the following way: a browser-side video scheduling method for a converged communication platform, which is based on the two-way real-time protocol Websocket to transmit the decoded video stream of the client to the browser side, use the web page drawing container Canvas to render the picture, and encapsulate the rendering component of the web page side as a front-end video stream playback component to achieve cross-platform, cross-browser, and cross-terminal browser playback of the conference picture; the details are as follows:

[0006] Real-time transmission of conference screen video stream;

[0007] Desktop plug-in for video stream decoding;

[0008] Use the web drawing container Canvas to render the screen.

[0009] As a preferred method, the real-time transmission of the conference screen video stream is as follows:

[0010] After receiving the video conference command, the central server merges the screens of each participant into a whole screen and transmits it to the desktop plug-in of the participant via the RTSP protocol;

[0011] Among them, RTSP (Real-time Streaming Protocol) is a protocol used to control data transmission between streaming media servers and clients. It supports real-time video and audio transmission. At the same time, RTSP has the advantages of high compression rate and small video stream size.

[0012] As a preferred embodiment, the desktop plug-in performs video stream decoding as follows:

[0013] The desktop plug-in has a built-in interface codec library FFmpeg; the interface codec library FFmpeg is an open source multimedia framework that can decode all known audio and video formats;

[0014] Use the interface codec library FFmpeg to convert the RTSP video stream into YUV420 color decoding;

[0015] The decrypted video stream is transmitted to the Canvas playback component on the web page via Websocket.

[0016] Preferably, OpenGL is used to decode the RTSP video stream through a desktop plug-in, thereby reducing the pressure on the central server.

[0017] Preferably, the web page drawing container Canvas is used to render the screen as follows:

[0018] After the web page receives the video stream transmitted by Websocket, it uses the Canvas playback component to render the picture frame by frame;

[0019] The Canvas playback component dynamically adjusts the number of frames per second by monitoring the current browser playback performance;

[0020] Deliver conference control instructions to the desktop plug-in via Websocket.

[0021] A browser-side video scheduling system for a converged communication platform, the system comprising a central server, a desktop-side plug-in and a web page:

[0022] The central server is used to merge the screens of each participant into a whole screen and transmit it to the desktop plug-in of the participant via RTSP protocol after receiving the video conference instruction;

[0023] The desktop plug-in is used to decode the video stream;

[0024] The web page is used to render the screen using the web drawing container Canvas.

[0025] Preferably, the desktop plug-in has a built-in interface codec library FFmpeg, and uses the interface codec library FFmpeg to convert the RTSP video stream into YUV420 color decoding, and the decrypted video stream is transmitted to the Canvas playback component of the web page through Websocket.

[0026] Preferably, after the web page receives the video stream transmitted by Websocket, it uses the Canvas playback component to render the picture frame by frame. The Canvas playback component monitors the current browser playback performance, dynamically adjusts the number of playback frames per second, and transmits conference control instructions to the desktop plug-in through Websocket.

[0027] An electronic device comprising: a memory and at least one processor;

[0028] Wherein, the memory stores a computer program;

[0029] The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the browser-side video scheduling method of the converged communication platform as described above.

[0030] A computer-readable storage medium stores a computer program, and the computer program can be executed by a processor to implement the browser-side video scheduling method of the converged communication platform as described above.

[0031] The browser-side video scheduling method, system, device and medium of the converged communication platform of the present invention have the following advantages:

[0032] (1) The present invention realizes the real-time transmission of conference screen video streams, and the client efficiently decodes and shares the decoding pressure of the central server, while the video screen supports the complete solution of browser playback;

[0033] (ii) The present invention transmits the decoded video stream from the client to the browser based on the two-way real-time protocol Websocket, uses the web drawing container Canvas to render the picture, and the rendering component of the web page can be encapsulated as a front-end video stream playback component, which has scalability, inheritability and integrability;

[0034] (III) The present invention can share the decoding pressure of the central server, reduce the bandwidth cost of system operation, and can play the conference screen in the browser across platforms, browsers and terminals;

[0035] (IV) The present invention realizes the overall converged communication browser-side playback, which allows users to view the video conference screen with low latency; from the developer's perspective, it saves the central server performance and bandwidth costs; from the open source perspective, it is integrable, scalable, and inheritable;

[0036] (V) The present invention can transmit video conference screens in real time: the real-time streaming protocol RTSP adopted can transmit video conference video streams with low delay;

[0037] (VI) The present invention can save bandwidth costs: the real-time streaming protocol RTSP adopted has a high compression ratio characteristic, and can transmit video conferencing video streams in a smaller volume, thereby reducing the system operation bandwidth cost;

[0038] (VII) The present invention can reduce the pressure on the central server: by using OpenGL to decode the RTSP video stream through the client desktop plug-in, the pressure on the central server is reduced and the stability of the central server is increased;

[0039] (VIII) The present invention supports web conference video playback: the web drawing container Canvas is used to render the screen, which can be encapsulated as a front-end video stream playback component, and has scalability, inheritability and integrability;

[0040] (IX) The present invention uses the video stream transmission protocol RTSP and front-end related technologies Websocket and Canvas, and is implemented with a modern browser as a carrier, making full use of the efficient rendering of the Canvas element in HTML5 and the rich API to realize the conference screen playback function. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The present invention is further described below in conjunction with the accompanying drawings.

[0042] Attached Figure 1 The flowchart is a flow chart of a browser-side video scheduling method for a converged communication platform. DETAILED DESCRIPTION

[0044] The browser-side video scheduling method, system, device and medium of the converged communication platform of the present invention are described in detail below with reference to the accompanying drawings and specific embodiments of the specification.

[0045] Embodiment 1:

[0046] As attached Figure 1As shown, this embodiment provides a browser-side video scheduling method for a converged communication platform. The method is based on the two-way real-time protocol Websocket to transmit the decoded video stream of the client to the browser side, use the web page drawing container Canvas to render the picture, and encapsulate the rendering component of the web page side as a front-end video stream playback component to achieve cross-platform, cross-browser, and cross-terminal browser playback of the conference picture; the details are as follows:

[0047] S1, real-time transmission of conference screen video stream;

[0048] S2, desktop plug-in to decode video stream;

[0049] S3. Use the web page drawing container Canvas to render the picture.

[0050] The real-time transmission of the conference screen video stream in step S1 of this embodiment is as follows:

[0051] After receiving the video conference command, the central server merges the screens of each participant into a whole screen and transmits it to the desktop plug-in of the participant via the RTSP protocol;

[0052] Among them, RTSP (Real-time Streaming Protocol) is a protocol used to control data transmission between streaming media servers and clients. It supports real-time video and audio transmission. At the same time, RTSP has the advantages of high compression rate and small video stream size.

[0053] The video stream decoding performed by the desktop plug-in in step S2 of this embodiment is specifically as follows:

[0054] S201, the desktop plug-in has a built-in interface codec library FFmpeg; wherein the interface codec library FFmpeg is an open source multimedia framework that can decode all known audio and video formats;

[0055] S202, using the interface codec library FFmpeg to convert the RTSP video stream into YUV420 color decoding;

[0056] S203, the decrypted video stream is transmitted to the Canvas playback component on the web page via Websocket.

[0057] In this embodiment, the RTSP video stream is decoded using OpenGL through a desktop plug-in, thereby reducing the pressure on the central server.

[0058] The specific steps of using the web page drawing container Canvas to render the screen in step S3 of this embodiment are as follows:

[0059] S301, after receiving the video stream transmitted by Websocket, the web page uses the Canvas playback component to render the picture frame by frame;

[0060] S302, the Canvas playback component dynamically adjusts the number of playback frames per second by monitoring the current browser playback performance;

[0061] S303. Deliver conference control instructions to the desktop plug-in via Websocket.

[0062] Embodiment 2:

[0063] This embodiment provides a browser-side video scheduling system for a converged communication platform, which includes a central server, a desktop-side plug-in, and a web page:

[0064] The central server is used to merge the screens of each participant into a whole screen and transmit it to the desktop plug-in of the participant via RTSP protocol after receiving the video conference instruction;

[0065] The desktop plug-in is used to decode the video stream;

[0066] The web page is used to render the screen using the web drawing container Canvas.

[0067] The desktop plug-in in this embodiment has a built-in interface codec library FFmpeg, which uses the interface codec library FFmpeg to convert the RTSP video stream into YUV420 color decoding, and the decrypted video stream is transmitted to the Canvas playback component on the web page through Websocket.

[0068] After the web page in this embodiment receives the video stream transmitted by Websocket, it uses the Canvas playback component to render the picture frame by frame. The Canvas playback component dynamically adjusts the number of playback frames per second by monitoring the current browser playback performance, and transmits conference control instructions to the desktop plug-in through Websocket.

[0069] Embodiment 3:

[0070] This embodiment also provides an electronic device, including: a memory and a processor;

[0071] Wherein, the memory stores computer-executable instructions;

[0072] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the browser-side video scheduling method of the converged communication platform in any embodiment of the present invention.

[0073] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or any conventional processor, etc.

[0074] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can also include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage period, a flash memory device, or other volatile solid-state storage devices.

[0075] Embodiment 4:

[0076] This embodiment also provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions are loaded by a processor, so that the processor executes the browser-side video scheduling method of the converged communication platform in any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code that implements the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0077] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0078] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0079] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0080] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A browser-side video scheduling method for a converged communication platform, characterized in that: The method is based on the two-way real-time protocol Websocket to transmit the decoded video stream from the client to the browser, use the web drawing container Canvas to render the screen, and encapsulate the rendering component of the web page as a front-end video stream playback component to achieve cross-platform, cross-browser, and cross-terminal browser playback of the conference screen; the details are as follows: Real-time transmission of conference screen video stream; Desktop plug-in for video stream decoding; Use the web drawing container Canvas to render the screen.

2. The browser-side video scheduling method of the converged communication platform according to claim 1 is characterized in that: The real-time transmission of conference screen video stream is as follows: After receiving the video conference command, the central server merges the screens of each participant into a whole screen and transmits it to the desktop plug-in of the participant via the RTSP protocol; Among them, RTSP is a protocol used to control data transmission between a streaming media server and a client.

3. The browser-side video scheduling method of the converged communication platform according to claim 1 is characterized in that: The desktop plug-in decodes the video stream as follows: The desktop plug-in has a built-in interface codec library FFmpeg; the interface codec library FFmpeg is an open source multimedia framework that can decode all known audio and video formats; Use the interface codec library FFmpeg to convert the RTSP video stream into YUV420 color decoding; The decrypted video stream is transmitted to the Canvas playback component on the web page via Websocket.

4. The browser-side video scheduling method of the converged communication platform according to claim 1 is characterized in that: Use OpenGL to decode RTSP video streams through desktop plug-ins, thereby reducing the pressure on the central server.

5. The browser-side video scheduling method of the converged communication platform according to any one of claims 1 to 4, characterized in that: The specific steps of using the web drawing container Canvas to render the screen are as follows: After the web page receives the video stream transmitted by Websocket, it uses the Canvas playback component to render the picture frame by frame; The Canvas playback component dynamically adjusts the number of frames per second by monitoring the current browser playback performance; Deliver conference control instructions to the desktop plug-in via Websocket.

6. A browser-side video scheduling system for a fusion communication platform, characterized in that: The system includes a central server, a desktop plug-in and a web page: The central server is used to merge the screens of each participant into a whole screen and transmit it to the desktop plug-in of the participant via RTSP protocol after receiving the video conference instruction; The desktop plug-in is used to decode the video stream; The web page is used to render the screen using the web drawing container Canvas.

7. The browser-side video scheduling system of the converged communication platform according to claim 6 is characterized in that: The desktop plug-in has a built-in interface codec library FFmpeg, which uses the interface codec library FFmpeg to convert the RTSP video stream into YUV420 color decoding, and the decrypted video stream is transmitted to the Canvas playback component of the web page through Websocket.

8. The browser-side video scheduling system of the converged communication platform according to claim 6 or 7, characterized in that: After the web page receives the video stream transmitted by Websocket, it uses the Canvas playback component to render the picture frame by frame. The Canvas playback component dynamically adjusts the number of playback frames per second by monitoring the current browser playback performance, and transmits conference control instructions to the desktop plug-in through Websocket.

9. An electronic device, characterized in that: include: memory and at least one processor; Wherein, the memory stores a computer program; The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the browser-side video scheduling method of the converged communication platform as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which can be executed by a processor to implement the browser-side video scheduling method of the converged communication platform as described in any one of claims 1 to 5.

Images