Data transmission method, device, terminal equipment and storage medium

Abstract

An embodiment of the present invention provides a data transmission method, apparatus, terminal device and storage medium, the method comprising: a streaming media server receives video data in a first encoding format sent by a web client; based on the video data in the first encoding format, determines a picture set; based on the picture set and a pre-established canvas, determines a YUV format picture corresponding to the video data; converts the YUV format picture into a video stream in a second encoding format; and sends the video stream in the second encoding format to a visual networking terminal so that the web client and the visual networking terminal can perform business interaction. By using the embodiment of the present invention, there is no need to decode the video data in the VP8 or VP9 encoding format, which simplifies the conversion steps and improves the conversion efficiency.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a data transmission method, device, terminal equipment and storage medium. Background Art

[0002] In the prior art, when a web client conducts business with a visual network terminal based on the WebRTC protocol, the web client supports multiple encoding formats such as VP8, VP9 or H264. When converting VP8 / VP9 into the H264 encoding format, since the data encoded in the VP8 / VP9 encoding format is encapsulated, professional technicians are required to configure system parameters on the web client and then decode the data. The parameter configuration process is complicated and tedious, and the conversion efficiency is slow. In this process, not only professional training of technicians is required, but it also takes a long time, wasting manpower and time. Summary of the invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a data transmission method, apparatus, terminal device and storage medium that overcome the above problems or at least partially solve the above problems.

[0004] In a first aspect, an embodiment of the present invention provides a data transmission method, which is applied to a streaming media server, wherein the streaming media server is respectively connected to a web client and a video network terminal for communication, and the method comprises:

[0005] Receiving video data in a first encoding format sent by the web client;

[0006] Determining a picture set according to the video data in the first coding format;

[0007] Determine, according to the picture set and the pre-established canvas, a YUV format picture corresponding to the video data;

[0008] Convert the YUV format image into a video stream in a second encoding format;

[0009] The video stream in the second encoding format is sent to the visual networking terminal so that the web client and the visual networking terminal can perform business interaction.

[0010] Optionally, determining a picture set according to the video data in the first coding format includes:

[0011] Through the WebRTC decoding interface, the video data in the first encoding format is intercepted every first preset time period to obtain a picture set, wherein the picture format in the picture set includes at least one of BMP, JPG, JPEG or PNG.

[0012] Optionally, determining a YUV format picture corresponding to the video data according to the picture set and a pre-established canvas includes:

[0013] Inserting the pictures in the picture set into a pre-established canvas to generate a canvas picture set;

[0014] Capture the canvas image in the canvas image set to obtain a YUV format image.

[0015] Optionally, capturing a canvas picture in the canvas picture set to obtain a YUV format picture includes:

[0016] Every second preset time period, a canvas picture in the canvas picture set is captured to obtain the YUV format picture.

[0017] Optionally, converting the YUV format picture into a video stream in a second encoding format includes:

[0018] The YUV format image is encoded according to preset encoding parameters to obtain a video stream in the second encoding format, wherein the preset encoding parameters include at least one of frame rate, bit rate, resolution, I frame or P frame.

[0019] Optionally, the step of sending the video stream in the second encoding format to a visual networking terminal so that the web client and the visual networking terminal perform service interaction includes:

[0020] Encapsulating the video stream in the second coding format into a video stream of a visual networking protocol;

[0021] The video stream of the visual networking protocol is sent to the visual networking terminal so that the web client and the visual networking terminal can perform business interaction.

[0022] In a second aspect, an embodiment of the present invention provides a data transmission device, which is applied to a streaming media server, and the streaming media server is respectively connected to a web client and a visual network terminal for communication, and the device includes:

[0023] A receiving module, used for receiving video data in a first encoding format sent by the web client;

[0024] An interception module, configured to determine a picture set according to the video data in the first coding format;

[0025] A screen capture module, used to determine a YUV format picture corresponding to the video data according to the picture set and a pre-established canvas;

[0026] A conversion module, used for converting the YUV format picture into a video stream in a second encoding format;

[0027] The sending module is used to send the video stream in the second encoding format to the visual networking terminal so that the web client and the visual networking terminal can perform business interaction.

[0028] Optionally, the interception module is used to:

[0029] Through the WebRTC decoding interface, the video data in the first encoding format is intercepted every first preset time period to obtain a picture set, wherein the picture format in the picture set includes at least one of BMP, JPG, JPEG or PNG.

[0030] Optionally, the screen capture module is used to:

[0031] Inserting the pictures in the picture set into a pre-established canvas to generate a canvas picture set;

[0032] Capture the canvas image in the canvas image set to obtain a YUV format image.

[0033] Optionally, the screen capture module is specifically used for:

[0034] Every second preset time period, a canvas picture in the canvas picture set is captured to obtain the YUV format picture.

[0035] Optionally, the conversion module is used to:

[0036] The YUV format image is encoded according to preset encoding parameters to obtain a video stream in the second encoding format, wherein the preset encoding parameters include at least one of frame rate, bit rate, resolution, I frame or P frame.

[0037] Optionally, the sending module is used to:

[0038] Encapsulating the video stream in the second coding format into a video stream of a visual networking protocol;

[0039] The video stream of the visual networking protocol is sent to the visual networking terminal so that the web client and the visual networking terminal can perform business interaction.

[0040] In a third aspect, an embodiment of the present invention provides a terminal device, including: at least one processor and a memory;

[0041] The memory stores a computer program; and the at least one processor executes the computer program stored in the memory to implement the data transmission method provided in the first aspect.

[0042] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed, the data transmission method provided in the first aspect is implemented.

[0043] The embodiments of the present invention include the following advantages:

[0044] The embodiment of the present invention applies the characteristics of the visual network, and the streaming media server receives video data in a first encoding format sent by a web client; determines a picture set based on the video data in the first encoding format; determines a YUV format picture corresponding to the video data based on the picture set and a pre-established canvas; converts the YUV format picture into a video stream in a second encoding format; and sends the video stream in the second encoding format to the visual network terminal, so that the web client and the visual network terminal can perform business interaction. The embodiment of the present invention does not need to decode the video data in the VP8 or VP9 encoding format, which simplifies the conversion steps and improves the conversion efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flowchart of a data transmission method embodiment of the present invention;

[0046] Figure 2 is a flowchart of another data transmission method embodiment of the present invention;

[0047] Figure 3 is a flowchart of steps of another data transmission method embodiment of the present invention;

[0048] Figure 4 is a structural block diagram of an embodiment of a data transmission device of the present invention;

[0049] Figure 5 It is a structural schematic diagram of a terminal device of the present invention. DETAILED DESCRIPTION

[0050] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] The visual Internet is an important milestone in the development of the Internet. It is a real-time network that can realize real-time transmission of high-definition video and push many Internet applications towards high-definition video and high-definition face-to-face communication.

[0052] The visual network adopts real-time high-definition video exchange technology. It can integrate the required services such as high-definition video conferencing, video monitoring, intelligent monitoring analysis, emergency command, digital radio and television, delayed TV, online teaching, live broadcast, VOD on demand, TV mail, personalized recording (PVR), intranet (self-operated) channels, intelligent video broadcast control, information release and dozens of other video, voice, picture, text, communication, data and other services on a single system platform, and realize high-definition quality video playback through TV or computer.

[0053] Based on the above characteristics of the visual network, one of the core concepts of an embodiment of the present invention is proposed, that is, a streaming media server receives video data in a first encoding format sent by a web client; determines a picture set based on the video data in the first encoding format; determines a YUV format picture corresponding to the video data based on the picture set and a pre-established canvas; converts the YUV format picture into a video stream in a second encoding format; and sends the video stream in the second encoding format to a visual network terminal, so that the web client and the visual network terminal can perform business interaction. The embodiment of the present invention does not need to decode the video data in VP8 or VP9 encoding format, which simplifies the conversion steps and improves the conversion efficiency.

[0054] The glossary is explained as follows:

[0055] WebRTC, Web Real-Time Communication, is an API that supports web browsers for real-time voice or video conversations.

[0056] VP8 or VP9: Open image compression formats that deliver higher quality video with less data and require less processing power to play video, providing an ideal solution for network TV, IPTV and video conferencing companies seeking to differentiate their products and services.

[0057] YUV: A color coding method adopted by the European television system (belonging to PAL), and is the color space adopted by the PAL and SECAM analog color television systems. In modern color television systems, a three-tube color camera or a color CCD camera is usually used to capture images, and then the obtained color image signals are separated, amplified and corrected to obtain RGB, and then the brightness signal Y and two color difference signals BY (i.e. U) and RY (i.e. V) are obtained through a matrix conversion circuit. Finally, the transmitter encodes the brightness and color difference signals separately and sends them out through the same channel. This color representation method is the so-called YUV color space representation.

[0058] The importance of using the YUV color space is that its brightness signal Y and color signals U and V are separated. "Y" represents brightness (Luminance or Luma), which is the grayscale value; while "U" and "V" represent chrominance (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel. "Brightness" is established through the RGB input signal by adding specific parts of the RGB signal together. "Chroma" defines two aspects of color - hue and saturation, represented by Cr and Cb respectively. Among them, Cr reflects the difference between the red part of the RGB input signal and the brightness value of the RGB signal. And Cb reflects the difference between the blue part of the RGB input signal and the brightness value of the RGB signal.

[0059] H264: A high-performance video codec technology with a high data compression ratio.

[0060] An embodiment of the present invention provides a data transmission method for converting data formats when a web client based on the WebRTC protocol performs services with a video network terminal. The execution subject of this embodiment is a data transmission device, which is arranged on a streaming media server.

[0061] Reference Figure 1 , shows a flowchart of a data transmission method embodiment of the present invention, which can be applied to a streaming media server, wherein the streaming media server is respectively connected to a web client and a video network terminal for communication, and specifically can include the following steps:

[0062] S101, receiving video data in a first encoding format sent by the web client;

[0063] Specifically, embodiments of the present invention are applied to instant communication between the visual network and the World Wide Web, such as real-time voice conversation or real-time video conversation, such as conducting a real-time video call or performing a live broadcast.

[0064] Among them, the streaming media server is a server that processes media streams in the visual network. It is the core server of the visual network. Its main function is to collect, cache, schedule, transmit and play audio, video and other video data transmitted in the visual network.

[0065] The web client is specifically a World Wide Web client. The World Wide Web is based on the Internet transmission protocol. Users can log in to the target web page on the World Wide Web client and initiate a videophone service or a live broadcast service, and communicate with the streaming media server through the Internet protocol.

[0066] When performing videophone services or live broadcast services, the web client and the video network terminal can process video data in different encoding formats. In an embodiment of the present invention, video data in a first encoding format and video data in a second encoding format are distinguished, and the web client sends video data in the first encoding format, such as video data in VP8 or VP9 encoding format, to the streaming media server.

[0067] The visual networking terminal is a physical terminal in the visual networking, and communicates with the streaming media server through the visual networking protocol.

[0068] S102, determining a picture set according to the video data in the first coding format;

[0069] It can be understood that in practical applications, the video data in the first encoding format may also carry conditions of the video data, such as a starting position, an ending position, etc., so as to obtain only the video data between the starting position and the ending position.

[0070] For example, VP8 or VP9 is a video coding standard for the World Wide Web, and video data in a WebRTC-based web client is stored in a VP8 or VP9 coding format.

[0071] After receiving the video data in VP8 or VP9 encoding format sent by the web client, the streaming media server extracts video frames in the video data at a first preset time interval to obtain at least one type of picture, which constitutes a picture set.

[0072] Exemplarily, for video data in VP8 or VP9 encoding format sent by a web client, 30 video frames are extracted within 1 second, and these video frames constitute a picture set.

[0073] S103, determining a YUV format picture corresponding to the video data according to the picture set and a pre-established canvas;

[0074] Specifically, the streaming media server creates a blank canvas according to the video data in the first encoding format sent by the web client, puts the pictures in the picture set into the blank canvas, and performs screen capture on the blank canvas with the pictures put into it to obtain multiple YUV format pictures.

[0075] At the same time, only one picture is placed in the blank canvas, and the YUV format picture after the screen capture also only includes this picture. In addition, new pictures are constantly refreshed in the blank canvas, and the screen capture is continuously performed to obtain multiple YUV format pictures.

[0076] S104, converting the YUV format picture into a video stream in a second coding format;

[0077] Specifically, the streaming media server encodes the YUV format image according to preset encoding parameters to obtain a video stream in the second encoding format, that is, encodes the YUV format of the pixel layer to obtain video data in the second encoding format of the encoding layer, such as video data in the H264 encoding format.

[0078] The streaming media server obtains the parameter information of the YUV format image, including pixel information, coordinate information and RGB information, and sets the encapsulation format, subtitle encapsulation format, frame rate, bit rate, resolution, I frame, P frame, encoding type, pixel format, video width and height, fps (frames per second), etc. of the video to be generated in the encoder for use in the encoding layer. H264 has the concept of slice group, in which the coded slices are divided into 5 types, I slice, P slice, B slice, SP slice and SI slice. In the encoding process of the encoder, it is necessary to encode in the order of the encoding header, the encoding data and the encoding tail, and the entire encoding process is completed to generate a video stream in the H264 encoding format.

[0079] S105. Send the video stream in the second encoding format to the visual networking terminal, so that the web client and the visual networking terminal can perform business interaction.

[0080] Specifically, H264 is a video encoding standard for visual networking, and visual networking terminals play video data in the H264 encoding format.

[0081] The streaming media server transmits the video stream in the second encoding format, that is, the encoded H264 encoding format, to the video networking terminal through the video networking protocol, thereby realizing business interaction between the web client and the video networking terminal, such as videophone service or live broadcast service.

[0082] The data transmission method provided by the embodiment of the present invention receives video data in a first encoding format sent by a web client through a streaming media server; determines a picture set based on the video data in the first encoding format; determines a YUV format picture corresponding to the video data based on the picture set and a pre-established canvas; converts the YUV format picture into a video stream in a second encoding format; and sends the video stream in the second encoding format to a visual networking terminal so that the web client and the visual networking terminal can perform business interaction. The embodiment of the present invention does not require decoding of video data in VP8 or VP9 encoding formats, which simplifies the conversion steps and improves conversion efficiency.

[0083] Another embodiment of the present invention further supplements the data transmission method provided in the above embodiment.

[0084] like Figure 2 FIG. 1 is a flowchart showing another data transmission method embodiment of the present invention, wherein the data transmission method comprises:

[0085] S201, receiving video data in a first encoding format sent by the web client;

[0086] Specifically, the video data in the first encoding format is video data in the VP8 or VP9 encoding format.

[0087] A virtual terminal corresponding to the web client is established on the streaming media server, and video data in VP8 or VP9 encoding format sent by the web client is received through the virtual terminal, or video data in VP8 or VP9 encoding format is sent to the web client through the virtual terminal.

[0088] The virtual terminal corresponds to a cache or memory on the streaming media server, so that the video data can be temporarily stored. In practical applications, the video data can be stored in real time, so that the storage speed can be improved. For example, the video data between the current cycle and the previous cycle is stored in a preset cycle, so that when the amount of video data is large, the video data is processed in different time periods, which helps to improve the storage speed and the real-time performance of voice calls and video calls.

[0089] Specifically, in response to the received video data, a virtual terminal based on web page instant messaging is dynamically created on the streaming media server, and storage space is allocated for the virtual terminal.

[0090] Specifically, an API (Application Programming Interface) may be called to establish a virtual terminal. In an embodiment of the present invention, a WebRTC (WEB Real-Time Communication) interface is used to establish a virtual terminal.

[0091] The storage space can be a preset size and allocated according to the storage configuration size, remaining size, and experience value of the streaming media server. When the storage configuration size is larger, the remaining storage is larger, the experience value is larger, and the allocated storage space is larger; when the storage configuration size is smaller, the remaining storage is smaller, the experience value is smaller, and the allocated storage space is smaller.

[0092] The embodiment of the present invention can dynamically and real-time create a virtual terminal, which can ensure real-time communication while minimizing unnecessary occupation of streaming media server space, thereby helping to improve the performance of the streaming media server.

[0093] The embodiment of the present invention can not only ensure the one-to-one correspondence between the virtual terminal and the web client, thereby improving the data acquisition speed of the web client, but also write the generated video data to it when creating the virtual terminal, saving the time consumption introduced by the format conversion of the video data.

[0094] Specifically, the data can be released and deleted periodically or after each call ends, which helps to improve the performance of the streaming media server.

[0095] S202: intercepting the video data in the first encoding format at intervals of a first preset time period through a WebRTC decoding interface to obtain a picture set, wherein the picture format in the picture set includes at least one of BMP (Bitmap, image file format), JPG, JPEG (Joint Photo graphic Experts Group) or PNG (Portable Network Graphic Format, image file storage format).

[0096] Specifically, since there may be multiple video frames with the same video content in the video data in the first encoding format, the streaming media server extracts the video data in the first encoding format according to the first preset time period through the WebRTC decoding interface to obtain a picture set, which not only saves time but also reduces storage space.

[0097] S203, inserting the pictures in the picture set into a pre-established canvas to generate a canvas picture set;

[0098] Specifically, the streaming media server creates a blank canvas according to the video data in the VP8 or VP9 encoding format sent by the web client, puts the pictures in the picture set into the blank canvas, and performs screen capture on the blank canvas with the pictures put into it to obtain multiple YUV format pictures.

[0099] S204. Capture the canvas pictures in the canvas picture set every second preset time period to obtain YUV format pictures.

[0100] Specifically, since a plurality of consecutive YUV format pictures may have the same picture content, the streaming media server captures the canvas pictures in the canvas picture set every second preset time period to generate YUV format pictures.

[0101] Exemplarily, the streaming media server captures the canvas images in the canvas image set every 10 ms.

[0102] The first preset time period and the second preset time period can be set as needed and are not specifically limited in the embodiment of the present invention.

[0103] S205. Encode the YUV format image according to preset encoding parameters to obtain a video stream in a second encoding format, wherein the second encoding format is an H264 encoding format, and the preset encoding parameters include at least one of a frame rate, a bit rate, a resolution, an I frame, or a P frame.

[0104] Since step S205 is Figure 1 Step S104 in the illustrated embodiment is the same. Figure 1 Step S104 has been described in detail in , so step S205 will not be described again here.

[0105] S206, encapsulating the video stream in the second coding format into a video stream of a video networking protocol;

[0106] S207, sending the video stream of the visual networking protocol to the visual networking terminal, so that the web client and the visual networking terminal can perform business interaction.

[0107] Specifically, the streaming media server encapsulates the video stream in H264 encoding format into a video stream of the visual networking protocol, and sends the video stream of the visual networking protocol to the visual networking terminal, so that the web client and the visual networking terminal can interact with each other, such as performing videophone services or live broadcast services.

[0108] Figure 3 is a flowchart of another embodiment of a data transmission method of the present invention, such as Figure 3 As shown, in the embodiment of the present invention, a virtual WebRTC end is established on the streaming media service based on the WebRTC protocol, and a Chromium browser is installed on the web client, so that the web client and the streaming media server perform end-to-end WebRTC video calls or live broadcast services, and the streaming media server acts as a WebRTC signaling server. When the web client sends a request to call a video network terminal through the signaling server, the signaling server receives the request sent by the web client, and dynamically creates a virtual WebRTC end to interact with the real web end (web client). At the same time, the signaling server performs video network services with the video network terminal through the virtual terminal;

[0109] After obtaining the VP8 video data from the web client, it is intercepted, converted into a picture set, and saved locally in the virtual terminal. According to the picture set and the pre-established canvas, the YUV format picture corresponding to the video data is determined; the YUV format picture is converted into a video stream in a second encoding format, namely, a video stream in H264 encoding format; the video stream in H264 encoding format is sent to the visual networking terminal, so that the web client and the visual networking terminal can interact with each other. In this case, the video data in VP8 or VP9 encoding format is converted into H264 by establishing a screen capture mode strategy. The video data in VP8 or VP9 encoding format is not internally decoded in detail, but the presented picture (image) is secondary-encoded and then interacts with the visual networking terminal.

[0110] The video data in VP8 or VP9 encoding format obtained from the web page is converted into bitmap buffer data through the decoding interface provided by WebRTC, and presented on a hidden canvas. The canvas screen is then captured, and the YUV data is converted and encoded into H264 buffer. The H264 video stream is encapsulated into a data packet of the v2v visual networking protocol and sent to the visual networking terminal.

[0111] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0112] The data transmission method provided by the embodiment of the present invention receives video data in a first encoding format sent by a web client through a streaming media server; determines a picture set based on the video data in the first encoding format; determines a YUV format picture corresponding to the video data based on the picture set and a pre-established canvas; converts the YUV format picture into a video stream in a second encoding format; and sends the video stream in the second encoding format to a visual networking terminal so that the web client and the visual networking terminal can perform business interaction. The embodiment of the present invention does not require decoding of video data in VP8 or VP9 encoding formats, which simplifies the conversion steps and improves conversion efficiency.

[0113] Another embodiment of the present invention provides a data transmission device, which is used to execute the data transmission method provided in the above embodiment.

[0114] Reference Figure 4 , shows a structural block diagram of an embodiment of a data transmission device of the present invention, the device can be applied to a streaming media server, the streaming media server is respectively connected to a web client and a visual network terminal, the device can specifically include the following modules: a receiving module 401, an intercepting module 402, a screen grabbing module 403, a conversion module 404 and a sending module 405, wherein:

[0115] The receiving module 401 is used to receive the video data in the first encoding format sent by the web client;

[0116] The interception module 402 is used to determine a picture set according to the video data in the first coding format;

[0117] The screen capture module 403 is used to determine a YUV format picture corresponding to the video data according to the picture set and the pre-established canvas;

[0118] The conversion module 404 is used to convert the YUV format picture into a video stream in a second coding format;

[0119] The sending module 405 is used to send the video stream in the second encoding format to the visual networking terminal so that the web client and the visual networking terminal can perform business interaction.

[0120] The data transmission device provided by the embodiment of the present invention receives video data in a first encoding format sent by a web client through a streaming media server; determines a picture set based on the video data in the first encoding format; determines a YUV format picture corresponding to the video data based on the picture set and a pre-established canvas; converts the YUV format picture into a video stream in a second encoding format; and sends the video stream in the second encoding format to a visual networking terminal so that the web client and the visual networking terminal can perform business interaction. The embodiment of the present invention does not require decoding of video data in VP8 or VP9 encoding formats, which simplifies the conversion steps and improves conversion efficiency.

[0121] Another embodiment of the present invention further supplements the data transmission device provided in the above embodiment.

[0122] Optionally, the interception module 402 is used to:

[0123] Through the WebRTC decoding interface, the video data in the first encoding format is intercepted every first preset time period to obtain a picture set, wherein the picture format in the picture set includes at least one of BMP, JPG, JPEG or PNG

[0124] Optionally, the screen capture module 403 is used to:

[0125] Inserting the pictures in the picture set into a pre-established canvas to generate a canvas picture set;

[0126] Capture the canvas image in the canvas image set to obtain a YUV format image.

[0127] Optionally, the screen capture module 403 is specifically used for:

[0128] Every second preset time period, a canvas picture in the canvas picture set is captured to obtain the YUV format picture.

[0129] Optionally, the conversion module 404 is used to:

[0130] The YUV format image is encoded according to preset encoding parameters to obtain a video stream in the second encoding format, wherein the preset encoding parameters include at least one of frame rate, bit rate, resolution, I frame or P frame.

[0131] Optionally, the sending module 405 is used to:

[0132] Encapsulating the video stream in the second coding format into a video stream of a visual networking protocol;

[0133] The video stream of the visual networking protocol is sent to the visual networking terminal so that the web client and the visual networking terminal can perform business interaction. It should be noted that each practicable method in this embodiment can be implemented separately or in any combination without conflict. This application does not limit it.

[0134] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0135] The data transmission device provided by the embodiment of the present invention receives video data in a first encoding format sent by a web client through a streaming media server; determines a picture set based on the video data in the first encoding format; determines a YUV format picture corresponding to the video data based on the picture set and a pre-established canvas; converts the YUV format picture into a video stream in a second encoding format; and sends the video stream in the second encoding format to a visual networking terminal so that the web client and the visual networking terminal can perform business interaction. The embodiment of the present invention does not require decoding of video data in VP8 or VP9 encoding formats, which simplifies the conversion steps and improves conversion efficiency.

[0136] Still another embodiment of the present invention provides a terminal device for executing the data transmission method provided in the above embodiment.

[0137] Figure 5 is a schematic diagram of the structure of a terminal device of the present invention, such as Figure 5 As shown, the terminal device includes: at least one processor 501 and a memory 502;

[0138] The memory stores a computer program; and the at least one processor executes the computer program stored in the memory to implement the data transmission method provided in the above embodiment.

[0139] The terminal device provided in this embodiment receives video data in a first encoding format sent by a web client through a streaming media server; determines a picture set based on the video data in the first encoding format; determines a YUV format picture corresponding to the video data based on the picture set and a pre-established canvas; converts the YUV format picture into a video stream in a second encoding format; and sends the video stream in the second encoding format to a visual networking terminal so that the web client and the visual networking terminal can perform business interaction. The embodiment of the present invention does not require decoding of video data in VP8 or VP9 encoding formats, which simplifies the conversion steps and improves conversion efficiency.

[0140] Yet another embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed, the data transmission method provided in any of the above embodiments is implemented.

[0141] According to the computer-readable storage medium of the present embodiment, video data in a first encoding format sent by a web client is received through a streaming media server; a picture set is determined based on the video data in the first encoding format; a YUV format picture corresponding to the video data is determined based on the picture set and a pre-established canvas; the YUV format picture is converted into a video stream in a second encoding format; the video stream in the second encoding format is sent to a visual networking terminal, so that the web client and the visual networking terminal can perform business interaction. The embodiment of the present invention does not require decoding of video data in VP8 or VP9 encoding formats, which simplifies the conversion steps and improves conversion efficiency.

[0142] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0143] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0144] The embodiments of the present invention are described with reference to flowcharts and / or block diagrams of methods, electronic devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing electronic device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing electronic device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0145] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing electronic device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0146] These computer program instructions can also be loaded onto a computer or other programmable data processing electronic device so that a series of operating steps are executed on the computer or other programmable electronic device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable electronic device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0147] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0148] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or electronic device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or electronic device. In the absence of further restrictions, the elements defined by the sentence "comprises one..." do not exclude the presence of other identical elements in the process, method, article or electronic device including the elements.

[0149] The data transmission method and the data transmission device provided by the present invention are introduced in detail above. The principle and implementation mode of the present invention are explained by using specific examples in this article. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and the scope of application. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A data transmission method, characterized in that: The method is applied to a streaming media server, and the streaming media server is respectively connected to a web client and a visual network terminal for communication, and the method comprises: Receiving video data in a first encoding format sent by the web client; Determining a picture set according to the video data in the first coding format; Determine, according to the picture set and the pre-established canvas, a YUV format picture corresponding to the video data; Convert the YUV format image into a video stream in a second encoding format; Sending the video stream in the second encoding format to the visual networking terminal so that the web client and the visual networking terminal can perform business interaction; The first encoding format is VP8 or VP9 encoding format, and the second encoding format is H264 encoding format; Determining a YUV format picture corresponding to the video data according to the picture set and a pre-established canvas includes: Inserting the pictures in the picture set into a pre-established canvas to generate a canvas picture set; wherein, at the same time, one picture in the picture set is inserted into the canvas, and the pictures inserted into the canvas are continuously refreshed to new pictures; Capture the canvas image in the canvas image set to obtain a YUV format image.

2. The method according to claim 1, characterized in that: The step of determining a picture set according to the video data in the first coding format includes: Through the WebRTC decoding interface, the video data in the first encoding format is intercepted every first preset time period to obtain a picture set, wherein the picture format in the picture set includes at least one of BMP, JPG, JPEG or PNG.

3. The method according to claim 1, characterized in that The capturing of the canvas picture in the canvas picture set to obtain a YUV format picture includes: Every second preset time period, a canvas picture in the canvas picture set is captured to obtain the YUV format picture.

4. The method according to claim 1, characterized in that: The converting of the YUV format picture into a video stream in a second coding format includes: The YUV format image is encoded according to preset encoding parameters to obtain a video stream in the second encoding format, wherein the preset encoding parameters include at least one of frame rate, bit rate, resolution, I frame or P frame.

5. The method according to claim 1, characterized in that The step of sending the video stream in the second encoding format to the visual networking terminal so that the web client and the visual networking terminal perform service interaction includes: Encapsulating the video stream in the second coding format into a video stream of a visual networking protocol; The video stream of the visual networking protocol is sent to the visual networking terminal so that the web client and the visual networking terminal can perform business interaction.

6. A data transmission device, characterized in that: The device is applied to a streaming media server, and the streaming media server is respectively connected to a web client and a visual network terminal for communication, and the device comprises: A receiving module, used for receiving video data in a first encoding format sent by the web client; An interception module, configured to determine a picture set according to the video data in the first coding format; A screen capture module, used to determine a YUV format picture corresponding to the video data according to the picture set and a pre-established canvas; A conversion module, used for converting the YUV format picture into a video stream in a second encoding format; A sending module, used for sending the video stream in the second coding format to the visual networking terminal, so that the web client and the visual networking terminal can perform business interaction; The first encoding format is VP8 or VP9 encoding format, and the second encoding format is H264 encoding format; The screen capture module is used for: Inserting the pictures in the picture set into a pre-established canvas to generate a canvas picture set; wherein, at the same time, one picture in the picture set is inserted into the canvas, and the pictures inserted into the canvas are continuously refreshed to new pictures; Capture the canvas image in the canvas image set to obtain a YUV format image.

7. A terminal device, characterized in that: include: at least one processor and memory; The memory stores a computer program; The at least one processor executes the computer program stored in the memory to implement the data transmission method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the data transmission method according to any one of claims 1 to 5 is implemented.

Images