Video data transmission method, device, storage medium and system

Abstract

The present application provides a video data transmission method, device, storage medium and system, which includes: in response to establishing a communication connection with a server, the client sends an acquisition request for obtaining a key frame to the server, so that the server encodes the current video picture to be transmitted into a first key frame. The client receives the first key frame fed back by the server, decodes the first key frame according to the first decoding information included in the first key frame, and displays the decoded video picture. After connecting to the server, the client actively requests an I frame from the server, and completes the decoding and display of the I frame based on the decoding information in the fed-back I frame, which can reduce the display delay of the first screen.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a video data transmission method, device, storage medium and system. Background Art

[0002] Taking live video streaming as an example, the server-side encoder encodes the multiple frames contained in the live video based on a set Group of Picture (GOP) length, generating GOP segments one by one. The client-side decoder decodes each received frame to render and display the decoded video. A GOP contains a series of consecutive images, consisting of an I-frame followed by multiple B-frames and P-frames. I-frames are intra-coded frames (also known as keyframes), P-frames are forward-predicted frames (also known as forward reference frames), and B-frames are bidirectionally interpolated frames (also known as bidirectional reference frames). In practice, the length of a GOP is the distance between two I-frames. For example, if the GOP length is 2 seconds, an I-frame is encoded every 2 seconds. Simply put, an I-frame represents a complete video frame, while P-frames and B-frames record changes relative to the I-frame. Therefore, without an I-frame, P-frames and B-frames cannot be decoded. In other words, the decoder starts decoding from the I-frame, as the decoding of B-frames and P-frames depends on the decoding results of the I-frame.

[0003] Applications involving video data transmission, such as long and short video applications, live streaming, cloud desktops, and other cloud applications, prioritize optimizing first-screen display latency, striving to shorten the initial display delay, allowing users to see the video faster and achieve a better user experience. First-screen display latency, specifically the time it takes to load the first screen, is a visual experience metric used to measure the time it takes for various video services, from opening an application or media file to the video appearing on the screen. If this delay exceeds seconds, users experience a lack of fast service access.

[0004] However, due to the existence of GOP, the decoder in the client needs to wait until a key frame is received before decoding. For example, when a user's client enters a live broadcast room, the image frame received from the server at this time is not a key frame. In this case, the decoder can only wait, and a black screen will appear. The waiting time may be close to the length of a GOP. For example, if a GOP contains 50 frames, the first frame is an I-frame. When the client accesses the live broadcast room, the video screen obtained is the 5th frame. Then, the subsequent frames obtained until the 50th frame cannot be decoded immediately. It must wait until the next I-frame is obtained before it can be decoded to display the first screen image. Summary of the Invention

[0005] Embodiments of the present invention provide a video data transmission method, device, storage medium, and system, which can shorten the display delay of the first screen image.

[0006] In a first aspect, an embodiment of the present invention provides a method for transmitting video data, applied to a client, the method comprising:

[0007] In response to establishing a communication connection with the server, sending an acquisition request for acquiring a key frame to the server, so that the server encodes the current video picture to be transmitted into a first key frame;

[0008] receiving the first key frame fed back by the server, where the first key frame includes first decoding information;

[0009] The first key frame is decoded according to the first decoding information to display the decoded video picture.

[0010] In a second aspect, an embodiment of the present invention provides a video data transmission device, applied to a client, the device comprising:

[0011] a sending module, configured to, in response to establishing a communication connection with a server, send a request for obtaining a key frame to the server, so that the server encodes the current video picture to be transmitted into a first key frame;

[0012] A receiving module, configured to receive the first key frame fed back by the server, where the first key frame includes first decoding information;

[0013] A decoding module is used to decode the first key frame according to the first decoding information to display the decoded video picture.

[0014] In a third aspect, an embodiment of the present invention provides an electronic device comprising: a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the video data transmission method as described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the video data transmission method described in the first aspect.

[0016] In a fifth aspect, an embodiment of the present invention provides a video data transmission method, applied to a server, the method comprising:

[0017] Receiving an acquisition request for acquiring a key frame sent by the client after establishing a communication connection with the server;

[0018] Encoding the current video picture to be transmitted into a key frame, wherein the key frame includes decoding information;

[0019] The key frame is sent to the client, so that the client displays the decoded video picture after decoding the key frame according to the decoding information.

[0020] In a sixth aspect, an embodiment of the present invention provides a video data transmission device, applied to a server, the device comprising:

[0021] A receiving module, configured to receive an acquisition request for acquiring a key frame sent by the client after establishing a communication connection with the server;

[0022] An encoding module, configured to encode a video image to be transmitted into a key frame, wherein the key frame includes decoding information;

[0023] The sending module is used to send the key frame to the client, so that the client displays the decoded video picture after decoding the key frame according to the decoding information.

[0024] In the seventh aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a communication interface; wherein, the memory stores executable code, and when the executable code is executed by the processor, the processor executes the video data transmission method as described in the fifth aspect.

[0025] In the eighth aspect, an embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the video data transmission method described in the fifth aspect.

[0026] In a ninth aspect, an embodiment of the present invention provides a video data transmission method, applied to a first client, the method comprising:

[0027] In response to a screen sharing operation triggered by a user on the first client, sending a request for obtaining a key frame to a first cloud desktop to which the first client is connected, so that the first cloud desktop encodes a video picture currently to be transmitted into a first key frame, where the video picture is a picture currently presented by the first cloud desktop;

[0028] Receive the first key frame fed back by the first cloud desktop, where the first key frame includes first decoding information;

[0029] The first key frame is sent to a second client, so that the second client decodes the first key frame according to the first decoding information and displays the decoded video picture, and the second client is connected to a second cloud desktop.

[0030] In a tenth aspect, an embodiment of the present invention provides a video data transmission device, applied to a first client, the device comprising:

[0031] a sending module, configured to, in response to a screen sharing operation triggered by a user on the first client, send a request for obtaining a key frame to a first cloud desktop to which the first client is connected, so that the first cloud desktop encodes a video picture currently to be transmitted into a first key frame, where the video picture is a picture currently presented by the first cloud desktop;

[0032] A receiving module, configured to receive the first key frame fed back by the first cloud desktop, where the first key frame includes first decoding information;

[0033] A decoding module is used to send the first key frame to a second client, so that the second client decodes the first key frame according to the first decoding information and displays the decoded video picture, and the second client is connected to a second cloud desktop.

[0034] In the eleventh aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a communication interface; wherein, the memory stores executable code, and when the executable code is executed by the processor, the processor executes the video data transmission method as described in the ninth aspect.

[0035] In the twelfth aspect, an embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the video data transmission method described in the ninth aspect.

[0036] In a thirteenth aspect, an embodiment of the present invention provides a video data transmission system, including:

[0037] A first cloud desktop, a second cloud desktop, a first client connected to the first cloud desktop, and a second client connected to the second cloud desktop;

[0038] The first client is configured to, in response to a screen sharing operation triggered by a user on the first client, send a request for obtaining a key frame to the first cloud desktop, receive a first key frame fed back by the first cloud desktop, the first key frame including first decoding information, and send the first key frame to the second client;

[0039] The first cloud desktop is configured to, in response to the acquisition request, encode a first video picture to be transmitted into the first key frame and feed the first key frame back to the first client, where the first video picture is a picture currently presented by the first cloud desktop;

[0040] The second client is configured to decode the first key frame according to the first decoding information, and display a second video image and the decoded first video image, wherein the second video image is a image currently presented by the second cloud desktop;

[0041] The second cloud desktop is used to transmit the video image corresponding to the second cloud desktop to the second client.

[0042] In an embodiment of the present invention, a server is used to transmit video stream data to a client, wherein the video stream data is composed of frames of video images, and in order to reduce the occupancy of network bandwidth, the server sends encoded frames to the client, that is, after the client establishes a communication connection with the server, the encoded video stream is pulled from the server for decoding and display. In fact, the server encodes the video stream according to the default GOP length. In an embodiment of the present invention, after the client establishes a communication connection with the server, the client can first send an acquisition request for obtaining an I frame to the server. After receiving the acquisition request, the server starts encoding a new GOP. Specifically, the current frame of video image to be transmitted is encoded into an I frame as the first frame of the new GOP. The I frame includes complete decoding information. Therefore, the client can decode the I frame based on the decoding information contained in the I frame fed back by the server, so that the decoded video image can be displayed immediately, reducing the display delay of the first screen image. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A flowchart of a video data transmission method provided by an embodiment of the present invention;

[0045] Figure 2 A flowchart of a video data transmission method provided by an embodiment of the present invention;

[0046] Figure 3 A flowchart of a video data transmission method provided by an embodiment of the present invention;

[0047] Figure 4 A schematic diagram of a cloud desktop-based video data transmission system provided by an embodiment of the present invention;

[0048] Figure 5 A flowchart of a video data transmission method provided by an embodiment of the present invention;

[0049] Figure 6 A schematic diagram of an application of a video data transmission method provided by an embodiment of the present invention;

[0050] Figure 7 A schematic diagram of the execution flow of a teacher client in an e-learning scenario provided by an embodiment of the present invention;

[0051] Figure 8 A schematic diagram of the execution flow of a student client in an e-learning scenario provided by an embodiment of the present invention;

[0052] Figure 9 A schematic structural diagram of a video data transmission device provided by an embodiment of the present invention;

[0053] Figure 10 A schematic structural diagram of an electronic device provided in this embodiment;

[0054] Figure 11 A schematic structural diagram of a video data transmission device provided by an embodiment of the present invention;

[0055] Figure 12 This is a schematic structural diagram of an electronic device provided in this embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In addition, the step timing in the following method embodiments is only an example and not a strict limitation.

[0057] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0058] First, some concepts involved in the embodiments of the present invention are explained.

[0059] A Group of Picture (GOP), generated by video encoding, is a group of pictures that contains a key frame (I frame). The I frame in a GOP is the first coded frame in the GOP, so the GOP length is the distance between two I frames: it can be expressed by the time length between two I frames.

[0060] The Sequence Parameter Set (SPS) contains decoding-related information, such as profile level, resolution, coding tool switch flags and related parameters in a certain profile, and time-domain scalability information.

[0061] The Picture Parameter Set (PPS) describes the common parameters used by the image, such as initial image control information, initialization parameters, and block information.

[0062] Instantaneous Decoding Refresh (IDR) frames contain the decoding information required for complete frame decoding, such as SPS and PPS. The decoder can decode and restore the original image without referring to other frames. The function of IDR frames is to refresh immediately so that errors do not propagate. When the decoder receives an IDR frame, it clears the reference frame list. In other words, for frames after an IDR frame, the decoder will not refer to any frames before the IDR frame for decoding. It should be noted that IDR frames belong to I frames, but I frames include but are not limited to IDR frames. In other words, I frames are divided into ordinary I frames and special I frames - IDR frames. Both contain complete decoding information and can be decoded without relying on other frames. Compared with IDR frames, P frames and B frames after ordinary I frames can skip the ordinary I frame and refer to the previous frames for decoding.

[0063] Multicast: A one-to-many communication mode between hosts, allowing one or more multicast sources to send the same message to multiple receivers in the same multicast group. Multicast groups are identified by multicast addresses.

[0064] Real-Time Stream Protocol (RTSP) is a text-based application layer protocol. Its message types are divided into request messages and response messages. A URL link starting with "rtsp" or "rtspu" is used to specify that the RTSP protocol is currently being used. It can be analyzed by some media tools and then request the corresponding media data from the media server. It is commonly used in streaming media applications such as live video and multicast.

[0065] Taking real-time video live streaming as an example, due to the existence of GOP, the video decoder on the client side, i.e. the playback side, needs to wait until the I-frame is received before decoding. If the first frame pulled from the server is not a key frame, the decoder can only wait, and a black screen will appear. It will not be able to decode until the next I-frame is received to display the first screen image.

[0066] To enable real-time video playback immediately after entering the live studio, a common optimization method used on the server side is to cache GOPs at the edge nodes of the Content Delivery Network (CDN), often caching the previous GOP. This edge node is typically one that is close to the client, or even directly connected to the client. The disadvantage of this method is that there is a playback delay, as the client always decodes and displays the previous I-frame, resulting in a display delay of at least one GOP.

[0067] To reduce playback latency, in some applications with high real-time requirements, the server can reduce the GOP length, setting a shorter GOP. However, this will result in an increase in the number of I-frames within the same timeframe. For example, a 2-second GOP originally contained one I-frame, but if the GOP length is set to 500 milliseconds, there would be four I-frames within the same 2-second period. In reality, compared to P-frames and B-frames, I-frames use intra-frame encoding, which has a lower compression rate. Therefore, more I-frames consume more network bandwidth, placing higher demands on network bandwidth.

[0068] In view of this, in the video data transmission solution provided in the embodiment of the present invention, when a communication connection is established between the client and the server and the client pulls the video stream from the server, first, the client actively sends a set signaling request to the server: a request for obtaining a key frame, so that the server encodes the video picture to be transmitted after receiving the acquisition request into a key frame and feeds it back to the client, so that the client can complete the decoding and display of the key frame in a timely manner based on the complete decoding information contained in the key frame, such as SPS, PPS and other information, so that the client can quickly display the first screen. Afterwards, the server continues to encode the subsequent video pictures into P frames and B frames and sends them to the client. The client decodes and displays the subsequently received P frames and B frames based on the decoding results of the above key frames.

[0069] In this way, the client actively requests key frames from the cloud server to overcome the problem of excessive delay in the first screen display caused by the inability to obtain sufficient decoding information in time when pulling the video stream on the client side, and it will not significantly increase the occupancy of the network bandwidth, forming an end-cloud collaborative optimization solution.

[0070] Figure 1 A flowchart of a video data transmission method provided by an embodiment of the present invention, which can be executed by a client, such as Figure 1 As shown, the method includes the following steps:

[0071] 101. In response to establishing a communication connection with a server, sending an acquisition request for acquiring an I frame to the server, so that the server encodes a video picture to be currently transmitted into a first I frame.

[0072] 102. Receive a first I frame fed back by the server, where the first I frame includes first decoding information.

[0073] 103. Decode the first I frame according to the first decoding information to display the decoded video picture.

[0074] Taking the live broadcast application scenario as an example, the client can be a live broadcast application installed on the user's terminal device. When the user launches the client and clicks to enter a live broadcast room to watch, the client establishes a communication connection with the server, including but not limited to a communication connection established based on the RTSP protocol. At this time, the client sends a request to the server to obtain the ID frame.

[0075] The server can be a server or server cluster corresponding to a live broadcast application in the cloud.

[0076] During the transmission of the video stream data, the server assumes that the GOP length included in the default encoding parameters is the second GOP length. For example, if the second GOP length is 2 seconds, the server will encode and generate an I frame every 2 seconds.

[0077] For ease of illustration, let's assume that, according to the second GOP length, the server encodes the 10 consecutive video frames F1, F2, F3, ..., F10 in the video stream data as follows: IPBBPBBPBB. These 10 encoded frames constitute a GOP (denoted as GOP1). The server can transmit each encoded frame to the connected client after generating it. In this hypothetical scenario, suppose a client establishes a communication connection with the server and the first encoded frame it receives from the server is a B-frame corresponding to video frame F3. Since decoding of the B-frame requires reference to the preceding P-frame and I-frame, and the client has not yet received the preceding P-frame and I-frame, the client cannot immediately decode the received B-frame. Similarly, subsequent B-frames and P-frames within GOP1 cannot be decoded. Only after receiving the I-frame contained in the next GOP can the client complete decoding of the I-frame based on the decoding information contained in the I-frame, such as the SPS and PPS, and thus display the decoded video.

[0078] The above describes the first screen delay that may occur on the client side when the server encodes and transmits the video based on the second GOP length configured by default after the client and server establish a communication connection.

[0079] In the solution provided by the embodiment of the present invention, after the client establishes a communication connection with the server, it immediately and actively sends the above-mentioned acquisition request for obtaining the I frame to the server. Assuming that the video picture currently to be transmitted by the server is the video picture F3 in the above example, based on the acquisition request, the server encodes the video picture F3 into an I frame (that is, the first I frame in the above step) and feeds it back to the client. Thereby, the client can complete the decoding of the first I frame based on the first decoding information included in the first I frame, such as SPS and PPS, and display the decoded video picture F3.

[0080] In fact, when the server receives the above acquisition request, it will start encoding a new GOP. Assume that the newly started GOP is GOP2. Then, for the 10 video frames F1-F10 in the above example, the encoding result of the server will become:

[0081] GOP1:IP

[0082] GOP2: IPBBPBBPBB, where the last two coded frames B are the coding results corresponding to the two video pictures following the above 10 video pictures.

[0083] This assumes that the length of the new GOP initiated by the server upon receiving the acquisition request is the same as the default second GOP length. In practice, the acquisition request sent by the client to the server may optionally include a first GOP length, where the first GOP length is smaller than the second GOP length. Thus, the length of the new GOP initiated by the server is the first GOP length.

[0084] To optimize the first screen display delay, this indicator can be optimized by allowing the client to obtain sufficient decoding information more quickly. Therefore, the length of the new GOP can be set to the first GOP length, which is smaller than the default second GOP length. Assuming that the new GOP contains 5 coded frames based on the first GOP length, the encoding results of the 10 video frames F1-F10 in the above example will become:

[0085] GOP1:IP

[0086] GOP2: IPBBP

[0087] GOP3: IPB…

[0088] The length of GOP3 will become the default second GOP length. That is, after encoding a new GOP based on a received request, the server will revert to the default encoding method—the second GOP length. This way, to optimize the first-screen display delay, the server only briefly generates a new I-frame, which doesn't significantly increase network bandwidth usage, but does reduce the first-screen display delay.

[0089] In addition, in an embodiment of the present invention, in response to the acquisition request for acquiring the I frame sent by the client, the I frame fed back to the client by the server may be a common I frame or an I DR frame.

[0090] In most video applications, generally, a cache queue is set up on the client side, and corresponding cache parameters are configured to cache a certain length of video stream data after establishing a communication connection with the server, in order to analyze the information required for decoding. By default, this process is relatively time-consuming, which will affect the delay in displaying the first screen. In order to ensure that sufficient decoding information can be parsed from the cached data, the above cache parameters are usually set to be large. Optionally, the cache parameter can be measured by the amount of cached data, such as setting the number of cached frames. The cache parameter can also be a cache duration, such as 1 second. The cache duration refers to the set duration after the start of receiving the encoded frame.

[0091] One approach to optimizing the client-side cache parameters mentioned above is to set them relatively low, thereby reducing the initial screen load time. However, if the cache parameters are set too low, the cached encoded frames will contain insufficient decoding information, making decoding impossible and resulting in playback failure.

[0092] The solution provided by the embodiment of the present invention can well balance the playback success rate and the first-screen display delay. Specifically, after establishing a communication connection with the server, the client actively sends an acquisition request to the server for obtaining the I frame, so that the server immediately generates a new I frame (the above-mentioned first I frame) and feeds it back to the client. The client stores the first I frame in the cache queue, so that the first I frame including sufficient decoding information is quickly stored in the cache queue after the client establishes a communication connection with the server. In this way, the cache parameter corresponding to the cache queue can be set to a smaller value, such as 200 milliseconds or caching 5 frames. In this way, the client's decoder only needs to wait for a shorter cache parameter value before starting decoding, shortening the display delay of the first screen.

[0093] Based on this, after receiving the above-mentioned first I frame feedback from the server, the client stores the received first I frame in the local cache queue. If the number of frames stored in the cache queue reaches the set number, or the cache duration of the cache queue reaches the set duration, the multiple encoded frames stored in the cache queue are read, and the above-mentioned first I frame contained therein is identified from these multiple encoded frames. The first decoding information, such as SPS and PPS, is parsed from the first I frame to complete the decoding and display of the first I frame according to the first decoding information.

[0094] The multiple encoded frames are the encoding results of multiple video frames, and are the encoded frames received sequentially from the server after establishing a communication connection with the server. For example, assuming the cache parameter is set to cache 5 frames, after the server feeds the first I frame back to the client, the P frames and B frames obtained by encoding the subsequent four video frames will also be fed back to the client in real time. The client will store the five received encoded frames in a cache queue. When the decoder in the client detects that the number of encoded frames stored in the cache queue has reached the set number of 5, it begins parsing the five encoded frames stored in the cache queue: determining whether an I frame is included, parsing decoding information from the included I frame, decoding and displaying the I frame based on the decoded decoding information, and decoding and displaying subsequent B frames and P frames based on the decoding result of the I frame. It should be noted that in the above example, starting from the fifth encoded frame, subsequent encoded frames received by the client from the server do not need to be placed in the cache queue and can be directly sent to the decoder for decoding.

[0095] Figure 2 A flowchart of a video data transmission method provided by an embodiment of the present invention, which can be executed by a client, such as Figure 2 As shown, the method includes the following steps:

[0096] 201. In response to establishing a communication connection with a server, sending an acquisition request for acquiring an I frame to the server, so that the server encodes a video picture to be currently transmitted into a first I frame.

[0097] 202. Receive a first I frame fed back by the server, where the first I frame includes first decoding information.

[0098] 203. Decode the first I frame according to the first decoding information to display the decoded video picture.

[0099] 204. Starting from the first moment, determine whether to send the acquisition request to the server again every set time period, and the first moment is the first sending moment of the acquisition request.

[0100] 205. Send the acquisition request to the server at a second time, wherein it is determined that the acquisition request is sent to the server again at the second time, and the second time is separated from the first time by at least the set time length.

[0101] 206. Receive a second I frame including second decoding information fed back by the server, and decode the second I frame according to the second decoding information to display a decoded video picture.

[0102] When the client establishes a communication connection with the server and completes the decoding of the first I frame based on the first decoding information contained in the first I frame requested from the server, and displays the corresponding first screen, the server will continuously send subsequent encoded frames to the client in sequence, and the client will perform corresponding decoding and display, thereby presenting the display effect of the video stream.

[0103] When the server transmits video data to the client, although the default GOP length is the second GOP length described above, the number of coded frames contained in different GOPs can vary. Generally speaking, if the scene corresponding to the video image is static, the frame interval when the server encodes the video image will be larger, resulting in fewer coded frames in a GOP. However, if the scene is dynamic, the frame interval will be smaller, resulting in more coded frames in a GOP.

[0104] In other words, the server can determine the frame interval based on the degree of change between adjacent video images. A small degree of change indicates no perceptible changes within a certain period of time. In this case, a larger frame interval is used, which can reduce the number of encoded frames and thus reduce network bandwidth usage. Conversely, if the degree of change is large, a smaller frame interval is used to ensure that users can accurately perceive changes in the video image and maintain their video viewing experience, thereby fully preserving the dynamic changes in the video image across more encoded frames.

[0105] Based on this, it can be understood that, assuming that the video stream currently transmitted by the server to the client corresponds to a static or low-frequency refresh scenario, in this case, the time interval between the server sending the encoded frames is relatively large. If the client fails to successfully receive the I frame in a GOP due to abnormalities such as network jitter, then because the frame interval is large, the P frame and B frame received after the I frame will not be decoded and displayed, which will cause obvious screen freezes on the client side. For dynamic change scenarios with a high degree of dynamic content change, the probability of the client failing to successfully receive the I frame will be lower, and the frame interval will be smaller. Even if a P frame or B frame is missed in the middle, the user will not obviously perceive the abnormality of the picture.

[0106] In this embodiment, in addition to sending the acquisition request for the first I frame after connecting to the server, the client can also determine whether it needs to send the acquisition request for the I frame to the server again at set intervals in the subsequent process.

[0107] Assume that the moment when the client sends the acquisition request to the server for the first time after establishing a communication connection with the server is recorded as the first moment, and the duration is set to 200 milliseconds. Then, starting from the first moment, a judgment is made every 200 milliseconds. Assume that at the second moment (for example, 600 milliseconds after the first moment), the judgment result is positive: the acquisition request needs to be sent to the server again. Then, at the second moment, the client sends the acquisition request to the server, and the server encodes the video picture that needs to be sent to the client at the second moment into an I frame, called the second I frame, and the decoding information contained therein is called the second decoding information. The server feeds the second I frame back to the client, and the client parses the second decoding information to decode the second I frame to display the decoded video picture.

[0108] In short, the purpose of the client sending the above-mentioned acquisition request to the server at a set interval is to determine whether the current video scene is dynamic or static, where low-frequency refresh rates are also considered static. Therefore, if it is determined to be a static scene at the second moment, the acquisition request is sent; otherwise, if it is determined to be a dynamic scene, the acquisition request is not sent.

[0109] In an optional embodiment, the above judgment process can be implemented as follows:

[0110] At the second moment, the current cumulative number of request judgments and the number of received frames are determined. If the difference between the number of received frames and the number of request judgments is less than or equal to the preset value, it is determined that the acquisition request will be sent to the server again at the second moment. Conversely, if the difference is greater than the preset value, it is determined that the acquisition request will not be sent to the server again at the second moment. The number of received frames refers to the cumulative number of encoded frames received from the server after a communication connection is established with the server. The number of request judgments is the number of times the above-mentioned acquisition request is judged to be sent to the server, which is actually the count number of the above-mentioned set duration starting from the first moment.

[0111] Specifically, assuming that the above-mentioned set duration is 200 milliseconds, and the preset value for comparison with the above-mentioned difference is 1, after the client is connected to the server, each time a coded frame is received, it will accumulate the count, for example, represented by A. Every 200 milliseconds, it will confirm whether it is necessary to request an I frame. The basis for judgment is to compare the current A value with the number of request judgments for the I frame. Assuming that the number of request judgments is represented by B, if the current AB≤1, it is considered that no new coded frames have been sent within 200 milliseconds. The client believes that the picture scene at this time is a static scene, and requests the server to obtain the I frame.

[0112] It is understandable that after each judgment, the count value B will be updated by one, regardless of whether the condition AB≤1 is met.

[0113] This shows that for the client, requesting an I-frame from the server every set duration when the current video scene is determined to be static can prevent network anomalies and other issues from impacting the user's video viewing experience. Furthermore, if the client sends an I-frame request to the server after establishing a connection with the server, but the request is not successfully received by the server due to network anomalies or other reasons, the above strategy of performing a judgment once per set duration can also ensure that the server returns the I-frame to the client as quickly as possible, improving stability.

[0114] Figure 3 A flowchart of a video data transmission method provided by an embodiment of the present invention, which can be executed by a server, such as Figure 3 As shown, the method includes the following steps:

[0115] 301. Receive an acquisition request for acquiring a key frame sent by a client after establishing a communication connection with a server.

[0116] 302. Encode the current video picture to be transmitted into a key frame, where the key frame includes decoding information.

[0117] 303. Send the key frame to the client, so that the client displays the decoded video image after decoding the key frame according to the decoding information.

[0118] This embodiment is a step executed by the server. The specific implementation process can refer to the relevant description in the above embodiment and will not be repeated here.

[0119] The above-mentioned end (client) cloud (server) collaboration solution provided by the embodiment of the present invention can optimize the display delay of the first screen video image, which can be applied to many application scenarios of video data transmission, including but not limited to live broadcast scenarios, and can also be applied to application scenarios such as cloud desktops.

[0120] The cloud desktop and the corresponding client can communicate through a streaming transmission protocol. Simply put, the cloud desktop will encode the screen content displayed on the desktop into a video stream and transmit it to the client for decoding and display.

[0121] Cloud desktops can be used in many specific application scenarios, such as office scenarios, teaching scenarios, etc. Office scenarios are the most common usage scenarios of cloud desktops and will not be discussed here.

[0122] Cloud desktops can be used in teaching scenarios, such as teacher instruction and student demonstrations. For example, a teacher's cloud desktop can share their screen content with all students in the same classroom in real time, achieving unified teaching. For example, a student can share their cloud desktop screen content with the teacher and other students in real time, facilitating sharing and mutual learning.

[0123] All of the above specific application scenarios are inseparable from screen display, so optimizing the display delay of the first screen is also very important. Delayed waiting not only affects the teacher's teaching rhythm, but also affects the efficiency of the class. Therefore, in the application scenarios exemplified above, the video data transmission solution provided by the embodiment of the present invention can be used to optimize the first screen display delay indicator.

[0124] The following is a detailed description of the video transmission process based on the cloud desktop.

[0125] Figure 4 A schematic diagram of a video data transmission system based on a cloud desktop provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, the system includes: a first cloud desktop, a second cloud desktop, a first client connected to the first cloud desktop, and a second client connected to the second cloud desktop.

[0126] As described above, the communication connection between the first cloud desktop and the first client can be a communication connection that supports a certain streaming transmission protocol. Similarly, the communication connection between the second cloud desktop and the second client can also be a communication connection that supports this streaming transmission protocol.

[0127] Optionally, in order to achieve video data transmission between different clients, such as Figure 4 As shown, the system may further include: a network forwarding server.

[0128] Assume that the video data is transmitted between the first client, the second client and the network forwarding server via the RTSP protocol. Figure 4 As shown in , the first client and the second client further include a communication component supporting the RTSP protocol: the RTSP communication component can realize the conversion of video data corresponding to the streaming transmission protocol and video data corresponding to the RTSP protocol.

[0129] In combination with some actual application needs, such as different employees in the same company may have the need to share cloud desktops, and teachers and students in an electronic classroom may have the need to share cloud desktops. Therefore, in an optional specific implementation process, the above-mentioned first client and the second client can be located in the same multicast group. At this time, a multicast group may include clients corresponding to many users (such as clients corresponding to a teacher and many students), and the above-mentioned first client and second client are only two of them. Since the principles of the video data transmission process between different clients are similar, only these two clients are used as an example for illustration in the embodiment of the present invention.

[0130] Based on the above system composition, taking the process of the first client sharing the screen content of the first cloud desktop to the second client as an example, the video data transmission solution is as follows:

[0131] The first client, in response to a screen sharing operation triggered by a user on the first client, sends an acquisition request for obtaining an I frame to the first cloud desktop, receives a first I frame fed back by the first cloud desktop, the first I frame includes first decoding information, and sends the first I frame to the second client.

[0132] The first cloud desktop, in response to the acquisition request, encodes the first video picture to be transmitted into a first I frame and feeds it back to the first client, where the first video picture is the picture currently presented by the first cloud desktop.

[0133] The second client decodes the first I frame according to the first decoding information, and displays the decoded first video picture on the second video picture, wherein the second video picture is the picture currently presented by the second cloud desktop.

[0134] The second cloud desktop transmits the video image corresponding to the second cloud desktop to the second client.

[0135] Assuming the operator of the first client is User 1 and the operator of the second client is User 2, after User 1 and User 2 log in to their respective clients, they establish a communication connection with their respective cloud desktops and can view the video images of the corresponding cloud desktops in their respective client interfaces. This process does not involve the display delay of the first screen.

[0136] When user 1 triggers a screen sharing operation on the first client, it means that user 1 wants to share the screen of the first cloud desktop with other users, assuming that it is shared with user 2. In response to the screen sharing operation triggered by user 1, the first client sends an acquisition request for obtaining an I frame to the first cloud desktop through the communication connection between the first client and the first cloud desktop. In response to the acquisition request, the first cloud desktop encodes the first video image to be transmitted into a first I frame and feeds it back to the first client. The first I frame includes first decoding information. The first video image is the screen content displayed on the first cloud desktop when the acquisition request is received.

[0137] After receiving the first I frame, the first client forwards the first I frame to the second client. The second client decodes the first I frame according to the first decoding information in the first I frame and displays the corresponding first video picture on the client.

[0138] It should be noted that after user 2 opens the second client, the second client is connected to the second cloud desktop. Therefore, the second cloud desktop will send the corresponding video stream of the second cloud desktop to the second client in real time, and the second cloud desktop will decode and display it. Therefore, when the second client decodes the first I frame to obtain the corresponding first video image, the second client will also display the second video image decoded from the video stream transmitted by the second cloud desktop. The second client can float the first video image on the second video image.

[0139] For example, at time T, the second client decodes the first video screen. At the same time, the screen presented on the second cloud desktop is the second video screen. A video playback window can be generated on the second client, and the first video screen is displayed in the video playback window. The positional relationship between the video playback window and the display area of ​​the second video screen is not specifically limited, and can be located inside or outside the display area of ​​the second video screen or partially overlap.

[0140] In addition, in the specific implementation process, the implementation method of forwarding the above-mentioned first I frame from the first client to the second client is not specifically limited. For example, when user 1 knows the IP address corresponding to user 2's second client, user 1 can enter the IP address corresponding to the second client when triggering the screen sharing operation, thereby realizing the one-to-one forwarding. For another example, when the first client and the second client belong to the same multicast group, the forwarding of the first I frame can also be realized by the following method:

[0141] The first client sends the first I-frame to a network forwarding server. The network forwarding server sends the first I-frame to the multicast address corresponding to the multicast group and generates a corresponding target URL link, wherein the target URL link includes the multicast address corresponding to the multicast group. The network forwarding server then feeds the target URL link back to the first client and sends the target URL link to the second client. After receiving the target URL link, the second client parses the target URL link and establishes a communication connection with the network forwarding server to obtain the first I-frame from the multicast address.

[0142] Since the first client and the second client are set in the same multicast group, ie, the local area network, they can establish a TCP connection, thereby forwarding the target URL link, and the video data with a large amount of data is forwarded through the network forwarding server.

[0143] Through the above solution, in the cloud desktop application scenario, the screen content corresponding to the cloud desktop can be shared between the clients corresponding to different cloud desktops. Among them, the first client as the sharing source requests an I frame from the corresponding first cloud desktop when the user triggers the screen sharing operation, and forwards it to the second client as the sharing destination, so that the second client can display the video screen of the first cloud desktop more quickly.

[0144] Figure 5 A flowchart of a video data transmission method provided by an embodiment of the present invention, the method can be executed by a first client connected to a first cloud desktop, such as Figure 5 As shown, the method includes the following steps:

[0145] 501. In response to a screen sharing operation triggered by a user on a first client, an acquisition request for obtaining an I frame is sent to a first cloud desktop to which the first client is connected, so that the first cloud desktop encodes the video picture currently to be transmitted into a first I frame, where the video picture is the picture currently presented by the first cloud desktop.

[0146] As described above, the acquisition request optionally includes a first GOP length, so that upon receiving the acquisition request, the first cloud desktop initiates encoding of a new GOP, where the length of the new GOP is the first GOP length, and the first I-frame is the first frame in the new GOP. If the first cloud desktop does not receive the acquisition request, the GOP length used is a second GOP length, which is greater than the first GOP length.

[0147] 502. Receive a first I-frame fed back by a first cloud desktop, where the first I-frame includes first decoding information.

[0148] 503. Send the first I frame to the second client, so that the second client decodes the first I frame according to the first decoding information and displays the decoded video image, and the second client connects to the second cloud desktop.

[0149] As described above, the second client can store the received first I-frame in a local cache queue. If the number of frames stored in the cache queue reaches a set number, or the cache duration of the cache queue reaches a set duration, the second client can read multiple encoded frames stored in the cache queue and parse the first I-frame contained in these multiple encoded frames to obtain the first decoding information. These multiple encoded frames are the encoding results of multiple frames of video images, and the multiple encoded frames are the encoded frames that the first client has received from the first cloud desktop and forwarded to the second client in sequence since the screen sharing operation was triggered.

[0150] 504. Starting from the first moment, determine whether to send the acquisition request to the first cloud desktop again every set time period, and the first moment is the first sending moment of the acquisition request.

[0151] 505. Send the acquisition request to the first cloud desktop at a second time, wherein it is determined to send the acquisition request to the first cloud desktop again at the second time, and the second time is separated from the first time by at least the set time length.

[0152] For any second moment determined after the first moment according to the set duration, at the second moment, the currently accumulated number of request determinations and the number of received frames are determined. If the difference between the number of received frames and the number of request determinations is less than or equal to a preset value, it is determined to resend the acquisition request to the first cloud desktop at the second moment. The number of received frames refers to the cumulative number of encoded frames received from the first cloud desktop after the screen sharing operation is triggered.

[0153] 506. Receive a second I frame fed back by the first cloud desktop, where the second I frame includes second decoding information, and send the second I frame to the second client, so that the second client decodes the second I frame according to the second decoding information and displays the decoded video picture.

[0154] For the contents not introduced in detail in this embodiment, please refer to the relevant descriptions in the aforementioned embodiments and will not be repeated here.

[0155] The following takes the electronic teaching scenario as an example to illustrate a specific implementation process of the video data transmission solution based on the cloud desktop. Specifically, combined with Figure 6 , taking the process of a teacher sharing the teaching content on his or her cloud desktop with students as an example.

[0156] In order to realize e-learning, the clients of teachers and students in a class generally need to include corresponding application software. Figure 6As shown in Figure 2, both the teacher and student clients are running teaching software. To conduct e-learning, the teacher and each student log in to their respective clients, open the teaching software, and select their own class from the displayed classes to initiate e-learning. When a teacher and multiple students select a class, they form a multicast group and are assigned a corresponding multicast group address. Figure 6 The student client shown in the figure is a client corresponding to any one of the multiple students, and the teacher client and the student client refer to clients connected to their respective corresponding cloud desktops (teacher cloud desktop, student cloud desktop).

[0157] When a teacher shares teaching content with students, the screen content displayed on the teacher's cloud desktop is the "teaching courseware," such as a PowerPoint presentation. After the teacher triggers the "screen sharing operation" on their teaching software, the teacher client sends an I-frame acquisition request to the teacher's cloud desktop. The teacher's cloud desktop then returns the target I-frame to the teacher client. This target I-frame is obtained by performing I-frame video encoding on the screen content displayed on the teacher's cloud desktop when the acquisition request is received. This target I-frame can be a regular I-frame or an IDR frame, which contains the decoding information required to decode the target I-frame.

[0158] After receiving the target I frame, the teacher client can forward the target I frame to Figure 6 The multicast server shown in the figure can actually be a network device such as a gateway.

[0159] Specifically, if Figure 6 As shown in , it is assumed that the communication protocol used between the teacher client and the teacher cloud desktop is ASP, and the communication protocol used between the teacher client and the multicast server and between the multicast server and the student client is RTSP. Then in actual applications, after the teacher client receives the above-mentioned target I frame, it sends the target I frame to the multicast server, so that the multicast server performs corresponding encapsulation processing on the target I frame corresponding to the RTSP protocol, and sends the encapsulated target I frame to the above-mentioned multicast group address for storage.

[0160] Afterwards, the multicast server generates a target URL link, which includes the multicast group address. By accessing the target URL link, the teacher client can obtain the various coded frames received from the teacher's cloud desktop, including the target I frame. The multicast server feeds the generated target URL link back to the teacher client, which connects the local teaching software to the teaching software of each student (for example, Figure 6 The target URL link is sent to the student client through the TCP connection shown in FIG.

[0161] After receiving the target URL, the student client parses it to determine the multicast group address contained therein. After accessing the target URL to establish a communication connection with the multicast server, the student client retrieves the encoded frames stored therein, including the target I-frame. Furthermore, the student client decodes and displays the target I-frame based on the decoding information contained in the target I-frame.

[0162] As mentioned above, on the one hand, the teacher client continues to forward the various encoded frames received from the teacher cloud desktop to the multicast server with the above-mentioned multicast group address as the destination address, so that the student client can pull these encoded frames for decoding and display. On the other hand, the teacher client can also determine whether to send the acquisition request again at set time intervals after sending the above-mentioned acquisition request. The specific implementation process refers to the relevant descriptions in the above-mentioned other embodiments and will not be repeated here.

[0163] The above briefly introduces the video transmission process of the teacher's teaching courseware shared by the teacher client to the student client. Figure 7 and Figure 8 An optional specific implementation process of the teacher client and the student client is described respectively.

[0164] The execution process on the teacher client side is as follows Figure 7 Specifically, first, the teacher opens the teaching software in the teacher client to trigger multicast teaching (ie, trigger the above-mentioned screen sharing function), and the teacher client sends a flow switching notification to the teacher cloud desktop based on the communication connection between the teacher client and the teacher cloud desktop.

[0165] In fact, before the teaching software is started on the teacher client, the teacher client also receives and displays the first video stream sent by the teacher cloud desktop. The first video stream is the screen image of the teacher cloud desktop before the teaching software is started. After the teaching software is started on the teacher client, the teacher client receives and displays the second video stream sent by the teacher cloud desktop. The video type, encoding method, etc. corresponding to the first video stream are different from those of the second video stream. That is to say, after starting the teaching software to start multicast teaching, the teacher client sends a stream switching notification to the teacher cloud desktop. Afterwards, the teacher cloud desktop encodes the screen image on the teacher cloud desktop with another video type and encoding method, and sends the obtained encoded frames to the teacher client.

[0166] In addition, in response to the teacher's operation of starting multicast teaching, the teacher client will also send an I frame acquisition request to the teacher's cloud desktop, thereby receiving an I frame obtained by the teacher's cloud desktop after I frame encoding the current video picture to be transmitted using the above-mentioned "another video type and encoding method".

[0167] The teacher client forwards each of the consecutively received coded frames, including the above-mentioned I frame, to the multicast server, so that the student client can pull each coded frame from the multicast server. The process of the multicast server generating the corresponding URL link is described above and will not be repeated here.

[0168] In addition, if Figure 7 As shown in , at each set time, the teacher client determines whether it needs to request an I-frame from the teacher's cloud desktop again. If so, it sends an I-frame request to the teacher's cloud desktop and then updates the request judgment condition. If not, it directly updates the request judgment condition and then receives the encoded frames sent by the teacher's cloud desktop normally.

[0169] As mentioned above, the basis for judgment is the relationship between the difference between the cumulative number of received coded frames and the number of requested judgment times and the set value. Therefore, the update of the request judgment condition is: after each judgment is executed, the number of requested judgment times is increased by one.

[0170] The execution process on the student client side is as follows Figure 8 Specifically, the student client receives the URL link sent by the teacher client and sets the cache parameters corresponding to the local cache space, such as the number of cached frames and the cache duration. The student client parses the URL link to establish a connection with the multicast server, then pulls the encoded frames from the multicast group address and stores them in the local cache.

[0171] When the number of coded frames stored in the local cache reaches the cache parameter value, the coded frames stored in the local cache are read to parse the decoding information therefrom.

[0172] Among them, due to the teacher client's request for the I frame, the student client's local cache can quickly store the I frame containing the required decoding information. In this way, the cache parameter value can be set to a smaller value. Since subsequent processing can only be triggered after the cache parameter value is reached, the setting of a smaller cache parameter value can enable faster decoding processing.

[0173] After parsing the decoding information contained in the I frame, the student client can complete the decoding and display of the I frame. Based on the decoding result of the I frame, the subsequent decoding and display of the P frame and B frame that depend on the I frame can be performed.

[0174] The above is an example of a scenario where a teacher shares teaching materials with students. In fact, when students need to demonstrate, that is, when the screen content of the student cloud desktop needs to be shared with other students and teachers, the processing process is similar and will not be repeated here.

[0175] The video data transmission device of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these devices can be constructed using commercially available hardware components and configured according to the steps taught in this solution.

[0176] Figure 9 A schematic diagram of the structure of a video data transmission device provided by an embodiment of the present invention, the device is applied to a client, such as Figure 9 As shown, the device includes: a sending module 11, a receiving module 12, and a decoding module 13.

[0177] The sending module 11 is configured to send an acquisition request for acquiring a key frame to the server in response to establishing a communication connection with the server, so that the server encodes the current video picture to be transmitted into a first key frame.

[0178] The receiving module 12 is configured to receive the first key frame fed back by the server, where the first key frame includes first decoding information.

[0179] The decoding module 13 is configured to decode the first key frame according to the first decoding information to display the decoded video picture.

[0180] Optionally, the acquisition request includes a first GOP length, so that the server starts encoding a new GOP, the length of the new GOP is the first GOP length, and the first key frame is the first frame in the new GOP;

[0181] The image group length used when the server does not receive the acquisition request is the second image group length, and the second image group length is greater than the first image group length.

[0182] Optionally, the device further includes: a judgment module for determining whether to resend the acquisition request to the server at set time intervals starting from a first moment, wherein the first moment is the first time the acquisition request is sent. The sending module 11 is also used to send the acquisition request to the server at a second moment, wherein it is determined that the acquisition request is sent to the server again at the second moment, and the second moment is separated from the first moment by at least one of the set time intervals. The receiving module 12 is also used to receive a second key frame fed back by the server, wherein the second key frame includes second decoding information. The decoding module 13 is also used to decode the second key frame according to the second decoding information to display the decoded video picture.

[0183] Optionally, the judgment module is specifically configured to: at the second moment, if it is determined that the current video picture change scene is a static scene, determine to send the acquisition request to the server again at the second moment.

[0184] Among them, optionally, the judgment module is specifically used to: determine the current cumulative number of request judgments and the number of received frames at the second moment, wherein the number of received frames refers to the cumulative number of encoded frames received from the server after establishing the communication connection with the server; if the difference between the number of received frames and the number of request judgments is less than or equal to a preset value, determine to send the acquisition request to the server again at the second moment.

[0185] Optionally, the decoding module 13 is specifically used to: store the received first key frame into a local cache queue; if the number of frames stored in the cache queue reaches a set number, or the cache duration of the cache queue reaches a set duration, read the multiple encoded frames stored in the cache queue, the multiple encoded frames are the encoding results of multiple frames of video images, and the multiple encoded frames are the encoded frames received in sequence from the server after establishing a communication connection with the server; parse the first decoding information from the first key frame contained in the multiple encoded frames.

[0186] Figure 9 The device shown can execute the steps executed by the client in the aforementioned embodiment. The detailed execution process and technical effects can be found in the description of the aforementioned embodiment and will not be repeated here.

[0187] In one possible design, the above Figure 9 The structure of the video data transmission device shown can be realized as an electronic device. Figure 10 As shown, the electronic device may include: a processor 21, a memory 22, and a communication interface 23. The memory 22 stores executable code, which, when executed by the processor 21, enables the processor 21 to at least implement the video data transmission method executed by the client in the aforementioned embodiment.

[0188] Figure 11 A schematic diagram of the structure of a video data transmission device provided by an embodiment of the present invention, the device is applied to a server, such as Figure 11 As shown, the device includes: a receiving module 31, an encoding module 32, and a sending module 33.

[0189] The receiving module 31 is configured to receive an acquisition request for acquiring a key frame sent by the client after establishing a communication connection with the server.

[0190] The encoding module 32 is configured to encode the video picture to be currently transmitted into key frames, wherein the key frames include decoding information.

[0191] The sending module 33 is configured to send the key frame to the client, so that the client displays the decoded video image after decoding the key frame according to the decoding information.

[0192] Figure 11 The device shown can execute the steps executed by the server in the aforementioned embodiment. The detailed execution process and technical effects can be found in the description of the aforementioned embodiment and will not be repeated here.

[0193] In one possible design, the above Figure 11 The structure of the video data transmission device shown can be realized as an electronic device. Figure 12 As shown, the electronic device may include: a processor 41, a memory 42, and a communication interface 43. The memory 42 stores executable code, which, when executed by the processor 41, enables the processor 41 to at least implement the video data transmission method executed by the server in the aforementioned embodiment.

[0194] In addition, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the video data transmission method provided in the aforementioned embodiment.

[0195] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Those skilled in the art can understand and implement the present invention without inventive effort.

[0196] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by a combination of hardware and software. Based on this understanding, the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A video data transmission method, characterized in that: Applied to a client, the method includes: In response to establishing a communication connection with the server, sending an acquisition request for acquiring a key frame to the server, so that the server encodes the current video picture to be transmitted into a first key frame; receiving the first key frame fed back by the server, where the first key frame includes first decoding information; Decoding the first key frame according to the first decoding information to display the decoded video picture; The acquisition request includes a first image group length, so that the server starts encoding a new image group. The length of the new image group is the first image group length, and the first key frame is the first frame in the new image group. The image group length used by the server when it does not receive the acquisition request is the second image group length, and the second image group length is greater than the first image group length. The length of the image group after the new image group is the second image group length.

2. The method according to claim 1, characterized in that The method further comprises: Starting from a first moment, determining whether to send the acquisition request to the server again at a set time interval, the first moment being the first time the acquisition request is sent; Sending the acquisition request to the server at a second time, wherein it is determined that the acquisition request needs to be sent to the server again at the second time, and the second time is separated from the first time by at least the set time duration; receiving a second key frame fed back by the server, where the second key frame includes second decoding information; The second key frame is decoded according to the second decoding information to display a decoded video picture.

3. The method according to claim 2, characterized in that The determining whether to resend the acquisition request to the server at set intervals includes: At the second moment, if it is determined that the current video picture change scene is a static scene, it is determined to send the acquisition request to the server again at the second moment.

4. The method according to claim 3, characterized in that At the second moment, determining whether the current video picture change scene is a static scene includes: At the second moment, determining a currently accumulated number of request determinations and a number of received frames, wherein the number of received frames refers to a cumulative number of coded frames received from the server after establishing the communication connection with the server; If the difference between the number of received frames and the number of requested judgment times is less than or equal to a preset value, it is determined that the current video picture change scene is a static scene.

5. The method according to any one of claims 1 to 4, characterized in that The decoding the first key frame according to the first decoding information to display the decoded video picture includes: Storing the received first key frame in a local cache queue; If the number of frames stored in the cache queue reaches a set number, or the cache duration of the cache queue reaches a set duration, reading a plurality of encoded frames stored in the cache queue, the plurality of encoded frames being encoding results of a plurality of frames of video images, and the plurality of encoded frames being encoded frames received sequentially from the server after establishing a communication connection with the server; The first decoding information is parsed from the first key frame included in the multiple coded frames.

6. A video data transmission method, characterized in that: Applied to the server, the method includes: Receiving an acquisition request for acquiring a key frame sent by the client after establishing a communication connection with the server; Encoding the current video picture to be transmitted into a key frame, wherein the key frame includes decoding information; Sending the key frame to the client, so that the client displays the decoded video picture after decoding the key frame according to the decoding information; The acquisition request includes a first image group length, so that the server starts encoding a new image group, the length of the new image group is the first image group length, and the key frame is the first frame in the new image group; the image group length used when the server does not receive the acquisition request is a second image group length, and the second image group length is greater than the first image group length; the length of the image group after the new image group is the second image group length.

7. A video data transmission method, characterized in that: Applied to a first client, the method includes: In response to a screen sharing operation triggered by a user on the first client, an acquisition request for acquiring a key frame is sent to a first cloud desktop connected to the first client, so that the first cloud desktop encodes a video picture currently to be transmitted into a first key frame, where the video picture is the picture currently presented by the first cloud desktop; the acquisition request includes a first image group length, so that the first cloud desktop starts encoding a new image group, the length of the new image group is the first image group length, and the first key frame is the first frame in the new image group; the image group length used by the first cloud desktop when the acquisition request is not received is a second image group length, where the second image group length is greater than the first image group length; the length of the image group after the new image group is the second image group length; Receive the first key frame fed back by the first cloud desktop, where the first key frame includes first decoding information; The first key frame is sent to a second client, so that the second client decodes the first key frame according to the first decoding information and displays the decoded video picture, and the second client is connected to a second cloud desktop.

8. The method according to claim 7, characterized in that The method further comprises: Starting from a first moment, determining whether to send the acquisition request to the first cloud desktop again at a set time interval, the first moment being the first time the acquisition request is sent; Sending the acquisition request to the first cloud desktop at a second time, wherein determining to send the acquisition request to the first cloud desktop again at the second time, the second time being separated from the first time by at least the set time length; Receive a second key frame fed back by the first cloud desktop, where the second key frame includes second decoding information; The second key frame is sent to the second client, so that the second client decodes the second key frame according to the second decoding information and displays the decoded video picture.

9. A video data transmission system, characterized in that: include: A first cloud desktop, a second cloud desktop, a first client connected to the first cloud desktop, and a second client connected to the second cloud desktop; The first client is configured to, in response to a screen sharing operation triggered by a user on the first client, send an acquisition request for acquiring a key frame to the first cloud desktop, receive a first key frame fed back by the first cloud desktop, the first key frame including first decoding information, and send the first key frame to the second client; the acquisition request includes a first picture group length, so that the first cloud desktop starts encoding a new picture group, the length of the new picture group is the first picture group length, and the first key frame is the first frame in the new picture group; the picture group length used by the first cloud desktop when the acquisition request is not received is a second picture group length, the second picture group length being greater than the first picture group length; the length of the picture group after the new picture group is the second picture group length; The first cloud desktop is configured to, in response to the acquisition request, encode a first video picture to be transmitted into the first key frame and feed the first key frame back to the first client, where the first video picture is a picture currently presented by the first cloud desktop; The second client is configured to decode the first key frame according to the first decoding information, and display a second video image and the decoded first video image, wherein the second video image is a image currently presented by the second cloud desktop; The second cloud desktop is used to transmit the video image corresponding to the second cloud desktop to the second client.

10. The system according to claim 9, characterized in that The first client and the second client are located in the same multicast group; the system further includes: a network forwarding server; The first client is specifically configured to: send the first key frame to a network forwarding server, receive a target URL link fed back by the network forwarding server, and send the target URL link to the second client; The network forwarding server is configured to receive the first key frame, send the first key frame to the multicast address corresponding to the multicast group, and generate the target URL link, wherein the target URL link includes the multicast address corresponding to the multicast group; The second client is configured to establish a communication connection with the network forwarding server according to the target URL link, so as to obtain the first key frame from the multicast address.

11. The system according to claim 9 or 10, characterized in that: The first client is further configured to determine whether to resend the acquisition request to the first cloud desktop at set intervals starting from a first moment, the first moment being the first time the acquisition request is sent; and to send the acquisition request to the first cloud desktop at a second moment, the second moment being separated from the first moment by at least one of the set intervals. receiving a second key frame fed back by the first cloud desktop, and sending the second key frame to a second client, where the second key frame includes second decoding information; The second client is further configured to decode the second key frame according to the second decoding information and display the decoded video picture.

12. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores an executable code, and when the executable code is executed by the processor, the processor executes the video data transmission method according to any one of claims 1 to 4, or the video data transmission method according to claim 5, or the video data transmission method according to any one of claims 6 to 8.

13. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor executes the video data transmission method as described in any one of claims 1 to 4, or executes the video data transmission method as described in claim 5, or executes the video data transmission method as described in any one of claims 6 to 8.

Images