Data sending method and device in weak network scene

Abstract

The invention relates to the technical field of weak media, and discloses a data sending method and device in a weak network scene. The method comprises the following steps: acquiring cache data and a latest frame in the cache data; wherein the latest frame is a data frame to be sent; determining a target key frame from the cached data according to the type of the latest frame; changing a video timestamp between the target key frame and the latest frame into a video timestamp corresponding to the target key frame to obtain a modified video frame; and packaging the modified video frame into a video stream, and sending the video stream to the client according to the target rate.

Description

Technical Field

[0001] The present invention relates to the field of streaming media technology, and in particular to a data sending method, device, computer equipment, storage medium and program product in a weak network scenario. Background Art

[0002] The current "quick start" playback strategy on the market mainly relies on cached data stored on the server to achieve fast startup. When a user initiates a playback request, the system will first read the data from the server's cache and immediately send it to the end user.

[0003] When the network condition is poor, quickly transmitting a large amount of data to the end user may further occupy network egress resources, which will lead to network bandwidth shortage and even network congestion, thus affecting the normal operation of other network applications. Summary of the invention

[0004] In view of this, the present invention provides a data sending method, apparatus, computer equipment, storage medium and program product in a weak network scenario.

[0005] In a first aspect, the present invention provides a method for sending data in a weak network scenario, the method comprising: obtaining cached data and the latest frame in the cached data; wherein the latest frame is a data frame to be sent; determining a target key frame from the cached data according to the type of the latest frame; changing the video timestamps between the target key frame and the latest frame to the video timestamps corresponding to the target key frame to obtain a modified video frame; encapsulating the modified video frame into a video stream, and sending the video stream to the client at a target rate.

[0006] The data transmission method in a weak network scenario provided by this embodiment reduces the amount of data transmitted when the network is poor by transmitting only key frames and necessary video data. Reducing the amount of data transmission means reducing the load on the network, thereby reducing network congestion and delay, and improving the stability and reliability of the overall network. At the same time, the sending rate of the video stream is dynamically adjusted according to the network conditions to ensure that the client can stably receive and play data.

[0007] In one possible implementation, a target key frame is determined from cached data according to the type of the latest frame, including: if the type of the latest frame is a bidirectional prediction frame, obtaining a preset cache time, a decoding time of the bidirectional prediction frame, and a display time of the decoded bidirectional prediction frame; obtaining a first target time according to the display time of the decoded bidirectional prediction frame, the decoding time of the bidirectional prediction frame, and the preset cache time; traversing the cached data to determine a first target data frame that differs from the latest frame by the first target time; wherein, if the first target data frame is a key frame, using the key frame as the target key frame.

[0008] The data transmission method in a weak network scenario provided by this embodiment can obtain an accurate first target time by accurately calculating the decoding time, display time and preset cache time of the bidirectional prediction frame. This time point is used to find a key frame that matches the target time in the cached data, thereby ensuring the synchronization of video playback. And because the playback time can be accurately controlled, the playback delay caused by waiting for data decoding or transmission can be reduced. At the same time, the bidirectional prediction frame usually needs to rely on the previous and next frames for decoding, and this method can reduce the decoding operation of unnecessary frames by determining the target key frame, thereby improving the decoding efficiency.

[0009] In one possible implementation, a target key frame is determined from cached data according to the type of the latest frame, including: if the type of the latest frame is a predicted frame, determining the previous data frame of the predicted frame; obtaining a preset cache time, a display time of the predicted frame after decoding, and a display time of the previous data frame of the predicted frame after decoding; determining a second target time according to the preset cache time, the display time of the predicted frame after decoding, and the display time of the previous data frame of the predicted frame after decoding; traversing the cached data to determine a second target data frame that differs from the data frame by the second target time; wherein, if the second target data frame is a key frame, using the key frame as the target key frame.

[0010] The data transmission method in a weak network scenario provided by this embodiment can ensure that when the predicted frame is played, the previous data frame it depends on is ready, thereby reducing the possibility of playback interruption. And because it can ensure that the dependent frame of the predicted frame has been decoded and available, the video can be played more smoothly, reducing the jamming caused by waiting for the dependent frame to be decoded. At the same time, by accurately calculating the second target time and determining the target key frame, the occupancy of unnecessary data in the cache can be reduced, thereby reducing the consumption of system resources.

[0011] In a possible implementation, the method further includes: if the first target data frame is not a key frame, continuing to traverse the cached data until the determined data frame is a key frame, and using the key frame as the target key frame.

[0012] The data transmission method in a weak network scenario provided by this embodiment, in video encoding, a key frame (such as an I frame) is a frame that can be decoded independently and does not depend on other frames. Therefore, if the first target data frame is not a key frame, continue to traverse the cached data until the determined data frame is a key frame, which can ensure that subsequent video frames can be correctly decoded based on this key frame. At the same time, if a non-key frame is used directly as the starting point for decoding, decoding failure or playback interruption may occur. By ensuring the existence of the target key frame, this risk can be reduced, thereby maintaining the continuity of video playback.

[0013] In one possible implementation, a video stream is sent to a client at a target rate, including: detecting whether a current confirmation packet returned by the client is a confirmation packet corresponding to a current data packet sent to the client; detecting whether a target confirmation packet returned by the client is not received within a preset time length; wherein the target confirmation packet is a confirmation packet corresponding to a current data packet sent by the client; if the current confirmation packet returned by the client is not a confirmation packet corresponding to a data packet sent by the client or the target confirmation packet returned by the client is not received within the preset time length, starting from the end of the video stream according to the target ratio, gradually moving forward to discard predicted frames until the number of discarded predicted frames reaches a preset threshold, and sending the adjusted video stream to the client at the target rate.

[0014] The data transmission method in a weak network scenario provided by this embodiment can ensure that the client has correctly received the data packet by detecting the confirmation packet. If the confirmation packet is not received or the confirmation packet does not match, the system can detect and take corresponding measures in time to avoid data loss or transmission errors. When it is detected that the client fails to return the target confirmation packet within the preset time, the system can gradually move forward from the end of the video stream and discard the predicted frame. This method helps to reduce video freezes or unsmooth playback caused by network delays or packet loss, and improves user experience.

[0015] In a second aspect, the present invention provides a data sending device in a weak network scenario, the device comprising: an acquisition module, used to acquire cached data and the latest frame in the cached data; wherein the latest frame is a data frame to be sent; a determination module, used to determine a target key frame from the cached data according to the type of the latest frame; a change module, used to change the video timestamps between the target key frame and the latest frame to the video timestamps corresponding to the target key frame, to obtain a modified video frame; a sending module, used to encapsulate the modified video frame into a video stream, and send the video stream to the client at the target rate.

[0016] In one possible implementation, the determination module includes: an acquisition unit, which is used to obtain a preset cache time, a decoding time of the bidirectional prediction frame, and a display time of the decoded bidirectional prediction frame if the type of the latest frame is a bidirectional prediction frame; a first determination unit, which is used to obtain a first target time based on the display time of the decoded bidirectional prediction frame, the decoding time of the bidirectional prediction frame, and the preset cache time; a second determination unit, which is used to traverse the cache data and determine a first target data frame that differs from the latest frame by the first target time; wherein, if the first target data frame is a key frame, the key frame is used as the target key frame.

[0017] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the data sending method in a weak network scenario of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the data sending method in a weak network scenario of the above-mentioned first aspect or any corresponding embodiment thereof.

[0019] In a fifth aspect, the present invention provides a computer program product, including computer instructions, which are used to enable a computer to execute the data sending method in a weak network scenario of the above-mentioned first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 is a flow chart of a method for sending data in a weak network scenario according to an embodiment of the present invention;

[0022] Figure 2 is a schematic diagram of a timestamp setting strategy provided according to an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of a data sending method in a weak network scenario according to an embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of a discard strategy provided according to an embodiment of the present invention;

[0025] Figure 5 is a structural block diagram of a data sending device in a weak network scenario according to an embodiment of the present invention;

[0026] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0028] The current "quick start" playback strategy on the market mainly relies on cached data stored on the server to achieve fast startup. When a user initiates a playback request, the system will first read the data from the server's cache and immediately send it to the end user.

[0029] When the network condition is poor, quickly transmitting a large amount of data to the end user may further occupy network egress resources, which will lead to network bandwidth shortage and even network congestion, thus affecting the normal operation of other network applications.

[0030] Based on this, the present invention provides a data transmission method in a weak network scenario, which reduces the amount of data transmitted when the network is poor by transmitting only key frames and necessary video data, thereby reducing the amount of data transmission, which means reducing the load on the network, thereby reducing network congestion and delay, and improving the stability and reliability of the overall network. At the same time, the sending rate of the video stream is dynamically adjusted according to the network conditions to ensure that the client can stably receive and play data.

[0031] According to an embodiment of the present invention, an embodiment of a method for sending data in a weak network scenario is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] In this embodiment, a data transmission method in a weak network scenario is provided, which can be used for computer devices, such as computers, servers, etc. Figure 1 is a flow chart of a method for sending data in a weak network scenario according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0033] Step S101, obtaining cached data and the latest frame in the cached data; wherein the latest frame is a data frame to be sent.

[0034] The cached data can be represented as data stored in the memory for temporary storage of data to be processed or sent. In video transmission, the cached data may contain multiple frames of video data. The cached data may be live data of 2 gops (Group of Pictures). The latest frame is used to represent the data frame to be sent. Specifically, the cached data is read from the memory and the latest frame is determined.

[0035] As an example, data is read through memory access operations, and cached data is managed using data structures (such as linked lists and queues) to quickly locate the latest frame.

[0036] Step S102: determining a target key frame from the cached data according to the type of the latest frame.

[0037] The types of the latest frames may include P-frames, I-frames, and B-frames.

[0038] I-frame is an intra-frame compressed coded frame, also known as an intra picture. It is a key frame in a video sequence, contains complete picture information, and can be decoded independently. I-frames can be decoded without relying on other frames. I-frame compression efficiency is relatively low, but there is no need to refer to other frames when decoding, so the decoding speed is fast. As a reference point for random access, I-frames can be used to quickly locate a frame in the video. In video editing and clipping, I-frames can be used as key frames for precise operations.

[0039] P frame is a coded frame that compresses the data of this frame based on the difference between this frame and the adjacent previous frame (I frame or P frame), also known as a predicted frame. It improves the efficiency of video compression by reducing redundant information. When playing a video, a P frame can rely on the previous I frame or P frame for decoding, thus ensuring the smoothness of the video. B frame is a coded frame that compresses the data of this frame based on the difference between the adjacent previous frame, this frame and the next frame, also known as a bidirectional predicted frame.

[0040] The target key frame may be an I frame. Specifically, the method of determining the target key frame from the buffered data is different depending on the type of the latest frame.

[0041] As an example, when the type of the latest frame is a P frame, it is necessary to determine the target key frame according to information of a previous data frame of the P frame.

[0042] As an example, when the type of the latest frame is a B frame, the target key frame needs to be determined through information of the B frame.

[0043] Step S103, changing the video timestamps between the target key frame and the latest frame to the video timestamps corresponding to the target key frame to obtain a modified video frame.

[0044] The video timestamp is used to identify the time position of the video frame in the video stream to ensure that the video frames can be played in the correct order. Specifically, after determining the target key frame, the video timestamps corresponding to all data frames between the target key frame and the latest frame are changed to the video timestamp corresponding to the target key frame.

[0045] As an example, first, a key frame is determined in the cache data as the target key frame according to specific rules (such as time interval, content change, etc.). Starting from the target key frame, all frames including the latest frame are traversed backwards, and all frames including these two frames are processed. For each frame traversed, its video timestamp is changed to the timestamp of the target key frame. This usually involves modifying the timestamp field in the frame header.

[0046] In a possible implementation, the frames whose timestamps have been modified may be recorded for subsequent processing or debugging.

[0047] Please refer to Figure 2 , Figure 2 is a schematic diagram of a timestamp setting strategy provided according to an embodiment of the present invention.

[0048] The video timestamps from the I frame to the history (latest frame) are all changed to the I frame timestamps. The audio timestamps remain unchanged.

[0049] As an example, changing all video timestamps from an I frame to a history (latest frame) to an I frame timestamp may be a process of changing a segment of data frames.

[0050] Step S104: encapsulate the modified video frames into a video stream, and send the video stream to the client at a target rate.

[0051] The modified video frames are encapsulated into a video stream. Specifically, the modified video frames can be encapsulated into a video stream by professional encapsulation software, or by other methods, etc., which are not specifically limited here and can be implemented by those skilled in the art.

[0052] As an example, according to the application scenario and client requirements, select a suitable video encapsulation format (such as MP4, FLV, MKV, etc.). The encapsulation format determines the organization structure of the video data and the storage method of metadata (such as timestamps, indexes, etc.). The modified video frames are packaged according to the requirements of the encapsulation format. Each video frame will be encapsulated into one or more data packets and the corresponding header information (such as frame type, timestamp, etc.) will be added. The encapsulated data packets will form a continuous video stream, which is convenient for network transmission and client decoding.

[0053] The target rate can represent a rate that is greater than the playback bit rate. Specifically, the target transmission rate of the video stream is determined based on factors such as network bandwidth, client processing power, and video quality requirements. The target rate should ensure smooth playback of the video stream while avoiding network congestion and client decoding pressure. Use appropriate rate control algorithms (such as TCP flow control, UDP rate limiting, etc.) to ensure that the video stream is sent at the target rate. The algorithm will dynamically adjust the sending rate according to network conditions (such as latency, packet loss rate, etc.) to maintain the stability and reliability of the transmission. Set buffers at the sending and receiving ends to store temporary data to cope with network fluctuations and data transmission delays. The sending end will adjust the sending rate according to the size and status of the buffer to avoid buffer overflow or underflow.

[0054] The data transmission method in a weak network scenario provided by this embodiment reduces the amount of data transmitted when the network is poor by transmitting only key frames and necessary video data. Reducing the amount of data transmission means reducing the load on the network, thereby reducing network congestion and delay, and improving the stability and reliability of the overall network. At the same time, the sending rate of the video stream is dynamically adjusted according to the network conditions to ensure that the client can stably receive and play data.

[0055] In a possible implementation, the above step S102 includes:

[0056] Step a1: If the type of the latest frame is a bidirectional prediction frame, obtain a preset cache time, a decoding time of the bidirectional prediction frame, and a display time of the decoded bidirectional prediction frame.

[0057] As can be seen from the above, the bidirectional prediction frame can be a B frame. Among them, if the type of the latest frame is a bidirectional prediction frame, the preset cache time, the decoding time of the bidirectional prediction frame, and the display time of the decoded bidirectional prediction frame are obtained. Among them, the preset cache time can be determined according to the needs of the system or application to ensure that the video frame has sufficient buffering between decoding and display. The decoding time of the bidirectional prediction frame can represent the time required to decode the current bidirectional prediction frame. The display time of the decoded bidirectional prediction frame can represent the timestamp of when the bidirectional prediction frame should be displayed to the user after decoding.

[0058] Step a2, obtaining a first target time according to the display time of the decoded bidirectional prediction frame, the decoding time of the bidirectional prediction frame and the preset cache time.

[0059] Based on the time parameter obtained in step a1, a value called "first target time" is calculated. This value can be obtained based on the display time, decoding time and preset cache time of the bidirectional prediction frame, and is used to find the frame with this time difference from the latest frame in the cache data.

[0060] As an example, the first target time may be determined by subtracting the preset buffering time from the sum of the display time of the decoded bidirectional prediction frame, the decoding time of the bidirectional prediction frame and the preset buffering time.

[0061] Step a3, traversing the cached data, determining a first target data frame that is different from the latest frame by a first target time; wherein, if the first target data frame is a key frame, the key frame is used as a target key frame.

[0062] According to the calculated first target time, a frame that differs from the first target time is searched in the cache data. Specifically, the position of the first target time can be searched forward from the position of the latest frame to determine the first target data frame. If the first target data frame is a key frame (such as an I frame), it is determined as the target key frame. If the first target data frame is not a key frame, it may be necessary to further determine the target key frame according to a specific rule or algorithm, or in some cases, it may be chosen not to change the current target key frame.

[0063] The data transmission method in a weak network scenario provided by this embodiment can obtain an accurate first target time by accurately calculating the decoding time, display time and preset cache time of the bidirectional prediction frame. This time point is used to find a key frame that matches the target time in the cached data, thereby ensuring the synchronization of video playback. And because the playback time can be accurately controlled, the playback delay caused by waiting for data decoding or transmission can be reduced. At the same time, the bidirectional prediction frame usually needs to rely on the previous and next frames for decoding, and this method can reduce the decoding operation of unnecessary frames by determining the target key frame, thereby improving the decoding efficiency.

[0064] In a possible implementation, the above step S102 includes:

[0065] Step b1: If the type of the latest frame is a predicted frame, determine the previous data frame of the predicted frame.

[0066] As can be seen from the above, the predicted frame can be a P frame. Figure 3 , when the predicted frame is a P frame, its previous data frame can be found. This previous data frame may be an I frame (key frame) or a P frame (another predicted frame, but here it is used as a reference frame).

[0067] Step b2, obtaining a preset cache time, a display time of a decoded predicted frame, and a display time of a previous data frame of the predicted frame after decoding.

[0068] The display time of the decoded predicted frame can represent the timestamp of when the predicted frame should be displayed to the user after being decoded. The display time of the previous data frame of the predicted frame after being decoded can represent the timestamp of when the previous data frame of the predicted frame should be displayed to the user after being decoded.

[0069] Step b3, determining a second target time according to a preset cache time, a display time of a decoded predicted frame, and a display time of a previous data frame of the predicted frame after decoding.

[0070] Please refer to Figure 2 , due to the existence of the cache time, it is necessary to start processing or transmitting the frame in advance. Among them, the display time of the previous data frame can also be considered to ensure that the interval between frames is reasonable. The display time of the previous data frame is mainly used to determine the order and dependency of the frames, rather than directly used to calculate the second target time. A safe second target time can be calculated using the display time of the previous data frame and the estimated decoding time of the current predicted frame.

[0071] As an example, the second target time may be obtained by subtracting the preset cache time from the sum of the display time of the decoded predicted frame and the display time of the previous data frame of the predicted frame after decoding.

[0072] Step b4, traversing the cached data, and determining a second target data frame that is different from the data frame by a second target time; wherein, if the second target data frame is a key frame, the key frame is used as the target key frame.

[0073] According to the calculated second target time, a frame that differs from the second target time is searched in the cache data. Specifically, the position of the second target time can be searched forward from the position of the latest frame to determine the second target data frame. If the second target data frame is a key frame (such as an I frame), it is determined as the target key frame. If the second target data frame is not a key frame, it may be necessary to further determine the target key frame according to a specific rule or algorithm, or in some cases, it may be chosen not to change the current target key frame.

[0074] The data transmission method in a weak network scenario provided by this embodiment can ensure that when the predicted frame is played, the previous data frame it depends on is ready, thereby reducing the possibility of playback interruption. And because it can ensure that the dependent frame of the predicted frame has been decoded and available, the video can be played more smoothly, reducing the jamming caused by waiting for the dependent frame to be decoded. At the same time, by accurately calculating the second target time and determining the target key frame, the occupancy of unnecessary data in the cache can be reduced, thereby reducing the consumption of system resources.

[0075] In a possible implementation, the method further includes: if the first target data frame is not a key frame, continuing to traverse the cached data until the determined data frame is a key frame, and using the key frame as the target key frame.

[0076] When the first target data frame is not a key frame, it indicates that the video stream data cannot be sent from the frame, and the cache data needs to be traversed forward or backward from the frame until the first key frame is found.

[0077] As an example, the first target data frame is a P frame, which is not a key frame. Therefore, the system needs to continue traversing the cache to find the key frame. The traversal process continues until an I frame is found, and this I frame is determined to be the target key frame. This process ensures that the system can find and process the key frame even if the frame initially received is not a key frame.

[0078] It should be noted that when the second target data frame is not a key frame, the processing method is the same as the processing method when the first target data frame is not a key frame, and will not be described in detail here.

[0079] The data transmission method in a weak network scenario provided by this embodiment, in video encoding, a key frame (such as an I frame) is a frame that can be decoded independently and does not depend on other frames. Therefore, if the first target data frame is not a key frame, continue to traverse the cached data until the determined data frame is a key frame, which can ensure that subsequent video frames can be correctly decoded based on this key frame. At the same time, if a non-key frame is used directly as the starting point for decoding, decoding failure or playback interruption may occur. By ensuring the existence of the target key frame, this risk can be reduced, thereby maintaining the continuity of video playback.

[0080] In a possible implementation, sending the video stream to the client at the target rate in step S104 includes:

[0081] Step c1, detecting whether the current confirmation packet returned by the client is the confirmation packet corresponding to the current data packet sent to the client.

[0082] After sending a data packet, the server will wait for an acknowledgment packet (ACK) returned by the client. This acknowledgment packet can contain information such as the sequence number or timestamp of the confirmed data packet, so that the server can confirm which data packet the client has received. Specifically, after the server sends a data packet, it needs to detect whether the acknowledgment packet corresponding to the data packet is returned to the server.

[0083] As an example, the server sends data packet A, data packet B and data packet C in sequence; when data packet C is sent, the returned confirmation packet is the confirmation packet of data packet A, then it is considered that data packet A is abnormal.

[0084] Step c2, detecting whether a target confirmation packet returned by the client is not received within a preset time period; wherein the target confirmation packet is a confirmation packet corresponding to the current data packet sent by the client.

[0085] The server can set a preset time (timeout) to wait for the target confirmation packet (i.e., the confirmation packet for the current data packet) returned by the client. If the target confirmation packet is not received within this preset time, the server believes that the current data packet may be lost or the client fails to process the data packet in time.

[0086] Step c3: If the current confirmation packet returned by the client is not the confirmation packet corresponding to the data packet sent by the client or the target confirmation packet returned by the client is not received within the preset detection time, start from the end of the video stream according to the target ratio and gradually move forward to discard the predicted frames until the number of discarded predicted frames reaches the preset threshold, and then send the adjusted video stream to the client at the target rate.

[0087] If the target confirmation packet is not received within this preset time, the server believes that the current data packet may be lost or the client fails to process the data packet in time, or the current confirmation packet returned by the client is not the confirmation packet corresponding to the data packet sent by the client, then the server starts to advance from the end of the video stream (that is, the most recent data frame) and gradually discards predicted frames (such as P frames). The number of discards is determined according to a target ratio. This ratio may be dynamically adjusted based on factors such as network conditions, packet loss rate, and client buffering capacity. The discarding process continues until the number of discarded predicted frames reaches a preset threshold.

[0088] Then, after discarding the predicted frames, the server resends the adjusted video stream to the client at the target rate. This target rate may also be adjusted according to the network conditions to ensure smooth transmission of the video stream.

[0089] Please refer to Figure 4 , Figure 4 It is a schematic diagram of a discarding strategy provided according to an embodiment of the present invention.

[0090] Check whether the nack signal is received normally. If the nack signal is not received, according to the received ACK ratio, in the completed video frame to be sent, the corresponding P frame is discarded from the end according to the ratio, and the I frame, B frame and audio frame are retained. This reduces the amount of transmitted data so that the data can be delivered to the end user quickly. For example, through the configuration item, adjust the corresponding discard ratio, for example, if the NACK response is xxs late, discard 10% of the P frames.

[0091] The data transmission method in a weak network scenario provided by this embodiment can ensure that the client has correctly received the data packet by detecting the confirmation packet. If the confirmation packet is not received or the confirmation packet does not match, the system can detect and take corresponding measures in time to avoid data loss or transmission errors. When it is detected that the client fails to return the target confirmation packet within the preset time, the system can gradually move forward from the end of the video stream and discard the predicted frame. This method helps to reduce video freezes or unsmooth playback caused by network delays or packet loss, and improves user experience.

[0092] In this embodiment, a data transmission device in a weak network scenario is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0093] This embodiment provides a data transmission device in a weak network scenario, such as Figure 5 As shown, including:

[0094] The acquisition module 501 is used to acquire the cached data and the latest frame in the cached data; wherein the latest frame is the data frame to be sent;

[0095] A determination module 502, configured to determine a target key frame from cached data according to the type of the latest frame;

[0096] A changing module 503, used to change the video timestamps between the target key frame and the latest frame to the video timestamps corresponding to the target key frame, so as to obtain a modified video frame;

[0097] The sending module 504 is used to encapsulate the modified video frames into a video stream, and send the video stream to the client at a target rate.

[0098] In one possible implementation, the determination module 502 includes: an acquisition unit, which is used to obtain a preset cache time, a decoding time of the bidirectional prediction frame, and a display time of the decoded bidirectional prediction frame if the type of the latest frame is a bidirectional prediction frame; a first determination unit, which is used to obtain a first target time according to the display time of the decoded bidirectional prediction frame, the decoding time of the bidirectional prediction frame, and the preset cache time; a second determination unit, which is used to traverse the cache data and determine a first target data frame that differs from the latest frame by the first target time; wherein, if the first target data frame is a key frame, the key frame is used as the target key frame.

[0099] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0100] The data transmission in the weak network scenario in this embodiment is presented in the form of a functional unit, where the functional unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0101] The embodiment of the present invention also provides a computer device having the above Figure 5 The data sending device in the weak network scenario is shown.

[0102] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0103] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0104] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0105] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0106] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0107] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0108] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0109] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0110] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A data transmission method in a weak network scenario, characterized in that: The method comprises: Acquire cached data and the latest frame in the cached data; wherein the latest frame is a data frame to be sent; Determining a target key frame from the cached data according to the type of the latest frame; The video timestamps between the target key frame and the latest frame are changed to the video timestamps corresponding to the target key frame to obtain a modified video frame; The modified video frames are encapsulated into a video stream, and the video stream is sent to the client at a target rate.

2. The data transmission method in a weak network scenario according to claim 1, characterized in that: The step of determining a target key frame from the cached data according to the type of the latest frame includes: If the type of the latest frame is a bidirectional prediction frame, obtaining a preset cache time, a decoding time of the bidirectional prediction frame, and a display time of the decoded bidirectional prediction frame; Obtaining a first target time according to the display time of the decoded bidirectional prediction frame, the decoding time of the bidirectional prediction frame and the preset cache time; The cached data is traversed to determine a first target data frame that is different from the latest frame by a first target time; wherein, if the first target data frame is a key frame, the key frame is used as a target key frame.

3. The data transmission method in a weak network scenario according to claim 1, characterized in that: The step of determining a target key frame from the cached data according to the type of the latest frame includes: If the type of the latest frame is a predicted frame, determining a previous data frame of the predicted frame; Obtaining the preset cache time, the display time of the decoded predicted frame, and the display time of the previous data frame of the predicted frame after decoding; Determine a second target time according to the preset cache time, the display time of the decoded predicted frame, and the display time of a previous data frame of the predicted frame after decoding; The cached data is traversed to determine a second target data frame that is different from the data frame by a second target time; wherein, if the second target data frame is a key frame, the key frame is used as a target key frame.

4. The data transmission method in a weak network scenario according to claim 2, characterized in that: The method further comprises: If the first target data frame is not a key frame, continue to traverse the cached data until the determined data frame is a key frame, and use the key frame as the target key frame.

5. The data transmission method in a weak network scenario according to claim 1, characterized in that: Sending the video stream to the client at a target rate includes: Detecting whether the current confirmation packet returned by the client is a confirmation packet corresponding to the current data packet sent to the client; Detecting whether a target confirmation packet returned by the client is not received within a preset time period; wherein the target confirmation packet is a confirmation packet corresponding to the current data packet sent by the client; If the current confirmation packet returned by the client is not the confirmation packet corresponding to the data packet sent by the client or the target confirmation packet returned by the client is not received within the preset detection time, start from the end of the video stream according to the target ratio and gradually move forward to discard the predicted frames until the number of discarded predicted frames reaches the preset threshold, and then send the adjusted video stream to the client at the target rate.

6. A data transmission device in a weak network scenario, characterized in that: The device comprises: An acquisition module, used to acquire cached data and the latest frame in the cached data; wherein the latest frame is a data frame to be sent; A determination module, configured to determine a target key frame from the cached data according to the type of the latest frame; A changing module, used for changing the video timestamps between the target key frame and the latest frame to the video timestamps corresponding to the target key frame, so as to obtain a modified video frame; The sending module is used to encapsulate the modified video frame into a video stream and send the video stream to the client at a target rate.

7. The data transmission device in a weak network scenario according to claim 6, characterized in that: The determination module comprises: an acquisition unit, configured to acquire, if the type of the latest frame is a bidirectional prediction frame, a preset cache time, a decoding time of the bidirectional prediction frame, and a display time of the decoded bidirectional prediction frame; A first determining unit, configured to obtain a first target time according to the display time of the decoded bidirectional prediction frame, the decoding time of the bidirectional prediction frame and a preset cache time; The second determination unit is used to traverse the cached data and determine a first target data frame that is different from the latest frame by a first target time; wherein, if the first target data frame is a key frame, the key frame is used as a target key frame.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data sending method in a weak network scenario according to any one of claims 1 to 5 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data sending method in a weak network scenario according to any one of claims 1 to 5.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer to execute the data sending method in a weak network scenario according to any one of claims 1 to 5.

Images