A data processing method, device, equipment and readable storage medium

Abstract

The application discloses a data processing method, device and equipment and a readable storage medium. The method comprises the following steps: obtaining target original data packets corresponding to a target data frame; performing redundancy coding on the target original data packets based on a redundancy transmission strategy to obtain redundancy data packets corresponding to the target original data packets; sending the target original data packets and the redundancy data packets to a receiving end; receiving feedback information returned by the receiving end; if the feedback information comprises a real-time packet loss rate for the target original data packets and the redundancy data packets, determining a function state of a redundancy coding function indicated by the redundancy transmission strategy based on the real-time packet loss rate when transmitting remaining data frames; the remaining data frames refer to data frames other than the target data frame and not yet transmitted in media data to which the target data frame belongs. By using the application, the data packet loss rate and the data transmission delay can be reduced in the data transmission process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a data processing method and device, equipment and readable storage medium. BACKGROUND

[0002] With the continuous popularity of the Internet, real-time video has been well developed, and watching real-time video has gradually become an entertainment way in daily life, especially in news release, online education, social network and self-media scenarios.

[0003] Generally, in the transmission process of video data, in order to reduce the adverse effects of video data loss and errors on decoding quality, some error control techniques can be used to improve the reliability of video data transmission on the network, and the error control techniques can include forward error correction (FEC). FEC detects and corrects data errors by generating certain redundant data. Although FEC wastes a certain amount of network bandwidth, the delay is short.

[0004] However, real-time video requires low latency, and in the transmission process of low-latency real-time video, since the logic of FEC is to start FEC if there is a packet loss, FEC can adjust the redundancy according to the packet loss rate. The higher the packet loss rate, the higher the redundancy. The growth of redundancy will also lead to the growth of redundant packets, which is likely to cause a vicious cycle. That is, in the transmission process of low-latency real-time video, the existing FEC strategy is applied, the packet loss rate is still high, and the data transmission delay cannot be well improved. SUMMARY

[0005] The embodiments of the present application provide a data processing method, device, equipment and readable storage medium, which can reduce the data packet loss rate and data transmission delay in the data transmission process.

[0006] The embodiments of the present application provide a data processing method, device, equipment and readable storage medium, which can reduce the data packet loss rate and data transmission delay in the data transmission process.

[0007] Obtaining target original data packets corresponding to a target data frame; the target original data packets refer to data packets obtained after data encoding of the target data frame;

[0008] Based on the redundancy transmission strategy, the target original data packets are redundantly encoded to obtain redundant data packets corresponding to the target original data packets, and the target original data packets and the redundant data packets are sent to a receiving end;

[0009] Receiving feedback information returned by the receiving end;

[0010] If the feedback information comprises a real-time packet loss rate for the target original data packet and the redundant data packet, a function state of a redundancy encoding function indicated by the redundancy transmission strategy is determined based on the real-time packet loss rate when transmitting a remaining data frame; the function state comprises a running state and a closed state; the remaining data frame refers to a data frame other than the target data frame and not transmitted in media data to which the target data frame belongs.

[0011] The embodiment of the present application provides a data processing device, comprising:

[0012] a packet obtaining module, configured to obtain a target original data packet corresponding to a target data frame; the target original data packet refers to a data packet obtained after data encoding is performed on the target data frame;

[0013] a redundancy encoding module, configured to perform redundancy encoding on the target original data packet based on a redundancy transmission strategy to obtain a redundant data packet corresponding to the target original data packet, and send the target original data packet and the redundant data packet to a receiving end;

[0014] a feedback receiving module, configured to receive feedback information returned by the receiving end;

[0015] a function state determining module, configured to, if the feedback information comprises a real-time packet loss rate for the target original data packet and the redundant data packet, determine a function state of a redundancy encoding function indicated by the redundancy transmission strategy based on the real-time packet loss rate when transmitting a remaining data frame; the function state comprises a running state and a closed state; the remaining data frame refers to a data frame other than the target data frame and not transmitted in media data to which the target data frame belongs.

[0016] In one embodiment, the function state determining module comprises:

[0017] a packet loss threshold determining unit, configured to obtain a service scenario type to which media data belongs, and determine a packet loss rate threshold corresponding to the media data according to the service scenario type;

[0018] a target time delay obtaining unit, configured to, if the real-time packet loss rate is greater than a preset lower limit value and the real-time packet loss is greater than the packet loss rate threshold, obtain a target network transmission time delay at an information receiving moment; the information receiving moment refers to a moment when the feedback information is received;

[0019] a function state determining unit, configured to determine the function state of the redundancy encoding function indicated by the redundancy transmission strategy according to the target network transmission time delay;

[0020] The function state determining unit is further configured to, if the real-time packet loss rate is greater than the packet loss rate threshold, determine the function state of the redundancy encoding function indicated by the redundancy transmission strategy as the closed state.

[0021] In one embodiment, the target time delay obtaining unit comprises:

[0022] a set of historical detection time periods is obtained, each of the set of historical detection time periods is used to detect a network transmission delay, and one historical detection time period corresponds to one detected historical network transmission delay;

[0023] a target time period obtaining sub-unit is configured to obtain, from the set of historical detection time periods, a target historical detection time period to which the information receiving moment belongs;

[0024] a target delay obtaining sub-unit is configured to determine the historical network transmission delay corresponding to the target historical detection time period as a target network transmission delay at the information receiving moment.

[0025] In one embodiment, the function state determining unit comprises:

[0026] a set of delays obtaining sub-unit is configured to obtain a set of historical network transmission delays, the set of historical network transmission delays comprises N historical network transmission delays, one historical network transmission delay refers to a detected network transmission delay in one historical detection time period, N is a positive integer, and each historical detection time period is earlier than the information receiving moment;

[0027] a curve constructing sub-unit is configured to construct a delay change curve for the set of historical network transmission delays and the target network transmission delay according to a time sequence of the N historical detection time periods and the information receiving moment;

[0028] a function state determining sub-unit is configured to determine a function state of a redundancy encoding function indicated by the redundancy transmission strategy according to a delay change trend indicated by the delay change curve.

[0029] In one embodiment, the function state determining sub-unit is further specifically configured to determine the function state of the redundancy encoding function indicated by the redundancy transmission strategy as a closed state if the delay change trend is an increasing trend.

[0030] The function state determining sub-unit is further specifically configured to determine the function state of the redundancy encoding function indicated by the redundancy transmission strategy as a running state if the delay change trend is a stable fluctuation trend.

[0031] In one embodiment, the frame type of the target data frame is an intra-frame encoding type.

[0032] The redundancy encoding module comprises:

[0033] a frame group obtaining unit is configured to obtain a unit data frame group to which the target data frame belongs based on a first redundancy encoding rule for the intra-frame encoding type indicated by the redundancy transmission strategy, the unit data frame group is composed of M continuous data frames in the media data, and M is a positive integer.

[0034] a time length obtaining unit, configured to obtain a frame group encoding time length corresponding to a unit data frame group, and a strategy maintaining time length corresponding to the redundancy transmission strategy;

[0035] a threshold obtaining unit, configured to obtain a service scenario type to which the media data belongs, and determine a packet loss rate threshold corresponding to the media data according to the service scenario type;

[0036] a redundancy encoding unit, configured to perform redundancy encoding on the original data packet according to the frame group encoding time length, the strategy maintaining time length and the packet loss rate threshold, to obtain a redundancy data packet corresponding to the target original data packet.

[0037] In an embodiment, the time length obtaining unit comprises:

[0038] a data encoding time length obtaining sub-unit, configured to obtain a data encoding time length corresponding to each data frame contained in the unit data frame group, to obtain M data encoding time lengths;

[0039] a time length determining sub-unit, configured to add the M data encoding time lengths to obtain the frame group encoding time length;

[0040] The time length determining sub-unit is further configured to obtain a current time and a strategy starting time of the redundancy transmission strategy.

[0041] The time length determining sub-unit is further configured to determine a difference time length between the current time and the strategy starting time as the strategy maintaining time length.

[0042] In an embodiment, the redundancy encoding unit comprises:

[0043] a redundancy degree determining sub-unit, configured to determine a first redundancy degree corresponding to the target data frame according to the frame group encoding time length, the strategy maintaining time length and the packet loss rate threshold;

[0044] a redundancy packet number determining sub-unit, configured to obtain an original packet number corresponding to the target original data packet, and determine a first redundancy packet number corresponding to the target data frame according to the first redundancy degree and the original packet number;

[0045] a redundancy encoding sub-unit, configured to perform redundancy encoding on the target original data packet according to the first redundancy degree and the first redundancy packet number, to obtain a redundancy data packet corresponding to the target original data packet; a data packet number corresponding to the redundancy data packet is the first redundancy packet number.

[0046] In an embodiment, the frame type of the target data frame is a forward encoding type.

[0047] The redundancy encoding module comprises:

[0048] The reference frame acquisition unit is configured to acquire a unit data frame group to which the target data frame belongs based on a second redundancy coding rule for the forward coding type indicated by the redundancy transmission strategy; the unit data frame group is composed of M continuous data frames in the media data; M is a positive integer;

[0049] The reference frame acquisition unit is further configured to acquire a reference data frame with an intra coding type and a transmitted state in the unit data frame group.

[0050] The reference packet acquisition unit is configured to acquire an original data packet corresponding to the reference data frame, and determine the original data packet corresponding to the reference data frame as a reference data packet.

[0051] The packet coding unit is configured to acquire a first data packet number corresponding to the target original data packet and a second data packet number corresponding to the reference data packet, and perform redundancy coding on the target original data packet according to the first data packet number and the second data packet number to obtain a redundancy data packet corresponding to the target original data packet.

[0052] In an embodiment, the packet coding unit is further configured to acquire a redundancy degree corresponding to the reference data frame, and determine the redundancy degree corresponding to the reference data frame as a reference redundancy degree.

[0053] The packet coding unit is further configured to determine a first candidate redundancy degree corresponding to the target data frame based on the reference redundancy degree, the first data packet number and the second data packet number.

[0054] The packet coding unit is further configured to acquire a second candidate redundancy degree for the forward coding type indicated by the second redundancy coding rule.

[0055] The packet coding unit is further configured to determine a minimum candidate redundancy degree between the first candidate redundancy degree and the second candidate redundancy degree as a second redundancy degree corresponding to the target data frame.

[0056] The packet coding unit is further configured to determine a second redundancy packet number corresponding to the target data frame according to the first data packet number and the second redundancy degree.

[0057] The packet coding unit is further configured to perform redundancy coding on the target original data packet according to the second redundancy degree and the second redundancy packet number to obtain a redundancy data packet corresponding to the target original data packet; a data packet number corresponding to the redundancy data packet is the second redundancy packet number.

[0058] In an embodiment, the data processing apparatus further comprises:

[0059] The original packet obtaining module is configured to obtain a to-be-transmitted original data packet corresponding to the remaining data frame if a function state of a redundancy coding function indicated by the redundancy transmission strategy is a closed state when the remaining data frame is being transmitted, wherein the to-be-transmitted original data packet refers to a data packet obtained after data coding is performed on the remaining data frame.

[0060] The original packet sending module is configured to send the to-be-transmitted original data packet to the receiving end.

[0061] In an embodiment, the data processing apparatus further comprises:

[0062] The retransmission module is configured to obtain a lost data packet from the cache library based on a retransmission request included in the feedback information if the feedback information includes the retransmission request for the lost data packet, and retransmit the lost data packet to the receiving end, wherein the lost data packet refers to an original data packet that is not received by the receiving end in the target original data packet.

[0063] An embodiment of the present application provides a computer device, which comprises a processor and a memory.

[0064] The memory stores a computer program, and the computer program is executed by the processor to enable the processor to execute the method in the embodiments of the present application.

[0065] An embodiment of the present application provides a computer readable storage medium, which stores a computer program. The computer program comprises program instructions, and the program instructions are executed by a processor to execute the method in the embodiments of the present application.

[0066] In an aspect of the present application, a computer program product or a computer program is provided, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to execute the method provided in an aspect of the embodiments of the present application.

[0067] In the embodiments of the present application, when transmitting data frames, for a certain target data frame, a redundant transmission strategy can be used to redundantly encode the original data packet of the target data frame (i.e. the data packet obtained after data encoding of the target data frame), and after obtaining the redundant data packet, the redundant data packet is sent to the receiving end together with the original data packet; in the present application, the feedback information returned by the receiving end can be used to control the redundant transmission strategy of the remaining data frames that have not been transmitted, for example, when the feedback information includes the real-time packet loss rate, the function state of the redundant encoding function of the redundant transmission strategy can be determined based on the real-time packet loss rate when transmitting the remaining data frames, wherein if the function state is a running state, it indicates that the redundant transmission strategy can be used for transmission when transmitting the remaining data frames; if the function state is a closed state, it indicates that the redundant transmission strategy can be closed when transmitting the remaining data frames. It should be understood that the present application can analyze the bandwidth of the transmission link through the real-time packet loss rate (for example, if the real-time packet loss rate is too large, it indicates that the data packet of the transmission link exceeds the bandwidth capacity of the link), and thus the redundant transmission strategy can be dynamically controlled. By controlling the redundant transmission strategy of the data frames, the redundant transmission strategy can be closed in time in the case of excessive redundant data packets (in the case that the data packet exceeds the bandwidth capacity of the transmission link), so as to reduce the packet loss rate; in the case of appropriate redundant data packets, the redundant transmission strategy can be continued to maintain, so as to reduce the retransmission rate of the data packet based on the redundant transmission strategy, and thus reduce the end-to-end delay caused by retransmission. In summary, the present application dynamically controls the redundant transmission strategy of the data frames, which can reduce the data packet loss rate and reduce the data transmission delay in the transmission process of the data frames. BRIEF DESCRIPTION OF DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0069] Figure 1 is a network architecture diagram provided by an embodiment of the present application;

[0070] Figure 2 is a system architecture diagram of low-latency live provided by an embodiment of the present application;

[0071] Figure 3 is a scene schematic diagram for data encoding provided by an embodiment of the present application;

[0072] Figure 4 is a flow schematic diagram of a data processing method provided by an embodiment of the present application;

[0073] Figure 5 is a flowchart of determining a function state of a redundancy coding function provided by an embodiment of the present application;

[0074] Figure 6 is a schematic diagram of drawing a time delay change curve provided by an embodiment of the present application;

[0075] Figure 7 is a flowchart of performing redundancy coding on an original data packet to obtain a redundancy data packet provided by an embodiment of the present application;

[0076] Figure 8 is a flowchart of performing redundancy coding on an original data packet to obtain a redundancy data packet provided by an embodiment of the present application;

[0077] Figure 9 is a structural schematic diagram of a data processing apparatus provided by an embodiment of the present application;

[0078] Figure 10 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0079] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative work fall within the protection scope of the present application.

[0080] Please refer to Figure 1 , Figure 1 is a structural schematic diagram of a network architecture provided by an embodiment of the present application. As shown in Figure 1 , the network architecture can include a service server 1000 and a terminal device cluster (i.e., a terminal device cluster). The terminal device cluster can include one or more terminal devices, and the number of terminal devices will not be limited here. As shown in Figure 1 , the plurality of terminal devices can specifically include a terminal device 100a, a terminal device 100b, a terminal device 100c, …, and a terminal device 100n. As shown in Figure 1As shown, the terminal device 100a, the terminal device 100b, the terminal device 100c, …, the terminal device 100n can be respectively connected with the service server 1000 in a network, so that each terminal device can interact with the service server 1000 through the network connection. The network connection herein is not limited to the connection mode, and can be directly or indirectly connected through wired communication, directly or indirectly connected through wireless communication, or connected through other modes, which are not limited herein.

[0081] Each terminal device can be integrated with a target application, which can interact with the service server 1000 when running in each terminal device. Figure 1 The target application can include an application with the function of displaying data information such as text, images, audio, and video. The application can include a social application, a multimedia application (e.g., a video application), an entertainment application (e.g., a game application), an educational application, a live broadcast application, and other applications with media data encoding functions (e.g., video encoding functions). The application can be a standalone application or an embedded sub-application integrated in an application (e.g., a social application, an educational application, and a multimedia application), which is not limited herein.

[0082] For ease of understanding, the embodiments of the present application can select one terminal device from the plurality of terminal devices as a target terminal device. For example, the embodiments of the present application can select the terminal device 100a as the target terminal device, which can be integrated with a target application with a video encoding function. At this time, the target terminal device can realize data interaction between the target application and the service server 1000 through a corresponding service data platform. Figure 1 Figure 1

[0083] It should be understood that the computer device (e.g., the terminal device 100a and the service server 1000) with a media data encoding function (e.g., a video encoding function) in the embodiments of the present application can realize data encoding and data transmission of multimedia data (e.g., video data) through cloud technology. The cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network to realize data calculation, storage, processing, and sharing in a wide area network or a local area network.

[0084] ​​Cloud technology can be a general term for network technology, information technology, integration technology, management platform technology, application technology, etc., can form a resource pool, on-demand use, flexible and convenient. Cloud computing technology will become an important support. The background service of the technical network system needs a large amount of computing and storage resources, such as video websites, picture websites and more portals. With the high development and application of the Internet industry, every item in the future may have its own identification mark and needs to be transmitted to the background system for logical processing. Different levels of data will be processed separately, and various industry data will need strong system support, which can only be realized through cloud computing.

[0085] For example, the data processing method provided by the embodiment of the present application can be applied to video watching scenarios, video call scenarios, video transmission scenarios, cloud conference scenarios, live broadcast scenarios and other high-resolution and high-frame-rate scenarios. Among them, cloud conference is an efficient, convenient and low-cost conference form based on cloud computing technology. Users only need to use the Internet interface and simple and easy-to-use operations to quickly and efficiently share voice, data files and video with teams and customers around the world, and the complex technology of data transmission and processing in the conference is operated by cloud conference service providers. At present, domestic cloud conferences mainly focus on SaaS (Software as a Service) mode as the main service content, including telephone, network, video and other service forms, and video conferencing based on cloud computing is called cloud conference. In the era of cloud conference, data transmission, processing and storage are all handled by computer resources of video conference manufacturers, and users no longer need to purchase expensive hardware and install cumbersome software. They only need to open a browser and log in to the corresponding interface to conduct efficient remote conferences. The cloud conference system supports multi-server dynamic cluster deployment and provides multiple high-performance servers, greatly improving the stability, security and availability of the conference. In recent years, video conferencing has been widely welcomed by many users because it can greatly improve communication efficiency, continuously reduce communication costs, and upgrade internal management levels. It has been widely used in transportation, finance, operators, education, enterprises and other fields. There is no doubt that video conferencing using cloud computing has stronger appeal in convenience, speed and ease of use, and will certainly stimulate a new high tide of video conference applications.

[0086] It should be understood that the computer device with media data encoding function (for example, the terminal device 100a with video encoding function) can perform encoding processing on the media data through the media data encoder (for example, the video encoder) to obtain the data stream corresponding to the media data (for example, the video stream corresponding to the video data), thereby improving the transmission efficiency of the media data. Wherein, when the media data encoder is a video encoder, the video encoder can be an AV1 video encoder, an H.266 video encoder, an AVS3 video encoder, etc., which will not be exemplified one by one here. Wherein, the video compression standard of the AV1 video encoder is the first generation video encoding standard developed by the Alliance for Open Media (AOM).

[0087] Wherein, for media data, the redundancy transmission strategy can be enabled by default when transmitting the media data, and the media data can be transmitted based on the redundancy transmission strategy. Wherein, the redundancy transmission strategy can be a forward error correction technology (FEC), and the sender (such as a service server) can generate a corresponding number of redundant packets (which can be referred to as redundant data packets) by XOR from the original data packets according to the configured redundancy, and then can send the original data packets and the redundant data packets to the receiver (such as a terminal device that will play the media data, etc.). After receiving the relevant data packets, even if there is a loss, but in the case that the number of lost data packets is less than or equal to the number of redundant data packets, the receiver can still recover the lost data packets by XOR from the received data packets, so as to obtain all the original data packets. In the present embodiment, the media data encoder can perform encoding processing on a certain data frame (which can be referred to as a target data frame) of the media data to obtain the encoding data packets (which can be referred to as original data packets, and can be understood as the data stream corresponding to the target data frame) corresponding to the target data frame. After obtaining the encoding data packets (i.e. original data packets) corresponding to the target data frame, the original data packets are subjected to redundant encoding processing based on the redundancy transmission strategy to obtain redundant data packets. Subsequently, the original data packets and the redundant data packets can be sent to the receiver, and the receiver can return a feedback information to the sender. In the feedback information, the real packet loss rate of the original data packets and the redundant data packets is included. In the present embodiment, when transmitting the remaining data frames (i.e. the data frames that have not been transmitted in the media data except the target data frame), the function state of the redundant encoding function of the redundancy transmission strategy can be determined based on the real packet loss rate (i.e. whether to continue to use the redundancy transmission strategy to transmit the remaining data frames). The specific implementation of determining the function state of the redundant encoding function of the redundancy transmission strategy based on the real packet loss rate can be referred to the description in the subsequent corresponding embodiments. Figure 4 corresponding embodiments.

[0088] It can be understood that the method for dynamically controlling the redundancy transmission strategy of the data frame provided in the embodiments of the present application can be applied in various scenarios, for example, can be applied in a low-latency live streaming scenario. For ease of understanding, the architecture of the low-latency live streaming scenario will be described below, please see Figure 2 , Figure 2 is a system architecture diagram of a low-latency live streaming system provided by the embodiments of the present application. As shown in Figure 2 , the system architecture of the scenario can include a terminal device 100a (corresponding to a sending end), a service server, and a terminal device 100b (corresponding to a receiving end).

[0089] It can be understood that in the low-latency live streaming scenario, the anchor uplink collects video stream data through a live terminal (such as the terminal device 100a), and then the live terminal can quantize, encode, and encapsulate the collected video through a video encoding device to obtain a video code stream. Subsequently, the terminal device 100a can send the video code stream to an uplink access module in the service server. The terminal device 100a can be configured with an audio device and a video device, and the terminal device 100a can be a computer or a smart terminal device configured with a camera and a sound pickup device. The terminal device 100a can upload the processed video stream data to the uplink access module using a real-time messaging protocol (RTMP), or can upload the processed video data using other methods, for example, can upload the processed video data to the live uplink access module using a WebRTC method, a GB.28181 method, and a private sdk protocol streaming method, etc. Subsequently, the live uplink access module can send the video code stream to a media data processing module in the service server, and the media data processing module can decode and re-encode (transcode) the original data packet according to the requirements of the downlink audio and video format or code rate, to obtain a transcoded video stream (which can be referred to as an original data packet). Subsequently, the transcoded data can be distributed to each content delivery network (CDN) based on a transmission protocol such as a user datagram protocol (UDP) transmission protocol in a downlink distribution module. For the terminal device 100b, the corresponding user can download and play the live video stream data of the corresponding format or code rate from the nearest content delivery network CDN according to the user demand, so as to enable the corresponding user to watch the live video.

[0090] It can be understood that the service server (such as the media processing module) in the present application can decode and re-encode the received video code stream to obtain the encoded data packet (which can be referred to as the original data packet), and the service server can perform redundancy coding processing on the original data packet to obtain the redundancy data packet, and then send the redundancy data packet together with the original data packet to the terminal device 100b. It should be understood that the present application can dynamically control the redundancy transmission mode of the video frame when transmitting the video data between the sending end (such as the service server) and the receiving end (such as the terminal device), thereby reducing the packet loss rate and the transmission delay. Alternatively, the terminal device 100a can also act as a sending end, and the service server can act as a receiving end. The video code stream obtained by the terminal device by encoding the collected video data can also be referred to as an original data packet. The terminal device 100a can also perform redundancy processing on the original data packet to obtain a redundancy data packet, and then send the redundancy data packet together with the original data packet to the service server.

[0091] Among them, it can be understood that the method provided by the embodiments of the present application can be executed by a computer device, which includes but is not limited to a terminal device or a service server. Among them, the service server can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms.

[0092] Optionally, it can be understood that the above computer device (such as the above business server 1000, terminal device 100a, terminal device 100b, etc.) can be a node in a distributed system, wherein the distributed system can be a blockchain system, which can be a distributed system formed by the plurality of nodes connected through network communication. Among them, nodes can form a peer-to-peer (P2P, Peer To Peer) network, and the P2P protocol is an application layer protocol running on the transmission control protocol (TCP, Transmission Control Protocol) protocol. In the distributed system, any form of computer device, such as a business server, a terminal device, and other electronic devices, can become a node in the blockchain system by joining the peer-to-peer network. For ease of understanding, the concept of blockchain will be described below: blockchain is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm, etc. computer technology, mainly used for arranging data in chronological order, and encrypting into a ledger, so that it cannot be tampered with and forged, while the data can be verified, stored and updated. When the computer device is a blockchain node, due to the tamper-proof and anti-forgery properties of the blockchain, the data (such as media data, raw data packets, redundant data packets, etc.) in the present application can have authenticity and security, so that the results obtained after related data processing based on these data are more reliable.

[0093] It should be noted that in the specific embodiments of the present application, the data related to user information, user data (such as video data containing users) need to be authorized by the user before being obtained. That is, when the above embodiments of the present application are applied to specific products or technologies, the user's permission or consent needs to be obtained, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0094] The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, intelligent transportation, assisted driving, etc. For ease of understanding, please refer to Figure 3 , Figure 3 is a scene diagram provided by an embodiment of the present application for data encoding. Among them, terminal device 2a can be a sending terminal for sending video data (for example, video data 1) shown in Figure 3 , and the user corresponding to the terminal device 2a can be user a. Terminal device 2b can be a receiving terminal for receiving video data (such as video data 1) shown in Figure 3 , and the user corresponding to the terminal device 2b can be user b. Among them, Figure 3The illustrated service server 200 can be a server having a network connection relationship with the terminal device 2a, and the service server 200 can be the above-mentioned Figure 1 The illustrated service server 1000.

[0095] It should be understood that in the video transmission scenario, the terminal device 2a can obtain video data 1 associated with the user a collected by an image collector (for example, a camera). Further, the terminal device 2a can perform encoding processing on the video data 1 through a video encoder (for example, an AV1 video encoder) to generate a video code stream 1 associated with the video data 1. At this time, the terminal device 2a can send the video code stream 1 to the service server 200. The service server 200, upon receiving the video code stream 1, can perform decoding processing on the video code stream to obtain video data (which can be referred to as YUV video data, also referred to as decoded video data, or as to-be-encoded video data) in a pixel image format (also referred to as YUV format), and then the service server 200 can perform encoding processing on the to-be-encoded video data (for example, by encoding the to-be-encoded video data through a video encoder) to obtain a video code stream 2 associated with the video data 1.

[0096] It should be understood that the video code stream can be understood as being composed of encapsulated data packets of each video frame, for example, the video data contains C (C is a positive integer) video frames, and each video frame corresponds to a different data packet obtained by encoding and encapsulating according to a video transmission protocol during transmission. Then, the video code stream corresponding to the video data can be composed of the encapsulated data packets corresponding to the C video frames (sequentially composed of the video code stream according to the time sequence of the video frames). In the present embodiment, the service server performs encoding processing on the video data 1 to obtain a video code stream 2 including data packets, which can be referred to as original data packets. When the service server transmits the video code stream 2 to the terminal device 2b, it can sequentially send the original data packets corresponding to each video frame to the terminal device 2b according to the time sequence of the video frames.

[0097] In the embodiments of the present application, in order to reduce the data retransmission rate and reduce the data transmission delay, the service server can use the redundant transmission strategy in the transmission process, and then dynamically control the redundant transmission strategy (such as continuing to use the redundant transmission strategy for transmission, or closing the redundant transmission strategy in the transmission process of the subsequent video frame) based on the feedback information returned by the receiving end (terminal device 2b). For example, taking the transmission of the first video frame (video frame 1) by the service server as an example, assuming that the original data packet corresponding to the video frame 1 is an original data packet 1, the service server can perform redundant encoding processing on the original data packet 1 based on the redundant transmission strategy through the redundant encoding function of the redundant transmission strategy to obtain the redundant data packet corresponding to the original data packet 1 (which can be referred to as redundant data packet 1). Further, the service server can send the original data packet 1 and the redundant data packet 1 to the terminal device 2b together, and the terminal device 2b can return a feedback information to the service server.

[0098] It can be understood that through the redundant transmission strategy, even if the data packets received by the receiving end (such as terminal device 2b) have packet loss (such as some data packets in the original data packet 1 and the redundant data packet 1 are lost), in the case that the total number of packet loss is less than or equal to the number of data packets of the redundant data packet 1, the receiving end can still recover the lost data packets through a certain way (such as XOR way) (that is, the receiving end can still recover all the original data packets). In the case that the total number of packet loss is greater than the number of data packets of the redundant data packet 1, the receiving end cannot recover the lost data packets. Then when the data packets cannot be recovered, the receiving end can return a retransmission request to the sending end (such as the service server), and the sending end can resend the lost data packets to the receiving end based on the retransmission request. In the present application, in the case that the receiving end can recover the lost data packets, the receiving end can calculate the packet loss rate and return the packet loss rate to the sending end, and the sending end can dynamically control the redundant transmission strategy (for example, continue to run the redundant transmission strategy or close the redundant transmission strategy) based on the packet loss rate.

[0099] For example, as shown in FIG. 2, the service server can send the original data packet 1 and the redundant data packet 1 to the terminal device 2b, and the terminal device 2b can return a feedback information to the service server. The feedback information can include the packet loss rate of the original data packet 1 and the redundant data packet 1. The service server can dynamically control the redundant transmission strategy based on the feedback information. For example, the service server can continue to use the redundant transmission strategy for transmission, or close the redundant transmission strategy in the transmission process of the subsequent video frame. Figure 3As shown, the terminal device 2b can recover all original data packets 1 based on the received data packets, and then the terminal device 2b can obtain a packet loss rate and generate a feedback information based on the packet loss rate and return the feedback information to the service server 200. The service server 200 can determine the function state (including the running state and the closed state) of the redundancy encoding function of the redundancy transmission strategy according to the packet loss rate, that is, the service server 200 can determine whether to continue to use the redundancy transmission mode to transmit the subsequent video frame (such as video frame 2) according to the packet loss rate. The specific implementation of determining the function state of the redundancy encoding function of the redundancy transmission strategy according to the packet loss rate can be referred to the description of the corresponding embodiments below. Figure 4 Figure 3 As shown, assuming that the service server 200 determines to continue to use the redundancy transmission mode to transmit the video frame 2 according to the packet loss rate, the service server 200 can perform redundancy encoding processing on the original data packet 2 of the video frame 2 based on the redundancy transmission strategy through the redundancy encoding function to obtain the redundancy data packet 2 corresponding to the video frame 2, and then transmit the original data packet 2 and the redundancy data packet 2 to the terminal device 2b. As for the video frame 1, the terminal device 2b can return a feedback information, and the service server 200 can determine whether to continue to use the redundancy transmission strategy to transmit the data packet of the subsequent video frame (such as video frame 3) based on the real-time packet loss rate of the original data packet 2 and the redundancy data packet 2 in the feedback information.

[0100] It should be understood that when the data packet is transmitted based on the redundancy transmission mode, the redundancy packet will gradually increase, and the real-time packet loss rate can be used to analyze the bandwidth of the transmission link (for example, if the real-time packet loss rate is too large, it means that the data packet of the transmission link exceeds the bandwidth capacity of the link), and then the redundancy transmission strategy can be dynamically controlled. Through the control of the redundancy transmission strategy on the data frame, the redundancy transmission strategy can be closed in time in the case of excessive redundancy data packets (in the case that the data packet exceeds the bandwidth capacity of the transmission link), so as to reduce the packet loss rate; and the redundancy transmission strategy can be continued to maintain in the case of appropriate redundancy data packets, so that the retransmission rate of the data packet can be reduced based on the redundancy transmission strategy, and the end-to-end delay caused by the retransmission can be reduced.

[0101] Further, please refer to Figure 4 Figure 4 is a flow diagram of a data processing method provided by an embodiment of the present application. The method can be performed by a terminal device (such as the terminal device 2b described above). Figure 1 ​​Any terminal device in the terminal device cluster in the corresponding embodiment, such as terminal device 100a, performs; the method can also be performed by the service server (such as the above Figure 1 The service server 1000 in the corresponding embodiment performs; the method can also be performed by the terminal device and the service server jointly. For example, the method is performed by the service server, as shown in Figure 4 The method flow can at least include the following steps S101-S104:

[0102] Step S101, obtaining a target original data packet corresponding to a target data frame; the target original data packet refers to a data packet obtained after data encoding is performed on the target data frame.

[0103] In this application, a computer device (for example, a terminal device) (such as a video encoding function) can obtain media data (such as video data) collected by an image collector (for example, a camera of the terminal device) in a data transmission scene. Further, the terminal device can perform encoding processing on the media data, thereby obtaining a data code stream (such as a video code stream) corresponding to the media data, and the terminal device can send the data code stream to the service server, and the service server can perform decoding processing on the data code stream to obtain media data with YUV format, which can be called to-be-encoded media data.

[0104] Further, the service server can re-encode the to-be-encoded media data. In re-encoding the to-be-encoded media data, the service server can encode in units of frame groups (a frame group can be referred to as a unit data frame group, for example, a frame group of video data can be referred to as a unit video frame group). For example, for video data, a frame group can be a group of pictures (GOP), and a frame group can include multiple video frames. It should be understood that a frame image can be determined according to a frame type of the frame image based on an encoding parameter setting and a bit rate control strategy. The frame type can include a first type, a second type, and a third type. For example, an intra picture (I-frame) can be referred to as the first type, a bi-directional interpolated prediction frame (B-frame) can be referred to as the second type, and a predictive-frame (P-frame) can be referred to as the third type. A GOP can be understood as an interval between two I-frames. For example, a video frame can include 20 frames, the first frame can be an I-frame, the second to eighth frames can be B-frames, the ninth frame can be a P-frame, the tenth frame can be an I-frame, the eleventh to nineteenth frames can be B-frames, and the twentieth frame can be a P-frame. A GOP (unit video frame group) can be formed with the first frame as a starting frame and the ninth frame as an ending frame, or a GOP (unit video frame group) can be formed with the tenth frame as a starting frame and the twentieth frame as an ending frame.

[0105] It should be noted that, when the GOP and the frame sequence in the GOP are fixed, the frame sequence of each GOP is fixed, for example, the fixed GOP is 120 frames, and an I frame can be generated every 120 frames, and the GOP frame sequence is fixed, for example, I B B P B B P…BB I; when the size of the fixed GOP is fixed but the frame sequence is not fixed, the frame sequence of each GOP is not fixed, for example, the fixed GOP is 120 frames, that is, an I frame is generated every 120 frames, and the GOP frame sequence can be determined according to the picture complexity and the P and B frame generation weight. When the size of the GOP and the frame sequence are not fixed, the GOP can be automatically generated according to the picture texture, motion complexity, and I, P, and B frame generation strategy and weight configuration. For the above division process of the GOP (for example, the first frame is the starting frame and the ninth frame is the ending frame to form a GOP), it is only an example to illustrate how to divide the GOP, and it has no practical reference significance.

[0106] It can be understood that the embodiments of the present application can take the GOP as the granularity to encode each video frame included in the GOP. During encoding, each GOP can be encoded in sequence according to the time sequence of each GOP, and each video frame in a certain GOP can also be encoded in sequence according to the arrangement order of the video frames. For example, a GOP is {I, P1, P2, P3, B1, B2}, and I frame can be encoded first, then P1 frame, then P2 frame, and finally B2 frame.

[0107] As known from the above, when the service server encodes a certain video frame in a certain GOP to obtain a data packet corresponding to the video frame for transmission, the video frame can be referred to as a target video frame, and the data packet obtained by encoding the target video frame is referred to as a target original data packet.

[0108] In step S102, the target original data packet is redundantly encoded based on a redundancy transmission strategy to obtain a redundant data packet corresponding to the target original data packet, and the target original data packet and the redundant data packet are sent to the receiving end.

[0109] In the present application, the redundancy transmission strategy can refer to transmitting original data packets by using forward error correction technology (FEC). The sending end can generate a corresponding number of redundant packets (which can be referred to as redundant data packets or FEC packets) from the original data packets by using XOR according to the configured redundancy, and then send the original data packets and the redundant data packets to the receiving end (such as a terminal device). For the receiving end, if some of the original data packets and the redundant data packets are lost, that is, the receiving end receives not all of the original data packets and the redundant data packets, the receiving end can still recover the lost data packets by using XOR based on the received part of the data packets, as long as the number of lost data packets is less than or equal to the number of redundant data packets. In the present application, the redundancy transmission strategy can be enabled by default for the first video frame in the first GOP, and the first video frame can be redundantly encoded and transmitted based on the redundancy transmission strategy. After the receiving end receives the original data packets and the redundant data packets corresponding to the first video frame, the receiving end can return a feedback information to the sending end. The sending end can determine whether to disable the redundancy transmission strategy (that is, not to perform redundant encoding and only send the original data packets to the receiving end) when transmitting the original data packets of the subsequent video frame (such as the next video frame of the first video frame) based on the feedback information. If the redundancy transmission strategy is still used for the next video frame, the receiving end can return a feedback information after the redundancy transmission, and the sending end can determine whether to continue using the redundancy transmission strategy when transmitting the subsequent video frame (such as the next video frame of the video frame) based on the feedback information. If the redundancy transmission strategy is to be continued, the redundancy transmission strategy can be continued when transmitting the next video frame. If the redundancy transmission strategy is not to be used, the redundancy transmission strategy can not be used for all subsequent video frames of the video frame (including the subsequent untransmitted video frames in the current GOP and the video frames in all subsequent GOPs).

[0110] For the target data frame, for example, the target data frame needs to be transmitted by using a redundant transmission strategy, the target original data packet of the target data frame can be redundantly encoded according to the frame type of the target data frame to obtain the redundant data packet corresponding to the target original data packet. It can be understood that different frame types can correspond to different redundant encoding modes, and then the application can obtain the frame type of the target data frame, and then redundantly encode the target original data packet according to the frame type of the target data frame to obtain the redundant data packet corresponding to the target original data packet. The frame type here can include the first type (I frame, also known as intra coding type), the second type (B frame, also known as bidirectional coding type) and the third type (P frame, also known as forward coding type). It should be noted that in the encoding scheme, if the B frame is an external reference-free encoding frame, the B frame can not be redundantly encoded in the embodiment of the application. For example, in the encoding scheme in the present scheme, the B frame is an external reference-free encoding frame, when a GOP is encoded and transmitted, the B frame can not be transmitted by using the redundant encoding mode, and only the redundant transmission strategy of the I frame and the P frame in a GOP needs to be controlled. When the frame type of the target data frame is the intra coding type, the specific implementation of redundantly encoding the target original data packet according to the frame type of the target data frame to obtain the redundant data packet corresponding to the target original data packet can be referred to the description in the corresponding embodiment below. Figure 7 When the frame type of the target data frame is the forward coding type, the specific implementation of redundantly encoding the target original data packet according to the frame type of the target data frame to obtain the redundant data packet corresponding to the target original data packet can be referred to the description in the corresponding embodiment below. Figure 8 When the frame type of the target data frame is the forward coding type, the specific implementation of redundantly encoding the target original data packet according to the frame type of the target data frame to obtain the redundant data packet corresponding to the target original data packet can be referred to the description in the corresponding embodiment below.

[0111] Step S103, receiving the feedback information returned by the receiving end.

[0112] In the present application, the service server can send the encoded target original data packets and the redundant data packets to the receiving end together. If some data packets (may be some target original data packets or some redundant data packets) are lost during transmission, and the number of the lost data packets is not greater than the number of the redundant data packets, the receiving end can recover the lost data packets by a certain way (such as XOR way), so that all the target original data packets can be obtained. When the receiving end can recover the original data packets, the receiving end can count the real-time packet loss rate (the number of the lost data packets / the total number of the transmitted data packets, wherein the total number of the transmitted data packets is the number of the target original data packets plus the number of the redundant data packets), and the receiving end can return a feedback information containing the real-time packet loss rate to the sending end. If the number of the lost data packets is greater than the number of the redundant data packets, the receiving end can not recover all the target original data packets, and in this case, the receiving end can generate a retransmission request for the lost data packets and return a feedback information containing the retransmission request to the sending end. The sending end (service server) can receive the feedback information for the target original data packets and the redundant data packets returned by the receiving end.

[0113] In step S104, if the feedback information includes the real-time packet loss rate for the target original data packets and the redundant data packets, the function state of the redundant encoding function indicated by the redundant transmission strategy is determined based on the real-time packet loss rate when transmitting the remaining data frames. The function state includes a running state and a closed state. The remaining data frames refer to the data frames in the media data other than the target data frames and not yet transmitted.

[0114] In the present application, when the feedback information includes the real-time packet loss rate for the target original data packets and the redundant data packets, the service server can determine the function state of the redundant encoding function indicated by the redundant transmission strategy according to the real-time packet loss rate. The function state can include a running state and a closed state. When the function state of the redundant encoding function is the running state, it can be indicated that the redundant transmission strategy can be continued to be used for transmission when transmitting the remaining data frames. When the function state of the redundant encoding function is the closed state, it can be indicated that the redundant transmission strategy can not be used for transmission when transmitting the remaining data frames. The remaining data frames can refer to the data frames in the media data other than the target data frames and not yet transmitted, such as the data frames in the GOP group other than the target data frames and not yet transmitted, and the data frames in the remaining GOP groups not yet transmitted. That is, the real-time packet loss rate of the target data frames can be used to determine whether the redundant transmission strategy is continued to be used when transmitting the subsequent data frames.

[0115] It can be understood that the specific implementation of determining the function state of the redundancy coding function indicated by the redundancy transmission strategy based on the real-time packet loss rate can be: the type of the service scenario to which the media data belongs can be obtained, and the packet loss rate threshold corresponding to the media data can be determined according to the type of the service scenario; if the real-time packet loss rate is greater than the preset lower limit value, and the real-time packet loss is greater than the packet loss rate threshold, the target network transmission delay at the information receiving moment can be obtained, and according to the target network transmission delay, the function state of the redundancy coding function indicated by the redundancy transmission strategy can be determined; wherein the information receiving moment can refer to the moment when the feedback information is received; and if the real-time packet loss rate is greater than the packet loss rate threshold, the function state of the redundancy coding function indicated by the redundancy transmission strategy can be determined as the closed state.

[0116] The specific implementation of obtaining the target network transmission delay at the information receiving moment can be: a set of historical detection time periods of network transmission delay can be obtained; each historical detection time period in a set of historical detection time periods is used to detect the network transmission delay; one historical detection time period corresponds to one detected historical network transmission delay; in the set of historical detection time periods, the target historical detection time period to which the information receiving moment belongs can be obtained; then, the historical network transmission delay corresponding to the target historical detection time period can be determined as the target network transmission delay at the information receiving moment.

[0117] It can be understood that when transmitting media data, a maximum allowed redundant packet loss packet loss rate (which can be referred to as a packet loss rate threshold or a maximum allowed packet loss rate) can be configured for each media data, which can be determined by the type of the service scenario to which the media data belongs, that is, different types of service scenarios can correspond to different packet loss rate thresholds. The type of the service scenario here can be determined according to the application type of the target application, for example, the target application installed in the terminal device is an offline short video application, the terminal device obtains media data (such as video data) through the camera when running the offline short video application, and the type of the service scenario to which the media data belongs can be offline short video type. For example, the target application installed in the terminal device is a live application, and the terminal device obtains live video data through the camera when running the live application, and the live video data can be used as media data, and the type of the service scenario to which the media data belongs can be live type. That is, if the media data is obtained through the target application, the target type of the service scenario to which the media data belongs can be the application type of the target application. The present application can dynamically configure different packet loss rate thresholds for different types of service scenarios (for example, the packet loss rate threshold can be configured by a person, such as 20%, 25%, and the like, which will not be illustrated one by one).

[0118] In addition, it should be understood that the application can also preset a lower limit value (which can be referred to as a preset lower limit value, which can be configured by a person, and the preset lower limit value should be less than the packet loss rate threshold) for the packet loss rate of the media data during transmission (for example, the preset lower limit value can be 0, 1%, 1.5%, and the like, which will not be enumerated one by one here). When the real-time packet loss rate is between the preset lower limit value and the packet loss rate threshold, the round-trip time (RTT) corresponding to the data packets (i.e., the target original data packet and the redundant data packet) for transmitting the target video frame can be obtained at this time, and the round-trip time can be used as the target network transmission delay corresponding to the target data frame. According to the target network transmission delay, the function state of the redundancy encoding function can be determined.

[0119] It can be understood that in the embodiments of the present application, the sending end can periodically send a network delay detection signal to the receiving end to detect the network transmission delay between the sending end and the receiving end. For example, the sending end can send a network delay detection signal to the receiving end every 50 ms, and the receiving end can return a delay feedback signal to the sending end when receiving the network delay detection signal. The sending end can obtain the sending time of the network delay detection signal and the receiving time of the delay feedback signal, and the time length between the sending time of the network delay detection signal and the receiving time of the delay feedback signal can be used as a network transmission delay between the sending end and the receiving end. The time period between the time of detecting the network transmission delay twice in the present application can be referred to as a detection time period. For example, the sending end detects the network transmission delay once every 50 ms, and the sending end can send a network delay detection signal to the receiving end at 0 ms, send a network delay detection signal to the receiving end at 50 ms, send a network delay detection signal to the receiving end at 100 ms, and so on (here, the examples will not be listed one by one). Then, 0 ms and 50 ms can form a detection time period (a time period containing 0 ms but not containing 50 ms), that is, the time period is [0 ms, 50 ms). Similarly, 50 ms and 100 ms can form a detection time period (a time period containing 50 ms but not containing 100 ms), that is, the time period is [50 ms, 100 ms). For the detection time period in which the network transmission delay has been detected, it can be referred to as a historical detection time period, and for the detected network transmission delay corresponding to the historical detection time period, it can be referred to as a historical network transmission delay. For example, the sending end detects the network transmission delay once every 50 ms, and detects the network transmission delay between the sending end and the receiving end to be RTT1 at 0 ms by sending a network delay detection signal. The detection time period [0 ms, 50 ms) can be used as a historical detection time period, and the RTT1 can be used as a historical network transmission delay detected by the historical detection time period. If the network transmission delay between the sending end and the receiving end is detected to be RTT2 at 50 ms by sending a network delay detection signal, the detection time period [50 ms, 100 ms) can be used as a historical detection time period, and the RTT2 can be used as a historical network transmission delay detected by the historical detection time period. As known from the above, by periodically detecting the network transmission delay between the sending end and the receiving end, continuous historical detection time periods can be obtained, and these continuous historical detection time periods can form a historical detection time period set, and each historical detection time period can correspond to a detected historical network transmission delay.

[0120] Further, when the sending end receives the feedback information returned by the receiving end for the target original data packet and the redundant data packet, the sending end can obtain the time when the feedback information is received (i.e., the information receiving time) and pull the corresponding network transmission delay at the information receiving time. The process of pulling the delay mainly includes: for example, the sending end detects the network transmission delay every 10 minutes, the time when the network transmission delay is detected for the first time is 11:00 on March 8, 2022 (which can be recorded as 0 min), the time when the network transmission delay is detected for the second time is 11:10 on March 8, 2022 (which can be recorded as 10 min), and so on. Different detection times of the network transmission delay can be obtained, and the two detection times can form a historical detection time period, wherein a historical detection time period can include a previous detection time and exclude a subsequent detection time, for example, 0 min and 10 min can form a historical detection time period [0 min, 10 min). When the information receiving time of the received feedback information is obtained, it can be determined which historical detection time period the information receiving time belongs to, and the historical detection time period to which the information receiving time belongs can be taken as a target historical detection time period. The historical network transmission delay corresponding to the target historical detection time period is the target network transmission delay corresponding to the information receiving time. For example, the information receiving time is 11:25 on March 8, 2022, at this time, the historical detection time period set includes the historical detection time period [0 min, 10 min) (the time period corresponding to the historical detection time period [0 min, 10 min) is from 11:00 on March 8, 2022 to 11:10 on March 8, 2022), the historical detection time period [10 min, 20 min) (the time period corresponding to the historical detection time period [10 min, 20 min) is from 11:10 on March 8, 2022 to 11:20 on March 8, 2022), and the historical detection time period [20 min, 30 min) (the time period corresponding to the historical detection time period [20 min, 30 min) is from 11:20 on March 8, 2022 to 11:30 on March 8, 2022), and the historical detection time period to which the information receiving time 11:25 on March 8, 2022 belongs is the historical detection time period [20 min, 30 min), the historical network transmission delay corresponding to the historical detection time period is RTT3, and the target network transmission delay corresponding to the information receiving time is RTT3.

[0121] Further, after obtaining the target network transmission delay, the transmission strategy of the data packet of the subsequent video frame can be dynamically controlled according to the target network transmission delay (redundant transmission strategy is adopted for transmission; or the redundant transmission mode is closed, and only the original data packet is transmitted). For the specific implementation of dynamically controlling the redundant transmission strategy according to the target network transmission delay, please refer to the subsequentFigure 5 The description in the corresponding embodiment.

[0122] When the real-time packet loss rate is greater than the packet loss rate threshold, it can be characterized that the transmission link of the data packet is not a stable packet loss link, and the data packet has exceeded the link bandwidth capacity. If the redundant transmission strategy is continuously used for transmission, the packet loss situation will become more and more serious, and the receiving end is extremely likely to fail to recover the data packet, thereby causing the retransmission probability to rise, affecting the transmission efficiency, and increasing the end-to-end transmission delay. Therefore, when the real-time packet loss rate is greater than the packet loss rate threshold, the redundant transmission strategy can be closed, that is, the function state of the redundancy coding function is determined to be a closed state, and in the subsequent transmission of the remaining video frames, the redundant transmission strategy is no longer used for transmission.

[0123] That is, if the function state of the redundancy coding function indicated by the redundant transmission strategy is a closed state when transmitting the remaining data frames, the remaining data frames can be obtained when transmitting the remaining data frames. The to-be-transmitted original data packet refers to the data packet obtained after data encoding of the remaining data frames. Then, the to-be-transmitted original data packet can be directly sent to the receiving end (without redundancy coding, but the corresponding original data packet is directly sent to the receiving end). In this way, the redundant data packet can be reduced, and transmission can be performed under the condition of meeting the link bandwidth capacity.

[0124] As can be seen from the above, in the case that the feedback information includes the real-time packet loss rate, the sending end can dynamically control the redundant transmission strategy based on the real-time packet loss rate. If the receiving end cannot recover all the original data packets, a retransmission request for the lost data packets can be generated, and the feedback information including the retransmission request can be returned to the sending end. If the feedback information includes the retransmission request for the lost data packets, the sending end can obtain the lost data packets from the cache library based on the retransmission request, and retransmit the lost data packets to the receiving end. The lost data packet refers to the original data packet that is not received by the receiving end in the target original data packet.

[0125] Optionally, it can be understood that the above is described by taking the service server as the sending end and the terminal device as the receiving end. In actual application, the terminal device that collects media data can also be taken as the sending end, and the service server can also be taken as the receiving end. The terminal device that collects media data can also use the redundant transmission method for transmission when transmitting to the service server, and can also use the method of the present application to dynamically control the redundant transmission strategy.

[0126] In the embodiments of the present application, the real-time packet loss rate and the network transmission delay are used to analyze the bandwidth of the transmission link (for example, if the real-time packet loss rate is too large, it indicates that the data packets of the transmission link exceed the bandwidth capacity of the link), and then the redundant transmission strategy can be dynamically controlled. Through the control of the redundant transmission strategy of the data frame, the redundant transmission strategy can be closed in time in the case of excessive redundant data packets (in the case of data packets exceeding the bandwidth capacity of the transmission link), so as to reduce the packet loss rate. In the case of appropriate redundant data packets, the redundant transmission strategy can be continued to maintain, so that the retransmission rate of the data packets can be reduced based on the redundant transmission strategy, and the end-to-end delay caused by retransmission can be reduced. In summary, the redundant transmission strategy of the data frame can be dynamically controlled to reduce the data packet loss rate and reduce the data transmission delay during the transmission of the data frame.

[0127] As can be seen from the above, when the real-time packet loss rate is between the preset lower limit value and the packet loss rate threshold, the target network transmission delay at the information receiving moment can be obtained, and the function state of the redundant encoding function indicated by the redundant transmission strategy (that is, dynamically controlling whether to use the redundant transmission strategy or not) can be determined according to the target network transmission delay. For ease of understanding, the specific implementation manner of determining the function state of the redundant encoding function indicated by the redundant transmission strategy according to the target network transmission delay will be described below. Please see Figure 5 , Figure 5 is a flowchart of determining the function state of the redundant encoding function provided by the embodiments of the present application. The flowchart can correspond to the flow of determining the function state of the redundant encoding function indicated by the redundant transmission strategy according to the target network transmission delay in the above Figure 4 As shown in Figure 5 , the flowchart can at least include the following steps S201-S203:

[0128] Step S201, obtaining a set of historical network transmission delays; the set of historical network transmission delays includes N historical network transmission delays, one historical network transmission delay refers to the detected network transmission delay in one historical detection time period; N is a positive integer; each historical detection time period is earlier than the information receiving moment.

[0129] Specifically, the set of historical network transmission delays can be obtained by the above Figure 4According to the above, the sending end can periodically detect the network transmission delay between the sending end and the receiving end. The time instants of detecting the delay before and after can constitute a detection time period. For the detection time period in which the network transmission delay has been detected, it can be referred to as a historical detection time period. For the detected network transmission delay corresponding to the historical detection time period, it can be referred to as a historical network transmission delay. Then, the historical network transmission delay set is obtained, that is, a set composed of the historical network transmission delays corresponding to the historical detection time periods earlier than the target historical detection time period (the historical detection time period to which the information receiving time instant belongs) in time. For example, in the above example Figure 4 In the example, the historical detection time period set includes [0min, 10min), [10min, 20min) and [20min, 30min), and the [20min, 30min) is the target historical detection time period. Then, the historical network transmission delay corresponding to the [0min, 10min) (assuming RTT1) and the historical network transmission delay corresponding to the [10min, 20min) (assuming RTT2) can constitute the historical network transmission delay set.

[0130] In step S202, a delay change curve for the historical network transmission delay set and the target network transmission delay is constructed according to the time sequence of the N historical detection time periods and the information receiving time instant.

[0131] Specifically, the historical detection time period can be taken as a time coordinate axis, and the network transmission delay can be taken as a delay coordinate axis. According to the time coordinate axis and the delay coordinate axis, a coordinate system can be constructed. According to the N historical detection time periods and the historical network transmission delays, a delay change curve can be drawn in the coordinate system.

[0132] For ease of understanding, please refer to Figure 6 , Figure 6 is a schematic diagram of drawing a delay change curve provided by an embodiment of the present application. As shown in Figure 6As shown, the X axis can be used to represent the time coordinate axis, the Y axis can be used to represent the time delay coordinate axis, the first historical detection time period is [0ms, 50ms), the second historical detection time period is [50ms, 100ms), the third historical detection time period is [100ms, 150ms), and the fourth historical detection time period is [150ms, 200ms). The historical network transmission time delay corresponding to the first historical detection time period is 50, the historical network transmission time delay corresponding to the second historical detection time period is 65, the historical network transmission time delay corresponding to the third historical detection time period is 67, and the historical network transmission time delay corresponding to the fourth historical detection time period is 60. Assuming that the target historical detection time period in which the information receiving moment is located is the fourth historical detection time period, according to the historical detection time period and the historical network transmission time delay corresponding thereto, a curve as shown in the figure can be constructed in the coordinate system, which can be referred to as a time delay change curve. Figure 6

[0133] In step S203, the function state of the redundancy encoding function indicated by the redundancy transmission strategy is determined according to the time delay change trend indicated by the time delay change curve.

[0134] Specifically, the function state of the redundancy encoding function indicated by the redundancy transmission strategy can be determined according to the time delay change trend indicated by the time delay change curve. For example, if the time delay change trend is an increasing trend, the function state of the redundancy encoding function indicated by the redundancy transmission strategy can be determined as a closed state; and if the time delay change trend is a stable fluctuation trend, the function state of the redundancy encoding function indicated by the redundancy transmission strategy can be determined as a running state.

[0135] It can be understood that when the real-time packet loss rate is between the preset lower limit value and the packet loss rate threshold, if the network transmission time delay continues to increase (the time delay change trend is an increasing trend), it can be represented that the transmission link is not a stable packet loss link, and actually the data packet exceeds the bandwidth capacity. At this time, in order to reduce the transmission time delay, the redundancy transmission mode can be closed, and the redundancy transmission strategy is not used for subsequent transmission, that is, the function state of the redundancy encoding function is determined as a closed state; and if the network transmission time delay does not fluctuate with the continuous running of the redundancy transmission mode, and is always in a stable fluctuation state (the fluctuation is less than a preset value, such as less than 10ms. That is, the time delay change trend is a stable fluctuation trend), it can be indicated that the transmission link is a relatively stable packet loss link, and the data packet does not exceed the link bandwidth capacity. The redundancy transmission strategy (that is, the function state of the redundancy encoding function is determined as a running state) can be continuously run to reduce the link retransmission and further reduce the end-to-end transmission delay.

[0136] ​In the embodiments of the present application, the packet loss rate and network transmission delay can be implemented to dynamically and real-timely control the transmission mode (such as using redundant transmission or not using redundant transmission) of the video frame. On the one hand, in the scene of transmitting data (such as in the scene of transmitting data in low-delay live broadcast), the bandwidth can be made more smooth, so that the network congestion control is more accurate. When the network link is congested, the redundant transmission can be closed in time to reduce the packet loss rate. On the other hand, in the case that the link capacity is not exceeded, the redundant transmission strategy can be used to reduce the NACK (backward error correction, NACK is also a notification technology. When a message is not received, it is notified to the sender that “the message is not received”, that is, it is notified that the message is not delivered. After the receiving end detects data packet loss, the NACK message is sent to the sending end. According to the sequence number in the NACK message, the corresponding data packet is found in the sending buffer and is retransmitted to the receiving end) retransmission waiting case, so that the end-to-end delay in the data transmission scene can be lower.

[0137] According to the above Figure 4 The embodiments corresponding to the above Figure 7 , Figure 7 are a flowchart of a process of performing redundant encoding on an original data packet to obtain a redundant data packet provided by the embodiments of the present application. As shown in Figure 7 , the process can include at least the following steps S301-S304:

[0138] In step S301, a unit data frame group to which the target data frame belongs is obtained based on a first redundant encoding rule for the intra-frame encoding type indicated by the redundant transmission strategy. The unit data frame group is composed of M continuous data frames in the media data. M is a positive integer.

[0139] Specifically, the present application can configure different redundant encoding rules for different frame types. Here, the redundant encoding rule can refer to the redundancy determination method, that is, different redundancy determination methods can exist for different frame types. For example, a first redundant encoding rule (first redundancy determination method) can be configured for the intra-frame encoding type (I frame). When the frame type of the target data frame is the intra-frame encoding type, the first redundant encoding rule can be obtained. For ease of understanding, please refer to formula (1), which can be the redundancy determination method (i.e. the first redundant encoding rule) configured by the present application for the intra-frame encoding type.

[0140] Formula (1)

[0141] wherein, as shown in formula (1) The strategy maintenance duration can be used to represent a duration from when the one-time redundancy transmission strategy is enabled to now, that is, a strategy maintenance duration corresponding to the redundancy transmission strategy. The frame group encoding duration can be used to represent a size duration for encoding one GOP, that is, a frame group encoding duration corresponding to a unit data frame group where the target data frame group is located. The packet loss rate threshold corresponding to the media data can be used to represent a redundancy determination manner indicated by the formula (1). In order to determine the redundancy corresponding to the target data frame, the unit data frame group to which the target data frame belongs (that is, the GOP where the target data frame is located) needs to be obtained.

[0142] In step S302, the frame group encoding duration corresponding to the unit data frame group and the strategy maintenance duration corresponding to the redundancy transmission strategy are obtained.

[0143] Specifically, according to the above formula (1), in order to determine the redundancy corresponding to the target data frame, the frame group encoding duration and the strategy maintenance duration need to be obtained. In the present application, the specific implementation manner for obtaining the frame group encoding duration corresponding to the unit data frame group and the strategy maintenance duration corresponding to the redundancy transmission strategy can be: the data encoding duration corresponding to each data frame included in the unit data frame group is obtained, and M data encoding durations are obtained; the M data encoding durations are added to obtain the frame group encoding duration; the current time and the strategy enabling time of the redundancy transmission strategy are obtained; and the difference duration between the current time and the strategy enabling time is determined as the strategy maintenance duration. That is, the frame group encoding duration can be the encoding duration corresponding to one GOP, and the strategy maintenance duration can be the duration from the one-time strategy enabling time to the current time.

[0144] In step S303, the service scenario type to which the media data belongs is obtained, and the packet loss rate threshold corresponding to the media data is determined according to the service scenario type.

[0145] Specifically, according to the above formula (1), in order to determine the redundancy corresponding to the target data frame, the packet loss rate threshold needs to be obtained. For the specific manner of obtaining the packet loss rate threshold, reference can be made to the description in the above Figure 4 embodiments, which will not be described herein again.

[0146] In step S304, the original data packet is redundantly encoded according to the frame group encoding duration, the strategy maintenance duration and the packet loss rate threshold, to obtain the redundancy data packet corresponding to the target original data packet.

[0147] Specifically, for the frame group encoding duration, the policy maintenance duration and the packet loss rate threshold, the specific implementation manner of performing redundant encoding on the original data packet to obtain the redundant data packet corresponding to the target original data packet can be: the first redundancy corresponding to the target data frame can be determined according to the frame group encoding duration, the policy maintenance duration and the packet loss rate threshold; then, the original packet quantity corresponding to the target original data packet can be obtained, and the first redundant packet quantity corresponding to the target data frame can be determined according to the first redundancy and the original packet quantity; the target original data packet can be redundantly encoded to obtain the redundant data packet corresponding to the target original data packet according to the first redundancy and the first redundant packet quantity; wherein the data packet quantity corresponding to the redundant data packet is the first redundant packet quantity.

[0148] For the convenience of understanding the above process, please refer to formula (2) for the definition of redundancy:

[0149] Formula (2)

[0150] As shown in formula (2) can be used to represent the redundant data packet, can be used to represent the original data packet. That is, the ratio between the total number of redundant data packets and the total number of redundant data packets and original data packets is the redundancy.

[0151] Therefore, according to the frame group encoding duration, the policy maintenance duration and the packet loss rate threshold and the above formula (1), the redundancy corresponding to the target data frame (which can be called the first redundancy) can be determined. As can be known from the above formula (1), the first redundancy can be at most the packet loss rate threshold. After determining the first redundancy, the packet quantity corresponding to the target original data packet (which can be called the original packet quantity) can be obtained. As can be known from the above formula (2), based on the original packet quantity (W) corresponding to the target original data packet and the first redundancy, the quantity of redundant data packets (Q, which can be called the first redundant packet quantity) corresponding to the target data frame can be determined. Further, according to the first redundancy and the first redundant packet quantity, the target original data packet can be redundantly encoded to obtain the corresponding redundant data packet.

[0152] In the embodiments of the present application, different redundant encoding strategies can be adopted for different frame types. By dynamically and real-timely controlling the FEC redundant transmission of I frames (timely closing the FEC redundant transmission strategy or continuing to adopt the FEC redundant transmission strategy), the situation of network congestion and packet loss caused by inaccurate network bandwidth evaluation due to the code rate fluctuation of I frames can be well improved.

[0153] Through the above Figure 4According to the corresponding embodiment, the target original data packet can be redundantly encoded according to the frame type of the target data frame. Here, the frame type of the target data frame is taken as an example of the forward encoding type, and the specific implementation of redundantly encoding the target original data packet according to the frame type of the target data frame to obtain the redundant data packet corresponding to the target original data packet is described. For ease of understanding, please refer to Figure 8 , Figure 8 is a flowchart provided by an embodiment of the present application for redundantly encoding an original data packet to obtain a redundant data packet. As shown in Figure 8 , the flowchart can include at least the following steps S401-S404:

[0154] In step S401, a unit data frame group to which the target data frame belongs is obtained based on the second redundancy encoding rule for the forward encoding type indicated by the redundancy transmission strategy. The unit data frame group is composed of M consecutive data frames in the media data, and M is a positive integer.

[0155] Specifically, the present application can configure different redundancy encoding rules for different frame types. Here, the redundancy encoding rule can refer to a redundancy determination method, that is, different redundancy determination methods can exist for different frame types. For example, the second redundancy encoding rule (second redundancy determination method) can be configured for the forward encoding type (i.e., P frame or third type), so that when the frame type of the target data frame is the forward encoding type, the second redundancy encoding rule can be obtained. For ease of understanding, please refer to formula (3), which is the redundancy determination method (i.e., the second redundancy encoding rule) configured by the present application for the forward encoding type.

[0156] Formula (3)

[0157] As shown in formula (3), the can be used to represent the total number of original data packets corresponding to the target data frame; can be used to represent the total number of original data packets corresponding to the I frame in the GOP in which the target data frame is located; can be used to represent the redundancy of the I frame in the GOP in which the target data frame is located. Through the redundancy determination method indicated by formula (1), the unit data frame group (i.e., the GOP in which the target data frame is located) to which the target data frame belongs needs to be obtained.

[0158] In step S402, a reference data frame with a frame type of intra-frame encoding type and a transmission state of transmitted state in the unit data frame group is obtained.

[0159] Specifically, it should be understood that one GOP can refer to a sequence of data frames starting with an I frame, the GOP can include I frames, P frames and B frames (in the coding scheme, B frames are not externally referenced coding frames, and the redundant transmission strategy can not be used), and for one GOP, the I frame is first encoded and transmitted, and then the P frame is encoded and transmitted. The reference data frame here refers to the I frame in the GOP.

[0160] In step S403, the original data packet corresponding to the reference data frame is obtained, and the original data packet corresponding to the reference data frame is determined as the reference data packet.

[0161] Specifically, it can be known from the above formula (3) that in order to determine the redundancy corresponding to the target data frame, the total number of original data packets corresponding to the I frame in the GOP needs to be obtained, so the original data packet corresponding to the reference data frame needs to be obtained, and thus the total number of packets corresponding thereto is obtained.

[0162] In step S404, the first data packet quantity corresponding to the target original data packet and the second data packet quantity corresponding to the reference data packet are obtained, and the target original data packet is redundantly encoded according to the first data packet quantity and the second data packet quantity, to obtain the redundant data packet corresponding to the target original data packet.

[0163] Specifically, for the specific implementation of redundantly encoding the target original data packet according to the first data packet quantity and the second data packet quantity to obtain the redundant data packet corresponding to the target original data packet, the redundancy corresponding to the reference data frame can be obtained, and then the redundancy corresponding to the reference data frame can be determined as the reference redundancy; based on the reference redundancy, the first data packet quantity and the second data packet quantity, the first candidate redundancy corresponding to the target data frame can be determined; the second candidate redundancy for the forward encoding type (or the third type) indicated by the second redundancy encoding rule can be obtained; the minimum candidate redundancy between the first candidate redundancy and the second candidate redundancy can be determined as the second redundancy corresponding to the target data frame; the second redundant packet quantity corresponding to the target data frame can be determined according to the first data packet quantity and the second redundancy; the target original data packet can be redundantly encoded according to the second redundancy and the second redundant packet quantity, to obtain the redundant data packet corresponding to the target original data packet; and the data packet quantity corresponding to the redundant data packet is the second redundant packet quantity.

[0164] It can be understood that when the frame type of the target data frame is the forward encoding type and the redundant transmission strategy is used to transmit it, it can be indicated that the I frame before the P frame is also transmitted by the redundant transmission method. Since the I frame in the GOP has been transmitted by the redundant transmission method at this time, the redundancy KI of the I frame when the redundant transmission is performed can be obtained (the specific determination of KI can be referred to the aboveFigure 7 (as described in the corresponding embodiment), and the total number of original data packets of the I-frame (i.e., From the above formula (3), it can be seen that after obtaining the number of the first data packets, the number of the second data packets, and the reference redundancy (KI), the formula in formula (3) can be used to determine the redundancy. ”, calculate a value, which can be called the first candidate redundancy; through the above formula (3), it can be seen that the minimum value between the value and 1 / 2 needs to be selected as the redundancy corresponding to the target data frame. Then the parameter 1 / 2 can be called the second candidate redundancy. The minimum value between the first candidate redundancy and the second candidate redundancy (can be called the minimum candidate redundancy) is the redundancy corresponding to the target data frame (can be called the second redundancy).

[0165] After determining the second redundancy, the number of packets corresponding to the target original data packet (i.e., the number of first data packets) can be obtained. According to the above formula (2), based on the number of original packets corresponding to the target original data packet (i.e., W) and the second redundancy, the number of redundant data packets corresponding to the target data frame (i.e., Q, which can be called the number of second redundant packets) can be determined. Furthermore, based on the second redundancy and the number of second redundant packets, the target original data packet can be subjected to redundancy encoding processing to obtain the corresponding redundant data packets.

[0166] In this embodiment, since a GOP consists of I, P, and B frames, the initial I frame within a GOP will have a high bitrate, while the P and B frames will have relatively low bitrate. This uneven bandwidth usage during transmission can easily lead to inaccurate bandwidth assessment and network congestion. This application, however, can employ different redundancy coding strategies for different frame types. By dynamically controlling the FEC redundancy transmission of I / P frames in real time (either disabling or continuing to use the FEC redundancy transmission strategy), it can improve the situation of inaccurate network bandwidth assessment and packet loss caused by I frame bitrate fluctuations. Furthermore, in data transmission scenarios (such as low-latency live streaming), it can make data bandwidth smoother, resulting in more precise network congestion control, reducing NACK retransmissions, and consequently reducing end-to-end latency caused by retransmission waiting.

[0167] Optionally, it can be understood that in a feasible embodiment, due to the fast transmission speed within a GOP, generally after the corresponding RTT is obtained after the I frame transmission is completed, if it is determined based on the RTT that the FEC redundancy transmission strategy can be continued to be used, then for all the P frames that have not been transmitted after the I frame in the GOP, the FEC redundancy transmission strategy can be directly used for transmission, that is, it is not necessary to determine whether the FEC redundancy transmission strategy is to be used for each P frame in the GOP. When the data frame transmission in the GOP is completed, for the first I frame of the next GOP, the redundancy transmission mode is continued to be used for transmission, and similarly, the real-time packet loss rate and network transmission delay corresponding thereto can be obtained, and based on the real-time packet loss rate and network transmission delay, it is determined whether the P frames in the GOP are to continue to use FEC for transmission. That is, the first I frame in a GOP can be obtained based on the real-time packet loss rate and network transmission delay after transmission, if it is determined based on the real-time packet loss rate and network transmission delay that the FEC redundancy transmission strategy is not to be used for subsequent transmission, then during the transmission of the P frames in the GOP and all subsequent GOPs, the FEC redundancy transmission mode can not be used for transmission; and if it is determined based on the real-time packet loss rate and network transmission delay that the FEC redundancy transmission strategy is to be continued to be used for transmission, then for the P frames in the GOP where the I frame is located, the FEC redundancy transmission can be continued, and for the I frame in the next GOP, the FEC redundancy transmission can also be continued, but after the transmission of the I frame in the next GOP is completed, the real-time packet loss rate and network transmission delay of the next I frame can be obtained again to determine whether the FEC is to be continued to be used for subsequent transmission. In succession, until the transmission of the last GOP of the media data is completed.

[0168] Further, please refer to Figure 9 , Figure 9 is a structural schematic diagram of a data processing device provided by an embodiment of the present application. The data processing device can be a computer program (including program code) running in a computer device, for example, the data processing device is an application software; the data processing device can be used to execute the method shown in Figure 3 . As shown in Figure 9 , the data processing device 1 can include a packet obtaining module 11, a redundancy encoding module 12, a feedback receiving module 13, and a function state determining module 14.

[0169] The packet obtaining module 11 is used for obtaining a target original data packet corresponding to a target data frame; the target original data packet refers to a data packet obtained after data encoding is performed on the target data frame;

[0170] a redundancy coding module 12, configured to perform redundancy coding on the target original data packet based on the redundancy transmission strategy to obtain a redundancy data packet corresponding to the target original data packet, and send the target original data packet and the redundancy data packet to the receiving end;

[0171] a feedback receiving module 13, configured to receive feedback information returned by the receiving end;

[0172] a function state determining module 14, configured to, if the feedback information includes a real-time packet loss rate for the target original data packet and the redundancy data packet, determine a function state of a redundancy coding function indicated by the redundancy transmission strategy based on the real-time packet loss rate when transmitting a remaining data frame; the function state includes a running state and a closed state; the remaining data frame refers to a data frame other than the target data frame and not yet transmitted in the media data to which the target data frame belongs.

[0173] The specific implementation of the packet obtaining module 11, the redundancy coding module 12, the feedback receiving module 13, and the function state determining module 14 can be referred to the descriptions of steps S101-S104 in the above Figure 4 corresponding embodiments, which will not be repeated here.

[0174] In one embodiment, the function state determining module 14 can include a packet loss threshold determining unit 141, a target delay obtaining unit 142, and a function state determining unit 143.

[0175] The packet loss threshold determining unit 141 is configured to obtain a service scenario type to which the media data belongs, and determine a packet loss rate threshold corresponding to the media data according to the service scenario type.

[0176] The target delay obtaining unit 142 is configured to, if the real-time packet loss rate is greater than a preset lower limit value and the real-time packet loss is greater than the packet loss rate threshold, obtain a target network transmission delay at an information receiving moment; the information receiving moment refers to a moment when the feedback information is received.

[0177] The function state determining unit 143 is configured to determine the function state of the redundancy coding function indicated by the redundancy transmission strategy according to the target network transmission delay.

[0178] The function state determining unit 143 is further configured to, if the real-time packet loss rate is greater than the packet loss rate threshold, determine the function state of the redundancy coding function indicated by the redundancy transmission strategy as the closed state.

[0179] The specific implementation of the packet loss threshold determining unit 141, the target delay obtaining unit 142, and the function state determining unit 143 can be referred to the description of step S104 in the above Figure 4 corresponding embodiments, which will not be repeated here.

[0180] In an embodiment, the target time delay obtaining unit 142 can include: a set obtaining sub-unit 1421, a target time period obtaining sub-unit 1422, and a target time delay obtaining sub-unit 1423.

[0181] The set obtaining sub-unit 1421 is configured to obtain a set of historical detection time periods for network transmission time delay; each historical detection time period in the set of historical detection time periods is used for detecting network transmission time delay; one historical detection time period corresponds to one detected historical network transmission time delay.

[0182] The target time period obtaining sub-unit 1422 is configured to obtain, from the set of historical detection time periods, a target historical detection time period to which the information receiving moment belongs.

[0183] The target time delay obtaining sub-unit 1423 is configured to determine the historical network transmission time delay corresponding to the target historical detection time period as the target network transmission time delay at the information receiving moment.

[0184] The specific implementation of the set obtaining sub-unit 1421, the target time period obtaining sub-unit 1422, and the target time delay obtaining sub-unit 1423 can refer to the description of step S104 in the above Figure 4 corresponding embodiment, which will not be described here in detail.

[0185] In an embodiment, the function state determining unit 143 can include: a time delay set obtaining sub-unit 1431, a curve constructing sub-unit 1432, and a function state determining sub-unit 1433.

[0186] The time delay set obtaining sub-unit 1431 is configured to obtain a set of historical network transmission time delays; the set of historical network transmission time delays includes N historical network transmission time delays, one historical network transmission time delay refers to a detected network transmission time delay in one historical detection time period; N is a positive integer; each historical detection time period is earlier than the information receiving moment.

[0187] The curve constructing sub-unit 1432 is configured to construct a time delay change curve for the set of historical network transmission time delays and the target network transmission time delay according to the time sequence of the N historical detection time periods and the information receiving moment.

[0188] The function state determining sub-unit 1433 is configured to determine the function state of the redundant encoding function indicated by the redundant transmission strategy according to the time delay change trend indicated by the time delay change curve.

[0189] The specific implementation of the time delay set obtaining sub-unit 1431, the curve constructing sub-unit 1432, and the function state determining sub-unit 1433 can refer to the description of step S104 in the above Figure 5The description of steps S201-S203 in the corresponding embodiment will not be repeated here.

[0190] In one embodiment, the function state determining sub-unit 1433 is further specifically configured to determine the function state of the redundancy coding function indicated by the redundancy transmission strategy as the closed state if the time delay variation trend is an increasing trend.

[0191] The function state determining sub-unit 1433 is further specifically configured to determine the function state of the redundancy coding function indicated by the redundancy transmission strategy as the running state if the time delay variation trend is a stable fluctuation trend.

[0192] In one embodiment, the frame type of the target data frame is an intra-frame coding type.

[0193] The redundancy coding module 12 can include a frame group obtaining unit 121, a time length obtaining unit 122, a threshold obtaining unit 123, and a redundancy coding unit 124.

[0194] The frame group obtaining unit 121 is configured to obtain a unit data frame group to which a target data frame belongs based on a first redundancy coding rule for the intra-frame coding type indicated by a redundancy transmission strategy; the unit data frame group is composed of M continuous data frames in the media data; M is a positive integer.

[0195] The time length obtaining unit 122 is configured to obtain a frame group coding time corresponding to the unit data frame group and a strategy maintenance time corresponding to the redundancy transmission strategy.

[0196] The threshold obtaining unit 123 is configured to obtain a service scenario type to which the media data belongs, and determine a packet loss rate threshold corresponding to the media data according to the service scenario type.

[0197] The redundancy coding unit 124 is configured to perform redundancy coding on an original data packet according to the frame group coding time, the strategy maintenance time, and the packet loss rate threshold, to obtain a redundancy data packet corresponding to the target original data packet.

[0198] The specific implementation of the frame group obtaining unit 121, the time length obtaining unit 122, the threshold obtaining unit 123, and the redundancy coding unit 124 can be referred to the above description. Figure 7 The description of steps S301-S304 in the corresponding embodiment will not be repeated here.

[0199] In one embodiment, the time length obtaining unit 122 can include a coding time length obtaining sub-unit 1221 and a time length determining sub-unit 1222.

[0200] The encoding duration obtaining subunit 1221 is configured to obtain a data encoding duration corresponding to each data frame included in the unit data frame group, to obtain M data encoding durations.

[0201] The duration determining subunit 1222 is configured to add the M data encoding durations to obtain a frame group encoding duration.

[0202] The duration determining subunit 1222 is further configured to obtain a current time and a policy start time of the redundancy transmission strategy.

[0203] The duration determining subunit 1222 is further configured to determine a difference duration between the current time and the policy start time as a policy maintenance duration.

[0204] The specific implementation of the encoding duration obtaining subunit 1221 and the duration determining subunit 1222 can refer to the description of step S302 in the above-mentioned Figure 7

[0205] In one embodiment, the redundancy encoding unit 124 can include a redundancy degree determining subunit 1241, a redundancy packet quantity determining subunit 1242, and a redundancy encoding subunit 1243.

[0206] The redundancy degree determining subunit 1241 is configured to determine a first redundancy degree corresponding to a target data frame according to the frame group encoding duration, the policy maintenance duration, and a packet loss rate threshold.

[0207] The redundancy packet quantity determining subunit 1242 is configured to obtain an original packet quantity corresponding to a target original data packet, and determine a first redundancy packet quantity corresponding to the target data frame according to the first redundancy degree and the original packet quantity.

[0208] The redundancy encoding subunit 1243 is configured to perform redundancy encoding on the target original data packet according to the first redundancy degree and the first redundancy packet quantity, to obtain a redundancy data packet corresponding to the target original data packet. The data packet quantity corresponding to the redundancy data packet is the first redundancy packet quantity.

[0209] The specific implementation of the redundancy degree determining subunit 1241, the redundancy packet quantity determining subunit 1242, and the redundancy encoding subunit 1243 can refer to the description of step S304 in the above-mentioned Figure 7

[0210] In one embodiment, the frame type of the target data frame is a forward encoding type.

[0211] The redundancy encoding module 12 can include a reference frame obtaining unit 125, a reference packet obtaining unit 126, and a packet encoding unit 127. ​​

[0212] The reference frame acquisition unit 125 is configured to acquire a unit data frame group to which the target data frame belongs based on the second redundancy coding rule for the forward coding type indicated by the redundancy transmission strategy. The unit data frame group is composed of M continuous data frames in the media data. M is a positive integer.

[0213] The reference frame acquisition unit 125 is further configured to acquire a reference data frame in the unit data frame group, which has the intra coding type and the transmitted state.

[0214] The reference packet acquisition unit 126 is configured to acquire an original data packet corresponding to the reference data frame, and determine the original data packet corresponding to the reference data frame as a reference packet.

[0215] The packet coding unit 127 is configured to acquire a first data packet quantity corresponding to the target original data packet and a second data packet quantity corresponding to the reference packet, perform redundancy coding on the target original data packet according to the first data packet quantity and the second data packet quantity, and obtain a redundancy data packet corresponding to the target original data packet.

[0216] The specific implementation of the reference frame acquisition unit 125, the reference packet acquisition unit 126, and the packet coding unit 127 can be referred to the description of steps S401-S404 in the above-mentioned Figure 8 corresponding embodiments, which will not be described here.

[0217] In one embodiment, the packet coding unit 127 is further configured to acquire a redundancy degree corresponding to the reference data frame, and determine the redundancy degree corresponding to the reference data frame as a reference redundancy degree.

[0218] The packet coding unit 127 is further configured to determine a first candidate redundancy degree corresponding to the target data frame based on the reference redundancy degree, the first data packet quantity, and the second data packet quantity.

[0219] The packet coding unit 127 is further configured to acquire a second candidate redundancy degree for the forward coding type indicated by the second redundancy coding rule.

[0220] The packet coding unit 127 is further configured to determine a minimum candidate redundancy degree between the first candidate redundancy degree and the second candidate redundancy degree as a second redundancy degree corresponding to the target data frame.

[0221] The packet coding unit 127 is further configured to determine a second redundancy packet quantity corresponding to the target data frame according to the first data packet quantity and the second redundancy degree.

[0222] The packet encoding unit 127 is further configured to encode the target original data packet redundantly according to the second redundancy and the second number of redundant packets, to obtain a redundant data packet corresponding to the target original data packet; the number of data packets corresponding to the redundant data packet is the second number of redundant packets.

[0223] In one embodiment, the data processing apparatus 1 can further comprise an original packet obtaining module 15 and an original packet sending module 16.

[0224] The original packet obtaining module 15 is configured to, if the function state of the redundant encoding function indicated by the redundant transmission strategy is the closed state when the remaining data frame is being transmitted, obtain a to-be-transmitted original data packet corresponding to the remaining data frame; the to-be-transmitted original data packet is a data packet obtained after data encoding is performed on the remaining data frame.

[0225] The original packet sending module 16 is configured to send the to-be-transmitted original data packet to the receiving end.

[0226] The specific implementation of the original packet obtaining module 15 and the original packet sending module 16 can be referred to the description of step S104 in the above Figure 4 corresponding embodiments, which will not be repeated here.

[0227] In one embodiment, the data processing apparatus 1 can further comprise a retransmission module 17.

[0228] The retransmission module 17 is configured to, if the feedback information comprises a retransmission request for a lost data packet, obtain the lost data packet from the cache library based on the retransmission request, and retransmit the lost data packet to the receiving end; the lost data packet is an original data packet that is not received by the receiving end in the target original data packet.

[0229] The specific implementation of the retransmission module 17 can be referred to the description of step S104 in the above Figure 4 corresponding embodiments, which will not be repeated here.

[0230] In the embodiments of the present application, since a GOP is composed of I, P and B frames, the code rate of the initial I frame in a GOP is large, the P and B frames in the middle occupy less, and the transmission bandwidth is not smooth, which can cause inaccurate bandwidth evaluation and network congestion. In the present application, different frame types can adopt different redundancy coding strategies, and the FEC redundancy transmission of I / P frames is dynamically controlled in real time (FEC redundancy transmission strategy is closed or continued), which can improve the network congestion and packet loss caused by inaccurate network bandwidth evaluation due to the code rate fluctuation of I frames, and can also make the data bandwidth smoother in the data transmission scene (such as low-delay live transmission scene), so as to make the network congestion control more accurate, reduce the NACK retransmission, and further reduce the end-to-end delay caused by retransmission waiting.

[0231] Further, please refer to Figure 10 , Figure 10 is a structural schematic diagram of a computer device provided by the embodiments of the present application. As shown in Figure 10 , the data processing device 1 in the embodiments corresponding to the above Figure 9 application can be applied to the above computer device 8000, and the computer device 8000 can include a processor 8001, a network interface 8004 and a memory 8005, in addition, the computer device 8000 further includes a user interface 8003 and at least one communication bus 8002. Wherein, the communication bus 8002 is used to realize the connection communication between these components. Wherein, the user interface 8003 can include a display screen (Display), a keyboard (Keyboard), and the optional user interface 8003 can further include a standard wired interface, a wireless interface. The network interface 8004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 8005 can be a high-speed RAM memory, or a non-volatile memory, for example, at least one disk memory. The memory 8005 can optionally be at least one storage device located away from the aforementioned processor 8001. As shown in Figure 10 , the memory 8005 as a computer readable storage medium can include an operating system, a network communication module, a user interface module and a device control application program.

[0232] In the computer device 8000 shown in Figure 10 , the network interface 8004 can provide network communication function; and the user interface 8003 is mainly used to provide an input interface for the user; and the processor 8001 can be used to call the device control application program stored in the memory 8005 to realize:

[0233] obtain a target original data packet corresponding to the target data frame; the target original data packet refers to a data packet obtained after data encoding is performed on the target data frame;

[0234] perform redundant encoding on the target original data packet based on a redundant transmission strategy to obtain a redundant data packet corresponding to the target original data packet, and send the target original data packet and the redundant data packet to the receiving end;

[0235] receive feedback information returned by the receiving end;

[0236] If the feedback information includes a real-time packet loss rate for the target original data packet and the redundant data packet, a function state of a redundant encoding function indicated by the redundant transmission strategy is determined based on the real-time packet loss rate when transmitting a remaining data frame; the function state includes a running state and a closed state; the remaining data frame refers to a data frame other than the target data frame and not yet transmitted in media data to which the target data frame belongs.

[0237] It should be understood that the computer device 8000 described in the embodiments of the present application can execute the foregoing description of the data processing method in the embodiments corresponding thereto. Figures 4 to 8 The foregoing description of the data processing device 1 in the embodiments corresponding thereto can also be executed, and the description of the beneficial effects of using the same method will not be repeated. Figure 9 The foregoing description of the data processing device 1 in the embodiments corresponding thereto can also be executed, and the description of the beneficial effects of using the same method will not be repeated.

[0238] In addition, it should be noted that the embodiments of the present application also provide a computer readable storage medium, and the aforementioned computer readable storage medium stores the computer program executed by the aforementioned data processing computer device 1000, and the aforementioned computer program includes program instructions, and when the aforementioned processor executes the aforementioned program instructions, the foregoing description of the data processing method in the embodiments corresponding thereto can be executed, and therefore, the description will not be repeated. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer readable storage medium embodiments of the present application, please refer to the description of the method embodiments of the present application. Figures 4 to 8 In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer readable storage medium embodiments of the present application, please refer to the description of the method embodiments of the present application.

[0239] The computer readable storage medium can be an internal storage unit of the data processing apparatus or the computer device, for example, a hard disk or a memory of the computer device. The computer readable storage medium can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the computer device. The computer readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0240] In an aspect of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to perform the method provided in an aspect of the embodiments of the present application.

[0241] The terms "first", "second", and the like in the description and claims and drawings of the embodiments of the present application are used to distinguish different objects, and are not used to describe a particular order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but can optionally include other steps or units not listed, or can optionally include other steps or units inherent to the process, method, device, product, or apparatus.

[0242] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software, or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in a general manner. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0243] The method and related apparatus provided by the embodiments of the present application are described with reference to the method flowchart and / or structural schematic diagram provided by the embodiments of the present application, and each flow and / or block of the method flowchart and / or structural schematic diagram and the combination of the flows and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. The computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowchart Figure One of one or more flows and / or the functions specified in the structural schematic Figure One of one or more blocks. The computer program instructions can also be stored in a computer readable memory capable of causing the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices that implement the functions specified in the flowchart Figure One of one or more flows and / or the functions specified in the structural schematic Figure One of one or more blocks. The computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flowchart Figure One of one or more flows and / or the functions specified in the structural schematic

[0244] The above disclosure is only the preferred embodiments of the present application, and of course cannot limit the scope of the rights of the present application, so the equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data processing method, characterized by, The method comprises: obtaining a target original data packet corresponding to a target data frame; the target original data packet is a data packet obtained after data encoding is performed on the target data frame; performing redundant encoding on the target original data packet based on a redundant transmission strategy to obtain a redundant data packet corresponding to the target original data packet, and sending the target original data packet and the redundant data packet to a receiving end; receiving feedback information returned by the receiving end; if the feedback information comprises a real-time packet loss rate for the target original data packet and the redundant data packet, determining a function state of a redundant encoding function indicated by the redundant transmission strategy based on the real-time packet loss rate when transmitting a remaining data frame; the function state comprises a running state and a closed state; the remaining data frame is a data frame that is not transmitted in addition to the target data frame in media data to which the target data frame belongs; wherein, when a frame type of the target data frame is an intra-frame encoding type, the performing redundant encoding on the target original data packet based on the redundant transmission strategy to obtain the redundant data packet corresponding to the target original data packet comprises: obtaining a unit data frame group to which the target data frame belongs based on a first redundant encoding rule for the intra-frame encoding type indicated by the redundant transmission strategy; the unit data frame group is composed of M continuous data frames in the media data; M is a positive integer; obtaining a frame group encoding duration corresponding to the unit data frame group and a strategy maintenance duration corresponding to the redundant transmission strategy; obtaining a service scenario type to which the media data belongs, and determining a packet loss rate threshold corresponding to the media data according to the service scenario type; performing redundant encoding on the target original data packet according to the frame group encoding duration, the strategy maintenance duration, and the packet loss rate threshold to obtain the redundant data packet corresponding to the target original data packet.

2. The method of claim 1, wherein, The determining the function state of the redundant encoding function indicated by the redundant transmission strategy based on the real-time packet loss rate comprises: obtaining a service scenario type to which the media data belongs, and determining a packet loss rate threshold corresponding to the media data according to the service scenario type; if the real-time packet loss rate is greater than a preset lower limit value and the real-time packet loss rate is less than the packet loss rate threshold, obtaining a target network transmission delay at an information receiving time, and determining the function state of the redundant encoding function indicated by the redundant transmission strategy according to the target network transmission delay; the information receiving time is a time when the feedback information is received; the preset lower limit value is less than the packet loss rate threshold; if the real-time packet loss rate is greater than the packet loss rate threshold, determining the function state of the redundant encoding function indicated by the redundant transmission strategy as a closed state.

3. The method of claim 2, wherein, The obtaining the target network transmission delay at the information receiving time comprises: obtaining a set of historical detection time periods for network transmission delay; each historical detection time period in the set of historical detection time periods is used for detecting network transmission delay; one historical detection time period corresponds to one detected historical network transmission delay; In the set of historical detection time periods, a target historical detection time period to which the information receiving moment belongs is acquired; A historical network transmission delay corresponding to the target historical detection time period is determined as a target network transmission delay at the information receiving moment.

4. The method of claim 2, wherein, The determining of the function state of the redundancy coding function indicated by the redundancy transmission strategy according to the target network transmission delay comprises: A set of historical network transmission delays is acquired; the set of historical network transmission delays comprises N historical network transmission delays, one historical network transmission delay being a detected network transmission delay in one historical detection time period; N is a positive integer; each historical detection time period is earlier than the information receiving moment; A delay change curve for the set of historical network transmission delays and the target network transmission delay is constructed according to the time sequence of N historical detection time periods and the information receiving moment; The function state of the redundancy coding function indicated by the redundancy transmission strategy is determined according to a delay change trend indicated by the delay change curve.

5. The method of claim 4, wherein, The determining of the function state of the redundancy coding function indicated by the redundancy transmission strategy according to the delay change trend indicated by the delay change curve comprises: If the delay change trend is an increasing trend, the function state of the redundancy coding function indicated by the redundancy transmission strategy is determined as a closed state; If the delay change trend is a stable fluctuation trend, the function state of the redundancy coding function indicated by the redundancy transmission strategy is determined as a running state.

6. The method of claim 1, wherein, The acquiring of the frame group coding duration corresponding to the unit data frame group and the strategy maintenance duration corresponding to the redundancy transmission strategy comprises: M data coding durations are obtained by acquiring a data coding duration corresponding to each data frame included in the unit data frame group; The M data coding durations are added to obtain the frame group coding duration; A current moment and a strategy opening moment of the redundancy transmission strategy are acquired; A difference duration between the current moment and the strategy opening moment is determined as the strategy maintenance duration.

7. The method of claim 1, wherein, The redundancy coding of the target original data packet according to the frame group coding duration, the strategy maintenance duration and the packet loss rate threshold to obtain a redundancy data packet corresponding to the target original data packet comprises: A first redundancy degree corresponding to the target data frame is determined according to the frame group coding duration, the strategy maintenance duration and the packet loss rate threshold; An original packet quantity corresponding to the target original data packet is acquired, and a first redundancy packet quantity corresponding to the target data frame is determined according to the first redundancy degree and the original packet quantity; The redundancy coding of the target original data packet according to the first redundancy degree and the first redundancy packet quantity to obtain a redundancy data packet corresponding to the target original data packet; a data packet quantity of the redundancy data packet is the first redundancy packet quantity.

8. The method of claim 1, wherein, The redundancy coding of the target original data packet based on the redundancy transmission strategy to obtain a redundancy data packet corresponding to the target original data packet comprises: When the frame type of the target data frame is a forward coding type, a second redundancy coding rule for the forward coding type indicated by the redundancy transmission strategy is used to obtain a unit data frame group to which the target data frame belongs; the unit data frame group is composed of M continuous data frames in the media data; M is a positive integer; A reference data frame with a frame type of intra-frame coding type and a transmission state of having been transmitted in the unit data frame group is obtained; An original data packet corresponding to the reference data frame is obtained, and the original data packet corresponding to the reference data frame is determined as a reference data packet; A first data packet quantity corresponding to the target original data packet and a second data packet quantity corresponding to the reference data packet are obtained, and the target original data packet is redundantly coded according to the first data packet quantity and the second data packet quantity, to obtain a redundancy data packet corresponding to the target original data packet.

9. The method of claim 8, wherein, The redundancy data packet corresponding to the target original data packet is obtained by redundantly coding the target original data packet according to the first data packet quantity and the second data packet quantity, including: A redundancy degree corresponding to the reference data frame is obtained, and the redundancy degree corresponding to the reference data frame is determined as a reference redundancy degree; A first candidate redundancy degree corresponding to the target data frame is determined based on the reference redundancy degree, the first data packet quantity, and the second data packet quantity; A second candidate redundancy degree for the forward coding type indicated by the second redundancy coding rule is obtained; A minimum candidate redundancy degree between the first candidate redundancy degree and the second candidate redundancy degree is determined as a second redundancy degree corresponding to the target data frame; A second redundancy packet quantity corresponding to the target data frame is determined according to the first data packet quantity and the second redundancy degree; The target original data packet is redundantly coded according to the second redundancy degree and the second redundancy packet quantity, to obtain a redundancy data packet corresponding to the target original data packet; a data packet quantity corresponding to the redundancy data packet is the second redundancy packet quantity.

10. The method of claim 1, wherein, The method further includes: If a function state of a redundancy coding function indicated by the redundancy transmission strategy is a closed state when transmitting a remaining data frame, a to-be-transmitted original data packet corresponding to the remaining data frame is obtained; the to-be-transmitted original data packet refers to a data packet obtained after data coding of the remaining data frame; The to-be-transmitted original data packet is sent to the receiving end.

11. The method of claim 1, wherein, The method further includes: If the feedback information includes a retransmission request for a lost data packet, the lost data packet is obtained from a cache library based on the retransmission request, and the lost data packet is retransmitted to the receiving end; the lost data packet refers to an original data packet that is not received by the receiving end in the target original data packet.

12. A data processing apparatus, characterized by including: A packet obtaining module is configured to obtain a target original data packet corresponding to a target data frame; the target original data packet refers to a data packet obtained after data coding of the target data frame; a redundancy coding module, configured to perform redundancy coding on the target original data packet based on a redundancy transmission strategy to obtain a redundancy data packet corresponding to the target original data packet, and send the target original data packet and the redundancy data packet to a receiving end; a feedback receiving module, configured to receive feedback information returned by the receiving end; a function state determining module, configured to, if the feedback information includes a real-time packet loss rate for the target original data packet and the redundancy data packet, determine a function state of a redundancy coding function indicated by the redundancy transmission strategy based on the real-time packet loss rate when transmitting a remaining data frame; the function state includes a running state and a closed state; the remaining data frame refers to a data frame other than the target data frame and not transmitted in media data to which the target data frame belongs; when the frame type of the target data frame is an intra-frame coding type, the redundancy coding on the target original data packet based on the redundancy transmission strategy to obtain the redundancy data packet corresponding to the target original data packet includes: acquiring a unit data frame group to which the target data frame belongs based on a first redundancy coding rule for the intra-frame coding type indicated by the redundancy transmission strategy; the unit data frame group is composed of M continuous data frames in the media data; M is a positive integer; acquiring a frame group coding duration corresponding to the unit data frame group and a strategy maintenance duration corresponding to the redundancy transmission strategy; acquiring a service scenario type to which the media data belongs, and determining a packet loss rate threshold corresponding to the media data according to the service scenario type; and performing redundancy coding on the target original data packet according to the frame group coding duration, the strategy maintenance duration, and the packet loss rate threshold to obtain the redundancy data packet corresponding to the target original data packet.

13. A computer device, comprising: comprise: a processor, a memory, and a network interface; the processor is connected with the memory and the network interface, wherein the network interface is configured to provide network communication function, the memory is configured to store program code, and the processor is configured to call the program code to enable the computer device to execute the method in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by the processor to execute the method in any one of claims 1-11.

15. A computer program product, characterised in that, The computer program product comprises computer instructions stored in the computer readable storage medium, and the computer instructions are suitable for being read and executed by the processor to enable the computer device with the processor to execute the method in any one of claims 1-11.

Images