Redundancy transmission method and device, related equipment, storage medium and program product

Abstract

The invention provides a redundancy transmission method and device, related equipment, a storage medium and a program product, which are applied to the technical field of communication, and the method comprises the following steps: a sending end respectively encodes a first I frame and a second I frame; performing inter-frame prediction on the encoded second I frame by taking the encoded first I frame as a reference frame to obtain an inter-frame prediction residual image; and after the first I frame is sent to a receiving end, taking the inter-frame prediction residual image as a redundant unit, and sending the second I frame and the redundant unit to the receiving end through a plurality of transmission paths between the sending end and the receiving end. Compared with a mode that the whole I frame is copied and then is transmitted as a redundant unit, the method has the advantages that the redundancy granularity is reduced, and the inter-frame prediction residual image is transmitted as the redundant unit, so that on the basis of ensuring the reliability of data transmission, the occupation of bandwidth resources can be reduced, and the resource consumption can be reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, and in particular to a redundancy transmission method and device, related equipment, a storage medium and a program product. BACKGROUND

[0002] Redundancy transmission involves adding extra redundant bits in original data, when the original data encounters bit damage or loss in the transmission process, the extra redundant bits can help the receiving end to reconstruct or correct information. By adding redundant information, even if part of the data is damaged, the system can use these redundant information to detect and correct errors, so as to recover the complete and true information, therefore, it significantly improves the probability of correct reception of the receiving end data packet. In the field of video data transmission, the transmission quality of I frame is directly related to the decoding effect of a group of pictures (GOP), once the I frame is lost or error occurs, the error will spread in the GOP, and the subsequent frames cannot be normally decoded, therefore, it is crucial to ensure the transmission reliability of I frame. The prior art is to copy the I frame and transmit it through multiple paths at the same time, if the I frame on one path is damaged, the I frame on other paths can be used as a redundant frame and the I frame information can be obtained therefrom, but the compression rate of I frame is low, and the data amount is large, thereby increasing the occupation of bandwidth resources, resulting in large resource consumption. SUMMARY

[0003] The embodiments of the present application provide a redundancy transmission method, device, related equipment, storage medium and program product to solve the problem of large resource consumption in the prior art.

[0004] To solve the above technical problems, the present application is implemented as follows:

[0005] In a first aspect, the embodiments of the present application provide a redundancy transmission method applied to a sending end, the method comprising:

[0006] encoding a first I frame and a second I frame, wherein the first I frame and the second I frame are any two adjacent I frames in a video stream, and the transmission order of the first I frame is earlier than that of the second I frame;

[0007] performing inter-frame prediction on the encoded second I frame with the encoded first I frame as a reference frame to obtain an inter-frame prediction residual image;

[0008] after sending the first I frame to a receiving end, sending the second I frame and a redundancy unit to the receiving end through multiple transmission paths between the sending end and the receiving end, wherein the redundancy unit is the inter-frame prediction residual image.

[0009] Optionally, the sending of the redundancy unit to the receiving end comprises:

[0010] The sending of the redundancy unit to the receiving end for the first preset number of times;

[0011] After the sending of the redundancy unit to the receiving end for the first preset number of times, the method further comprises:

[0012] Calculating a difference between a first transmission delay and a second transmission delay, the first transmission delay being an overall transmission delay between the sending end and the receiving end before the sending of the redundancy unit to the receiving end for the first preset number of times, and the second transmission delay being an overall transmission delay between the sending end and the receiving end after the sending of the redundancy unit to the receiving end for the first preset number of times;

[0013] In a case where the difference is greater than a preset threshold, adjusting the first preset number of times.

[0014] Optionally, the overall transmission delay is calculated based on the following manner:

[0015] Obtaining a transmission delay of each transmission path between the sending end and the receiving end;

[0016] According to a data amount transmitted by each transmission path and an overall transmission data amount of the plurality of transmission paths, calculating a delay weight of each transmission path;

[0017] According to the delay weight of each transmission path and the transmission delay of each transmission path, calculating the overall transmission delay.

[0018] Optionally, the adjusting of the first preset number of times comprises:

[0019] According to the first transmission delay, the second transmission delay and the first preset number of times, calculating a second preset number of times;

[0020] Adjusting the first preset number of times to the second preset number of times.

[0021] Optionally, in a case where a plurality of redundancy units are obtained based on a plurality of adjacent I frames of the video stream and the first preset number of times has a decimal part, the sending of the redundancy unit to the receiving end for the first preset number of times comprises:

[0022] Sending each redundancy unit of the plurality of redundancy units to the receiving end according to an integer part of the first preset number of times;

[0023] According to a decimal part of the first preset number of times of the plurality of redundancy units and a quantity of the plurality of redundancy units, determining a shared sending number of times of the plurality of redundancy units.

[0024] sending N redundancy units of the plurality of redundancy units to the receiving end according to the shared sending times, wherein N is a non-negative integer.

[0025] In a second aspect, the embodiments of the present application provide a redundancy transmission method applied to a receiving end, the method comprising:

[0026] receiving a first I-frame sent by a sending end;

[0027] processing a redundancy unit according to the redundancy unit and a receiving condition of a second I-frame sent by the sending end, wherein the redundancy unit is an inter-frame prediction residual image of the first I-frame and the second I-frame, the first I-frame and the second I-frame are any two adjacent I-frames in a video stream, and the transmission order of the first I-frame is earlier than that of the second I-frame.

[0028] Optionally, the processing of the redundancy unit according to the redundancy unit and the receiving condition of the second I-frame sent by the sending end comprises any one of the following conditions:

[0029] in a case where the second I-frame is correctly received, discarding the redundancy unit;

[0030] in a case where there already exists a redundancy unit identical to the redundancy unit, discarding the duplicate redundancy unit;

[0031] in a case where the second I-frame is not correctly received at a target time, performing inter-frame decoding based on the redundancy unit and the first I-frame to obtain the second I-frame, and deleting the redundancy unit and the first I-frame.

[0032] In a third aspect, the embodiments of the present application further provide a redundancy transmission device applied to a sending end, the redundancy transmission device comprising:

[0033] a first encoding module configured to encode a first I-frame and a second I-frame respectively, wherein the first I-frame and the second I-frame are any two adjacent I-frames in a video stream, and the transmission order of the first I-frame is earlier than that of the second I-frame;

[0034] a first prediction module configured to perform inter-frame prediction on the encoded second I-frame with the encoded first I-frame as a reference frame to obtain an inter-frame prediction residual image;

[0035] a first sending module configured to send the second I-frame and the inter-frame prediction residual image as a redundancy unit to a receiving end through a plurality of transmission paths between the sending end and the receiving end after sending the first I-frame to the receiving end.

[0036] In a fourth aspect, the embodiments of the present application further provide a redundancy transmission apparatus applied to a receiving end, comprising:

[0037] a first receiving module, configured to receive a first I-frame sent by a sending end;

[0038] a first processing module, configured to process a redundancy unit according to a receiving condition of the second I-frame, wherein the redundancy unit is an inter-frame prediction residual image of the first I-frame and the second I-frame, the first I-frame and the second I-frame are any two adjacent I-frames in a video stream, and the transmission sequence of the first I-frame is earlier than that of the second I-frame.

[0039] In a fifth aspect, the embodiments of the present application further provide a sending end, comprising a transceiver and a processor, wherein the processor is configured to:

[0040] encode the first I-frame and the second I-frame respectively, wherein the first I-frame and the second I-frame are any two adjacent I-frames in a video stream, and the transmission sequence of the first I-frame is earlier than that of the second I-frame;

[0041] perform inter-frame prediction on the encoded second I-frame with the encoded first I-frame as a reference frame to obtain an inter-frame prediction residual image;

[0042] the transceiver is configured to:

[0043] after the first I-frame is sent to the receiving end, send the second I-frame and the redundancy unit to the receiving end through a plurality of transmission paths between the sending end and the receiving end, with the inter-frame prediction residual image as the redundancy unit.

[0044] In a sixth aspect, the embodiments of the present application further provide a receiving end, comprising a transceiver and a processor, wherein the transceiver is configured to:

[0045] receive a first I-frame sent by a sending end;

[0046] the processor is configured to:

[0047] process a redundancy unit according to a receiving condition of the second I-frame, wherein the redundancy unit is an inter-frame prediction residual image of the first I-frame and the second I-frame, the first I-frame and the second I-frame are any two adjacent I-frames in a video stream, and the transmission sequence of the first I-frame is earlier than that of the second I-frame.

[0048] In a seventh aspect, an electronic device is provided, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and when the computer program is executed by the processor, the steps of the redundant transmission method described above are implemented.

[0049] In an eighth aspect, a computer readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the redundant transmission method described above are implemented.

[0050] In a ninth aspect, a computer program product is provided, which includes computer instructions, and when the computer instructions are executed by a processor, the steps of the redundant transmission method described above are implemented.

[0051] In the redundant transmission method of the embodiments of the present application, the first I frame and the second I frame are encoded by the sending end, wherein the first I frame and the second I frame are any two adjacent I frames in a video stream, and the transmission order of the first I frame is earlier than that of the second I frame; the second I frame after encoding is inter-predicted with the first I frame after encoding as a reference frame to obtain an inter-predicted residual image; after the first I frame is sent to the receiving end, the second I frame and the inter-predicted residual image as a redundant unit are sent to the receiving end through multiple transmission paths between the sending end and the receiving end. Compared with copying the entire I frame as a redundant unit for transmission, the method reduces the redundancy granularity and transmits the inter-predicted residual image as a redundant unit, which can reduce the occupation of bandwidth resources and reduce resource consumption on the basis of ensuring data transmission reliability. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the description of the embodiments of the present application will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0053] Figure 1 is a schematic diagram of the redundant transmission method provided by the embodiments of the present application;

[0054] Figure 2 is one of the flowcharts of the redundant transmission method provided by the embodiments of the present application;

[0055] Figure 3 is a processing schematic diagram of the inter-predicted residual image provided by the embodiments of the present application;

[0056] Figure 4This is the second flowchart of the redundant transmission method provided in the embodiments of this application;

[0057] Figure 5 This is an architecture diagram of the sending end and receiving end provided in the embodiments of this application;

[0058] Figure 6 This is one of the structural diagrams of the redundant transmission device provided in the embodiments of this application;

[0059] Figure 7 This is the second structural diagram of the redundant transmission device provided in the embodiments of this application;

[0060] Figure 8 This is a structural diagram of the transmitting end provided in an embodiment of this application;

[0061] Figure 9 This is a structural diagram of a receiving end provided in an embodiment of this application. Detailed Implementation

[0062] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] To further illustrate the technical solution of this application, the background technology involved in this application is first introduced: Multiple transmission paths exist between the sending end and the receiving end, and the sending end can use multiple transmission paths to transmit data to the receiving end. Redundant transmission involves the sending end selecting a portion of the data, copying it into multiple copies, and transmitting it simultaneously using different paths, such as... Figure 1 As shown. Selective redundancy transmission is suitable for scenarios with large data volumes. By using redundant data, it ensures the reliability of critical data transmission and improves resource utilization efficiency under transmission resource constraints. This application is mainly used in audio and video service scenarios, such as live streaming and online conferencing.

[0064] In audio and video service scenarios, the transmission quality of I-frames in a video stream directly affects the decoding performance of the entire Group of Pictures (GOP). If an I-frame is lost or erroneous, the error will propagate within the GOP, preventing subsequent frames from being decoded correctly. Therefore, ensuring the reliability of I-frame transmission is crucial, and current technologies consider using I-frames as redundant units for redundant transmission. However, I-frames use intra-frame compression coding, containing all the information of the entire image, and their compression ratio can only reach about 1 / 6. Generally speaking, I-frames have the lowest compression efficiency, while B-frames have the highest. Assuming an I-frame bitrate of 100, a P-frame has about twice the compression efficiency of an I-frame (bitrate of 50), while a B-frame has nearly twice the compression efficiency of a P-frame (bitrate of 25).

[0065] I-frames contain significantly more data than P-frames and B-frames. Directly using I-frames as redundant units would result in substantial additional bandwidth usage, potentially impacting normal data transmission. Therefore, it is necessary to further reduce the granularity of redundancy.

[0066] In video image sequences, moving subjects exhibit high correlation in both spatial and temporal dimensions. In the temporal domain, due to the very small sampling interval between adjacent frames, the values ​​of corresponding pixels are highly correlated; in the spatial domain, the values ​​of adjacent pixels are usually also very close. Since the length of a Group of Pictures (GOP) can be set according to the network environment and terminal transmission requirements, when the GOP length is short, there must be a strong correlation between I-frames of adjacent GOPs. For example, in a video conferencing scenario, only local images change for a long period, while most of the background image remains unchanged. However, video compression coding methods do not allow for the generation of only one keyframe over a long period. If an error occurs in the transmission of that keyframe, subsequent video frames in that GOP cannot be decoded. Therefore, at certain intervals, the encoder automatically inserts an I-frame to shorten the GOP length and prevent error propagation. In this case, the difference between two adjacent I-frames is small, and the correlation is high.

[0067] This application provides a redundant transmission method applied at the transmitting end. Borrowing the concept of inter-frame prediction, it performs inter-frame prediction on adjacent I-frames, thereby reducing temporal redundancy, improving the compression ratio, and refining the granularity of I-frame redundancy. Specifically, see... Figure 2 , Figure 2 This is a flowchart of the redundant transmission method provided in the embodiments of this application, such as... Figure 2 As shown, it includes the following steps:

[0068] Step 201: Encode the first I-frame and the second I-frame respectively, wherein the first I-frame and the second I-frame are any two adjacent I-frames in the video stream, and the transmission order of the first I-frame is before the transmission order of the second I-frame.

[0069] The specific process in this step is as follows:

[0070] (1) The encoder at the transmitting end first determines whether to perform intra-frame or inter-frame coding on the current image f(t) based on the GOP configuration parameters and the terminal transmission requirements;

[0071] (2) The encoder encodes the current image f(t) into F(t);

[0072] (3) Store F(t) after encoding the first I-frame corresponding to the current image. I ;

[0073] (4) Repeat steps (1) to (3) to obtain the encoded F(t+GOP) of the second I-frame. I And store F(t+GOP). I .

[0074] Step 202: Perform inter-frame prediction on the encoded second I-frame using the encoded first I-frame as the reference frame to obtain the inter-frame prediction residual image;

[0075] In this step, see Figure 3 The encoded second I-frame F(t+GOP) I The first I-frame after encoding, F(t). I Inter-frame prediction is performed using the reference frame to obtain the inter-frame prediction residual image F'(t+GOP) of the I-frame. I =F(t+GOP) I -F(t) I The inter-frame prediction residual image F'(t+GOP) is then used. I As a redundant unit, it performs Discrete Cosine Transform (DCT), quantization, and entropy coding.

[0076] Step 103: After sending the first I-frame to the receiving end, the second I-frame and the redundancy unit are sent to the receiving end through multiple transmission paths between the sending end and the receiving end, using the inter-frame prediction residual image as a redundancy unit.

[0077] In this step, after sending the first I-frame to the receiver, the sending end continues to send the second I-frame. Furthermore, the sending end also needs to send the inter-frame prediction residual image between the first and second I-frames as a redundancy unit to the receiver via a different transmission path than the second I-frame. Thus, if an error occurs during the transmission of the second I-frame, the second I-frame can be predicted and recovered using the first I-frame and the inter-frame prediction residual image.

[0078] In one embodiment, the transmitting end encodes a first I-frame and a second I-frame, wherein the first I-frame and the second I-frame are any two adjacent I-frames in the video stream, and the transmission order of the first I-frame precedes the transmission order of the second I-frame. The encoded second I-frame is then used as a reference frame for inter-frame prediction to obtain an inter-frame prediction residual image. After the first I-frame is sent to the receiving end, the inter-frame prediction residual image is used as a redundancy unit, and the second I-frame and the redundancy unit are transmitted to the receiving end through multiple transmission paths between the transmitting end and the receiving end. Compared to transmitting a copied I-frame as a redundancy unit, this embodiment reduces the redundancy granularity by using the inter-frame prediction residual image as the redundancy unit. This reduces bandwidth consumption and resource usage while ensuring data transmission reliability.

[0079] Optionally, sending the redundant unit to the receiving end includes:

[0080] Send the redundant unit a first preset number of times to the receiving end;

[0081] After sending the redundant unit to the receiving end a first preset number of times, the method further includes:

[0082] Calculate the difference between the first transmission delay and the second transmission delay. The first transmission delay is the overall transmission delay of multiple transmission paths between the sender and the receiver before sending the redundant unit to the receiver a first preset number of times. The second transmission delay is the overall transmission delay of multiple transmission paths between the sender and the receiver after sending the redundant unit to the receiver a first preset number of times.

[0083] If the difference is greater than a preset threshold, the first preset number of times is adjusted.

[0084] In one implementation, the primary principle of redundant transmission is to ensure that it does not affect the transmission of the original data stream, and secondly, to use as little redundant data as possible to guarantee the correct reception of the data stream. To prevent redundant data from affecting the transmission of the original data stream, the sending end needs to further control the amount of redundant data based on the state of the transmission path. In network transmission, it is generally considered that when the overall transmission delay of the transmission path increases significantly, it indicates that the transmission capacity of the transmission path is approaching saturation. Moreover, the capacity of redundant data should not exceed the difference between the path transmission capacity and the original data capacity to avoid a significant increase in path transmission delay.

[0085] Therefore, when the overall transmission delay of multiple transmission paths between the sending and receiving ends varies significantly, it is necessary to adjust the number of transmission redundancy units. This helps to reduce the impact on the transmission of the original data stream.

[0086] Optionally, the overall transmission delay is calculated based on the following method:

[0087] Obtain the transmission delay of each transmission path among multiple transmission paths between the sending end and the receiving end;

[0088] The delay weight of each transmission path is calculated based on the amount of data transmitted along each transmission path and the total amount of data transmitted along the multiple transmission paths.

[0089] The overall transmission delay is calculated based on the delay weight of each transmission path and the transmission delay of each transmission path.

[0090] In one implementation, the sending end periodically updates the transmission delay of each path. Path delay detection methods are relatively mature, typically involving the sending and receiving ends working together to measure delays using experimental probe packets. The sending end performs weighted smoothing on the obtained delay data to obtain the path delay value. The overall transmission delay D of multiple transmission paths between the sending and receiving ends can be calculated based on the following formulas (1) and (2):

[0091] w i =Len i / LEN (1)

[0092]

[0093] Among them, w i Len represents the delay weight of transmission path i. i represents the amount of data (in bytes) transmitted along transmission path i, LEN represents the total amount of data (in bytes) transmitted, and d i Let LEN represent the transmission delay of transmission path i, and N represent the number of paths used for transmission. (After D is updated, LEN and Leni are reset, and a new round of statistical counting begins).

[0094] In this implementation, by calculating latency weights, paths with high transmission efficiency and low latency can be identified and prioritized. This encourages more data to pass through better channels, improving overall transmission performance.

[0095] Optionally, adjusting the first preset number of times includes:

[0096] Calculate the second preset number of times based on the first transmission delay, the second transmission delay, and the first preset number of times;

[0097] Adjust the first preset number of times to the second preset number of times.

[0098] In one implementation, the transmitting end determines the overall transmission delay D. If the change in delay does not exceed a preset threshold (usually 20%), the transmission state is considered relatively stable, and there is no need to adjust the transmission count of the redundant unit. Otherwise, the transmission count of the redundant unit needs to be adjusted. The specific calculation of the adjustment count can be found in the following calculation formula (3).

[0099]

[0100] Where RTR represents the first preset number of times, RTR' represents the second preset number of times, D' represents the second transmission delay, and D represents the first transmission delay.

[0101] In this implementation, when the overall transmission delay of the multiple transmission paths between the sending and receiving ends varies significantly, it is necessary to adjust the number of transmission redundancy units. This helps to reduce the impact on the original data stream transmission and ensure the overall quality of data transmission.

[0102] Optionally, if multiple redundant units are obtained based on multiple adjacent I-frames of the video stream and the first preset number of times has a fractional part, the step of sending the first preset number of redundant units to the receiving end includes:

[0103] Each of the plurality of redundant units is sent to the receiving end according to the integer part of the first preset number of times;

[0104] The number of times the multiple redundant units are shared is determined based on the fractional part of the first preset number of times the multiple redundant units are used and the number of the multiple redundant units.

[0105] N redundant units from the plurality of redundant units are sent to the receiving end according to the shared transmission count, where N is a non-negative integer.

[0106] In one implementation, multiple redundant units, denoted as set P = (p1, p2, ..., pn), can be obtained by residual calculation between multiple adjacent I-frames of the video stream. The redundancy transmission ratio (RTR) represents the number of times each redundant unit in set P is transmitted, with a default RTR = 1. When the RTR has a fractional part, multiple redundant units share the cumulative number of transmissions of the fractional part.

[0107] For example, if there are three redundant units P1, P2, and P3 in set P, and RTR = 1.4, with the integer part being 1, then P1, P2, and P3 are each transmitted once, with the fractional part being 0.4. The sum of the three redundant units is 1.2 (rounded down to 1, i.e., 1 transmission, shared by P1, P2, and P3). If a redundant unit is randomly selected from P1, P2, and P3 for an additional transmission, then N is 1.

[0108] If set P contains three redundant units P1, P2, and P3, and RTR = 1.2, with the integer part being 1, then P1, P2, and P3 are transmitted once each. With the fractional part being 0.2, the cumulative sum of the three redundant units is 0.6 (rounded down to 0, which is insufficient for complete transmission). Therefore, P1, P2, and P3 cannot be transmitted any further, and in this case, N is 0.

[0109] If set P contains three redundant units P1, P2, and P3, with RTR = 1.7 (the integer part is 1), then P1, P2, and P3 are each transmitted once, and the fractional part is 0.7. The sum of the three redundant units is 2.1 (rounded down to 2, i.e., 2 transmissions, shared by P1, P2, and P3). Two different redundant units are randomly selected from P1, P2, and P3 for an additional transmission. In this case, N is 2.

[0110] This implementation reduces unnecessary transmissions while ensuring a certain level of redundancy. Furthermore, if the first preset number of transmissions has a fractional portion, sharing the transmission count among multiple redundant units fully utilizes transmission resources and reduces waste.

[0111] This application provides a redundant transmission method applied at the receiving end; see [link to relevant documentation]. Figure 4 , Figure 4 This is a flowchart of the redundant transmission method provided in the embodiments of this application, such as... Figure 4 As shown, it includes the following steps:

[0112] Step 401: Receive the first I-frame sent by the sending end;

[0113] Step 402: Process the redundant unit according to the reception status of the redundant unit and the second I-frame sent by the sending end. The redundant unit is the inter-frame prediction residual image of the first I-frame and the second I-frame. The first I-frame and the second I-frame are any two adjacent I-frames in the video stream. The transmission order of the first I-frame is before the transmission order of the second I-frame.

[0114] In one embodiment, the receiving end receives video frames from the video stream in the order they are received. The receiving end first receives a first I-frame sent by the sending end, and then receives a second I-frame and a redundancy unit (the inter-frame prediction residual image of the first and second I-frames) sent by the sending end through different transmission paths. Thus, if an error occurs during the transmission of the second I-frame, the second I-frame can be predicted and recovered using the first I-frame and the inter-frame prediction residual image. Compared to transmitting a copied I-frame as a redundancy unit, this embodiment reduces the redundancy granularity by using the inter-frame prediction residual image as the redundancy unit. This reduces bandwidth consumption and resource usage while ensuring data transmission reliability.

[0115] Optionally, the process of processing the redundant units based on the redundancy units sent by the transmitting end and the reception status of the second I-frame includes any of the following:

[0116] If the second I-frame is received correctly, the redundant unit is discarded;

[0117] If a redundant unit identical to the redundant unit already exists, the duplicate redundant unit is discarded;

[0118] If the second I-frame is not correctly received at the target time, inter-frame decoding is performed based on the redundant unit and the first I-frame to obtain the second I-frame, and the redundant unit and the first I-frame are deleted.

[0119] In one implementation, the redundancy unit is the inter-frame prediction residual of the first I-frame and the second I-frame. The receiver stores the redundancy unit and the first I-frame respectively, and processes the redundancy unit based on factors such as the reception status of the second I-frame, the uniqueness and validity of the redundancy unit. Specifically, this may include the following scenarios:

[0120] (1) Determine the validity of the redundant unit: If the second I-frame is received correctly, discard the redundant unit;

[0121] (2) Determine the uniqueness of the redundant unit: If a redundant unit with the same name already exists in the receive buffer, discard the duplicate redundant unit.

[0122] (3) If the second I-frame is not received correctly when the playback time of the second I-frame is close, perform inter-frame decoding based on the redundant unit and the first I-frame to obtain the second I-frame, and delete the saved redundant unit and the first I-frame.

[0123] In this implementation, once the second I-frame is correctly received, the redundant unit becomes useless and can be immediately deleted to free up storage space and avoid bandwidth consumption. By identifying and discarding duplicate redundant units, unnecessary data transmission can be reduced, thereby alleviating network burden and potentially improving overall network transmission speed and response time. Inter-frame decoding is performed using the redundant unit and the first I-frame, enabling reconstruction of the second I-frame if it is lost or not received. This redundancy strategy ensures data integrity; even if data is lost, information loss will not occur.

[0124] See Figure 5 , Figure 5 This is an architecture diagram of the sending end and receiving end provided in the embodiments of this application. The sending end includes: a sending buffer, a sending record module, a delay detection module, a redundancy unit generation module, and a redundancy transmission control module.

[0125] The receiving end includes: a receiving buffer and a redundant unit removal module.

[0126] The transmit buffer is used to receive data from upper-layer applications and the redundancy unit generation module, and then send the data to the network. The sender maintains a transmit buffer for each path and keeps track of its status information.

[0127] The sending record module is used to record basic information about the data packets sent, such as packet length and transmission path identifier.

[0128] The delay detection module is used to measure the transmission delay of a path. Existing path delay measurement methods are relatively mature; for example, they are usually used in conjunction with the delay detection module at the other end to complete delay detection by sending and responding to probe packets.

[0129] The redundant transmission control module is used to control the amount of data transmitted redundantly and to select the transmission path for the redundant units.

[0130] The receive buffer is used to receive data packets from the network, reassemble them, and deliver them to the upper-layer application, and synchronize the receive status to the redundancy removal module.

[0131] The redundant unit removal module is used to remove received redundant units and restore I-frame data.

[0132] See Figure 6 , Figure 6 This is a structural diagram of a redundant transmission device provided in an embodiment of this application. This redundant device is applied at the transmitting end, such as... Figure 6 As shown, the redundant transmission device 600 includes:

[0133] The first encoding module 601 is used to encode the first I-frame and the second I-frame respectively, wherein the first I-frame and the second I-frame are any two adjacent I-frames in the video stream, and the transmission order of the first I-frame is before the transmission order of the second I-frame.

[0134] The first prediction module 602 is used to perform inter-frame prediction on the encoded second I-frame with the encoded first I-frame as the reference frame to obtain an inter-frame prediction residual image.

[0135] The first transmitting module 603 is configured to, after transmitting the first I-frame to the receiving end, use the inter-frame prediction residual image as a redundancy unit and transmit the second I-frame and the redundancy unit to the receiving end through multiple transmission paths between the transmitting end and the receiving end.

[0136] Optionally, the first transmitting module includes:

[0137] The first transmitting unit is used to transmit the redundant unit a first preset number of times to the receiving end;

[0138] The device further includes:

[0139] The first calculation module is used to calculate the difference between the first transmission delay and the second transmission delay. The first transmission delay is the overall transmission delay of multiple transmission paths between the sending end and the receiving end before sending the redundant unit to the receiving end a first preset number of times. The second transmission delay is the overall transmission delay of multiple transmission paths between the sending end and the receiving end after sending the redundant unit to the receiving end a first preset number of times.

[0140] The first adjustment module is used to adjust the first preset number of times when the difference is greater than a preset threshold.

[0141] Optionally, the overall transmission delay is calculated based on the following method:

[0142] Obtain the transmission delay of each transmission path among multiple transmission paths between the sending end and the receiving end;

[0143] The delay weight of each transmission path is calculated based on the amount of data transmitted along each transmission path and the total amount of data transmitted along the multiple transmission paths.

[0144] The overall transmission delay is calculated based on the delay weight of each transmission path and the transmission delay of each transmission path.

[0145] Optionally, the first adjustment module includes:

[0146] The first calculation unit is used to calculate the second preset number of times based on the first transmission delay, the second transmission delay, and the first preset number of times;

[0147] The first adjustment unit is used to adjust the first preset number of times to the second preset number of times.

[0148] Optionally, if multiple redundant units are obtained based on multiple adjacent I-frames of the video stream and the first preset number has a fractional part, the first sending unit includes:

[0149] The first transmitting subunit is used to transmit each of the plurality of redundant units to the receiving end according to the integer part of the first preset number of times;

[0150] The first determining subunit is used to determine the shared transmission count of the multiple redundant units based on the fractional part of the first preset number of the multiple redundant units and the number of the multiple redundant units;

[0151] The second transmitting subunit is used to transmit N redundant units from the plurality of redundant units to the receiving end according to the shared transmitting number, where N is a non-negative integer.

[0152] See Figure 7 , Figure 7 This is a structural diagram of a redundant transmission device provided in an embodiment of this application. This redundant device is applied at the receiving end, such as... Figure 7 As shown, the redundant transmission device 700 includes:

[0153] The first receiving module 701 is used to receive the first I-frame sent by the sending end;

[0154] The first processing module 702 is used to process the redundant unit according to the reception status of the redundant unit and the second I-frame sent by the transmitting end. The redundant unit is the inter-frame prediction residual image of the first I-frame and the second I-frame. The first I-frame and the second I-frame are any two adjacent I-frames in the video stream. The transmission order of the first I-frame is before the transmission order of the second I-frame.

[0155] Optionally, the first processing module includes any one of the following units:

[0156] The first discarding unit is used to discard the redundant unit if the second I-frame is correctly received;

[0157] The second discarding unit is used to discard duplicate redundant units when there is already a redundant unit identical to the redundant unit.

[0158] The first deletion unit is used to perform inter-frame decoding based on the redundancy unit and the first I-frame to obtain the second I-frame and delete the redundancy unit and the first I-frame if the second I-frame is not correctly received at the target time.

[0159] This application also provides a transmitting end. Since the principle by which the transmitting end solves the problem is similar to the redundant transmission method in this application, the implementation of this transmitting end can be found in the implementation of the method, and repeated details will not be described again. Figure 8 As shown in the embodiment of this application, the transmitting end includes: a processor 800, configured to read a program from the memory 820 and execute the following processes:

[0160] The first I-frame and the second I-frame are encoded separately, wherein the first I-frame and the second I-frame are any two adjacent I-frames in the video stream, and the transmission order of the first I-frame is before the transmission order of the second I-frame.

[0161] The encoded second I-frame is used as the reference frame for inter-frame prediction to obtain the inter-frame prediction residual image.

[0162] The processor 800 is used to read the program from the memory 820 and execute the following process via the transceiver 810:

[0163] After the first I-frame is sent to the receiving end, the second I-frame and the redundant unit are sent to the receiving end through multiple transmission paths between the sending end and the receiving end, using the inter-frame prediction residual image as a redundancy unit.

[0164] Among them, Figure 8 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 800) and memory (memory 820). The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 810 can be multiple elements, including transmitters and transceivers, providing a unit for communicating with various other devices over a transmission medium. The processor 800 is responsible for managing the bus architecture and general processing, and the memory 820 can store data used by the processor 800 during operation.

[0165] Optionally, the processor 800 is configured to read the program from the memory 820 and execute the following processes:

[0166] Send the redundant unit a first preset number of times to the receiving end;

[0167] After sending the redundant unit to the receiving end a first preset number of times, the method further includes:

[0168] Calculate the difference between the first transmission delay and the second transmission delay. The first transmission delay is the overall transmission delay of multiple transmission paths between the sender and the receiver before sending the redundant unit to the receiver a first preset number of times. The second transmission delay is the overall transmission delay of multiple transmission paths between the sender and the receiver after sending the redundant unit to the receiver a first preset number of times.

[0169] If the difference is greater than a preset threshold, the first preset number of times is adjusted.

[0170] Optionally, the overall transmission delay is calculated based on the following method:

[0171] Obtain the transmission delay of each transmission path among multiple transmission paths between the sending end and the receiving end;

[0172] The delay weight of each transmission path is calculated based on the amount of data transmitted along each transmission path and the total amount of data transmitted along the multiple transmission paths.

[0173] The overall transmission delay is calculated based on the delay weight of each transmission path and the transmission delay of each transmission path.

[0174] Optionally, the processor 800 is configured to read the program from the memory 820 and execute the following processes:

[0175] Calculate the second preset number of times based on the first transmission delay, the second transmission delay, and the first preset number of times;

[0176] Adjust the first preset number of times to the second preset number of times.

[0177] Optionally, if multiple redundant units are obtained based on multiple adjacent I-frames of the video stream and the first preset number has a fractional part, the processor 800 is configured to read the program in the memory 820 and execute the following process: via transceiver 810:

[0178] Each of the plurality of redundant units is sent to the receiving end according to the integer part of the first preset number of times;

[0179] The number of times the multiple redundant units are shared is determined based on the fractional part of the first preset number of times the multiple redundant units are used and the number of the multiple redundant units.

[0180] N redundant units from the plurality of redundant units are sent to the receiving end according to the shared transmission count, where N is a non-negative integer.

[0181] This application also provides a receiving end. Since the principle behind the problem-solving approach of the receiving end is similar to the redundant transmission method in this application, the implementation of this receiving end can be found in the implementation of the method, and repeated details will not be elaborated further. Figure 9 As shown in the embodiment of this application, the transmitting end includes: a processor 900, configured to read a program from a memory 920 and execute the following process: via transceiver 910:

[0182] Receive the first I-frame sent by the sending end;

[0183] The processor 900 is used to read the program in the memory 920 and execute the following processes:

[0184] The redundant units are processed according to the reception status of the redundant units and the second I-frame sent by the transmitting end. The redundant units are the inter-frame prediction residual images of the first I-frame and the second I-frame. The first I-frame and the second I-frame are any two adjacent I-frames in the video stream. The transmission order of the first I-frame is before the transmission order of the second I-frame.

[0185] Among them, Figure 9 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 900) and memory (memory 920). The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 910 can be multiple elements, including transmitters and transceivers, providing a unit for communicating with various other devices over a transmission medium. The processor 900 is responsible for managing the bus architecture and general processing, and the memory 920 can store data used by the processor 900 during operation.

[0186] Optionally, the processor 900 is configured to read the program from the memory 920 and execute the following processes:

[0187] If the second I-frame is received correctly, the redundant unit is discarded;

[0188] If a redundant unit identical to the redundant unit already exists, the duplicate redundant unit is discarded;

[0189] If the second I-frame is not correctly received at the target time, inter-frame decoding is performed based on the redundant unit and the first I-frame to obtain the second I-frame, and the redundant unit and the first I-frame are deleted.

[0190] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the redundant transmission method embodiments described above and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0191] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-described... Figure 2 or Figure 4The various processes of the method embodiments shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0192] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0193] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0194] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A redundant transmission method, characterized in that, Applied to the sending end, the method includes: The first I-frame and the second I-frame are encoded separately, wherein the first I-frame and the second I-frame are any two adjacent I-frames in the video stream, and the transmission order of the first I-frame is before the transmission order of the second I-frame. The encoded second I-frame is used as the reference frame for inter-frame prediction to obtain the inter-frame prediction residual image. After the first I-frame is sent to the receiving end, the second I-frame and the redundant unit are sent to the receiving end through multiple transmission paths between the sending end and the receiving end, using the inter-frame prediction residual image as a redundancy unit.

2. The redundant transmission method according to claim 1, characterized in that, Sending the redundant unit to the receiving end includes: Send the redundant unit a first preset number of times to the receiving end; After sending the redundant unit to the receiving end a first preset number of times, the method further includes: Calculate the difference between the first transmission delay and the second transmission delay. The first transmission delay is the overall transmission delay of multiple transmission paths between the sender and the receiver before sending the redundant unit to the receiver a first preset number of times. The second transmission delay is the overall transmission delay of multiple transmission paths between the sender and the receiver after sending the redundant unit to the receiver a first preset number of times. If the difference is greater than a preset threshold, the first preset number of times is adjusted.

3. The redundant transmission method according to claim 2, characterized in that, The overall transmission delay is calculated based on the following method: Obtain the transmission delay of each transmission path among multiple transmission paths between the sending end and the receiving end; The delay weight of each transmission path is calculated based on the amount of data transmitted along each transmission path and the total amount of data transmitted along the multiple transmission paths. The overall transmission delay is calculated based on the delay weight of each transmission path and the transmission delay of each transmission path.

4. The redundant transmission method according to claim 2, characterized in that, The adjustment of the first preset number of times includes: Calculate the second preset number of times based on the first transmission delay, the second transmission delay, and the first preset number of times; Adjust the first preset number of times to the second preset number of times.

5. The redundant transmission method according to claim 2, characterized in that, When multiple redundant units are obtained based on multiple adjacent I-frames of the video stream and the first preset number of times has a fractional part, the step of sending the first preset number of redundant units to the receiving end includes: Each of the plurality of redundant units is sent to the receiving end according to the integer part of the first preset number of times; The number of times the multiple redundant units are shared is determined based on the fractional part of the first preset number of times the multiple redundant units are used and the number of the multiple redundant units. N redundant units from the plurality of redundant units are sent to the receiving end according to the shared transmission count, where N is a non-negative integer.

6. A redundant transmission method, characterized in that, Applied to the receiving end, the method includes: Receive the first I-frame sent by the sending end; The redundant units are processed according to the reception status of the redundant units and the second I-frame sent by the transmitting end. The redundant units are the inter-frame prediction residual images of the first I-frame and the second I-frame. The first I-frame and the second I-frame are any two adjacent I-frames in the video stream. The transmission order of the first I-frame is before the transmission order of the second I-frame.

7. The redundant transmission method according to claim 6, characterized in that, The process of processing the redundant units based on the redundancy units sent by the transmitting end and the reception status of the second I-frame includes any of the following: If the second I-frame is received correctly, the redundant unit is discarded; If a redundant unit identical to the redundant unit already exists, the duplicate redundant unit is discarded; If the second I-frame is not correctly received at the target time, inter-frame decoding is performed based on the redundant unit and the first I-frame to obtain the second I-frame, and the redundant unit and the first I-frame are deleted.

8. A redundant transmission device, characterized in that, Applied to the transmitting end, the device includes: The first encoding module is used to encode the first I-frame and the second I-frame respectively, wherein the first I-frame and the second I-frame are any two adjacent I-frames in the video stream, and the transmission order of the first I-frame is before the transmission order of the second I-frame. The first prediction module is used to perform inter-frame prediction on the encoded second I-frame with the encoded first I-frame as the reference frame to obtain the inter-frame prediction residual image. The first transmitting module is configured to, after transmitting the first I-frame to the receiving end, use the inter-frame prediction residual image as a redundancy unit to transmit the second I-frame and the redundancy unit to the receiving end through multiple transmission paths between the transmitting end and the receiving end.

9. A redundant transmission device, characterized in that, Applied to the receiving end, the device includes: The first receiving module is used to receive the first I-frame sent by the sending end; The first processing module is used to process the redundant unit according to the reception status of the redundant unit and the second I-frame sent by the sending end. The redundant unit is the inter-frame prediction residual image of the first I-frame and the second I-frame. The first I-frame and the second I-frame are any two adjacent I-frames in the video stream. The transmission order of the first I-frame is before the transmission order of the second I-frame.

10. A transmitter, characterized in that, The transmitting end includes a transceiver and a processor, the processor being used for: The first I-frame and the second I-frame are encoded separately, wherein the first I-frame and the second I-frame are any two adjacent I-frames in the video stream, and the transmission order of the first I-frame is before the transmission order of the second I-frame. The encoded second I-frame is used as the reference frame for inter-frame prediction to obtain the inter-frame prediction residual image. The transceiver is used for: After the first I-frame is sent to the receiving end, the second I-frame and the redundant unit are sent to the receiving end through multiple transmission paths between the sending end and the receiving end, using the inter-frame prediction residual image as a redundancy unit.

11. A receiving end, characterized in that, The receiving end includes a transceiver and a processor, the transceiver being used for: Receive the first I-frame sent by the sending end; The processor is used for: The redundant units are processed according to the reception status of the redundant units and the second I-frame sent by the transmitting end. The redundant units are the inter-frame prediction residual images of the first I-frame and the second I-frame. The first I-frame and the second I-frame are any two adjacent I-frames in the video stream. The transmission order of the first I-frame is before the transmission order of the second I-frame.

12. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, it implements the steps of the redundant transmission method as described in any one of claims 1 to 5, or when the computer program is executed by the processor, it implements the steps of the redundant transmission method as described in any one of claims 6 to 7.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the redundant transmission method as described in any one of claims 1 to 5, or when executed by the processor, implements the steps of the redundant transmission method as described in any one of claims 6 to 7.

14. A computer program product, characterized in that, The method includes computer instructions that, when executed by a processor, implement the steps of the redundant transmission method as described in any one of claims 1 to 5, and when executed by a processor, implement the steps of the redundant transmission method as described in any one of claims 6 to 7.

Images