Forward error correction coding method, decoding method, storage medium and electronic device

Abstract

The embodiment of the invention provides a forward error correction coding method, a decoding method, a storage medium and an electronic device, and the method comprises the steps: segmenting a to-be-transmitted video data block into a plurality of data packets according to the size of a preset data packet, and determining the code length of the plurality of data packets, determining the redundancy of the coded video data block in the next time period according to the code lengths of the plurality of data packets and the predicted packet loss rate in the next time period, and determining a target generation matrix from a plurality of pre-constructed generation matrixes according to the code lengths and the redundancy of the plurality of data packets, and performing forward error correction coding on the plurality of data packets according to the target generation matrix to obtain a plurality of coding packets. According to the embodiment of the invention, the problem of poor channel fluctuation adaptability of forward error correction coding and decoding in related technologies can be solved, the coding redundancy is determined by predicting the packet loss rate, the channel fluctuation condition can be better adapted, and the redundancy of overall sent data is reduced.

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular, to a forward error correction encoding method, a decoding method, a storage medium and an electronic device. Background Art

[0002] Ultra-high-definition video is a new round of major technological innovation following the digitization and high-definition of video, which will lead to profound changes in all links of the industry chain, including video acquisition, production, transmission, presentation, and application. In terms of video coding and decoding, the Audio Video Coding Standard (AVS) Working Group has completed the formulation of the AVS3 benchmark level of my country's third-generation national digital video coding and decoding technology standard, indicating the importance of high-definition real-time video transmission technology to social development.

[0003] However, packet loss or bit errors caused by poor network environment will have a serious impact on video quality. For example, wireless channels cause bit errors due to multipath fading, Doppler effect and the presence of noise; wired channels cause packet loss due to network congestion caused by limited buffer. For the former, there are well-known channel coding methods to solve the bit error problem of the wireless physical layer; for the latter, the existing solutions mainly include forward error correction coding (FEC) or retransmission technology. Retransmission is one of the most effective technologies to resist packet loss, but retransmission at a high round-trip time (RTT) will increase the cumulative delay at the receiving end, thereby increasing the entire end-to-end picture delay.

[0004] There are still many problems in the existing forward error correction coding schemes: (1) The redundancy is obtained by table lookup, which has an upper limit on the coding rate and cannot adapt to higher packet loss environments, and has poor adaptability to channel fluctuations; (2) The forward error correction method based on XOR check is too inefficient in error correction protection, and the coding efficiency needs to be improved; (3) Coding long codes will cause a lot of computational overhead, and the inefficient coding rate adaptation method will produce too many redundant packets in the transmission link, which may affect other high-definition videos under limited bandwidth conditions, and lead to increased overall transmission loss and poor user viewing experience; (4) The characteristics of video data packets are not taken into account, and the relationship between data redundancy and network quality cannot be accurately described, making it difficult to effectively balance redundancy and video quality.

[0005] In summary, there is no good solution to the problem that the forward error correction coding code in the related art has poor adaptability to channel fluctuations. Summary of the invention

[0006] The embodiments of the present application provide a forward error correction encoding method, a decoding method, a storage medium and an electronic device to at least solve the problem of poor channel fluctuation adaptability of forward error correction coding in the related art.

[0007] According to one embodiment of the present application, a forward error correction coding method is provided, which is applied to a transmitting end, and the method includes: dividing a video data block to be sent into multiple data packets according to a preset data packet size, and determining the code length of the multiple data packets, wherein the code length of the multiple data packets is the number of the multiple data packets; determining the redundancy of the encoded video data block in the next time period according to the code length of the multiple data packets and the predicted packet loss rate in the next time period; determining a target generation matrix from multiple pre-constructed generation matrices according to the code length of the multiple data packets and the redundancy; performing forward error correction coding on the multiple data packets according to the target generation matrix to obtain multiple coding packets, wherein the number of the multiple coding packets is greater than the number of the multiple data packets.

[0008] According to another embodiment of the present application, a forward error correction decoding method is provided, which is applied to a receiving end, and the method includes: performing Gaussian elimination decoding on part or all of the multiple coding packets of the sending end to obtain multiple data packets, wherein the multiple coding packets are obtained by the sending end dividing the video data blocks to be sent into the multiple data packets according to a preset data packet size, and determining the code length of the multiple data packets, determining the redundancy of the encoded video data blocks in the next time period according to the code length of the multiple data packets and the predicted packet loss rate in the next time period, determining a target generating matrix from a plurality of pre-constructed generating matrices according to the code length of the multiple data packets and the redundancy, and performing forward error correction encoding on the multiple data packets according to the target generating matrix, wherein the number of the multiple coding packets is greater than the number of the multiple data packets, and the code length of the multiple data packets is the number of the multiple data packets; and reorganizing the multiple data packets to obtain the video data block.

[0009] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program executes the steps of any of the above method embodiments when executed by a processor.

[0010] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0011] Through the embodiments of the present application, by predicting the packet loss rate, the coding redundancy is adaptively adjusted according to the channel fluctuation situation without limiting the coding code length, which can solve the problem of poor channel fluctuation adaptability of the forward error correction coding code in the related technology, and can better adapt to the channel fluctuation situation and reduce the redundancy of the overall transmitted data. At the same time, the forward error correction coding also has the characteristic of being able to effectively resist data packet loss and reduce the picture delay of video transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a hardware structure block diagram of the forward error correction encoding and decoding method of the embodiment of the present application;

[0013] Figure 2 is a flowchart of a forward error correction coding method according to an embodiment of the present application;

[0014] Figure 3 is a flowchart of a forward error correction decoding method according to an embodiment of the present application;

[0015] Figure 4 is a schematic diagram of video frame level segmentation according to an embodiment of the present application;

[0016] Figure 5 is a schematic diagram of receiving end redundancy according to an embodiment of the present application;

[0017] Figure 6 is a schematic diagram of sub-packet coding according to an embodiment of the present application;

[0018] Figure 7 is a block diagram of a forward error correction encoding and decoding device according to an embodiment of the present application;

[0019] Figure 8 It is a flowchart of forward error correction coding according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0022] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 is a hardware structure block diagram of the forward error correction encoding and decoding method of the embodiment of the present application, such as Figure 1 As shown, the hardware board may include one or more ( Figure 1Only one is shown in the figure) a processor 12 (the processor 12 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device) and a memory 14 for storing data, wherein the mobile terminal may also include a transmission device 16 for communication functions and an input and output device 18. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0023] The memory 14 can be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the forward error correction coding and decoding methods in the embodiments of the present application. The processor 12 executes various functional applications and forward error correction coding and decoding methods by running the computer programs stored in the memory 14, that is, implementing the above-mentioned methods. The memory 14 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 14 may further include a memory remotely arranged relative to the processor 12, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0024] The transmission device 16 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider. In one example, the transmission device 16 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 16 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0025] In one embodiment of the present application, a forward error correction coding method is provided, which is applied to a transmitting end. Figure 2 is a flowchart of a forward error correction coding method according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0026] Step S202, dividing the video data block to be sent into a plurality of data packets according to a preset data packet size, and determining the code length of the plurality of data packets, wherein the code length of the plurality of data packets is the number of the plurality of data packets;

[0027] Step S204, determining the redundancy of the encoded video data blocks in the next time period according to the code lengths of the multiple data packets and the predicted packet loss rate in the next time period;

[0028] Step S206, determining a target generator matrix from a plurality of pre-constructed generator matrices according to the code lengths of the plurality of data packets and the redundancy;

[0029] Step S208, performing forward error correction encoding on the multiple data packets according to the target generation matrix to obtain multiple coded packets, wherein the number of the multiple coded packets is greater than the number of the multiple data packets.

[0030] In an embodiment of the present application, through the above-mentioned steps S202 to S208, it is possible to better adapt to channel fluctuations, adaptively adjust coding redundancy without limiting the video encoding code length, perform forward error correction coding while ensuring successful data transmission decoding, and reduce the redundancy of overall data transmission and the amount of resources occupied by video transmission.

[0031] In some embodiments, before step S202, the method further includes: step S201, according to a preset data block size interval and a preset upper limit of the number of data frames, the video data to be sent is frame-level blocked to obtain a plurality of the video data blocks, wherein the size of each of the video data blocks is within the preset data block size interval, the number of data frames contained in each of the video data blocks is less than or equal to the preset upper limit of the number of data frames, and the preset data block size interval is determined based on the product of a preset code length range and the preset data packet size.

[0032] In this embodiment, each data block also includes a coding method and / or a block size (unit: byte). For example, the block size can be placed in the header of the data block as basic information of the block.

[0033] In this embodiment, the upper limit of the number of preset data frames in step S201 is K. frame , reorganize the frames into blocks, one block contains one frame or multiple frames, the encoding and decoding process of the application layer FEC is based on the entire block, and the entire video data of the block can be obtained after successful decoding.

[0034] In some embodiments, the preset code length range in step S201 is an optimal code length interval with higher coding and decoding recovery accuracy, and the optimal code length intervals for short codes and medium-long codes can be set separately. For example, the optimal code length interval for short codes can be [1,20], and the optimal code length interval for medium-long codes can be [N min ,N max ], where the upper limit of the code length of the long code is N maxIt can be set to 100, and the optimal code length range can be set according to the actual video scene.

[0035] In an exemplary embodiment, the preset data packet size in step S201 and step S202 may be set to 1400 bytes according to the Maximum Transmission Unit (MTU).

[0036] Furthermore, the preset data block size interval corresponding to the short code can be 1×1400 bytes to 20×1400 bytes. Generally, the amount of data in B frames and P frames is small, and several consecutive B frames or P frames can be used as a block. It is only necessary to ensure that the amount of block data is not greater than 28000 bytes; the preset data block size interval corresponding to the medium and long codes can be N min ×1400 bytes to N max × 1400 bytes. Due to the coding characteristics of Low Density Parity Check Codes (LDPC), longer code lengths have better decoding performance. I / IDR frames contain a large amount of information and can be directly used as a block, or several P frames with large amounts of data can be combined into a block. This application does not limit the combination of data frames.

[0037] In this embodiment, when performing frame-level segmentation in step S201 , as many data frames as possible are combined within a preset data block size interval and a preset upper limit on the number of data frames.

[0038] In an exemplary embodiment, frame weights may also be introduced, and blocks containing frames with larger weights may be subsequently encoded using the maximum code rate.

[0039] In some embodiments, step S202 may include the following steps:

[0040] Step S2022, dividing the video data block into a plurality of data packets of equal size according to the preset data packet size;

[0041] Step S2024: Determine the number of the plurality of data packets as the code length of the plurality of data packets, wherein the code length of the plurality of data packets is within the preset code length range.

[0042] In this embodiment, the size of each data packet in step S2022 is a preset data packet size, such as 1400 bytes. If the size of the last data packet in the video data block is less than 1400 bytes, it can be padded with 0 to 1400 bytes.

[0043] In some embodiments, before step S204, the method may further include step S203, obtaining a predicted packet loss rate in a next time period, which may specifically include the following steps:

[0044] Step S2032, obtaining a first packet loss rate predicted in the previous time period through a pre-built Kalman filter;

[0045] Step S2034, obtaining an actual packet loss rate in the previous time period, and correcting the first packet loss rate according to the actual packet loss rate to obtain a corrected second packet loss rate;

[0046] Step S2036: predicting the packet loss rate in the next time period according to the second packet loss rate by using the Kalman filter to obtain the predicted packet loss rate.

[0047] In this embodiment, a Kalman filter (KF) is constructed to predict the packet loss rate by calculating the linear minimum mean square error of the data and continuously recursively. Further, the prediction process of the Kalman filter algorithm includes: using the optimal result predicted at time k-1 and the measured value at time k to calculate and update the optimal result of the state prediction at time k, and continue to iterate to obtain the predicted value at the next time.

[0048] In this embodiment, the Kalman filter consists of two processes, including prediction and correction. In the prediction stage, the filter uses the packet loss rate estimate of the previous state to make a prediction of the current state. In the correction stage, the filter uses the measured value of the current state (actual packet loss rate) to correct the predicted value (first packet loss rate) obtained in the prediction stage, that is, the feedback value of the packet loss rate, to obtain a new estimated value (second packet loss rate) that is closer to the true value. This new estimated value is the Kalman estimated value, and the next time the Kalman estimation is performed, the estimated value is used as the estimate of the previous state. In the case where modeling cannot be achieved due to the uncertainty of process noise and measurement noise, the mean square error between the true value and the estimated value can be minimized by continuously correcting the estimation model through the Kalman filter.

[0049] In some embodiments, step S204 may include the following steps:

[0050] Step S2042, when the code lengths of the plurality of data packets are within a first preset code length interval, determining the redundancy according to the code lengths of the plurality of data packets and the predicted packet loss rate;

[0051] Step S2044, when the code length of the multiple data packets is within the second preset code length interval, determine the redundancy ratio of the receiving end in the next time period according to the pre-established redundancy cost function of the receiving end, and determine the redundancy according to the predicted packet loss rate and the redundancy ratio.

[0052] In this embodiment, the preset code length range includes a first preset code length interval and a second preset code length interval, the first preset code length interval is an optimal code length interval for short codes, and the second preset code length interval is an optimal code length interval for medium and long codes. Step S204 performs adaptive adjustment according to the code lengths of the plurality of data packets, and determines redundancy for the short codes and the medium and long codes respectively.

[0053] In some embodiments, step S2042 is for short codes. According to the coding characteristics of the short codes in the present application, the receiving end only needs to receive any k coded packets to achieve correct decoding. Therefore, the relationship between the code length k, predicted packet loss rate c and redundancy r of multiple data packets is: c=r / (k+r).

[0054] In some embodiments, the redundancy cost function of the receiving end in step S2044 is:

[0055] Cost t =[Q(k,rm)-δ·P rtt (k,rm)] 2 ;

[0056] Q(k,rm)=(rm) / k;

[0057] Among them, Cost t is the receiving end redundancy cost of the t-th video data block, Q(k,rm) is the redundancy ratio, k is the code length of the multiple data packets, r is the redundancy, m is the number of packet losses, P rtt (k, rm) is the decoding failure rate corresponding to the redundancy ratio, and δ is the weight of the decoding failure item.

[0058] In some embodiments, the redundancy may be determined in step S2044 according to the following formula:

[0059]

[0060] Wherein, r is the redundancy, k is the code length of the plurality of data packets, r rec is the redundancy ratio, and c is the predicted packet loss rate.

[0061] In the embodiment of the present application, the packet loss rate in a future time period can be predicted through a multi-step Kalman filter, and the coding redundancy is obtained by combining the receiving end redundancy ratio, thereby ensuring that the sending redundancy can match the channel packet loss rate, and actively solving the problem of packet loss rate and redundancy adaptation.

[0062] In some embodiments, before step S206, the method may further include step S205 of constructing a generator matrix. The generator matrix includes a Reed-Solomon code (RS) matrix and a Low Density Parity Check Codes (LDPC) matrix, and the element value of the generator matrix is ​​0 or 1.

[0063] In some embodiments, step S205 may include the following steps:

[0064] Step S2052, traversing code length values ​​within a preset code length range and redundancy values ​​within a preset redundancy range, wherein the preset code length range includes a first preset code length interval and a second preset code length interval;

[0065] Step S2054, when the code length values ​​of the plurality of data packets are within the first preset code length interval, constructing a plurality of Reed-Solomon code RS matrices based on the respective redundancy values ​​within the preset redundancy range, the respective code length values ​​of the first preset code length interval, and the Vandermonde matrix;

[0066] Step S2056, when the code length values ​​of the multiple data packets are within the second preset code length interval, a low-density parity-check code LDPC matrix is ​​constructed based on the various redundancy values ​​within the preset redundancy range, the various code length values ​​of the second preset code length interval, and a progressive edge growth algorithm.

[0067] In some embodiments, the element value of the RS matrix is ​​typically [0,255], the Galois Field (GF, also known as the computational field, finite field) is GF(2^8), and the element value of the LDPC matrix is ​​[0,1], and the computational field is GF(2).

[0068] In other embodiments, in order to reduce the computational complexity of RS coding and unify the coding and decoding processes of RS coding and LDPC coding, it is necessary to transform and expand the computational domain of the RS coding matrix from GF(2^8) to GF(2).

[0069] Furthermore, each element value in [0,255] can correspond to an 8×8 [0,1] matrix. The 8×8 [0,1] matrix can be used to transform the n×k [0,255] matrix into an 8n×8k matrix [0,1], where n is the total code length of the encoded packet, which is also the sum of the code length k and redundancy r of multiple data packets.

[0070] In this embodiment, the physical layer coding characteristics of LDPC and RS are utilized to establish an application layer forward error correction coding scheme based on the two coding methods. On the basis of the LDPC-PEG algorithm, an LDPC check matrix with high-performance packet loss compensation is designed. At the same time, by expanding the calculation domain of the RS coding matrix, the encoding and decoding processes of short codes and medium and long codes are unified, which reduces the computational complexity and enhances the scalability.

[0071] In some embodiments, step S206 may include the following steps:

[0072] Step S2062, when the code lengths of the plurality of data packets are within the first preset code length interval, determining an RS matrix corresponding to the code lengths of the plurality of data packets and the redundancy as the target generation matrix;

[0073] Step S2064: When the code lengths of the plurality of data packets are within the second preset code length interval, an LDPC matrix corresponding to the code lengths of the plurality of data packets and the redundancy is determined as the target generation matrix.

[0074] In the embodiment of the present application, the generator matrix used for encoding can be constructed in advance through step S205, and the generator matrix corresponding to the code length and redundancy can be directly called through step S206, thereby avoiding time consumption due to the generator matrix during the encoding process.

[0075] In some embodiments, step S208 may include the following steps:

[0076] Step S2082, when the code lengths of the multiple data packets are within the first preset code length interval, forward error correction encoding is performed on each byte data in the multiple data packets according to the target generation matrix to obtain the multiple coded packets, wherein each byte data includes 8 bits of data;

[0077] Step S2084, when the code length of the multiple data packets is within the second preset code length interval, forward error correction encoding is performed on each bit data in the multiple data packets according to the target generation matrix to obtain the multiple coded packets.

[0078] In some embodiments, step S2082 may include: performing forward error correction encoding on the multiple data packets according to the following formula to obtain the multiple encoded packets:

[0079] G 8n×8k ×S 8k =Sp 8n Among them, G 8n×8k is the target generation matrix, n is the number of the plurality of coded packets, k is the number of the plurality of data packets, S 8kis the byte data sequence of k packets, Sp 8n is a byte data sequence of n coded packets, wherein the element value of the byte data sequence is 0 or 1.

[0080] In some embodiments, step S2084 may include: performing forward error correction encoding on the multiple data packets according to the following formula to obtain the multiple encoded packets:

[0081] G n×k ×S k =Sp n Among them, G n×k Generate the matrix for the target, S k is the bit data sequence of k packets, Sp n is a bit data sequence of n coded packets, and the element value of the bit data sequence is 0 or 1.

[0082] In some embodiments, after step S208, the method further includes step S209 to encapsulate and send the encoded packet.

[0083] In some embodiments, step S209 may include the following steps:

[0084] Step S2092, numbering the plurality of encoding packets in sequence to obtain a serial number of each encoding packet;

[0085] Step S2094, determining the row information corresponding to the sequence number in the target generation matrix as protection information of the corresponding coded packet;

[0086] Step S2096, encapsulating the multiple coded packets using packet header information to obtain multiple encapsulated coded packets, wherein the packet header information includes: code lengths of the multiple data packets, the sequence numbers, and the protection information;

[0087] Step S2098, sending the multiple encapsulated coding packets.

[0088] In the embodiment of the present application, an efficient coding method based on the application layer is designed, which can reduce the probability of retransmission as much as possible after determining the redundancy. The embodiment of the present application can cope with continuous burst packet loss and different application scenarios, and can reduce the computational complexity of coding while covering various code lengths.

[0089] In some embodiments, the above-mentioned forward error correction coding method can be applied to video transmission of virtual reality technology (Virtual Reality, VR for short). With the rapid growth of VR video data volume and the surge in sales of head-mounted display devices, VR video will become mainstream in the future. Since VR video usually has a large amount of data, the encoding scheme in the embodiment of the present application can provide long code protection for transmission. The method of adaptively allocating redundancy in the above-mentioned method embodiment can also be combined with the viewport prediction (Viewport Predict) of the VR video to adaptively allocate the encoding bit rate according to the viewport prediction.

[0090] In yet another embodiment of the present application, a forward error correction decoding method is provided, which is applied to a receiving end.

[0091] Figure 3 is a flowchart of a forward error correction decoding method according to an embodiment of the present application, such as Figure 3 As shown, the process includes the following steps:

[0092] Step S302, performing Gaussian elimination decoding on part or all of the multiple coded packets of the transmitting end to obtain multiple data packets;

[0093] Step S304: reassemble the multiple data packets to obtain video data blocks.

[0094] In this embodiment, the multiple coding packets in step S302 are obtained by the transmitter dividing the video data block to be sent into the multiple data packets according to a preset data packet size, determining the code length of the multiple data packets, determining the redundancy of the encoded video data block in the next time period according to the code length of the multiple data packets and the predicted packet loss rate in the next time period, determining the target generation matrix from a plurality of pre-constructed generation matrices according to the code length of the multiple data packets and the redundancy, and performing forward error correction encoding on the multiple data packets according to the target generation matrix, the number of the multiple coding packets is greater than the number of the multiple data packets, and the code length of the multiple data packets is the number of the multiple data packets.

[0095] In traditional linear block codes, it is usually necessary to wait until all data is received before decoding can be performed. Once decoding fails, the entire data needs to be retransmitted, which leads to a large amount of delay. Through the embodiment of the present application, decoding is performed in the manner of Gaussian Elimination (GE) decoding, without waiting for all data to be received. The receiving end can only perform Gaussian elimination decoding on part of the received coded packets to obtain the encoding results of all multiple data packets. Even if data retransmission is required, only a small number of lost coded packets need to be retransmitted, reducing the transmission delay caused by overall retransmission.

[0096] In some embodiments, step S302 may include the following steps:

[0097] Step S3022: each time a packaged code packet sent by the transmitting end is received, the packaged code packet is parsed to obtain a corresponding code packet and row information of the target generator matrix, and a coefficient matrix is ​​updated according to the row information of the target generator matrix;

[0098] Step S3024, each time the coefficient matrix is ​​updated, Gaussian elimination decoding is performed on part or all of the coded packets according to the coefficient matrix, and the multiple data packets are obtained when Gaussian elimination decoding is successful, wherein part or all of the coded packets are coded packets that have been received and parsed by the receiving end.

[0099] The embodiment of the present application establishes a decoding scheme with fountain code characteristics, which is performed in a receiving and decoding manner, which can reduce the decoding delay at the receiving end, improve the real-time performance of video transmission, and enhance the user viewing experience.

[0100] In some embodiments, performing Gaussian elimination decoding on part or all of the multiple coded packets of the transmitter in step S3024 to obtain multiple data packets may include the following steps:

[0101] Step S3024-2: each time one of the encapsulated coding packets is received, the encapsulated coding packet is parsed to obtain a coding packet and corresponding protection information, wherein the encapsulated coding packet is obtained by encapsulating any one of the multiple coding packets using packet header information, the packet header information includes code lengths of the multiple data packets, the serial number of the coding packet, and the protection information, and the protection information is row information corresponding to the serial number in the target generation matrix;

[0102] Step S3024-4: each time a piece of protection information is obtained, the coefficient matrix is ​​updated according to the protection information, and Gaussian elimination decoding is performed on part or all of the coded packets according to the coefficient matrix;

[0103] Step S3024-6, when the coefficient matrix after Gaussian elimination decoding meets the preset conditions, determine that Gaussian elimination decoding is successful and obtain the multiple data packets;

[0104] Step S3024-8, when the coefficient matrix after Gaussian elimination decoding does not meet the preset condition, continue to receive and parse new encapsulated coding packets until Gaussian elimination decoding is successful and the multiple data packets are obtained.

[0105] In some embodiments, each time the protection information is obtained in step S3024-4, updating the coefficient matrix according to the protection information may include: each time the protection information is obtained, inserting the protection information as the first row of the coefficient matrix into the top of the coefficient matrix, wherein the total number of rows of the coefficient matrix is ​​equal to the number of encapsulated coding packets received.

[0106] Furthermore, the protection information carries row information corresponding to the generator matrix. Each row of the coefficient matrix corresponds to a received coding packet. Since packets may be lost during data transmission, the coefficient matrix will lack some row information compared to the original generator matrix.

[0107] In some embodiments, performing Gaussian elimination decoding on part or all of the coding packets according to the coefficient matrix in step S3024-4 may include: for each new row of the coefficient matrix, determining the other rows in the coefficient matrix except the first row as target rows in sequence, and using the first row of the coefficient matrix to perform Gaussian elimination on the target row to step the coefficient matrix; in the Gaussian elimination process, performing an exclusive OR (XOR) operation on the coding packet corresponding to the first row and the coding packet corresponding to the target row.

[0108] Furthermore, the Gaussian elimination decoding process in the embodiment of the present application is performed iteratively in sequence. When the next encapsulated coding packet is received, it is necessary to continue decoding based on the decoding result of the previously received encapsulated coding packet (including the coefficient matrix and the corresponding coding packet sequence after XOR operation). Through Gaussian elimination decoding, the receiving end only needs to receive any k coding packets from multiple coding packets to realize the decoding of the video data block with a code length of k, where the video data block contains k data packets.

[0109] In some embodiments, the preset condition in step S3024-6 and step S3024-8 is that the coefficient matrix is ​​full rank, that is, the coefficient matrix after Gaussian elimination decoding is a stepped unit matrix.

[0110] In some embodiments, step S3024-6 may include: when the coefficient matrix after Gaussian elimination decoding is a unit matrix, determining that Gaussian elimination decoding is successful, and determining the coded packets processed by XOR operation corresponding to the unit matrix as the multiple data packets.

[0111] In some embodiments, after step S3024, the method further includes: determining that decoding has failed when the coefficient matrix after Gaussian elimination decoding does not satisfy the preset condition and new encapsulated coding packets cannot continue to be received; when decoding has failed, determining the lost sequence number based on the sequence numbers of some or all of the coding packets; and feeding back the lost sequence number to the sending end so that the sending end resends the corresponding encapsulated coding packet based on the lost sequence number.

[0112] In the embodiment of the present application, only the lost coded packets need to be retransmitted, which can greatly shorten the retransmission delay and improve the user's video viewing experience.

[0113] Figure 4 is a schematic diagram of video frame level segmentation according to an embodiment of the present application, such as Figure 4 As shown, each video data block consists of several frame data.

[0114] In this embodiment, the video is divided into blocks according to frame data, and one video data block includes one or several frames, thereby ensuring that the FEC encoding and decoding is synchronized with the video timestamp as much as possible.

[0115] In this embodiment, without considering the requirements of the video scene for the coding block, the optimal interval of the short code and the medium and long code and the upper limit of the number of frames in a block can be used for block division, and the optimal interval refers to the interval code length under which the encoding and decoding recovery correctness is high. The sending end obtains the video frame information by calling the ffmpeg library tool, but the present application is not limited to this.

[0116] Furthermore, the upper limit of the number of data frames is K frame The optimal code length interval for short codes can be [1,20], and the optimal code length interval for medium and long codes can be expressed as [N min ,N max ].

[0117] In this embodiment, the encoding and decoding process of the application layer FEC is performed based on the entire video data block, and the entire video data of a block can be obtained if the decoding is successful.

[0118] In this embodiment, the encoding method is divided into two cases: short code and medium-long code according to different information bit code lengths (i.e., the number of data packets in each video data block). The short code adopts the RS encoding and decoding method, and the medium-long code adopts the LDPC encoding and decoding method. In order to ensure high accuracy and low latency characteristics of decoding, both short codes and medium-long codes should try to select the optimal interval, and the optimal interval is selected according to the encoding and decoding characteristics of RS codes and LDPC codes.

[0119] In this embodiment, each video data block is divided into coding information source packets (i.e., data packets). We set the coding information source packet size to 1400 bytes according to the MTU. For short codes, the block information bit size ranges from 1×1400 bytes to 20×1400 bytes. Generally, the data volume of B frames and P frames is small. We take several consecutive B frames or P frames as a block. We only need to ensure that the block data volume is not greater than 28,000 bytes and the number of frames in the block is not greater than K. frame .

[0120] In some embodiments, the data block header is basic information of the data block, including encoding method and block size (unit: byte).

[0121] In some embodiments, the optimal interval selection of medium-length codes is more flexible. Due to the characteristics of LDPC coding, longer code lengths have better decoding performance. In addition, different code lengths and coding rates correspond to different LDPC generation matrices, so the complexity of the storage space occupied by the LDPC generation matrix is ​​O(n 2 ), we can set the upper limit of the long code interval N max Set to 100. For high-bitrate video frames, I / IDR frames contain a large amount of information and can be used as a block. We can also combine several P frames with large data volumes into a block. Regardless of the block grouping method, the number of frames should not exceed K when long code blocks are used. frame And the block data volume is not greater than N max ×1400, take the larger value.

[0122] Furthermore, frame weights may be introduced, and maximum bit rate encoding may be used for data blocks containing frames with larger weights.

[0123] In another embodiment of the present application, a packet loss rate prediction module based on a multi-step Kalman filter system is also established. The prediction is performed by calculating the linear minimum mean square error of the data and continuously recursively. The prediction process of the algorithm: the optimal result of the prediction at time k-1 and the measured value at time k are used to calculate and update the optimal result of the state prediction at time k, and continue to iterate to obtain the predicted value at the next moment.

[0124] In this embodiment, a state space equation is first determined to estimate the channel packet loss rate state;

[0125] The state prediction equation is: k =Ac k-1 +Bu k +w k , where A is the state transfer matrix, c k-1 is the last predicted value, B is the input gain matrix, u k System input vector. w k is process noise, specifically, the mean is 0, the covariance matrix And the process noise obeys the normal distribution.

[0126] The state measurement equation is: k =c k +v k , where v k is the measurement noise, specifically the mean is 0, and the covariance matrix is And the measurement noise obeys the normal distribution.

[0127] In this embodiment, the Kalman filter consists of two processes, including prediction and correction. In the prediction stage, the filter uses the packet loss rate estimate of the previous state to make a prediction of the current state. In the correction stage, the filter uses the measured value of the current state (actual packet loss rate) to correct the predicted value (first packet loss rate) obtained in the prediction stage, that is, the feedback value of the packet loss rate, to obtain a new estimated value (second packet loss rate) that is closer to the true value. This new estimated value is the Kalman estimated value, and the next time the Kalman estimation is performed, the estimated value is used as the estimate of the previous state. In the case where modeling cannot be achieved due to the uncertainty of process noise and measurement noise, the mean square error between the true value and the estimated value can be minimized by continuously correcting the estimation model through the Kalman filter.

[0128] In some embodiments, the main steps of packet loss rate prediction using a Kalman filter are as follows:

[0129] Step 1,

[0130] Step 2,

[0131] Step 3,P(k)=(1-Kg(k))(P(k-1)+W);

[0132] in, is the error variance of the Kalman filter model, Kg(k) is the Kalman gain, is the final predicted value, It is the state prediction at time k-1 for time k.

[0133] In this embodiment, we obtain a packet loss rate under a time scale through the Kalman filter, which is an important parameter for subsequent adaptive redundancy calculation. By predicting the packet loss rate, redundancy can better adapt to channel fluctuations in a weak network environment.

[0134] Figure 5 is a schematic diagram of receiving end redundancy according to an embodiment of the present application, such as Figure 5As shown in FIG. 1 , the receiving end redundancy of LDPC and XOR when the code length is 100, the horizontal axis is the number of additional received packets at the receiving end / the number of information source packets (ie, the receiving end redundancy ratio), and the vertical axis is the decoding failure retransmission rate.

[0135] In this embodiment, the receiving end redundancy is another important parameter in the adaptive redundancy calculation. The LDPC coding method used by the long code is not an ideal code, and a receiving end redundancy cost function needs to be established to describe the mapping relationship between the number of additional packets that the receiving end needs to receive and the decoding failure retransmission rate.

[0136] In some embodiments, the redundancy cost function of the receiving end can be represented as:

[0137] Cost t =[Q(k,rm)-δ·P rtt (k,rm)] 2 ;

[0138] Among them, Cost t is the redundancy cost of the t-th video data block at the receiving end, that is, the total cost of transmitting block t at the current bit rate, Q(k,rm) is the redundancy ratio of the receiving end, k is the source information bit code length (that is, the code length of multiple data packets), r is the redundant bit code length (that is, redundancy), m is the number of packet losses, the encoding bit rate can be expressed as m / (k+r), k+rm is the number of packets received by the receiving end, and Q(k,rm) is expressed as a function related to k and rm.

[0139] In an exemplary embodiment, the received redundancy ratio P can be expressed as Q(k,rm)=(rm) / k. rtt (k,rm), is the decoding failure rate corresponding to the redundancy ratio, that is, the ratio of overall decoding errors of blocks during the simulation process. Decoding failure requires block retransmission. δ is the weight of the decoding failure item. An increase in δ indicates that the current decoding failure rate is unacceptable.

[0140] In some embodiments, by inversely solving the redundancy cost function, it can be obtained that when LDPC coding is used, when the retransmission expectation is not greater than P, the number of packets that the receiving end needs to receive additionally accounts for r rec . Combined with the predicted packet loss rate, the redundancy is determined according to the following formula:

[0141] Wherein, r is the redundancy, k is the code length of the plurality of data packets, r rec is the redundancy ratio, and c is the predicted packet loss rate.

[0142] Figure 6 is a schematic diagram of sub-packet coding according to an embodiment of the present application, such as Figure 6As shown, a video data block can be divided into k information source packets (ie data packets).

[0143] In this embodiment, the video data block is the basic unit of encoding, and is divided into k information source packets containing 1400 bytes of video data during encoding, and zeros are added if the number is less than an integer multiple of 1400.

[0144] In this embodiment, the information source packet includes a 1400-byte data portion, and the packet header information is a 2-byte packet sequence, which can represent a sequence between [0, 65535]. The 1402-byte portion is encoded, and the LDPC encoding is based on bits, and the RS encoding is based on bytes.

[0145] In this embodiment, forward error correction encoding is performed on k information source packets to obtain n coded packets, where n is the sum of the source code length k of the information source packet and the number of redundant packets r.

[0146] In this embodiment, the header information of the coded packet obtained after encoding includes a 2-byte source code length, a 2-byte coded packet sequence and 94 bytes of protection information. The protection information is the row information corresponding to the generating matrix. By sending the protection information, the receiving end can reorganize the coefficient matrix to perform Gaussian elimination decoding.

[0147] In this embodiment, the corresponding pre-constructed generator matrix G can be retrieved according to the information source code length k and the redundancy r, and the coding sequence Sp with a length of n can be constructed in the following way: n : G n×k S k =Sp n ;

[0148] In some embodiments, the LDPC matrix adopts the Progressive Edge Growth (PEG) method during construction, and during construction, edges are added to each variable node in turn to connect the check nodes to construct the desired Tannar graph.

[0149] Furthermore, the selection of variable node degree Dv will affect the performance of LDPC coding. For example, the optimal Dv value under different code lengths can be selected to construct the LDPC matrix. The constructed matrix elements are [0,1], and the specific operations are performed in the computational domain GF(2).

[0150] In some embodiments, the RS matrix is ​​constructed using the vandemonde construction method, the constructed matrix elements are values ​​in [0, 255], and the specific operations are performed in GF(2^8).

[0151] Furthermore, in order to reduce computational complexity and unify the encoding and decoding process with LDPC, the computational domain of the RS encoding matrix can be transformed, and an 8×8 [0,1] matrix corresponding to each [0,255] can be designed, and the specific operations can be transformed into GF(2).

[0152] In another embodiment, a forward error correction decoding method is provided, specifically a Gaussian elimination algorithm for decoding while receiving.

[0153] In this embodiment, decoding can be performed immediately when partial data is received, which greatly reduces decoding delay. In addition, when decoding fails and retransmission is required, only a small portion of the lost coded packets need to be retransmitted, which can avoid transmission delay caused by overall retransmission.

[0154] In this embodiment, the receiving end decodes and reorganizes the received coded data packets to obtain a coefficient matrix G'. The rows in the coefficient matrix G' correspond to the code bits in the coded packet sequence Sp'. As long as the reorganized coefficient matrix is ​​full rank, it meets the Gaussian elimination decoding requirements, that is: G' m×k S k =Sp' m ; Among them, G' m×k is the coefficient matrix, Sp' m is a sequence of m coded packets, S k is a sequence of k data packets, i.e., the final decoding result.

[0155] In this embodiment, by O(n 2 ) complexity can restore the original data, and decoding and packet receiving are carried out simultaneously.

[0156] Specifically, when the receiving end receives a coded packet, it parses and obtains the protection information carried in the coded packet, that is, the corresponding row of the generator matrix, and introduces it into the coefficient matrix G' to perform O(n) complexity elimination to achieve matrix ladderization. Then, the coefficient matrix G' is expanded by continuously receiving packets until the coefficient matrix G' is full rank, that is, the decoding is successful.

[0157] Through the embodiments of the present application, decoding can be started when the first coded packet is received without increasing the complexity of Gaussian elimination decoding, thereby greatly reducing the decoding delay.

[0158] In some embodiments, the specific implementation of Gaussian elimination decoding is shown in Table 1 below:

[0159] Table 1:

[0160]

[0161]

[0162] In some embodiments, if decoding fails, a retransmission mechanism is invoked. The retransmission mechanism may include: feeding back the unreceived coded packet sequence to the sender, and the sender retransmits the lost coded packet sequence.

[0163] Furthermore, the receiving end may also feed back the overall packet loss rate of the video data block to the sending end according to the obtained total number of sent packets.

[0164] In some embodiments, the method further includes: after the decoding is successful, arranging the obtained information packets in sequence (ie, reorganizing), and extracting the corresponding video data according to the header information of the video data block.

[0165] According to another aspect of an embodiment of the present application, a forward error correction encoding and decoding device is also provided.

[0166] Figure 7 is a block diagram of a forward error correction encoding and decoding device according to an embodiment of the present application, such as Figure 7 As shown, the device can be divided into two parts: a transmitting end and a receiving end.

[0167] In this embodiment, the transmitting end mainly includes an FEC encoding unit 72, which is used to perform packet loss rate prediction, redundancy allocation and packet encoding.

[0168] In this embodiment, the packet loss rate prediction is to establish a prediction system based on a Kalman filter, by inputting channel packet loss rate feedback of a certain time scale, and calculating through the time equation and the prediction equation to obtain a fine-grained packet loss rate prediction.

[0169] In this embodiment, redundancy allocation is based on code length and predicted packet loss rate, and the code rate is allocated in consideration of the decoding success rate at the receiving end, and packet coding is the implementation process of forward error correction coding (Forward Error Correction, referred to as FEC).

[0170] In this embodiment, the receiving end mainly includes an FEC decoding unit 74, which is used for packet loss rate calculation, FEC decoding and video data recovery.

[0171] In this embodiment, the receiving end counts the packet loss rate of the link received and sent packets and returns it to the sending end.

[0172] In this embodiment, FEC decoding is to use Gaussian elimination decoding after re-arranging the received packets, determine whether retransmission is required based on the restored check sequence, and finally re-arrange the successfully decoded packet sequence to restore the original video data.

[0173] Figure 8 is a schematic diagram of the process of forward error correction coding according to an embodiment of the present application, such as Figure 8 As shown, the complete process of forward error correction coding includes the following steps:

[0174] Step S801, frame-level block reorganization and packetization;

[0175] Step S802, predicting the packet loss rate through a Kalman filter system;

[0176] Step S803, obtaining LDPC receiving end redundancy;

[0177] Step S804, LDPC medium and long code encoding;

[0178] Step S805, packet sending;

[0179] Step S806, receiving and decoding packets, and calculating the packet loss rate;

[0180] Step S807, determining whether the decoding is successful;

[0181] Step S808, block and restore the video data.

[0182] In some embodiments, the process further includes step S809, RS medium and long code encoding, which is used to replace the medium and long code encoding scheme in steps S803 to S804.

[0183] In this embodiment, step S807 executes step S808 when the decoding is successful, and returns to step S805 to continue receiving packets when the decoding fails.

[0184] In this embodiment, the packet loss rate in step S806 may be fed back to the Kalman filter system at the transmitting end to correct the predicted packet loss rate.

[0185] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program executes the steps of any of the above method embodiments when executed by a processor.

[0186] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0187] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0188] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0189] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0190] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0191] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A forward error correction coding method, applied to a transmitting end, characterized in that: The method comprises: Dividing the video data block to be sent into a plurality of data packets according to a preset data packet size, and determining the code length of the plurality of data packets, wherein the code length of the plurality of data packets is the number of the plurality of data packets; Determining the redundancy of the encoded video data blocks in the next time period according to the code lengths of the multiple data packets and the predicted packet loss rate in the next time period; Determining a target generator matrix from a plurality of pre-constructed generator matrices according to the code lengths of the plurality of data packets and the redundancy; Forward error correction encoding is performed on the multiple data packets according to the target generation matrix to obtain multiple coded packets, wherein the number of the multiple coded packets is greater than the number of the multiple data packets.

2. The method according to claim 1, characterized in that: Before dividing the video data block to be sent into a plurality of data packets according to the preset data packet size and determining the code lengths of the plurality of data packets, the method further includes: According to a preset data block size interval and a preset upper limit on the number of data frames, the video data to be sent is divided into frame-level blocks to obtain a plurality of the video data blocks, wherein the size of each of the video data blocks is within the preset data block size interval, the number of data frames contained in each of the video data blocks is less than or equal to the preset upper limit on the number of data frames, and the preset data block size interval is determined based on the product of a preset code length range and a preset data packet size.

3. The method according to claim 2, characterized in that Dividing the video data block to be sent into a plurality of data packets according to a preset data packet size, and determining the code lengths of the plurality of data packets, including: Dividing the video data block into a plurality of data packets of equal size according to the preset data packet size; The number of the plurality of data packets is determined as the code length of the plurality of data packets, wherein the code length of the plurality of data packets is within the preset code length range.

4. The method according to claim 1, characterized in that: Before determining the redundancy of the encoded video data blocks in the next time period according to the code lengths of the multiple data packets and the predicted packet loss rate in the next time period, the method further includes: The first packet loss rate predicted in the previous time period is obtained through the pre-built Kalman filter; Acquire an actual packet loss rate in a previous time period, and correct the first packet loss rate according to the actual packet loss rate to obtain a corrected second packet loss rate; The Kalman filter is used to predict the packet loss rate in the next time period according to the second packet loss rate to obtain the predicted packet loss rate.

5. The method according to claim 1, characterized in that The step of determining the redundancy of the encoded video data blocks in the next time period according to the code lengths of the multiple data packets and the predicted packet loss rate in the next time period includes: In a case where the code lengths of the plurality of data packets are within a first preset code length interval, determining the redundancy according to the code lengths of the plurality of data packets and the predicted packet loss rate; When the code lengths of the multiple data packets are within a second preset code length interval, the redundancy ratio of the receiving end in the next time period is determined according to a pre-established redundancy cost function of the receiving end, and the redundancy is determined according to the predicted packet loss rate and the redundancy ratio.

6. The method according to claim 5, characterized in that The determining the redundancy according to the predicted packet loss rate and the redundancy ratio includes: The redundancy is determined according to the following formula: Wherein, r is the redundancy, k is the code length of the plurality of data packets, r rec is the redundancy ratio, and c is the predicted packet loss rate.

7. The method according to claim 1, characterized in that Before determining the target generator matrix from a plurality of pre-constructed generator matrices according to the code lengths of the plurality of data packets and the redundancy, the method further comprises: Traversing code length values ​​within a preset code length range and redundancy values ​​within a preset redundancy range, wherein the preset code length range includes a first preset code length interval and a second preset code length interval; In a case where the code length values ​​of the plurality of data packets are within a first preset code length interval, constructing a plurality of Reed-Solomon code RS matrices based on the respective redundancy values ​​within the preset redundancy range, the respective code length values ​​of the first preset code length interval, and the Vandermonde matrix; In a case where the code length values ​​of the plurality of data packets are within a second preset code length interval, constructing a low-density parity-check code LDPC matrix based on each redundancy value within the preset redundancy range, each code length value of the second preset code length interval, and a progressive edge growth algorithm; The generator matrix includes the RS matrix and the LDPC matrix, and the element value of the generator matrix is ​​0 or 1.

8. The method according to claim 7, characterized in that The step of determining a target generator matrix from a plurality of pre-constructed generator matrices according to the code lengths of the plurality of data packets and the redundancy comprises: In a case where the code lengths of the plurality of data packets are within the first preset code length interval, determining an RS matrix corresponding to the code lengths of the plurality of data packets and the redundancy as the target generation matrix; In a case where the code lengths of the plurality of data packets are within the second preset code length interval, an LDPC matrix corresponding to the code lengths of the plurality of data packets and the redundancy is determined as the target generation matrix.

9. The method according to claim 7, characterized in that: The performing forward error correction encoding on the multiple data packets according to the target generation matrix to obtain multiple coded packets includes: When the code lengths of the multiple data packets are within the first preset code length interval, forward error correction encoding is performed on each byte data in the multiple data packets according to the target generation matrix to obtain the multiple coded packets, wherein each byte data includes 8 bits of data; In a case where the code lengths of the multiple data packets are within the second preset code length interval, forward error correction encoding is performed on each bit of data in the multiple data packets according to the target generation matrix to obtain the multiple coded packets.

10. The method according to claim 1, characterized in that After performing forward error correction encoding on the multiple data packets according to the target generation matrix to obtain multiple coded packets, the method further includes: Numbering the multiple encoding packets in sequence to obtain a serial number of each encoding packet; Determining row information corresponding to the sequence number in the target generation matrix as protection information of the corresponding coded packet; Encapsulating the multiple coded packets using packet header information to obtain multiple encapsulated coded packets, wherein the packet header information includes: code lengths of the multiple data packets, the sequence numbers, and the protection information; The multiple encapsulated encoded packets are sent.

11. A forward error correction decoding method, applied to a receiving end, characterized in that: The method comprises: Gaussian elimination decoding is performed on part or all of the multiple coded packets of the transmitter to obtain multiple data packets, wherein the multiple coded packets are obtained by the transmitter dividing the video data blocks to be sent into the multiple data packets according to a preset data packet size, determining the code lengths of the multiple data packets, determining the redundancy of the encoded video data blocks in the next time period according to the code lengths of the multiple data packets and the predicted packet loss rate in the next time period, determining a target generation matrix from multiple pre-constructed generation matrices according to the code lengths of the multiple data packets and the redundancy, and performing forward error correction encoding on the multiple data packets according to the target generation matrix, wherein the number of the multiple coded packets is greater than the number of the multiple data packets, and the code length of the multiple data packets is the number of the multiple data packets; The multiple data packets are reassembled to obtain a video data block.

12. The method according to claim 11, characterized in that The Gaussian elimination decoding is performed on part or all of the multiple coded packets of the transmitting end to obtain multiple data packets, including: Each time an encapsulated coding packet sent by the transmitting end is received, the encapsulated coding packet is parsed to obtain a corresponding coding packet and row information of the target generation matrix, and a coefficient matrix is ​​updated according to the row information of the target generation matrix; Each time the coefficient matrix is ​​updated, Gaussian elimination decoding is performed on part or all of the coded packets according to the coefficient matrix, and the multiple data packets are obtained when Gaussian elimination decoding is successful, wherein part or all of the coded packets are coded packets that have been received and parsed by the receiving end.

13. The method according to claim 12, characterized in that The Gaussian elimination decoding is performed on part or all of the multiple coded packets of the transmitting end to obtain multiple data packets, including: Each time one of the encapsulated coding packets is received, the encapsulated coding packet is parsed to obtain a coding packet and corresponding protection information, wherein the encapsulated coding packet is obtained by encapsulating any one of the multiple coding packets using packet header information, the packet header information includes code lengths of the multiple data packets, the serial number of the coding packet, and the protection information, and the protection information is row information corresponding to the serial number in the target generation matrix; Each time the protection information is obtained, the coefficient matrix is ​​updated according to the protection information, and Gaussian elimination decoding is performed on part or all of the coded packets according to the coefficient matrix; When the coefficient matrix after Gaussian elimination decoding satisfies a preset condition, determining that Gaussian elimination decoding is successful and the multiple data packets are obtained; When the coefficient matrix after Gaussian elimination decoding does not satisfy the preset condition, new encapsulation coding packets continue to be received and parsed until Gaussian elimination decoding is successful and the multiple data packets are obtained.

14. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program executes the method described in any one of claims 1 to 10 and 11 to 13 when executed by a processor.

15. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method described in any one of claims 1 to 10 and 11 to 13.

Images