METHOD, APPARATUS, SYSTEM AND MEDIUM FOR SELF-ADAPTIVE SYSTEM CODE FEC ENCODING AND DECODING BASED ON MEDIA CONTENT - Patent application

Abstract

To provide a method, device, system and medium for self-adaptive system code forward error correction (FEC) encoding and decoding based on a media content, which reduce complexity of the FEC encoding at a sending end, improve efficiency of the FEC encoding, and have stronger adaptability.SOLUTION: A method comprises: according to importance of a media content, dividing source data into N types of source data packets according to a priority; according to the N types of the source data packets and the priorities thereof, generating N types of intermediate codes; according to the N types of the intermediate codes, setting recovery data of the N types of the source data according to a channel condition, generating encoded symbols of N types of system codes; receiving the encoded symbols and sorting and arranging the encoded symbols according to a decoding requirement; and, according to the number of the received encoded symbols, decoding an intermediate codeword depending on conditions and recovering a corresponding source data packet according to the intermediate code.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] This invention claims priority to Chinese patent application CN201710973473.3 (filing date: October 18, 2017, title: Self-adaptive system code FEC method, apparatus and system based on media content) and Chinese patent application CN201710561850.2 (filing date: July 11, 2017, title: Self-adaptive system code FEC encoding and decoding method based on media content).

[0002] The present invention relates to the field of multimedia transmission technology, and in particular to a self-adaptive system code FEC (Forward Error Correction) encoding and decoding method, device, system and medium based on media content. [Background technology]

[0003] In a heterogeneous network media service system, content is transmitted to terminals via Internet protocols or broadcasting protocols. On the Internet, media data is transmitted via IP / TCP or UDP messages, while in broadcasting, content is transmitted via MPEG2-TS. UDP messages may be lost after passing through multiple network devices. Broadcasting TS streams may experience bit errors due to the influence of the transmission environment, resulting in damage to the screen on the terminal side or interruption of the audio.

[0004] FEC (Forward Error Correction) technology is a coding technology that is widely used in communication systems. The server side performs error correction coding on the media data and then adds redundant information before transmitting it, and the terminal side performs reverse FEC decoding to recover lost messages. Taking a typical block symbol as an example, the basic principle is as follows: At the transmitting end, k bits of information are encoded as one block, and (nk) bits of redundant correction information are added to form a codeword of length n bits. After the codeword reaches the receiving end through the channel, if the error is within the correctable range, the erroneous bit can be checked and corrected by decoding. This makes it possible to resist the interference caused by the channel, effectively reduce the bit error rate of the system, and improve the reliability of the communication system. However, since FEC processing reduces the bit error rate of the system at the cost of redundancy overhead, excessive FEC encoding may put pressure on the real-time nature of the system and the state of the network.

[0005] Unequal Error Protection (UEP) is a kind of joint source channel coding. Its core idea is to use different channel protection mechanisms for each part of the data based on the different importance of each part of the data in the code stream, i.e., to prioritize protection for important code streams. UEP reduces the noise tolerance of non-important code streams, but helps to improve the overall bit error prevention performance of the system.

[0006] As one of the FEC (Forward Error Correction) coding techniques, Digital Fountain Code does not require feedback and automatic retransmission mechanisms during the transmission process, thus avoiding the problems of time delays in signal transmission and reception and feedback explosion in broadcast applications. The basic idea of ​​Digital Fountain is as follows: The transmitting end divides the original data into k data symbols, encodes these data symbols, and outputs a code stream of encoded symbols of any length. The receiving end can recover all k data symbols with a high probability by simply receiving n (n is slightly larger than k) encoded symbols correctly. The digital fountain code itself has UEP performance, so it can realize the protection of data with different importance. Summary of the Invention

[0007] In view of the problems of the prior art, the present invention aims to provide a self-adaptive system code FEC encoding and decoding method, apparatus, system and medium based on media content.

[0008] (1) A self-adaptive system code FEC encoding and decoding method based on media content according to the present invention includes: dividing the source data into N (N integers greater than 1) types of source data packets according to priority levels based on the importance of the media contents; generating N kinds of intermediate codes based on the source data and the priorities of the N kinds of source data packets; determining N kinds of restored data of source data according to the N kinds of intermediate codes in accordance with channel conditions, and generating N kinds of encoded symbols; receiving the coded symbols, arranging and ordering the coded symbols according to decoding needs, and decoding the coded symbols; Decoding the intermediate code with a higher priority according to the received encoded symbol status, and recovering the corresponding source data packet according to the intermediate code; Includes.

[0009] (2) Preferably, the step of generating N types of intermediate codes based on the source data and the priorities of the N types of source data packets includes: A, a coding matrix corresponding to N kinds of source data packets i (i=1,2,…N),

number

number

number

[0010] (3) Preferably, the restored data of N types of source data set according to the channel conditions based on the N types of intermediate codes is

number

[0011] (4) Preferably, when N is 2, the following corresponding intermediate codes can be generated for data with different importance:

number

number

[0012] (5) Preferably, when N is 2, the step of receiving the encoded symbols and decoding the encoded symbols includes: During the first stage decoding, obtain an intermediate codeword according to the reconstruction matrix W and the received source data packet;

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0013] (6) Preferably, since some data is lost, a restoration matrix needs to be generated according to the received coding symbols, i.e., the matrix rows corresponding to the lost coding symbols are erased, and a corresponding restoration matrix (denoted as REV_G_ENC) is generated; and intermediate codewords are generated case-by-case according to the number of received data packets, specifically, the method includes the following steps A1 to A4: (Step A1) When l priority data packets are received, the following restoration matrix equation is obtained:

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0014] (7) preferably analyzing significant information of a header of each subpacket in the source data packet; Mapping each subpacket in the source data packet to a different priority based on the critical information; Rearrange the subpackets according to the priority of the subpackets to obtain intermediate data packets; converting the intermediate data packet into a plurality of symbols, performing forward error correction (FEC) coding on the intermediate data packet converted into the plurality of symbols, and then obtaining a restored data packet including the plurality of symbols; adding an index field and / or a priority indication field to the recovered data packet, the index field indicating a sequence of subpackets in the recovered data packet and a sequence of subpackets in the recovered data packet with a priority corresponding to the priority indication field; adding an index field and / or a priority indication field to the intermediate data packet, the index field indicating a sequence of subpackets in the intermediate data packet and / or a sequence of subpackets in the intermediate data packet with a priority corresponding to the priority indication field; Adding corresponding indication information to the restored data packet and the intermediate data packet, or transmitting corresponding indication information when transmitting the restored data packet and the intermediate data packet, the indication information indicating subpackets of each priority level of the restored data packet, the number of subpackets belonging to the corresponding priority level or the proportion of the subpackets in the restored data packet, the indication information indicating subpackets of each priority level of the intermediate data packet, and the number of subpackets belonging to the corresponding priority level or the proportion of the subpackets in the intermediate data packet, Send a restored data packet and an intermediate data packet containing instruction information to the terminal side, or send a restored data packet, an intermediate data packet and corresponding instruction information to the terminal side; The method further includes the steps of:

[0015] (8) Preferably, analyzing the important information of the header of each subpacket in the source data packet includes: This includes reading a priority field in the header of each subpacket and determining a priority for the corresponding subpacket based on the information in the priority field.

[0016] (9) Preferably, the step of mapping subpackets in the source data packet to different predetermined priorities based on the critical information comprises: Determining the number of priorities actually used in the source data packet and the mapping relationship of each priority with the priority field; and Mapping the subpackets to corresponding predefined priorities based on the mapping relationship.

[0017] (10) Preferably, converting the intermediate data packet into a plurality of symbols, performing forward error correction (FEC) coding on the intermediate data packet converted into the plurality of symbols, and then obtaining a restored data packet including the plurality of symbols includes: performing FEC coding on the intermediate data packet by a self-adaptive FEC coding method of a RaptorQ system code to obtain the following intermediate codeword;

number

number

[0018] (11) A self-adaptive system code FEC encoding and decoding method based on media content according to the present invention includes: Receive the restored data packet and the intermediate data packet sent by the server, or receive the restored data packet, the intermediate data and the corresponding instruction information; Analyze an index field included in the recovered data packet and the intermediate data packet, and generate a coding matrix according to the index field, where the index field indicates a sequence of subpackets in the recovered data packet and subpackets at each priority level of the intermediate data packet; directly obtain index information according to a priority level indication field of the recovered data packet and the intermediate data packet, and generate a coding matrix according to the index information, where the index information indicates a sequence of subpackets in the recovered data packet and subpackets at each priority level of the intermediate data packet; performing forward error correction (FEC) decoding on the coding matrix to recover intermediate data packets; analyzing information in a header of the intermediate data packet and reordering the subpacket-wise sequence of the intermediate data packet to restore the source data packet; The method includes the steps of:

[0019] (12) Preferably, analyzing the index field included in the restored data packet and the intermediate data packet includes: Obtaining a corresponding row index field of the coding matrix corresponding to the source data according to the following general formula: index=FEC payload ID- number of packets of all priorities before this packet Here, index indicates the corresponding row index of the coding matrix corresponding to the source data, FEC payload ID indicates the index information of the header of the data packet, and number of packets of all priorities before this packet indicates the total number of data packets of all priorities before this data packet.

[0020] (13) Preferably, performing forward error correction (FEC) decoding on the coding matrix to restore the intermediate data packet includes: performing a linear row transformation on the coding matrix, arranging the symbols in the received corresponding restored data packet, and determining the priority level to which the symbols belong and their post-transformation positions based on the value of index information FECpayload_ID in the header of the restored data packet and the number of symbols in the corresponding priority level; or determining the priority level to which the symbols belong and their post-transformation positions based on the priority level field in the header of the restored data packet and index information in the priority level corresponding to the priority level field; adjusting a sequence of symbols in the recovered data packet and decoding the sequence of symbols using a Gaussian elimination decoder and an RFC6330 decoder to obtain an intermediate data packet including the recovered symbols; and converting the intermediate data packet, which includes the plurality of symbols, into an intermediate data packet.

[0021] (14) A self-adaptive system code FEC encoding and decoding device based on media content according to the present invention comprises: an analysis module for analyzing important information of a header of each subpacket in the source data packet; a mapping module for mapping each sub-packet in the source data packet to a different predetermined priority based on the critical information; an ordering module for obtaining intermediate data packets by rearranging the subpackets based on a predetermined priority; an encoding module that converts the intermediate data packet into a plurality of symbols and performs forward error correction (FEC) encoding on the intermediate data packet converted into the plurality of symbols to obtain a restored data packet including the plurality of symbols; an index module that adds an index field and / or a priority indication field to the recovered data packet, the index field indicating a sequence of subpackets in the recovered data packet and / or a sequence of subpackets in the recovered data packet with a priority corresponding to the priority indication field, and adds an index field and / or a priority indication field to the intermediate data packet, the index field indicating a sequence of subpackets in the intermediate data packet and / or a sequence of subpackets in the intermediate data packet with a priority corresponding to the priority indication field; an indication information generating module for adding indication information corresponding to the restored data packet and the intermediate data packet, or for transmitting the corresponding indication information when transmitting the restored data packet and the intermediate data packet, the indication information indicating subpackets of each priority level of the restored data packet, and the number or proportion of subpackets belonging to the corresponding priority level of the restored data packet, and the indication information indicating subpackets of each priority level of the intermediate data packet, and the number or proportion of subpackets belonging to the corresponding priority level of the intermediate data packet; A transmitting module for transmitting the restored data packet, the intermediate data packet and corresponding signaling information to a terminal side, or transmitting the restored data packet, the intermediate data packet and corresponding indication information to a terminal side; Includes.

[0022] (15) Preferably, the encoding module specifically comprises: The intermediate data packet is FEC-encoded according to the self-adaptive FEC encoding method of the RaptorQ system code to obtain the following intermediate code word:

number

number

[0023] (16) A self-adaptive system code FEC encoding and decoding device based on media content according to the present invention includes a receiving module, a coding matrix generating module, a decoding module and a restoration module; The receiving module receives the restored data packet, the intermediate data packet, or receives the restored data packet, the intermediate data, and the corresponding instruction information sent by the server; a coding matrix generating module that analyzes an index field included in the recovered data packet and the intermediate data packet, and generates a coding matrix based on the index field, the index field indicating a sequence of subpackets in the recovered data packet and subpackets at each priority level in the intermediate data packet; or Directly obtain index information according to the priority indication fields of the recovered data packet and the intermediate data packet, and generate a coding matrix according to the index information, where the index information indicates a sequence of subpackets in the recovered data packet and subpackets at each priority level of the intermediate data packet; The decoding module performs forward error correction (FEC) decoding on the coding matrix to recover intermediate data packets; A restoration module analyzes information in a header of the intermediate data packet, rearranges a sequence of sub-packets of the intermediate data packet, and restores the source data packet.

[0024] (17) Preferably, the coding matrix generation module specifically includes: Obtain the index field of the corresponding row of the coding matrix corresponding to the source data according to the following general formula: index=FEC payload ID- number of packets of all priorities before this packet Here, index indicates the corresponding row index of the coding matrix corresponding to the source data, FEC payload ID indicates the index information of the header of the data packet, and number of packets of all priorities before this packet indicates the total number of data packets of all priorities before this data packet.

[0025] (18) Preferably, the decoding module specifically comprises: performing a linear row transformation on the coding matrix to arrange the symbols in the received corresponding recovered data packets, and determining the priority level and the transformed position of the symbols according to the value of index information FECpayload_ID in the header of the recovered data packets and the number of symbols in the corresponding priority level; or determining a priority level and a post-conversion position of the symbol based on a priority level field in the header of the recovered data packet and index information at a priority level corresponding to the priority level field; adjusting a sequence of symbols in the restored data packet and decoding the sequence of symbols using a Gaussian elimination decoder and an RFC6330 decoder to obtain an intermediate data packet including the restored symbols; An intermediate data packet including a plurality of symbols is converted into an intermediate data packet.

[0026] (19) The media content-based self-adaptive system code FEC encoding and decoding system of the present invention includes a server and at least one terminal, wherein the server executes the media content-based self-adaptive system code FEC encoding and decoding method described in any one of (7) to (10) above, and the terminal executes the media content-based self-adaptive system code FEC encoding and decoding method described in any one of (11) to (13) above.

[0027] (20) The present invention further provides a computer-readable storage medium having a computer program stored therein, the computer program being executed by a processor to perform the steps of the method for detecting, positioning and cropping a striped fabric based on an image according to any one of (7) to (13) above.

[0028] Compared with the prior art, the present invention has the following beneficial effects: The self-adaptive system code FEC method based on media content of the present invention arranges media contents according to the importance according to a priority order and adopts unequal error protection (UEP), thereby maximally guaranteeing the quality of media contents and reducing data redundancy caused by FEC. In addition, the encoding and decoding of the system code is more flexible, and the source data stream does not need to be multi-streamed, so the complexity of the transmitting end FEC encoding is reduced and the efficiency of FEC encoding is improved. In addition, the encoding scheme can be adjusted based on the changes of the current network conditions, so it has stronger adaptability to time-varying networks. [Brief description of the drawings]

[0029] Other features, objects and advantages of the invention will become more apparent from the detailed description of the non-limiting embodiments given below with reference to the drawings, in which: [Figure 1] 1 is a schematic diagram of a two-layer FEC structure for media source data; [Diagram 2] FIG. 2 is a schematic diagram of the dependency relationships between frames in an image group. [Diagram 3] FIG. 2 is a schematic diagram of the dependency relationships between frames in an image group. [Figure 4] FIG. 1 is a schematic diagram of a Fountain Code unequal error protection coding scheme. [Diagram 5] FIG. 1 is a schematic diagram of a system configuration for unequal error protection using a fountain code. [Figure 6] 1 is a schematic diagram of the coding matrix structure of the RaptorQ code system. [Figure 7] FIG. 2 is a schematic diagram of the structure of the coding matrix of the unequal error protection of the RaptorQ code system; [Figure 8] FIG. 1 is a schematic diagram of an unequal error protection structure of a LT coding matrix. [Figure 9] 1 is a flowchart of a multiple priority unequal error protection system fountain code encoding. [Figure 10] 1 is a flowchart of multiple priority unequal error protection system fountain code decoding. [Figure 11] FIG. 1 is a frame diagram of a self-adaptive system code FEC system based on media content. [Figure 12] FIG. 13 is a schematic diagram of a mapping relationship of each priority order. [Figure 13] FIG. 2 is a schematic diagram of reordering data packet streams according to priority. [Figure 14] 13 is a processing flowchart for source data packets and recovered data packets after FEC encoding. [Figure 15] A schematic diagram of a receiving end analyzing signaling information and data packet information. [Figure 16] FIG. 2 is a schematic diagram of generating a coding matrix based on received data packets; [Figure 17] 11 is a flow chart for processing a received data packet. [Figure 18] FIG. 2 is a schematic diagram of the results of an initial sequence of recovered data packet streams. [Figure 19] 2 is a schematic diagram of a receiving end analyzing header information of a data packet; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] The present invention will be described in detail below with reference to specific embodiments. The following embodiments are merely examples for those skilled in the art to more easily understand the present invention, and are not intended to limit the present invention in any manner. In addition, those skilled in the art can make various modifications and improvements within the scope of the present invention, and it should be noted that these also fall within the scope of the present invention.

[0031] The terms used in this specification are interpreted as follows. 1) Unequal Error Protection (UEP), a type of joint source channel coding. Its core idea is Based on the different importance of the data in each part of the stream, different channel protection mechanisms are used for each part of the data, i.e., more protection is given to the important code streams. UEP reduces the noise tolerance performance of the non-important code streams, but helps to improve the overall bit error prevention performance of the system.

[0032] 2) Digital Fountain Code does not require feedback and automatic retransmission mechanisms in the transmission process, thus avoiding the problems of time delays in signal transmission and reception and feedback explosion in broadcast applications. The basic idea of ​​Digital Fountain is as follows: The sender divides the original data into k data symbols, encodes these data symbols, and outputs a code stream of encoded symbols of any length. The receiver can recover all k data symbols with a high probability by simply receiving n (n is slightly larger than k) encoded symbols correctly.

[0033] The Digital Fountain Code itself has UEP characteristics and can be used to distinguish data of different importance. Compared with traditional constant bit rate channel coding methods, the digital fountain code has the following obvious advantages:

[0034] a) Ideal scalability: One-way broadcasting has no feedback, so the sender is not affected by the increase in the number of users. Therefore, the sender can provide services to any number of users.

[0035] b. It is suitable for time-varying channels and utilizes the channel capacity efficiently. The user's decoding characteristics are not related to the channel's drop probability and bandwidth. Even if the channel has a high packet loss rate and poor conditions, it will not affect the decoding at the receiving side, that is, the receiving side can decode normally as long as it receives a sufficient number of encoded data, and it has higher adaptability.

[0036] c) Low encoding and decoding complexity: In an ideal situation, each coding symbol generated by the Fountain Code has linear encoding and decoding complexity, which helps to simplify the design of the codec on the transmitting and receiving sides and to realize software implementation.

[0037] d. High applicability to heterogeneous users. The bit rateless property of fountain code ensures that users with different packet loss rates or bandwidths do not affect each other, and that high-quality users are not affected by low-quality users. In addition, digital fountain code supports multiple service modes, such as resuming transmission after interruption and asynchronous access.

[0038] Figure 1 shows the FEC two-layer structure of media resource data. As shown in Figure 1, the first layer divides a source packet block into many small blocks and performs FEC protection for each block, and the second layer performs FEC protection for the entire block. The fine division in the first layer can provide small delay, and the second layer ensures resilience and small redundancy. However, this method requires FEC encoding twice, which reduces the encoding efficiency and is not flexible enough.

[0039] A data stream can be divided into different types of data packets according to their contents. Different types of data packets have different importance. For example, the importance of I, B, and P frames in a data packet is shown in Figures 2 and 3. The dependency of each frame in a picture group shows that the dependency degree and importance of different frames in a picture group are different, and the I frame is the most important, and the importance of B and P frames is comparatively low. Therefore, unequal error protection can be achieved according to the importance of the data content. Specifically, when considering the situation of transmitting source data with two levels of importance, the source data is partitioned according to the above standard, rearranged according to the importance, and the structure of the system fountain code coding matrix is ​​changed to realize unequal error protection for different contents. The basic configuration of the system is shown in Figure 5. The data generated by the unequal error protection coding of the system fountain code is shown in Figure 4.

[0040] [Example 1] In this embodiment, the RaptorQ system code is taken as an example, and the coding matrix structure of the system RaptorQ code is shown in Figure 6. Here, D represents source data, C represents intermediate code, G_LDPC represents LDPC matrix, I_S, I_H represent identity matrix, and G_ENC represents LT coding matrix, and the coding matrix can be expressed as follows:

number

[0041] When performing FEC encoding, an intermediate code is generated from the source data and the coding matrix shown in FIG.

number

[0042] Furthermore, the source data and the restored data are generated based on the LT matrix in the coding matrix and the obtained intermediate code. R denotes the restored data obtained by encoding, and G_ENC' is the LT matrix generated by the same generation method as G_ENC.

number

[0043] When encoding source data by the above method, it is necessary to redesign the matrix structures of the coding matrix A and the LT matrix G_ENC, since the source data is prioritized and unequal error protection coding cannot be adopted. Taking the case where there are two data with different priorities in the source data as an example, the priorities are set to Priority 1 and Priority 2, respectively, and the data with different priorities in the source data are divided and different redundancies are assigned to each, thereby achieving the effect of unequal error protection.

[0044] <Encoding process> The coding matrix structure is shown in Figure 7. The coding matrix is ​​expressed by general formula (6), where G_ENC1 and G_ENC2 are coding matrices generated by different rules, and the degree distribution of G_ENC1 matrix can be increased to ensure the protection of important data.

number

number

[0045] In order to achieve unequal error protection for data with different importance, the structure of the LT coding matrix can be changed as shown in FIG. 8 to increase the redundant data for protecting the important data. Here, G_ENC1' and G_ENC1 have the same generation method, and G_ENC2' and G_ENC2 have the same generation method. The obtained restored data R1 is related only to data with priority 1, and the restored data R is related to data with priorities 1 and 2. This increases the redundancy of the important data and increases the protection strength for it. Due to the restriction of the channel condition, the total redundancy of the data is limited, and the ratio of R1 and R2 generated is determined by the ratio of the number of rows of [G_ENC1' 0] and [G_ENC1'' G_ENC2'], so the protection strength for the data with priority 1 can be improved by increasing the number of rows of [G_ENC1' 0].

number

[0046] <Decryption process> After passing through a lossy channel, some data packets are lost, so in order to generate a restoration matrix based on the received symbol ISI, it is necessary to erase the rows of the matrix corresponding to the lost symbols ISI. For example, in the case of a matrix REV_G_ENC1, the ISI of the lost data can be obtained based on the received REV_D1 and D1 of the transmitting end, and the rows corresponding to G_ENC1 can be deleted. Similar to the encoding process, during the first stage decoding, an intermediate codeword is obtained based on the restoration matrix and the received data packets.

number

[0047] A linear row transformation of the above matrix equation gives the following form:

number

number

[0048] the total number of received coding symbols is less than the number of source data coding symbols, and

number

[0049] The total number of received coding symbols is less than the number of source data coding symbols, but

number

number

number

[0050] The total number of received coding symbols is greater than the number of source data coding symbols, and

number

number

number

number

number

number

number

number

[0051] If the total number of received coding symbols is greater than or equal to the number of source data coding symbols,

number

number

number

[0052] As can be seen from the first stage of the decoding process above, C1 must be calculated accurately in any case. The intermediate codewords C1 and C2 recover important data of the source data in the second stage of decoding, so by increasing the success rate of decoding C1, the probability of recovering the source data accurately is increased.

[0053] A system code encoding symbol is generated based on the resulting intermediate codeword.

number

[0054] In addition, this embodiment is not only applicable to two-priority scenes, but also applicable to multi-priority source data. The specific process is as follows:

[0055] <Encoding process> First, generate intermediate codewords based on the source data and the priority of the source data. If the source data is divided into l data packets,

number

number

number

[0056] The whole encoding process is shown in Figure 9. The source data and restored data are sent from the transmitting end to the receiving end. Due to channel attenuation, some data may be lost. After the receiving end receives the data, it decodes it according to the received data to restore the source data.

[0057] <Decryption process> Since some data has been lost, we need to generate a recovery matrix based on the received symbols ISI, i.e., erase the matrix rows corresponding to the lost symbols ISI and generate the corresponding matrix (denoted as REV_G_ENC). Depending on the number of received data, we generate intermediate codewords on a case-by-case basis.

[0058]

number

number

number

number

number

number

number

number

number

number

[0059] For a given i value, if cumFullRand(i)==1 does not hold, then the corresponding C i If neither of the above two conditions is satisfied until i==1, C1, C2...C lNeither of these can be calculated, and the decryption fails.

[0060] The above process is repeated by subsequently cycling i=i-1, ie, i=l:-1:1.

[0061] The entire decoding process is shown in FIG. According to the method of this embodiment, resources are effectively saved. At the receiving end, sometimes received data (e.g., B frames) are artificially discarded, which results in a waste of transmission resources. According to the above embodiment, the problem is solved from the source end, and important packets are protected to a greater extent by making undesirable packets lose with a greater probability during the transmission process.

[0062] The method of this embodiment provides a unique transmission mode. By designing an unequal error protection mode based on channel conditions, user experience, etc., the video transmission becomes more flexible and precise. In addition, even if the encoding mode of the transmitting end becomes flexible, the receiving end can restore the original data based on the list information.

[0063] The method of this embodiment provides a more flexible coding matrix design, and according to the actual media application scenario, data prioritization is performed based on media content, and FEC coding matrix can be more flexible designed according to actual needs, so as to achieve the effect of unequal error protection.

[0064] The method of the present embodiment provides a more flexible decoding scheme, and adopts different decoding algorithms according to the number of data of each priority level received, so that the data of higher priority can be restored to the maximum extent under limited bandwidth resources, and the protection strength for the data of higher priority can be increased.

[0065] [Example 2] Each sub-packet in the source data packet can be mapped to a different predetermined priority, or the first and second embodiments can be combined.

[0066] FIG. 11 is a frame diagram of a self-adaptive system code FEC system based on media content. The system includes a server and at least one terminal. First, the server side divides multiple priorities based on actual application scenarios and media content, analyzes important information related to the data packets in the headers of the data packets, and maps each data packet to a different priority. Furthermore, based on the priority status and the mapping relationship, rearranges the data packets that require forward error correction coding (FEC) from high to low according to priority. For the rearranged source data, encoding is performed according to the actual channel situation by an unequal error protection FEC encoding method to generate repair data (repair symbols). After the FEC encoding is completed, an index field indicating the current data packet sequence is added to the source data packets and the repaired data packets. The source data and the repaired data at each priority are statistically collected, and corresponding signaling information is generated, and the number or proportion of data packets at each priority and the number or proportion of corresponding repaired data at each priority are indicated in the signaling, so that the data can be accurately restored after the receiving end receives the data. Finally, it transmits the source data packets, the restored data packets and the signaling to a terminal (client or receiving end).

[0067] Correspondingly, the terminal side first analyzes the signaling information and the index field in the received data packet to obtain the number of source data packets and restored data packets received in each priority order and the corresponding data packet index.Furthermore, generate a coding matrix at the receiving end according to the obtained data packet index information.Finally, perform FEC decoding on a case-by-case basis according to the generated coding matrix to restore the source data packets.Since the sequence of the restored source data packets is a rearranged sequence, the initial data of the source data packets can be obtained by analyzing the sequence of the source data packets in the header of the data packet and rearranging the source data packets.

[0068] The following specific examples are provided to further illustrate the present invention, and are not intended to limit the scope of the present invention.

[0069] Example 2.1 As shown in FIG. 12, it may include the following steps. S1: Analyze the header of the encoded and packaged data packet, read the priority field in the DU header, and determine the number of priorities actually adopted according to actual applications and the mapping relationship with the priority field for each priority.

[0070] S2: Based on the separated priorities and the mapping relationship, rearrange the original source data streams according to priorities from high to low, and collect statistics on the number of data packets at each priority (Figure 13).

[0071] S3: The reordered source data stream is FEC-encoded using the self-adaptive FEC encoding scheme of the RaptorQ system code to generate reconstructed data.

[0072] Specifically, the intermediate data packet is FEC-encoded using the self-adaptive FEC encoding method of the RaptorQ system code to obtain the following intermediate codeword:

number

number

[0073] Based on the data stream output from the FEC encoder (the data stream after being rearranged), source data packets and restored data packets are generated and an FEC payload ID is added (Figure 14).

[0074] S4: Generate AL-FEC message, and indicate the number of data packets of each priority and the number of corresponding restored data packets of each priority in the signaling, so that the receiving end can accurately restore the data after receiving the data. Specific signaling is shown in Table 1.

[0075] [Table 1]

[0076] In Table 1, number_of_priority indicates the number of priorities. feCpriority_value (8 bits) indicates a value corresponding to each priority, and this value may be at most the same as the priority defined by the priority in the DU header, and may correspond to priorities in multiple DU headers. number_of_source_symbol (16 bits) indicates the number of source symbols with a priority value indicated by feCpriority_value. number_of_repair_symbol (16th) indicates the number of repair symbols with a priority value indicated by feCpriority_value. private_feCflag is an indication bit and indicates whether or not a private FEC encoding scheme is used. private_flag is an indication bit and indicates whether or not there is one private_field that describes the private FEC encoding scheme to be used. private_field_length is a length field that describes the length of the field of the private FEC encoding scheme. private_field describes detailed information of the private FEC scheme. priority_id is a priority id that indicates the priority of the MMT packet. feCcode_id_for_repair_flow describes the FEC coding scheme used. repair_flow_id (8-bit integer) indicates the correspondence between the generated FEC repair flow and the packet id in the header of the FEC repair packet. maximum_k_for_repair_flow (24-bit integer) indicates the maximum number of source symbols in a source sysmbol block. maximum_p_for_repair_flow (24-bit integer) indicates the maximum number of repair symbols in a repair sysmbol block.protection_window_time is a protection window time (unit: milliseconds) indicating the maximum time difference between the transmission of the first source or repair packet and the transmission of the last source or repair packet in FEC encoding. protection_window_size is a protection window value indicating the maximum count value between the load transmitting the first FEC packet and the load transmitting the last FEC packet in an FEC encoded stream.

[0077] As shown in FIG. 15, according to the number of data packets in each FEC priority level, the index of the corresponding row of the coding matrix corresponding to the source data is obtained by the following formula: index=FEC payload ID-number of packets of all priorities before this packet.

[0078] Wherein, index indicates the index of the corresponding row in the coding matrix of the source data; FEC payload ID indicates the index information of the header of the data packet; and number of packets of all priorities before this packet indicates the total number of data packets of all priorities before the data packet.

[0079] Here, four priorities, each of which has a number of 100, are taken as an example for description (FIG. 16). The source_FECpayload_ID is arranged according to the sequence of the source data packets after rearrangement. If the priority is 1, the ISI (index) for generating the coding matrix can be determined directly based on the current source_FECpayload_ID. If the priority is greater than 1, the priority can be determined based on the value of the source_FECpayload_ID and the number of source symbols in each priority when analyzing the header. For example, if source_FECpayload_ID=108, it can be determined that it belongs to priority 2, and the above general formula shows that the ISI=8. For the restored data packet, the ISI of the generator matrix can be determined based on the repair_FECpayload_ID in the header. Since the transmitting end and the receiving end adopt the same FEC encoding mechanism, the corresponding coding matrix can be generated based on the two indexes.

[0080] According to the generated FEC coding matrix, the coding matrix is ​​transformed by a first-order row, the received data packets are arranged correspondingly, and the position of the restored symbols in the symbol block is adjusted by determining which priority it belongs to according to the value of repair_FECpayload_ID and the number of restored symbols in each priority, thereby achieving the same change (Fig. 17). According to the number of received data packets of different priorities, decoding is performed on a case-by-case basis, the flowchart of which is shown in Fig. 10. Finally, the restored source data is obtained, and the obtained source data packets are further restored to the initial sequence according to the packet_sequence_number in the header (Fig. 18).

[0081] Example 2.2 As shown in FIG. 12, it may include the following steps. S1: Analyze the header of the encoded and packaged data packet, read the priority field in the DU header, and determine the number of priorities actually adopted according to actual applications and the mapping relationship with the priority field for each priority.

[0082] S2: Based on the separated priorities and the mapping relationship, rearrange the original source data streams according to priorities from high to low, and collect statistics on the number of data packets at each priority (Figure 13).

[0083] S3: The reordered source data stream is FEC-encoded using the self-adaptive FEC encoding scheme of the RaptorQ system code to generate reconstructed data.

[0084] Specifically, the intermediate data packet is FEC-encoded using the self-adaptive FEC encoding method of the RaptorQ system code to obtain the following intermediate codeword:

number

number

[0085] S4: Generate corresponding indication information in the header extension of the data packet to indicate the number of data packets at each priority level and the number of corresponding restored data packets at each priority level, so that the receiving end can accurately restore the data after receiving the data. Details are shown in Table 2.

[0086] [Table 2]

[0087] In Table 2, type (16 bits) indicates the type of the current header extension, where it indicates the intermediate data packet or the recovered data packet used for self-adaptive forward error correction coding. length (16 bits) indicates the length of the header_extension. FECpriority (8 bits) indicates the priority to which the current data packet belongs. sub_encoded_symbol_id (16 bits) indicates the index of the number of intermediate data packets or recovered data packets at this priority. N (8 bits) indicates the total number of priorities. SSBL[i] (24 bits) indicates the number of source data packets of each priority, where i indicates the i-th priority. RSBL[i] (24 bits) indicates the number of recovered data packets of each priority, where i indicates the i-th priority.

[0088] As shown in FIG. 19, according to the number of data packets in each FEC priority level, the index of the corresponding row of the coding matrix corresponding to the source data is obtained by the following formula: index=FEC payload ID-number of packets of all priorities before this packet.

[0089] Here, index indicates the index of the corresponding row in the coding matrix of the source data, FEC payload ID indicates index information of the header of the data packet, and number of packets of all priorities before this packet indicates the total number of data packets of all priorities before the data packet.

[0090] According to the generated FEC coding matrix, the coding matrix is ​​transformed by a first-order row, the received data packets are arranged correspondingly, and the position of the restored symbols in the symbol block is adjusted by determining which priority it belongs to according to the value of repair_FECpayload_ID and the number of restored symbols in each priority, thereby achieving the same change (Fig. 17). According to the number of received data packets of different priorities, decoding is performed on a case-by-case basis, the flowchart of which is shown in Fig. 10. Finally, the restored source data is obtained, and the obtained source data packets are further restored to the initial sequence according to the packet_sequence_number in the header (Fig. 18).

[0091] In this embodiment, the server side can divide different data in the data stream according to their importance, so that the data with different importance can be given different protection; the header information and signaling information of the transmitted data packets are marked; the system code method is adopted, and the coding matrix of the system fountain code is self-adaptively adjusted according to the channel condition and the importance of the data, so as to achieve unequal error protection for the data with different importance; the terminal side analyzes the signaling information and header information, and adopts a flexible decoding algorithm according to the number of data with different priorities received, so as to achieve flexible encoding and decoding, and finally restores the source data stream according to the header information; thus, the following beneficial effects are obtained:

[0092] 1) Resources are saved. For example, artificially discarding received data (e.g., B frames) at the terminal side (receiving end) sometimes results in a waste of transmission resources. According to the method of the present embodiment, the problem is solved from the source end, and important data packets are protected to a greater extent by making undesired packets lose with a greater probability during the transmission process.

[0093] 2) Achieve personalized transmission methods. For example, by designing unequal error protection methods based on channel conditions, user experience, etc., video transmission becomes more flexible and precise. In addition, even if the encoding method of the transmitting end becomes flexible, the receiving end can restore the original data based on the list information.

[0094] 3) Achieve more flexible coding matrix design. For example, according to the actual media application scenario, data prioritization can be performed based on media content, and the FEC coding matrix can be designed more flexibly according to actual needs, thereby achieving the effect of unequal error protection.

[0095] 4) A more flexible decoding scheme is achieved. For example, by adopting different decoding algorithms according to the number of data of each priority level received, the data of higher priority can be restored to the maximum extent possible under limited bandwidth resources, and the protection strength for the data of relatively higher priority can be increased.

[0096] It should be noted that the steps of the self-adaptive system code FEC encoding and decoding method based on media content provided in the present invention are achieved by corresponding modules, devices, units, etc. in the self-adaptive system code FEC system based on media content. Those skilled in the art can realize the steps of the method based on the technical scheme of the system. That is, the embodiment in the system can be understood as a preferred example for realizing the method, and therefore the description is omitted here.

[0097] The present invention can also be applied to a computer-readable storage medium, which stores a computer program and executes the steps of the self-adaptive system code FEC encoding and decoding method based on media content described in the above embodiment.

[0098] As known to those skilled in the art, in addition to implementing the system provided by the present invention and each of its devices, modules, and units in the form of pure computer-readable program code, the system provided by the present invention and each of its devices, modules, and units can realize the same functions by logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc., by logically programming the steps of the method. Therefore, the system provided by the present invention and each of its devices, modules, and units can be regarded as hardware components, and the devices, modules, and units for achieving each function contained therein can be regarded as structures within the hardware components. Furthermore, the devices, modules, and units for achieving each function can also be regarded as software modules or structures within the hardware components that can achieve the method.

[0099] The above is a description of specific embodiments of the present invention. It is to be understood that the present invention is not limited to the above specific embodiments, and that those skilled in the art may make various changes or modifications within the scope of the claims, and these changes or modifications do not provide for the substantial content of the present invention. Unless there is a contradiction, the embodiments of the present invention and the features in the embodiments may be arbitrarily combined.

Claims

1. 1. A method for transmitting an FEC encoded codeword, comprising: Dividing the source data into N types of source data packets based on priority, where N is an integer greater than 1; rearrange the subpackets in the N types of source data packets based on priority to generate N types of intermediate data packets; constructing a coding matrix corresponding to each type of intermediate data packet, each coding matrix corresponding to the intermediate data packet includes two sub-matrices, one sub-matrix being a combined matrix consisting of an LDPC matrix and an identity matrix corresponding to the intermediate data packet, and the other sub-matrix being an LT coding matrix corresponding to the intermediate data packet, and constructing coding matrices corresponding to the N types of intermediate data packets based on the coding matrices corresponding to each of the intermediate data packets; generating N intermediate codewords based on the N intermediate data packets and their corresponding coding matrices; generating N recovered data packets based on the N intermediate codewords; adding an index field and / or a priority indication field to the intermediate data packet and / or the recovered data packet; transmitting the intermediate data packet, the restored data packet and corresponding indication information; 1. A method for transmitting FEC encoded codewords, comprising:

2. Before dividing the source data into N types of source data packets based on priority, 2. The method for transmitting FEC encoded codewords as claimed in claim 1, further comprising the step of: analyzing information in a header of each subpacket in the source data packet to determine a priority level corresponding to the subpacket.

3. The step of generating N types of recovered data packets based on the N types of intermediate codewords comprises: constructing an extension matrix of the LT coding matrix corresponding to the N types of intermediate data packets based on the LT coding matrix corresponding to the N types of intermediate data packets; generating N types of recovered data packets according to an extension matrix of the LT coding matrix corresponding to the N types of intermediate data packets and the N types of intermediate codewords; 2. The method of claim 1, comprising:

4. The method of claim 3, characterized in that the construction of the extension matrix of the LT coding matrix corresponding to the N types of intermediate data packets involves added redundant information.

5. 2. The method of claim 1, wherein the index field added to the recovered data packet indicates a sequence of subpackets in the recovered data packet, a sequence of subpackets in the recovered data packet at a priority corresponding to the priority indication field, and / or a number of subpackets or a proportion of the subpackets in the recovered data packet that belong to a corresponding priority.

6. 2. The method of claim 1, wherein the index field added to the intermediate data packet indicates a sequence of subpackets in the intermediate data packet and / or a sequence of subpackets in the intermediate data packet at a priority level corresponding to the priority indication field.

7. 2. The method of claim 1, wherein the indication information indicates subpackets at each priority level of the recovered data packet and / or subpackets at each priority level of the intermediate data packet.

8. 1. A method for decoding an FEC encoded codeword, comprising the steps of: receiving the transmitted recovery data packet, the intermediate data packet and corresponding indication information; Analyzing an index field and / or a priority indication field included in the restored data packet and the intermediate data packet, and the indication information, and generating a restoration matrix according to a coding matrix corresponding to the intermediate data packet and a loss situation of the restored data packet and the intermediate data packet, wherein the coding matrix corresponding to the intermediate data packet includes a combination matrix consisting of an LDPC matrix and an identity matrix, and an LT coding matrix; generating intermediate codewords based on the reconstruction matrix and the received reconstruction data packets and intermediate data packets; generating a decoded intermediate data packet according to a LT coding matrix and an intermediate codeword corresponding to the intermediate data packet; Analyzing total information of headers of subpackets in the intermediate data packet after being decoded, and rearranging the subpackets in the intermediate data packet after being decoded; recovering the source data packets; 1. A method for decoding an FEC encoded codeword, comprising:

9. 9. The method of claim 8, wherein the index field included in the recovered data packet indicates a sequence of subpackets in the recovered data packet, a sequence of subpackets in the recovered data packet at a priority corresponding to the priority indication field, and / or a number of subpackets or a proportion of the subpackets in the recovered data packet that belong to a corresponding priority.

10. 9. The method of claim 8, wherein the index field included in the intermediate data packet indicates a sequence of subpackets in the intermediate data packet and / or a sequence of subpackets in the intermediate data packet at a priority level corresponding to the priority indication field.

11. 9. The method of claim 8, wherein the indication information indicates subpackets of each priority level of the recovered data packet and / or subpackets of each priority level of the intermediate data packet.

12. a mapping module for dividing the source data into N (N is an integer greater than 1) types of source data packets according to priority; an ordering module for generating N types of intermediate data packets by rearranging subpackets in the N types of source data packets according to a priority order; an encoding module for constructing a coding matrix corresponding to each type of intermediate data packet, the coding matrix corresponding to each of the intermediate data packets respectively including two sub-matrices, one sub-matrix being a combined matrix consisting of an LDPC matrix and an identity matrix corresponding to the intermediate data packet, and the other sub-matrix being an LT coding matrix corresponding to the intermediate data packet; an encoding module for constructing coding matrices corresponding to the N types of intermediate data packets according to the coding matrices corresponding to each of the intermediate data packets, generating N types of intermediate codewords according to the N types of intermediate data packets and their corresponding coding matrices, and generating N types of restored data packets according to the N types of intermediate codewords; an index module for adding an index field and / or a priority indication field to the intermediate data packet and / or the recovered data packet; a transmitting module for transmitting the intermediate data packet, the restored data packet, and corresponding indication information; 1. An apparatus for transmitting FEC encoded codewords, comprising:

13. The apparatus for transmitting FEC encoded codewords as claimed in claim 12, further comprising: a parsing module for parsing information of a header of each subpacket in the source data packet, and determining a priority level corresponding to the subpacket.

14. The encoding module generates N types of intermediate code words by FEC encoding the N types of intermediate data packets, constructing a coding matrix corresponding to each intermediate data packet; constructing coding matrices corresponding to the N types of intermediate data packets based on a coding matrix corresponding to each intermediate data packet; and generating N intermediate codewords based on the N intermediate data packets and corresponding coding matrices; 13. An apparatus for transmitting FEC encoded codewords according to claim 12, comprising:

15. The device for transmitting FEC encoded codewords according to claim 12, characterized in that the construction of the extension matrix of the LT coding matrix corresponding to the N types of intermediate data packets is related to the added redundant information.

16. 13. The apparatus of claim 12, wherein the index module adds the index field to the recovered data packet to indicate a sequence of subpackets in the recovered data packet, a sequence of subpackets in the recovered data packet with a priority corresponding to the priority indication field, and / or a number of subpackets or a proportion of the subpackets in the recovered data packet that belong to a corresponding priority.

17. 13. The apparatus for transmitting FEC encoded code words according to claim 12, characterized in that the index module adds the index field to the intermediate data packet to indicate a sequence of subpackets in the intermediate data packet and / or a sequence of subpackets in the intermediate data packet with a priority corresponding to the priority indication field.

18. 13. The apparatus for transmitting FEC encoded code words according to claim 12, characterized in that the indication information indicates subpackets of each priority level of the recovered data packet and / or subpackets of each priority level of the intermediate data packet.

19. a receiving module for receiving the transmitted restored data packets, intermediate data packets and corresponding indication information; a reconstruction matrix generating module for analyzing an index field and / or a priority indication field included in the reconstruction data packet and the intermediate data packet, and the indication information, and generating a reconstruction matrix based on a coding matrix corresponding to the intermediate data packet and a loss situation of the reconstruction data packet and the intermediate data packet, the coding matrix corresponding to the intermediate data packet including a combination matrix consisting of an LDPC matrix and an identity matrix, and an LT coding matrix; a decoding module for generating an intermediate codeword according to the reconstruction matrix and the received reconstruction data packet and intermediate data packet, and generating a decoded intermediate data packet according to the LT coding matrix and the intermediate codeword corresponding to the intermediate data packet; A restoration module for analyzing total information of headers of subpackets in the intermediate data packet after being decoded, and rearranging the subpackets in the intermediate data packet after being decoded to restore the source data packet; 1. An apparatus for decoding FEC encoded codewords, comprising:

20. 20. The apparatus of claim 19, wherein the index field included in the recovered data packet indicates a sequence of subpackets in the recovered data packet, a sequence of subpackets in the recovered data packet with a priority corresponding to the priority indication field, and / or a number of subpackets or a proportion of the subpackets in the recovered data packet that belong to a corresponding priority.

21. 20. The apparatus of claim 19, wherein the index field included in the intermediate data packet indicates a sequence of subpackets in the intermediate data packet and / or a sequence of subpackets in the intermediate data packet in a priority order corresponding to the priority indication field.

22. 20. The device for decoding FEC encoded code words according to claim 19, wherein the indication information indicates subpackets of each priority level of the recovered data packet and / or subpackets of each priority level of the intermediate data packet.

23. A computer-readable storage medium having a computer program stored thereon, A computer-readable storage medium, characterized in that the computer program, when executed by a processor, realizes the method for transmitting FEC encoded codewords according to any one of claims 1 to 7 and / or the method for decoding FEC encoded codewords according to any one of claims 8 to 11.

Images