A method for reliable transmission of h.264 video frames

Abstract

This invention discloses a reliable transmission method for H.264 video frames. Specifically, when sending data, the transmitting end performs block buffering based on Group of Pictures (GOPs). A Merkle Tree data structure is constructed for each GOP, and the root node value is signed and written into the SEI structure before the first I-frame of the GOP. This completes the processing of one GOP and the data is then sent. The receiving end acquires and detects the received data. When the first I-frame is detected, the root node value is also calculated and compared with the root node value stored in the SEI structure before the first I-frame of the GOP for signature verification. If the signature verification is successful, the data transmission is complete and error-free. If the signature verification fails, data loss or alteration has occurred during transmission, and the transmitting end is notified to retransmit. The beneficial effects of this invention are: utilizing the H.264 supplemental enhancement information (SEI) structure to store signature information, the data signature information is integrated with the video frame data, eliminating the need for additional information transmission channels, and enabling rapid tracing of problematic frame locations, thus achieving reliable transmission of H.264 video data.

Description

Technical Field

[0001] This invention relates to the field of video data transmission reliability verification, and in particular to a reliable transmission method for H.264 video frames. Background Technology

[0002] Video data, containing rich and comprehensive image and temporal information, has become an important medium for information exchange. Video is also a crucial form of evidence. The authenticity and integrity of video data are vital in fields such as news media, education and research, and judicial evidence collection. However, video data transmission may traverse various network environments, including 4G / 5G, ordinary broadband, inter-provincial trunk lines, and different operators. The network environment is difficult to maintain consistently stable, so data loss can occur at any stage of long-distance, high-capacity, high-definition video acquisition, transmission, and distribution, affecting video transmission quality. Traditional reliable video transmission methods require additional storage of verification information, which is detrimental to video data transmission. Summary of the Invention

[0003] The purpose of this invention is to overcome the above-mentioned defects in the prior art and provide a reliable transmission method for H.264 video frames. This method can quickly determine whether the data transmission maintains its originality and integrity without the need for an additional information transmission channel, thus ensuring the reliable transmission of video frames.

[0004] To achieve the above-mentioned objective, this invention provides a reliable transmission method for H.264 video frames, the method comprising the following steps:

[0005] Step S1: Acquire and detect the H.264 video stream data to be sent, divide it into blocks and buffer it in units of GOP image groups, and take one GOP for subsequent processing;

[0006] Step S2: When the first I-frame in the GOP is detected, Merkle Tree operation is started with each frame in the current GOP as the basic calculation unit to obtain the leaf node value, intermediate node value and root node value. Signature calculation is performed on the root node value. All node values ​​and root node signature value of the Merkle Tree data structure are stored in the supplementary enhancement information (SEI) structure before the first I-frame in the current GOP to complete the processing of one GOP image group. Then the GOP data is sent to the receiving end.

[0007] Step S3: Continue the detection and repeat steps S1 and S2;

[0008] Step S4: Acquire and detect the received video stream data, and construct a Merkle Tree data structure in units of GOP image groups; when the first I-frame in the GOP is detected, start the Merkle Tree operation with each frame in the current GOP as the basic calculation unit to obtain the leaf node value, intermediate node value and root node value; perform signature verification calculation with the root node signature value stored in the SEI structure before the first I-frame of the current GOP.

[0009] Step S5: If the signature verification in step S4 is successful, the data transmission is complete and error-free;

[0010] Step S6: If the signature verification fails in step S4, it indicates that data loss or alteration has occurred during data transmission. By comparing the MerkleTree nodes to locate the problematic data, the receiving end notifies the sending end to return to step S1 and resend the data.

[0011] Prior to performing step S1, both the sending end and the receiving end have obtained asymmetric encryption keys.

[0012] Preferably, step S2 specifically comprises:

[0013] Step S201: When the first I-frame in the GOP is detected, Merkle Tree calculation is started, and a hash operation is performed on the I-frame, with its value recorded as Node. 0,O,send ;

[0014] Step S202: Perform hash operations on each subsequent data frame in the same GOP image group, and record the values ​​as Nodes in sequence. 0,j,send (j=0…m), complete the calculation of the leaf node values ​​of the Merkle Tree, where m+1 is the total number of frames of the GOP image group;

[0015] Step S203: Connect two adjacent Nodes 0,j,send and Node 0,j+1,send Connect the preceding and following nodes, perform another hash operation, and record them as Nodes in sequence. i+1,j / 2,send =hash(Node) i,j,send +Node i,j+1,send ), i≥0;

[0016] Step S204: Connect two adjacent Nodes i,j,send Node i,j+1,send Connect the preceding and following nodes together, then perform a hash operation to obtain the Node. i+1,j / 2,send This process involves obtaining several intermediate node values ​​until the root node value (Node) is finally obtained. r,send ;

[0017] Step S205: For Node r,send Sign(Node) is obtained by performing signature calculation.r，send ), and then combine the leaf node values, intermediate node values, root node values, and Sign(Node) values. r，send Write it into the SEI structure before the first I-frame in the current GOP.

[0018] Preferably, step S4 specifically comprises:

[0019] Step S401: Acquire and detect the received video stream data. When the first I-frame in the GOP is detected, start the Merkle Tree calculation, perform a hash operation on the I-frame, and record its value as Node. 0,0,recv ;

[0020] Step S402: Perform hash operations on each subsequent data frame in the same GOP image group, and record the values ​​as Nodes in sequence. 0,j,recv (j=0…m), complete the calculation of the leaf node values ​​of the Merkle Tree, where m+1 is the total number of frames of the GOP image group;

[0021] Step S403: Connect two adjacent Nodes 0,j,recv and Node 0,j+1,recv Connect the preceding and following nodes, perform another hash operation, and record them as Nodes in sequence. i+1,j / 2,recv =hash(Node) i,j,,recv +Node i,j+1,recv ), i≥0;

[0022] Step S404: Connect two adjacent Nodes i,j,recv Node i,j+1,recv Connect the preceding and following nodes together, then perform a hash operation to obtain the Node. i+1,j / 2,recv This process yields several intermediate node values ​​until the root node value (Node) is calculated. r,recv .

[0023] Preferably, in step S6, if the signature verification fails, it indicates that data loss or alteration has occurred during data transmission. In this case, the Merkle Tree data structures of the receiving end and the sending end are compared level by level until the position of the problematic leaf node value is located, i.e., the location of the problematic frame is obtained. The receiving end then notifies the sending end to return to step S1 to retransmit.

[0024] The beneficial effects of this invention are: when the video is sent, the video data is signed, and the signature information is stored using the H.264 supplementary enhancement information (SEI) structure. The verification information is fused with the video frame data, eliminating the need for an additional information transmission channel. Moreover, the video data can be verified and erroneous data frames can be quickly located during each process of transmission, storage, distribution, and playback. Attached Figure Description

[0025] Figure 1This is a flowchart of the H.264 reliable transmission sending and receiving process of the present invention.

[0026] Figure 2 The Merkle Tree structure calculation diagram of H.264 video frames in this invention.

[0027] Figure 3 This is a diagram of the H.264SEI NALU storage Merkle Tree signature chain structure of the present invention.

[0028] Figure 4 This is a flowchart of the video sending end of the present invention.

[0029] Figure 5 This is a flowchart illustrating the workflow of the video receiver of the present invention. Detailed Implementation

[0030] To clearly illustrate the technical features of this solution, the following detailed implementation method will be used to explain the solution.

[0031] Example 1

[0032] See Figure 1 This invention provides a reliable transmission method for H.264 video frames, specifically: using GOP (Group of Pictures) as the unit:

[0033] Sending end: Calculate the Merkle Tree data structure M1 of a GOP image group (i.e., the leaf node value, intermediate node value and root node value obtained by Merkle Tree operation), and the signature value S1 of the root node value, and write the data structure M1 and the signature value S1 into the SEI structure before the first I frame of the current GOP image group;

[0034] The receiving end calculates the Merkle Tree data structure M2 for the GOP image group (i.e., the leaf node values, intermediate node values, and root node values ​​obtained through Merkle Tree operations). It then performs a signature verification calculation by comparing the root node value of data structure M2 with the root node value stored in the SEI structure preceding the first I-frame of the same GOP image group. If the signature verification passes, the process ends; otherwise, it compares M1 and M2 level by level to locate the problematic frame and notifies the sending end to retransmit.

[0035] Example 2

[0036] See Figure 1 This invention provides a reliable transmission method for H.264 video frames, comprising the following steps:

[0037] Step S1: Acquire and detect the H.264 video stream data to be sent, divide it into blocks and buffer it in units of GOP image groups, and take one GOP for subsequent processing;

[0038] Step S2: When the first I-frame in the GOP is detected, Merkle Tree operation is started with each frame in the current GOP as the basic calculation unit to obtain the leaf node value, intermediate node value and root node value. Signature calculation is performed on the root node value. All node values ​​and root node signature value of the Merkle Tree data structure are stored in the supplementary enhancement information (SEI) structure before the first I-frame in the current GOP to complete the processing of one GOP image group. Then the GOP data is sent to the receiving end.

[0039] Step S3: Continue the detection and repeat steps S1 and S2;

[0040] Step S4: Acquire and detect the received video stream data, and construct a Merkle Tree data structure in units of GOP image groups; when the first I-frame in the GOP is detected, start the Merkle Tree operation with each frame in the current GOP as the basic calculation unit to obtain the leaf node value, intermediate node value and root node value; perform signature verification calculation with the root node signature value stored in the SEI structure before the first I-frame of the current GOP.

[0041] Step S5: If the signature verification in step S4 is successful, the data transmission is complete and error-free;

[0042] Step S6: If the signature verification fails in step S4, it indicates that data loss or alteration has occurred during data transmission. By comparing the MerkleTree nodes to locate the problematic data, the receiving end notifies the sending end to return to step S1 and resend the data.

[0043] Before executing step S1, the sending end and the receiving end have obtained the asymmetric encryption key respectively.

[0044] See Figure 2 Step S2 is as follows:

[0045] Step S201: When the first I-frame in the GOP is detected, Merkle Tree calculation is started, and a hash operation is performed on the I-frame, with its value recorded as Node. 0,0,send ;

[0046] Step S202: Perform hash operations on each subsequent data frame in the same GOP image group, and record the values ​​as Nodes in sequence. 0,j,send (j=0…m), complete the calculation of the leaf node values ​​of the Merkle Tree, where m+1 is the total number of frames of the GOP image group;

[0047] Step S203: Connect two adjacent Nodes 0,j,send and Node 0,j+1,send Connect the preceding and following nodes, perform another hash operation, and record them as Nodes in sequence. i+1,j / 2,send=hash(Node) i,j,send +Node i,j+1,send ), i≥0;

[0048] Step S204: Connect two adjacent Nodes i,j,send Node i,j+1,send Connect the preceding and following nodes together, then perform a hash operation to obtain the Node. i+1,j / 2,send This process involves obtaining several intermediate node values ​​until the root node value (Node) is finally obtained. r,send ;

[0049] Step S205: For Node r,send Sign(Node) is obtained by performing signature calculation. r，send ), and then combine the leaf node values, intermediate node values, root node values, and Sign(Node) values. r，send Write it into the SEI structure before the first I-frame in the current GOP.

[0050] Specifically, step S4 is as follows:

[0051] Step S401: Acquire and detect the received video stream data. When the first I-frame in the GOP is detected, start the Merkle Tree calculation, perform a hash operation on the I-frame, and record its value as Node. 0,0,recv ;

[0052] Step S402: Perform hash operations on each subsequent data frame in the same GOP image group, and record the values ​​as Nodes in sequence. 0,j,recv (j=0…m), complete the calculation of the leaf node values ​​of the Merkle Tree, where m+1 is the total number of frames of the GOP image group;

[0053] Step S403: Connect two adjacent Nodes 0,j,recv and Node 0,j+1,recv Connect the preceding and following nodes, perform another hash operation, and record them as Nodes in sequence. i+1,j / 2,recv =hash(Node) i,j,,recv +Node i,j+1,recv ), i≥0;

[0054] Step S404: Connect two adjacent Nodes i,j,recv Node i,j+1,recv Connect the preceding and following nodes together, then perform a hash operation to obtain the Node. i+1,j / 2,recv This process yields several intermediate node values ​​until the root node value (Node) is calculated. r,recv .

[0055] In step S6, if the signature verification fails, it indicates that the data transmission has been lost or altered. The Merkle Tree data structures of the receiving end and the sending end are compared level by level until the position of the problematic leaf node value is located, that is, the location of the problematic frame is obtained. The receiving end then notifies the sending end to return to step S1 to retransmit.

[0056] Example 3

[0057] See Figure 1 This invention provides a reliable transmission method for H.264 video frames, comprising the following steps:

[0058] S1: Before starting the video signature operation, the video sending end and the receiving end have obtained the asymmetric encryption key respectively. The acquisition method belongs to the existing technology and is therefore not within the scope of the technical discussion of this invention.

[0059] S2: The sending end detects the H.264 video stream data to be sent. When the first I-frame is detected, the corresponding data structure is created and the Merkle Tree calculation is started.

[0060] S3: Perform a hash operation on the first complete I-frame data, and denote its value as Node. 0,0,send ;

[0061] S4: Perform a hash operation on the subsequent P or B frame data, and denote the value as Node. 0,j,send (j=0…m), where m+1 is the number of consecutive frames before the next I-frame;

[0062] S5: Connect two adjacent nodes 0,j,send and Node 0,j+1,send Connect the two nodes together, perform another hash operation, and record it as a Node. 1,j =hash(Node) 0,j,send +Node 0,j+1,send ); Node i+1,j / 2,send =hash(Node) i,j.send +Node i,j+1,send );

[0063] S6: Repeat S5, continuously changing the two adjacent nodes. i,j,send Node i,j+1,send Connect the preceding and following nodes together, then perform a hash operation to obtain the Node. i+1,j / 2,send This continues until the last root node value, Node, is obtained. r,send ;

[0064] S7: For Node r,send Sign(Node) is obtained by performing signature calculation. r,send And write it into the SEI structure of the GOP image group;

[0065] S8: After the receiving end obtains the H.264 data, it calculates the value according to steps S2-S8, and finally obtains the root node value Node. r,recv ;

[0066] S9: The receiver obtains the Sign(Node) from the current GOP image group SEI structure. r,send ) value, and use Node r,recv The data is verified. If the verification fails, it means that the video data was illegally modified or lost during transmission and is different from the original data.

[0067] S10: When S9 signature verification fails, the problematic video frame can be quickly located by comparing the Merkle Tree hash chains calculated by the receiving end and the sending end respectively.

[0068] For details, see Figure 4 When the sending end detects the first I-frame data, it initiates Merkle Tree computation. Starting from the first I-frame until the next I-frame, each frame of data undergoes a hash operation to obtain a Node. 0,j,send (j=0…m)(e.g.) Figure 2 As shown), for two consecutive adjacent Nodes 0,j,send Node 0,j+1,send Perform another hash calculation to obtain the Node. 1,i,send Calculate Node in sequence i,j,send (i = 0…n, j = 0…m), until the final unique node value Noder is obtained. r,send For Node r,send Sign(Node) is obtained by performing signature calculation. r,send ).

[0069] See Figure 5 At the receiving end, the same method is used to calculate the Merkle Tree hash chain value and the root node value Node of the received data. s,send The system performs a signature verification calculation. If the signature verification passes, it means that the video frame data has not been lost or modified during transmission and distribution. Otherwise, it means that the received data is different from the original data, and the video data has been illegally modified or lost.

[0070] By comparing the Merkle Tree signature chain, the location of data tampering can be quickly pinpointed. The sender can assign the value of each node in the Merkle Tree to the Node. i,j,send (i = 0…n, j = 0…m) are sent along with the video stream, and the receiving end uses the same method to calculate the Node for receiving data. i,j,recv (i = 0…n, j = 0…m), first check Sign(Node) r,sendIf the signature verification fails, it means that the video frame data has been tampered with. At this time, comparing the Merkle Tree structure of the receiving end and the sending end can quickly locate the location of the problematic data.

[0071] See Figure 3 Based on the H.264 / AVC standard definition, this invention proposes to use the SEI structure of H.264 supplementary enhancement information to store the Merkel Tree hash chain and tree top signature information, which is fully compatible with the H.264 video structure, does not affect the original video data, and does not affect the operation of existing codecs.

[0072] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A reliable transmission method for H.264 video frames, characterized in that, The method includes the following steps: Step S1: Acquire and detect the H.264 video stream data to be sent, divide it into blocks and buffer it in units of GOP image groups, and take one GOP for subsequent processing; Step S2: When the first I-frame in the GOP is detected, MerkleTree operation is started with each frame in the current GOP as the basic calculation unit to obtain the leaf node value, intermediate node value and root node value. The root node value is then signed. All node values ​​and the root node signature value of the MerkleTree data structure are stored in the supplementary enhancement information (SEI) structure before the first I-frame in the current GOP to complete the processing of one GOP image group. The GOP data is then sent to the receiving end. Step S3: Continue the detection and repeat steps S1 and S2; Step S4: Acquire and detect the received video stream data, and construct a Merkle Tree data structure in units of GOP image groups; when the first I-frame in the GOP is detected, start the Merkle Tree operation with each frame in the current GOP as the basic calculation unit to obtain the leaf node value, intermediate node value and root node value; perform signature verification calculation with the root node signature value stored in the SEI structure before the first I-frame of the current GOP. Step S5: If the signature verification in step S4 is successful, the data transmission is complete and error-free; Step S6: If the signature verification fails in step S4, it indicates that data loss or alteration has occurred during data transmission. By comparing the Merkle Tree nodes to locate the problematic data, the receiving end notifies the sending end to return to step S1 and resend the data.

2. The method according to claim 1, characterized in that, Before performing step S1, the sending end and the receiving end have each obtained the asymmetric encryption key.

3. The method according to claim 1, characterized in that, Step S2 specifically involves: Step S201: When a first I frame in a GOP (Group of Pictures) is detected, hash operations are performed on each video frame in the current GOP respectively, starting from the first I frame, to obtain m+1 leaf node values, denoted as Node 0,j,send , j=0, …, m, where Node 0,0,send is the hash value of the first I frame; Step S202: from the leaf node layer, the adjacent node values in the same layer of the Merkle Tree are spliced in front and back according to the arrangement order, a hash operation is performed on the spliced result, a corresponding parent node value in the previous layer is obtained, and the obtained parent node value is repeatedly spliced and hashed until the root node value Node r,send is obtained. Among them, Node i,j,send This represents the value of the j-th node at level i in the Merkle Tree of the sending end, where i represents the level of the node, i=0 represents the leaf node level, j represents the sequence number of the node in the corresponding level, r represents the level of the root node, and send represents the sending end; Step S203: For the root node value Node r,send Perform signature calculation to obtain the root node signature value Sign(Node). r,send The leaf node value, intermediate node value, root node value, and root node signature value are written into the supplementary enhancement information (SEI) structure before the first I-frame in the current GOP image group.

4. The method according to claim 1, characterized in that, Step S4 specifically includes: Step S401: Acquire and detect the received video stream data. When the first I-frame in the current GOP image group is detected, starting from the first I-frame, perform a hash operation on each video frame in the current GOP image group to obtain m+1 leaf node values. The leaf node values ​​are sequentially denoted as Node. 0,j,recv j = 0, ..., m, where Node 0,0,recv The hash value of the first I-frame; Step S402: Starting from the leaf node level, concatenate pairs of adjacent node values ​​in the same level of the Merkle Tree according to their order of arrangement. Perform a hash operation on the concatenation result to obtain the corresponding parent node value in the previous level. Repeat the concatenation and hash operation on the obtained parent node value until the root node value Node is obtained. r,recv ; Among them, Node i,j,recv Represents the value of the j-th node at level i in the Merkle Tree at the receiving end, where i represents the level of the node, i=0 represents the leaf node level, j represents the sequence number of the node at the corresponding level, r represents the level of the root node, and recv represents the receiving end. Step S403: Obtain the root node signature value (Sign) from the Supplemental Enhancement Information (SEI) structure preceding the first I-frame in the current GOP image group. r,send ), and the root node value Node r,recv With the root node signature value Sign(Node) r,send ) Perform verification and calculation.

5. The method according to claim 1, characterized in that, In step S6, if the signature verification fails, it indicates that data transmission has been lost or altered. The Merkle Tree data structures of the receiving end and the sending end are compared level by level until the position of the problematic leaf node value is located, that is, the position of the problematic frame is obtained. The receiving end then notifies the sending end to return to step S1 to retransmit.

Images