Image frame verification method, device, transmitting end device and storage medium

Abstract

The present invention provides an image frame verification method, device, transmitting end device and storage medium, which relates to the field of data verification technology. The method includes: for each GOP, determining the minimum unit corresponding to the GOP, and based on the minimum unit corresponding to the GOP, dividing the GOP into at least two sub-GOPs; adding the model construction parameters corresponding to the GOP to the first frame data of the first sub-GOP in the GOP; determining the first verification model corresponding to each sub-GOP, and adding each first verification model to the first frame data of the corresponding sub-GOP to obtain each updated sub-GOP; and sending each updated sub-GOP to the receiving end device in sequence. The present invention can shorten the time for performing image frame verification on the GOP, thereby improving the efficiency of image frame verification.

Description

Technical Field

[0001] The present invention relates to the field of data verification technology, and in particular to an image frame verification method, device, sending end equipment and storage medium. Background Art

[0002] H.264 encoding technology is widely used in video streaming, real-time communication, storage and other fields due to its efficient compression capability. With the improvement of video quality and resolution, the use of GOP (Group Of Pictures) in H.264 encoding technology is becoming more and more common. GOP refers to a series of continuous image frames. Frames are transmitted between the sending and receiving devices in units of GOP, and Merkle Tree data structures are constructed based on GOP to verify image frames.

[0003] However, the receiving device needs to construct the Merkle Tree data structure for image frame verification only after receiving the entire GOP sent by the sending device. The GOP length is relatively long, and it takes a long time for the sending device and the receiving device to construct the Merkle Tree data structure according to the GOP respectively, resulting in low image frame verification efficiency, which seriously affects the frame transmission process. Summary of the invention

[0004] The present invention provides an image frame verification method, an apparatus, a transmitting end device and a storage medium, which are used to solve the defects in the prior art that the image frame verification efficiency is low and the frame transmission process is seriously affected.

[0005] The present invention provides an image frame verification method, which is applied to a sending end device. The method comprises the following steps.

[0006] For each group of continuous image frames GOP, a minimum unit corresponding to the GOP is determined, and based on the minimum unit corresponding to the GOP, the GOP is divided into at least two sub-GOPs.

[0007] The model building parameters corresponding to the GOP are added to the first frame data of the first sub-GOP in the GOP.

[0008] A first verification model corresponding to each of the sub-GOPs is determined, and each of the first verification models is added to the first frame data of the corresponding sub-GOP to obtain updated sub-GOPs.

[0009] The updated sub-GOPs are sent to a receiving device in sequence, instructing the receiving device to construct a second verification model for each sub-GOP based on the model construction parameters in the first frame data of the first sub-GOP and the first verification model, and to perform image frame verification on each sub-GOP based on the second verification model and the first verification model corresponding to each sub-GOP.

[0010] According to the image frame checking method provided by the present invention, the determining the minimum unit corresponding to the GOP includes:

[0011] Determine a sub-GOP combination corresponding to at least two pre-divided units corresponding to the GOP;

[0012] Determine the number of sub-hash operations and the number of sub-specific operations corresponding to each of the sub-GOP combinations;

[0013] Determining the pre-construction duration of the pre-verification model corresponding to each of the sub-GOP combinations based on the number of sub-hash operations and the number of sub-specific operations corresponding to each of the sub-GOP combinations;

[0014] Based on each of the pre-constructed durations and the original construction duration of the original verification model corresponding to the GOP, a minimum unit corresponding to the GOP is determined from all the pre-divided units.

[0015] According to the image frame verification method provided by the present invention, the determining of the number of sub-hash operations and the number of sub-specific operations corresponding to each sub-GOP combination includes:

[0016] Determine the pre-division quantity of all sub-GOPs in each sub-GOP combination based on the total length corresponding to the GOP and each pre-division unit;

[0017] Determine the number of sub-hash operations corresponding to each of the sub-GOP combinations based on the total length corresponding to the GOP, and the pre-division quantity and pre-division unit corresponding to each of the sub-GOP combinations;

[0018] The total length corresponding to the GOP is determined as the number of sub-specific operations corresponding to each sub-GOP combination.

[0019] According to the image frame verification method provided by the present invention, the method of determining the minimum unit corresponding to the GOP from all pre-segmented units based on the pre-constructed durations and the original construction duration of the original verification model corresponding to the GOP includes:

[0020] Determine at least one candidate build duration that is shorter than the original build duration from all pre-build durations;

[0021] The pre-division unit corresponding to the minimum candidate construction duration among the at least one candidate construction duration is determined as the minimum unit corresponding to the GOP.

[0022] According to the image frame verification method provided by the present invention, the determining of the first verification model corresponding to each sub-GOP includes:

[0023] For each of the GOPs, obtaining an initial hash value corresponding to the GOP;

[0024] Determine a single-frame hash value corresponding to each frame image in each of the sub-GOPs;

[0025] Perform a specific operation on the single-frame hash value corresponding to the first frame image in each of the sub-GOPs and the initial hash value to obtain a check value corresponding to the first frame image in each of the sub-GOPs;

[0026] Performing a specific operation on the single-frame hash value of each non-first frame image in each sub-GOP and the single-frame hash value of the previous frame image to obtain a check value corresponding to each non-first frame image in each sub-GOP;

[0027] Based on all the check values ​​corresponding to the sub-GOPs and the initial hash value, a first check model corresponding to each of the sub-GOPs is determined.

[0028] According to the image frame verification method provided by the present invention, the method further includes:

[0029] Obtaining historical minimum units corresponding to at least two historical GOPs corresponding to the GOP;

[0030] Determine an average value of all historical minimum units and the minimum unit corresponding to the GOP;

[0031] The average value is determined as the minimum unit after the GOP is updated.

[0032] According to the image frame verification method provided by the present invention, the model construction parameters include the total length corresponding to the GOP, the minimum unit and the number of all sub-GOPs.

[0033] The present invention also provides an image frame checking device, which is applied to a sending end device, and the device includes the following modules.

[0034] The segmentation module is used to determine the minimum unit corresponding to each continuous image frame group GOP, and divide the GOP into at least two sub-GOPs based on the minimum unit corresponding to the GOP.

[0035] The adding module is used to add the model building parameters corresponding to the GOP to the first frame data of the first sub-GOP in the GOP.

[0036] The determination module is used to determine the first verification model corresponding to each sub-GOP, and add each first verification model to the first frame data of the corresponding sub-GOP to obtain the updated sub-GOP.

[0037] A sending module is used to send the updated sub-GOPs to a receiving device in sequence, instructing the receiving device to build a second verification model for each sub-GOP based on the model construction parameters in the first frame data of the first sub-GOP and the first verification model, and to perform image frame verification on each sub-GOP based on the second verification model and the first verification model corresponding to each sub-GOP.

[0038] The present invention also provides a transmitting end device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned image frame verification methods is implemented.

[0039] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the image frame verification method described in any one of the above is implemented.

[0040] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the image frame verification method described above is implemented.

[0041] The image frame verification method, device, transmitting end device and storage medium provided by the present invention divide a GOP into at least two sub-GOPs with the minimum unit as the interval, add the model construction parameters corresponding to the GOP to the first frame data of the first sub-GOP, construct the corresponding first verification model with the sub-GOP as the unit, and add the first verification model to the first frame data of the corresponding sub-GOP. Afterwards, the updated sub-GOPs are sent to the receiving end device one by one in sequence with the sub-GOP as the unit, indicating that the receiving end device can start to build the corresponding second verification model after detecting the first frame data of the first sub-GOP, so as to perform image frame verification on each sub-GOP according to the second verification model and the first verification model. In the present invention, by reducing the GOP to the minimum unit, the complexity corresponding to the construction of the verification model is reduced, thereby shortening the construction time of the verification model. At the same time, the receiving end device can start to build the second verification model after detecting the first frame data, without having to build the verification model after receiving the entire GOP. The time when the receiving end device builds the second verification model is advanced, shortening the time for image frame verification of the GOP, thereby improving the efficiency of image frame verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a flowchart of an image frame verification method provided by an embodiment of the present invention.

[0044] Figure 2 It is a comparative schematic diagram of the time envelope diagram corresponding to the construction verification model provided by the embodiment of the present invention.

[0045] Figure 3 It is a structural schematic diagram of an image frame checking device provided by an embodiment of the present invention.

[0046] Figure 4 It is a structural diagram of a sending end device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In the prior art, the image frame verification process using GOP (Group Of Pictures) includes: after the sending device constructs a Merkle Tree data structure based on the GOP, it performs a signature calculation on the root node value of the Merkle Tree data structure, and writes the calculation result into the SEI (Supplemental enhancement information) structure before the first I frame of the GOP. After writing the calculation result, the GOP can be sent to the receiving device. After receiving the complete GOP, the receiving device also constructs a Merkle Tree data structure and calculates the root node value, extracts the root node value calculated by the sending device from the first I frame of the GOP, and performs signature verification calculation on the two root node values. If the signature verification succeeds, the GOP transmission is complete and correct. If the signature verification fails, frame loss or changes occur in the data transmission, and the sending device can be notified to resend.

[0049] However, the method of constructing a Merkle Tree data structure according to GOP to perform image frame verification in the prior art has the following two problems.

[0050] On the one hand, when the GOP length is fixed, in an environment where the video content changes little or the network bandwidth is low, the GOP length is longer. When the sending device and the receiving device construct the Merkle Tree data structure based on the GOP, there are more leaf nodes and a deeper level. The construction process performs more hash operations, resulting in a longer construction time for the Merkle Tree data structure, resulting in a longer time-consuming image frame verification for the GOP, which leads to lower image frame verification efficiency and even affects the frame transmission process.

[0051] On the other hand, the frame transmission between the transmitting device and the receiving device is performed in units of GOP, and the receiving device can only construct the Merkle Tree data structure after receiving the entire GOP. When the GOP length changes dynamically, that is, the lengths of the transmitted GOPs are different, the construction time lengths of the Merkle Tree data structure based on different GOPs are different, resulting in uneven construction time intervals of two adjacent Merkle Tree data structures. Based on this, there is a mismatch between the construction speeds of the Merkle Tree data structure constructed by the receiving device and the transmitting device. For example, when the length of the first GOP is shorter than the length of the second GOP, after the transmitting device sends the first GOP to the receiving device, the transmitting device constructs the Merkle Tree data structure based on the second GOP. At the same time, the receiving device constructs the Merkle Tree data structure based on the first GOP. Since the length of the first GOP is shorter than the length of the second GOP, the construction time length of the Merkle Tree data structure constructed by the receiving device is shorter than the construction time length of the Merkle Tree data structure constructed by the transmitting device. After verifying the first GOP, the receiving device needs to wait for the transmitting device to transmit the second GOP, resulting in a large delay in the frame transmission process, which seriously affects the frame transmission process.

[0052] Therefore, in view of the above problems in the prior art, an embodiment of the present invention provides an image frame verification method, which is applied to a sending end device. Figure 1 FIG. 1 is a flow chart of an image frame verification method provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps 110 to 140.

[0053] Step 110: For each GOP, determine the minimum unit corresponding to the GOP, and divide the GOP into at least two sub-GOPs based on the minimum unit corresponding to the GOP.

[0054] GOP refers to a group of continuous image frames, which includes at least one group of I frames, P frames and B frames. Among them, I frame is a key frame, which means a complete image. I frame can be decoded without referring to other frames. P frame is a predicted frame, which is predicted and encoded based on at least one previous decoded frame (usually I frame or P frame). B frame is a bidirectional predicted frame, which is predicted and encoded based on the two previous decoded frames.

[0055] For each GOP, determine the minimum unit of the GOP, which is the maximum number of image frames included in each sub-GOP after segmentation, and segment the GOP with the minimum unit as an interval to obtain at least two sub-GOPs, so as to facilitate the subsequent construction of a verification model based on the sub-GOP and frame transmission in units of sub-GOPs.

[0056] Step 120: Add the model building parameters corresponding to the GOP to the first frame data of the first sub-GOP in the GOP.

[0057] Specifically, after a GOP is divided into at least two sub-GOPs, the model construction parameters corresponding to the GOP can be determined, and the model construction parameters include the total length corresponding to the GOP, the minimum unit, and the number of all sub-GOPs. The total length corresponding to the GOP is the number of image frames included in the GOP. The number of all sub-GOPs is calculated based on the total length and the minimum unit of the GOP, and the number of all sub-GOPs can be Among them, m represents the total length of the GOP, and n represents the smallest unit. represents rounding down, a represents a constant, and , when the total length corresponding to the GOP is an integer multiple of the minimum unit, a=0, and when the total length corresponding to the GOP is not an integer multiple of the minimum unit, a=1. That is, when the total length corresponding to the GOP is an integer multiple of the minimum unit, the lengths of all sub-GOPs obtained by segmentation are the minimum unit, and when the total length corresponding to the GOP is not an integer multiple of the minimum unit, the length of the last sub-GOP obtained by segmentation is less than the minimum unit.

[0058] After determining the model construction parameters corresponding to the GOP, the model construction parameters can be added to the first frame data of the first sub-GOP after the GOP is divided, so that the receiving device can clearly understand the construction rules of the second verification model corresponding to each sub-GOP after receiving the first frame data of the first sub-GOP.

[0059] Step 130: Determine the first verification model corresponding to each of the sub-GOPs, and add each of the first verification models to the first frame data of the corresponding sub-GOP to obtain updated sub-GOPs.

[0060] Specifically, the sending device constructs a first verification model corresponding to each sub-GOP according to the order of each sub-GOP, and adds the determined first verification model to the first frame data of the corresponding sub-GOP, so that after each sub-GOP is sent to the receiving device, the receiving device can obtain the first verification model after detecting the first frame data of each sub-GOP, and start to construct the second verification model corresponding to each sub-GOP, thereby shortening the time for image frame verification of the GOP and improving the efficiency of image frame verification.

[0061] Step 140, sending the updated sub-GOPs to the receiving device in sequence, instructing the receiving device to construct a second verification model for each sub-GOP based on the model construction parameters in the first frame data of the first sub-GOP and the first verification model, and perform image frame verification on each sub-GOP based on the second verification model and the first verification model corresponding to each sub-GOP.

[0062] Specifically, the sub-GOPs after segmentation are arranged in order, and after the sending end device completes constructing the first verification model corresponding to each sub-GOP, the sub-GOP can be sent to the receiving end device. After receiving the first frame data of the first sub-GOP, the receiving end device can extract the model construction parameters from the first frame data of the first sub-GOP, clarify the construction rules of each sub-GOP in the GOP, and extract the first verification model corresponding to the first sub-GOP. Afterwards, the receiving end device can start to construct the second verification model corresponding to the sub-GOP every time it receives the first frame data of a sub-GOP, and advance the time when the receiving end device constructs the second verification model, so there is no need to construct the verification model after receiving the entire GOP, thereby shortening the time for image frame verification of the GOP and improving the efficiency of image frame verification.

[0063] The image frame verification method provided by the embodiment of the present invention divides a GOP into at least two sub-GOPs with the minimum unit as the interval, adds the model construction parameters corresponding to the GOP to the first frame data of the first sub-GOP, constructs the corresponding first verification model with the sub-GOP as the unit, and adds the first verification model to the first frame data of the corresponding sub-GOP. Afterwards, the updated sub-GOPs are sent to the receiving end device one by one in sequence with the sub-GOP as the unit, indicating that the receiving end device can start to build the corresponding second verification model after detecting the first frame data of the first sub-GOP, so as to perform image frame verification on each sub-GOP according to the second verification model and the first verification model. In the present invention, by reducing the GOP to the minimum unit, the complexity corresponding to the construction of the verification model is reduced, thereby shortening the construction time of the verification model. At the same time, the receiving end device can start to build the second verification model after detecting the first frame data, without having to build the verification model after receiving the entire GOP. The time when the receiving end device builds the second verification model is advanced, shortening the time for image frame verification of the GOP, thereby improving the efficiency of image frame verification.

[0064] Further, the determining of the first verification model corresponding to each of the sub-GOPs includes:

[0065] For each of the GOPs, obtaining an initial hash value corresponding to the GOP;

[0066] Determine a single-frame hash value corresponding to each frame image in each of the sub-GOPs;

[0067] Perform a specific operation on the single-frame hash value corresponding to the first frame image in each of the sub-GOPs and the initial hash value to obtain a check value corresponding to the first frame image in each of the sub-GOPs;

[0068] Performing a specific operation on the single-frame hash value of each non-first frame image in each sub-GOP and the single-frame hash value of the previous frame image to obtain a check value corresponding to each non-first frame image in each sub-GOP;

[0069] Based on all the check values ​​corresponding to the sub-GOPs and the initial hash value, a first check model corresponding to each of the sub-GOPs is determined.

[0070] Specifically, since there is a problem of mismatch between the construction speed of the receiving device and the sending device when constructing the Merkle Tree data structure according to the GOP in the prior art, in the embodiment of the present invention, the initial hash value of the GOP is randomly generated. For each sub-GOP, a single-frame hash value corresponding to each frame image in the sub-GOP is generated. Afterwards, a specific operation is performed on the single-frame hash value of the first frame image in the sub-GOP and the initial hash value to obtain a check value corresponding to the first frame image in the sub-GOP. The check value corresponding to the first frame image in the sub-GOP can be expressed as: , where D1 represents the check value corresponding to the first frame image, H1 represents the single-frame hash value corresponding to the first frame image, and H0 represents the initial hash value corresponding to the GOP. Represents a specific operation, which may include an XOR operation or an append operation, etc. The append operation is used to perform element concatenation, which is not limited in the embodiment of the present invention.

[0071] Afterwards, starting from the second frame image of the sub-GOP, a specific operation is performed on the single-frame hash value corresponding to each frame image and the hash value corresponding to the previous frame image to determine the check value corresponding to each frame image. The check value corresponding to each non-first frame image in the sub-GOP can be expressed as: Among them, D i Indicates the check value corresponding to each non-first frame image; H i Indicates the single-frame hash value corresponding to the i-th non-first frame image; H i-1 Indicates the single-frame hash value corresponding to the i-1th non-first frame image; i is an integer, and 1 <i<n。

[0072] Afterwards, the check values ​​corresponding to each frame image in the sub-GOP are sorted according to the arrangement order of each frame image to obtain a hash chain, and the initial hash value and the hash chain corresponding to the sub-GOP are used as the first check model corresponding to the sub-GOP.

[0073] Further, the determining the minimum unit corresponding to the GOP includes:

[0074] Determine a sub-GOP combination corresponding to at least two pre-divided units corresponding to the GOP;

[0075] Determine the number of sub-hash operations and the number of sub-specific operations corresponding to each of the sub-GOP combinations;

[0076] Determining the pre-construction duration of the pre-verification model corresponding to each of the sub-GOP combinations based on the number of sub-hash operations and the number of sub-specific operations corresponding to each of the sub-GOP combinations;

[0077] Based on each of the pre-constructed durations and the original construction duration of the original verification model corresponding to the GOP, a minimum unit corresponding to the GOP is determined from all the pre-divided units.

[0078] Specifically, for each GOP, at least two pre-division units corresponding to the GOP are obtained, and the GOP is pre-divided with the pre-division units as intervals. Since the values ​​of the pre-division units are different, different sub-GOP combinations will be obtained after the GOP is divided, that is, the number of pre-divisions of the sub-GOPs included in each sub-GOP combination is different. For example, taking the total length corresponding to the GOP as 10, and the two pre-division units as 2 and 3 respectively, when the pre-division unit is 2, the GOP can be divided into a first sub-GOP combination, and the first sub-GOP combination includes 5 sub-GOPs of equal length. When the pre-division unit is 3, the GOP can be divided into a second sub-GOP combination, and the second sub-GOP combination includes 4 sub-GOPs, and the lengths of the first 3 sub-GOPs are equal and are all 3, and the length of the last sub-GOP is 1.

[0079] Further, the determining of the number of sub-hash operations and the number of sub-specific operations corresponding to each of the sub-GOP combinations includes:

[0080] Determine the pre-division quantity of all sub-GOPs in each sub-GOP combination based on the total length corresponding to the GOP and each pre-division unit;

[0081] Determine the number of sub-hash operations corresponding to each of the sub-GOP combinations based on the total length corresponding to the GOP, and the pre-division quantity and pre-division unit corresponding to each of the sub-GOP combinations;

[0082] Determine the total length corresponding to the GOP as the number of sub-specific operation times corresponding to each combination of sub-GOPs.

[0083] Specifically, after dividing the GOP at intervals of each pre-segmentation unit, determine the number of pre-segmentations of all sub-GOPs included in each combination of sub-GOPs. Then, for each combination of sub-GOPs, according to the process of constructing a corresponding pre-verification model for each sub-GOP in the combination of sub-GOPs, determine the number of sub-hash operation times and the number of sub-specific operation times required for constructing the corresponding pre-verification model for all sub-GOPs in the combination of sub-GOPs. The number of sub-hash operation times is as shown in formula (1), and formula (1) is:

[0084] .

[0085] Among them, represents the number of sub-hash operation times corresponding to the combination of sub-GOPs; n j represents the pre-segmentation unit corresponding to the j-th combination of sub-GOPs; j is an integer, and 1 < j < P, where P represents the total number of all pre-segmentation units. It should be noted that when the total length corresponding to the GOP is an integer multiple of the j-th pre-segmentation unit, , that is, the number of pre-segmentations corresponding to the combination of sub-GOPs is ; when the total length corresponding to the GOP is not an integer multiple of the j-th pre-segmentation unit, the corresponding quantity is 1, that is, the number of pre-segmentations corresponding to the combination of sub-GOPs is .

[0086] The number of sub-specific operation times is equal to the total length of the GOP, that is, , among them, represents the number of sub-specific operation times.

[0087] After determining the number of sub-hash operation times and the number of sub-specific operation times, the product of the number of sub-hash operation times and the time-consuming of a single hash operation can be determined as the sub-hash operation time-consuming corresponding to the combination of sub-GOPs, and the sub-hash operation time-consuming can be expressed as: , where T1 j represents the sub-hash operation time-consuming corresponding to the j-th pre-segmentation unit, and t Hash represents the time-consuming of a single hash operation.

[0088] At the same time, the sub-specific operation time-consuming corresponding to the combination of sub-GOPs can be determined according to the product of the number of sub-specific operation times and the time-consuming of a single specific operation, and the sub-specific operation time-consuming is expressed as: , among them, represents the sub-specific operation time-consuming corresponding to the combination of sub-GOPs, represents the time-consuming of a single specific operation, and , represents the time ratio coefficient, which is determined by the rate of the specific operation and the hash operation, and .

[0089] After determining the sub-hash operation time and the sub-specific operation time corresponding to the sub-GOP combination, the sum of the sub-hash operation time and the sub-specific operation time is determined as the pre-construction duration of the pre-verification model corresponding to the sub-GOP combination.

[0090] Afterwards, the original construction time of constructing the original verification model according to the GOP is obtained. The original verification model can be a Merkle Tree data structure. The Merkle Tree data structure is a binary tree. When constructing the Merkle Tree data structure, the number of hash operations required to be performed at the bottom layer is equal to the total length of the GOP, and the number of hash operations performed at the previous layer is half of the number of hash operations performed at the next layer. Based on this rule, the original number of hash operations H required to be performed to construct the Merkle Tree data structure according to the GOP can be determined. m As shown in formula (2), formula (2) is:

[0091] .

[0092] After determining the original number of hash operations, the product of the original number of hash operations and the time consumption of a single hash operation can be determined as the original construction time for constructing the original verification model according to the GOP. The original construction time T m It can be expressed as: .

[0093] It should be noted that the number of original hash operations required to construct the Merkle Tree data structure according to GOP is greater than the number of sub-hash operations corresponding to each sub-GOP combination in this application, that is, For example, if the total length of a GOP is 16 frames, the number of hash operations required to construct the Merkle Tree data structure is equal to the total length of the GOP, and the number of hash operations performed on the previous layer is half of the number of hash operations performed on the next layer. Therefore, the original number of hash operations required to construct the Merkle Tree data structure based on the GOP is The corresponding number of sub-hash operations in the embodiment of the present invention is Second-rate, .

[0094] After determining the pre-construction duration corresponding to each sub-GOP combination, each pre-construction duration can be compared with the original construction duration of the original verification model constructed based on the GOP. According to the comparison result, the optimal minimum unit of the GOP is determined, so that the total time spent on constructing the first verification model corresponding to each sub-GOP based on the minimum unit is less than the original construction duration.

[0095] Further, the determining the minimum unit corresponding to the GOP from all pre-segmented units based on the pre-built durations and the original built duration of the original verification model corresponding to the GOP includes:

[0096] Determine at least one candidate build duration that is shorter than the original build duration from all pre-build durations;

[0097] The pre-division unit corresponding to the minimum candidate construction duration among the at least one candidate construction duration is determined as the minimum unit corresponding to the GOP.

[0098] Specifically, after determining the pre-built duration and the original built duration corresponding to each pre-segmented unit, each pre-built duration can be compared with the original built duration, and at least one candidate built duration that is smaller than the original built duration can be determined from all pre-built durations. After that, all candidate built durations are sorted in ascending order to determine the minimum candidate built duration, which is the minimum unit corresponding to the GOP.

[0099] Figure 2 is a schematic diagram of the comparison of the time envelope diagram corresponding to the construction of the verification model provided by the embodiment of the present invention, such as Figure 2 The total length of the GOP is used as the horizontal axis, and the construction time of the verification model is used as the vertical axis to construct the time envelope diagrams corresponding to the fixed GOP length in the prior art, the dynamic change of the GOP length, and the embodiment of the present application. Figure 2 It can be seen that in the prior art, when the GOP length is fixed, the construction time of the verification model corresponding to each GOP is relatively balanced, but the construction time is relatively long. In the prior art, when the GOP length changes dynamically, the construction time is relatively long, and the envelope fluctuation is relatively large. In the embodiment of the present invention, the construction time of the verification model corresponding to each GOP is relatively short, and the envelope fluctuation is relatively small.

[0100] In the embodiment of the present invention, the total time consumption of the first verification model corresponding to each sub-GOP constructed according to the minimum unit is less than the original construction time, which shortens the construction time of the verification model constructed by the transmitting device and the receiving device. In addition, after the GOP is minimized, the transmitting device and the receiving device perform frame transmission in sub-GOP units, and the lengths of two adjacent sub-GOPs are relatively balanced, so that the construction time intervals of the verification model constructed by the transmitting device and the receiving device are relatively uniform, and the construction speeds corresponding to the transmitting device and the receiving device are matched. The delay in the frame transmission process is greatly reduced, ensuring the smoothness, reliability and security of the frame transmission process.

[0101] Furthermore, the method further comprises:

[0102] Obtaining historical minimum units corresponding to at least two historical GOPs corresponding to the GOP;

[0103] Determine an average value of all historical minimum units and the minimum unit corresponding to the GOP;

[0104] The average value is determined as the minimum unit after the GOP is updated.

[0105] Specifically, taking into account the different lengths of the sub-GOPs of each GOP, the minimum unit can also be averaged based on the historical minimum units corresponding to at least two historical GOPs before the GOP by combining the historical GOPs, that is, the average value of all historical minimum units and the minimum unit is calculated, and the average value is used as the optimal minimum unit of the GOP.

[0106] It should be noted that after the receiving end device detects the first frame data of the first sub-GOP of the GOP, it can extract the model construction parameters of the GOP and the first verification model corresponding to the first sub-GOP from the first frame data of the first sub-GOP, and after detecting the first frame data, it can start to construct the second verification model corresponding to the first sub-GOP according to the model construction parameters and the initial hash value in the first verification model. The construction process of the second verification model is the same as the construction process of the first verification model, that is, a single-frame hash value corresponding to each frame image in the first sub-GOP is generated, and the single-frame hash value of the first frame image and the initial hash value are specifically calculated to obtain the verification value corresponding to the first frame image in the first sub-GOP. Afterwards, taking the second frame image of the first sub-GOP as the starting point, the single-frame hash value of each frame image and the single-frame hash value of the previous frame image are specifically calculated to determine the verification value corresponding to each frame image, and the verification values ​​corresponding to each frame image are sorted according to the arrangement order of each frame image to obtain a hash chain, and the hash chain and the initial hash value are used as the second verification model of the first sub-GOP. After constructing the second verification model, the receiving device can compare the hash chain in the second verification model with the hash chain in the first verification model. If the hash chain in the second verification model is exactly the same as the hash chain in the first verification model, it indicates that the first sub-GOP data is complete and not lost. If the hash chain in the second verification model is not exactly the same as the hash chain in the first verification model, it indicates that the first sub-GOP has lost frames. The lost image frame can be determined based on the difference between the two hash chains. If the lost image frame is an I frame, the loss of the I frame will cause video interruption or serious errors. At this time, the receiving device can instruct the sending device to resend the lost I frame or the sub-GOP where the frame is lost. If the lost image frame is a P frame or a B frame, the loss of the P frame may cause partial picture distortion, and the loss of the B frame may cause temporary picture distortion, which has a small impact on video transmission. The receiving device can instruct the sending device to resend the lost P frame or B frame or the corresponding sub-GOP, or it may not require the sending device to retransmit.

[0107] The image frame checking process of each subsequent sub-GOP may refer to the image frame checking process of the first sub-GOP, and the embodiment of the present invention will not be described in detail here.

[0108] The image frame verification device provided by the present invention is described below. The image frame verification device described below and the image frame verification method described above can be referenced to each other.

[0109] The embodiment of the present invention further provides an image frame checking device, which is applied to a sending end device. Figure 3 is a schematic diagram of the structure of an image frame checking device provided by an embodiment of the present invention, such as Figure 3 As shown, the image frame checking device 300 includes: a segmentation module 310 , an adding module 320 , a determination module 330 and a sending module 340 .

[0110] The segmentation module 310 is used to determine, for each continuous image frame group GOP, a minimum unit corresponding to the GOP, and to segment the GOP into at least two sub-GOPs based on the minimum unit corresponding to the GOP.

[0111] The adding module 320 is used to add the model building parameters corresponding to the GOP to the first frame data of the first sub-GOP in the GOP.

[0112] The determination module 330 is used to determine the first verification model corresponding to each of the sub-GOPs, and add each of the first verification models to the first frame data of the corresponding sub-GOP to obtain each of the updated sub-GOPs.

[0113] The sending module 340 is used to send the updated sub-GOPs to the receiving device in sequence, instructing the receiving device to build a second verification model for each sub-GOP based on the model construction parameters in the first frame data of the first sub-GOP and the first verification model, and perform image frame verification on each sub-GOP based on the second verification model and the first verification model corresponding to each sub-GOP.

[0114] The image frame verification device provided by the embodiment of the present invention divides a GOP into at least two sub-GOPs with the minimum unit as the interval, adds the model construction parameters corresponding to the GOP to the first frame data of the first sub-GOP, constructs the corresponding first verification model with the sub-GOP as the unit, and adds the first verification model to the first frame data of the corresponding sub-GOP. Afterwards, the updated sub-GOPs are sent to the receiving end device in sequence one by one with the sub-GOP as the unit, indicating that the receiving end device can start to build the corresponding second verification model after detecting the first frame data of the first sub-GOP, so as to perform image frame verification on each sub-GOP according to the second verification model and the first verification model. In the present invention, by reducing the GOP to the minimum unit, the complexity corresponding to the construction of the verification model is reduced, thereby shortening the construction time of the verification model. At the same time, the receiving end device can start to build the second verification model after detecting the first frame data, without having to build the verification model after receiving the entire GOP. The time when the receiving end device builds the second verification model is advanced, shortening the time for image frame verification of the GOP, thereby improving the efficiency of image frame verification.

[0115] Optionally, the segmentation module 310 is specifically configured to:

[0116] Determine a sub-GOP combination corresponding to at least two pre-divided units corresponding to the GOP;

[0117] Determine the number of sub-hash operations and the number of sub-specific operations corresponding to each of the sub-GOP combinations;

[0118] Determining the pre-construction duration of the pre-verification model corresponding to each of the sub-GOP combinations based on the number of sub-hash operations and the number of sub-specific operations corresponding to each of the sub-GOP combinations;

[0119] Based on each of the pre-constructed durations and the original construction duration of the original verification model corresponding to the GOP, a minimum unit corresponding to the GOP is determined from all the pre-divided units.

[0120] Optionally, the segmentation module 310 is specifically configured to:

[0121] Determine the pre-division quantity of all sub-GOPs in each sub-GOP combination based on the total length corresponding to the GOP and each pre-division unit;

[0122] Determine the number of sub-hash operations corresponding to each of the sub-GOP combinations based on the total length corresponding to the GOP, and the pre-division quantity and pre-division unit corresponding to each of the sub-GOP combinations;

[0123] The total length corresponding to the GOP is determined as the number of sub-specific operations corresponding to each sub-GOP combination.

[0124] Optionally, the segmentation module 310 is specifically configured to:

[0125] Determine at least one candidate build duration that is shorter than the original build duration from all pre-build durations;

[0126] The pre-division unit corresponding to the minimum candidate construction duration among the at least one candidate construction duration is determined as the minimum unit corresponding to the GOP.

[0127] Optionally, the determination module 330 is specifically configured to:

[0128] For each of the GOPs, obtaining an initial hash value corresponding to the GOP;

[0129] Determine a single-frame hash value corresponding to each frame image in each of the sub-GOPs;

[0130] Perform a specific operation on the single-frame hash value corresponding to the first frame image in each of the sub-GOPs and the initial hash value to obtain a check value corresponding to the first frame image in each of the sub-GOPs;

[0131] Performing a specific operation on the single-frame hash value of each non-first frame image in each sub-GOP and the single-frame hash value of the previous frame image to obtain a check value corresponding to each non-first frame image in each sub-GOP;

[0132] Based on all the check values ​​corresponding to the sub-GOPs and the initial hash value, a first check model corresponding to each of the sub-GOPs is determined.

[0133] Optionally, the image frame checking device 300 further includes an updating module, which is specifically used for:

[0134] Obtaining historical minimum units corresponding to at least two historical GOPs corresponding to the GOP;

[0135] Determine an average value of all historical minimum units and the minimum unit corresponding to the GOP;

[0136] The average value is determined as the minimum unit after the GOP is updated.

[0137] Optionally, the model building parameters include the total length corresponding to the GOP, the minimum unit and the number of all sub-GOPs.

[0138] Figure 4 is a schematic diagram of the structure of a transmitting end device provided in an embodiment of the present invention, such as Figure 4 As shown, the transmitting end device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the image frame verification method, which includes:

[0139] For each continuous image frame group GOP, determining a minimum unit corresponding to the GOP, and dividing the GOP into at least two sub-GOPs based on the minimum unit corresponding to the GOP;

[0140] Adding the model building parameters corresponding to the GOP to the first frame data of the first sub-GOP in the GOP;

[0141] Determine a first verification model corresponding to each of the sub-GOPs, and add each of the first verification models to the first frame data of the corresponding sub-GOP to obtain updated sub-GOPs;

[0142] The updated sub-GOPs are sent to a receiving device in sequence, instructing the receiving device to construct a second verification model for each sub-GOP based on the model construction parameters in the first frame data of the first sub-GOP and the first verification model, and to perform image frame verification on each sub-GOP based on the second verification model and the first verification model corresponding to each sub-GOP.

[0143] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0144] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the image frame verification method provided by the above methods, the method includes:

[0145] For each continuous image frame group GOP, determining a minimum unit corresponding to the GOP, and dividing the GOP into at least two sub-GOPs based on the minimum unit corresponding to the GOP;

[0146] Adding the model building parameters corresponding to the GOP to the first frame data of the first sub-GOP in the GOP;

[0147] Determine a first verification model corresponding to each of the sub-GOPs, and add each of the first verification models to the first frame data of the corresponding sub-GOP to obtain updated sub-GOPs;

[0148] The updated sub-GOPs are sent to a receiving device in sequence, instructing the receiving device to construct a second verification model for each sub-GOP based on the model construction parameters in the first frame data of the first sub-GOP and the first verification model, and to perform image frame verification on each sub-GOP based on the second verification model and the first verification model corresponding to each sub-GOP.

[0149] In another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the image frame verification method provided by the above methods is implemented, and the method comprises:

[0150] For each continuous image frame group GOP, determining a minimum unit corresponding to the GOP, and dividing the GOP into at least two sub-GOPs based on the minimum unit corresponding to the GOP;

[0151] Adding the model building parameters corresponding to the GOP to the first frame data of the first sub-GOP in the GOP;

[0152] Determine a first verification model corresponding to each of the sub-GOPs, and add each of the first verification models to the first frame data of the corresponding sub-GOP to obtain updated sub-GOPs;

[0153] The updated sub-GOPs are sent to a receiving device in sequence, instructing the receiving device to construct a second verification model for each sub-GOP based on the model construction parameters in the first frame data of the first sub-GOP and the first verification model, and to perform image frame verification on each sub-GOP based on the second verification model and the first verification model corresponding to each sub-GOP.

[0154] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0155] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image frame verification method, characterized in that: Applied to a transmitting device, the method comprises: For each continuous image frame group GOP, determining a minimum unit corresponding to the GOP, and dividing the GOP into at least two sub-GOPs based on the minimum unit corresponding to the GOP; Adding the model building parameters corresponding to the GOP to the first frame data of the first sub-GOP in the GOP; Determine a first verification model corresponding to each of the sub-GOPs, and add each of the first verification models to the first frame data of the corresponding sub-GOP to obtain updated sub-GOPs; The updated sub-GOPs are sent to a receiving device in sequence, instructing the receiving device to construct a second verification model for each sub-GOP based on the model construction parameters in the first frame data of the first sub-GOP and the first verification model, and to perform image frame verification on each sub-GOP based on the second verification model and the first verification model corresponding to each sub-GOP.

2. The image frame checking method according to claim 1, characterized in that: The determining the minimum unit corresponding to the GOP includes: Determine a sub-GOP combination corresponding to at least two pre-divided units corresponding to the GOP; Determine the number of sub-hash operations and the number of sub-specific operations corresponding to each of the sub-GOP combinations; Determining the pre-construction duration of the pre-verification model corresponding to each of the sub-GOP combinations based on the number of sub-hash operations and the number of sub-specific operations corresponding to each of the sub-GOP combinations; Based on each of the pre-constructed durations and the original construction duration of the original verification model corresponding to the GOP, a minimum unit corresponding to the GOP is determined from all the pre-divided units.

3. The image frame verification method according to claim 2, characterized in that: The determining of the number of sub-hash operations and the number of sub-specific operations corresponding to each of the sub-GOP combinations includes: Determine the pre-division quantity of all sub-GOPs in each sub-GOP combination based on the total length corresponding to the GOP and each pre-division unit; Determine the number of sub-hash operations corresponding to each of the sub-GOP combinations based on the total length corresponding to the GOP, and the pre-division quantity and pre-division unit corresponding to each of the sub-GOP combinations; The total length corresponding to the GOP is determined as the number of sub-specific operations corresponding to each sub-GOP combination.

4. The image frame checking method according to claim 2, characterized in that: The determining the minimum unit corresponding to the GOP from all pre-segmented units based on the pre-built durations and the original built duration of the original verification model corresponding to the GOP comprises: Determine at least one candidate build duration that is shorter than the original build duration from all pre-build durations; The pre-division unit corresponding to the minimum candidate construction duration among the at least one candidate construction duration is determined as the minimum unit corresponding to the GOP.

5. The image frame verification method according to any one of claims 1 to 4, characterized in that: The determining of the first verification model corresponding to each of the sub-GOPs includes: For each of the GOPs, obtaining an initial hash value corresponding to the GOP; Determine a single-frame hash value corresponding to each frame image in each of the sub-GOPs; Perform a specific operation on the single-frame hash value corresponding to the first frame image in each of the sub-GOPs and the initial hash value to obtain a check value corresponding to the first frame image in each of the sub-GOPs; Performing a specific operation on the single-frame hash value of each non-first frame image in each sub-GOP and the single-frame hash value of the previous frame image to obtain a check value corresponding to each non-first frame image in each sub-GOP; Based on all the check values ​​corresponding to the sub-GOPs and the initial hash value, a first check model corresponding to each of the sub-GOPs is determined.

6. The image frame verification method according to any one of claims 1 to 4, characterized in that: The method further comprises: Obtaining historical minimum units corresponding to at least two historical GOPs corresponding to the GOP; Determine an average value of all historical minimum units and the minimum unit corresponding to the GOP; The average value is determined as the minimum unit after the GOP is updated.

7. The image frame verification method according to any one of claims 1 to 4, characterized in that: The model building parameters include the total length corresponding to the GOP, the minimum unit and the number of all sub-GOPs.

8. An image frame checking device, characterized in that: Applied to a transmitting device, the device comprises: A segmentation module, for determining, for each continuous image frame group GOP, a minimum unit corresponding to the GOP, and segmenting the GOP into at least two sub-GOPs based on the minimum unit corresponding to the GOP; An adding module, used for adding the model building parameters corresponding to the GOP to the first frame data of the first sub-GOP in the GOP; A determination module, used for determining a first verification model corresponding to each of the sub-GOPs, and adding each of the first verification models to the first frame data of the corresponding sub-GOP to obtain each of the updated sub-GOPs; A sending module is used to send the updated sub-GOPs to a receiving device in sequence, instructing the receiving device to build a second verification model for each sub-GOP based on the model construction parameters in the first frame data of the first sub-GOP and the first verification model, and to perform image frame verification on each sub-GOP based on the second verification model and the first verification model corresponding to each sub-GOP.

9. A transmitting end device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the image frame verification method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image frame verification method according to any one of claims 1 to 7 is implemented.

Images