Video auditing method and electronic equipment

Abstract

The invention provides a video auditing method and electronic equipment, and the method comprises the steps: generating a corresponding matrix for each to-be-audited video frame of a to-be-audited video based on the gray value of each pixel point in the to-be-audited video frame, dividing the matrix into a plurality of matrix blocks with the same size, and carrying out the auditing of the to-be-audited video frame based on the information entropy and weight matrix corresponding to each matrix block, the total information entropy of the to-be-reviewed video frame is calculated, the information entropy is obtained through calculation based on numerical values in the matrix blocks, and the weight is obtained through calculation based on the number of the matrix blocks in the to-be-reviewed video frame and the positions of the matrix blocks; for any two adjacent to-be-reviewed video frames, on the basis of the total information entropy of the two to-be-reviewed video frames, deleting any one to-be-reviewed video frame under the condition that the two to-be-reviewed video frames are determined to be similar frames; based on the deleted to-be-audited video frame, auditing the to-be-audited video; by comparing the total information entropies of the to-be-audited video frames, the similar frames are accurately identified and deleted, the number of the to-be-audited video frames is reduced, the analysis and auditing time is shortened, and the auditing efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the multimedia technical field, and particularly relates to a video auditing method and an electronic device. BACKGROUND

[0002] With the rapid development and wide application of multimedia technology, the video volume on the network continues to grow rapidly. In order to effectively filter the illegal information involved, efficient and accurate auditing of video content has become a key link in platform operation. The existing common auditing method usually adopts an image recognition algorithm, decodes the video into continuous frame images, and introduces a repeated frame detection and deduplication mechanism to identify and determine the content of each frame.

[0003] However, as a pre-step, image decoding needs to occupy a large amount of computing resources and takes a long time, resulting in low video auditing efficiency. SUMMARY

[0004] The present application provides a video auditing method and an electronic device to solve the problem of low video auditing efficiency in the prior art.

[0005] In a first aspect, the present application provides a video auditing method, comprising:

[0006] extracting a plurality of to-be-audited video frames from a to-be-audited video;

[0007] For each to-be-audited video frame, a corresponding matrix is generated based on the gray value of each pixel point in the to-be-audited video frame, the matrix is divided into a plurality of matrix blocks of the same size, and the total information entropy of the to-be-audited video frame is calculated based on the information entropy of each matrix block and the weight of each matrix block, wherein the information entropy of the matrix block is calculated based on the values in the matrix block, and the weight of the matrix block is calculated based on the number of matrix blocks in the to-be-audited video frame and the position of the matrix block;

[0008] For any two adjacent to-be-audited video frames, based on the total information entropy of the two to-be-audited video frames, in the case of determining that the two to-be-audited video frames are similar frames, one to-be-audited video frame in the two to-be-audited video frames is deleted;

[0009] Based on the to-be-audited video frames after deletion, the to-be-audited video is audited.

[0010] In a possible implementation manner, the information entropy of the matrix block is calculated in the following manner:

[0011] The values in the matrix block are normalized to obtain a first matrix block;

[0012] Each value in the first matrix block is discretized to obtain a second matrix block;

[0013] determining, for each value in the second matrix block, a ratio of a number of the value to a total number of values in the second matrix block;

[0014] calculating, based on the determined ratio, an information entropy of the matrix block.

[0015] In a possible implementation, the normalization processing of the values in the matrix block to obtain the first matrix block comprises:

[0016] calculating an average of all values in the matrix block;

[0017] calculating a standard deviation of the matrix block based on all values in the matrix block and the average;

[0018] calculating a first difference value of each value in the matrix block and the average, and calculating a first sum value of the standard deviation and a preset constant;

[0019] calculating a first quotient value of the first difference value and the first sum value, and taking a matrix composed of the first quotient value corresponding to each value in the matrix block as the first matrix block.

[0020] In a possible implementation, based on the determined ratio, the information entropy of the matrix block is calculated by the following formula:

[0021]

[0022] wherein E ij is the information entropy of the matrix block, p k is the ratio of the number of the value k in the second matrix block to the total number, and δ is a preset smoothing constant.

[0023] In a possible implementation, the weight of the matrix block is calculated in the following manner:

[0024] calculating a spatial weight of the matrix block based on a number of matrix blocks in the video frame under review and a position of the matrix block;

[0025] calculating a second sum value of spatial weights of all matrix blocks in the video frame under review;

[0026] calculating a second quotient value of the spatial weight of the matrix block and the second sum value, and taking the second quotient value as the weight of the matrix block.

[0027] In a possible implementation, the number of matrix blocks in the matrix corresponding to the video frame under review is N×M, where N and M are both positive integers.

[0028] The spatial weight corresponding to the matrix block is calculated by the following formula:

[0029]

[0030] wherein W ij is the spatial weight corresponding to the matrix block, N is the number of rows of matrix blocks in the matrix, M is the number of columns of matrix blocks in the matrix, i is the row position of the matrix block in the matrix, and j is the column position of the matrix block in the matrix.

[0031] In a possible implementation, the calculating the total information entropy of the to-be-reviewed video frame based on the information entropy of each matrix block and the weight corresponding to each matrix block comprises:

[0032] calculating, for each matrix block in the to-be-reviewed video frame, the product of the information entropy of the matrix block and the weight of the matrix block;

[0033] taking the third sum of all the calculated products as the total information entropy of the to-be-reviewed video frame.

[0034] In a possible implementation, the deleting one of the two to-be-reviewed video frames in a case where the two to-be-reviewed video frames are determined to be similar frames based on the total information entropy of the two to-be-reviewed video frames comprises:

[0035] calculating the difference of the total information entropy of the two to-be-reviewed video frames;

[0036] if the difference is less than or equal to a first preset threshold, determining that the two to-be-reviewed video frames are similar frames; if the difference is greater than the first preset threshold and less than or equal to a second preset threshold, performing sampling verification, and determining that the two to-be-reviewed video frames are similar frames when the sampling verification result meets a preset condition;

[0037] deleting one of the two to-be-reviewed video frames.

[0038] In a possible implementation, the performing sampling verification comprises:

[0039] selecting, from the matrices of the two to-be-reviewed video frames, multiple groups of values at the same positions;

[0040] calculating a second difference of the values at the same positions in each group;

[0041] determining the number of second differences that are less than or equal to a preset proportion threshold, and calculating the proportion of the number in the total number of the calculated second differences;

[0042] the determining that the two to-be-reviewed video frames are similar frames when the sampling verification result meets a preset condition comprises:

[0043] determining that the ratio is greater than a preset ratio, determining that the two to-be-audited video frames are similar frames.

[0044] In a second aspect, the present application provides an electronic device, comprising:

[0045] a memory for storing program instructions;

[0046] a processor for calling the program instructions stored in the memory and performing the steps included in the method of any one of the first aspect according to the obtained program instructions.

[0047] The beneficial effects of the present application are as follows:

[0048] The video auditing method provided by the present application, for each to-be-audited video frame of a to-be-audited video, generates a corresponding matrix based on the gray value of each pixel point in the to-be-audited video frame and divides the matrix into a plurality of matrix blocks of the same size, calculates the total information entropy of the to-be-audited video frame based on the information entropy corresponding to each matrix block and the weight matrix, wherein the information entropy is calculated based on the values in the matrix block, and the weight is calculated based on the number of matrix blocks in the to-be-audited video frame and the position of the matrix block; for any two adjacent to-be-audited video frames, based on the total information entropy of the two to-be-audited video frames, in the case of determining that the two to-be-audited video frames are similar frames, deleting any one to-be-audited video frame; based on the to-be-audited video frame after deletion, auditing the to-be-audited video; the present application generates a corresponding matrix based on the gray value of each pixel point in the to-be-audited video frame and divides the matrix into a plurality of matrix blocks of the same size, skips the decoding step, compares the total information entropy of the to-be-audited video frame, accurately identifies and deletes similar frames, simplifies the number of to-be-audited video pictures, avoids decoding overhead at the same time, reduces analysis and auditing time, and improves auditing efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0050] Figure 1 A flowchart of a video auditing method provided by an embodiment of the present application;

[0051] Figure 2 A matrix diagram of a to-be-audited video frame provided by an embodiment of the present application;

[0052] Figure 3 A flowchart of calculating the information entropy of a matrix block provided by an embodiment of the present application;

[0053] Figure 4A structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0055] It should be noted that the terms “first”, “second”, and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present application. Rather, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0056] In current video review, the existing technology is to extract part of the video frames, decode the video frame image, extract features such as color histogram and direction gradient histogram (Histogram of Oriented Gradients, HOG) or hash code (such as perceptual hash), and perform image recognition and comparison and video review. This process consumes considerable system resources, and the review time is also relatively long, and a considerable number of similar frames of the same scene will be repeatedly reviewed, further wasting resources and time.

[0057] In order to solve the above problems, the present application provides a video review method and an electronic device. In order to facilitate understanding, the video review method and the electronic device provided by the embodiments of the present application will be described in detail below with reference to the drawings.

[0058] As shown in FIG. 1, a flowchart of a video review method provided by an embodiment of the present application is shown, and the specific process is as follows: Figure 1 As shown in FIG. 1, a flowchart of a video review method provided by an embodiment of the present application is shown, and the specific process is as follows:

[0059] S101: Extracting a plurality of to-be-reviewed video frames from a to-be-reviewed video;

[0060] S102: For each to-be-audited video frame, a corresponding matrix is generated based on the gray value of each pixel point in the to-be-audited video frame, the matrix is divided into a plurality of matrix blocks of the same size, and the total information entropy of the to-be-audited video frame is calculated based on the information entropy of each matrix block and the weight of each matrix block, wherein the information entropy of the matrix block is calculated based on the numerical value in the matrix block, and the weight matrix of the matrix block is calculated based on the number of matrix blocks in the to-be-audited video frame and the position of the matrix block.

[0061] S103: For any two adjacent to-be-audited video frames, based on the total information entropy of the two to-be-audited video frames, in the case of determining that the two to-be-audited video frames are similar frames, one of the two to-be-audited video frames is deleted.

[0062] S104: Based on the to-be-audited video frame after deletion, the to-be-audited video is audited.

[0063] The video auditing method provided in the application is used for each to-be-audited video frame of the to-be-audited video, a corresponding matrix is generated based on the gray value of each pixel point in the to-be-audited video frame, the matrix is divided into a plurality of matrix blocks of the same size, the total information entropy of the to-be-audited video frame is calculated based on the information entropy of each matrix block and the weight matrix, wherein the information entropy is calculated based on the numerical value in the matrix block, and the weight is calculated based on the number of matrix blocks in the to-be-audited video frame and the position of the matrix block; for any two adjacent to-be-audited video frames, based on the total information entropy of the two to-be-audited video frames, in the case of determining that the two to-be-audited video frames are similar frames, one of the two to-be-audited video frames is deleted; based on the to-be-audited video frame after deletion, the to-be-audited video is audited; the application generates a corresponding matrix based on the gray value of each pixel point in the to-be-audited video frame and divides it into a plurality of matrix blocks of the same size, skips the decoding step, compares the total information entropy of the to-be-audited video frame, accurately identifies and deletes similar frames, simplifies the number of to-be-audited video frames, avoids decoding overhead, reduces analysis and auditing time, and improves auditing efficiency.

[0064] In the embodiment of the application, a plurality of to-be-audited video frames are extracted from the to-be-audited video, for each to-be-audited video frame, it is regarded as a two-dimensional matrix with a height of H and a width of W, each numerical value in the matrix represents the gray value of each pixel point in the to-be-audited video frame, and each numerical value exists in the form of bytes, for example, the byte is an 8-bit binary number, and the value range of the numerical value is 0-255 (from 00000000 to 11111111);

[0065] The matrix is divided into a plurality of NXM matrix blocks of the same size, wherein N represents that the matrix is divided into N rows, M represents that the matrix is divided into M columns, each matrix block is represented by B ij , i is the row position of the matrix block in the matrix, j is the column position of the matrix block in the matrix, the height of the matrix block is h (that is, The width of the matrix block is w (i.e., ), where 1≦i≦N, 1≦j≦M.

[0066] It should be noted that, in the following detailed description of the embodiments, decimal values ​​will be used to represent the values ​​in the matrix for ease of intuitive understanding and numerical demonstration. This representation is only for the purpose of simplifying the explanation and is not intended to limit the technical solution of this patent. Those skilled in the art should understand that binary values ​​are processed in practical applications.

[0067] For example, such as Figure 2 The diagram shown is a matrix representation of a video frame to be examined, provided in an embodiment of this application. The matrix has a height of H and a width of W. The matrix is ​​divided into multiple 4×4 matrix blocks of the same size. The matrix block corresponding to the first row and first column is matrix block B. 11 The matrix block corresponding to the first row and second column in the matrix is ​​matrix block B. 12 The matrix block corresponding to the 1st row and 3rd column is matrix block B. 13 The matrix block corresponding to the 1st row and 4th column is matrix block B. 14 The matrix block corresponding to the second row and first column in the matrix is ​​matrix block B. 21 The matrix block corresponding to the second row and second column in the matrix is ​​matrix block B. 22 The matrix block corresponding to the 2nd row and 3rd column in the matrix is ​​matrix block B. 23 The matrix block corresponding to the 2nd row and 4th column in the matrix is ​​matrix block B. 24 The matrix block corresponding to the 3rd row and 1st column in the matrix is ​​matrix block B. 31 The matrix block corresponding to the 3rd row and 2nd column is matrix block B. 32 The matrix block corresponding to the 3rd row and 3rd column in the matrix is ​​matrix block B. 33 The matrix block corresponding to the 3rd row and 4th column is matrix block B. 34 The matrix block corresponding to the 4th row and 1st column is matrix block B. 41 The matrix block corresponding to the 4th row and 2nd column is matrix block B. 42 The matrix block corresponding to the 4th row and 3rd column is matrix block B. 43 The matrix block corresponding to the 4th row and 4th column in the matrix is ​​matrix block B. 44 The height corresponding to all the above matrix blocks is 1. The width of all matrix blocks is [missing information].

[0068] In one embodiment, such as Figure 3 The diagram shown is a flowchart illustrating a method for calculating the information entropy of a matrix block according to an embodiment of this application, including:

[0069] S301: normalizing the numerical values in the matrix block, and taking the processed matrix block as a first matrix block;

[0070] First, the average of all numerical values in the matrix block is calculated, and second, the standard deviation of the matrix block is calculated based on all numerical values in the matrix block and the average. Specifically, the average of all numerical values in the matrix block is calculated by the following formula:

[0071]

[0072] wherein, u ij is the average of the data corresponding to the matrix block in the ith row and jth column, and p and q are the row index and column index of traversing all numerical values in the matrix block, respectively.

[0073] The standard deviation of the matrix block is calculated by the following formula:

[0074]

[0075] wherein, σ ij represents the standard deviation of the data corresponding to the matrix block in the ith row and jth column.

[0076] Then, the first difference value of each numerical value in the matrix block and the average is calculated, and the first sum value of the standard deviation and a preset constant is calculated;

[0077] Finally, the first quotient value of the first difference value and the first sum value is calculated, and the matrix composed of the obtained first quotient value corresponding to each numerical value in the matrix block is taken as the first matrix block. The specific formula is as follows:

[0078]

[0079] wherein, ε is a zero prevention constant, and in the embodiment of the present application, ε = 10 -8 .

[0080] For example, the matrix is which is divided into 2x2 matrix blocks (i.e., N is 2 and M is 2) of the same size, and the matrix block B 11 is The matrix block B 12 is The matrix block B 21 is The matrix block B 22 is

[0081] Taking the matrix block B 11 as an example, the corresponding average u 11 is (100+120+115+125) / 4 = 115, and the standard deviation is The corresponding formula 3 of the matrix block B 11 is Computing the matrix block B 11 The first quotient value corresponding to each numerical value in the first matrix block For

[0082] S302: Discretize each numerical value in the first matrix block, and take the processed first matrix block as a second matrix block;

[0083] In this application, in order to convert the numerical value in the matrix block into a form suitable for subsequent calculation, it is necessary to discretize each numerical value in the first matrix block, and round each numerical value in the first matrix block to the nearest integer, so as to map the continuous floating point numerical value to a discrete integer value, wherein the range of the discretized integer value is determined by the bit depth of the video frame under review (i.e. the binary number corresponding to the byte).

[0084] As in the above example, it is necessary to convert the numerical value of the first matrix block to a discrete integer value, i.e. to the integer value range of [0, 255], specifically, -1.60x64+128≈25.6≈26, 0.53x64+128≈162.9≈163, 0x64+128=128, 1.070x64+128≈197.5≈198, to obtain the second matrix block .

[0085] S303: For each numerical value in the second matrix block, determine the ratio pk of the number of numerical values to the total number of numerical values in the second matrix block, and the specific formula is as follows:

[0086]

[0087] Wherein, is an indicator function, representing whether the numerical value in the second matrix block is equal to the preset integer value.

[0088] As in the above example, the ratio of the number of numerical value 26 in the second matrix block to the total number of numerical values in the second matrix block is pk=1 / 4, the ratio of the number of numerical value 163 in the second matrix block to the total number of numerical values in the second matrix block is pk=1 / 4, the ratio of the number of numerical value 128 in the second matrix block to the total number of numerical values in the second matrix block is pk=1 / 4, and the ratio of the number of numerical value 198 in the second matrix block to the total number of numerical values in the second matrix block The ratio of the total number of values in the matrix block B

[0089] S304: Calculate the information entropy of the matrix block based on the determined ratio.

[0090] The information entropy of the matrix block is calculated by the following formula:

[0091]

[0092] Wherein, E ij is the information entropy of the matrix block, p k is the ratio of the number of values k in the second matrix block to the total number, and δ is a preset smoothing constant, δ = 10 -10 in the embodiment of the present application.

[0093] As the above example, the information entropy of the matrix block B 11 is calculated as follows: E 11 = - [0.25 x log2(0.25 + 10 -10 ) + 0.25 x log2(0.25 + 10 -10 ) + 0.25 x log2(0.25 + 10 -10 ) + 0.25 x log2(0.25 + 10 -10 )] = 2, that is, the information entropy E 11 of the matrix block B 11 is 2.

[0094] It should be noted that the steps of calculating the information entropy of the matrix block B 12 , the matrix block B 21 and the matrix block B 22 are the same as the steps of calculating the information entropy of the matrix block B 11 , which will not be repeated here.

[0095] In one embodiment, the weight of the matrix block is calculated in the following manner:

[0096] Based on the number of matrix blocks in the video frame under review and the position of the matrix block, the spatial weight corresponding to the matrix block is calculated.

[0097] Calculate the second sum value of the spatial weights corresponding to all matrix blocks in the video frame under review.

[0098] Calculate the second quotient value of the spatial weight corresponding to the matrix block and the second sum value, and take the second quotient value as the weight of the matrix block.

[0099] Specifically, the spatial weight corresponding to the matrix block is calculated by the following formula, wherein W ij is the spatial weight corresponding to the matrix block B ij .

[0100]

[0101] The weight of the matrix block is calculated by the following formula:

[0102]

[0103] wherein, is the matrix block B ij The corresponding weight, the denominator in formula 5 represents the second sum value.

[0104] As the above example, by calculating the information entropy E 11 of the matrix block B 11 is 2, the information entropy E 12 of the matrix block B 12 is 1.8, the information entropy E 21 of the matrix block B 21 is 2.2, the information entropy E 22 of the matrix block B 22 is 1.9;

[0105] The corresponding spatial weight W 11 of the matrix block B 12 The corresponding spatial weight W 21 of the matrix block B 221 The corresponding spatial weight W 11 of the matrix block B 11 The corresponding spatial weight W 12 of the matrix block B 12 The corresponding spatial weight W 21 of the matrix block B 21 The corresponding spatial weight W 221 of the matrix block B 22 The second sum value is 1+0.7071+0.7071+0.5774=2.9916;

[0107] The corresponding second quotient value of the matrix block B 11 is 1 / 2.9916≈0.3342, that is, the weight W 11 of the matrix block B 12 is 0.3342; the corresponding second quotient value of the matrix block B 12 is 0.7071 / 2.9916≈0.2364, that is, the weight W 21 of the matrix block B 21 is 0.2364; the corresponding second quotient value of the matrix block B 22 is 0.7071 / 2.9916≈0.2364, that is, the weight W 22 of the matrix block B ij is 0.2364; the corresponding second quotient value of the matrix block B 11 is 0.5774 / 2.9916≈0.1935, that is, the weight W 11 of the matrix block B 11 is 0.1935.21 The corresponding second quotient value is 0.7071 / 2.9916≈0.2364, i.e. the weight of the matrix block B 21 is 0.2364; the weight of the matrix block B 22 The corresponding second quotient value is 0.5774 / 2.9916≈0.193, i.e. the weight of the matrix block B 22 is 0.193.

[0108] In an embodiment, the total information entropy of the video frame under review is calculated based on the information entropy of each matrix block and the weight corresponding to each matrix block, comprising:

[0109] For each matrix block in the video frame under review, the information entropy E ij of the matrix block is calculated, and the weight of the matrix block is multiplied by the information entropy of the matrix block.

[0110] The third sum of all the products is taken as the total information entropy S of the video frame under review.

[0111] Specifically, the total information entropy of the video frame under review is calculated by the following formula:

[0112]

[0113] As in the above example, the information entropy E 11 of the matrix block B 11 is 2, the weight of the matrix block B 12 is 0.3342, the information entropy E 12 of the matrix block B 12 is 1.8, the weight of the matrix block B 21 is 0.2364, the information entropy E 21 of the matrix block B 21 is 2.2, the weight of the matrix block B 22 is 0.2364, the information entropy E 22 of the matrix block B 22 is 1.9, and the weight of the matrix block B is 0.193, then the total information entropy S of the video frame under review is 1.9807.

[0114] In an embodiment, based on the total information entropy of two video frames under review, in the case of determining that the two video frames under review are similar frames, one of the two video frames under review is deleted, comprising: ​​

[0115] calculating a difference of total information entropy of the two to-be-reviewed video frames;

[0116] if the difference is less than or equal to a first preset threshold, determining that the two to-be-reviewed video frames are similar frames;

[0117] if the difference is greater than the first preset threshold and less than or equal to a second preset threshold, performing a sampling verification, and determining that the two to-be-reviewed video frames are similar frames when a sampling verification result meets a preset condition;

[0118] if the difference is greater than the second preset threshold, determining that the two to-be-reviewed video frames are not similar frames;

[0119] in a case where the two to-be-reviewed video frames are determined to be similar frames, deleting one of the two to-be-reviewed video frames.

[0120] The sampling verification includes selecting a plurality of groups of values at the same positions from the matrices of the two to-be-reviewed video frames, calculating a second difference of the values at the same positions in each group, determining a number of second differences less than or equal to a preset proportion threshold, calculating a proportion of the number in a total number of calculated second differences, and determining that the two to-be-reviewed video frames are similar frames when the proportion is greater than a preset proportion.

[0121] For example, the first threshold a is 0.05, the second threshold β is 0.15, the total entropy value S1 of the first to-be-reviewed video frame is 2.1, the total entropy value S2 of the second to-be-reviewed video frame is 2.04, and the difference ΔS of the total information entropy of the two to-be-reviewed video frames is 0.06. Therefore, a(0.05) < ΔS(0.06) ≤ β(0.15), so the byte difference verification is started.

[0122] Suppose the matrix of the first to-be-reviewed video frame is the matrix of the second to-be-reviewed video frame is

[0123] The randomly selected positions are (1, 2), (3, 1), and (4, 2), i.e., the first row and the second column, the third row and the first column, and the fourth row and the second column. The values corresponding to the first position are 120 and 118, and the corresponding second difference is 2. The values corresponding to the second position are 108 and 105, and the corresponding second difference is 3. The values corresponding to the third position are 117 and 124, and the corresponding second difference is 7.

[0124] If the preset proportion threshold is 5.5, the number of the second difference values less than or equal to the preset proportion threshold is 2, the proportion of the number in the total number of the calculated second difference values is 2 / 3≈0.67, the preset proportion is 0.5, it is determined that the proportion is greater than the preset proportion, it is determined that the numerical difference between the matrix of the first to-be-audited video frame and the matrix of the second to-be-audited video frame is small, it is determined that the first to-be-audited video frame and the second to-be-audited video frame are similar frames, and then the second to-be-audited video frame in the two to-be-audited video frames can be deleted, and the first to-be-audited video frame is retained.

[0125] Based on the same inventive concept, the embodiment of the present application also provides an electronic device, the implementation principle of the electronic device is similar to that of the foregoing video auditing method, and the specific implementation manner of the electronic device can be referred to the foregoing video auditing method embodiment, and the repeated parts will not be described herein.

[0126] As shown in Figure 4 , it is a structural schematic diagram of an electronic device provided by the embodiment of the present application, which comprises:

[0127] The memory 41 is configured to store program instructions.

[0128] The processor 42 is configured to invoke the program instructions stored in the memory, and execute the steps included in the foregoing video auditing method according to the obtained program instructions.

[0129] The video auditing method provided by the present application is used for each to-be-audited video frame of a to-be-audited video, generates a corresponding matrix based on the gray value of each pixel point in the to-be-audited video frame, divides the matrix into a plurality of matrix blocks of the same size, calculates the total information entropy of the to-be-audited video frame based on the information entropy and the weight matrix corresponding to each matrix block, wherein the information entropy is calculated based on the values in the matrix block, and the weight is calculated based on the number and position of the matrix blocks in the to-be-audited video frame; for any two adjacent to-be-audited video frames, based on the total information entropy of the two to-be-audited video frames, in the case of determining that the two to-be-audited video frames are similar frames, deleting any one to-be-audited video frame; based on the to-be-audited video frame after deletion, auditing the to-be-audited video; the present application generates a corresponding matrix based on the gray value of each pixel point in the to-be-audited video frame, divides the matrix into a plurality of matrix blocks of the same size, skips the decoding step, compares the total information entropy of the to-be-audited video frame, accurately identifies and deletes similar frames, simplifies the number of to-be-audited video pictures, avoids decoding overhead, reduces analysis and auditing time, and improves auditing efficiency.

[0130] Those skilled in the art will appreciate that embodiments of the present application can be readily used as software, hardware, or a combination of software and hardware. In one

[0131] The present application is described in reference to the flow diagrams and / or block diagrams of the methods, apparatus (systems) and computer program products according to this application. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks.

[0132] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks.

[0134] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the apparatus and methods disclosed herein, equivalents and substitutions thereof could be made by one of ordinary skill in the art without departing from the spirit and scope of the application. Any and all such modifications and variations are intended to be included herein within the scope of the present application and the present expressive equivalents thereof.

Claims

1. A method of video auditing, the method comprising: The method comprises the following steps: extracting a plurality of to-be-reviewed video frames from a to-be-reviewed video; for each to-be-reviewed video frame, generating a corresponding matrix based on the gray value of each pixel point in the to-be-reviewed video frame, and dividing the matrix into a plurality of matrix blocks of the same size; based on the information entropy of each matrix block and the weight of each matrix block, calculating the total information entropy of the to-be-reviewed video frame, wherein the information entropy of the matrix block is calculated based on the values in the matrix block, and the weight of the matrix block is calculated based on the number of matrix blocks in the to-be-reviewed video frame and the position of the matrix block; for any two adjacent to-be-reviewed video frames, based on the total information entropy of the two to-be-reviewed video frames, in the case of determining that the two to-be-reviewed video frames are similar frames, deleting one to-be-reviewed video frame in the two to-be-reviewed video frames; based on the to-be-reviewed video frame after deletion, reviewing the to-be-reviewed video.

2. The method of claim 1, wherein, The information entropy of the matrix block is calculated in the following manner: normalizing the values in the matrix block to obtain a first matrix block; discretizing each value in the first matrix block to obtain a second matrix block; for each value in the second matrix block, determining the ratio of the number of the value to the total number of values in the second matrix block; based on the determined ratio, calculating the information entropy of the matrix block.

3. The method of claim 2, wherein, The normalization of the values in the matrix block to obtain the first matrix block comprises: calculating the average value of all values in the matrix block; based on all values in the matrix block and the average value, calculating the standard deviation of the matrix block; calculating the first difference value of each value in the matrix block and the average value, and calculating the first sum value of the standard deviation and a preset constant; calculating the first quotient value of the first difference value and the first sum value, and taking the matrix composed of the first quotient value corresponding to each value in the matrix block as the first matrix block.

4. The method of claim 2, wherein, Based on the determined ratio, the information entropy of the matrix block is calculated by the following formula: wherein E ij is the information entropy of the matrix block, p k is the ratio of the number of values k in the second matrix block to the total number, and δ is a predetermined smoothing constant.

5. The method of claim 1, wherein, The weight of the matrix block is calculated in the following manner: based on the number of matrix blocks in the to-be-reviewed video frame and the position of the matrix block, calculating the spatial weight of the matrix block; calculating the second sum value of the spatial weights of all matrix blocks in the to-be-reviewed video frame; calculating the second quotient value of the spatial weight of the matrix block and the second sum value, and taking the second quotient value as the weight of the matrix block.

6. The method of claim 5, wherein, The number of matrix blocks in the matrix corresponding to the to-be-reviewed video frame is N×M, wherein N and M are both positive integers; the spatial weight corresponding to the matrix block is calculated by the following formula: where W ij is the spatial weight corresponding to the matrix block, N is the number of rows of matrix blocks in the matrix, M is the number of columns of matrix blocks in the matrix, i is the row position of the matrix block in the matrix, and j is the column position of the matrix block in the matrix.

7. The method of claim 1, wherein, based on the information entropy of each matrix block and the weight corresponding to each matrix block, the total information entropy of the to-be-reviewed video frame is calculated by the following formula: for each matrix block in the to-be-reviewed video frame, calculating the product of the information entropy of the matrix block and the weight of the matrix block; taking the third sum value of all the calculated products as the total information entropy of the to-be-reviewed video frame.

8. The method according to any one of claims 1 to 7, characterized in that, based on the total information entropy of the two to-be-reviewed video frames, in the case of determining that the two to-be-reviewed video frames are similar frames, deleting one to-be-reviewed video frame in the two to-be-reviewed video frames, comprises: calculating a difference value of total information entropy of the two to-be-judged video frames; if the difference value is less than or equal to a first preset threshold, determining that the two to-be-judged video frames are similar frames; if the difference value is greater than the first preset threshold and less than or equal to a second preset threshold, performing sampling verification, and when a sampling verification result meets a preset condition, determining that the two to-be-judged video frames are similar frames; deleting one of the two to-be-judged video frames.

9. The method of claim 8, wherein, The sampling verification includes: selecting multiple groups of values at the same positions from matrices of the two to-be-judged video frames; calculating a second difference value of each group of values at the same positions; determining a number of second difference values less than or equal to a preset proportion threshold, and calculating a proportion of the number in a total number of calculated second difference values; when the sampling verification result meets the preset condition, determining that the two to-be-judged video frames are similar frames, includes: when the proportion is greater than a preset proportion, determining that the two to-be-judged video frames are similar frames.

10. An electronic device, comprising: include: a memory for storing program instructions; a processor for calling the program instructions stored in the memory and performing the steps included in the method of any one of claims 1-9 according to the obtained program instructions.

Images