A target detection method, device, computing device and storage medium

Abstract

The present application provides a target detection method, apparatus, computing device and storage medium, including: obtaining N discrete cosine transform DCT matrices of the first size corresponding to each video frame to be detected in the video to be analyzed; wherein each DCT matrix of the first size corresponds to a slice of the video frame; determining the key frame in the video to be analyzed according to the N DCT matrices of the first size corresponding to each video frame to be detected in the video to be analyzed; based on the kernel matrix of the DCT transform, converting the N DCT matrices of the first size corresponding to the key frame into a DCT matrix of the second size; the DCT matrix of the second size contains information of all slices corresponding to the key frame; and performing target detection on the key frame according to the DCT matrix of the second size. This scheme can realize fast and effective target detection.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a target detection method, device, computing equipment and storage medium. Background Art

[0002] With the popularization of digital devices and the advancement of high-definition camera technology, the amount of video data generated has exploded, bringing huge storage and management challenges.

[0003] Target detection is a key step in video analysis, which aims to extract specific targets (such as people, objects or actions) from massive video data and improve the utilization efficiency of video data.

[0004] The traditional frame-by-frame detection method is difficult to meet the demand due to its large amount of calculation and slow speed; there is also a method in the prior art that first extracts key frames and then performs target detection on the key frames; however, the prior art often determines the key frames by comparing the histogram distribution probability and the joint histogram distribution probability on each color component, which has poor robustness.

[0005] On the other hand, when performing target detection on video frames in the H.264 video encoding format, it is often necessary to convert the DCT matrix into raw pixel data; this method has large computational complexity and low efficiency, and is difficult to handle scenes with large amounts of video data.

[0006] So, how to perform target detection quickly and effectively remains to be solved. Summary of the invention

[0007] The present application provides a target detection method, apparatus, computing device and storage medium, which can realize fast and effective target detection.

[0008] In the first aspect, an embodiment of the present application provides a target detection method, which can be executed by a target detection device, which can be a terminal device or a module for a terminal device, or a server or a module for a server. The present application does not limit the execution subject of the method. The method includes: obtaining N discrete cosine transform DCT matrices of a first size corresponding to each video frame to be detected in the video to be analyzed; wherein each DCT matrix of the first size corresponds to a slice of the video frame; according to the N first size DCT matrices corresponding to each video frame to be detected in the video to be analyzed, determining the key frame in the video to be analyzed; based on the kernel matrix of the DCT transform, converting the N first size DCT matrices corresponding to the key frame into a second size DCT matrix; the second size DCT matrix contains information of all slices corresponding to the key frame; according to the second size DCT matrix, performing target detection on the key frame.

[0009] The above scheme, on the one hand, determines the key frames in the video to be analyzed based on the DCT matrix, which is more robust and can improve the accuracy of determining the key frames; on the other hand, the kernel matrix based on the DCT transform can directly convert N first-size DCT matrices into a second-size DCT matrix. Therefore, when performing target detection on the key frames, there is no need to convert the DCT matrix into original pixel data, and the second-size DCT matrix can be directly used for target detection, thereby improving the efficiency of target detection.

[0010] In a possible implementation method, each video frame in the video to be analyzed is analyzed by a key frame extraction algorithm to obtain an initial key frame in the video to be analyzed; and N first-size DCT matrices corresponding to the initial key frame in the video to be analyzed are obtained.

[0011] The above scheme extracts the initial key frame first, which can reduce the number of subsequent key frames. This method can not only improve the accuracy of key frame determination, but also improve the efficiency of object detection in the video to be analyzed when the number of final key frames is reduced.

[0012] In a possible implementation method, for the i-th key frame in the video to be analyzed, the frame difference is determined according to the N first-size DCT matrices corresponding to the j-th frame in the video to be analyzed, the N first-size DCT matrices corresponding to the i-th key frame, and a weight matrix; the weight matrix is ​​used to increase the low-frequency information in the DCT matrix and reduce the high-frequency information in the DCT matrix; wherein the j-th frame is located after the i-th key frame in the video to be analyzed; i and j are positive integers; if the frame difference is greater than the difference threshold, the j-th frame is determined to be the i+1-th key frame.

[0013] The above solution can accurately and effectively determine the key frames in the video to be analyzed.

[0014] In a possible implementation method, the weights in any weight matrix decrease layer by layer according to the ladder layer, and the weights in the same ladder layer are the same; the ladder layer is a parallel oblique line formed with reference to the diagonal line from the upper right corner to the lower left corner of the matrix.

[0015] In the above scheme, the low-frequency information of the video frame is generally concentrated in the upper left corner of the DCT matrix, and the low-frequency information generally includes the key information of the video frame. The weights in the weight matrix are reduced layer by layer according to the ladder layer, which can increase the low-frequency information in the video frame. Further, it can increase the accuracy of key frame determination.

[0016] In a possible implementation method, for the odd rows of the first matrix and the second matrix, a first difference is determined based on the difference between the values ​​of the first matrix and the second matrix at corresponding positions and a weight matrix; the first matrix and the second matrix correspond to a first-size DCT matrix of the same slice in a video frame; for the even rows of the first matrix and the second matrix, a second difference is determined based on the difference between the values ​​of the first matrix and the second matrix at corresponding positions and the standard deviation of the first matrix and the second matrix; based on the first difference and the second difference, a first difference sum between the first matrix and the second matrix is ​​determined; and based on the first difference sum corresponding to N first-size DCT matrices, the inter-frame difference is determined.

[0017] The above solution can accurately and effectively determine the key frames in the video to be analyzed.

[0018] In a possible implementation method, according to the N first-size DCT matrices corresponding to the j-th frame in the video to be analyzed and the weight matrix corresponding to each first-size DCT matrix, the N third sub-matrices corresponding to the N first-size DCT matrices in the j-th frame are determined; the N third sub-matrices corresponding to the j-th frame are spliced ​​to obtain a third matrix; according to the N first-size DCT matrices corresponding to the i-th key frame in the video to be analyzed and the weight matrix corresponding to each first-size DCT matrix, the N fourth sub-matrices corresponding to the N first-size DCT matrices in the i-th key frame are determined; the N fourth sub-matrices corresponding to the i-th key frame are spliced ​​to obtain a fourth matrix; the dimension of the fourth matrix is ​​equal to the dimension of the third matrix; the frame difference is determined according to the difference between the third matrix and the fourth matrix.

[0019] The above solution can accurately and effectively determine the key frames in the video to be analyzed.

[0020] In a possible implementation method, a first kernel matrix and a second kernel matrix are determined; the dimension of the first kernel matrix is ​​the same as the dimension of the first-size DCT matrix; the dimension of the second kernel matrix is ​​the same as the dimension of the second-size DCT matrix; based on the first kernel matrix and the second kernel matrix, the N first-size DCT matrices corresponding to the key frame are converted into a second-size DCT matrix.

[0021] The above solution can accurately and effectively convert N DCT matrices of the first size corresponding to the key frame into a DCT matrix of the second size.

[0022] In a possible implementation method, the second kernel matrix is ​​divided according to the dimension of the first-size DCT matrix to obtain N sub-matrices; wherein each sub-matrix is ​​a subset of the second kernel matrix; a transformation matrix is ​​determined according to the N sub-matrices and the first inverse matrix; each value in the transformation matrix is ​​equal to the product of the corresponding positions of the N sub-matrices and the first inverse matrix; the first inverse matrix is ​​the inverse matrix corresponding to the first kernel matrix; the second-size DCT matrix is ​​determined according to the N first-size DCT matrices corresponding to the key frame and the transformation matrix.

[0023] The above solution can accurately and effectively convert N DCT matrices of the first size corresponding to the key frame into a DCT matrix of the second size.

[0024] In the second aspect, the embodiment of the present application provides a target detection device, including: an acquisition unit, a determination unit and a detection unit. The acquisition unit is used to acquire N discrete cosine transform DCT matrices of the first size corresponding to each video frame to be detected in the video to be analyzed; wherein each DCT matrix of the first size corresponds to a slice of the video frame; the determination unit is used to determine the key frame in the video to be analyzed according to the N first-size DCT matrices corresponding to each video frame to be detected in the video to be analyzed; based on the kernel matrix of the DCT transform, the N first-size DCT matrices corresponding to the key frame are converted into a second-size DCT matrix; the second-size DCT matrix contains information of all slices corresponding to the key frame; the detection unit is used to perform target detection on the key frame according to the second-size DCT matrix.

[0025] In a possible implementation method, the acquisition unit is used to analyze each video frame in the video to be analyzed by using a key frame extraction algorithm to obtain an initial key frame in the video to be analyzed; and obtain N first-size DCT matrices corresponding to the initial key frame in the video to be analyzed.

[0026] In a possible implementation method, the determination unit is used to determine the frame difference for the i-th key frame in the video to be analyzed based on N first-size DCT matrices corresponding to the j-th frame in the video to be analyzed, N first-size DCT matrices corresponding to the i-th key frame, and a weight matrix; the weight matrix is ​​used to increase the low-frequency information in the DCT matrix and reduce the high-frequency information in the DCT matrix; wherein the j-th frame is located after the i-th key frame in the video to be analyzed; i and j are positive integers; if the frame difference is greater than a difference threshold, the j-th frame is determined to be the i+1-th key frame.

[0027] In a possible implementation method, the weights in any weight matrix decrease layer by layer according to the ladder layer, and the weights in the same ladder layer are the same; the ladder layer is a parallel oblique line formed with reference to the diagonal line from the upper right corner to the lower left corner of the matrix.

[0028] In a possible implementation method, the determination unit is used to determine, for odd rows of a first matrix and a second matrix, a first difference according to a difference between the values ​​of the first matrix and the second matrix at corresponding positions and a weight matrix; the first matrix and the second matrix correspond to a first-size DCT matrix of the same slice in a video frame; for even rows of the first matrix and the second matrix, a second difference is determined according to a difference between the values ​​of the first matrix and the second matrix at corresponding positions and a standard deviation of the first matrix and the second matrix; a first difference sum between the first matrix and the second matrix is ​​determined according to the first difference and the second difference; and the inter-frame difference is determined according to the first difference sums corresponding to N first-size DCT matrices respectively.

[0029] In a possible implementation method, the determination unit is used to determine the N third sub-matrices corresponding to the N first-size DCT matrices in the j-th frame according to the N first-size DCT matrices corresponding to the j-th frame in the video to be analyzed and the weight matrix corresponding to each first-size DCT matrix; splice the N third sub-matrices corresponding to the j-th frame to obtain a third matrix; determine the N fourth sub-matrices corresponding to the N first-size DCT matrices in the i-th key frame according to the N first-size DCT matrices corresponding to the i-th key frame in the video to be analyzed and the weight matrix corresponding to each first-size DCT matrix; splice the N fourth sub-matrices corresponding to the i-th key frame to obtain a fourth matrix; the dimension of the fourth matrix is ​​equal to the dimension of the third matrix; and determine the frame difference according to the difference between the third matrix and the fourth matrix.

[0030] In a possible implementation method, the determination unit is used to determine a first kernel matrix and a second kernel matrix; the dimension of the first kernel matrix is ​​the same as the dimension of the first-size DCT matrix; the dimension of the second kernel matrix is ​​the same as the dimension of the second-size DCT matrix; based on the first kernel matrix and the second kernel matrix, the N first-size DCT matrices corresponding to the key frame are converted into a second-size DCT matrix.

[0031] In a possible implementation method, the determination unit is used to divide the second kernel matrix according to the dimension of the first-size DCT matrix to obtain N sub-matrices; wherein each sub-matrix is ​​a subset of the second kernel matrix; a transformation matrix is ​​determined according to the N sub-matrices and the first inverse matrix; each value in the transformation matrix is ​​equal to the product of the corresponding positions of the N sub-matrices and the first inverse matrix; the first inverse matrix is ​​the inverse matrix corresponding to the first kernel matrix; and the second-size DCT matrix is ​​determined according to the N first-size DCT matrices corresponding to the key frame and the transformation matrix.

[0032] In a third aspect, an embodiment of the present application further provides a computing device, including:

[0033] A memory for storing program instructions;

[0034] The processor is used to call the program instructions stored in the memory, and execute any method of implementing the above-mentioned first aspect according to the obtained program instructions.

[0035] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which computer-readable instructions are stored. When a computer reads and executes the computer-readable instructions, any method of the above-mentioned first aspect is implemented.

[0036] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising a computer program executable by a computer device, wherein when the program is run on the computer device, the computer device executes any method for implementing the above-mentioned first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of a target detection method provided in an embodiment of the present application;

[0038] Figure 2 A schematic diagram of a flow chart of a method for determining an inter-frame difference provided in an embodiment of the present application;

[0039] Figure 3 A schematic diagram of a flow chart of a method for determining an inter-frame difference provided in an embodiment of the present application;

[0040] Figure 4 A schematic diagram of a target detection method provided in an embodiment of the present application;

[0041] Figure 5 A schematic diagram of the structure of a target detection method provided in an embodiment of the present application;

[0042] Figure 6 A schematic diagram of the structure of a target detection device provided in an embodiment of the present application;

[0043] Figure 7 A schematic diagram of the structure of a target detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] Figure 1 A flowchart of a target detection method provided in an embodiment of the present application, the method can be executed by a target detection device, the target detection device can be a terminal device or a module for a terminal device, or a server or a module for a server. The present application does not limit the execution subject of the method.

[0045] The method comprises the following steps:

[0046] Step 101: Obtain N discrete cosine transform (DCT) matrices of a first size corresponding to each video frame to be detected in the video to be analyzed.

[0047] Each DCT matrix of the first size corresponds to a slice of the video frame.

[0048] In one possible implementation method, the video to be analyzed is a video in H.264 encoding format. H.264 is an efficient video compression standard that is widely used in streaming media and high-definition content storage. It uses I frames (Intra-frame) to perform key frame compression. These frames are independently encoded and do not depend on other frames, which helps to quickly jump and restore the video. At the same time, H.264 uses inter-frame compression technology to reduce redundant data between video frames, thereby improving compression efficiency. Its core also includes discrete cosine DCT transform, which converts the spatial information of the image into frequency domain information, making it easier to compress low-frequency data, further optimizing the encoding effect and reducing bandwidth requirements.

[0049] In one possible implementation method, discrete cosine transform (DCT) is a mathematical transform that converts a signal from the time domain or space domain to the frequency domain, and is often used in data compression and signal processing. The core idea is to represent an image or signal as a weighted sum of cosine functions of different frequencies, usually concentrating most of the energy on lower frequencies, which allows a small amount of high-frequency components to be retained and the amount of data to be reduced.

[0050] In a possible implementation method, each video frame to be detected includes N slices, and each slice corresponds to a DCT matrix of a first size. The present application does not limit the number of N, and does not limit the size of the first size. Exemplarily, if the first size is 4*4 and the video frame to be detected is 32*32, the video frame to be detected includes 64 slices, that is, 64 first-size DCT matrices corresponding to the video frame to be detected; if the first size is 8*8 and the video frame to be detected is 32*32, the video frame to be detected includes 16 slices, that is, 16 first-size DCT matrices corresponding to the video frame to be detected.

[0051] In a possible implementation method, N first-size DCT matrices corresponding to each frame in the video to be analyzed can be obtained, or N first-size DCT matrices corresponding to the initial key frame can be obtained after the initial key frame is extracted from the video to be analyzed. After obtaining the initial key frame, the key frame is further extracted through the following step 102 to reduce the number of key frames. This method can not only improve the accuracy of key frame determination, but also improve the efficiency of target detection in the video to be analyzed when the number of key frames is reduced.

[0052] In a possible implementation method, each video frame in the video to be analyzed is analyzed by a key frame extraction algorithm to obtain an initial key frame in the video to be analyzed. This application does not limit the key frame extraction algorithm.

[0053] Step 102: determining a key frame in the video to be analyzed according to N DCT matrices of a first size corresponding to each video frame to be detected in the video to be analyzed.

[0054] Step 103: based on the kernel matrix of DCT transformation, convert the N DCT matrices of the first size corresponding to the key frame into a DCT matrix of the second size.

[0055] The DCT matrix of the second size contains information of all slices corresponding to the key frame.

[0056] In a possible implementation method, the size of the key frame is equal to the second size; for example, the size of the key frame is 32*32, the first size is 4*4, and N is 64; then 64 4*4 DCT matrices are converted into a 32*32 DCT matrix; that is, the size of the second size is 32*32. Of course, the size of the second size DCT matrix may not be the same as the size of the key frame, and this application does not limit this.

[0057] Step 104: Perform object detection on the key frame according to the DCT matrix of the second size.

[0058] In a possible implementation method, target detection is performed on the key frame according to a detection classifier, wherein the detection classifier is a target detection model trained and processed by machine learning. This application does not limit the machine learning algorithm.

[0059] For example, a Gaussian radial basis function is used as a kernel function, and LibSVM is used to train the training samples. During the training, the penalty factor C is set to 100, and the kernel parameter is set to 0.008; after training, a target detection classifier is finally obtained.

[0060] In a possible implementation method, the input of the detection classifier is a DCT matrix of the second size, and the output is the classification and position information of the target object.

[0061] The above scheme, on the one hand, determines the key frames in the video to be analyzed based on the DCT matrix, which is more robust and can improve the accuracy of determining the key frames; on the other hand, the kernel matrix based on the DCT transform can directly convert N first-size DCT matrices into a second-size DCT matrix. Therefore, when performing target detection on the key frames, there is no need to convert the DCT matrix into original pixel data, and the second-size DCT matrix can be directly used for target detection, thereby improving the efficiency of target detection.

[0062] In one embodiment, the above step 102, determining the key frame in the video to be analyzed according to the N first-size DCT matrices corresponding to each video frame to be detected in the video to be analyzed, includes: for the i-th key frame in the video to be analyzed, determining the frame difference according to the N first-size DCT matrices corresponding to the j-th frame in the video to be analyzed, the N first-size DCT matrices corresponding to the i-th key frame, and the weight matrix; the weight matrix is ​​used to increase the low-frequency information in the DCT matrix and reduce the high-frequency information in the DCT matrix; wherein the j-th frame is located after the i-th key frame in the video to be analyzed; i and j are positive integers; if the frame difference is greater than the difference threshold, the j-th frame is determined to be the i+1-th key frame. This solution can accurately and effectively determine the key frame in the video to be analyzed.

[0063] This application does not limit the specific value of the difference threshold.

[0064] In a possible implementation method, the first frame in the video to be analyzed is selected as the key frame.

[0065] In a possible implementation method, the weights in any weight matrix are reduced layer by layer and the weights in the same layer are reduced from top to bottom.

[0066] For example, the weight matrix is ​​shown in the following formula (1):

[0067]

[0068] That is, the size relationship of each weight in the weight matrix is ​​as follows: 11 >w 12 >w 21 >w 13 >w 22 >w 31 >……>w 34 >w 43 >w 44 .

[0069] In a possible implementation method, the weights in any weight matrix decrease layer by layer according to the ladder layer, and the weights in the same ladder layer are the same; the ladder layer is a parallel oblique line formed with reference to the diagonal line from the upper right corner to the lower left corner of the matrix.

[0070] That is, when (i1+j1)<(i2+j2), the value corresponding to the i1th row and j1th column in the weight matrix is ​​greater than the value corresponding to the i2th row and j2th column in the weight matrix; when (i1+j1)=(i2+j2), the value corresponding to the i1th row and j1th column in the weight matrix is ​​equal to the value corresponding to the i2th row and j2th column in the weight matrix.

[0071] For example, the magnitude relationship of each weight in the weight matrix is ​​shown in the following formula (2):

[0072]

[0073] In the above scheme, the low-frequency information of the video frame is generally concentrated in the upper left corner of the DCT matrix, and the low-frequency information generally includes the key information of the video frame. The weights in the weight matrix are reduced layer by layer according to the ladder layer, which can increase the low-frequency information in the video frame. Further, it can increase the accuracy of key frame determination.

[0074] In a possible implementation method, the method for determining the frame difference is as follows: Figure 2 As shown, the following steps are included:

[0075] Step 201 : for odd rows of a first matrix and a second matrix, determine a first difference according to a difference between values ​​of the first matrix and the second matrix at corresponding positions and a weight matrix.

[0076] The first matrix and the second matrix correspond to a first-size DCT matrix of the same slice in the video frame.

[0077] Step 202: for the even rows of the first matrix and the second matrix, determine a second difference according to the difference between the values ​​of the first matrix and the second matrix at corresponding positions and the standard deviation of the first matrix and the second matrix.

[0078] Step 203: Determine a first difference sum between the first matrix and the second matrix according to the first difference and the second difference.

[0079] Step 204: determine the inter-frame difference according to the first difference values ​​and corresponding to the N first-size DCT matrices respectively.

[0080] Optionally, for the even rows of the first matrix and the second matrix, the first difference can be determined based on the difference between the values ​​of the first matrix and the second matrix at corresponding positions and the weight matrix; for the odd rows of the first matrix and the second matrix, the second difference can be determined based on the difference between the values ​​of the first matrix and the second matrix at corresponding positions and the standard deviation of the first matrix and the second matrix; this application does not limit this.

[0081] In a possible implementation method, the calculation method of the above steps 201 to 204 is shown in the following formula (3):

[0082]

[0083] Among them, FED represents the frame difference, w ij is the weight matrix, a ij is the value in the i-th row and j-th column of the first matrix; b ij is the value in the i-th row and j-th column of the second matrix; σ ij is the standard deviation; n is the size of the first dimension, for example, if the first dimension is 4*4, then n is 4.

[0084] The present application does not limit the method for obtaining the standard deviation of the first matrix and the second matrix. Optionally, the method for obtaining the standard deviation of the first matrix and the second matrix is ​​as shown in the following formula (4):

[0085]

[0086] Among them, u ij is the mean of the first matrix and the second matrix position (i, j), u ij The calculation method is shown in the following formula (5):

[0087]

[0088] Among them, σ ij is the standard deviation, K is the number of matrices. Here we need to calculate the standard deviation of the first matrix and the second matrix, so K is 2; Represents the value in the i-th row and j-th column of the K-th matrix.

[0089] Optionally, a method for obtaining the standard deviation of the first matrix and the second matrix is ​​shown in the following formula (6):

[0090]

[0091] Among them, u ij is the mean of the first matrix and the second matrix position (i, j), u ij The calculation method is shown in the following formula (7):

[0092]

[0093] Among them, a ij is the value in the i-th row and j-th column of the first matrix; b ij is the value in the i-th row and j-th column of the second matrix.

[0094] In a possible implementation method, the method for determining the frame difference is as follows: Figure 3 As shown, the following steps are included:

[0095] Step 301 : Determine N third sub-matrices corresponding to the N first-size DCT matrices in the j-th frame according to the N first-size DCT matrices corresponding to the j-th frame in the video to be analyzed and the weight matrix corresponding to each first-size DCT matrix.

[0096] In a possible implementation method, N first-size DCT matrices corresponding to the j-th frame are multiplied by weight matrices corresponding to each first-size DCT matrix to obtain N third sub-matrices corresponding to the N first-size DCT matrices corresponding to the j-th frame.

[0097] Step 302: splice the N third sub-matrices corresponding to the j-th frame to obtain a third matrix.

[0098] In a possible implementation method, the position of each third submatrix in the third matrix has the same position as each slice in the jth frame. For example, the third submatrix 1 corresponds to the first slice in the jth frame, and the third submatrix 2 corresponds to the second slice in the jth frame, then the third submatrix 2 is spliced ​​after the third submatrix 1.

[0099] Exemplarily, the size of each third sub-matrix is ​​4*4, N is 64, and the dimension of the third matrix is ​​32*32.

[0100] Step 303: Determine N fourth sub-matrices corresponding to the N first-size DCT matrices in the i-th key frame according to the N first-size DCT matrices corresponding to the i-th key frame in the video to be analyzed and the weight matrix corresponding to each first-size DCT matrix.

[0101] In a possible implementation method, the N first-size DCT matrices corresponding to the i-th key frame are multiplied by the weight matrix corresponding to each first-size DCT matrix to obtain N fourth sub-matrices corresponding to the N first-size DCT matrices corresponding to the i-th key frame.

[0102] Step 304 , concatenating the N fourth sub-matrices corresponding to the i-th key frame to obtain a fourth matrix.

[0103] The dimension of the fourth matrix is ​​equal to the dimension of the third matrix.

[0104] In a possible implementation method, the position of each fourth submatrix in the fourth matrix has the same position as that of each slice in the i-th key frame.

[0105] In a possible implementation method, the dimension of the fourth matrix is ​​equal to the dimension of the third matrix. For example, if the dimension of the third matrix is ​​32*32, the dimension of the fourth matrix is ​​also 32*32.

[0106] Step 305: determine the inter-frame difference according to the difference between the third matrix and the fourth matrix.

[0107] In a possible implementation method, the difference between the corresponding positions of the third matrix and the fourth matrix is ​​obtained, and the sum of all the differences is used as the inter-frame difference.

[0108] The above solution can accurately and effectively determine the key frames in the video to be analyzed.

[0109] In a possible implementation method, in the above steps 301 to 305, the N first-size DCT matrices corresponding to the j-th frame can be first concatenated, and then calculated with the weight matrix to obtain a third matrix; the N first-size DCT matrices corresponding to the i-th key frame can be first concatenated, and then calculated with the weight matrix to obtain a fourth matrix; the specific implementation of this method is similar to the above steps 301 to 305, and will not be repeated here.

[0110] In one embodiment, the above step 103, based on the kernel matrix of DCT transformation, converts N DCT matrices of the first size corresponding to the key frame into a DCT matrix of the second size, as shown in the specific implementation process. Figure 4 As shown, the following steps are included:

[0111] Step 401: determine a first kernel matrix and a second kernel matrix.

[0112] The dimension of the first kernel matrix is ​​the same as the dimension of the DCT matrix of the first size; the dimension of the second kernel matrix is ​​the same as the dimension of the DCT matrix of the second size.

[0113] In a possible implementation method, a first kernel matrix and a second kernel matrix are determined according to a DCT kernel transformation. The first kernel matrix and the second kernel matrix are orthogonal matrices respectively, and are used to transform input data.

[0114] In a possible implementation method, the calculation method of the first kernel matrix and the second kernel matrix is ​​as shown in the following formula (8):

[0115]

[0116] Among them, Cuv is a kernel matrix of dimension u*v, cos is the cosine function, u, v = 0, 1, 2, ..., N-1.

[0117] Wherein, α(u) is the normalization coefficient. The calculation method of α(u) is shown in the following formula (9):

[0118]

[0119] Step 402: Convert the N DCT matrices of the first size corresponding to the key frame into a DCT matrix of the second size according to the first kernel matrix and the second kernel matrix.

[0120] The above solution can accurately and effectively convert N DCT matrices of the first size corresponding to the key frame into a DCT matrix of the second size.

[0121] In a possible implementation method, the above-mentioned step 402, according to the first kernel matrix and the second kernel matrix, converts the N first-size DCT matrices corresponding to the key frame into a second-size DCT matrix, including: dividing the second kernel matrix according to the dimension of the first-size DCT matrix to obtain N sub-matrices; wherein each sub-matrix is ​​a subset of the second kernel matrix; determining a conversion matrix according to the N sub-matrices and the first inverse matrix; each value in the conversion matrix is ​​equal to the product of the corresponding position of the N sub-matrices and the first inverse matrix; the first inverse matrix is ​​the inverse matrix corresponding to the first kernel matrix; and determining the second-size DCT matrix according to the N first-size DCT matrices corresponding to the key frame and the conversion matrix.

[0122] In a possible implementation method, the method for determining the DCT matrix of the second size is shown in the following formula (10):

[0123]

[0124] Wherein, Y is the DCT matrix of the second size, S is the transformation matrix; S T is the inverse of the transformation matrix; F ij is the i*jth first-size DCT matrix, n*n=N.

[0125] The calculation method of the conversion matrix S is shown in the following formula (11):

[0126]

[0127] Wherein, S is the transformation matrix; C -1 is the inverse matrix corresponding to the first kernel matrix; A ij is the i*jth submatrix; n*n=N.

[0128] Wherein, the second kernel matrix is ​​shown in the following formula (12):

[0129]

[0130] In a possible implementation method, it is proved in the following way: the second-size DCT matrix determined by the present application through the above steps 401 to 402 is the same as the DCT matrix converted from the original pixel values.

[0131] like Figure 5 As shown, the original pixel matrix f can be converted into a large-sized DCT coefficient matrix; after the original pixel matrix f is divided into multiple blocks, each block can also be converted into a small-sized DCT coefficient matrix; generally, the video to be analyzed is composed of multiple small-sized DCT coefficient matrices; but in the target detection task, it is definitely necessary to perform target detection on the entire video frame, which requires multiple small-sized DCT coefficient matrices to be composed of a large-sized DCT coefficient matrix for target detection; the existing technology converts multiple small-sized DCT coefficient matrices into corresponding original pixel blocks respectively, and then merges them into the entire original pixel matrix, and then converts the original pixel matrix into a large-sized DCT coefficient matrix, and performs target detection based on the large-sized DCT coefficient matrix.

[0132] The present application does not need to convert multiple small-sized DCT coefficient matrices into corresponding original pixel blocks, but can directly convert multiple small-sized DCT coefficient matrices into large-sized DCT coefficient matrices that can be used for target detection based on the kernel matrix.

[0133] The second size DCT matrix determined in this application is the same as the DCT matrix converted from the original pixel value, and the proof process is as follows:

[0134] Assume f is an MxM pixel block, A is the kernel matrix of f for DCT transformation, f is divided into NxN pixel blocks (M is an integer multiple of N), and A is also divided into NxN data blocks, as shown in the following formulas (13) and (14):

[0135]

[0136] C is the first kernel matrix, size is n*n, F ij f ij The result after DCT transformation, assuming Y is the result after f is transformed by DCT, then Y can be expressed as shown in formula (15):

[0137]

[0138] The matrix form of DCT transformation is as follows:

[0139] F ij =Cf ij C T = >f ij =C -1 F ij (C T ) -1 ……(16)

[0140] Because C -1 =C T , then f ij As shown in the following formula (17):

[0141] f ij =C -1 F ij C……(17)

[0142] Substituting formula (17) into formula (15), we can obtain formula (18).

[0143]

[0144] Let the transformation matrix

[0145] but

[0146] For the data after DCT transformation of a video frame of size n*n, most of the energy of the image is concentrated in the low-frequency coefficients in the upper left corner. Only these coefficients are used during detection. The coefficients in the lower right corner of the data have no use value and can be directly assigned to 0 without calculation.

[0147] In one embodiment, setting some values ​​of the conversion matrix S to 0 can achieve the purpose of reducing the amount of calculation.

[0148] In a possible implementation method, the last K rows of the transformation matrix S are set to 0; K is related to the valid input data of the detection classifier. For example, the input of the detection classifier is a 32*32 DCT matrix. After a large number of experiments, it is found that after extracting features from the 32*32 DCT matrix, only the 8*8 DCT matrix in the upper left corner of the sample has a greater effect on target detection. Therefore, only the first 8 rows of the transformation matrix S can be set to valid values, and the remaining rows can be set to 0.

[0149] In a possible implementation method, the transformation matrix S is shown in the following formula (19).

[0150]

[0151] Based on the same technical concept, Figure 6An object detection device 600 provided in an embodiment of the present application is exemplarily shown. Figure 6 As shown, it includes: an acquisition unit 601, a determination unit 602 and a detection unit 603. The acquisition unit 601 is used to acquire N discrete cosine transform DCT matrices of the first size corresponding to each video frame to be detected in the video to be analyzed; wherein each DCT matrix of the first size corresponds to a slice of the video frame; the determination unit 602 is used to determine the key frame in the video to be analyzed according to the N DCT matrices of the first size corresponding to each video frame to be detected in the video to be analyzed; based on the kernel matrix of the DCT transformation, the N DCT matrices of the first size corresponding to the key frame are converted into a DCT matrix of the second size; the DCT matrix of the second size contains information of all slices corresponding to the key frame; the detection unit 603 is used to perform target detection on the key frame according to the DCT matrix of the second size.

[0152] In a possible implementation method, the acquisition unit 601 is used to analyze each video frame in the video to be analyzed by using a key frame extraction algorithm to obtain an initial key frame in the video to be analyzed; and obtain N first-size DCT matrices corresponding to the initial key frame in the video to be analyzed.

[0153] In a possible implementation method, the determination unit 602 is used to determine the frame difference for the i-th key frame in the video to be analyzed based on N first-size DCT matrices corresponding to the j-th frame in the video to be analyzed, N first-size DCT matrices corresponding to the i-th key frame, and a weight matrix; the weight matrix is ​​used to increase the low-frequency information in the DCT matrix and reduce the high-frequency information in the DCT matrix; wherein the j-th frame is located after the i-th key frame in the video to be analyzed; i and j are positive integers; if the frame difference is greater than the difference threshold, the j-th frame is determined to be the i+1-th key frame.

[0154] In a possible implementation method, the weights in any weight matrix decrease layer by layer according to the ladder layer, and the weights in the same ladder layer are the same; the ladder layer is a parallel oblique line formed with reference to the diagonal line from the upper right corner to the lower left corner of the matrix.

[0155] In a possible implementation method, the determination unit 602 is used to determine, for odd rows of the first matrix and the second matrix, a first difference according to the difference between the values ​​of the first matrix and the second matrix at corresponding positions and a weight matrix; the first matrix and the second matrix correspond to a first-size DCT matrix of the same slice in a video frame; for even rows of the first matrix and the second matrix, a second difference is determined according to the difference between the values ​​of the first matrix and the second matrix at corresponding positions and a standard deviation of the first matrix and the second matrix; a first difference sum between the first matrix and the second matrix is ​​determined according to the first difference and the second difference; and the inter-frame difference is determined according to the first difference sums corresponding to N first-size DCT matrices respectively.

[0156] In a possible implementation method, the determination unit 602 is used to determine the N third sub-matrices corresponding to the N first-size DCT matrices in the j-th frame according to the N first-size DCT matrices corresponding to the j-th frame in the video to be analyzed and the weight matrix corresponding to each first-size DCT matrix; splice the N third sub-matrices corresponding to the j-th frame to obtain a third matrix; determine the N fourth sub-matrices corresponding to the N first-size DCT matrices in the i-th key frame according to the N first-size DCT matrices corresponding to the i-th key frame in the video to be analyzed and the weight matrix corresponding to each first-size DCT matrix; splice the N fourth sub-matrices corresponding to the i-th key frame to obtain a fourth matrix; the dimension of the fourth matrix is ​​equal to the dimension of the third matrix; and determine the frame difference according to the difference between the third matrix and the fourth matrix.

[0157] In a possible implementation method, the determination unit 602 is used to determine a first kernel matrix and a second kernel matrix; the dimension of the first kernel matrix is ​​the same as the dimension of the first-size DCT matrix; the dimension of the second kernel matrix is ​​the same as the dimension of the second-size DCT matrix; according to the first kernel matrix and the second kernel matrix, the N first-size DCT matrices corresponding to the key frame are converted into a second-size DCT matrix.

[0158] In a possible implementation method, the determination unit 602 is used to divide the second kernel matrix according to the dimension of the first-size DCT matrix to obtain N sub-matrices; wherein each sub-matrix is ​​a subset of the second kernel matrix; a transformation matrix is ​​determined according to the N sub-matrices and the first inverse matrix; each value in the transformation matrix is ​​equal to the product of the corresponding positions of the N sub-matrices and the first inverse matrix; the first inverse matrix is ​​the inverse matrix corresponding to the first kernel matrix; and the second-size DCT matrix is ​​determined according to the N first-size DCT matrices corresponding to the key frame and the transformation matrix.

[0159] Based on the same technical concept, the embodiment of the present application provides a target detection device 700, which can be a computing device, for example. Figure 7 As shown, a target detection device 700 includes at least one processor 701 and a memory 702 connected to the at least one processor. The specific connection medium between the processor 701 and the memory 702 is not limited in the embodiment of the present application. Figure 7 For example, the processor 701 and the memory 702 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0160] In the embodiment of the present application, the memory 702 stores instructions that can be executed by at least one processor 701, and the at least one processor 701 can execute the above-mentioned target detection method by executing the instructions stored in the memory 702.

[0161] Among them, the processor 701 is a control center of the target detection device 700, which can use various interfaces and lines to connect various parts of the computer equipment, and perform resource settings by running or executing instructions stored in the memory 702 and calling data stored in the memory 702. Optionally, the processor 701 may include one or more determination units, and the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 701. In some embodiments, the processor 701 and the memory 702 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on independent chips.

[0162] Processor 701 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware processor for execution, or can be executed by a combination of hardware and software modules in the processor.

[0163] The memory 702 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 702 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 702 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 702 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0164] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer-executable program. The computer-executable program is used to enable a computer to execute a target detection method listed in any of the above methods.

[0165] An embodiment of the present application provides a computer program product, including a computer program executable by a computer device. When the program is run on the computer device, the computer device executes a target detection method listed in any of the above-mentioned ways.

[0166] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0167] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0168] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0170] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A target detection method, characterized in that: include: Obtain N discrete cosine transform (DCT) matrices of a first size corresponding to each video frame to be detected in the video to be analyzed; wherein each DCT matrix of the first size corresponds to a slice of the video frame; Determine a key frame in the video to be analyzed according to N DCT matrices of the first size corresponding to each video frame to be detected in the video to be analyzed; Based on the kernel matrix of DCT transformation, the N first-size DCT matrices corresponding to the key frame are converted into a second-size DCT matrix; the second-size DCT matrix contains information of all the slices corresponding to the key frame; Perform object detection on the key frame according to the DCT matrix of the second size.

2. The method according to claim 1, characterized in that The step of obtaining N discrete cosine transform (DCT) matrices of a first size corresponding to each video frame to be detected in the video to be analyzed includes: Analyze each video frame in the video to be analyzed by a key frame extraction algorithm to obtain an initial key frame in the video to be analyzed; Obtain N DCT matrices of the first size corresponding to the initial key frames in the video to be analyzed.

3. The method according to claim 1, characterized in that The step of determining the key frames in the video to be analyzed according to the N first-size DCT matrices corresponding to each video frame to be detected in the video to be analyzed includes: For the i-th key frame in the video to be analyzed, the frame difference is determined according to the N first-size DCT matrices corresponding to the j-th frame in the video to be analyzed, the N first-size DCT matrices corresponding to the i-th key frame, and a weight matrix; the weight matrix is ​​used to increase the low-frequency information in the DCT matrix and reduce the high-frequency information in the DCT matrix; wherein the j-th frame is located after the i-th key frame in the video to be analyzed; i and j are positive integers; If the inter-frame difference is greater than the difference threshold, the j-th frame is determined to be the (i+1)-th key frame.

4. The method according to claim 3, characterized in that The weights in any weight matrix decrease layer by layer according to the ladder layer, and the weights in the same ladder layer are the same; the ladder layer is a parallel oblique line formed with the diagonal line from the upper right corner to the lower left corner of the matrix as a reference.

5. The method according to claim 3, characterized in that The determining the frame difference according to the N first-size DCT matrices corresponding to the j-th frame in the video to be analyzed, the N first-size DCT matrices corresponding to the i-th key frame, and the weight matrix includes: For odd rows of a first matrix and a second matrix, a first difference is determined according to a difference between values ​​of the first matrix and the second matrix at corresponding positions and a weight matrix; the first matrix and the second matrix correspond to a first-size DCT matrix of the same slice in a video frame; For even-numbered rows of the first matrix and the second matrix, determine a second difference according to a difference between values ​​of the first matrix and the second matrix at corresponding positions and a standard deviation of the first matrix and the second matrix; Determine a first difference sum between the first matrix and the second matrix according to the first difference and the second difference; The inter-frame difference is determined according to the first difference values ​​and respectively corresponding to the N DCT matrices of the first size.

6. The method according to claim 3, characterized in that The determining the frame difference according to the N first-size DCT matrices corresponding to the j-th frame in the video to be analyzed, the N first-size DCT matrices corresponding to the i-th key frame, and the weight matrix includes: Determine N third sub-matrices corresponding to the N first-size DCT matrices in the j-th frame according to the N first-size DCT matrices corresponding to the j-th frame in the video to be analyzed and the weight matrix corresponding to each first-size DCT matrix; splicing the N third sub-matrices corresponding to the j-th frame to obtain a third matrix; Determine N fourth sub-matrices corresponding to the N first-size DCT matrices in the i-th key frame according to the N first-size DCT matrices corresponding to the i-th key frame in the video to be analyzed and the weight matrix corresponding to each first-size DCT matrix; splicing the N fourth sub-matrices corresponding to the i-th key frame to obtain a fourth matrix; the dimension of the fourth matrix is ​​equal to the dimension of the third matrix; The inter-frame difference is determined according to a difference between the third matrix and the fourth matrix.

7. The method according to any one of claims 1 to 6, characterized in that: Based on the kernel matrix of DCT transformation, N first-size DCT matrices corresponding to the key frames are transformed into a second-size DCT matrix, including: Determine a first kernel matrix and a second kernel matrix; the dimension of the first kernel matrix is ​​the same as the dimension of the first-size DCT matrix; the dimension of the second kernel matrix is ​​the same as the dimension of the second-size DCT matrix; According to the first core matrix and the second core matrix, the N DCT matrices of the first size corresponding to the key frame are converted into a DCT matrix of the second size.

8. The method according to claim 7, characterized in that The converting the N DCT matrices of the first size corresponding to the key frame into a DCT matrix of the second size according to the first kernel matrix and the second kernel matrix includes: Splitting the second kernel matrix according to the dimension of the DCT matrix of the first size to obtain N sub-matrices; wherein each sub-matrix is ​​a subset of the second kernel matrix; Determine a conversion matrix according to the N sub-matrices and the first inverse matrix; each value in the conversion matrix is ​​equal to the product of the N sub-matrices and the corresponding position of the first inverse matrix; the first inverse matrix is ​​the inverse matrix corresponding to the first kernel matrix; The DCT matrix of the second size is determined according to the N DCT matrices of the first size corresponding to the key frame and the conversion matrix.

9. A target detection device, characterized in that: Including acquisition unit, determination unit and detection unit: The acquisition unit is used to acquire N discrete cosine transform DCT matrices of a first size corresponding to each video frame to be detected in the video to be analyzed; wherein each DCT matrix of the first size corresponds to a slice of the video frame; The determining unit is used to determine the key frame in the video to be analyzed according to the N first-size DCT matrices corresponding to each video frame to be detected in the video to be analyzed; based on the kernel matrix of DCT transformation, the N first-size DCT matrices corresponding to the key frame are converted into a second-size DCT matrix; the second-size DCT matrix contains information of all the slices corresponding to the key frame; The detection unit is used to perform target detection on the key frame according to the DCT matrix of the second size.

10. A computing device, characterized in that include: A memory for storing program instructions; A processor is used to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 8 according to the obtained program instructions.

11. A computer-readable storage medium, characterized in that: The method comprises computer-readable instructions, and when a computer reads and executes the computer-readable instructions, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that The invention comprises a computer program executable by a computer device, and when the program is run on the computer device, the computer device is caused to execute the steps of the method according to any one of claims 1 to 8.

Images