A method for calculating video vectors

Abstract

The present invention relates to a method for calculating video vectors, and belongs to the field of image processing. The present invention extracts representative image frames from a video; calculates the eigenvectors of the extracted image frames; calculates the eigenvalues of the covariance matrix of the matrix composed of the eigenvectors of the extracted image frames, and these eigenvalues constitute the eigenvectors of the video; and performs an orthogonal transformation on the video eigenvectors to obtain the video vectors. The method of calculating video vectors using eigenvalues and orthogonal transformation proposed by the present invention is simple to calculate and takes into account some semantic information of the video. The vector differences of similar videos are small, and the vector differences of dissimilar videos are large. It can effectively distinguish different videos and has important application value in video management.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a method for calculating video vectors. Background Art

[0002] With the development of information technology, a vast amount of video has accumulated online, generating video big data. For in-depth video processing and analysis, video representation has become a crucial challenge. Video representation involves representing a video using a low-dimensional, dense, floating-point feature vector, so that the feature vectors of different videos reflect the similarities between them. Video representation, also known as video vectorization or embedding, plays an important role in video recommendation, video retrieval, and video copyright protection.

[0003] Video vectorization methods primarily rely on deep learning. This requires a large amount of manual video annotation and computing power to perform deep learning and build a video vectorization model before vectorizing new videos. This resource-intensive process hinders the application of video vectorization.

[0004] Therefore, utilizing existing deep learning results to reduce the resources required for video vectorization is of great significance for video vectorization and video big data management. This invention is generated based on this practical need. Summary of the Invention

[0005] (1) Technical issues to be resolved

[0006] The technical problem to be solved by the present invention is how to provide a method for calculating video vectors to solve the problem that existing video vectorization requires a large amount of resources, which affects the application of video vectorization.

[0007] (2) Technical solution

[0008] In order to solve the above technical problems, the present invention proposes a method for calculating a video vector, which includes the following steps:

[0009] S1, video frame separation step: extract m representative image frames from the video, and separate a video into an image frame sequence F1, F2, ..., F m ;

[0010] S2, the step of calculating the feature vector of the image frame; calculating the feature vector of the extracted image frame, and converting an image frame F i Vectorization, i = 1, ..., m, represented as an n-dimensional vector X i =(x i1 ,x i2 ,…,x in );

[0011] S3, eigenvalue calculation step: Calculate the eigenvalue λ of the matrix composed of the eigenvectors of the extracted m image frames j ,j=1,2,…,n,the eigenvalues form the eigenvectors of the video (λ1,λ2,…,λ n );

[0012] S4, orthogonal transformation step: perform orthogonal transformation on the video eigenvector to obtain the orthogonal transformation coefficients (y1, y2, ..., y n ), the orthogonal transform coefficients are used as video vectors.

[0013] Furthermore, in step S1, F i , i=1,…,m, take the I frame in the compressed video and discard the P frame and B frame.

[0014] Furthermore, in step S2, the input image frame is processed using a trained deep neural network, and the output of the last connection layer in the adopted neural network is intercepted as the preliminary feature vector of the image frame. The preliminary feature vector of the image frame is subjected to discrete cosine transform, and the first n low-frequency coefficients are intercepted as the feature vector of the image frame.

[0015] Furthermore, n=256.

[0016] Furthermore, n=128.

[0017] Furthermore, in step S3, the n-dimensional feature vectors of the m image frames are arranged into a matrix X with m rows and n columns.

[0018]

[0019] Calculate A=X T ×X, get a real symmetric matrix A with n rows and n columns, matrix X T is the transposed matrix of matrix X; calculate the eigenvalues of matrix A: λ1,λ2,…,λ n ; Arrange the eigenvalues in order and combine them into a video eigenvector D = (λ1,λ2,…,λ n ).

[0020] Furthermore, the Jacobi method is used to calculate the eigenvalues of the matrix A.

[0021] Furthermore, the eigenvalues are arranged in descending order: λ1≥λ2≥…≥λ n ≥0.

[0022] Furthermore, the eigenvalues are arranged in descending order: 0≤λ1≤λ2≤…≤λ n .

[0023] Furthermore, in step S4, the orthogonal transform adopts discrete cosine transform, Hadamard transform or Haar transform.

[0024] (3) Beneficial effects

[0025] The present invention proposes a method for calculating video vectors. The method proposed in the present invention uses eigenvalues and orthogonal transformation to calculate video vectors. The calculation is simple and takes into account some semantic information of the video. The vector differences of similar videos are small, and the vector differences of dissimilar videos are large. Different videos can be effectively distinguished and have important application value in video management. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Flowchart of the method for calculating video vectors according to the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below with reference to the accompanying drawings and examples.

[0028] The present invention relates to a method for calculating a video vector, and more specifically, to a method for calculating video features by using representative image frames in video analysis so as to determine the similarity between videos.

[0029] The present invention discloses a method for calculating video vectors, which includes: (1) a step of separating video image frames. Representative image frames are extracted from the video. (2) a step of calculating image frame feature vectors. The feature vectors of the extracted image frames are calculated. (3) a step of calculating eigenvalues. The eigenvalues of the covariance matrix of the matrix composed of the feature vectors of the extracted image frames are calculated, and these eigenvalues constitute the eigenvectors of the video. (4) an orthogonal transformation step. The video eigenvectors are orthogonally transformed to obtain the video vectors. The method of calculating video vectors using eigenvalues and orthogonal transformation proposed by the present invention is simple to calculate and takes into account some semantic information of the video. The vector differences of similar videos are small, and the vector differences of dissimilar videos are large. It can effectively distinguish different videos and has important application value in video management.

[0030] The present invention discloses a method for calculating video vectors, which includes: (1) a step of separating video image frames. Representative image frames are extracted from the video. (2) a step of calculating image frame feature vectors. The feature vectors of the extracted image frames are calculated. (3) a step of calculating eigenvalues. The eigenvalues of the covariance matrix of the matrix composed of the feature vectors of the extracted image frames are calculated, and these eigenvalues constitute the eigenvectors of the video. (4) an orthogonal transformation step. The video eigenvectors are orthogonally transformed to obtain the video vectors. The method of calculating video vectors using eigenvalues and orthogonal transformation proposed by the present invention is simple to calculate and takes into account some semantic information of the video. The vector differences of similar videos are small, and the vector differences of dissimilar videos are large. Different videos can be effectively distinguished and have important application value in video management.

[0031] The purpose of the present invention is to provide a method for calculating video vectors to meet the demand for calculating video feature vectors in video big data management.

[0032] To achieve the above object, the present invention proposes a method for calculating a video vector, the method comprising:

[0033] S1, video frame separation step. Extract m representative image frames from the video. Separate a video into an image frame sequence F1, F2, ..., F m .

[0034] S2, step of calculating the feature vector of the image frame. Calculate the feature vector of the extracted image frame. i Vectorized, represented as an n-dimensional vector X i =(x i1 ,x i2 ,…,x in ).

[0035] S3, eigenvalue calculation step. Calculate the eigenvalue λ of the matrix composed of the eigenvectors of the extracted m image frames j , j = 1, 2, ..., n, these eigenvalues constitute the eigenvectors of the video (λ1, λ2, ..., λ n ).

[0036] S4, orthogonal transformation step. Perform orthogonal transformation on the video eigenvector to obtain the orthogonal transformation coefficients (y1, y2, ..., y n ), the orthogonal transform coefficients are used as video vectors.

[0037] Example 1:

[0038] Figure 1 This is a flow chart of a method for calculating video vectors of the present invention. Figure 1 As shown, the method includes:

[0039] S1, video frame separation step. Extract m representative image frames from the video. Separate a video into an image frame sequence F1, F2, ..., F m .

[0040] In specific implementation, F i , i = 1, ..., m, take the I frame in the compressed video and discard the P frame and B frame. In this way, the extracted image frame is clearer.

[0041] S2, step of calculating the feature vector of the image frame. Calculate the feature vector of the extracted image frame. i Vectorized, represented as an n-dimensional vector X i =(x i1 ,x i2 ,…,x in ).

[0042] In specific implementation, the method in 202210247400.7 "A method for calculating image hash values using convolutional neural networks and orthogonal transformations" can be adopted. The trained deep neural network is used to process the input image frame, and the output of the last connection layer in the neural network is intercepted as the preliminary feature vector of the image frame. The preliminary feature vector of the image frame is subjected to discrete cosine transform, and the first n low-frequency coefficients are intercepted as the feature vector of the image frame, where n = 256 or 128.

[0043] S3, eigenvalue calculation step. Calculate the eigenvalue λ of the matrix composed of the eigenvectors of the extracted image frame j , j = 1, 2, ..., n, these eigenvalues constitute the eigenvectors of the video (λ1, λ2, ..., λ n ).

[0044] In specific implementation, the n-dimensional feature vectors of m image frames are arranged into a matrix X with m rows and n columns.

[0045]

[0046] Calculate A=X T ×X, we get a real symmetric matrix A with n rows and n columns. T is the transposed matrix of matrix X.

[0047] Use Jacobi and other methods to calculate the eigenvalues of matrix A, λ1,λ2,…,λ n Arrange these eigenvalues in descending order: λ1≥λ2≥…≥λ n ≥0. In another embodiment, these eigenvalues are arranged in order from large to small: 0≤λ1≤λ2≤…≤λ n .

[0048] The eigenvalues are combined into a video eigenvector D = (λ1,λ2,…,λ n ).

[0049] S4, orthogonal transformation step. Perform orthogonal transformation on the video eigenvector to obtain the orthogonal transformation coefficients (y1, y2, ..., y n ), the orthogonal transform coefficient is the video vector.

[0050] During specific implementation, the orthogonal transform may adopt discrete cosine transform, Hadamard transform, Haar transform, etc.

[0051] The method proposed in the present invention for calculating video vectors using eigenvalues and orthogonal transformation is simple to calculate and takes into account some semantic information of the video. The vector differences of similar videos are small, while the vector differences of dissimilar videos are large. It can effectively distinguish different videos and has important application value in video management.

[0052] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for calculating a video vector, characterized in that: The method comprises the following steps: S1, video frame separation step: extract m representative image frames from the video, and separate a video into an image frame sequence F1, F2, ..., F m ; S2, step of calculating the image frame feature vector; Calculate the feature vector of the extracted image frame and convert an image frame F i Vectorization, i=1,…,m, represented as an n-dimensional vector X i =(x i1 , x i2 ,…, x in ); S3, step of calculating eigenvalue; Calculate the eigenvalue λ of the matrix composed of the eigenvectors of the extracted m image frames j ,j=1,2,…,n,the eigenvalues form the eigenvectors of the video (λ1, λ2,…, λ n ); S4, orthogonal transformation step: perform orthogonal transformation on the video eigenvector to obtain the orthogonal transformation coefficients (y1, y2, ..., y n ), orthogonal transform coefficients as video vectors; in, In the step S1, F i , i=1,…,m, take the I frame in the compressed video and discard the P frame and B frame; In step S2, the input image frame is processed using a trained deep neural network, the output of the last connection layer in the adopted neural network is intercepted as the preliminary feature vector of the image frame, the preliminary feature vector of the image frame is subjected to discrete cosine transform, and the first n low-frequency coefficients are intercepted as the feature vector of the image frame.

2. The method for calculating a video vector according to claim 1, wherein: n=256。 3. The method for calculating a video vector according to claim 1, wherein: n=128。 4. The method for calculating a video vector according to any one of claims 1 to 3, wherein: In step S3, the n-dimensional feature vectors of the m image frames are arranged into a matrix X with m rows and n columns. Calculate A=X T ×X, get a real symmetric matrix A with n rows and n columns, matrix X T is the transposed matrix of matrix X; calculate the eigenvalues of matrix A: λ1,λ2, …, λ n ; Arrange the eigenvalues in order and combine them into the video eigenvector D=(λ1,λ2, …, λ n ).

5. The method for calculating a video vector according to claim 4, wherein: Compute the eigenvalues of matrix A using the Jacobi method.

6. The method for calculating a video vector according to claim 4, wherein: Arrange the eigenvalues in descending order: λ1≥λ2≥…≥λ n ≥0.

7. The method for calculating a video vector according to claim 4, wherein: Arrange the eigenvalues in descending order: 0≤λ1≤λ2≤…≤λ n .

8. The method for calculating a video vector according to claim 4, wherein: In step S4, the orthogonal transformation adopts discrete cosine transform, Hadamard transform or Haar transform.

Images