A fusion video compression method and system
By using two samplings in different directions and fusion feature space-time in full-connection layer in video compression, using neural network to solve the loss function, the problem of large amount of video compression and restoration operations in the existing technology is solved, and efficient real-time compression of video data is achieved reduction.
Patent Information
- Application Number
- CN202210674214.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-15
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-06-15
AI Technical Summary
When existing video compression and restoration methods process videos with complex scenes and large amounts of movement of objects, the calculation amount is large and time-consuming, making it difficult to achieve efficient compression and restoration.
Two sampling processes in different directions are adopted to reduce the resolution and dimension of video data, extract the features and then improve the resolution and dimension, and use the full connection layer and neural network to fusion the feature space-time and space, real-time compression of video data is achieved through loss function solution. reduction.
It significantly reduces the amount of computing, realizes real-time compression and restoration of video data, and improves processing efficiency.
Smart Images

Figure CN115103192B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of network multimedia, and in particular to a fusion video compression method and system. Background Art
[0002] Existing video compression and restoration methods mostly use regularization theoretical models. However, for videos with complex scenes and a large number of moving objects, this method requires a large amount of iterative operations. Whether it is compression or restoration, the amount of calculation is large and very time-consuming.
[0003] Therefore, there is an urgent need for a targeted fusion video compression method and system. The deep learning theory of the machine is introduced into video compression and restoration, and the full connection layer and spatiotemporal correlation principle are used to improve the video compression and restoration operation. Summary of the invention
[0004] The purpose of the present invention is to provide a fusion video compression method and system, which adopts two sampling processes in different directions to first reduce the resolution and dimension of video data, extract features and then increase the resolution and dimension of video data, use the principle of fully connected layer to fuse the two features together in time and space, and finally call a neural network to solve the optimal solution of the loss function to achieve real-time compression and restoration of video data.
[0005] In a first aspect, the present application provides a method for compressing a fused video, the method comprising:
[0006] Obtain video data, determine the convolution window width and dimension value M in the video sampling process according to the current network environment, and determine the number of layers N of the convolution window used for sampling according to the dimension value, where M and N are positive integers;
[0007] The convolution window is used to perform M first sampling on the video data, and the width of the convolution window needs to be reduced proportionally after each first sampling, and the results after the M samplings are combined into one-dimensional data according to a preset rule, and then the first video feature is extracted;
[0008] Submitting the convolution window parameters to the server, receiving the deconvolution window parameters returned by the server, and generating a deconvolution window locally;
[0009] Performing a second sampling on the video data using the deconvolution window, and extracting a second video feature of the second sampled data;
[0010] According to the first video feature and the second video feature, according to the correlation between different features, different features with correlation higher than a preset threshold are mapped together, and fused to reconstruct new video data;
[0011] Calling a neural network model to learn the new video data and the deconvolution window parameters, obtaining a current loss function, finding an optimal solution for the loss function, and obtaining a new dimension value M;
[0012] The compression coefficient is obtained by querying the server according to the new dimension value M, and a real-time compression and restoration operation is performed on the video data.
[0013] In combination with the first aspect, in a first possible implementation manner of the first aspect, the first sampling is to reduce the resolution of the video data, and the second sampling is to increase the resolution of the video data.
[0014] In combination with the first aspect, in a second possible implementation manner of the first aspect, the first sampling and the second sampling adopt a parallel structure.
[0015] In combination with the first aspect, in a third possible implementation manner of the first aspect, the correlation between different features includes: extracting the first video feature and the second video feature in sequence to form a feature vector pair, and calculating the correlation between the feature vector pairs.
[0016] In a second aspect, the present application provides a fusion video compression system, the system comprising a processor and a memory:
[0017] The memory is used to store program code and transmit the program code to the processor;
[0018] The processor is used to execute any one of the four possible methods of the first aspect according to the instructions in the program code.
[0019] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute any one of the four possible methods in the first aspect.
[0020] Beneficial Effects
[0021] The present invention provides a fusion video compression method and system. By adopting two sampling processes in different directions, the resolution and dimension are first reduced, and then the resolution and dimension are increased after the features are extracted. The principle of the fully connected layer is fully utilized to fuse the two features together in time and space. Finally, a neural network is called to solve the optimal solution of the loss function, thereby realizing real-time compression and restoration of video data and greatly reducing the amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0023] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION
[0024] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.
[0025] Figure 1 A flowchart of a fusion video compression method provided in this application includes:
[0026] Obtain video data, determine the convolution window width and dimension value M in the video sampling process according to the current network environment, and determine the number of layers N of the convolution window used for sampling according to the dimension value, where M and N are positive integers;
[0027] The convolution window is used to perform M first sampling on the video data, and the width of the convolution window needs to be reduced proportionally after each first sampling, and the results after the M samplings are combined into one-dimensional data according to a preset rule, and then the first video feature is extracted;
[0028] Submitting the convolution window parameters to the server, receiving the deconvolution window parameters returned by the server, and generating a deconvolution window locally;
[0029] Performing a second sampling on the video data using the deconvolution window, and extracting a second video feature of the second sampled data;
[0030] According to the first video feature and the second video feature, according to the correlation between different features, different features with correlation higher than a preset threshold are mapped together, and fused to reconstruct new video data;
[0031] Calling a neural network model to learn the new video data and the deconvolution window parameters, obtaining a current loss function, finding an optimal solution for the loss function, and obtaining a new dimension value M;
[0032] The compression coefficient is obtained by querying the server according to the new dimension value M, and a real-time compression and restoration operation is performed on the video data.
[0033] In some preferred embodiments, the first sampling is to reduce the resolution of the video data, and the second sampling is to increase the resolution of the video data.
[0034] In some preferred embodiments, the first sampling and the second sampling adopt a parallel structure.
[0035] In some preferred embodiments, the step of calculating the correlation between different features includes: extracting the first video feature and the second video feature in sequence to form a feature vector pair, and calculating the correlation between the feature vector pairs.
[0036] The present application provides a fusion video compression system, the system comprising: the system comprising a processor and a memory:
[0037] The memory is used to store program code and transmit the program code to the processor;
[0038] The processor is used to execute the method described in any one of all embodiments of the first aspect according to the instructions in the program code.
[0039] The present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the method described in any one of the embodiments of the first aspect.
[0040] In a specific implementation, the present invention further provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment of the present invention. The storage medium may be a disk, an optical disk, a read-only storage memory (abbreviated as: ROM) or a random access memory (abbreviated as: RAM), etc.
[0041] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention or some parts of the embodiments.
[0042] The same and similar parts between the various embodiments of this specification can be referred to each other. In particular, for the embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.
[0043] The above-described embodiments of the present invention do not limit the protection scope of the present invention.
Claims
1. A method for compressing fused video, characterized in that: The method comprises: Obtain video data, determine the convolution window width and dimension value M in the video sampling process according to the current network environment, and determine the number of layers N of the convolution window used for sampling according to the dimension value, where M and N are positive integers; The convolution window is used to perform M first sampling on the video data, and the width of the convolution window needs to be reduced proportionally after each first sampling, and the results after the M samplings are combined into one-dimensional data according to a preset rule, and then the first video feature is extracted; Submitting the convolution window parameters to the server, receiving the deconvolution window parameters returned by the server, and generating a deconvolution window locally; Performing a second sampling on the video data using the deconvolution window to extract a second video feature of the second sampled data; According to the first video feature and the second video feature, according to the correlation between different features, different features with correlation higher than a preset threshold are mapped together, and fused to reconstruct new video data; Calling a neural network model to learn the new video data and the deconvolution window parameters, obtaining a current loss function, finding an optimal solution for the loss function, and obtaining a new dimension value M; The compression coefficient is obtained by querying the server according to the new dimension value M, and a real-time compression and restoration operation is performed on the video data.
2. The method according to claim 1, characterized in that: The first sampling is to reduce the resolution of the video data, and the second sampling is to increase the resolution of the video data.
3. The method according to claim 2, characterized in that: The first sampling and the second sampling adopt a parallel structure.
4. The method according to claim 3, characterized in that: The method according to the correlation between different features includes: extracting the first video feature and the second video feature in sequence to form a feature vector pair, and calculating the correlation between the feature vector pairs.
5. A fusion video compression system, characterized in that: The system comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the method according to any one of claims 1 to 4 according to the instructions in the program code.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the method according to any one of claims 1 to 4.
