A neural network-based video compression method, system, device, and medium

Abstract

The application discloses a video compression method based on a neural network, comprising the following steps: constructing a bidirectional multi-frame prediction network based on single-frame compression and one-way prediction; inputting the obtained reconstructed frame and the original frame into a motion vector network to learn the motion vector between the two frames; obtaining an inaccurate motion vector through linear transformation to learn and obtain a predicted motion vector; encoding the difference between the predicted motion vector and the real motion vector to obtain the reconstructed motion vector; and using a residual encoder to encode the difference between the real current frame and the predicted frame. The application does not directly encode the real motion vector for motion information coding, but encodes the difference value through the auto-encoder, so as to reduce the coding complexity; the bidirectional multi-frame compensation network is constructed, the edge position of the current frame is limited through the bidirectional edge video frame, the quality of the compensation frame is improved, and then the residual information is encoded with fewer bytes.

Description

Technical Field

[0001] This invention relates to the field of video coding technology, and in particular to a video compression method, system, device and medium based on neural networks. Background Technology

[0002] With the rapid development of the economy and society, and the continuous improvement and upgrading of video surveillance systems, Yunnan Power Grid Company's video supervision platform and substation video surveillance system have generated massive amounts of video surveillance data in recent years. This data represents a crucial resource for the power grid, making it particularly important to compress, transmit, and archive this video data on a data management platform. Traditional video compression tools often struggle with compressing large amounts of video data, especially in areas with high motion or new content where inter-frame correlation is typically low and residual energy is high. These limitations stem from traditional compression methods. By employing adaptive learning through inter-frame coding in neural networks, frames acquire high-dimensional contextual features, offering advantages in restoring high-frequency details and compressing high-resolution video. Summary of the Invention

[0003] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0004] In view of the problems existing in the above and / or existing video compression methods, the present invention is proposed.

[0005] Therefore, the problem to be solved by this invention is to improve the compression efficiency of areas with large motion or areas where new content appears in a video.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, embodiments of the present invention provide a video compression method based on a neural network, comprising,

[0008] A bidirectional multi-frame prediction network is constructed based on single-frame compression and unidirectional prediction.

[0009] The current frame is predicted based on a bidirectional multi-frame prediction network. The reconstructed frame and the original frame are input into a motion vector network to learn the motion vectors between the two frames.

[0010] Inaccurate motion vectors obtained through linear transformation are learned to obtain predicted motion vectors.

[0011] The difference between the predicted motion vector and the actual motion vector is encoded to obtain the reconstructed motion vector;

[0012] The predicted frame is obtained by performing distortion and edge extraction operations on the reconstructed motion vector and the reconstructed frame. The difference between the real current frame and the predicted frame is encoded using a residual encoder.

[0013] As a preferred embodiment of the neural network-based video compression method of the present invention, the network structure for bidirectional multi-frame prediction includes:

[0014] Five fixed layers are constructed, each used to predict different video frames;

[0015] Set the GOP to a fixed size of 13;

[0016] For each GOP, the 13 video frames to be processed are divided into 5 parts;

[0017] Different compression methods are used for different video frames.

[0018] As a preferred embodiment of the neural network-based video compression method of the present invention, the step of compressing different video frames in different ways includes:

[0019] The first layer of frames includes the first video frame and the thirteenth video frame, which are encoded using single-frame compression;

[0020] The second layer of frames includes the 3rd, 5th, 9th and 11th video frames, which are reconstructed through unidirectional prediction and used as input to the bidirectional predictable multi-frame learning video compression (BPM-LVC) network.

[0021] The third layer frame includes the seventh video frame, which is compressed using the BPM-LVC network;

[0022] The fourth layer frame includes the 4th, 6th, 8th and 10th video frames, and the fifth layer frame includes the 2nd and 12th video frames. Both the fourth layer frame and the fifth layer frame compress the corresponding video frames through the TRF-LVC network.

[0023] As a preferred embodiment of the neural network-based video compression method of the present invention, wherein: the reconstructed frame and the original frame are jointly input into a motion vector network to learn the motion vectors between the two frames, including...

[0024] Using a motion vector network, obtain four motion vectors from the 3rd, 5th, 9th, and 11th reconstructed frames to the 7th frame of the current frame;

[0025] Each motion vector is viewed as two feature maps, each representing the x-direction and y-direction of the motion vector, respectively.

[0026] The four motion vectors are combined to form eight feature maps, which are then input into the motion vector network.

[0027] The feature map is reconstructed into four motion vectors by performing convolution operations and nonlinear transformations through a motion vector network.

[0028] As a preferred embodiment of the video compression method based on neural networks described in this invention, the step of obtaining inaccurate motion vectors through linear transformation and learning to obtain predicted motion vectors includes obtaining four inaccurate motion vectors from the 3rd, 5th, 9th and 11th video frames to the 7th frame of the current frame through linear transformation.

[0029] The inaccurate motion vectors are concatenated and used as input to learn the true motion vectors, resulting in four predicted motion vectors.

[0030] As a preferred embodiment of the neural network-based video compression method of the present invention, the step of encoding the difference between the predicted motion vector and the real motion vector using a residual autoencoder includes,

[0031] The difference between the predicted motion vector and the actual motion vector is used to obtain the difference motion vector.

[0032] The obtained difference motion vectors are concatenated and used as input, and then encoded using a codec.

[0033] The concatenated reconstructed difference motion vector is obtained through the decoder;

[0034] The reconstructed motion vector is obtained by adding the predicted motion vector to the reconstructed difference motion vector.

[0035] As a preferred embodiment of the neural network-based video compression method of the present invention, the step of obtaining a predicted frame by performing distortion and edge extraction operations on the reconstructed motion vector and the reconstructed frame, and encoding the difference between the real current frame and the predicted frame using a residual encoder includes,

[0036] The reconstructed motion vectors are concatenated with the reconstructed frames and then warped to obtain the warped video frames.

[0037] Perform edge extraction on the obtained distorted video frames to obtain edge video frames;

[0038] Distort the edge video frame to the current frame to obtain the distorted edge video frame;

[0039] The distorted video frame is stitched together with the distorted edge video frame to learn and obtain the prediction frame; the difference between the real current frame and the prediction frame is encoded by a residual encoder.

[0040] Secondly, embodiments of the present invention provide a video compression system based on a neural network, which includes: a bidirectional multi-frame prediction network module, used to construct a bidirectional multi-frame prediction network based on single-frame compression and unidirectional prediction;

[0041] The motion vector network module is used to predict the current frame based on the bidirectional multi-frame prediction network. The reconstructed frame and the original frame are input into the motion vector network to learn the motion vectors between the two frames.

[0042] The motion prediction module is used to learn from inaccurate motion vectors obtained through linear transformation to obtain predicted motion vectors.

[0043] The motion compensation module is used to encode the difference between the predicted motion vector and the true motion vector to obtain the reconstructed motion vector;

[0044] The residual module is used to obtain the prediction frame by performing distortion and edge extraction operations on the reconstructed motion vector and the reconstructed frame, and to encode the difference between the real current frame and the prediction frame using the residual encoder.

[0045] Thirdly, embodiments of the present invention provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any step of the above-described method.

[0046] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any step of the above-described method.

[0047] The beneficial effects of this invention are as follows: a motion vector prediction network and a bidirectional multi-frame compensation network are designed. For motion information encoding, the actual motion vectors are not directly encoded; instead, the difference is taken and encoded after passing through an autoencoder, thereby reducing encoding complexity. A bidirectional multi-frame compensation network is constructed to restrict the edge position of the current frame through bidirectional edge video frames, improving the quality of the compensation frame and thus using fewer bytes to encode residual information. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0049] Figure 1 This is a flowchart of a video compression method based on neural networks.

[0050] Figure 2 This is a schematic diagram of a bidirectional predictable multi-frame learning video compression network framework.

[0051] Figure 3 A schematic diagram of the network framework for learning and predicting motion vectors.

[0052] Figure 4 The image shows the comparison results of frame 7 obtained from testing the JCT-VC dataset. Detailed Implementation

[0053] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0054] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0055] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0056] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0057] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0058] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0059] Example 1

[0060] Reference Figure 1 and Figure 2 This is the first embodiment of the present invention, which provides a video compression method based on a neural network, including:

[0061] S100: Construct a bidirectional multi-frame prediction network based on single-frame compression and unidirectional prediction.

[0062] It should be noted that a fixed set of 5 layers is constructed, each used to predict different video frames; a fixed GOP size of 13 is set; for each GOP, the 13 video frames to be processed are divided into 5 parts; and different compression methods are applied to different video frames, specifically including:

[0063] The first layer of frames includes the first video frame and the thirteenth video frame, which are encoded using single-frame compression;

[0064] The second layer of frames includes the 3rd, 5th, 9th and 11th video frames, which are reconstructed through unidirectional prediction and used as input to the bidirectional predictable multi-frame learning video compression (BPM-LVC) network.

[0065] The third layer frame includes the seventh video frame, which is compressed using the BPM-LVC network;

[0066] The fourth layer frame includes the 4th, 6th, 8th and 10th video frames, and the fifth layer frame includes the 2nd and 12th video frames. Both the fourth layer frame and the fifth layer frame compress the corresponding video frames through the TRF-LVC network.

[0067] Taking the first GOP as an example, the 7th video frame is compressed using a BPM-LVC network with a symmetrical structure. The following descriptions of the bidirectional multi-frame prediction network will also use the 7th frame as an example, because the 4th and 5th video frames both employ the bidirectional multi-frame prediction network.

[0068] First, the network structure for the 7th video frame uses the already reconstructed video frames (frames 3, 5, 9, and 11) as input data to predict the current 7th frame. Frames 3, 5, 9, and 11 are reconstructed video frames obtained through one-way prediction. Using such video frames to predict the 7th frame prevents error accumulation. Here, the first video frame is obtained through single-frame compression and only intra-frame coding, resulting in the highest quality reconstructed image. However, using this reconstructed frame for one-way prediction, where each prediction uses the video frame obtained from the previous frame to predict the next frame, leads to error accumulation. Therefore, only two one-way predictions are performed on each of the single-frame compressed reconstructed frames. This prevents error accumulation, and the two reconstructed frames obtained through one-way prediction have the highest quality.

[0069] Therefore, compared to single-frame compressed video frames, the reconstructed frames (frames 3, 5, 9, and 11) obtained through unidirectional prediction are of the second-highest quality, only slightly lower than the single-frame compressed frames. These four video frames are used to predict the 7th frame, and subsequent video frames are predicted using these higher-quality frames. The higher the quality of the video frames used for prediction, the closer they will be to the original video frames, and the higher the accuracy. Thus, the predicted video frames are of relatively high quality.

[0070] S200: The current frame is predicted based on a bidirectional multi-frame prediction network. The reconstructed frame and the original frame are input into a motion vector network to learn the motion vectors between the two frames.

[0071] It should be noted that the motion vectors from frame 3 to frame 7 are obtained through a motion vector network (MV-net) to obtain inter-frame motion information, and the reconstructed frame from frame 3 is then used. The original frame x6 (frame 7) is input into MV-net to learn the motion vector v between the two frames. 2→6 :

[0072]

[0073] Where MV represents the MV-net learning process, [:, :] represents the concatenation between channels, and the motion vectors from the 5th to the 7th frame are v. 4→6 The motion vector from frame 9 to frame 7 is v 8→6 The motion vector v from frame 11 to frame 7 10→6 They all use MV-net to obtain the data.

[0074]

[0075] It should be noted that a pyramid network is used here to process motions with a large time span in order to obtain motion information. This method can obtain a larger receptive field and improve the accuracy of motion vectors. Here, all motion vectors from the reconstructed frame to the current frame are obtained. The MV network structure can estimate pixel-level motion from coarse to fine between frames, and the motion vector field is more accurate.

[0076] Furthermore, each motion vector can be viewed as two feature maps, each representing the x-direction and y-direction of the motion vector. The four motion vectors are merged together to form eight feature maps, which are then fed into the next network. The motion vector network performs convolution and nonlinear transformations on these feature maps to obtain the reconstructed four motion vectors. Here, the motion vectors are encoded using an autoencoder network to compress them.

[0077] S300: Learn from inaccurate motion vectors obtained through linear transformation to obtain predicted motion vectors.

[0078] It should be noted that four inaccurate motion vectors are obtained from the 3rd, 5th, 9th and 11th video frames to the 7th frame of the current frame through linear transformation; the obtained inaccurate motion vectors are concatenated and used as input to learn the real motion vectors, resulting in four predicted motion vectors.

[0079] It should be noted that learning from inaccurate motion vectors obtained through linear transformation does not deviate significantly from the true motion vectors. A small "U-shaped" network framework is designed here to learn and predict motion vectors, the purpose of which is to obtain predicted motion vectors so that the difference between the predicted and true motion vectors can be compressed later.

[0080] S400: Encode the difference between the predicted motion vector and the true motion vector to obtain the reconstructed motion vector.

[0081] It should be noted that the difference between the predicted motion vector and the true motion vector is used to obtain the difference motion vector; the difference motion vector is concatenated and used as input, and then encoded using an encoder-decoder; the concatenated reconstructed difference motion vector is obtained through the decoder; and the predicted motion vector and the reconstructed difference motion vector are added together to obtain the reconstructed motion vector.

[0082] S500: The predicted frame is obtained by performing distortion and edge extraction operations on the reconstructed motion vector and the reconstructed frame. The difference between the real current frame and the predicted frame is encoded using a residual encoder.

[0083] It should be noted that the reconstructed motion vectors are concatenated with the reconstructed frame and warped to obtain a warped video frame; edge extraction is performed on the warped video frame to obtain an edge video frame; the edge video frame is warped to the current frame to obtain a warped edge video frame; the warped video frame and the warped edge video frame are concatenated and learned to obtain a predicted frame; and the difference between the real current frame and the predicted frame is encoded by a residual encoder.

[0084] It should be noted that distorted frames and frames with distorted edges are fed together into the network to learn the current frame. These feature maps are then concatenated and fed into the network to learn the true current frame. To remove artifacts from the video frame, it needs to be as close to the current frame as possible. To learn more features from distorted and distorted edge frames, the number of channels is gradually increased. The upsampled video frame has the same size as the original video frame. A total of four upsampling operations are performed, so the resolution of all video frames should be a multiple of 16. The resulting predicted frame has a slight error compared to the true current frame. To more accurately reconstruct the current frame, a residual autoencoder is used. The residual encoder network is used to compare the true current frame x6 with the predicted frame. The difference between them is encoded. The compensated... The difference between the current frame and the real frame (x6) is fed into the residual autoencoder. The residual autoencoder does not require fine feature extraction, but it needs the largest possible field of view to encompass the difference features of the entire video frame.

[0085] The above describes the process of generating inaccurate motion vectors, which are used in the motion vector prediction process for each video frame. The main idea behind these two horizontal frames is to utilize information from high-quality frames to improve the quality of the reconstructed frames. We can see that there is no prediction relationship between the remaining six video frames; these frames are all predicted from previous higher-quality video frames.

[0086] Furthermore, this embodiment also provides a video compression system based on a neural network, including:

[0087] A bidirectional multi-frame prediction network module is used to construct a bidirectional multi-frame prediction network based on single-frame compression and unidirectional prediction.

[0088] The motion vector network module is used to predict the current frame based on the bidirectional multi-frame prediction network. The reconstructed frame and the original frame are input into the motion vector network to learn the motion vectors between the two frames.

[0089] The motion prediction module is used to learn from inaccurate motion vectors obtained through linear transformation to obtain predicted motion vectors.

[0090] The motion compensation module is used to encode the difference between the predicted motion vector and the true motion vector to obtain the reconstructed motion vector;

[0091] The residual module is used to obtain the prediction frame by performing distortion and edge extraction operations on the reconstructed motion vector and the reconstructed frame, and to encode the difference between the real current frame and the prediction frame using the residual encoder.

[0092] This embodiment also provides a computer device applicable to video compression methods based on neural networks, including:

[0093] The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor executes these instructions to implement the neural network-based video compression method proposed in the above embodiments.

[0094] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0095] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements the video compression method based on a neural network as proposed in the above embodiments.

[0096] The storage medium proposed in this embodiment and the data storage method proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0097] Example 2

[0098] Reference Figures 2-4 This is the second embodiment of the present invention, which provides a video compression method based on neural networks. In order to verify the beneficial effects of the present invention, the experimental results were obtained by testing each group of videos in the JCT-VC test set, as shown in Table 1.

[0099] Table 1. Performance comparison of various algorithms on the JCT-VC test set.

[0100]

[0101] As shown in Table 1, for video compression tasks, a higher PSNR value indicates higher image quality, while a lower BPP value is better. The values ​​in bold represent the best experimental results. The JCT-VC test set includes three video classes (Class A, Class B, and Class C). The average result for each class is taken. Except for Class A, which has two videos, the other classes each have three videos. All videos have a minimum of 150 frames and a maximum of 600 frames.

[0102] The average PSNR value of this method is higher than that of DVC and HLVC, and the BPP value is lower than that of DVC and HLVC, based on the average values ​​of Class A and Class B video results. This proves that the video frame quality obtained by this method is better than that of DVC and HLVC, and the encoding cost is lower than that of DVC and HLVC, thus verifying the effectiveness of this method.

[0103] Based on the results of Class C videos, the video frame quality of this method is higher. Although the encoding cost of HLVC is lower than that of this method, the encoding cost of DVC is higher than that of this method.

[0104] Furthermore, a video frame was extracted from several videos in the JCT-VC test set as the visualization result, such as... Figure 4 As shown. For each selected video, its 7th frame, reconstructed from the video frame, is used for comparison. Each row represents the test results for different videos. Column 1 represents the original image, column 2 represents the test results obtained using our proposed method, column 3 represents the test results obtained using DVC, and column 4 represents the test results obtained using HLVC.

[0105] As shown in the figures, the results of this method are superior to DVC and HLVC. In the first row of reconstructed frames, compared with the original video, the accuracy of the railings in the DVC and HLVC results is lower. In the reconstructed video frames obtained by this method, the number of railings corresponds, and the reconstruction quality is higher. In the second row of reconstructed frames, the car in the HLVC result image is more distorted. In the third row of reconstructed frames, the textbook in the DVC result image has obvious distortion. In the fourth row of reconstructed frames, the reconstruction results of the sleeve texture in the DVC and HLVC results are poor.

[0106] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A video compression method based on neural networks, characterized in that: include, A bidirectional multi-frame prediction network is constructed based on single-frame compression and unidirectional prediction. The current frame is predicted based on a bidirectional multi-frame prediction network. The reconstructed frame and the original frame are input into a motion vector network to learn the motion vectors between the two frames. Inaccurate motion vectors obtained through linear transformation are learned to obtain predicted motion vectors. The difference between the predicted motion vector and the actual motion vector is encoded to obtain the reconstructed motion vector; The predicted frame is obtained by performing distortion and edge extraction operations on the reconstructed motion vector and the reconstructed frame. The difference between the real current frame and the predicted frame is encoded using a residual encoder. The bidirectional multi-frame prediction network structure includes, Five fixed layers are constructed, each used to predict different video frames; Set the GOP to a fixed size of 13; For each GOP, the 13 video frames to be processed are divided into 5 parts; Different compression methods are used for different video frames; The method of compressing different video frames using different methods includes... The first layer of frames includes the first video frame and the thirteenth video frame, which are encoded using single-frame compression; The second layer of frames includes the 3rd, 5th, 9th and 11th video frames, which are reconstructed through unidirectional prediction and used as input to the bidirectional predictable multi-frame learning video compression (BPM-LVC) network. The third layer frame includes the seventh video frame, which is compressed using the BPM-LVC network; The fourth layer frame includes the 4th, 6th, 8th and 10th video frames, and the fifth layer frame includes the 2nd and 12th video frames. Both the fourth layer frame and the fifth layer frame compress the corresponding video frames through the TRF-LVC network. The process of learning from inaccurate motion vectors obtained through linear transformation to obtain predicted motion vectors includes, Four inaccurate motion vectors from the 3rd, 5th, 9th and 11th video frames to the 7th frame of the current frame are obtained by linear transformation. The inaccurate motion vectors are concatenated and used as input to learn the true motion vectors, resulting in 4 predicted motion vectors. The difference between the predicted motion vector and the true motion vector is encoded using a residual autoencoder. The difference between the predicted motion vector and the actual motion vector is used to obtain the difference motion vector. The obtained difference motion vectors are concatenated and used as input, and then encoded using an encoder. The concatenated reconstructed difference motion vector is obtained through the decoder; The reconstructed motion vector is obtained by adding the predicted motion vector to the reconstructed difference motion vector. The process of obtaining a predicted frame by performing distortion and edge extraction operations on the reconstructed motion vectors and the reconstructed frame, and then encoding the difference between the real current frame and the predicted frame using a residual encoder includes... The reconstructed motion vectors are concatenated with the reconstructed frames and then warped to obtain the warped video frames. Perform edge extraction on the obtained distorted video frames to obtain edge video frames; Distort the edge video frame to the current frame to obtain the distorted edge video frame; The distorted video frame is stitched together with the distorted edge video frame to learn and obtain the prediction frame; the difference between the real current frame and the prediction frame is encoded by a residual encoder.

2. The video compression method based on neural networks as described in claim 1, characterized in that: The The reconstructed frame and the original frame are input together into the motion vector network to learn the motion vectors between the two frames, including... Using a motion vector network, obtain four motion vectors from the 3rd, 5th, 9th, and 11th reconstructed frames to the 7th frame of the current frame; Each motion vector is viewed as two feature maps, each representing the x-direction and y-direction of the motion vector, respectively. The four motion vectors are combined to form eight feature maps, which are then input into the motion vector network. The feature map is reconstructed into four motion vectors by performing convolution operations and nonlinear transformations through a motion vector network.

3. A video compression system based on neural networks, characterized in that, The method of claim 1 is used, comprising: A bidirectional multi-frame prediction network module is used to construct a bidirectional multi-frame prediction network based on single-frame compression and unidirectional prediction. The motion vector network module is used to predict the current frame based on the bidirectional multi-frame prediction network. The reconstructed frame and the original frame are input into the motion vector network to learn the motion vectors between the two frames. The motion prediction module is used to learn from inaccurate motion vectors obtained through linear transformation to obtain predicted motion vectors. The motion compensation module is used to encode the difference between the predicted motion vector and the true motion vector to obtain the reconstructed motion vector; The residual module is used to obtain the prediction frame by performing distortion and edge extraction operations on the reconstructed motion vector and the reconstructed frame, and to encode the difference between the real current frame and the prediction frame using the residual encoder.

4. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 2.

Images