Video coding and decoding method, video codec and electronic device

Abstract

The present application discloses a video encoding and decoding method, a video codec and an electronic device, and belongs to the field of electronic technology. The video encoding and decoding method comprises: obtaining a first video frame sequence to be encoded; inputting the first video frame sequence into a time-series reversible neural network to obtain a video encoding code stream corresponding to a first preset distribution outputted in a forward direction by the time-series reversible neural network; in the case of decoding, inputting the video encoding code stream and a second preset distribution into the time-series reversible neural network to obtain a second video frame sequence outputted in a reverse direction by the time-series reversible neural network.

Description

Technical Field

[0001] The present application belongs to the field of electronic technology, and specifically relates to a video encoding and decoding method, a video codec and an electronic device. Background Art

[0002] Video plays an important role in our daily lives, such as video calls, short video shooting, and video watching. With the development of network technology, the massive amount of video data has brought huge challenges to the transmission, storage, and processing of videos, and video encoding and decoding technology has become crucial. Coding technology can be used to encode and compress videos to facilitate storage and transmission. Decoding is the reverse process of encoding, which is used to restore the encoded data into video.

[0003] At present, video coding and decoding are mainly achieved through standard coding and decoding technologies such as H.264 / H.265. The H.264 / H.265 standard coding and decoding technology first predicts the coded image, then obtains the residual from the original image and the predicted image, and then transforms, quantizes and entropy encodes the residual to form a bit stream. When this method uses a lower bit rate encoding, there is too little information in the coded data, resulting in a large difference between the reconstructed video and the original video, and the video quality is poor. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a video coding and decoding method, a video codec and an electronic device, which can solve the problem of poor video quality caused by coding and decoding.

[0005] In a first aspect, an embodiment of the present application provides a video encoding and decoding method, the method comprising:

[0006] Acquire a first video frame sequence to be encoded; input the first video frame sequence into a time-reversible neural network to obtain a video encoding stream corresponding to a first preset distribution output by the time-reversible neural network; wherein the first preset distribution includes a distribution of high-frequency feature codes of video frames in the first video frame sequence, and the video encoding stream includes low-frequency feature codes of video frames in the first video frame sequence; in the case of decoding, input the video encoding stream and a second preset distribution into the time-reversible neural network to obtain a second video frame sequence output by the time-reversible neural network in reverse; wherein the time-reversible neural network samples the second preset distribution to obtain high-frequency feature codes, and determines the video frame sequence in combination with the high-frequency feature codes and the video encoding stream.

[0007] In the above technical solution provided by the present application, the video to be encoded is input into the time-series reversible neural network, which can learn the high-frequency features in the video as a preset distribution and map the low-frequency features corresponding to the high-frequency features into the encoded bitstream. The high-frequency features do not need to be encoded, which can make the bit rate of the encoded bitstream much lower than the original video, improve the compression rate, and realize the compression encoding of the video.

[0008] During decoding, the preset distribution can be used to resample the high-frequency features of the video, and then the high-frequency features can be combined with the low-bitrate encoded code stream to restore the high-resolution video and avoid the distortion caused by video encoding and decoding. In addition, compared with the prior art, the technical solution of the present application does not need to calculate the residual of the image before and after encoding, and also saves the process of processing the residual, so the encoding efficiency is higher. At the same time, the residual part does not need to be encoded, which can also improve the compression rate.

[0009] In a second aspect, an embodiment of the present application provides a video codec, including:

[0010] A first acquisition module is used to acquire a first video frame sequence to be encoded; a first encoding module is used to input the first video frame sequence into a time-series reversible neural network to obtain a video encoding code stream corresponding to a first preset distribution output by the time-series reversible neural network; wherein the first preset distribution includes the distribution of high-frequency feature codes of video frames in the first video frame sequence, and the video encoding code stream includes low-frequency feature codes of video frames in the first video frame sequence; a first decoding module is used to input the video encoding code stream and a second preset distribution into the time-series reversible neural network in the case of decoding, to obtain a second video frame sequence output by the time-series reversible neural network in reverse; wherein the time-series reversible neural network samples the second preset distribution to obtain high-frequency feature codes, and determines the video frame sequence in combination with the high-frequency feature codes and the video encoding code stream.

[0011] In the above technical solution provided by the present application, when encoding is required, the video frame sequence in the first acquisition module can be encoded by the first encoding module to obtain a corresponding encoding code stream. The first encoding module can learn the high-frequency features in the video as a preset distribution, and map the low-frequency features corresponding to the high-frequency features to the encoding code stream. The high-frequency features do not need to be encoded, which can make the bit rate of the encoding code stream much lower than the original video, improve the compression rate, and realize the compression encoding of the video.

[0012] When decoding is required, the video encoding code stream to be decoded can be decoded by the first decoding module to obtain a decoded video. The first decoding module can sample from a preset distribution to obtain high-frequency features, and decode in combination with the high-frequency features to restore a high-resolution video, improve the quality of the video, and avoid distortion caused by video encoding and decoding. In addition, the technical solution is simple to operate and has high decoding efficiency.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the video encoding and decoding method described in the first aspect is implemented.

[0014] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the video encoding and decoding method as described in the first aspect is implemented.

[0015] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the video encoding and decoding method as described in the first aspect.

[0016] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the video encoding and decoding method as described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flowchart of a video encoding and decoding method provided in an embodiment of the present application;

[0018] Figure 2 It is one of the structural schematic diagrams of the time-series reversible neural network in the video encoding and decoding method provided in the embodiment of the present application;

[0019] Figure 3 This is the second structural diagram of the time-series reversible neural network in the video encoding and decoding method provided in the embodiment of the present application;

[0020] Figure 4 This is the third structural diagram of the time-series reversible neural network in the video encoding and decoding method provided in the embodiment of the present application;

[0021] Figure 5 is a schematic diagram of the structure of a video encoding and decoding device provided in an embodiment of the present application;

[0022] Figure 6 It is one of the structural schematic diagrams of the electronic device provided in the embodiment of the present application;

[0023] Figure 7 This is the second structural schematic diagram of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0025] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0026] The video encoding and decoding method, video codec, and electronic device provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0027] This embodiment first provides a video encoding and decoding method. Exemplarily, the video encoding and decoding method can be applied to electronic devices such as mobile phones, tablet computers, personal computers (PCs), wearable electronic devices (such as smart watches), augmented reality (AR) / virtual reality (VR) devices, and vehicle-mounted devices, and this embodiment of the application does not impose any restrictions on this.

[0028] The video needs to be encoded before transmission, and the original video signal is encoded into a binary code stream suitable for transmission and storage, so that the electronic device can transmit and store it. Similarly, the video on the electronic device is saved in the format of a binary code stream. When playing the video, the binary code stream needs to be decoded to reconstruct the video for playback. The electronic device can implement the above encoding or decoding function through the video encoding and decoding method provided in this embodiment.

[0029] Figure 1 FIG. 1 is a flowchart of a video encoding and decoding method provided by an embodiment of the present application. Figure 1The video encoding and decoding method includes two parts: encoding (step 100 and step 200) and decoding (step 300). In this embodiment, the two parts are described in the order of encoding and decoding. However, the electronic device can implement encoding and decoding at the same time, or can implement only one of the parts separately, and this embodiment does not make any limitation on this.

[0030] Next, Figure 1 Each step of the video encoding and decoding method is described in detail.

[0031] Step 100: Obtain a first video frame sequence to be encoded.

[0032] A video frame is an image, and a video is a continuous sequence of images. The greater the frame rate, the more frames are contained in the video, and the smoother the video picture. According to the frame rate of the video, the video can be converted into multiple images that are continuous in time, that is, a video frame sequence. For ease of distinction, the video frame sequence corresponding to the video in the encoding process is recorded as the first video frame sequence. Exemplarily, by capturing images from the acquired video at a frame rate of 24FPS, a first video frame sequence of 24 frames per second can be obtained. The total number of frames included in the first video frame sequence can be determined according to the duration of the video. The first video frame sequence can include a total of N video frames, and N can be a positive integer.

[0033] Step 200: Input the first video frame sequence into a time-reversible neural network to obtain a video encoding stream corresponding to a first preset distribution that is a forward output of the time-reversible neural network; wherein the first preset distribution includes the distribution of high-frequency features of the video frames in the first video frame sequence, and the video encoding stream includes the encoding of low-frequency features of the video frames in the first video frame sequence.

[0034] The temporal reversible neural network may include an invertible neural network (INN), which can encode video data, i.e., a first video frame sequence, during forward propagation, and output a coded bitstream of the video frame sequence. During reverse propagation, the coded bitstream can also be decoded to output a corresponding video frame sequence. Below, the process of encoding a video frame sequence by the temporal reversible neural network is first introduced.

[0035] Specifically, the first video frame sequence is input into a time-series reversible neural network, which is trained to extract the features of each video frame in the first video frame sequence in the forward direction, and determine the high-frequency features in the video frame that obey the first preset distribution, as well as the low-frequency features other than the high-frequency features. The encoded code stream of the first video frame sequence includes the encoding of the low-frequency features of each video frame. The high-frequency features obey the preset distribution (such as the first preset distribution), and therefore can be discarded. That is to say, the encoded code stream only includes the low-frequency information of the original video. Compared with the original video, the bit rate is greatly reduced, and the compression encoding of the original video can be achieved. Compared with the encoding method in the prior art, the video encoding and decoding method in this embodiment does not require downsampling processing, and the encoding process is more efficient.

[0036] When training a time-series reversible neural network, a first preset distribution to be learned by the time-series reversible neural network can be set. For example, the first preset distribution can be set to a Gaussian distribution, a uniform distribution, a Rayleigh distribution, etc., and this embodiment does not specifically limit this. The trained time-series reversible neural network can extract high-frequency features that obey the first preset distribution and low-frequency features that do not obey the first preset distribution in the input video, and then output the encoding of the low-frequency features. High-frequency features refer to information about areas with high color change frequencies in the image, and low-frequency features refer to information about areas with slow color changes.

[0037] Exemplarily, the time-series reversible neural network is composed of N sequentially connected time-series reversible units, where N is a positive integer. The time-series reversible unit can be a reversible neural network. One time-series reversible unit can be used to encode or decode one video frame, and N time-series reversible units can encode N video frames in parallel, which can improve encoding efficiency.

[0038] Specifically, the N video frames in the first video frame sequence are respectively input into N sequential reversible units to obtain a coded stream corresponding to the first preset distribution outputted in the forward direction by each sequential reversible unit. Then, the N coded streams outputted in the forward direction by the N sequential reversible units are spliced ​​in the order of the video frames to obtain a video coded stream corresponding to the N video frames. If the number of video frames in the first video frame sequence is greater than N, the sequential reversible units can be input in batches. For example, the first to Nth video frames are respectively input into N sequential reversible units to obtain the coded streams of the first N video frames, and then the N+1th to N+Nth video frames are input into the sequential reversible unit to obtain the coded streams of the next N video frames, and finally all the coded streams are spliced ​​to obtain the overall video coded stream.

[0039] Exemplarily, the above-mentioned temporal reversible unit may include a spatiotemporal long short-term memory network (ST-LSTM) and a reversible neural network INN. In this embodiment, the spatiotemporal long short-term memory network is combined with the reversible neural network to construct a new model structure, namely, a temporal reversible neural network. When the video is encoded by the temporal reversible neural network, the ST-LSTM can extract the inter-frame features of the video and improve the continuity of the video encoding; the INN can learn the distribution of high-frequency information in the frame, and output the encoded code stream according to the distribution information, thereby reducing distortion.

[0040] Figure 2 FIG. 1 shows a structural diagram of a sequential reversible unit. Figure 2 As shown, the time-series reversible unit includes an ST-LSTM unit 201, an ST-LSTM unit 202 and an INN unit 203. For example, the first video frame in the video frame sequence is first input into the time-series reversible unit for encoding. The ST-LSTM unit 201 in the time-series reversible unit can extract the inter-frame features of the first video frame, merge them with the inter-frame features of the previous video frame, obtain the inter-frame coding features between the first video frame and the previous video frame, and pass the inter-frame coding features to the next (second) time-series reversible unit for calculating the inter-frame coding features of the next video frame. Since there are no other video frames before the first video frame, the inter-frame coding features of the previous video frame input to the time-series reversible unit can be determined as 0. In other words, the ST-LSTM unit 201 in the first time-series reversible unit can extract the inter-frame features of the first video frame and merge them with 0, and the result of the merging is output as the inter-frame coding features to the ST-LSTM of the next time-series reversible unit. At the same time, the inter-frame coding features output by the first time-reversible unit can be input into the INN unit 203, and the INN unit 203 can extract the intra-frame features corresponding to the first preset distribution in the first video frame. The intra-frame features can be low-frequency features in the first video frame, and then the intra-frame features are fused with the inter-frame coding features output by the ST-LSTM unit 201, and the fused features are mapped into a coded bit stream.

[0041] It can be seen that each time-reversible unit can be encoded not only based on the characteristics of the video frame, but also based on the inter-frame characteristics between the video frames, so that the encoded bitstream not only contains the information of the video frame, but also contains the information between the video frames, which can improve the accuracy of the encoding.

[0042] From the above, we can see that the input of each time-reversible unit is a video frame and the inter-frame coding features between the previous video frames, and the output is the encoded code stream corresponding to the input video frame and the inter-frame coding features common to the video frame and the previous video frame.

[0043] Furthermore, for the i-th (i=2, 3, 4, ..., N-1, N) video frame in the video frame sequence, the i-th video frame and the inter-frame coding features output by the i-1th temporal reversible unit are input into the i-th temporal reversible unit, and the coded code stream of the i-th video frame and the inter-frame coding features between the first i video frames can be obtained. Among them, the inter-frame coding features output by the i-1th temporal reversible unit are the inter-frame coding features between the first i-1 video frames. The serial numbers of the temporal reversible units are arranged in the forward order of the N layers of sequentially connected temporal reversible units. For example, when i is 1, it refers to the first temporal reversible unit arranged in the forward order, the inter-frame coding features input by the first temporal reversible unit are 0, and the inter-frame coding features output are the inter-frame coding features of the first video frame. It can be understood that the first temporal reversible unit in the forward order is the last temporal reversible unit in the reverse order.

[0044] For example, if the video frame sequence is (T1, T2, T3, ... TN), when the video frames T1 to TN in the video frame sequence are input into the time-series reversible neural network, each time-series reversible unit processes one video frame respectively. Figure 3 As shown, the time-series reversible unit 301 is used as the first time-series reversible unit in the forward sequence. The first video frame T1 is input into the first time-series reversible unit 301, and the inter-frame coding feature F0 also needs to be input into the time-series reversible unit 301. The inter-frame coding feature F0 can be set to 0. The time-series reversible unit 301 can encode the video frame T1, and output the coded code stream S1 of the video frame T1 and the inter-frame coding feature F1 of the previous video frame. The inter-frame coding feature F1 can be a feature obtained by merging the inter-frame feature in the video frame T1 on the basis of F0.

[0045] Similarly, the second video frame T2 in the video frame sequence can be input into the second time-reversible unit 302, and the time-reversible unit 302 can extract the inter-frame features of the video frame T2, merge the inter-frame features with the inter-frame coding features F1 output by the previous time-reversible unit 301, and output the merged inter-frame coding features F2. The time-reversible unit 302 encodes the video frame T2 according to the inter-frame coding features F2, and outputs the coded code stream S2 of the video frame T2. Multiple time-reversible units process each video frame in the video frame sequence in turn to obtain the coded code stream S3 of the third video frame T3 and the inter-frame coding features F3 between the first three video frames, until the coded code stream SN of the Nth video frame TN. The Nth time-reversible unit 304 can process the inter-frame coding features F1 output by the N-1th time-reversible unit according to the inter-frame coding features F2. N-1 To determine the inter-frame coding feature F of the video frame TN N , and based on the inter-frame coding feature F NOutput the coded bit stream SN of the video frame TN. The Nth video frame TN is the last video frame in the video frame sequence, and the Nth time-series reversible unit 304 outputs the inter-frame coding feature F N It does not participate in the encoding of other video frames, so the inter-frame coding feature F N No output required.

[0046] Exemplarily, if the number of video frames included in the video frame sequence is greater than N, the video frame sequence can be divided into multiple parts, each part including N frames. The N frames of the first part are first input into the above-mentioned time-series reversible neural network, and the inter-frame coding feature F output by the last time-series reversible unit is N Need to participate in the encoding of the next part of N frames. Therefore, the last time-reversible unit can output F N , the inter-frame coding feature F N It can be used as the input of the first time-reversible unit of the next part N frames. That is, when processing the second part, F0 can be replaced by F N , continue encoding this second part.

[0047] Through N time-reversible units, the coded streams of N video frames can be obtained, and then the N coded streams are spliced ​​in sequence, that is, the coded stream S1, the coded stream S2, the coded stream S3 until the coded stream SN are spliced ​​together, and a complete coded stream of the video frame sequence, that is, the video coded stream, can be obtained.

[0048] For example, Figure 4 Another structural diagram of a sequential reversible unit is shown. Figure 4 As shown, the temporal reversible unit includes an inter-frame feature extraction layer 401, a feature extraction layer 402, an attention mechanism layer 403 and an ST-LSTM layer 404. When the i-th video frame is input into the temporal reversible unit, the inter-frame feature f1 of the i-th video frame is extracted by the inter-frame feature extraction layer 401, and the extracted inter-frame feature f1 is extracted again by the attention mechanism layer 403 to obtain the extracted inter-frame feature f2. The attention mechanism layer 403 can learn important features and increase the weight of important features, so that the important features can be transmitted in a longer time sequence. The inter-frame feature f2 output by the attention mechanism layer 403 is consistent with the inter-frame feature F of the first i-1 video frames learned in the ST-LSTM layer 404. i-1 Accumulate and obtain the inter-frame coding feature F of the temporal reversible unit i. The feature extraction layer 402 can extract intra-frame features of the video frame to obtain intra-frame features. The temporal reversible unit also includes a hidden layer 405 and a reversible neural network layer 406. Among them, the hidden layer 405 can determine the distribution information of the i-th video frame that obeys the first preset distribution based on the extracted intra-frame features, and the obtained distribution information is input into the reversible neural network layer 406 as an intermediate feature. The reversible neural network layer 406 can establish a forward mapping based on the intermediate features to obtain the encoded bitstream of the i-th video frame.

[0049] The reversible neural network layer can determine the high-frequency features in the video frame that obey the preset distribution, and the low-frequency features that do not obey the preset distribution, and then map the features into a coded bitstream according to the low-frequency features. Compared with the encoding method of the prior art, not only does it not require downsampling, encoder encoding and other processing, making the encoding process more efficient; it can also retain as much information of the original video frame as possible in the coded bitstream to avoid encoding distortion. The inter-frame coding features between video frames also retain the details between frames, improving the readiness of encoding.

[0050] Optionally, the inter-frame feature extraction layer 401 and the feature extraction layer 402 may be ResBlock. Res Block can improve the efficiency of model training and reduce the difficulty of deep model optimization. In addition, the inter-frame feature extraction layer 401 or the feature extraction layer 402 may also adopt other structures, such as Inception, Xception, etc., which is not particularly limited in this embodiment.

[0051] The above-mentioned time-series reversible neural network needs to be trained before application. The time-series reversible neural network can be trained based on sample videos and sample coding streams. The sample coding stream can be obtained by downsampling and encoding the sample video. Specifically, the training process includes the following:

[0052] A certain number of videos can be obtained in advance, and n frames can be obtained from each video to obtain a video frame sequence T corresponding to each video as a sample video. Then, the sample video T can be downsampled to obtain a downsampled video sequence D. The way to downsample the sample video can include various downsampling methods such as point drop downsampling, nearest neighbor, bilinear, region, bicubic, maximum pooling, average pooling, etc.; the downsampling ratio can be a value greater than or equal to 1, such as 2, 3, etc.; this embodiment is not limited to this.

[0053] After the sample video is downsampled, the downsampled video sequence D is encoded to obtain an encoded sample encoding stream. Thus, the high-bitrate sample video is converted into a low-bitrate sample encoding stream. Optionally, the downsampled video sequence D can be encoded by encoding methods such as x264 and x265.

[0054] Then, the sample video is input into a time-reversible neural network, which can extract features from the sample video and output a predicted bitstream in a forward direction when determining high-frequency features that obey a first preset distribution. The first loss between the predicted bitstream and the above-mentioned sample coding bitstream is determined, and the first loss is the forward loss of the time-reversible neural network. In addition, the predicted bitstream output by the time-reversible neural network is input into the time-reversible neural network again to obtain a predicted video outputted in reverse by the time-reversible neural network. The time-reversible neural network can reversely determine the corresponding low-frequency features based on the predicted bitstream, and then sample the high-frequency features on the second preset distribution, and merge the low-frequency features with the high-frequency features to obtain a predicted video. The second loss between the predicted video and the sample video is determined, and the time-reversible neural network is forwardly trained and reversely trained based on the first loss and the second loss.

[0055] Exemplarily, the first preset distribution and the second preset distribution are the same type of distribution, taking Gaussian distribution as an example. During forward training, the model can learn the features of the video frame that obey the Gaussian distribution and obtain the distribution features of the video frame, for example, the video frame obeys a Gaussian distribution with a mean of 1 and a variance of 0.5. During reverse training, a random Gaussian distribution can be input, for example, a Gaussian distribution with a mean of 1 and a variance of 0, and the model can sample on the input Gaussian distribution to obtain the features of the video frame.

[0056] Exemplarily, during forward training, the loss between the predicted bitstream and the sample coded bitstream of the sample video can be calculated by using the L1 loss function or the L2 loss function. During reverse training, the loss between the predicted video and the sample video can also be calculated by using the L1 loss function or the L2 loss function. Exemplarily, the loss between the predicted bitstream and the sample coded bitstream of the sample video is recorded as loss1; the loss between the predicted video and the sample video is recorded as loss2. In addition, loss1 or loss2 can also be calculated by other loss functions, such as the VGG loss function, etc., but the present embodiment is not limited thereto.

[0057] After calculating the loss1 and loss2 corresponding to the sample video, the total loss of the temporal reversible neural network can be calculated. The total loss of the temporal reversible neural network is the sum of the forward loss loss1 and the reverse loss loss2. The parameters of each layer of the temporal reversible neural network are adjusted by the total loss. Optionally, the method for adjusting the parameters may include a stochastic gradient descent method, a Newton method, an adaptive descent method, etc. The parameters of the temporal reversible neural network are repeatedly adjusted by the total loss corresponding to multiple sample videos. Until the total loss obtained by the forward loss loss1 and the reverse loss loss2 between the predicted bitstream output by the temporal reversible neural network for a certain sample video and the sample encoding bitstream corresponding to the sample video meets the preset conditions, the forward training and reverse training of the temporal reversible neural network are completed. The above-mentioned preset condition may be that the loss is less than a preset value, for example, the loss is less than 0.2, 0.3, etc., which is not specifically limited in this embodiment.

[0058] Finally, the time-series reversible neural network after training is saved, and the video frame sequence in step 100 is encoded using the time-series reversible neural network after training. By simultaneously performing forward training and reverse training on the reversible neural network, the training efficiency can be improved, so that the reversible neural network can encode in the forward direction and decode in the reverse direction, thereby realizing the encoding and decoding of the video.

[0059] In addition, the time-series reversible neural network can learn the distribution information of video frames through the intermediate hidden layer during forward transmission, and can restore the corresponding video frames through this distribution information during reverse transmission. Therefore, there is almost no loss in the encoding and decoding process, which can solve the problem of video distortion caused by encoding and decoding.

[0060] The video decoding process provided by this embodiment is described below. Figure 1 As shown, the decoding process may include the following steps:

[0061] Step 300: Input the video encoding code stream and the second preset distribution into a time-series reversible neural network to obtain a second video frame sequence that is a reverse output of the time-series reversible neural network; wherein the time-series reversible neural network samples the second preset distribution to obtain high-frequency features, and combines the high-frequency features with the video encoding code stream to determine the second video frame sequence.

[0062] The video encoding stream to be decoded may be video data stored in a disk or cache, or may be video data transmitted from other electronic devices.

[0063] Exemplarily, the time-series reversible neural network may include multiple sequentially connected time-series reversible units. Each time-series reversible unit may process a segment of the video encoding stream to determine the corresponding video frame, and the multiple time-series reversible units may output multiple video frames respectively, and the multiple video frames are the decoded video.

[0064] Specifically, the video encoding stream is divided into N encoding streams of the same size according to a preset size. The preset size is the size of the stream that each time-reversible unit can output. The size of the stream output by each time-reversible unit can be the same. According to the size of the stream output by each time-reversible unit, the video encoding stream is divided into multiple encoding streams of the same size. Then the N encoding streams are arranged in sequence and input into the N time-reversible units. At the same time, the second preset distribution is also input into the N time-reversible units to obtain the decoded frames output in reverse by each time-reversible unit. The N decoded frames output in reverse by the N time-reversible units are merged to obtain a decoded video frame sequence, that is, a second video frame sequence.

[0065] In this implementation, the video is decoded by the time-series reversible unit, each time-series reversible unit can be responsible for decoding a section of the coded bitstream, and multiple time-series reversible units can perform decoding simultaneously, which can improve decoding efficiency.

[0066] The structure and training process of the timing reversible unit can refer to the above implementation mode, such as Figure 2 , Figure 3 , Figure 4 etc., which will not be described in detail here. Figure 2 As an example, the video coding stream is input into the temporal reversible unit, which is composed of a spatiotemporal long short-term memory network and a reversible neural network. The input coding stream is mapped into image features through the reversible neural network and merged with the high-frequency features sampled by the reversible neural network on the second preset distribution. The inter-frame coding features between the video coding streams can be extracted through the spatiotemporal long short-term memory network, and the merged image features are merged with the inter-frame coding features again to obtain the final image, i.e., the decoded frame.

[0067] The inter-frame coding feature of a coded stream may be a feature between other coded streams before the coded stream. For example, a certain video data includes two coded streams, and there is no other video data before the first coded stream, so the inter-frame coding feature of the first coded stream may be 0. For the second coded stream, the inter-frame coding feature may be the inter-frame coding feature of the first coded stream. The inter-frame coding feature corresponding to the first coded stream at the start may be determined in advance, for example, the inter-frame coding feature may be 0.

[0068] The time-reversible unit can extract inter-frame coding features between video coding streams, and restore the video corresponding to the video coding stream based on the inter-frame coding features, thereby improving the continuity of the video.

[0069] Specifically, the first temporal reversible unit can output the video frame corresponding to the first segment of the coded code stream and the inter-frame coding features between the first segment of the coded code stream and the previous coded code stream according to the input first segment of the coded code stream and the input inter-frame coding features. If there is no other coded code stream before the first segment of the coded code stream, the inter-frame coding features input by the first temporal reversible unit can be 0. It should be understood that the temporal reversible units are arranged in reverse order during decoding, that is, the first temporal reversible unit is the last temporal reversible unit during encoding.

[0070] In summary, according to the reverse order of the N layers of sequentially connected time-series reversible units, the jth coded bitstream and the inter-frame coding features output by the j-1th time-series reversible unit are input into the jth time-series reversible unit, and the jth decoded frame corresponding to the jth coded bitstream and the inter-frame coding features between the first j coded bitstreams can be obtained; 1<j≤N. j is the sequence number of the reverse order of the time-series reversible unit, and the inter-frame coding features input to the first time-series reversible unit in the reverse order are 0.

[0071] refer to Figure 4 , after the jth coded code stream is input into the jth temporal reversible unit, the reversible neural network layer 406 in the jth temporal reversible unit can map the jth coded code stream into the features of the image. The features are input into the hidden layer 405, and the hidden layer 405 can sample according to the second preset distribution, and merge the sampled features with the features input by the reversible neural network layer 406 as the intermediate layer features. Then, the intermediate layer features can be extracted again through the feature extraction layer 402 to obtain intra-frame features. The intra-frame features are weighted by the attention mechanism layer 403, and the weighted results are merged with the inter-frame coding features output by the i-1th temporal reversible unit of the ST-LSTM layer to obtain the merged result. The merged result contains the features between frames, and finally the features between frames are extracted again by the inter-frame feature extraction layer 401 to obtain the final image, that is, the decoded frame.

[0072] Each time-series reversible unit can not only decode based on the video coding stream, but also decode based on the inter-frame coding features, which can restore more details in the encoded video frame and improve the video quality. The long short-term memory network can extract the inter-frame coding features between each segment of the coding stream to improve the continuity of the video; the reversible neural network can restore the high-frequency features corresponding to each segment of the coding stream, and obtain the decoded frame based on the high-frequency features, thereby reducing the loss of information and reducing distortion.

[0073] Furthermore, the video coding and decoding method provided in the embodiment of the present application can be executed by a video codec. In the embodiment of the present application, the video codec executing the video coding and decoding method is taken as an example to illustrate the video codec provided in the embodiment of the present application.

[0074] Figure 5 FIG. 4 shows a schematic diagram of the structure of the video codec provided in this embodiment. Figure 5 As shown, the video codec 500 may include a first acquisition module 501, a first encoding module 502 and a first decoding module 503. Specifically, the first acquisition module 501 may be used to acquire a first video frame sequence to be encoded. The first encoding module 502 may be used to input the first video frame sequence into a time-series reversible neural network to obtain a video encoding stream corresponding to a first preset distribution outputted in a forward direction of the time-series reversible neural network. The first preset distribution includes the distribution of high-frequency features of video frames in the first video frame sequence, and the video encoding stream includes the encoding of low-frequency features of video frames in the first video frame sequence. The first decoding module 503 may be used to input the video encoding stream and the second preset distribution into the time-series reversible neural network in the case of decoding, to obtain a second video frame sequence outputted in reverse direction by the time-series reversible neural network; wherein the time-series reversible neural network samples the second preset distribution to obtain high-frequency features, and determines the second video frame sequence in combination with the high-frequency features and the video encoding stream.

[0075] In an exemplary embodiment, the time-reversible neural network includes N layers of sequentially connected time-reversible units, N ≥ 1; the first encoding module 502 may specifically include: a first input module, used to input N video frames in a first video frame sequence into N time-reversible units respectively, to obtain a coded code stream corresponding to a first preset distribution output by each time-reversible unit; a first splicing module, used to splice N segments of coded code streams output forward by N time-reversible units to obtain a video coded code stream.

[0076] In an exemplary embodiment, the first input module is configured to: input the inter-frame coding features of the i-th video frame in the video frame sequence and the i-1-th temporal reversible unit forward output into the i-th temporal reversible unit in the forward order of the N layers of sequentially connected temporal reversible units, and obtain the coded bit stream of the i-th video frame corresponding to the first preset distribution output by the i-th temporal reversible unit forward, and the inter-frame coding features between the first i video frames. Wherein, 1<i≤N, the inter-frame coding features input by the first temporal reversible unit in the forward order are 0.

[0077] In an exemplary embodiment, the video codec 500 also includes: a second acquisition module, which can be used to acquire a sample video, and downsample and encode the sample video to obtain a sample encoding code stream of the sample video; a first prediction module, which can be used to input the sample video into a time-series reversible neural network to obtain a predicted code stream of the forward output of the time-series reversible neural network; when the high-frequency feature encoding in the sample video conforms to the first preset distribution, the time-series reversible neural network outputs a predicted code stream; a first determination module, which can be used to determine a first loss between the predicted code stream and the sample encoding code stream; a second prediction module, which is specifically used to input the predicted code stream and the second preset distribution into the time-series reversible neural network to obtain a predicted video of the reverse output of the time-series reversible neural network; a second determination module, which can be used to determine a second loss between the predicted video and the sample video; a first training module, which can be used to perform forward training and reverse training on the time-series reversible neural network based on the first loss and the second loss.

[0078] In an exemplary embodiment, the first decoding module 503 may specifically include: a first division module, used to divide the video encoding stream into N encoding streams of the same size according to a preset size; a second input module, used to input the N encoding streams and the second preset distribution into N time-reversible units respectively, to obtain decoded frames output in reverse by each time-reversible unit; a first merging module, used to merge the N decoded frames output in reverse by the N time-reversible units into a second video frame sequence.

[0079] In an exemplary embodiment, the second input module is specifically configured as follows: in the reverse order of the sequentially connected N layers of sequentially connected sequentially reversible units, the jth segment of the coded code stream, the second preset distribution and the inter-frame coding features output by the j-1th sequentially reversible unit are input into the jth sequentially reversible unit, and the jth decoded frame corresponding to the jth segment of the coded code stream and the inter-frame coding features of the first j segments of the coded code stream are obtained. Wherein, 1<j≤N, the inter-frame coding features input by the first sequentially reversible unit in the reverse order are 0.

[0080] The video codec in this embodiment uses a spatiotemporal long short-term memory network and a reversible neural network to construct a new model structure, namely a temporal reversible neural network. When the temporal reversible neural network is used to encode or decode a video, the spatiotemporal long short-term memory network can extract inter-frame coding features to improve the continuity of the video. The reversible neural network can learn the distribution of features within a frame, and encode according to the distribution information, which can avoid the distortion of the video caused by encoding and decoding and improve the quality of the video picture.

[0081] The video codec in the embodiment of the present application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0082] The video codec in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0083] The video codec provided in the embodiment of the present application can achieve the following Figures 1 to 4 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0084] Alternatively, if Figure 6 As shown, an embodiment of the present application also provides an electronic device 600, including a processor 601 and a memory 602, wherein the memory 602 stores a program or instruction that can be executed on the processor 601, and when the program or instruction is executed by the processor 601, the various steps of the embodiment of the above-mentioned video encoding and decoding method are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0085] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0086] Figure 7 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0087] The electronic device 700 includes but is not limited to: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, and a processor 710.

[0088] Those skilled in the art will appreciate that the electronic device 700 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 710 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0089] Processor 710 is used to execute: obtaining a first video frame sequence to be encoded; inputting the first video frame sequence into a time-reversible neural network to obtain a video coding stream corresponding to a first preset distribution output by the time-reversible neural network; wherein the first preset distribution includes the distribution of high-frequency features of video frames in the first video frame sequence, and the video coding stream includes the encoding of low-frequency features of video frames in the first video frame sequence; in the case of decoding, inputting the video coding stream and the second preset distribution into the time-reversible neural network to obtain a second video frame sequence output by the time-reversible neural network in reverse; wherein the time-reversible neural network samples the second preset distribution to obtain high-frequency features, and determines the second video frame sequence in combination with the high-frequency features and the video coding stream.

[0090] It should be understood that in the embodiment of the present application, the input unit 704 may include a graphics processor (Graphics Processing Unit, GPU) 7041 and a microphone 7042, and the graphics processor 7041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 706 may include a display panel 7061, and the display panel 7061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 707 includes a touch panel 7071 and at least one of other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include two parts: a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0091] The memory 709 can be used to store software programs and various data. The memory 709 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 709 may include a volatile memory or a non-volatile memory, or the memory 709 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 709 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0092] The processor 710 may include one or more processing units; optionally, the processor 710 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 710.

[0093] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned video encoding and decoding method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0094] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0095] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned video encoding and decoding method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0096] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0097] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned video encoding and decoding method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0098] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0099] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0100] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A video encoding and decoding method, characterized in that: include: Obtaining a first video frame sequence to be encoded; Inputting the first video frame sequence into a time-series reversible neural network, obtaining a video encoding stream corresponding to a first preset distribution outputted by the time-series reversible neural network; wherein the first preset distribution includes a distribution of high-frequency features of video frames in the first video frame sequence, and the video encoding stream includes encodings of low-frequency features of video frames in the first video frame sequence; In the case of decoding, the video encoding code stream and the second preset distribution are input into the time-series reversible neural network to obtain a second video frame sequence reversely output by the time-series reversible neural network; wherein the time-series reversible neural network samples the second preset distribution to obtain high-frequency features, and combines the high-frequency features with the video encoding code stream to determine the second video frame sequence; The time-series reversible neural network comprises N layers of sequentially connected time-series reversible units, where N is a positive integer, and the time-series reversible units comprise: a spatiotemporal long short-term memory network and a reversible neural network; the spatiotemporal long short-term memory network is used to extract inter-frame features of video frames, and the reversible neural network is used to learn the distribution of high-frequency features within the frames of the video frames; The step of inputting the first video frame sequence into a time-series reversible neural network to obtain a video encoding bitstream corresponding to a first preset distribution outputted by the time-series reversible neural network in a forward direction includes: Inputting N video frames in the first video frame sequence into N time-series reversible units respectively, and obtaining a coded bit stream corresponding to the first preset distribution outputted in a forward direction by each of the time-series reversible units; The N segments of coded bit streams forwardly outputted by the N sequential reversible units are spliced ​​to obtain the video coded bit stream.

2. The video encoding and decoding method according to claim 1, characterized in that: Inputting N video frames in the first video frame sequence into N time-series reversible units respectively, and obtaining a coded bit stream corresponding to the first preset distribution outputted in a forward direction by each of the time-series reversible units, comprising: According to the forward order of the N layers of sequentially connected temporal reversible units, the i-th video frame in the video frame sequence and the inter-frame coding features forwardly output by the i-1-th temporal reversible unit are input into the i-th temporal reversible unit to obtain the coded bit stream of the i-th video frame corresponding to the first preset distribution forwardly output by the i-th temporal reversible unit and the inter-frame coding features between the first i video frames; Among them, 1<i≤N, the inter-frame coding feature input by the first time-reversible unit in the forward order is 0.

3. The video encoding and decoding method according to claim 1, characterized in that: The method further comprises: Acquire a sample video, and perform downsampling and encoding processing on the sample video to obtain a sample encoding bitstream of the sample video; Inputting the sample video into the time-series reversible neural network to obtain a predicted bit stream of the forward output of the time-series reversible neural network; The time-series reversible neural network outputs the predicted bitstream when the high-frequency features in the sample video conform to the first preset distribution; Determining a first loss between the predicted code stream and the sample encoded code stream; Inputting the predicted bitstream and the second preset distribution into the time-series reversible neural network to obtain a predicted video outputted inversely by the time-series reversible neural network; determining a second loss between the predicted video and the sample video; The temporal reversible neural network is forward trained and backward trained based on the first loss and the second loss.

4. The video encoding and decoding method according to claim 1, characterized in that: The step of inputting the video encoding code stream and the second preset distribution into the time-series reversible neural network to obtain a second video frame sequence reversely output by the time-series reversible neural network includes: Dividing the video encoding stream into N encoding streams of the same size according to a preset size; Input the N coded code streams and the second preset distribution into the N time-series reversible units respectively, to obtain a decoded frame reversely outputted by each time-series reversible unit; The N decoded frames reversely outputted by the N time-reversible units are merged into a second video frame sequence.

5. The video encoding and decoding method according to claim 4, characterized in that: The step of inputting the N coded code streams and the second preset distribution into the N time-series reversible units respectively to obtain a decoded frame reversely outputted by each time-series reversible unit comprises: According to the reverse order of the N layers of sequentially connected time-series reversible units, the j-th coded bitstream, the second preset distribution and the inter-frame coding features output by the j-1-th time-series reversible unit are input into the j-th time-series reversible unit to obtain the j-th decoded frame corresponding to the j-th coded bitstream and the inter-frame coding features of the previous j-th coded bitstreams; Among them, 1<j≤N, the inter-frame coding feature input by the first time-reversible unit in the reverse order is 0.

6. A video codec, characterized in that: include: A first acquisition module, used to acquire a first video frame sequence to be encoded; A first encoding module, configured to input the first video frame sequence into a time-series reversible neural network, and obtain a video encoding stream corresponding to a first preset distribution outputted by the time-series reversible neural network; wherein the first preset distribution includes a distribution of high-frequency features of video frames in the first video frame sequence, and the video encoding stream includes encoding of low-frequency features of video frames in the first video frame sequence; A first decoding module is used for, in the case of decoding, inputting the video coding stream and the second preset distribution into the time-series reversible neural network to obtain a second video frame sequence reversely output by the time-series reversible neural network; wherein the time-series reversible neural network samples the second preset distribution to obtain high-frequency features, and determines the video frame sequence by combining the high-frequency features with the video coding stream; The temporal reversible neural network comprises N layers of sequentially connected temporal reversible units, where N is a positive integer; the temporal reversible units comprise: a spatiotemporal long short-term memory network and a reversible neural network; the spatiotemporal long short-term memory network is used to extract inter-frame features of video frames, and the reversible neural network is used to learn the distribution of high-frequency features within the frames of the video frames; the first encoding module comprises: A first input module, used for inputting N video frames in the first video frame sequence into N time-series reversible units respectively, to obtain a coded bit stream corresponding to the first preset distribution outputted in a forward direction by each of the time-series reversible units; The first splicing module is used to splice N coded code streams forwardly outputted by N sequential reversible units to obtain the video coded code stream.

7. The video codec according to claim 6, characterized in that: The first input module is configured as: According to the forward order of the N layers of sequentially connected temporal reversible units, the i-th video frame in the video frame sequence and the inter-frame coding features forwardly output by the i-1-th temporal reversible unit are input into the i-th temporal reversible unit to obtain the coded bit stream of the i-th video frame corresponding to the first preset distribution forwardly output by the i-th temporal reversible unit and the inter-frame coding features between the first i video frames; Among them, 1<i≤N, the inter-frame coding feature input by the first time-reversible unit in the forward order is 0.

8. The video codec according to claim 6, characterized in that The video codec also includes: A second acquisition module is used to acquire a sample video, and perform downsampling and encoding processing on the sample video to obtain a sample encoding bit stream of the sample video; A first prediction module, used for inputting the sample video into the time-series reversible neural network to obtain a predicted bit stream of the forward output of the time-series reversible neural network; The time-series reversible neural network outputs the predicted bitstream when the high-frequency feature encoding in the sample video conforms to the first preset distribution; A first determining module, configured to determine a first loss between the predicted bitstream and the sample coded bitstream; A second prediction module, used for inputting the predicted bitstream and the second preset distribution into the time-series reversible neural network to obtain a predicted video outputted inversely by the time-series reversible neural network; A second determination module, used to determine a second loss between the predicted video and the sample video; The first training module is used to perform forward training and reverse training on the temporal reversible neural network based on the first loss and the second loss.

9. The video codec according to claim 6, characterized in that: The first decoding module comprises: A first division module, used for dividing the video coding stream into N coding streams of the same size according to a preset size; A second input module, used for inputting the N coded code streams and the second preset distribution into the N time-series reversible units respectively, to obtain a decoded frame reversely outputted by each time-series reversible unit; The first merging module is used to merge the N decoded frames reversely output by the N time-reversible units into a second video frame sequence.

10. The video codec according to claim 9, characterized in that: The second input module is configured as: According to the reverse order of the N layers of sequentially connected time-series reversible units, the j-th coded bitstream, the second preset distribution and the inter-frame coding features output by the j-1-th time-series reversible unit are input into the j-th time-series reversible unit to obtain the j-th decoded frame corresponding to the j-th coded bitstream and the inter-frame coding features of the previous j-th coded bitstreams; Among them, 1<j≤N, the inter-frame coding feature input by the first time-reversible unit in the reverse order is 0.

11. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the video encoding and decoding method according to any one of claims 1 to 5 is implemented.

12. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the video encoding and decoding method according to any one of claims 1 to 5 is implemented.

Images