Video coding and decoding method based on novel algorithm

Abstract

The invention provides a video coding and decoding method based on a novel algorithm, which comprises the following steps: after a coding end obtains an original video, processing the original video to obtain video feature data; acquiring network state data in real time, and determining coded data according to the network state data; encoding the video feature data according to the encoded data to obtain compressed video data; obtaining transmission data according to the compressed video data and the coded data; and outputting the transmission data to a decoding end for decoding based on the adaptive bit rate streaming media technology and the network state data to obtain a reconstructed video.

Description

Technical Field

[0001] The present invention relates to the technical field of video encoding and decoding, and in particular to a video encoding and decoding method based on a novel algorithm. Background Art

[0002] Today, when digital images are widely used, whether in digital photography, e-commerce or telemedicine, efficient storage and transmission of image data are particularly important. Since high-quality images take up a lot of storage space and transmission bandwidth, image compression technology has become a key means to reduce this demand. Image compression technology can reduce file size while ensuring image quality, making image transmission faster and greatly improving user experience.

[0003] Traditional video encoding and decoding methods mainly rely on manually designed algorithms and models, which cannot fully capture the key information in the video, resulting in low encoding efficiency or impaired video quality. In addition, traditional methods usually lack adaptability and cannot dynamically adjust encoding parameters and algorithms according to the characteristics of the video content and network conditions, making it difficult to meet diverse application needs. Summary of the invention

[0004] The present invention provides a video encoding and decoding method based on a novel algorithm, which is used to solve the problem that the traditional method cannot dynamically adjust the encoding parameters and the algorithm according to the characteristics of the video content and the network status.

[0005] According to an embodiment of the present invention, a video encoding and decoding method based on a novel algorithm comprises the following steps:

[0006] After the encoding end obtains the original video, it processes the original video to obtain video feature data;

[0007] Acquire network status data in real time and determine encoding data according to the network status data;

[0008] Encoding the video feature data according to the encoding data to obtain compressed video data;

[0009] Obtaining transmission data according to the compressed video data and the encoded data;

[0010] Based on adaptive bit rate streaming technology and network status data output, the data is transmitted to the decoding end for decoding to obtain the reconstructed video.

[0011] As an embodiment of the present invention, the original video is processed to obtain video feature data, including:

[0012] Based on human vision and machine vision, the original video is extracted and converted to obtain video feature data.

[0013] As an embodiment of the present invention, determining the coded data according to the network status data includes:

[0014] Based on AI technology, the coding data is determined through network status data; wherein the coding data includes: coding parameters and coding algorithms.

[0015] As an embodiment of the present invention, based on AI technology, the coded data is determined by network status data, including:

[0016] The network status data is input into the trained coding data determination model for processing to obtain the coding data.

[0017] As an embodiment of the present invention, the trained coding data determination model is a trained deep learning model or a machine learning algorithm model.

[0018] As an embodiment of the present invention, encoding video feature data according to encoding data to obtain compressed video data includes:

[0019] Intra-frame prediction and inter-frame prediction are performed on the video feature data to obtain compressed video data.

[0020] As an embodiment of the present invention, based on the adaptive bit rate streaming media technology and the network status data output, the transmission data is sent to the decoding end for decoding to obtain the reconstructed video, including:

[0021] After the data is transmitted to the decoding end based on the adaptive bit rate streaming technology and the network status data output, the decoding operation is performed according to the encoding parameters, encoding algorithm and compressed video data to obtain the reconstructed video.

[0022] As an embodiment of the present invention, the decoding operation of the transmission data at the decoding end is an inverse operation of encoding the video feature data according to the encoding data.

[0023] As an embodiment of the present invention, the encoding end includes:

[0024] Encoder and entropy coding module; wherein, the encoder and entropy coding module are both used to encode the original video.

[0025] As an embodiment of the present invention, the encoding end further includes:

[0026] Codeword estimation module and rate-distortion optimization module; wherein, the codeword estimation module is used to remove statistical redundancy in the encoding operation, and the rate-distortion optimization module is used to obtain network status data and determine the encoded data, and output the transmission data to the decoding end.

[0027] As an embodiment of the present invention, a quantization module is provided at an output port of the encoder, and the quantization module is used to reduce output transmission delay of the encoder.

[0028] As an embodiment of the present invention, the decoding end includes:

[0029] Dequantization module and decoder;

[0030] The decoder is used to perform decoding operations according to encoding parameters, encoding algorithms and compressed video data to obtain reconstructed video.

[0031] On the other hand, the present invention further provides a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0032] Compared with the prior art, the present invention has the following beneficial effects: by integrating the characteristics of human vision and machine vision, it is possible to more accurately capture the key information of the video content, reduce the processing of redundant data, and thus improve the encoding efficiency; and, by utilizing target detection and feature extraction technology, it is possible to deeply understand and analyze the video content, and provide more valuable feature data for subsequent encoding optimization; at the same time, combined with adaptive bitrate streaming media technology, it is possible to dynamically adjust the video bitrate according to the real-time network conditions to ensure a smooth viewing experience in different network environments; finally, in the decoding process, the inverse operation corresponding to the encoding process is adopted, and the neural network and the encoded data are used for reconstruction, so that a reconstructed video that is highly similar to the original video content can be restored, thereby achieving high-fidelity reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of a flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0035] like Figure 1 As shown, the embodiment of the present invention proposes a video encoding and decoding method based on a novel algorithm, comprising the following steps:

[0036] After the encoding end obtains the original video, it processes the original video to obtain video feature data;

[0037] Acquire network status data in real time and determine encoding data according to the network status data;

[0038] Encoding the video feature data according to the encoding data to obtain compressed video data;

[0039] Obtaining transmission data according to the compressed video data and the encoded data;

[0040] Based on adaptive bit rate streaming technology and network status data output, the data is transmitted to the decoding end for decoding to obtain the reconstructed video;

[0041] The original video is processed to obtain video feature data, including:

[0042] Extract and transform the original video features based on human vision and machine vision to obtain video feature data;

[0043] The working principle of the above technical solution is as follows: in actual use, color management and motion estimation technology are used to simulate human visual perception, and at the same time, combined with machine vision, the video content of the original video is deeply analyzed through target detection and feature extraction technology to obtain video feature data; at the same time, in the encoding process, the network status data of the current network is obtained in real time, and the encoding data is adjusted in real time according to the network status data, and then the video feature data is encoded according to the encoding data to obtain compressed video data; then the transmission data is obtained according to the compressed video data and the encoding data, and the transmission data is output to the decoding end for decoding based on the adaptive bit rate streaming technology and network status data to obtain the reconstructed video; wherein, the video feature data includes but is not limited to color distribution, motion vector, texture information, etc., to ensure that the subsequent encoding process can make full use of this information for efficient compression processing and encoding optimization;

[0044] The beneficial effects of the above technical solution are as follows: Through the above technical solution, by integrating the characteristics of human vision and machine vision, it is possible to more accurately capture the key information of the video content, reduce the processing of redundant data, and thus improve the encoding efficiency; and, by utilizing target detection and feature extraction technology, it is possible to deeply understand and analyze the video content, and provide more valuable feature data for subsequent encoding optimization; at the same time, combined with adaptive bitrate streaming technology, it is possible to dynamically adjust the video bitrate according to the real-time network conditions to ensure a smooth viewing experience in different network environments; finally, in the decoding process, the inverse operation corresponding to the encoding process is adopted, and the neural network and the encoded data are used for reconstruction, so that a reconstructed video that is highly similar to the original video content can be restored, achieving high-fidelity reconstruction.

[0045] In one embodiment, determining the encoding parameters and the encoding algorithm according to the network status data includes:

[0046] Based on AI technology, the coding data is determined through the network status data; wherein the coding data includes: coding parameters and coding algorithms;

[0047] Based on adaptive bitrate streaming technology and network status data output, the data is transmitted to the decoding end for decoding to obtain the reconstructed video, including:

[0048] After the data is transmitted to the decoding end based on the adaptive bit rate streaming technology and network status data output, the decoding operation is performed according to the encoding parameters, encoding algorithm and compressed video data to obtain the reconstructed video;

[0049] The decoding operation of the transmission data at the decoding end is the inverse operation of encoding the video feature data according to the encoded data;

[0050] Working principle and beneficial effects of the above technical solution: Through the above technical solution, advanced AI technologies, such as deep learning and machine learning algorithms, are introduced to intelligently adjust encoding parameters and encoding algorithms according to the complexity and dynamic characteristics of video content; at the same time, combined with adaptive bitrate streaming technology, the network environment and user device performance are monitored in real time to obtain network status data, thereby dynamically adjusting the resolution, frame rate and bit rate of the video stream to ensure that high-quality viewing experience can be provided under various network conditions; further, through machine learning algorithms, the complexity and dynamic changes of video content, such as scene switching frequency, object movement speed, etc., are deeply analyzed, and the encoding strategy is dynamically adjusted, such as adjusting quantization parameters, selecting more appropriate encoding modes, etc., to ensure that efficient compression ratios and excellent image quality can be achieved in different video scenes; adaptive bitrate streaming technology not only considers the user's network conditions, such as bandwidth, delay, etc., but also fully considers the decoding capabilities and display effects of the user's device, and adjusts the resolution, frame rate and bit rate of the video stream in real time to ensure that a smooth and non-stuttering viewing experience can be provided under various devices and network environments;

[0051] At the same time, the decoding process uses the inverse operation that is completely corresponding to the encoding process, and uses the same neural network structure and encoding parameter information to accurately reconstruct the compressed video data; through the powerful capabilities of deep learning and neural network technology, it is possible to restore video quality that is highly similar to the original video content and achieve high-fidelity video reconstruction; in the decoding process, the inverse operation corresponding to the encoding process is used, and reconstruction is performed using neural networks and encoding parameter information. This method can restore video quality that is highly similar to the original video content and achieve high-fidelity reconstruction; high-fidelity reconstruction not only improves the user's viewing experience, but also provides a more reliable data foundation for subsequent video processing and analysis.

[0052] In one embodiment, encoding the video feature data according to the encoding data to obtain compressed video data includes:

[0053] Performing intra-frame prediction and inter-frame prediction on the video feature data respectively to obtain compressed video data;

[0054] The working principle and beneficial effects of the above technical solution are as follows: in the process of video encoding, a neural network model is deeply integrated in the video encoding framework to efficiently compress video information; specifically, intra-frame prediction uses a convolutional neural network (CNN) or a multi-layer perceptron (MLP) to capture the complex relationship between pixels within a frame; inter-frame prediction uses a recurrent neural network (RNN) or CNN to accurately predict the motion trajectory of an object and achieve efficient motion compensation; in addition, by training a specific CNN model to optimize the entropy coding process and accurately predict the probability distribution of the coding mode, information redundancy in the coding process is significantly reduced and compression efficiency is improved; further, intra-frame prediction and inter-frame prediction are two parallel and complementary processes. Intra-frame prediction uses CNN or MLP to deeply explore the spatial correlation between pixels within a frame, while inter-frame prediction uses RNN or CNN to capture the temporal dependency between frames. The combination of the two can significantly improve the overall efficiency and quality of video encoding. Among them, making predictions within a picture is called intra-frame prediction; predicting the motion trajectory of a series of pictures, such as a video of a dunk action, belongs to inter-frame prediction; the frame used as a reference is called I-frame, which is a key frame. It has the largest amount of information and can only be compressed within the frame, usually with a very low compression rate; and those images that are subsequently predicted by referring to the information of the I-frame to obtain the difference do not store the original pixel values ​​at all, but the residuals of the original images, which are called predicted frames; predicted frames sometimes use a mixture of intra-frame prediction and inter-frame prediction, depending on which algorithm the block is more suitable for; the current frame image predicted based on the previous image is called a P frame; the current frame image obtained by combining the previous image and the next image for bidirectional prediction calculation is called a B frame; a group of images based on a key I-frame image plus a series of corresponding predicted images such as B frames and P frames is called a GOP;

[0055] The present invention uses deep learning of neural networks to automatically optimize coding parameters and algorithms. Compared with traditional methods, it can more finely control compression ratio and distortion and improve compression performance. In addition, by optimizing the entropy coding process using convolutional neural networks, it can more accurately predict the probability distribution of coding modes, reduce information redundancy in the coding process, and further improve compression efficiency.

[0056] In one embodiment, the encoding end includes:

[0057] Encoder, quantization module, entropy coding module, codeword estimation module and rate-distortion optimization module;

[0058] Among them, the encoder and entropy coding modules are both used to encode the original video, the codeword estimation module is used to remove statistical redundancy in the encoding operation, and the rate-distortion optimization module is used to obtain network status data and determine the encoded data, and output the transmission data to the decoding end.

[0059] The decoding end includes:

[0060] Dequantization module and decoder;

[0061] The decoder is used to perform decoding operations according to the encoding parameters, the encoding algorithm and the compressed video data to obtain a reconstructed video;

[0062] The working principle and beneficial effects of the above technical solution: the video coding framework includes an encoding end and a decoding end. The encoding end includes: an encoder, a quantization module, an entropy coding module, a codeword estimation module and a rate-distortion optimization module; the decoding end includes: an inverse quantization module and a decoder; the encoder is used to convert the image into compressed features, and the decoder is used to restore the original image from the compressed features; the encoder and the decoder can be designed and constructed using modules such as convolution, pooling, and nonlinearity.

[0063] In one embodiment, the present invention also provides example comparison data between the present invention and the traditional method, as shown below:

[0064] The present invention outputs a high-definition video with a resolution of 1920×1080, 24-bit color, and 30 frames per second, and the video bit rate for one second reaches 1.5Gbps. After the video encoding and decoding method based on the new algorithm provided by the present invention, the video bit rate is compressed to 6Mbps, which is 1 / 250 of the source video data volume. The specific process is as follows:

[0065] First, the video is split into N images according to the time series, and then a single image is divided into small blocks of 10*10 pixels, and then compressed.

[0066] The first step of compression is prediction processing. For each block of an image, the value of the next pixel point in the specified direction is predicted using a prediction formula based on the values ​​of four adjacent pixels, so as to obtain the predicted pixel value of the point to construct a complete image.

[0067] The prediction formula used above is:

[0068]

[0069] Where x and y are the position coordinates in the current pixel block respectively; Res|pos| is the reference pixel value. The calculation method of pos will also change according to the different video angle modes. ω is the weighting factor, which is calculated according to the positions of x and y.

[0070] Furthermore, the calculation process of pos and ω is as follows:

[0071] pos=(y*offset[M])>>5

[0072] ω=(y*offset[M])&31

[0073] Where offset[M] is the offset value predefined according to the video angle mode M; M is the number of the video angle mode; the numbers 32, 5, and 31 are all scaling factors used to balance the contribution between the predicted value and the reference pixel value. This factor ensures that the calculation result of the predicted value is within a reasonable range and can be processed by the quantizer;

[0074] If there are two consecutive moving images, one image is not changed and the original pixel value is saved. Then, based on the value of this image, the other image is predicted using a formula. That is, the value of a certain pixel in the first image is calculated by the prediction formula to predict the pixel value of the specified position in the second image. This process is repeated to obtain a complete predicted image.

[0075] Comparative Example: In order to compare with the video encoding and decoding method based on the new algorithm in Example 1, we use the existing H.264 / AVC video encoding standard for encoding. Taking the output of a high-definition video with a resolution of 1920×1080, 24-bit color, and 30 frames per second as an example, the original bit rate of one second of video is also 1.5Gbps;

[0076] The specific process is as follows: split the video into N image frames according to the time sequence, and each image frame is subsequently processed in units of 10*10 pixels;

[0077] In the H.264 standard, block matching is used for motion estimation and compensation. For each image frame, the best matching block in the previous frame is searched to obtain the motion vector;

[0078] Use the motion vector to predict the current macroblock and get the predicted block. Then calculate the residual between the current macroblock and the predicted block and encode the residual;

[0079] During the encoding process, the residual block is transformed using DCT (discrete cosine transform) to remove spatial redundancy. The transform coefficients are then quantized and further entropy coded to compress the data volume;

[0080] The final output video has a bit rate of 10Mbps, which is 1 / 150 of the original video data volume.

[0081] In summary, the data results of Example 1 and Comparative Example 1 are compared:

[0082] The method of the first embodiment compresses the video bit rate to 1 / 250 of the source video data volume, that is, 6 Mbps;

[0083] The method of comparative example 1 compresses the video bit rate to 1 / 150 of the source video data volume, that is, 10Mbps;

[0084] Obviously, the method of Example 1 is superior to that of Comparative Example 1 in terms of compression efficiency. Generally, the video encoding and decoding method based on deep learning can achieve efficient compression while retaining more detailed information, thus providing better video quality.

[0085] The method of the first embodiment combines the characteristics of human vision and machine vision, and uses a neural network for efficient compression processing and coding optimization; the method of the first embodiment uses a traditional block matching method for motion estimation and compensation, and uses technologies such as DCT transformation and entropy coding; there are significant differences between the two in technology, and the method of the first embodiment is more innovative and advanced;

[0086] When the present invention is used, a video coding framework is constructed based on human vision and machine vision, and color management, motion estimation and compensation technology are used to simulate the intuitive perception of human vision on video content. At the same time, target detection and feature extraction technology are used to deeply understand and analyze video content. The features of video information are extracted and converted through this framework to obtain video feature data, which provides a basis for subsequent coding processing;

[0087] A computer program, wherein when the processor executes the computer program, any step in the above method is implemented.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A video encoding and decoding method based on a new algorithm, characterized in that: The following steps are involved: After the encoding end obtains the original video, it processes the original video to obtain video feature data; Acquire network status data in real time and determine encoding data according to the network status data; Encoding the video feature data according to the encoding data to obtain compressed video data; Obtaining transmission data according to the compressed video data and the encoded data; Based on adaptive bit rate streaming technology and network status data output, the data is transmitted to the decoding end for decoding to obtain the reconstructed video.

2. A video encoding and decoding method based on a novel algorithm as claimed in claim 1, characterized in that: The original video is processed to obtain video feature data, including: Based on human vision and machine vision, the original video is extracted and converted to obtain video feature data.

3. A video encoding and decoding method based on a novel algorithm as claimed in claim 1, characterized in that: Determine the coded data according to the network status data, including: Based on AI technology, the coding data is determined through network status data; wherein the coding data includes: coding parameters and coding algorithms.

4. A video encoding and decoding method based on a novel algorithm as claimed in claim 3, characterized in that: Based on AI technology, the coded data is determined through network status data, including: The network status data is input into the trained coding data determination model for processing to obtain the coding data.

5. A video encoding and decoding method based on a novel algorithm as claimed in claim 4, characterized in that: The trained encoded data determines that the model is a trained deep learning model or a machine learning algorithm model.

6. The video encoding and decoding method based on the novel algorithm as claimed in claim 1, characterized in that: The video feature data is encoded according to the encoding data to obtain compressed video data, including: Intra-frame prediction and inter-frame prediction are performed on the video feature data to obtain compressed video data.

7. The video encoding and decoding method based on the novel algorithm as claimed in claim 1, characterized in that: Based on adaptive bitrate streaming technology and network status data output, the data is transmitted to the decoding end for decoding to obtain the reconstructed video, including: After the data is transmitted to the decoding end based on the adaptive bit rate streaming technology and the network status data output, the decoding operation is performed according to the encoding parameters, encoding algorithm and compressed video data to obtain the reconstructed video.

8. A video encoding and decoding method based on a novel algorithm as claimed in claim 7, characterized in that: The decoding operation of the transmission data at the decoding end is the inverse operation of encoding the video feature data according to the encoding data.

9. The video encoding and decoding method based on a novel algorithm as claimed in claim 1, characterized in that: The encoding end includes: Encoder and entropy coding module; wherein, the encoder and entropy coding module are both used to encode the original video.

10. The video encoding and decoding method based on a novel algorithm as claimed in claim 1, characterized in that: The encoding side also includes: Codeword estimation module and rate-distortion optimization module; wherein, the codeword estimation module is used to remove statistical redundancy in the encoding operation, and the rate-distortion optimization module is used to obtain network status data and determine the encoded data, and output the transmission data to the decoding end.

11. The video encoding and decoding method based on a novel algorithm as claimed in claim 9, characterized in that: The output port of the encoder is provided with a quantization module, and the quantization module is used to reduce the output transmission delay of the encoder.

12. The video encoding and decoding method based on a novel algorithm as claimed in claim 1, characterized in that: The decoding end includes: Dequantization module and decoder; The decoder is used to perform decoding operations according to encoding parameters, encoding algorithms and compressed video data to obtain reconstructed video.

13. A computer program, characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented.

Images