Video coding method and device, electronic equipment and computer program product
By updating the image parameter set and rewriting the code stream in the video encoding process, the problem that sequence-level parameter decisions cannot achieve frame-level adaptive decisions are solved, and the encoding performance and compression efficiency are improved.
Patent Information
- Application Number
- CN202411515639.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-28
- Publication Date
- 2025-02-11
AI Technical Summary
In video encoding processing, sequence-level parameter decisions cannot achieve frame-level adaptive decisions, resulting in impaired encoding performance.
By obtaining the video frame to be encoded, the initial image parameter set is determined for encoding processing, the current encoding statistical information is obtained, the image parameter set is updated based on this information, and the code stream rewrite operation is performed to realize frame-level adaptive decision-making.
It realizes frame-level adaptive decision making during one encoding process, saves codewords and improves compression efficiency.
Smart Images
Figure CN119420936A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of video coding and decoding technology, and in particular to a video coding method, a video coding device, an electronic device and a computer program product. Background Art
[0002] Video coding refers to the method of converting files in the original video format into files in another video format through compression technology. Since the highly compressed digital video codec standard (H.264 / AVC standard), a layered structure has been used in the compression process, and most of the syntax elements shared by the group of picture (Group of Picture, GOP) layer, the slice layer or the slice layer (Slice layer) have been separated to form a sequence parameter set (Sequence Parameter Set, SPS) and a picture parameter set (Picture Parameter Set, PPS). Among them, PPS contains the common parameters used for an image, that is, all slices in an image refer to the same PPS. Its general content includes initial image control information, such as initial quantization parameter (Quantization Parameter, QP) and block information.
[0003] In addition, in order to be compatible with the extension of the standard in other applications, such as scalable video encoders and multi-viewpoint video encoders, a video parameter set (VPS) was added to the syntax architecture after High Efficiency Video Coding (H.265 / HEVC). Its content generally includes syntax elements shared by multiple sublayers and other specific information that does not belong to SPS. Summary of the invention
[0004] The present disclosure provides a video encoding method, device, electronic device, computer-readable storage medium and computer program product to at least solve the problem in the related art that video encoding processing is sequence-level parameter decision-making and frame-level adaptive decision-making cannot be achieved, resulting in impaired encoding performance. The technical solution of the present disclosure is as follows:
[0005] According to a first aspect of an embodiment of the present disclosure, a video encoding method is provided, comprising: obtaining a video to be encoded, the video to be encoded comprising a plurality of video frames; determining an initial image parameter set corresponding to a current video frame, encoding the current video frame using the initial image parameter set to obtain current encoding statistical information; performing an encoding tool update operation based on the current encoding statistical information to obtain an updated image parameter set; and performing a code stream rewriting operation on the current video frame using the updated image parameter set.
[0006] In an exemplary embodiment of the present disclosure, determining the initial image parameter set corresponding to the current video frame includes: determining a preset encoding mode for video encoding, and a specified image parameter set corresponding to the preset encoding mode; and using the specified image parameter set as the initial image parameter set.
[0007] In an exemplary embodiment of the present disclosure, the encoding process of the current video frame using the initial image parameter set to obtain current encoding statistical information includes: determining a syntax element corresponding to the initial image parameter set, the syntax element including one or more of an algorithm switch, image coding distortion, and encoding intermediate information; during the encoding process of the current video frame, performing information statistics on the syntax element to obtain the current encoding statistical information.
[0008] In an exemplary embodiment of the present disclosure, the syntax element includes an algorithm switch, and the information statistics of the syntax element are performed to obtain the current encoding statistical information, including: determining the number of transform skip units of the current video frame, the number of transform skip units is the number of coding units selected for transform skipping in the current video frame in transform skip mode; and using the number of transform skip units as the current encoding statistical information.
[0009] In an exemplary embodiment of the present disclosure, the current encoding statistical information includes the number of transform skip units, and the encoding tool update operation is performed based on the current encoding statistical information to obtain an updated image parameter set, including: selecting a matching image parameter set from a predefined candidate image parameter set according to the number of transform skip units; and replacing the initial image parameter set with the matching image parameter set as the updated image parameter set.
[0010] In an exemplary embodiment of the present disclosure, the using the updated image parameter set to perform a codestream rewriting operation on the current video frame includes: determining a target coding unit in the current video frame, using a position identifier of the target coding unit as a target unit index identifier, and the target coding unit is a coding unit that does not use transform skipping; when the updated image parameter set is an initial image parameter set, adding a transform skip closing identifier to the target unit index identifier in the codestream rewriting operation; when the updated image parameter set is not the initial image parameter set, omitting the target unit index identifier in the codestream rewriting operation.
[0011] In an exemplary embodiment of the present disclosure, the determining of the initial image parameter set corresponding to the current video frame further includes: determining a preceding frame before the current video frame, obtaining a preceding frame encoding result of the preceding frame under a preset encoding mode, the preceding frame encoding result including preceding frame encoding statistical information and a selected encoding tool of the preceding frame; obtaining frame feature information of the preceding frame, and constructing a coding tool prediction model based on the frame feature information; determining, by the coding tool prediction model, a preceding frame tool selection result of the preceding frame under the frame feature information based on the preceding frame encoding statistical information and the selected encoding tool of the preceding frame; and determining the initial image parameter set according to the preceding frame tool selection result.
[0012] In an exemplary embodiment of the present disclosure, determining the initial image parameter set according to the previous frame tool selection result includes: determining current frame features corresponding to the current video frame; and predicting the initial image parameter set corresponding to the current video frame according to the previous frame tool selection result and the current frame features.
[0013] In an exemplary embodiment of the present disclosure, the method also includes: determining an actual image parameter set corresponding to the current video frame; when the actual image parameter set is inconsistent with the initial image parameter set, updating the encoding tool prediction model based on the current encoding statistical information to obtain an updated encoding tool prediction model.
[0014] According to a second aspect of an embodiment of the present disclosure, a video encoding device is provided, comprising: a video acquisition module, used to acquire a video to be encoded, wherein the video to be encoded comprises a plurality of video frames; a statistical information determination module, used to determine an initial image parameter set corresponding to a current video frame, and use the initial image parameter set to perform encoding processing on the current video frame to obtain current encoding statistical information; a parameter set updating module, used to perform an encoding tool update operation based on the current encoding statistical information to obtain an updated image parameter set; and a video encoding module, used to perform a code stream rewriting operation on the current video frame using the updated image parameter set.
[0015] In an exemplary embodiment of the present disclosure, the statistical information determination module includes a first parameter set determination unit, which is used to: determine a preset encoding mode for video encoding, and a specified image parameter set corresponding to the preset encoding mode; and use the specified image parameter set as the initial image parameter set.
[0016] In an exemplary embodiment of the present disclosure, the statistical information determination module includes a statistical information determination unit, which is used to: determine the syntax elements corresponding to the initial image parameter set, the syntax elements including one or more of algorithm switches, image coding distortion and coding intermediate information; during the encoding process of the current video frame, perform information statistics on the syntax elements to obtain the current encoding statistical information.
[0017] In an exemplary embodiment of the present disclosure, the syntax element includes an algorithm switch, and the statistical information determination unit includes a statistical information determination sub-unit, which is used to: determine the number of transform skip units of the current video frame, the number of transform skip units being the number of coding units selected for transform skipping in the current video frame in transform skip mode; and use the number of transform skip units as the current encoding statistical information.
[0018] In an exemplary embodiment of the present disclosure, the current encoding statistical information includes the number of transform skip units, and the parameter set update module includes a parameter set update unit, which is used to: select a matching image parameter set from a predefined candidate image parameter set according to the number of transform skip units; and replace the initial image parameter set with the matching image parameter set as the updated image parameter set.
[0019] In an exemplary embodiment of the present disclosure, the video encoding module includes a video encoding unit, which is used to: determine a target coding unit in the current video frame, use a position identifier of the target coding unit as a target unit index identifier, and the target coding unit is a coding unit that does not adopt transform skipping; when the updated image parameter set is an initial image parameter set, add a transform skip closing identifier to the target unit index identifier in the code stream rewriting operation; when the updated image parameter set is not the initial image parameter set, omit the target unit index identifier in the code stream rewriting operation.
[0020] In an exemplary embodiment of the present disclosure, the statistical information determination module includes a second parameter set determination unit, which is used to: determine a preceding frame before the current video frame, obtain a preceding frame encoding result of the preceding frame under a preset encoding mode, and the preceding frame encoding result includes preceding frame encoding statistical information and a selected encoding tool of the preceding frame; obtain frame feature information of the preceding frame, and construct a coding tool prediction model based on the frame feature information; determine, by the coding tool prediction model, a preceding frame tool selection result of the preceding frame under the frame feature information based on the preceding frame encoding statistical information and the selected encoding tool of the preceding frame; and determine the initial image parameter set according to the preceding frame tool selection result.
[0021] In an exemplary embodiment of the present disclosure, the second parameter set determination unit includes a second parameter set determination subunit, which is used to: determine the current frame features corresponding to the current video frame; and predict the initial image parameter set corresponding to the current video frame based on the previous frame tool selection result and the current frame features.
[0022] In an exemplary embodiment of the present disclosure, the video encoding device also includes a model updating module, which is used to: determine an actual image parameter set corresponding to the current video frame; when the actual image parameter set is inconsistent with the initial image parameter set, update the encoding tool prediction model based on the current encoding statistical information to obtain an updated encoding tool prediction model.
[0023] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement any one of the above-described video encoding methods.
[0024] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any one of the video encoding methods described above.
[0025] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, which implements any one of the above-mentioned video encoding methods when executed by a processor.
[0026] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects: by adopting a multi-PPS encoding scheme, during a single encoding process, a frame-level adaptive decision can be made as to whether to adopt a certain encoding tool, thereby saving code words and improving compression efficiency.
[0027] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.
[0029] Figure 1 It is a hierarchical structure diagram used in related coding schemes to determine the slice information in an image.
[0030] Figure 2 The figure is a flowchart of a video encoding method according to an exemplary embodiment.
[0031] Figure 3 The figure is a diagram showing an implementation process of a frame-level lossless quality control solution according to an exemplary embodiment.
[0032] Figure 4 It is a diagram showing the implementation process of video encoding based on an adaptive frame-level control model of a Bayesian model according to an exemplary embodiment.
[0033] Figure 5 The invention is a block diagram of a video encoding device according to an exemplary embodiment.
[0034] Figure 6 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0035] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.
[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0037] In video encoding based on the H.265 / HEVC standard, for a certain Slice, the PPS used by it is referenced, the PPS in turn references its corresponding SPS, and the SPS in turn references its corresponding VPS, and finally the common information of the Slice is obtained. Figure 1 As shown, Figure 1 It is a hierarchical structure diagram used in related coding schemes to determine the slice information in an image. Due to the diversity of video content and the influence of coding configuration and bit rate range, the codec tool does not bring more accurate prediction in certain scenes or frames. Instead, it will reduce the compression performance because of the need to use codewords to encode such syntax elements.
[0038] In various standards and implementations, since the tool switches are already specified in shared parameter sets such as VPS, SPS, and PPS, they cannot be switched adaptively, resulting in encoding losses. In some related schemes, multiple encodings are performed to analyze whether the encoding tool is beneficial for certain frames or sequences, and then the tool is turned off in the next encoding. However, the multiple encoding method may have the following problems: (1) Excessively high transcoding machine and time costs. (2) Parameter decisions are still made at the sequence level, and frame-level adaptation cannot be achieved.
[0039] Based on this, according to the embodiments of the present disclosure, a video encoding method, a video encoding device, an electronic device, a computer-readable storage medium, and a computer program product are proposed.
[0040] Figure 2 is a flowchart of a video encoding method according to an exemplary embodiment. Figure 1 As shown, the video encoding method can be used in a computer device, wherein the computer device described in the present disclosure may include mobile terminal devices such as mobile phones, tablet computers, laptop computers, PDAs, and fixed terminal devices such as desktop computers. This exemplary embodiment uses the method applied to a computer device as an example, and it can be understood that the method can also be applied to a server, and can also be applied to a system including a computer device and a server, and is implemented through the interaction between the computer device and the server. Specifically, the following steps are included.
[0041] In step S210, a video to be encoded is obtained, where the video to be encoded includes a plurality of video frames.
[0042] In an exemplary embodiment of the present disclosure, before performing video encoding, a video to be encoded that needs to be encoded is obtained. For example, in video communication scenarios such as video conferencing, online education, and telemedicine, a high-definition audio and video encoder is used to achieve real-time, high-definition, and smooth audio and video communication, and the video to be encoded is encoded and compressed for subsequent communication transmission. The video to be encoded is usually composed of multiple video frames, and the multiple video frames can form a corresponding video frame sequence.
[0043] In step S220, an initial image parameter set corresponding to the current video frame is determined, and the current video frame is encoded using the initial image parameter set to obtain current encoding statistical information.
[0044] In an exemplary embodiment of the present disclosure, the current video frame may be a video frame to be currently encoded. The initial image parameter set may be an image parameter set specified for the current video frame in an initial state. The current encoding statistical information may be statistical information obtained by performing information statistics on relevant syntax elements when encoding the current video frame based on the initial image parameter set.
[0045] The video encoding method disclosed in the present invention can realize the frame-level adaptive decision whether to turn off or on a switch controlled by PPS during one encoding process. Therefore, during the encoding process, encoding processing is performed frame by frame, the initial image parameter set corresponding to the current video frame is used to encode the current video frame, and information statistics are performed on related syntax elements to obtain current encoding statistical information.
[0046] In step S230, a coding tool update operation is performed based on the current coding statistical information to obtain an updated image parameter set.
[0047] In an exemplary embodiment of the present disclosure, the encoding tool update operation may be an operation of updating the image parameter set used by the current video frame. Updating the image parameter set may be updating the initial image parameter set to another image parameter set that is more suitable for the current video frame based on the current encoding statistics.
[0048] After obtaining the current coding statistical information, it can be determined based on the current coding statistical information whether to adaptively adjust the current initial image parameter set, update it to other PPSs that are more suitable for the current video frame, and use the updated image parameter set to rewrite the code stream of the current video frame.
[0049] In step S240, a code stream rewriting operation is performed on the current video frame using an updated image parameter set.
[0050] In an exemplary embodiment of the present disclosure, the code stream rewriting operation may be a processing operation of encoding the current video frame using an updated image parameter set and writing the encoding result into the code stream.
[0051] After the encoding tool update operation is performed, the updated image parameter set is a PPS that is more suitable for the current video frame. The updated image parameter set is used to rewrite the code stream of the current video frame, so that the multi-PPS solution can adaptively decide whether to use a certain PPS at the frame level during a single encoding process.
[0052] According to the video encoding method in this example embodiment, a multi-PPS encoding scheme is adopted. In one encoding process, a frame-level adaptive decision can be made whether to use a certain encoding tool, thereby saving code words and improving compression efficiency.
[0053] The video encoding method in this exemplary embodiment will be further described below.
[0054] In an exemplary embodiment of the present disclosure, for step S220, determining the initial image parameter set corresponding to the current video frame includes: determining a preset encoding mode for video encoding, and a specified image parameter set corresponding to the preset encoding mode; and using the specified image parameter set as the initial image parameter set.
[0055] The preset coding mode may be a coding mode pre-configured when performing video coding. The specified image parameter set may be a default image parameter set used in the preset coding mode. The PPS stores coding parameters of the video frame, including but not limited to entropy coding mode, slice segmentation type, initial quantization parameter, chrominance quantization parameter, etc.
[0056] In video encoding, PPS controls the switches of tools including but not limited to adaptive block-level quantization parameters (QuantizerParameter, QP), weighted prediction parameters (Weighted Predict), parameter decisions of deblocking filters (Deblock), and transform switches. When encoding the current video frame, the encoding mode can be configured as a preset encoding mode. For example, the preset encoding mode can be an encoding mode that sets various encoding tools in PPS to a fully open state. In this mode, the switches of the above encoding tools are all set to fully open.
[0057] Get the specified image parameter set in the preset encoding mode (that is, the mode with all algorithms turned on), refer to Figure 3 , Figure 3 FIG. 1 is a diagram showing an implementation process of a frame-level quality lossless control solution according to an exemplary embodiment. Figure 3 It can be seen that the designated image parameter set in the algorithm full-on mode in the present disclosure is PPS0, therefore, PPS0 is used as the initial image parameter set for encoding the current video frame, and the algorithm full-on mode is subsequently used for encoding. Through the above steps, the image parameter set for encoding processing is configured in advance for the current video frame for subsequent encoding.
[0058] In an exemplary embodiment of the present disclosure, for step S230, the current video frame is encoded using an initial image parameter set to obtain current encoding statistical information, including: determining the syntax elements corresponding to the initial image parameter set, the syntax elements including one or more of an algorithm switch, image coding distortion, and encoding intermediate information; during the encoding process of the current video frame, performing information statistics on the syntax elements to obtain current encoding statistical information.
[0059] The syntax element may be a syntax element of an image parameter set, which defines the encoding parameters of the image, thereby affecting the compression efficiency and quality of the video. The algorithm switch may be an identifier for controlling whether a certain algorithm is turned on. The image encoding distortion may be distortion information generated by encoding the current video frame using a certain PPS. The encoding intermediate information may be intermediate information generated by encoding the current video frame using the initial image parameter set, for example, the intermediate information may include the size of the encoded codeword, etc.
[0060] Continue to refer Figure 3 In the encoding process, the encoding information statistics step is performed to perform information statistics on the relevant syntax elements of the initial image parameter set. For example, the syntax elements of the PPS may include but are not limited to identifiers and reference information, availability flags of encoding tools, syntax elements related to the quantization process, syntax elements related to tiles and deblocking filters, control information and shared information, etc. In this embodiment, statistics are performed on the syntax elements encoded using PPS0, including but not limited to the algorithm switch in PPS0, image coding distortion and encoding intermediate information, and other related syntax elements, to obtain current encoding statistics information. Based on the obtained current encoding statistics information, whether to turn off or on a switch controlled by PPS can be adaptively adjusted at the frame level.
[0061] In an exemplary embodiment of the present disclosure, information statistics are performed on syntax elements to obtain current coding statistical information, including: determining the number of transform skip units of the current video frame, the number of transform skip units is the number of coding units selected for transform skipping in the current video frame in transform skip mode; and using the number of transform skip units as the current coding statistical information.
[0062] The number of transform skip units may be the number of coding units determined based on the transform skip algorithm used in the initial image parameter set. The transform skip mode may be a mode for performing video coding using the transform skip algorithm.
[0063] When performing information statistics on related syntax elements, taking the Transform Skip algorithm as an example, the number of Transform Skip selected in the coding unit is counted, such as the coding unit can be a Coding Tree Unit (CTU) or a Coding Unit (CU). The number of coding units selected for Transform Skip in the current video frame is counted and used as a current coding statistical information, so that the image parameter set selected in the current frame can be adaptively decided based on the number of Transform Skip units.
[0064] In an exemplary embodiment of the present disclosure, for step S230, a coding tool update operation is performed based on current coding statistical information to obtain an updated image parameter set, including: selecting a matching image parameter set from a predefined candidate image parameter set according to the number of transform skip units; and replacing the initial image parameter set with the matching image parameter set as the updated image parameter set.
[0065] After determining the number of transform skip units, a matching image parameter set is selected from a predefined candidate image parameter set according to the number of transform skip units. For example, if the number of coding units for which Transform Skip is selected in the current video frame is 0, that is, the number of transform skip units is equal to 0, then a PPS whose transform skip flag (transform_skip_enable_flag) is 0 can be selected from multiple candidate PPSs as a matching image parameter set corresponding to the current video frame. The matching image parameter set can be an image parameter set that matches the number of transform skip units of the current video frame.
[0066] Continue to refer Figure 3 After the coding information is counted, the PPS adaptive selection module selects the current video frame that is more compatible with PPS1. Therefore, the matching image parameter set is used to replace the initial image parameter set as the updated image parameter set, so that the current video frame can be rewritten with PPS1 later. By counting the information in an algorithm tool, it is possible to adaptively decide whether the current video frame uses a certain PPS, and then turn off or on the switch controlled by a certain PPS.
[0067] In an exemplary embodiment of the present disclosure, for step S240, a code stream rewriting operation is performed on the current video frame using an updated image parameter set, including: determining a target coding unit in the current video frame, using a position identifier of the target coding unit as a target unit index identifier, the target coding unit being a coding unit that does not use transform skipping; when the updated image parameter set is an initial image parameter set, adding a transform skip closing identifier to the target unit index identifier in the code stream rewriting operation; when the updated image parameter set is not the initial image parameter set, omitting the target unit index identifier in the code stream rewriting operation.
[0068] The target coding unit may be a coding unit that is not encoded using the transform skip mode. The position identifier may be an identifier of the position of the target coding unit in the current video frame. The target unit index identifier may be an index identifier used by the target coding unit during the bitstream rewriting process. The transform skip off identifier may be an identifier used to indicate that a certain coding unit does not use the transform skip algorithm.
[0069] This implementation is described by taking the current video frame using PPS0 encoding or PPS1 encoding as an example. For the target coding unit in the current video frame, the target coding unit is a coding unit that does not use transform skipping, that is, a coding unit that does not use TransformSkip. After determining the target coding unit, the position identifier (such as CU position) of the target coding unit is used as the target unit index identifier.
[0070] For example, if the updated image parameter set is the initial image parameter set, that is, PPS0 encoding continues to be used, it is necessary to mark the CU at the corresponding CU position that does not use Transform Skip. If the updated image parameter set is not the initial image parameter set, it means that the current video frame no longer uses PPS0 encoding, but directly uses PPS1 encoding, and the corresponding CU position identifier, that is, the target unit index identifier (flag), can be omitted. Since the identifier of each coding unit needs to occupy a certain codeword, omitting the CU position identifier can achieve the purpose of saving codeword writing.
[0071] In the above process, some video frames may not select the Transform Skip mode. PPS1 is used to control the CU codeword writing to be turned off, which is lossless for the final image quality. However, since the CU-level flag writing is omitted, the purpose of losslessly improving the compression efficiency is achieved.
[0072] The above processing process provides a lossless quality control solution, which can only achieve lossless improvement of compression efficiency. In actual video encoding, due to the dependency of the current video frame on the preceding block during the encoding process, PPS0 is directly used for encoding. Unless a certain algorithm is detected to be completely closed, it is still impossible to adaptively select a more appropriate PPS for encoding. Based on this, the present disclosure further proposes a Bayesian model based on the prior probability of the preceding frame, and uses the Bayesian model to construct a probability model, so that in the subsequent process, the appropriate PPS is predicted based on the video features, and the subsequent encoding information statistics are used for probability model update.
[0073] In an exemplary embodiment of the present disclosure, for step S220, determining the initial image parameter set corresponding to the current video frame also includes: determining a preceding frame before the current video frame, obtaining a preceding frame encoding result of the preceding frame under a preset encoding mode, the preceding frame encoding result including preceding frame encoding statistical information and a selected encoding tool of the preceding frame; obtaining frame feature information of the preceding frame, and constructing a coding tool prediction model based on the frame feature information; determining, by the coding tool prediction model, a preceding frame tool selection result of the preceding frame under the frame feature information based on the preceding frame encoding statistical information and the selected encoding tool of the preceding frame; and determining the initial image parameter set according to the preceding frame tool selection result.
[0074] Among them, the preceding frame may be a video frame that is before the current video frame in time sequence, for example, the preceding frame may be the first N frames that have been encoded, and the current video frame is the N+1th frame at this time. The encoding result of the preceding frame may be the encoding result obtained by encoding the preceding frame in a preset encoding mode. The preceding frame encoding statistical information may be information obtained by encoding the preceding frame in a preset encoding mode and performing information statistics on related syntax elements. The selected encoding tool of the preceding frame may be an image parameter set adaptively selected for the preceding frame during the encoding process. The encoding tool prediction model may be a calculation model for predicting the probability value of the encoding tool used for a certain video frame. The preceding frame tool selection result may be the PPS actually selected for the preceding frame.
[0075] refer to Figure 4 , Figure 4 It is a diagram of the implementation process of video encoding based on an adaptive frame-level control model based on a Bayesian model according to an exemplary embodiment. In this embodiment, the N video frames in the video frame sequence that are before the current video frame are used as the preceding frames (the first N frames). In the learning and training stage, the probability model is constructed based on the prior probability of the first N frames and the Bayesian model. Specifically, the encoding result corresponding to the preceding frame under the preset encoding mode is obtained. For the first N frames, the algorithm is fully open for encoding (the default PPS0), and the preceding frame encoding result corresponding to the preceding frame is determined. The preceding frame encoding result may include but is not limited to the feature information and actual selection (i.e., the selected encoding tool) of the preceding frame encoding, and the corresponding selected algorithm in the PPS adaptive selection module is recorded.
[0076] It should be noted that the process of initially encoding the first N frames using PPS0 and subsequently adaptively adjusting the PPS has been described above and will not be repeated in this embodiment.
[0077] When constructing the coding tool prediction model, the frame feature information corresponding to the previous frame is obtained. The frame feature information may include but is not limited to quantization parameters, frame types, texture features, etc. Then, the coding tool prediction model is constructed based on the frame feature information. In the coding statistics process of the first N frames, the corresponding selected algorithms in the adaptive selection module of the first N frames are recorded. For the algorithm switches that need to be decided in the coding, based on the Bayesian principle, the probability under the corresponding features is counted, and the posterior probability is estimated, as shown in Formula 1.
[0078]
[0079] Among them, f = on / off can indicate whether the final tool (such as algorithm) is turned on or off, F0, F1, ..., F n It can represent the feature information of the selected frame; p0 can represent the frame features F0, F1, ..., F nThe probability of a tool (algorithm) being closed. P(X|Y) can represent the probability of event X occurring when event Y has already occurred. P(f=off|F0,F1,…,F n ) can be expressed in frame features F0, F1, ..., F n The probability of the tool closing.
[0080] After the coding tool prediction model is constructed according to Formula 1, the initial image parameter set can be determined according to the previous frame tool selection result predicted by the coding tool prediction model. For example, a video frame closest to the current video frame is selected from the previous frame, and the initial image parameter set is determined according to the previous frame tool selection result of the video frame. By constructing the coding tool prediction model, the image parameter set can be adaptively selected according to the results and probabilities of the previous training, instead of selecting the image parameter set of the current video frame as PPS0 by default, thereby further improving the adaptability of the image parameter set corresponding to the current video frame.
[0081] In an exemplary embodiment of the present disclosure, an initial image parameter set is determined based on a previous frame tool selection result, including: determining current frame features corresponding to a current video frame; and predicting an initial image parameter set corresponding to the current video frame based on the previous frame tool selection result and the current frame features.
[0082] The current frame feature may be feature information contained in the current video frame, and may include but is not limited to quantization parameters, frame types, texture features, and the like.
[0083] Continue to refer Figure 4 In the model application stage, after the coding tool prediction model is constructed, based on the tool selection results (i.e., training results) of the previous N frames, after the N+1 frame, the PPS is adaptively selected based on the results and probability of the previous training and the current frame features contained in the current video frame, instead of selecting the PPS of the current video frame as PPS0 by default. By training and learning the tool selection results of the previous frames, the posterior probability of the current video frame with the above frame feature information can be estimated based on the prior probability of the previous frame under the corresponding features, and then a more suitable PPS can be adaptively selected for encoding.
[0084] In an exemplary embodiment of the present disclosure, an actual image parameter set corresponding to the current video frame is determined; when the actual image parameter set is inconsistent with the initial image parameter set, the encoding tool prediction model is updated based on the current encoding statistical information to obtain an updated encoding tool prediction model.
[0085] After the initial image parameter set of the current video frame is predicted by the encoding tool prediction model, the initial image parameter set may not be PPS0, but may be other PPS (such as PPS1). The selected PPS is used to encode the current video frame, and data statistics are continuously performed, and information statistics are performed on related syntax elements, including but not limited to algorithm switches, various distortions in the calculation process, intermediate information, etc.
[0086] After determining the initial image parameter set corresponding to the current video frame, the original PPS adaptive selection module is maintained in the subsequent process, and a new PPS is selected for the bitstream rewriting operation. Figure 4 For example, the initial image parameter set determined by the coding tool prediction model for the current video frame is PPS1, but the subsequent PPS adaptive selection module adaptively adjusts the PPS1 of the current video frame to PPS0.
[0087] Furthermore, the actual image parameter set finally selected for the current video frame is obtained; if the actual image parameter set is inconsistent with the initial image parameter set, it means that the prediction accuracy of the coding tool prediction model is not high, and the probability model is updated again. For example, the coding tool prediction model is updated by combining the coding statistics of the previous frame with the current coding statistics of the current frame to obtain an updated coding tool prediction model. By continuously updating the coding tool prediction model, the accuracy of the model prediction results can be further improved.
[0088] In summary, the video encoding method disclosed in the present invention obtains a video to be encoded, wherein the video to be encoded includes multiple video frames; determines an initial image parameter set corresponding to a current video frame, uses the initial image parameter set to encode the current video frame, and obtains current encoding statistical information; performs an encoding tool update operation based on the current encoding statistical information to obtain an updated image parameter set; and uses the updated image parameter set to rewrite the code stream of the current video frame. On the one hand, by adopting a multi-PPS encoding scheme, a frame-level adaptive decision can be made during a single encoding process to determine whether to use a certain encoding tool, thereby saving code words and improving compression efficiency. On the other hand, during the encoding process, a direct decision or prediction is made based on the intermediate result of the encoding to select whether to use the corresponding algorithm tool, thereby achieving the effect of saving bit rate from the PPS used in the frame-level adaptive decision.
[0089] Figure 5 FIG. 1 is a block diagram of a video encoding device according to an exemplary embodiment. Figure 5 The video encoding device 500 includes: a video acquisition module 510, a statistical information determination module 520, a parameter set updating module 530 and a video encoding module 540.
[0090] Specifically, the video acquisition module 510 is used to acquire the video to be encoded, which includes multiple video frames; the statistical information determination module 520 is used to determine the initial image parameter set corresponding to the current video frame, and use the initial image parameter set to encode the current video frame to obtain current encoding statistical information; the parameter set update module 530 is used to perform an encoding tool update operation based on the current encoding statistical information to obtain an updated image parameter set; the video encoding module 540 is used to use the updated image parameter set to perform a code stream rewrite operation on the current video frame.
[0091] In an exemplary embodiment of the present disclosure, the statistical information determination module 520 includes a first parameter set determination unit, which is used to: determine a preset coding mode for video coding, and a specified image parameter set corresponding to the preset coding mode; and use the specified image parameter set as an initial image parameter set.
[0092] In an exemplary embodiment of the present disclosure, the statistical information determination module 520 includes a statistical information determination unit, which is used to: determine the syntax elements corresponding to the initial image parameter set, the syntax elements including one or more of algorithm switches, image coding distortion and coding intermediate information; during the encoding process of the current video frame, perform information statistics on the syntax elements to obtain current encoding statistical information.
[0093] In an exemplary embodiment of the present disclosure, the syntax element includes an algorithm switch, and the statistical information determination unit includes a statistical information determination sub-unit, which is used to: determine the number of transform skip units of the current video frame, the number of transform skip units is the number of coding units selected for transform skipping in the current video frame in the transform skip mode; and use the number of transform skip units as the current encoding statistical information.
[0094] In an exemplary embodiment of the present disclosure, the current encoding statistical information includes the number of transform skip units, and the parameter set update module 530 includes a parameter set update unit, which is used to: select a matching image parameter set from a predefined candidate image parameter set according to the number of transform skip units; and replace the initial image parameter set with the matching image parameter set as an updated image parameter set.
[0095] In an exemplary embodiment of the present disclosure, the video encoding module 540 includes a video encoding unit, which is used to: determine a target coding unit in a current video frame, use a position identifier of the target coding unit as a target unit index identifier, and the target coding unit is a coding unit that does not adopt transform skipping; when the updated image parameter set is the initial image parameter set, add a transform skip closing identifier to the target unit index identifier in a code stream rewriting operation; when the updated image parameter set is not the initial image parameter set, omit the target unit index identifier in the code stream rewriting operation.
[0096] In an exemplary embodiment of the present disclosure, the statistical information determination module 520 also includes a second parameter set determination unit, which is used to: determine a previous frame before the current video frame, obtain a previous frame encoding result of the previous frame under a preset encoding mode, the previous frame encoding result including the previous frame encoding statistical information and the selected encoding tool of the previous frame; obtain frame feature information of the previous frame, and construct a coding tool prediction model based on the frame feature information; determine the previous frame tool selection result of the previous frame under the frame feature information based on the previous frame encoding statistical information and the selected encoding tool of the previous frame by the coding tool prediction model; and determine the initial image parameter set according to the previous frame tool selection result.
[0097] In an exemplary embodiment of the present disclosure, the second parameter set determination unit includes a second parameter set determination subunit, which is used to: determine the current frame features corresponding to the current video frame; and predict the initial image parameter set corresponding to the current video frame based on the previous frame tool selection result and the current frame features.
[0098] In an exemplary embodiment of the present disclosure, the video encoding device 500 also includes a model updating module, which is used to: determine the actual image parameter set corresponding to the current video frame; when the actual image parameter set is inconsistent with the initial image parameter set, update the encoding tool prediction model based on the current encoding statistical information to obtain an updated encoding tool prediction model.
[0099] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0100] Reference below Figure 6 hereinafter describes an electronic device 600 according to such an embodiment of the present disclosure. Figure 6 The electronic device 600 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0101] like Figure 6 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), and a display unit 640.
[0102] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.
[0103] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 621 and / or a cache memory unit 622 , and may further include a read-only memory unit (ROM) 623 .
[0104] The storage unit 620 may include a program / utility 624 having a set (at least one) of program modules 625, such program modules 625 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0105] Bus 630 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0106] The electronic device 600 may also communicate with one or more external devices 670 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 660. As shown, the network adapter 660 communicates with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0107] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of the device to complete the above video encoding method. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0108] In an exemplary embodiment, a computer program product is further provided, comprising a computer program, and when the computer program is executed by a processor, the video encoding method described in any one of the above is implemented.
[0109] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0110] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A video encoding method, characterized in that: include: Acquire a video to be encoded, where the video to be encoded includes multiple video frames; Determine an initial image parameter set corresponding to a current video frame, and use the initial image parameter set to encode the current video frame to obtain current encoding statistical information; Performing a coding tool update operation based on the current coding statistical information to obtain an updated image parameter set; The updated image parameter set is used to perform a code stream rewriting operation on the current video frame.
2. The method according to claim 1, characterized in that The determining of the initial image parameter set corresponding to the current video frame includes: Determine a preset encoding mode for video encoding, and a specified image parameter set corresponding to the preset encoding mode; The designated image parameter set is used as the initial image parameter set.
3. The method according to claim 1, characterized in that The encoding process of the current video frame by using the initial image parameter set to obtain current encoding statistical information includes: Determine a syntax element corresponding to the initial image parameter set, where the syntax element includes one or more of an algorithm switch, image coding distortion, and coding intermediate information; During the encoding process of the current video frame, information statistics are performed on the syntax elements to obtain the current encoding statistical information.
4. The method according to claim 3, characterized in that The syntax element includes an algorithm switch, and the performing information statistics on the syntax element to obtain the current coding statistical information includes: Determine a number of transform skip units for the current video frame, wherein the number of transform skip units is the number of coding units selected for transform skipping in the current video frame in a transform skip mode; The number of transform skip units is used as the current encoding statistical information.
5. The method according to claim 1, characterized in that The current encoding statistical information includes the number of transform skip units, and the encoding tool update operation is performed based on the current encoding statistical information to obtain an updated image parameter set, including: Selecting a matching image parameter set from a predefined candidate image parameter set according to the number of transformation skip units; The matching image parameter set is used to replace the initial image parameter set as the updated image parameter set.
6. The method according to claim 1, characterized in that The adopting the updated image parameter set to perform a code stream rewriting operation on the current video frame includes: Determine a target coding unit in the current video frame, and use a position identifier of the target coding unit as a target unit index identifier, wherein the target coding unit is a coding unit that does not adopt transformation skipping; When the updated image parameter set is an initial image parameter set, adding a transform skip close flag to the target unit index flag in the code stream rewriting operation; When the updated image parameter set is not the initial image parameter set, the target unit index identifier is omitted in the code stream rewriting operation.
7. The method according to claim 1, characterized in that The step of determining an initial image parameter set corresponding to the current video frame further includes: Determine a preceding frame before the current video frame, and obtain a preceding frame encoding result of the preceding frame in a preset encoding mode, wherein the preceding frame encoding result includes preceding frame encoding statistical information and a selected encoding tool of the preceding frame; Acquire frame feature information of the preceding frame, and construct a coding tool prediction model based on the frame feature information; The coding tool prediction model determines a preceding frame tool selection result of the preceding frame under the frame feature information based on the preceding frame coding statistical information and the selected coding tool of the preceding frame; The initial image parameter set is determined according to the preceding frame tool selection result.
8. The method according to claim 7, characterized in that The determining the initial image parameter set according to the preceding frame tool selection result includes: Determining a current frame feature corresponding to the current video frame; An initial image parameter set corresponding to the current video frame is predicted based on the previous frame tool selection result and the current frame features.
9. The method according to claim 7, characterized in that: The method further comprises: Determining an actual image parameter set corresponding to the current video frame; When the actual image parameter set is inconsistent with the initial image parameter set, the coding tool prediction model is updated based on the current coding statistical information to obtain an updated coding tool prediction model.
10. A video encoding device, characterized in that: include: A video acquisition module, used to acquire a video to be encoded, wherein the video to be encoded includes a plurality of video frames; A statistical information determination module, used to determine an initial image parameter set corresponding to a current video frame, and use the initial image parameter set to encode the current video frame to obtain current encoding statistical information; A parameter set updating module, configured to perform an encoding tool updating operation based on the current encoding statistical information to obtain an updated image parameter set; The video encoding module is used to perform a code stream rewriting operation on the current video frame using the updated image parameter set.
11. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the video encoding method according to any one of claims 1 to 9.
12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the video encoding method according to any one of claims 1 to 9 is implemented.