Video processing method and apparatus, electronic device, and computer program product

Abstract

The present disclosure relates to a video processing method and device, electronic equipment and computer program product, and relates to the technical field of video coding. The method comprises: obtaining a to-be-processed video, the to-be-processed video comprising a video frame set, the video frame set comprising video coding reference information of a to-be-processed video frame; obtaining a pre-configured coding configuration parameter, the coding configuration parameter comprising a plurality of candidate resampling rates, the candidate resampling rates being determined based on a pre-configured candidate resolution; determining a target resampling rate corresponding to the video frame set from the plurality of candidate resampling rates according to the video coding reference information; and performing coding processing on the to-be-processed video frame after performing downsampling processing on the to-be-processed video frame based on the target resampling rate, to obtain a coded video frame. The present disclosure realizes coding and decoding processing of any resampling rate, supports fine selection of coding resolution, further reduces coding rate under the premise of ensuring video reconstruction quality, and improves compression efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of video encoding and decoding technology, and more specifically, to a video processing method, a video processing apparatus, an electronic device, and a computer program product. Background Technology

[0002] With the widespread application of ultra-high-definition video, the amount of video data is growing rapidly, placing higher demands on video storage and transmission bandwidth. Video coding technology is a digital processing technique that uses compression algorithms to reduce image data redundancy. Its core objective is to reduce data volume while maintaining image quality, facilitating storage and transmission. Video compression coding technology can be divided into two main categories: lossless compression and lossy compression. Lossless compression, also known as reversible coding, means that when the compressed data is reconstructed (i.e., decompressed), the reconstructed data is completely identical to the original data. Lossy compression, also known as irreversible coding, means that when the compressed data is reconstructed (i.e., decompressed), the reconstructed data differs from the original data, but this does not affect the misunderstanding of the information expressed in the original data.

[0003] For example, resampling-based video coding techniques effectively reduce bitrate by downsampling at the encoding end and upsampling at the decoding end, but inevitably increase distortion. The resampling ratio is a crucial factor in balancing bitrate savings and quality distortion, determining overall compression performance. Within the existing framework of the VVC standard test model (VTM), resampling encoding is implemented using the Group of Pictures (GOP)-based Reference Picture Resampling (RPR) tool.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide a video processing method, video processing apparatus, electronic device, computer-readable storage medium, and computer program product, thereby overcoming, at least to some extent, the problem that the limited candidate space for resampling ratios in the reference image resampling coding scheme makes it difficult to adapt to different video content and coding conditions.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0007] According to a first aspect of this disclosure, a video processing method is provided, comprising: acquiring a video to be processed, the video to be processed including a set of video frames, the set of video frames including video encoding reference information of the video frames to be processed; acquiring pre-configured encoding configuration parameters, the encoding configuration parameters including a plurality of candidate resampling ratios, the candidate resampling ratios being determined based on pre-configured candidate resolutions; determining a target resampling ratio corresponding to the set of video frames from the plurality of candidate resampling ratios according to the video encoding reference information; performing downsampling processing on the video frames to be processed based on the target resampling ratio; and performing encoding processing on the downsampled video frames to be processed to obtain encoded video frames.

[0008] In one exemplary embodiment of this disclosure, the encoding configuration parameters include reference image resampling configuration parameters and an index identifier sequence. Obtaining the pre-configured encoding configuration parameters includes: determining a candidate resolution corresponding to the video encoding operation; determining a candidate resampling ratio based on the candidate resolution; constructing reference image resampling configuration parameters based on the candidate resampling ratio; constructing a candidate image parameter set based on the candidate resampling ratio and creating an index identifier sequence corresponding to the candidate image parameter set; and constructing the encoding configuration parameters based on the reference image resampling configuration parameters and the index identifier sequence.

[0009] In one exemplary embodiment of this disclosure, constructing reference image resampling configuration parameters based on the candidate resampling magnification includes: determining the number of candidate resampling magnifications; constructing a resampling magnification list based on the candidate resampling magnifications, wherein the number of candidate resampling magnifications is the same as the number of candidate resolutions; and constructing a luminance offset list and a chrominance offset list corresponding to the candidate resampling magnifications, wherein the number of candidate resampling magnifications, the number of first elements in the luminance offset list, and the number of second elements in the chrominance offset list are the same.

[0010] In one exemplary embodiment of this disclosure, the step of constructing a candidate image parameter set based on the candidate resampling ratio and creating an index identifier sequence corresponding to the candidate image parameter set includes: constructing an initial index identifier array according to the number of candidate resampling ratios; determining the image parameter set corresponding to each candidate resampling ratio; using the image parameter set identifier corresponding to the original resolution as a base index identifier, and determining an extended index identifier corresponding to the image parameter set of each candidate resampling ratio according to the base index identifier; storing the base index identifier and the extended index identifier in the initial index identifier array to obtain the index identifier sequence.

[0011] In one exemplary embodiment of this disclosure, determining the target resampling ratio corresponding to the video frame set from the plurality of candidate resampling ratios based on the video coding reference information includes: obtaining a pre-trained ratio decision model, the ratio decision model being trained based on candidate video coding feature information and candidate codec schemes corresponding to the plurality of candidate resolutions, the candidate codec schemes including candidate downsampling ratios and matching candidate upsampling scheme identifiers; and determining the target resampling ratio and target upsampling scheme identifier corresponding to the video frame set by the ratio decision model based on the video coding reference information.

[0012] In one exemplary embodiment of this disclosure, the video frame set includes at least one video frame to be processed. The step of downsampling the video frame to be processed based on the target resampling ratio and encoding the downsampled video frame to obtain an encoded video frame includes: initializing the encoding configuration parameters to obtain candidate video encoding parameters; determining initial video encoding parameters based on the target resampling ratio and the candidate video encoding parameters; adjusting the initial video encoding parameters to obtain target video encoding parameters; downsampling the video frame to be processed based on the target resampling ratio to obtain a downsampled video frame to be processed; and encoding the downsampled video frame to be processed based on the target video encoding parameters to obtain an encoded video frame.

[0013] In one exemplary embodiment of this disclosure, the target video coding parameters include target luminance quantization parameters. The step of adjusting the initial video coding parameters to obtain the target video coding parameters includes: in response to an image-level parameter calculation instruction, determining an initial luminance quantization parameter and a first image parameter set identifier corresponding to the video frame set based on the initial video coding parameters; in response to the first image parameter set identifier being located in an index identifier sequence, determining a first target index position from the index identifier sequence; determining a luminance offset from a luminance offset list based on the first target index position; and adjusting the initial luminance quantization parameters according to the luminance offset to obtain the target luminance quantization parameters.

[0014] In one exemplary embodiment of this disclosure, the target video coding parameters include target chroma quantization parameters. The step of adjusting the initial video coding parameters to obtain the target video coding parameters further includes: in response to a slice-level parameter calculation instruction, determining the initial chroma quantization parameters corresponding to the initial video coding parameters, and a second image parameter set identifier corresponding to the video frame set; in response to the second image parameter set identifier being located in an index identifier sequence, determining a second target index position from the index identifier sequence; determining a chroma offset from a luminance offset list based on the second target index position; and adjusting the initial chroma quantization parameters according to the chroma offset to obtain the target chroma quantization parameters.

[0015] In one exemplary embodiment of this disclosure, the method further includes: obtaining video frame size parameters of the encoded video frames in the reconstructed pixel buffer, and a list of image parameter set objects corresponding to each candidate resolution, wherein the list of image parameter set objects includes at least one reference size parameter; using the reference size parameter that is the same as the video frame size parameter as the target encoding size parameter; determining the image parameter set identifier and the target upsampling identifier corresponding to the target encoding size parameter; and writing the video frame set into the video bitstream based on the image parameter set identifier and the target upsampling identifier, wherein the target upsampling identifier is used to instruct the decoding end to reconstruct the encoded video frames.

[0016] According to a second aspect of this disclosure, a video processing apparatus is provided, comprising: a video acquisition module for acquiring a video to be processed, the video to be processed including a set of video frames, the set of video frames including video encoding reference information of the video frames to be processed; a configuration parameter acquisition module for acquiring pre-configured encoding configuration parameters, the encoding configuration parameters including a plurality of candidate resampling ratios, the candidate resampling ratios being determined based on pre-configured candidate resolutions; a target ratio determination module for determining a target resampling ratio corresponding to the set of video frames from the plurality of candidate resampling ratios according to the video encoding reference information; and a video encoding module for downsampling the video frames to be processed based on the target resampling ratio, and encoding the downsampled video frames to be processed to obtain encoded video frames.

[0017] In one exemplary embodiment of this disclosure, the encoding configuration parameters include reference image resampling configuration parameters and an index identifier sequence. The configuration parameter acquisition module includes a configuration parameter construction unit, configured to: determine a candidate resolution corresponding to a video encoding operation; determine a candidate resampling ratio based on the candidate resolution; construct reference image resampling configuration parameters based on the candidate resampling ratio; construct a candidate image parameter set based on the candidate resampling ratio and create an index identifier sequence corresponding to the candidate image parameter set; and construct the encoding configuration parameters based on the reference image resampling configuration parameters and the index identifier sequence.

[0018] In one exemplary embodiment of this disclosure, the configuration parameter construction unit includes a configuration parameter construction subunit, configured to: determine the number of candidate resampling magnifications; construct a resampling magnification list based on the candidate resampling magnifications, wherein the number of candidate resampling magnifications is the same as the number of candidate resolutions; and construct a luminance offset list and a chrominance offset list corresponding to the candidate resampling magnifications, wherein the number of candidate resampling magnifications, the number of first elements in the luminance offset list, and the number of second elements in the chrominance offset list are the same.

[0019] In one exemplary embodiment of this disclosure, the configuration parameter construction unit includes an identifier sequence construction subunit, configured to: construct an initial index identifier array based on the number of candidate resampling ratios; determine the image parameter set corresponding to each candidate resampling ratio; use the image parameter set identifier corresponding to the original resolution as a base index identifier, and determine the extended index identifier corresponding to the image parameter set of each candidate resampling ratio based on the base index identifier; store the base index identifier and the extended index identifier in the initial index identifier array to obtain the index identifier sequence.

[0020] In one exemplary embodiment of this disclosure, the target scaling factor determination module includes a target scaling factor determination unit, configured to: acquire a pre-trained scaling factor decision model, the scaling factor decision model being trained based on candidate video coding feature information and multiple candidate codec schemes corresponding to the candidate resolutions, the candidate codec schemes including candidate downsampling scaling factors and matching candidate upsampling scheme identifiers; and determine the target resampling scaling factor and target upsampling scheme identifier corresponding to the video frame set by the scaling factor decision model based on the video coding reference information.

[0021] In one exemplary embodiment of this disclosure, the video frame set includes at least one video frame to be processed, and the video encoding module includes a video encoding unit, configured to: initialize the encoding configuration parameters to obtain candidate video encoding parameters; determine initial video encoding parameters based on the target resampling ratio and the candidate video encoding parameters; perform parameter adjustment processing on the initial video encoding parameters to obtain target video encoding parameters; perform downsampling processing on the video frame to be processed based on the target resampling ratio to obtain a downsampled video frame to be processed; and perform encoding processing on the downsampled video frame to be processed based on the target video encoding parameters to obtain an encoded video frame.

[0022] In one exemplary embodiment of this disclosure, the target video coding parameters include target luminance quantization parameters, and the video coding unit includes a first parameter adjustment subunit, configured to: in response to an image-level parameter calculation instruction, determine an initial luminance quantization parameter and a first image parameter set identifier corresponding to the video frame set based on the initial video coding parameters; in response to the first image parameter set identifier being located in an index identifier sequence, determine a first target index position from the index identifier sequence; determine a luminance offset from a luminance offset list based on the first target index position, and adjust the initial luminance quantization parameter according to the luminance offset to obtain the target luminance quantization parameter.

[0023] In one exemplary embodiment of this disclosure, the target video encoding parameters include target chroma quantization parameters, and the video encoding unit further includes a second parameter adjustment subunit, configured to: in response to a slice-level parameter calculation instruction, determine the initial chroma quantization parameters corresponding to the initial video encoding parameters, and the second image parameter set identifier corresponding to the video frame set; in response to the second image parameter set identifier being located in an index identifier sequence, determine a second target index position from the index identifier sequence; determine a chroma offset from a luminance offset list based on the second target index position, and adjust the initial chroma quantization parameters according to the chroma offset to obtain the target chroma quantization parameters.

[0024] In one exemplary embodiment of this disclosure, the video processing apparatus further includes a bitstream writing module, configured to: obtain video frame size parameters of the encoded video frames in the reconstructed pixel buffer, and an image parameter set object list corresponding to each candidate resolution, wherein the image parameter set object list includes at least one reference size parameter; use the reference size parameter that is the same as the video frame size parameter as the target encoding size parameter; determine the image parameter set identifier and the target upsampling identifier corresponding to the target encoding size parameter; and write the video frame set into the video bitstream based on the image parameter set identifier and the target upsampling identifier, wherein the target upsampling identifier is used to instruct the decoding end to reconstruct the encoded video frames.

[0025] According to a third aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory storing computer-readable instructions that, when executed by the processor, implement the video processing method according to any one of the preceding claims.

[0026] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the video processing method according to any one of the preceding claims.

[0027] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the video processing method described in any of the preceding claims.

[0028] The technical solution provided in this disclosure may include the following beneficial effects: The video processing method in the exemplary embodiments of this disclosure, on the one hand, implements a reference image resampling scheme with arbitrary resolution. By implementing encoding and decoding processing with arbitrary resampling ratios, it supports fine-grained selection of encoding resolution, further reducing the encoding bitrate and improving video compression efficiency while ensuring video reconstruction quality. On the other hand, it expands the candidate resampling ratios through pre-configured candidate resolutions, improving the flexibility and adaptability of encoding resolution selection.

[0029] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0030] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1 A flowchart illustrating an exemplary embodiment of the video processing method of this disclosure is shown schematically. Figure 2 This schematic diagram illustrates the overall architecture of a video processing scheme according to an exemplary embodiment of the present disclosure; Figure 3 This illustration schematically shows the result of constructing a global image parameter set index identifier mapping table according to an exemplary embodiment of this disclosure; Figure 4 This schematically illustrates a flowchart of writing a video bitstream after the encoder has completed encoding, according to an exemplary embodiment of this disclosure; Figure 5 A block diagram of a video processing apparatus according to an exemplary embodiment of the present disclosure is shown schematically; Figure 6 The schematic diagram illustrates a computer-readable storage medium according to an exemplary embodiment of the present disclosure; Figure 7 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown schematically. Detailed Implementation

[0031] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0032] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details described, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known structures, methods, apparatuses, implementations, materials, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0033] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, or in one or more software-hardened modules, or in different network and / or processor devices and / or microcontroller devices.

[0034] According to embodiments of this disclosure, a video processing method, a video processing apparatus, a computer-readable storage medium, an electronic device, and a computer program product are proposed.

[0035] The principles and spirit of this disclosure will be explained in detail below with reference to several representative embodiments.

[0036] GOP-based RPR tools optimize video compression efficiency by reducing resolution at the encoding end and restoring resolution at the decoding end. The overall encoding / decoding architecture and workflow of this scheme includes an encoding-end control mechanism and a decoding-end processing mechanism; the encoding-end control mechanism includes: (1) Control Parameter Configuration. The encoder enables and controls the RPR tool through preset syntax elements. Its core control parameters include: global switch identifier for the RPR tool (e.g., GOPBasedRPR), initial quantization parameter (QP) threshold for triggering RPR encoding (e.g., GOPBasedRPRQPTh), and a fixed number (3) of scalar resampling ratios (e.g., scaling factors in the horizontal and vertical directions ScalingRatioHor / Ver 1~3). In addition, peak signal-to-noise ratio (PSNR) thresholds and luminance / chroma quantization parameter offsets (QpOffsetRPR 1~3, QpOffsetChromaRPR 1~3) corresponding to each ratio are also configured.

[0037] (2) Initialization of Picture Parameter Set (PPS). In the preprocessing stage before encoding, the system predefines and allocates four possible sets of encoded picture parameters based on the original geometric resolution of the input video and the three fixed resampling ratios mentioned above. These four PPS correspond to the original resolution and the resolution after downsampling by three fixed ratios (height and width are reduced proportionally according to the corresponding scaling factors).

[0038] (3) Magnification decision and adaptive downsampling. The encoder adopts a specific magnification decision algorithm. By comparing the real-time distortion index of the video frame with the preset PSNR threshold, it selects the single target resolution that best matches the current scene from four candidate resolutions (one original resolution and three downsampling resolutions), and calls the corresponding downsampling filter to scale the input video frame accordingly.

[0039] (4) Quantization parameter adjustment and bitstream encapsulation. After determining the target resolution, the encoder retrieves the corresponding fixed luminance and chrominance QP offsets based on the currently active PPS index to correct the quantization step size of the current image. Finally, the encoder completes pixel encoding and encapsulates the encoding resolution and other key syntax elements such as the original resolution into the video bitstream.

[0040] The decoding process includes: (1) Bitstream parsing. The decoder parses the encoded resolution and original resolution information of the current image from the received video bitstream. (2) Adaptive scale restoration. The decoder determines the geometric dimensions based on the parsed resolution information. If the encoded resolution is consistent with the original resolution, the decoded reconstructed image is directly output; if they are inconsistent, it indicates that the frame has been downsampled at the encoding end, and the decoder will call the upsampling filter to restore the decoded reconstructed image to the original resolution before outputting it.

[0041] While the introduction of the GOP-based RPR mechanism has enriched the control methods of video coding to some extent, it has revealed the following significant shortcomings in practical applications: First, the resolution candidate space is extremely limited. The resampling rate of this scheme relies entirely on a small number of fixed predefined scalar values ​​(usually only supporting 4 fixed levels). Due to the overly narrow candidate space for resolution switching, the encoder is unable to perform fine-grained spatial resolution adaptive matching for ultra-high-definition video scenes with complex textures or variable motion characteristics.

[0042] Second, the resolution control is coarse-grained and lacks flexibility. When faced with different bitrate ranges and different subjective quality control requirements, a fixed multiplier and an unchanging judgment threshold can easily lead to suboptimal resolution selection decisions. This coarse-grained resolution control limits the upper limit of RPR tools in improving video compression efficiency and optimizing reconstructed image quality, and cannot fully realize the bandwidth-saving potential of compressing ultra-high-definition video in ultra-low bitrate transmission environments.

[0043] Based on this, in this example embodiment, a video processing method is first provided. The video processing method of this disclosure can be implemented using a server or using a terminal device. The terminal described in this disclosure may include mobile terminals such as mobile phones, tablets, laptops, handheld computers, and personal digital assistants (PDAs), as well as fixed terminals such as desktop computers. Figure 1 A schematic diagram illustrating a video processing method flow according to some embodiments of the present disclosure is provided. Reference Figure 1 The video processing method may include the following steps: Step S110: Obtain the video to be processed. The video to be processed includes a set of video frames, and the set of video frames includes video encoding reference information of the video frames to be processed. Step S120: Obtain pre-configured encoding configuration parameters, which include multiple candidate resampling ratios, and the candidate resampling ratios are determined based on pre-configured candidate resolutions; Step S130: Determine the target resampling factor corresponding to the video frame set from multiple candidate resampling factors based on the video coding reference information; Step S140: The video frame to be processed is downsampled based on the target resampling ratio, and the downsampled video frame to be processed is encoded to obtain an encoded video frame.

[0044] According to the video processing method in this example embodiment, on the one hand, this disclosure implements a reference image resampling scheme with arbitrary resolution. By implementing encoding and decoding processing with arbitrary resampling ratio, it supports fine selection of encoding resolution, further reducing the encoding bitrate and improving video compression efficiency while ensuring video reconstruction quality. On the other hand, it expands the candidate resampling ratio through pre-configured candidate resolutions, thereby improving the flexibility and adaptability of encoding resolution selection.

[0045] The video processing method in this example embodiment will be further explained below.

[0046] For step S110, the video to be processed is obtained. The video to be processed includes a set of video frames, and the set of video frames includes video coding reference information for the video frames to be processed. The video to be processed can be the original video content to be compressed and encoded, usually existing in the form of an image sequence. The video to be processed can be used as input to the encoder, and is the object that the encoder will compress. The video to be processed can contain one or more sets of video frames. A set of video frames can be a single video frame or a Group of Pictures (GOP). A GOP is a group of consecutive frames, and a GOP can contain one or more video frames. The video frames contained in the set of video frames are the video frames to be processed. The video coding reference information can be coding reference information determined based on the video frames to be processed, and can be used for subsequent resampling rate decisions.

[0047] In one exemplary embodiment of this disclosure, step S120, obtaining pre-configured encoding configuration parameters, includes: determining a candidate resolution corresponding to a video encoding operation, determining a candidate resampling ratio based on the candidate resolution; constructing reference image resampling configuration parameters based on the candidate resampling ratio; constructing a candidate image parameter set based on the resampling ratio, and creating an index identifier sequence corresponding to the candidate image parameter set; and constructing encoding configuration parameters based on the reference image resampling configuration parameters and the index identifier sequence.

[0048] To address the limited candidate space for resampling rates, this disclosure introduces a configurable variable for the number of resampling rates through syntax element extension and encoder parameter management. It then constructs dynamically mapped arrays of resampling rates, luminance quantizer parameter (QP) offsets, and chrominance QP offsets, replacing the original fixed scalar configuration method. (Reference) Figure 2 , Figure 2 The diagram illustrates the overall architecture of a video processing scheme according to an exemplary embodiment of the present disclosure. Figure 2 The encoding configuration parameters are configured through the configuration parameter management operation in step S210 and the encoder parameter management operation in step S220.

[0049] Encoding configuration parameters can be pre-configured parameters for the encoder to compress and encode the video to be processed. These parameters can include configuration parameters corresponding to any resolution to support encoding and decoding operations at arbitrary resampling rates. Encoding configuration parameters can include reference image resampling configuration parameters and an index identifier sequence. The reference image resampling configuration parameters can be configuration parameters used to support reference image resampling at arbitrary resolutions. The index identifier sequence can be a sequence composed of the index identifiers of the image parameter set PPS corresponding to the original resolution and the index identifiers of the dynamically expanded image parameter set PPS. Before configuring the encoding configuration parameters, candidate resolutions corresponding to the video encoding operation can be determined. The video encoding operation can be the operation of compressing and encoding the video to be processed. The candidate resolution can be any resolution that may correspond to the video to be processed in the video encoding task.

[0050] In this embodiment, the encoder's configuration module dynamically imports and parses configuration parameters for supporting resampling of reference images at arbitrary resolutions. The corresponding resampling factor is determined based on the candidate resolutions. Typically, each candidate resolution has a corresponding resampling factor. After determining the resampling factor, reference image resampling configuration parameters are constructed based on the candidate resampling factors through configuration parameter management operations. Furthermore, a candidate image parameter set is constructed through encoder parameter management operations, along with an index identifier sequence corresponding to the candidate image parameter set. The constructed reference image resampling configuration parameters and index identifier sequence are used as encoding configuration parameters. By using pre-configured candidate resolutions, the reference image resampling configuration parameters and PPS index identifiers can be dynamically expanded, thereby enabling encoding and decoding processing that supports arbitrary resampling factors.

[0051] In one exemplary embodiment of this disclosure, constructing reference image resampling configuration parameters based on candidate resampling magnifications includes: determining the number of candidate resampling magnifications; constructing a resampling magnification list based on the candidate resampling magnifications, wherein the number of candidate resampling magnifications is the same as the number of candidate resolutions; and constructing a luminance offset list and a chrominance offset list corresponding to the candidate resampling magnifications, wherein the number of resampling magnifications, the number of the first element in the luminance offset list, and the number of the second element in the chrominance offset list are the same.

[0052] refer to Figure 2 In step S210, the configuration parameter management also configures GOPBasedRPR and GOPBasedRPRQPTh. GOPBasedRPR is a control switch indicating whether to enable the use of the reference image resampling scheme for video encoding. GOPBasedRPRQPTh refers to the initial quantization parameter threshold that triggers RPR encoding. When the initial quantization parameter is greater than or equal to the pre-configured GOPBasedRPRQPTh value, the reference image resampling scheme is used for video encoding.

[0053] After determining the candidate resampling ratios based on the candidate resolutions, the number of candidate resampling ratios is determined. The number of candidate resampling ratios (RPR_ScaleNum) specifies the total number of candidate arbitrary resolutions supported in the current encoding task. This number can be configured according to video encoding requirements, for example, to 10. A resampling ratio list (RprScalingRatioList) can be constructed based on the number of candidate resampling ratios. This list can be a floating-point array containing all candidate resampling ratio factors. For example, the resampling ratio list could be {1.1, 1.2, 1.3, 1.4, 1.4, 1.5, 1.7, 1.8, 1.9, 2.0}, which can include candidate resampling ratios corresponding to the original resolution and any configured resolution.

[0054] Additionally, a luminance offset list and a chrominance offset list corresponding to the candidate resampling factor can be constructed. The luminance QP offset list (QpOffsetRPRList) is an integer array with the same number of elements as the resampling factor list, storing the quantization parameter compensation values ​​for the luminance layer at each arbitrary resolution (e.g., QpOffsetRPRList={-1,-2,-2,-3,-4,-4,-5,-5,-6,-6}). The chrominance QP offset list (QpOffsetChromaRPRList) is an integer array storing the quantization parameter compensation values ​​for the chrominance layer at each arbitrary resolution (e.g., QpOffsetChromaRPRList ={-1,-2,-2,-3,-4,-4,-5,-5,-6,-6}).

[0055] When the encoder starts, the configuration class uses the multi-value input interface to read the above configuration and saves it to the member vector of the encoder core configuration object, which serves as the global control parameter for the entire subsequent encoding chain. By dynamically configuring the configuration parameters of arbitrary resolution RPR, RPR encoding and decoding that supports arbitrary resampling rates can be implemented during the subsequent encoding process.

[0056] In one exemplary embodiment of this disclosure, a candidate image parameter set is constructed based on the resampling ratio, and an index identifier sequence corresponding to the candidate image parameter set is created, including: constructing an initial index identifier array according to the number of candidate resampling ratios; determining the image parameter set corresponding to each candidate resampling ratio; using the image parameter set identifier corresponding to the original resolution as a base index identifier, and determining the extended index identifier corresponding to the image parameter set of each candidate resampling ratio according to the base index identifier; storing the base index identifier and the extended index identifier in the initial index identifier array to obtain the index identifier sequence.

[0057] To manage dynamically generated arbitrary resolutions within the encoder, this disclosure extends the allocation rules for PPS identifiers (PPSIDs). First, based on the configured number of candidate resampling rates N, a set of continuous or discrete arrays is reserved in a public definition as an initial index identifier array. This initial index identifier array can be an array used to store extended PPS IDs, and can be represented as PPS_ID_Arbitrary, specifically corresponding to each arbitrary resolution. The initial index identifier array does not store specific PPS IDs; it is merely a reserved array for subsequent PPS ID storage.

[0058] For each candidate resampling factor, the corresponding image parameter set can be determined. The image parameter set can include parameters used in the video encoding process for coordinate alignment, display alignment, slice alignment and prevention of edge artifacts. For example, the image parameter set may include, but is not limited to, parameters for forcibly disabling temporal motion prediction, cropping and scaling windows, block layout reconstruction parameters and locking virtual boundary filtering parameters.

[0059] After determining the image parameter set corresponding to each candidate resampling factor, we can first determine the image parameter set identifier corresponding to the original resolution and use it as the base index identifier. The base index identifier can be the index identifier value corresponding to the image parameter set of the original resolution. The base index identifier can usually be 0 or 1. The base index identifier can serve as the basis for the PPS identifier of the extended candidate resampling factors.

[0060] refer to Figure 3 , Figure 3 The diagram illustrates the result of constructing a global image parameter set index identifier mapping table according to an exemplary embodiment of this disclosure. Figure 3 The image parameter set identifier corresponding to the original resolution is used as the base index identifier, marked as Base ID. The extended index identifier corresponding to the image parameter set of each candidate resampling rate is determined based on the base index identifier. The extended index identifier can be determined based on the base index identifier, such as... Figure 3 In this context, the extended index identifier can be ID_arb_1, ID_arb_2, ID_arb_3, ..., ID_arb_N.

[0061] The determined base index identifier and extended index identifier are stored in the initial index identifier array to obtain the index identifier sequence. The index identifier sequence can be an identifier sequence constructed from the base index identifier and extended index identifier. The elements contained in the final index identifier sequence are Base ID, ID_arb_1, ID_arb_2, ID_arb_3, ..., ID_arb_N. Through the above steps, a global total PPS ID mapping table is constructed. The base PPS ID corresponding to the original resolution is combined and arranged with these N extended PPS IDs to form a complete resampling parameter set identifier sequence, which is used for subsequent PPS ID matching and acquisition.

[0062] In one exemplary embodiment of this disclosure, step S130, determining the target resampling ratio corresponding to the video frame set from multiple candidate resampling ratios based on video coding reference information, includes: obtaining a pre-trained ratio decision model, which is trained based on candidate video coding feature information and candidate encoding / decoding schemes corresponding to multiple candidate resolutions, wherein the candidate encoding / decoding schemes include candidate downsampling ratios and matching candidate upsampling scheme identifiers; and determining the target resampling ratio and target upsampling scheme identifier corresponding to the video frame set by the ratio decision model based on the video coding reference information.

[0063] Since the number of candidate resampling ratios in relevant GOP-based RPR schemes is limited, the encoding resolution can be selected through a preset threshold rule. However, when this disclosure extends to support any number of candidate resampling ratios and simultaneously supports the dynamic selection of multiple upsampling reconstruction methods, the original fixed threshold decision mechanism is difficult to apply effectively to larger-scale joint decision spaces, which can easily lead to suboptimal encoding configuration selection, thereby affecting the overall compression performance and reconstruction quality.

[0064] To address the aforementioned issues, this embodiment, after constructing the candidate resampling rate space, further introduces a joint decision-making scheme to determine the optimal encoding configuration for the current image or image group from multiple candidate resampling rates and multiple candidate upsampling methods. Before making the rate decision, this disclosure first configures the candidate resampling rate set as follows: ,in, This can represent the i-th downsampling factor; the set of candidate upsampling schemes is... ,in, Let represent the upsampling method ID. Then, the candidate encoding / decoding scheme can be represented as: A candidate encoding / decoding scheme can represent multiple downsampling ratios and an upsampling scheme identifier for each downsampling ratio. Based on the upsampling scheme identifier, the corresponding upsampling scheme can be invoked for subsequent video decoding operations.

[0065] After determining the corresponding candidate encoding / decoding schemes based on the candidate resolutions, the pre-built initial decision model can be trained using multiple candidate resolutions and their corresponding candidate encoding / decoding schemes to obtain the scaling decision model. The scaling decision model can be a content-adaptive decision model based on machine learning or deep learning.

[0066] The video coding reference information of the video to be processed is input into the trained scaling decision model. This video coding reference information can be used to guide or influence video coding decisions, and may include, but is not limited to, video content features, coding parameter information, and historical coding statistics. Specifically, video content features may include, but are not limited to, texture complexity features, spatial information features, motion information features, frequency domain energy features, and image statistical features; coding parameter information may include, but is not limited to, quantization parameter QP, target bitrate, and coding configuration; and historical coding statistics may include, but are not limited to, reference frame distortion, prediction residual statistics, and bitrate change trends.

[0067] Continue to refer to Figure 2 In step S230, an encoding preprocessing operation is performed, which includes determining the resampling ratio for downsampling. The ratio decision model predicts the optimal resampling ratio based on the extracted features, selects the encoding resolution corresponding to the video frame set from the candidate encoding / decoding scheme set, and determines the target resampling ratio and target upsampling scheme identifier (i.e., the optimal upsampling scheme identifier) ​​corresponding to the video frame set. Through the ratio decision model, the optimal encoding configuration that best matches the current video frame set can be determined. Using the optimal encoding configuration to compress the video can reduce the encoding bitrate and improve video compression efficiency.

[0068] In one exemplary embodiment of this disclosure, step S140, which involves downsampling the video frame to be processed based on a target resampling ratio and then encoding the downsampled video frame to obtain an encoded video frame, includes: initializing encoding configuration parameters to obtain candidate video encoding parameters; determining initial video encoding parameters based on the target resampling ratio and the candidate video encoding parameters; adjusting the initial video encoding parameters to obtain target video encoding parameters; downsampling the video frame to be processed based on the target resampling ratio to obtain a downsampled video frame to be processed; and encoding the downsampled video frame to be processed based on the target video encoding parameters to obtain an encoded video frame.

[0069] Continue to refer to Figure 2 In step S230, the encoding preprocessing operation includes an initialization encoding PPS operation. Before compressing and encoding the video frame set, this embodiment can also perform calculation and initialization operations based on PPS parameters of arbitrary magnification. During the encoder initialization stage (when the current QP meets the RPR enabling threshold), the resampling magnification list containing N elements is traversed, and the following parameter construction logic is executed for each magnification factor.

[0070] Target encoding width and height calculation. Obtain the effective display size of the original image at the brightness sampling points, as shown in Formula 1: (Formula 1) in, These are the original valid image width and height, used to output the display width and display height, respectively; This refers to the width and height of the original image, i.e., the width and height of the encoded image recorded in the bitstream; These are the boundary parameters of the original scaling window, representing the number of cropped samples on the left, right, top, and bottom sides, respectively. These are the sampling units corresponding to the chroma format, representing the horizontal and vertical unit steps, respectively. For the current resampling factor... The scaled target size is calculated as shown in Formula 2: (Formula 2) in, These are the original valid image width and height, used to output the display width and display height, respectively; Scale the width of the target; Scale the height of the target; This represents the current resampling rate.

[0071] To meet the minimum code block size constraint in the coding standard, the target size needs to be aligned upwards. Let the minimum code block alignment unit be... The final encoding size is shown in Formula 3: (Formula 3) in: The actual encoded width and height written to PPS are determined to ensure that the image size meets the block division boundary requirements. The smallest coded block alignment unit; Scale the width of the target; Scale the height of the target.

[0072] Since the aligned size may be larger than the scaled target size, the corresponding window parameters are generated based on the difference between the two, as shown in Formula 4: (Formula 4) in, The actual encoded width and height written to PPS are determined to ensure that the image size meets the block division boundary requirements. Scale the width of the target; Scale the height of the target; This is the sampling unit corresponding to the chroma format; The window parameter for the x-coordinate. The window parameters are for the y-coordinate. The window parameters are then constructed. This window parameter is written to PPS as both a ConformanceWindow and a Scaling Window to identify the actual valid image area.

[0073] After generating the current resolution, iterate through all generated reference resolutions and check whether they meet the preset scale constraints, as shown in Formula 5: (Formula 5) in, This represents the maximum resolution of the sequence. For existing reference resolution; Scale the width of the target; Scale the height of the target; This refers to the actual width and height of the encoded data written to PPS. This is the minimum coding block alignment unit. This constraint limits the range of variation between different scales, avoiding resampling relationships that do not conform to coding standards. Based on the configured scaling factor array and the original video geometry, the scaling window, consistency window, and aligned luminance and chrominance sampling point width and height are automatically calculated for each arbitrary resolution, enabling resource allocation and initialization for multiple extended PPS. Based on the above initialization operations, the corresponding candidate video coding parameters for each scaling factor can be calculated. The candidate video coding parameters can be the coding parameters used in the video compression coding process.

[0074] After initialization, the initial video coding parameters are determined based on the target resampling ratio and candidate video coding parameters determined by the ratio decision model. The initial video coding parameters can be determined from the candidate video coding parameters based on the target resampling ratio, and can be used to perform compression coding on the current video frame to be processed. To further refine the quantization parameters of the video frame to be processed, this embodiment can adjust the initial video coding parameters before formal encoding to obtain the target video coding parameters. The target video coding parameters can be the coding parameters used in the actual encoding operation of the video frame to be processed.

[0075] When the encoder compresses and encodes the video frame to be processed, it can first downsample the video frame based on the target resampling rate. Downsampling reduces the image resolution and can include, but is not limited to, anti-aliasing low-pass filtering and decimation / resampling steps. After downsampling, a downsampled video frame is obtained. This downsampled video frame is then encoded based on the target video coding parameters to obtain an encoded video frame. The encoded video frame can be the video frame obtained after compression and encoding. Through these steps, fine-grained compensation of the quantization parameters can be achieved and used in the video encoding operation, further reducing the encoding bitrate while ensuring video reconstruction quality.

[0076] In one exemplary embodiment of this disclosure, parameter adjustment processing is performed on initial video coding parameters to obtain target video coding parameters, including: in response to an image-level parameter calculation instruction, determining initial luminance quantization parameters and a first image parameter set identifier corresponding to a set of video frames based on the initial video coding parameters; in response to the first image parameter set identifier being located in an index identifier sequence, determining a first target index position from the index identifier sequence; determining a luminance offset from a luminance offset list based on the first target index position; and adjusting the initial luminance quantization parameters according to the luminance offset to obtain the target luminance quantization parameters.

[0077] Before formal coding, a second-level fine-tuning operation of the quantization parameters can be performed. (Continue to refer to...) Figure 2 In step S240, an encoding parameter adjustment operation is performed. The target video encoding parameters may include target luminance quantization parameters, which can be luminance quantization parameters obtained after secondary fine-tuning of the initial luminance quantization parameters. The image-level parameter calculation instruction can be a calculation instruction used to trigger the calculation of image-level quantization parameters for the current video frame to be processed. The initial luminance quantization parameters can be luminance quantization parameters directly determined based on the initial video encoding parameters. The first image parameter set identifier can be the image parameter set identifier corresponding to the video frame set during the luminance quantization parameter adjustment process. The first target index position can be the index position of the ID of the current PPS object bound to the video frame set in the index identifier sequence during the luminance quantization parameter adjustment process.

[0078] The luminance QP offset fine-tuning operation can be implemented through the following steps: In the function that calculates the image-level QP for the set of video frames (including one or more video frames to be processed), the ID of the current PPS object bound to the video frame to be processed is read, and it is traversed to check whether it hits the preset extended PPS ID array. If it does, the corresponding array loop index position is extracted, and the corresponding luminance offset (usually an integer offset) is retrieved from the global luminance QP offset list. The luminance offset can be a constant compensation value that is directly added to the luminance pixel value of the reference frame during weighted prediction; and the luminance offset is superimposed on the initial luminance quantization parameter base QP of the video frame to be processed to achieve fine compensation of the luminance quantization parameter.

[0079] In one exemplary embodiment of this disclosure, parameter adjustment processing is performed on the initial video coding parameters to obtain the target video coding parameters. The method further includes: in response to a slice-level parameter calculation instruction, determining the initial chroma quantization parameters corresponding to the initial video coding parameters, and the second image parameter set identifier corresponding to the video frame set; in response to the second image parameter set identifier being located in the index identifier sequence, determining the second target index position from the index identifier sequence; determining the chroma offset from the luminance offset list based on the second target index position; and adjusting the initial chroma quantization parameters according to the chroma offset to obtain the target chroma quantization parameters.

[0080] The target video coding parameters may also include target colorimetric parameters, which can be obtained by fine-tuning the initial colorimetric parameters. The slice-level parameter calculation instruction can be an instruction used to trigger slice-level coding of slices in the current video frame to be processed; where a slice can be a rectangular or raster scan region in a video frame that can be decoded independently. For example, a video frame can be divided into many coding tree units (CTUs), and a slice is an ordered set of several consecutive CTUs.

[0081] The initial colorimetric parameters can be directly determined based on the initial video encoding parameters. The second image parameter set identifier can be the image parameter set identifier corresponding to the video frame set during the colorimetric parameter adjustment process. The second target index position can be the index position of the ID of the current PPS object bound to the video frame set in the index identifier sequence during the colorimetric parameter adjustment process.

[0082] The chroma QP offset fine-tuning operation can be implemented through the following steps: During slice-level encoding initialization, such as in the initEncSlice function, which contains the encoder's processing flow and logic for the slice data structure, the PPS ID corresponding to the currently active slice parameter set is also read. If it is determined that the current slice is under the jurisdiction of an extended PPS of arbitrary resolution, the chroma offset (i.e., the chroma QP increment Delta value) is applied to the mapping calculation flow corresponding to the blue color difference component (Cb) and the red color difference component (Cr) using a mapping function, thereby achieving coordinated adaptive adjustment of the chroma quantization step size.

[0083] In one exemplary embodiment of this disclosure, the video frame size parameters of the encoded video frames in the reconstructed pixel buffer and the image parameter set object list corresponding to each candidate resolution are obtained. The image parameter set object list includes at least one reference size parameter. The reference size parameter that is the same as the video frame size parameter is used as the target encoding size parameter. The image parameter set identifier and the target upsampling identifier corresponding to the target encoding size parameter are determined. The video frame set is written into the video bitstream based on the image parameter set identifier and the target upsampling identifier. The target upsampling identifier is used to instruct the decoding end to reconstruct the encoded video frames.

[0084] Continue to refer to Figure 2 After encoding is completed, in step S250, a multi-resolution bitstream output operation is performed. At the stage where encoding is complete and the reconstructed image and the final encapsulated Network Abstraction Layer (NAL) unit bitstream are written, the encoder performs the following multi-resolution matching decision: It reads the video frame size parameters from the current output reconstructed pixel buffer (PelUnitBuf). These video frame size parameters can be parameters that identify the video frame size; for example, they can be represented using YUV components or size parameters in RGB color mode. This embodiment uses the luminance Y component as an example to illustrate the video frame size parameter.

[0085] refer to Figure 4 , Figure 4 The diagram illustrates a flowchart of the encoder writing a video stream after encoding, according to an exemplary embodiment of this disclosure. In step S410, input data is acquired, which may be the luminance Y component of the reconstructed pixel buffer. In step S420, the current Y component size of the reconstructed pixel buffer is read, i.e., the actual pixel geometric width and height.

[0086] In step S430, the extended PPS object list corresponding to N arbitrary resolutions within the encoder is iterated through, and the brightness pixel width and height recorded in each PPS are compared. In step S440, the comparison result between the Y component size in the current buffer and the size of any PPS component is determined. If the Y component size in the current buffer is inconsistent with the size of any PPS component, then step S430 is continued to perform a size consistency check.

[0087] A PPS is determined to be the actual dynamic parameter set used for current image encoding only if its width and height are exactly the same as the geometric dimensions of the reconstruction buffer. In step S450, the corresponding PPSID is extracted as the syntax element identifier of the current image frame. In step S460, the upsampling scheme identifier corresponding to the PPS ID is written into the video bitstream. In step S470, the loop is exited, and the compatibility NAL unit bitstream is written. In step S480, the NAL unit bitstream is finally encapsulated. Through the above steps, in the reconstructed image writing stage, by comparing the actual geometric dimensions of the reconstruction pixel buffer with the resolution parameters of the initialized PPS, the corresponding parameter set identifier is matched and written into the video bitstream, facilitating subsequent image reconstruction by the decoder.

[0088] Furthermore, this disclosure maintains full compatibility with the decoding end, as the decoding end of existing coding frameworks only receives the encoded resolution and the original resolution, requiring no modification. This disclosure proposes a coding scheme supporting any number of candidate resampling ratios and provides a unified interface for content-adaptive resolution decision algorithms, further enhancing the scalability and flexibility of video coding systems. Testing is performed on VVC test sets (such as the JVET Class A sequence set) using standard VVC testing tools. Compared to related schemes, when QP is 22, 27, 32, 37, 42, the BD rate (Bjøntegaard-Delta Rate, BD-rate) of this disclosure for each component {Y, U, V} is: {-0.34%, 1.28%, 1.08%}; when QP is 27, 32, 37, 42, 47, the BD-rate of this disclosure for each component {Y, U, V} is: {-1.36%, 4.24%, 2.15%}; when QP is 32, 37, 42, 47, 52, the BD-rate of this disclosure for each component {Y, U, V} is: {-2.62%, 2.80%, 0.19%}.

[0089] In summary, the video processing method disclosed herein involves: acquiring a video to be processed, which includes a set of video frames and video encoding reference information for the video frames to be processed; acquiring pre-configured encoding configuration parameters, which include multiple candidate resampling rates determined based on pre-configured candidate resolutions; determining a target resampling rate corresponding to the video frame set from the multiple candidate resampling rates according to the video encoding reference information; performing downsampling processing on the video frames to be processed based on the target resampling rate; and encoding the downsampled video frames to be processed to obtain encoded video frames. On one hand, this disclosure implements a reference image resampling scheme with arbitrary resolution. By implementing encoding and decoding processing with arbitrary resampling rates, it supports fine-grained selection of encoding resolution, further reducing the encoding bitrate and improving video compression efficiency while ensuring video reconstruction quality. On the other hand, by expanding the candidate resampling rates through pre-configured candidate resolutions, it improves the flexibility and adaptability of encoding resolution selection. On the other hand, this disclosure automatically generates the corresponding encoding parameter set according to the preset magnification, and completes the decision of resampling magnification and upsampling method, adjustment of brightness and color quantization parameters, and bitstream output, so as to realize reference image resampling between arbitrary resolutions.

[0090] It should be noted that although the steps of the method in this invention are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0091] Furthermore, in this example embodiment, a video processing apparatus is also provided. (See reference...) Figure 5 The video processing device 500 may include: a video acquisition module 510, a configuration parameter acquisition module 520, a target magnification determination module 530, and a video encoding module 540.

[0092] Specifically, the video acquisition module 510 is used to acquire the video to be processed, which includes a set of video frames and video encoding reference information of the video frames to be processed; the configuration parameter acquisition module 520 is used to acquire pre-configured encoding configuration parameters, which include multiple candidate resampling ratios, and the candidate resampling ratios are determined based on pre-configured candidate resolutions; the target ratio determination module 530 is used to determine the target resampling ratio corresponding to the set of video frames from multiple candidate resampling ratios according to the video encoding reference information; and the video encoding module 540 is used to downsample the video frames to be processed based on the target resampling ratio, and then encode the downsampled video frames to obtain encoded video frames.

[0093] In one exemplary embodiment of this disclosure, the encoding configuration parameters include reference image resampling configuration parameters and an index identifier sequence. The configuration parameter acquisition module 520 includes a configuration parameter construction unit, used for: determining a candidate resolution corresponding to the video encoding operation; determining a candidate resampling ratio based on the candidate resolution; constructing reference image resampling configuration parameters based on the candidate resampling ratio; constructing a candidate image parameter set based on the resampling ratio and creating an index identifier sequence corresponding to the candidate image parameter set; and constructing encoding configuration parameters based on the reference image resampling configuration parameters and the index identifier sequence.

[0094] In one exemplary embodiment of this disclosure, the configuration parameter construction unit includes a configuration parameter construction subunit, configured to: determine the number of candidate resampling magnifications; construct a resampling magnification list based on the candidate resampling magnifications, wherein the number of candidate resampling magnifications is the same as the number of candidate resolutions; and construct a luminance offset list and a chrominance offset list corresponding to the candidate resampling magnifications, wherein the number of resampling magnifications, the number of the first element of the luminance offset list, and the number of the second element of the chrominance offset list are the same.

[0095] In one exemplary embodiment of this disclosure, the configuration parameter construction unit includes an identifier sequence construction subunit, configured to: construct an initial index identifier array based on the number of candidate resampling ratios; determine the image parameter set corresponding to each candidate resampling ratio; use the image parameter set identifier corresponding to the original resolution as a base index identifier, and determine the extended index identifier corresponding to the image parameter set of each candidate resampling ratio based on the base index identifier; and store the base index identifier and the extended index identifier into the initial index identifier array to obtain an index identifier sequence.

[0096] In one exemplary embodiment of this disclosure, the target scaling factor determination module 530 includes a target scaling factor determination unit, configured to: acquire a pre-trained scaling factor decision model, the scaling factor decision model being trained based on candidate video coding feature information and candidate encoding / decoding schemes corresponding to multiple candidate resolutions, the candidate encoding / decoding schemes including candidate downsampling scaling factor and matching candidate upsampling scheme identifiers; and determine the target resampling scaling factor and target upsampling scheme identifier corresponding to the video frame set by the scaling factor decision model based on video coding reference information.

[0097] In one exemplary embodiment of this disclosure, the video frame set includes at least one video frame to be processed, and the video encoding module 540 includes a video encoding unit, configured to: initialize encoding configuration parameters to obtain candidate video encoding parameters; determine initial video encoding parameters based on a target resampling ratio and candidate video encoding parameters; perform parameter adjustment processing on the initial video encoding parameters to obtain target video encoding parameters; perform downsampling processing on the video frame to be processed based on the target resampling ratio to obtain a downsampled video frame to be processed; and perform encoding processing on the downsampled video frame to be processed based on the target video encoding parameters to obtain an encoded video frame.

[0098] In one exemplary embodiment of this disclosure, the target video coding parameters include target luminance quantization parameters, and the video coding unit includes a first parameter adjustment subunit, configured to: in response to an image-level parameter calculation instruction, determine an initial luminance quantization parameter and a first image parameter set identifier corresponding to a set of video frames based on the initial video coding parameters; in response to the first image parameter set identifier being located in an index identifier sequence, determine a first target index position from the index identifier sequence; determine a luminance offset from a luminance offset list based on the first target index position, and adjust the initial luminance quantization parameter according to the luminance offset to obtain the target luminance quantization parameter.

[0099] In one exemplary embodiment of this disclosure, the target video coding parameters include target chromaticity parameters, and the video coding unit further includes a second parameter adjustment subunit, configured to: in response to a slice-level parameter calculation instruction, determine the initial chromaticity parameters corresponding to the initial video coding parameters, and the second image parameter set identifier corresponding to the video frame set; in response to the second image parameter set identifier being located in the index identifier sequence, determine the second target index position from the index identifier sequence; determine the chromaticity offset from the luminance offset list based on the second target index position, and adjust the initial chromaticity parameters according to the chromaticity offset to obtain the target chromaticity parameters.

[0100] In one exemplary embodiment of this disclosure, the video processing apparatus 500 further includes a bitstream writing module, configured to: obtain video frame size parameters of encoded video frames in the reconstructed pixel buffer, and a list of image parameter set objects corresponding to each candidate resolution, the list of image parameter set objects including at least one reference size parameter; use the reference size parameter that is the same as the video frame size parameter as the target encoding size parameter; determine the image parameter set identifier and the target upsampling identifier corresponding to the target encoding size parameter; and write the video frame set into the video bitstream based on the image parameter set identifier and the target upsampling identifier, the target upsampling identifier being used to instruct the decoding end to reconstruct the encoded video frames.

[0101] The specific details of the virtual modules of each video processing device mentioned above have been described in detail in the corresponding video processing methods. For any undisclosed details, please refer to the implementation methods in the method section, and therefore will not be repeated here.

[0102] It should be noted that although several modules or units of the video processing apparatus have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0103] Exemplary embodiments of this disclosure also provide a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the video processing method described above.

[0104] In one implementation, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing the computer program. (See reference...) Figure 6 , Figure 6 The schematic diagram illustrates a computer-readable storage medium 600 according to an exemplary embodiment of the present disclosure. The computer-readable storage medium 600 can be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory, hard disk drive (HDD), solid-state drive (SSD), etc. Exemplarily, a computer program product can be implemented as a non-volatile storage medium storing a computer program, such as read-only memory, NAND flash memory, etc.

[0105] In one implementation, the computer program product can be an intangible product containing a computer program. For example, the computer program product can be implemented as a virtual digital product, such as an executable file, installation package, or other digital file storing the computer program.

[0106] Computer program code can be written in one or more programming languages. Examples of programming languages ​​include C, Java, and C++. Program code can execute entirely on the user's computing device, partially on the user's computing device, or as a standalone software package. It can also execute partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, such as a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via an internet connection provided by a mobile network operator).

[0107] Computer programs can be carried or transmitted via signals such as electrical, magnetic, optical, electromagnetic, and infrared rays. Electronic devices can convert the signals carrying computer programs into digital signals, thereby running the computer programs. When a computer program runs on an electronic device, its code is used to cause the electronic device to execute (more specifically, to execute) the method steps of various exemplary embodiments of this disclosure, such as the video processing method described above.

[0108] Exemplary embodiments of this disclosure also provide an electronic device, which may include a processor and a memory. The memory stores executable instructions of the processor, such as a computer program. The processor executes the executable instructions to perform the method steps of various exemplary embodiments of this disclosure. Furthermore, the electronic device may also include a display for displaying a graphical user interface.

[0109] The following is for reference. Figure 7 The electronic device is illustrated by way of a general-purpose computing device. It should be understood that... Figure 7 The electronic device 700 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0110] like Figure 7 As shown, the electronic device 700 may include: a processor 710, a memory 720, a bus 730, an I / O (input / output) interface 740, a network adapter 750, and a display 760.

[0111] The memory 720 may include volatile memory, such as RAM 721 and cache unit 722, and may also include non-volatile memory, such as ROM 723. The memory 720 may also include one or more program modules 724, including but not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. For example, program module 724 may include the modules described above.

[0112] The processor 710 may include one or more processing units, such as an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor, and / or an NPU (Neural-Network Processing Unit).

[0113] The processor 710 can be used to execute executable instructions stored in the memory 720, such as the video processing method described above.

[0114] Bus 730 is used to connect different components of electronic device 700 and may include a data bus, an address bus and a control bus.

[0115] Electronic device 700 can communicate with one or more external devices 800 (such as keyboard, mouse, external controller, etc.) through I / O interface 740.

[0116] Electronic device 700 can communicate with one or more networks via network adapter 750. For example, network adapter 750 can provide mobile communication solutions such as 3G / 4G / 5G, or wireless communication solutions such as wireless LAN, Bluetooth, and near-field communication. Network adapter 750 can communicate with other modules of electronic device 700 via bus 730.

[0117] The electronic device 700 can display a graphical user interface via a display 760, such as a display interface showing the original video to be processed and the video content after compression, transmission and decoding.

[0118] although Figure 7 As not shown in the diagram, other hardware and / or software modules may also be configured in the electronic device 700, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0119] As can be seen from the above, the technical solutions disclosed herein can be implemented as methods, apparatus, systems, computer program products, storage media, electronic devices, etc. Those skilled in the art will understand that various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which may be referred to as "circuit," "module," or "system," respectively.

[0120] It should be understood that this disclosure is not limited to the specific methods, steps, or structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. Those skilled in the art will readily conceive of other embodiments based on the specific implementations provided in this disclosure. Therefore, the specific implementations provided in this disclosure are merely exemplary, and the scope and spirit of this disclosure are indicated by the claims, and should cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary technical means in the art not disclosed in this disclosure.

Claims

1. A video processing method, characterized in that, include: Obtain the video to be processed, which includes a set of video frames and video encoding reference information of the video frames to be processed; Obtain pre-configured encoding configuration parameters, which include multiple candidate resampling ratios, and the candidate resampling ratios are determined based on pre-configured candidate resolutions; Based on the video coding reference information, the target resampling factor corresponding to the video frame set is determined from the plurality of candidate resampling factors; The video frame to be processed is downsampled based on the target resampling ratio, and then encoded to obtain an encoded video frame.

2. The method according to claim 1, characterized in that, The process of obtaining pre-configured encoding configuration parameters includes: Determine the candidate resolution corresponding to the video encoding operation, and determine the candidate resampling ratio based on the candidate resolution; Construct reference image resampling configuration parameters based on the candidate resampling magnification; A candidate image parameter set is constructed based on the candidate resampling ratio, and an index identifier sequence corresponding to the candidate image parameter set is created; The encoding configuration parameters are constructed based on the reference image resampling configuration parameters and the index identifier sequence.

3. The method according to claim 2, characterized in that, The step of constructing reference image resampling configuration parameters based on the candidate resampling ratio includes: Determine the number of candidate resampling ratios, and construct a resampling ratio list based on the candidate resampling ratios, wherein the number of candidate resampling ratios is the same as the number of candidate resolutions; Construct a luminance offset list and a chrominance offset list corresponding to the candidate resampling ratios, wherein the number of candidate resampling ratios, the number of the first element in the luminance offset list, and the number of the second element in the chrominance offset list are the same.

4. The method according to claim 2, characterized in that, The step of constructing a candidate image parameter set based on the candidate resampling ratio and creating an index identifier sequence corresponding to the candidate image parameter set includes: Construct an initial index identifier array based on the number of candidate resampling rates; Determine the image parameter set corresponding to each candidate resampling factor; The image parameter set identifier corresponding to the original resolution is used as the base index identifier, and the extended index identifier corresponding to the image parameter set of each candidate resampling factor is determined according to the base index identifier. The base index identifier and the extended index identifier are stored in the initial index identifier array to obtain the index identifier sequence.

5. The method according to claim 1, characterized in that, The step of determining the target resampling factor corresponding to the video frame set from the plurality of candidate resampling factors based on the video coding reference information includes: A pre-trained scaling factor decision model is obtained, which is trained based on candidate video coding feature information and multiple candidate encoding / decoding schemes corresponding to the candidate resolutions. The candidate encoding / decoding schemes include candidate downsampling scaling factors and matching candidate upsampling scheme identifiers. The target resampling ratio and target upsampling scheme identifier corresponding to the set of video frames are determined by the ratio decision model based on the video coding reference information.

6. The method according to claim 1, characterized in that, The video frame set includes at least one video frame to be processed. The process of downsampling the video frame to be processed based on the target resampling ratio, and then encoding the downsampled video frame to obtain an encoded video frame, includes: The encoding configuration parameters are initialized to obtain candidate video encoding parameters; Based on the target resampling ratio and the candidate video coding parameters, the initial video coding parameters are determined; The initial video encoding parameters are adjusted to obtain the target video encoding parameters; The video frame to be processed is downsampled based on the target resampling ratio to obtain the video frame to be processed after the downsampling process. Based on the target video coding parameters, the downsampled video frame to be processed is encoded to obtain the encoded video frame.

7. The method according to claim 6, characterized in that, The target video coding parameters include target luminance quantization parameters. The step of adjusting the initial video coding parameters to obtain the target video coding parameters includes: In response to the image-level parameter calculation instruction, the initial luminance quantization parameters and the first image parameter set identifier corresponding to the video frame set are determined based on the initial video coding parameters. In response to the first image parameter set identifier being located in the index identifier sequence, a first target index position is determined from the index identifier sequence; Based on the first target index position, the brightness offset is determined from the brightness offset list, and the initial brightness quantization parameter is adjusted according to the brightness offset to obtain the target brightness quantization parameter.

8. The method according to claim 6, characterized in that, The target video coding parameters include target colorimetric parameters. The step of adjusting the initial video coding parameters to obtain the target video coding parameters further includes: In response to the slice-level parameter calculation instruction, the initial color quantization parameters corresponding to the initial video coding parameters and the second image parameter set identifier corresponding to the video frame set are determined. In response to the second image parameter set identifier being located in the index identifier sequence, the second target index position is determined from the index identifier sequence; Based on the second target index position, the chromaticity offset is determined from the luminance offset list, and the initial chromaticity quantization parameter is adjusted according to the chromaticity offset to obtain the target chromaticity quantization parameter.

9. The method according to claim 1, characterized in that, The method further includes: Obtain the video frame size parameters of the encoded video frame in the reconstructed pixel buffer, and the image parameter set object list corresponding to each candidate resolution, wherein the image parameter set object list includes at least one reference size parameter; The reference size parameter, which is the same as the video frame size parameter, is used as the target encoding size parameter; Determine the image parameter set identifier and target upsampling identifier corresponding to the target encoding size parameter; The video frame set is written into the video bitstream based on the image parameter set identifier and the target upsampling identifier, wherein the target upsampling identifier is used to instruct the decoding end to reconstruct the encoded video frames.

10. A video processing apparatus, characterized in that, include: The video acquisition module is used to acquire a video to be processed, the video to be processed including a set of video frames, the set of video frames including video encoding reference information of the video frames to be processed; The configuration parameter acquisition module is used to acquire pre-configured encoding configuration parameters, which include multiple candidate resampling ratios, and the candidate resampling ratios are determined based on pre-configured candidate resolutions. The target resampling ratio determination module is used to determine the target resampling ratio corresponding to the video frame set from the plurality of candidate resampling ratios based on the video coding reference information; The video encoding module is used to downsample the video frame to be processed based on the target resampling ratio, and to encode the downsampled video frame to obtain an encoded video frame.

11. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the video processing method as described in any one of claims 1 to 9.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the video processing method according to any one of claims 1 to 9.

Images