Quantization method, device and equipment of video model and medium

Abstract

The present disclosure relates to a video model quantization method, device, equipment and medium, and belongs to the technical field of video models. The method comprises: determining an optimal quantization scale of a constructed model based on a preset finite difference operator to determine a time sequence tangential distortion between a reference model and the constructed model; determining an output fitting error of the reference model and the constructed model based on the optimal quantization scale to obtain an optimized rounding variable; obtaining quantized weights of the constructed model based on the optimal quantization scale, the optimized rounding variable and floating-point weights of the reference model; and obtaining the constructed model after quantization according to the optimal quantization scale, the optimized rounding variable and the quantized weights.

Description

Technical Field

[0001] This disclosure relates to the technical field of video models, and more specifically, to a method, apparatus, device, and medium for quantizing video models. Background Technology

[0002] With the development of generative artificial intelligence, diffusion models have become an important technical approach for image and video generation. In recent years, the Transformer architecture has further incorporated diffusion models, forming generative frameworks such as Video Diffusion Transformer. These models can generate video content with continuous motion, scene changes, and multi-frame semantic relationships based on text prompts, showing promising application prospects in the field of high-quality video generation. Video data contains both spatial and temporal dimensions, requiring the processing of longer spatiotemporal token sequences. Furthermore, diffusion models typically require multi-step reverse denoising during the inference phase, with each step involving a complete network computation. To reduce deployment costs, post-training quantization has gradually become an important technique for efficient inference in video generation models like Video Diffusion. However, existing benchmark video inference models, when constructed using post-training quantization, struggle to fully maintain the dynamic evolution of the video, leading to texture flicker, motion drift, and decreased scene consistency under low-bit inference conditions. Summary of the Invention

[0003] One objective of this disclosure is to provide a new technical solution for the quantization of video models.

[0004] According to a first aspect of this disclosure, a method for quantizing a video model is provided, the method comprising: Based on the temporal tangential distortion between the baseline model and the constructed model determined by the preset finite difference operator, the optimal quantization scale of the constructed model is determined. Based on the optimal quantization scale, the output fitting error of the benchmark model and the constructed model is determined, and the optimized rounding variable is obtained; Based on the optimal quantization scale, the optimized rounding variable, and the floating-point weights of the baseline model, the quantization weights of the constructed model are obtained. The quantized model is obtained based on the optimal quantization scale, the optimized rounding variable, and the quantization weight.

[0005] Optionally, before determining the optimal quantization scale of the constructed model based on the temporal tangential distortion between the baseline model and the constructed model determined by the preset finite difference operator, the method further includes: The first temporal cut vector of the target video sample is based on the preset finite difference operator and the benchmark model; The second temporal cut vector of the target video sample is based on the finite difference operator and the constructed model; Based on the difference between the first temporal tangent vector and the second temporal tangent vector, the temporal tangential distortion between the baseline model and the constructed model is obtained.

[0006] Optionally, determining the optimal quantization scale of the constructed model based on the temporal tangential distortion between the baseline model and the constructed model determined by the preset finite difference operator includes: Based on the first loss term constructed from the temporal tangential distortion and the second loss term representing the state reconstruction loss, the tangential joint loss is obtained; The optimal quantization scale for the constructed model is obtained by minimizing the tangential joint loss on a preset calibration set using a pass-through estimator.

[0007] Optionally, determining the output fitting error of the baseline model and the constructed model based on the optimal quantization scale to obtain the optimized rounding variable includes: Obtain the binary regularized term constructed from learnable rounding variables; Determine the quantization output of the constructed model under the optimal quantization scale and the learnable rounding variable; Based on the baseline output of the baseline model, the output fitting error of the quantized output, and the binarization regularization term under set weights, the optimized rounding variable is obtained.

[0008] Optionally, obtaining the binary regularization term constructed from learnable rounding variables includes: The learnable rounding variable is input into a preset activation function to obtain a controlled rounding variable; The controlled rounding variable is input into the binary regularization term function to obtain the binary regularization term.

[0009] Optionally, after obtaining the optimized rounding variable based on the benchmark output of the benchmark model, the output fitting error of the quantized output, and the binarization regularization term under set weights, the method further includes: Based on the optimized rounding variables, a set of continuous rounding variables is obtained; A hardening operation is performed on the set of continuous rounding variables to obtain the final rounding decision set, which serves as the hardened optimized rounding variable.

[0010] Optionally, obtaining the quantization weights of the constructed model based on the optimal quantization scale, the optimized rounding variable, and the floating-point weights of the baseline model includes: The optimal quantization scale, the optimized rounding variable, and the floating-point weights of the baseline model are input into a preset quantization mapping function to obtain the quantization weights of the constructed model.

[0011] According to a second aspect of this disclosure, a quantization apparatus for a video model is also provided, the apparatus comprising: The first determining module is used to determine the optimal quantization scale of the constructed model based on the temporal tangential distortion between the baseline model determined by the preset finite difference operator and the constructed model. The second determining module is used to determine the output fitting error of the benchmark model and the constructed model based on the optimal quantization scale, and to obtain the optimized rounding variable; The first obtaining module is used to obtain the quantization weights of the constructed model based on the optimal quantization scale, the optimized rounding variable, and the floating-point weights of the baseline model. The second obtaining module is used to obtain the quantized construction model based on the optimal quantization scale, the optimized rounding variable, and the quantization weight.

[0012] According to a third aspect of this disclosure, an electronic device is also provided, including a memory and a processor, the memory being used to store a computer program; the processor being used to execute the computer program to implement the method according to a first aspect of this disclosure.

[0013] According to a fourth aspect of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the method described according to a first aspect of this disclosure.

[0014] According to a fifth aspect of this disclosure, a computer program product is also provided, including a computer program that, when executed by a processor, implements the method described according to a first aspect of this disclosure.

[0015] One beneficial effect of the embodiments of this disclosure is that the quantization method of the video model provided by the present invention can determine the optimal quantization scale by bringing the temporal tangential distortion before the quantization scale, optimize the rounding variables separately based on the optimal quantization scale, and then output a low-bit quantized model with good dynamic preservation capability and high output fidelity through the optimal quantization scale, optimized rounding variables and quantization weights.

[0016] Other features and advantages of the embodiments of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with their description, serve to explain the principles of the embodiments of the present disclosure.

[0018] Figure 1 This is a flowchart illustrating a quantization method for a video model according to one embodiment; Figure 2 This is a block diagram of a quantization device for a video model according to one embodiment; Figure 3 This is a schematic diagram of the hardware structure of an electronic device according to one embodiment. Detailed Implementation

[0019] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the parts and steps set forth in these embodiments do not limit the scope of the invention.

[0020] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0021] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0022] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0023] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0024] <Method Implementation> Figure 1 This is a flowchart illustrating a video model quantization method according to one embodiment. The implementer is an electronic device with data processing capabilities, such as a personal computer, server, or dedicated computing device.

[0025] like Figure 1 As shown, the quantization method for the video model in this embodiment may include the following steps S110 to S140: Step S110: Based on the temporal tangential distortion between the baseline model and the constructed model determined by the preset finite difference operator, determine the optimal quantization scale of the constructed model.

[0026] In this embodiment, the baseline model This refers to the original, full-precision video diffusion Transformer model, such as the Open-Sora model, which serves as a reference for the teacher model in subsequent quantization processes. Model construction. This refers to the student model to be low-bit quantized. Temporal tangential distortion is a key concept used to quantize the differences between the baseline model and the constructed model in describing the dynamic changes between video frames. Specifically, it calculates the variation of the denoised response output by the model along the video frame dimension (i.e., the temporal tangential vector) using a finite difference operator and compares the deviations of the baseline model and the constructed model in this variation. The optimal quantization scale is the quantization step size parameter that minimizes this temporal tangential distortion; it determines the spacing and dynamic range of the low-bit quantization lattice.

[0027] In this embodiment, the dynamic consistency requirement for video generation is brought forward to the determination of the quantization scale. By minimizing temporal tangential distortion, the quantization scale no longer serves only the reconstruction of a single frame state, but prioritizes ensuring that the low-bit discrete space can express key inter-frame dynamic changes, thereby avoiding the compression or loss of motion-sensitive information in the early stages of quantization.

[0028] In some examples, steps S210 to S230 are included before step S110: Step S210: The first temporal cut vector of the target video sample is obtained based on the preset finite difference operator and the benchmark model.

[0029] Step S220: Based on the finite difference operator and the constructed model, the second temporal cut vector of the target video sample is obtained.

[0030] Step S230: Based on the difference between the first temporal tangent vector and the second temporal tangent vector, the temporal tangential distortion between the baseline model and the constructed model is obtained.

[0031] Specifically, for calibration sets For the i-th sample, first obtain the complete denoised response of the baseline model (teacher model). And obtain the complete denoised response of the constructed model (quantization model). Both are organized according to the video frame order. Then, a finite difference operator D is applied along the video frame dimension to obtain the temporal tangent vector of the baseline model. and the temporal tangent vector for building the model Finally, temporal tangential distortion is characterized as the square of the L2 norm of the difference between the two, i.e. This distortion metric directly reflects the degree of deviation between the quantization model and the baseline model in the direction of inter-frame dynamic evolution.

[0032] In these examples, by introducing temporal cut vectors and finite difference operators, the quantization constraints are extended from static state reconstruction to dynamic inter-frame evolution preservation, providing a direct and effective dynamic loss basis for subsequent optimization of quantization scale.

[0033] In some embodiments, step S110 may include the following steps S310 and S320: Step S310: Based on the first loss term constructed from the temporal tangential distortion and the second loss term representing the state reconstruction loss, the tangential joint loss is obtained.

[0034] Step S320: Minimize the tangential joint loss on the preset calibration set using a pass-through estimator to obtain the optimal quantization scale for the constructed model.

[0035] In this embodiment, a total loss function consisting of two parts is first constructed. The first part is the temporal tangential preservation loss. The aforementioned timing tangential distortion is expressed as follows:

[0036] The second part is the state reconstruction loss. This is used to constrain the quantized model to maintain a close approximation of the complete denoised output of the baseline model. The weighted combination of the two yields the tangential joint loss, expressed as follows:

[0037] Expanded to:

[0038] in, This is a weighting parameter used to balance dynamic preservation and state reconstruction (e.g., set to 50).

[0039] Constructing the initial set of quantization scales: .in, Indicates the number of layers to be quantized. Indicates the first Quantization scale parameters of the layer.

[0040] First, the calibration set... Perform forward inference and cache the denoised response of the full-precision teacher model:

[0041] Then, the finite difference of the teacher's response is calculated along the frame dimension:

[0042] The finite difference is used to describe the dynamic changes between frames of the sample.

[0043] In determining the optimal quantization scale Δ, the rounding rule is fixed as nearest neighbor rounding, with the objective of minimizing the tangential joint loss on the preset calibration set. Since the rounding function in the quantization operation is non-differentiable, a pass-through estimator is used for backpropagation to finally obtain the optimal quantization scale Δ*, expressed as follows:

[0044] In these examples, by constructing a joint loss that incorporates both state and dynamic constraints and by using a pass-through estimator to address the non-differentiability problem, end-to-end optimization of the quantization scale is achieved. This enables the final low-bit quantization lattice to maintain the fidelity of single-frame output while effectively representing the motion trends between video frames.

[0045] For example, the quantization scale determines the spacing and dynamic range of low-bit lattice points. If the lattice is too coarse, true inter-frame dynamics may occur. After quantization, it is compressed into no change. This can lead to motion collapse or stiffness between frames. Therefore, advancing dynamic representability to the quantization scale learning stage through this step, allowing low-bit lattices to preferentially adapt to the direction of changes between video frames, is key to solving the state-dynamic mismatch problem.

[0046] Step S120: Based on the optimal quantization scale, determine the output fitting error of the benchmark model and the constructed model to obtain the optimized rounding variable.

[0047] In this embodiment, after obtaining the optimal quantization scale Δ* through step S310, the quantization scale and the quantization lattice structure determined by it are fixed and no longer changed. The output fitting error refers to the difference between the quantized output of the constructed model under this fixed quantization lattice and the original floating-point output of the benchmark model, typically measured by mean square error. The optimized rounding variables are a set of learnable parameters used to control whether each weight element is rounded up or down when quantized to a lattice point. The goal of this step is to find an optimal set of rounding variables by minimizing the output fitting error, thereby restoring the denoised output accuracy of the constructed model as much as possible within a fixed low-bit representation space.

[0048] The design principle of this step is: after solving the problem of "whether the video dynamics can be expressed" in step S310, this step focuses on solving the problem of "whether the output can approximate the full-precision model as much as possible after fixing the dynamic lattice", that is, to achieve the restoration of noise reduction fidelity.

[0049] In some examples, step S120 above includes steps S410 to S430 as follows: Step S410: Obtain the binary regularization term constructed from the learnable rounding variables.

[0050] Step S420: Determine the quantization output of the constructed model under the optimal quantization scale and the learnable rounding variable.

[0051] Step S430: Based on the benchmark output of the benchmark model and the output fitting error of the quantized output, and the binarization regularization term under set weights, the optimized rounding variable is obtained.

[0052] Specifically, firstly, learnable rounding variables are introduced for each weight layer to be quantized. ,in, This represents the total number of weight layers to be quantized. Indicates the first One weight layer to be quantized; This indicates the learnable rounding variable corresponding to this layer. The dimension is matched with the weights to be quantized in that layer. Then, an objective function is constructed that includes the output fitting error and a binarization regularization term. The expression for the output fitting error is as follows:

[0053] in, It is the output of the baseline model. This is the quantized output of the model built on a fixed scale Δ and a learnable rounding variable A. λ_round is the weight set by the rounding regularization term. The binarization regularization term Ω(A) is used to gradually push the rounding variable closer to discrete 0 or 1, so as to obtain a definite rounding decision after optimization.

[0054] In these examples, by introducing a binarized regularization term into the objective function, the rounding direction can be continuously learned during the optimization process and eventually converge to a discrete rounding decision. Compared to fixed or heuristic rounding strategies, this approach can more flexibly and effectively approximate the output of the baseline model on a given quantitative lattice.

[0055] In some embodiments, step 410 may include the following steps S510 and S520: Step S510: Input the learnable rounding variable into a preset activation function to obtain a controlled rounding variable.

[0056] Step S520: Input the controlled rounding variable into the binarization regularization term function to obtain the binarization regularization term.

[0057] Specifically, for a learnable rounding variable First, through a continuous relaxation function This controls the rounding direction, where σ(·) is the Sigmoid function. , This function will learn rounding variables. This is mapped to a continuous, controlled rounding variable between 0 and 1. The controlled rounding variable is then input into a binary regularization function, as shown in the following expression:

[0058] The binarization regularization term Ω(A) is used to penalize intermediate values ​​that are neither 0 nor 1, thereby driving... Approaching 0 or 1.

[0059] In these examples, discrete rounding decisions are transformed into differentiable continuous optimization problems through continuous relaxation functions, and combined with specialized regularization terms, gradient-based optimization methods can effectively learn rounding variables and ultimately obtain a clear rounding direction.

[0060] In some embodiments, after step S430, the method further includes the following steps S610 and S620: Step S610: Based on the optimized rounding variables, obtain a set of continuous rounding variables.

[0061] Step S620: Perform a hardening operation on the set of continuous rounding variables to obtain the final rounding decision set, which is used as the hardened optimized rounding variable.

[0062] Specifically, after optimizing the rounding variables, a set of optimized continuous rounding variables is obtained. To obtain the discrete rounding decisions required for practical quantization, it is necessary to harden the continuous variables, for example, by using indicator functions. When the optimized continuous variable When ≥0, it indicates that the final rounding decision is to round up (or map to 1); otherwise, it is to round down (or map to 0). For all Perform this operation to obtain the final discrete rounding decision set. ,Right now 。

[0063] In these examples, the continuous variables obtained from optimization are converted into actual, discrete rounding decisions through hardening operations, ensuring the determinism and executability of the quantization process and completing the transformation from learnable parameters to fixed model parameters.

[0064] For example, after fixing the dynamic lattice in this step, the output can approximate the full-precision model as closely as possible. Therefore, this step does not change the temporal geometry again, but instead finds the optimal projection weights on the existing tangential sensing lattice to restore the residual denoising fidelity.

[0065] Step S130: Based on the optimal quantization scale, the optimized rounding variable, and the floating-point weights of the baseline model, the quantization weights of the constructed model are obtained.

[0066] In this embodiment, after determining the optimal quantization scale And the hardened optimized rounding variables Then, the original full-precision floating-point weights of the baseline model can be... Mapped to by On the determined low-bit quantization lattice points. This mapping process, i.e., the quantization operation, results in the final low-bit quantization weights for constructing the model. .

[0067] In some embodiments, step S130 may include the following step S710: Step S710: The optimal quantization scale, the optimized rounding variable, and the floating-point weights of the baseline model are input into a preset quantization mapping function to obtain the quantization weights of the constructed model.

[0068] Specifically, the expression for the quantization mapping function Q(·) is as follows:

[0069] The quantization mapping function Q(·) is based on the optimized rounding variable. Instructions (such as rounding up or down) will assign weights to each floating-point number. Mapped to the optimal quantization scale The closest value at the discrete quantized lattice point.

[0070] In these examples, a unified quantization mapping function is used to combine the two key parameters optimized in the early stage (scale and rounding decision) with the original weights to directly generate the final low-bit quantization weights, ensuring the consistency and reproducibility of the quantization process.

[0071] Step S140: Based on the optimal quantization scale, the optimized rounding variable, and the quantization weight, the quantized construction model is obtained.

[0072] In this embodiment, the optimal quantization scale Δ obtained in step S310, the optimized rounding variable obtained in step S320 (hardened to A_hard), and the quantization weight obtained in step S330 are used. These parameters, as the final configuration parameters for building the model, are applied to the original model (i.e., the student network). Specifically, the original floating-point weights of the model are replaced with quantized weights. And in subsequent inference calculations, the optimal quantization scale is used. The activation values ​​are then quantized. This yields a complete, low-bit quantization model that can be directly used for efficient video generation inference.

[0073] As can be seen from steps S310 to S340 above, this embodiment first determines the optimal quantization scale using temporal tangential distortion, prioritizing the ability of the low-bit quantization lattice to express dynamic changes between video frames; then, based on this optimal scale, the output fitting error is minimized by optimizing rounding variables, maximizing the fidelity of the denoised output while maintaining a fixed dynamic representation capability; finally, quantization weights are generated based on the optimized scale and rounding variables to obtain the final quantization model. This process decouples and optimizes "dynamic structure preservation" and "output accuracy recovery" in stages, effectively solving the problem of mismatch between state reconstruction and dynamic evolution in existing technologies, and significantly improving the temporal consistency and output quality of the low-bit video generation model.

[0074] <Equipment Example 1> Figure 2 This is a schematic block diagram of a quantization device for a video model according to one embodiment. Figure 2 As shown, the quantization device 200 for the video model may include: The first determining module 210 is used to determine the optimal quantization scale of the constructed model based on the temporal tangential distortion between the baseline model determined by the preset finite difference operator and the constructed model. The second determining module 220 is used to determine the output fitting error of the benchmark model and the constructed model based on the optimal quantization scale, and obtain the optimized rounding variable; The first obtaining module 230 is used to obtain the quantization weights of the constructed model based on the optimal quantization scale, the optimized rounding variable, and the floating-point weights of the baseline model. The second module 240 is used to obtain the quantized construction model based on the optimal quantization scale, the optimized rounding variable, and the quantization weight.

[0075] In some embodiments, the apparatus further includes a third obtaining module, configured to: obtain a first temporal tangent vector of the target video sample based on a preset finite difference operator and the baseline model; obtain a second temporal tangent vector of the target video sample based on the finite difference operator and the constructed model; and obtain the temporal tangential distortion between the baseline model and the constructed model based on the difference between the first temporal tangent vector and the second temporal tangent vector.

[0076] In some embodiments, the first determining module 210 is further configured to obtain a tangential joint loss based on a first loss term constructed from the temporal tangential distortion and a second loss term representing the state reconstruction loss; and to obtain the optimal quantization scale of the constructed model by minimizing the tangential joint loss on a preset calibration set through a pass-through estimator.

[0077] In some embodiments, the second determining module 220 is further configured to obtain a binarization regularization term constructed from learnable rounding variables; determine the quantization output of the constructed model under the optimal quantization scale and the learnable rounding variables; and obtain optimized rounding variables based on the benchmark output of the benchmark model, the output fitting error of the quantization output, and the binarization regularization term under set weights.

[0078] In some embodiments, the second determining module 220 is further configured to input the learnable rounding variable into a preset activation function to obtain a controlled rounding variable; and input the controlled rounding variable into a binarization regularization term function to obtain a binarization regularization term.

[0079] In some embodiments, the apparatus further includes a hardening module for obtaining a set of continuous rounding variables based on the optimized rounding variables; and performing a hardening operation on the set of continuous rounding variables to obtain a final rounding decision set as the hardened optimized rounding variables.

[0080] In some embodiments, the first obtaining module 230 is further configured to input the optimal quantization scale, the optimized rounding variable, and the floating-point weights of the baseline model as inputs into a preset quantization mapping function to obtain the quantization weights of the constructed model.

[0081] <Equipment Example 2> Figure 3 This is a schematic diagram of the hardware structure of an electronic device according to another embodiment.

[0082] like Figure 3 As shown, the electronic device 300 includes a processor 310 and a memory 320, the memory 320 being used to store an executable computer program, and the processor 310 being used to execute methods as described in any of the above method embodiments under the control of the computer program.

[0083] Each module of the quantization device 200 for the above video model can be implemented by the processor 310 in this embodiment executing the computer program stored in the memory 320, or it can be implemented by other structures, which are not limited here.

[0084] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium loaded with computer-readable program instructions for causing a processor to implement any of the methods in the foregoing embodiments of this disclosure. A computer-readable storage medium may be a tangible device capable of holding and storing instructions used by an instruction execution device. For example, a computer-readable storage medium may include an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), compact disc-read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any combination thereof. Computer-readable storage media as used herein is not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires. The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include one or more of copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to computer-readable storage media in the respective computing / processing device.The computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object programs written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as the "C" language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network (e.g., a local area network or a wide area network), or it may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays, or programmable logic arrays, may be customized by utilizing state information from the computer-readable program instructions to execute the computer-readable program instructions, thereby implementing various aspects of the embodiments of this disclosure. Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It should be noted that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are all equivalent. Various embodiments of the present disclosure have been described above; the above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A quantization method for a video model, characterized in that, The method includes: Based on the temporal tangential distortion between the baseline model and the constructed model determined by the preset finite difference operator, the optimal quantization scale of the constructed model is determined. Based on the optimal quantization scale, the output fitting error of the benchmark model and the constructed model is determined, and the optimized rounding variable is obtained; Based on the optimal quantization scale, the optimized rounding variable, and the floating-point weights of the baseline model, the quantization weights of the constructed model are obtained. The quantized model is obtained based on the optimal quantization scale, the optimized rounding variable, and the quantization weight. Before determining the optimal quantization scale of the constructed model based on the temporal tangential distortion between the baseline model and the constructed model determined by the preset finite difference operator, the method further includes: The first temporal cut vector of the target video sample is based on the preset finite difference operator and the benchmark model; The second temporal cut vector of the target video sample is based on the finite difference operator and the constructed model; Based on the difference between the first temporal tangent vector and the second temporal tangent vector, the temporal tangential distortion between the baseline model and the constructed model is obtained; The determination of the optimal quantization scale for the constructed model, based on the temporal tangential distortion between the baseline model and the constructed model determined by the preset finite difference operator, includes: Based on the first loss term constructed from the temporal tangential distortion and the second loss term representing the state reconstruction loss, the tangential joint loss is obtained; The optimal quantization scale for the constructed model is obtained by minimizing the tangential joint loss on a preset calibration set using a pass-through estimator.

2. The method according to claim 1, characterized in that, Based on the optimal quantization scale, the output fitting error of the baseline model and the constructed model is determined to obtain optimized rounding variables, including: Obtain the binary regularized term constructed from learnable rounding variables; Determine the quantization output of the constructed model under the optimal quantization scale and the learnable rounding variable; Based on the baseline output of the baseline model, the output fitting error of the quantized output, and the binarization regularization term under set weights, the optimized rounding variable is obtained.

3. The method according to claim 2, characterized in that, The process of obtaining the binary regularized term constructed from learnable rounding variables includes: The learnable rounding variable is input into a preset activation function to obtain a controlled rounding variable; The controlled rounding variable is input into the binary regularization term function to obtain the binary regularization term.

4. The method according to claim 2, characterized in that, After obtaining the optimized rounding variable based on the benchmark output of the benchmark model, the output fitting error of the quantized output, and the binarization regularization term under set weights, the method further includes: Based on the optimized rounding variables, a set of continuous rounding variables is obtained; A hardening operation is performed on the set of continuous rounding variables to obtain the final rounding decision set, which serves as the hardened optimized rounding variable.

5. The method according to claim 1, characterized in that, The process of obtaining the quantization weights of the constructed model based on the optimal quantization scale, the optimized rounding variables, and the floating-point weights of the baseline model includes: The optimal quantization scale, the optimized rounding variable, and the floating-point weights of the baseline model are input into a preset quantization mapping function to obtain the quantization weights of the constructed model.

6. A quantization device for a video model, characterized in that, The device includes: The first determining module is used to determine the optimal quantization scale of the constructed model based on the temporal tangential distortion between the baseline model determined by the preset finite difference operator and the constructed model. The second determining module is used to determine the output fitting error of the benchmark model and the constructed model based on the optimal quantization scale, and to obtain the optimized rounding variable; The first obtaining module is used to obtain the quantization weights of the constructed model based on the optimal quantization scale, the optimized rounding variable, and the floating-point weights of the baseline model. The second obtaining module is used to obtain the quantized construction model based on the optimal quantization scale, the optimized rounding variable, and the quantization weight; The third obtaining module is used to obtain a first temporal tangent vector of the target video sample based on a preset finite difference operator and the benchmark model; a second temporal tangent vector of the target video sample based on the finite difference operator and the constructed model; and to obtain the temporal tangent distortion between the benchmark model and the constructed model based on the difference between the first temporal tangent vector and the second temporal tangent vector. The first determining module is further configured to obtain the tangential joint loss based on the first loss term constructed from the temporal tangential distortion and the second loss term representing the state reconstruction loss; and to obtain the optimal quantization scale of the constructed model by minimizing the tangential joint loss on a preset calibration set through a pass-through estimator.

7. An electronic device, characterized in that, The device includes a memory and a processor, the memory being used to store a computer program; the processor being used to execute the computer program to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images