A post-quantization method and system of a moire convolutional neural network

Abstract

The application provides a post-quantization method and system of a de-moiring convolutional neural network. The method comprises: replacing a convolutional layer in a trained de-moiring convolutional neural network with a quantized convolutional layer; performing first-stage calibration processing on each quantized convolutional layer to determine a channel smoothing coefficient of each quantized convolutional layer; performing second-stage calibration processing on each quantized convolutional layer according to the channel smoothing coefficient of each quantized convolutional layer to determine an input activation value interval of each quantized convolutional layer; performing pseudo-quantization on the input activation value of each quantized convolutional layer according to the input activation value interval; performing mixed-precision quantization processing on the weight of each quantized convolutional layer according to the channel smoothing coefficient of each quantized convolutional layer; and determining a trained post-quantized de-moiring convolutional neural network. According to the application, the quality of the de-moiring task is maintained without back propagation and retraining, and fast, efficient and accurate post-quantization of the de-moiring model is achieved.

Description

Technical Field

[0001] This application relates to the fields of deep learning and computer vision, and more specifically, to a post-quantization method and system for demoired convolutional neural networks. Background Technology

[0002] Existing demoiré convolutional neural networks often improve detail reproduction through multi-scale feature extraction and pixel rearrangement, but their high-precision floating-point inference faces challenges in low latency and low power consumption when deployed on edge devices. Traditional Quantization-Aware Training (QAT) requires retraining, which is costly and requires large amounts of data; while conventional Post-Training Quantization (PTQ) methods often face the following problems: 1) Differences in channel range: The amplitude distribution of activation and weights varies greatly across different channels, and direct MinMax quantization will produce a large error; 2) Long-tail weights: A small number of extreme weights dominate the quantization range, leading to a decrease in overall quantization accuracy; 3) Outliers in activation values ​​cause quantization errors: Determining the quantization interval based on the extreme values ​​of the full tensor is easily affected by individual outlier points; For example, in the patent "CN119047518B; Quantization method, system, electronic device and storage medium of convolutional neural network", the pre-trained CNN model is pruned first, and then the pruned model is subjected to layer-by-layer MinMax quantization. The weights are calculated using the absolute maximum and minimum method, and the activations are calculated using the histogram method. At the same time, the sensitive convolutional layers are identified by calculating the mean square error of the parameters before and after quantization, and channel-by-channel quantization and channel / spatial dimension splitting and reconstruction are performed on them. The non-sensitive layers are still quantized layer by layer, and finally mixed precision quantization is achieved to balance the model accuracy and inference efficiency. However, the layer-by-layer MinMax quantization directly used after pruning does not truncate or quantize the long-tail weights or outlier activation values, and the non-sensitive layers do not consider the difference in amplitude distribution between channels. This makes it easy for extreme values ​​to dominate the quantization range and compress the effective quantization range. The difference in distribution between channels will also aggravate the quantization error. The overall accuracy is greatly affected by the inherent problems of traditional MinMax quantization.

[0003] Therefore, there is an urgent need for a post-training quantization method that requires no training and can balance deployment efficiency and recovery quality. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the purpose of this application is to provide a post-quantization method and system for demoired convolutional neural networks.

[0005] A first aspect of this application provides a post-quantization method for demoired convolutional neural networks, comprising: Replace the convolutional layers in the trained demoired convolutional neural network with quantized convolutional layers; The first-stage calibration process is performed on each of the quantized convolutional layers using an unlabeled moiré image calibration dataset to determine the channel smoothing coefficient of each of the quantized convolutional layers. Based on the channel smoothing coefficient of each quantized convolutional layer, a second-stage calibration process is performed on each quantized convolutional layer to determine the input activation value range of each quantized convolutional layer. Based on the input activation value range of each quantized convolutional layer, pseudo-quantization is performed on the input activation value of each quantized convolutional layer to determine the activation value after pseudo-quantization. Based on the channel smoothing coefficient of each quantized convolutional layer, the weights of each quantized convolutional layer are subjected to mixed precision quantization to determine the mixed precision weights. Based on the pseudo-quantized activation values ​​and the mixed precision weights, a demoired convolutional neural network after training and quantization is determined.

[0006] Optionally, the first-stage calibration process using an unlabeled moiré image calibration dataset to each quantized convolutional layer to determine the channel smoothing coefficients of each quantized convolutional layer includes: The unlabeled moiré image calibration dataset is used to perform a preset batch of calibration processing on each of the quantized convolutional layers, and the maximum absolute activation value of each input channel of each quantized convolutional layer and the maximum absolute weight value of each input channel of each quantized convolutional layer in each batch of calibration processing are determined. For each input channel of the quantized convolutional layer in each batch of calibration processing, the maximum absolute activation value is averaged for each input channel to determine the upper bound of the activation value amplitude of each input channel of the quantized convolutional layer. For each input channel of the quantized convolutional layer in each batch of calibration processing, the average value of the maximum absolute weight is calculated for each input channel to determine the upper bound of the weight magnitude of each input channel of the quantized convolutional layer. The channel smoothing coefficient of each quantized convolutional layer in each input channel is determined by the power average of the upper bound of the activation value magnitude of each quantized convolutional layer in each input channel and the upper bound of the weight magnitude of each quantized convolutional layer in each input channel.

[0007] Optionally, the step of performing a second-stage calibration process on each quantized convolutional layer based on the channel smoothing coefficient of each quantized convolutional layer to determine the input activation value range of each quantized convolutional layer includes: The input activation value of each quantized convolutional layer in each batch of calibration is divided channel by channel smoothing coefficient of each quantized convolutional layer in each input channel to determine the channel smoothed input activation value of each quantized convolutional layer in each batch of calibration. For each quantized convolutional layer, a preset number of elements are randomly sampled from the channel-smoothed input activation values ​​in each batch of calibration processing to determine the random sampling elements; Based on the random sampling elements, determine the maximum and minimum values ​​of the input activation values ​​for each quantized convolutional layer in each batch of calibration processing; The maximum and minimum input activation values ​​of each quantized convolutional layer in each batch of calibration processing are averaged over the batch to determine the maximum and minimum input activation values ​​of each quantized convolutional layer, and to determine the input activation value range of each quantized convolutional layer.

[0008] Optionally, the step of performing pseudo-quantization on the input activation values ​​of each quantized convolutional layer based on the input activation value range of each quantized convolutional layer, and determining the activation values ​​after pseudo-quantization, includes: The input activation value range of each quantized convolutional layer is used as the quantization range of each quantized convolutional layer. The quantization stride and zero point are determined based on the quantization interval and quantization bit width of each quantized convolutional layer. Based on the quantization step size and the zero point, the input activation value of each quantized convolutional layer is converted into an integer representation, and the integer representation of the input activation value of each quantized convolutional layer is determined. The integer representation of the input activation value of each quantized convolutional layer is dequantized to determine the pseudo-quantized activation value.

[0009] Optionally, the step of performing mixed-precision quantization on the weights of each quantized convolutional layer based on the channel smoothing coefficient of each quantized convolutional layer to determine the mixed-precision weights includes: The weight of each input channel of each quantized convolutional layer in each batch of calibration processing is divided channel by channel smoothing coefficient of each quantized convolutional layer in each input channel to determine the weight of each quantized convolutional layer after channel smoothing scaling. Based on the weights of each quantized convolutional layer after channel smoothing and scaling, as well as the preset first quantile threshold and the preset second quantile threshold, the outlier and non-outlier portions of each quantized convolutional layer after channel smoothing and scaling are determined. The outlier portion is represented using full-precision floating-point representation, and the weight of the outlier portion is determined. The MinMax method is used to perform channel-by-channel quantization on the non-outlier portion to determine the quantization weight of the non-outlier portion; The mixed precision weight is determined based on the weights of the outlier components and the quantization weights of the non-outlier components.

[0010] A second aspect of this application provides a post-quantization system for demoired convolutional neural networks, comprising: The quantized convolutional layer replacement module is used to replace the convolutional layers in a trained demoired convolutional neural network with quantized convolutional layers. The first-stage calibration processing module is used to perform first-stage calibration processing on each of the quantized convolutional layers using an unlabeled moiré image calibration dataset to determine the channel smoothing coefficient of each of the quantized convolutional layers. The second-stage calibration processing module is used to perform a second-stage calibration process on each quantized convolutional layer according to the channel smoothing coefficient of each quantized convolutional layer, and to determine the input activation value range of each quantized convolutional layer. The activation value pseudo-quantization module is used to perform pseudo-quantization on the input activation value of each quantized convolutional layer according to the input activation value range of each quantized convolutional layer, and determine the activation value after pseudo-quantization. The weighted mixed precision quantization module is used to perform mixed precision quantization on the weights of each quantized convolutional layer according to the channel smoothing coefficient of each quantized convolutional layer, and determine the mixed precision weights. The post-quantization model determination module is used to determine the trained post-quantized demoired convolutional neural network based on the activation values ​​after pseudo-quantization and the mixed precision weights.

[0011] A third aspect of this application provides a method for removing moiré patterns from an image, comprising: Obtain the moiré image to be processed and the trained Efficient Scale-Diverse Network model; The trained Efficient Scale-Diverse Network model is post-quantized using the post-quantization method for demoired convolutional neural networks described in any one of the first aspects of this application, thereby determining the post-quantized Efficient Scale-Diverse Network model. The moiré image to be processed is input into the post-quantized Efficient Scale-Diverse Network model to determine the demoiré image.

[0012] A fourth aspect of this application provides a video super-resolution processing method, comprising: Acquire the video to be processed and the trained video super-resolution model; The trained video super-resolution model is post-quantized using the post-quantization method of the demoired convolutional neural network described in any one of the first aspects of this application, thereby determining the post-quantized video super-resolution model. The video to be processed is input into the post-quantized video super-resolution model to determine the video that has undergone super-resolution processing.

[0013] A fifth aspect of this application provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of any of the methods provided in the first aspect of this application.

[0014] A sixth aspect of this application provides an electronic device comprising: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of any of the methods provided in the first aspect of this application.

[0015] The post-quantization method for demoired convolutional neural networks in this application uses an unlabeled moiré image calibration dataset to perform a first-stage calibration process on each quantized convolutional layer, obtains the channel smoothing coefficient of each quantized convolutional layer, and reduces the amplitude distribution difference of activation values ​​and weights of each quantized convolutional layer in different input channels based on the channel smoothing coefficient, thereby reducing post-quantization error. In addition, pseudo-quantization is performed on the input activation values ​​in combination with the input activation value range of each quantized convolutional layer, and mixed-precision quantization is performed on the weights using the channel smoothing coefficient. This effectively reduces the bit value of the demoired convolutional neural network and effectively improves the quality of maintaining demoired texture. Moreover, the entire process only requires forward propagation calibration, without the need for labeled data or backpropagation and retraining, achieving fast, efficient and accurate low-bit post-quantization of demoired convolutional neural networks, which can be adapted to scenarios with limited time or computing power.

[0016] Other technical effects resulting from the additional features will be further illustrated in the corresponding embodiments. Attached Figure Description

[0017] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating a post-quantization method for a demoired convolutional neural network according to an exemplary embodiment.

[0018] Figure 2 This is a schematic diagram of the structure of a post-quantization system for a demoired convolutional neural network according to an exemplary embodiment. Detailed Implementation

[0019] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0020] The terms "comprising" and "having," and any variations thereof, in the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or devices.

[0021] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature.

[0022] Traditional quantization training for existing demoired convolutional neural networks requires retraining, which is costly and requires a large amount of data. Conventional post-quantization after training faces challenges such as significant differences in the amplitude distribution of activations and weights across different channels, leading to large errors when directly performing MinMax quantization; extreme weights dominating the quantization range affecting overall quantization accuracy; and the influence of outlier activation values ​​causing quantization errors. To address these issues, this application provides a post-quantization method for demoired convolutional neural networks to solve these problems.

[0023] Figure 1 This is a flowchart illustrating a post-quantization method for a demoired convolutional neural network according to an exemplary embodiment.

[0024] Reference Figure 1 As shown in one embodiment of this application, a post-quantization method for a demoired convolutional neural network is provided, including steps S11 to S16.

[0025] S11 replaces the convolutional layers in the trained demoired convolutional neural network with quantized convolutional layers.

[0026] Specifically, quantized convolutional layers support channel-wise weight quantization and tensor-wise activation quantization.

[0027] For example, the demoired convolutional neural network trained in this embodiment can be an EfficientScale-Diverse Network model, or it can be applied to other demoired convolutional neural networks.

[0028] S12, using an unlabeled moiré image calibration dataset, performs the first-stage calibration process on each quantized convolutional layer to determine the channel smoothing coefficient of each quantized convolutional layer.

[0029] For example, an unlabeled moiré image calibration dataset can use a small moiré image training set, such as the moiré image datasets UHDM or FHDMi.

[0030] The unlabeled moiré image calibration dataset is used only for forward calibration, with a batch size of 1 to approximate real-world deployments.

[0031] S13, based on the channel smoothing coefficient of each quantized convolutional layer, perform a second-stage calibration process on each quantized convolutional layer to determine the input activation value range of each quantized convolutional layer.

[0032] S14. Based on the input activation value range of each quantized convolutional layer, perform pseudo-quantization on the input activation value of each quantized convolutional layer to determine the activation value after pseudo-quantization.

[0033] S15, based on the channel smoothing coefficient of each quantized convolutional layer, perform mixed precision quantization on the weights of each quantized convolutional layer to determine the mixed precision weights.

[0034] S16, based on the pseudo-quantized activation values ​​and mixed precision weights, determine the trained and quantized demoired convolutional neural network.

[0035] In the embodiments described above, an unlabeled moiré image calibration dataset is used to perform a first-stage calibration process on each quantized convolutional layer. The channel smoothing coefficient of each quantized convolutional layer is obtained, and the amplitude distribution difference of the activation value and weight of each quantized convolutional layer between different input channels is reduced based on the channel smoothing coefficient, thereby reducing the post-quantization error. The input activation value is pseudo-quantized in combination with the input activation value range of each quantized convolutional layer, and the weight is mixed-precision quantization is performed using the channel smoothing coefficient. This effectively reduces the bit value of the demoiré convolutional neural network while effectively maintaining the demoiré quality. Furthermore, the entire process only requires forward propagation calibration, without the need for labeled data or backpropagation and retraining, achieving fast, efficient, and accurate low-bit post-quantization of the demoiré convolutional neural network, which can be adapted to scenarios with limited time or computing power.

[0036] To achieve channel-wise weight quantization and tensor-wise activation quantization, in some specific embodiments of this application, for S11, the convolutional layers in the trained demoired convolutional neural network are replaced with quantized convolutional layers.

[0037] Specifically, the quantized convolutional layer includes quantization parameters, which include weights, biases, and channel smoothing coefficients.

[0038] The weights and biases of the convolutional layers are used as the weights and biases of the quantized convolutional layers, and all other quantization parameters of the quantized convolutional layers are initialized to 0, in order to determine the quantized convolutional layers of each convolutional layer. Iterate through the convolutional layers, replacing each convolutional layer with the quantized convolutional layer corresponding to each convolutional layer.

[0039] The weights and biases of the quantized convolutional layer are the same as those of the original convolutional layer.

[0040] In the above embodiments of this application, the convolutional layers in the trained demoired convolutional neural network are replaced with quantized convolutional layers, which serve as the basis for subsequent channel-wise weight quantization and tensor-wise activation quantization.

[0041] In order to obtain the channel smoothing coefficient of each quantized convolutional layer, in some specific embodiments of this application, for S12, the first-stage calibration process of each quantized convolutional layer is performed using an unlabeled moiré image calibration dataset to determine the channel smoothing coefficient of each quantized convolutional layer, which can be implemented using S121 to S124.

[0042] S121, using an unlabeled moiré image calibration dataset, perform a preset batch of calibration processing on each quantized convolutional layer, and determine the maximum absolute activation value of each input channel of each quantized convolutional layer in each batch of calibration processing and the maximum absolute weight value of each input channel of each quantized convolutional layer in each batch of calibration processing.

[0043] Specifically, the preset batch size is set to N.

[0044] Without modifying the weights, obtain the maximum activation value of each input channel of each quantized convolutional layer in each batch of calibration processing, that is, the upper bound of the magnitude of the input activation per input channel of each quantized convolutional layer, as follows: The first The quantized convolutional layer at the ... The activation value entered during each batch of calibration processing is: ; in, Indicates the first The quantized convolutional layer at the ... Enter the activation value during each batch of calibration processing. Indicates the first The number of input channels for the input activation values ​​of a quantized convolutional layer. Indicates the first The length of the input activation value feature map of a quantized convolutional layer. Indicates the first The width of the feature map of the input activation values ​​of a quantized convolutional layer, where R represents the real number field.

[0045] For the Each quantized convolutional layer, batch Input channel c, calculate the maximum absolute activation value: ; in, Indicates the first Each quantized convolutional layer, batch The maximum absolute value of activation in input channel c. Indicates the first The length of the input activation value feature map of each quantized convolutional layer Positive integers within 1000 Indicates the first The width of the input activation value feature map of each quantized convolutional layer Positive integers within 10, |·| represents the absolute value symbol. Indicates the first The quantized convolutional layer at the ... Enter the activation value during each batch of calibration processing. In input channel c, the position coordinates are The component at that location.

[0046] Let the first The weights of each quantized convolutional layer are: ; in, Indicates the first The weights of each quantized convolutional layer Indicates the first Number of output channels of each quantized convolutional layer Indicates the first The number of input channels for the input activation values ​​of a quantized convolutional layer. Indicates the first The height of the convolution kernel of each quantized convolutional layer Indicates the first The width of the convolution kernel of each quantized convolutional layer, where R represents the real number field.

[0047] For the Each quantized convolutional layer, batch Input channel c, calculate the maximum absolute weight value: ; in, Indicates the first Each quantized convolutional layer, batch The maximum absolute weight in input channel c. Indicates the first A positive integer less than or equal to the number of output channels of a quantized convolutional layer. Indicates the first Positive integers within the height of the convolutional kernels of each quantized convolutional layer. Indicates the first Positive integers within the width of the convolution kernel of each quantized convolutional layer, where |·| represents the absolute value sign. Indicates the first Each quantized convolutional layer has an output channel of... Input channel is c The position coordinates are The component at that location.

[0048] S122, average the maximum absolute activation value of each input channel for each quantized convolutional layer in each batch of calibration processing, and determine the upper bound of the activation value amplitude of each quantized convolutional layer in each input channel.

[0049] Specifically, the maximum absolute activation value of each quantized convolutional layer in each batch of calibration processing is averaged across each input channel to obtain the upper bound of the activation value amplitude of each quantized convolutional layer in each input channel: ; in, Indicates the first The upper bound of the activation value amplitude of each quantized convolutional layer in the input channel c, where N represents the preset batch size. Indicates the first Each quantized convolutional layer, batch The maximum absolute value of activation in input channel c.

[0050] S123, average the maximum absolute value of the weight of each input channel for each quantized convolutional layer in each batch of calibration processing for each input channel, and determine the upper bound of the weight magnitude of each quantized convolutional layer in each input channel.

[0051] Specifically, the maximum absolute value of the weights of each quantized convolutional layer in each batch of calibration processing is averaged across the batch for each input channel to obtain the upper bound of the weight magnitude of each quantized convolutional layer in each input channel: ; in, Indicates the first The upper bound of the weight magnitude of each quantized convolutional layer in the input channel c. Indicates the first Each quantized convolutional layer, batch The maximum absolute weight in input channel c.

[0052] S124, determine the channel smoothing coefficient of each quantized convolutional layer in each input channel based on the power average of the upper bound of the activation value amplitude of each quantized convolutional layer in each input channel and the upper bound of the weight amplitude of each quantized convolutional layer in each input channel.

[0053] Specifically, for the first The channel smoothing coefficients for the input channels c of each quantized convolutional layer are: ; in, Indicates the first Channel smoothing coefficients of input channel c of each quantized convolutional layer Indicates the first Each quantized convolutional layer in the input channel c The upper bound of the activation value amplitude, Indicates the first The upper bound of the weight magnitude of each quantized convolutional layer in the input channel c. This represents hyperparameters.

[0054] Specifically, for Hyperparameters within the interval are used to control the trade-off between weights and activation values. For example, hyperparameters... It can be set to 0.5.

[0055] In the above embodiments of this application, a first-stage calibration process is performed on each quantized convolutional layer using an unlabeled moiré image calibration dataset. Based on the power average of the upper bound of the activation value amplitude of each quantized convolutional layer in each input channel and the upper bound of the weight amplitude of each quantized convolutional layer in each input channel, a channel smoothing coefficient of each quantized convolutional layer is obtained. This coefficient is used to smooth the activation values ​​and weights, thereby reducing the distribution differences of activation values ​​and weights in different input channels.

[0056] In order to obtain the sampling activation value range, in some specific embodiments of this application, for S13, a second-stage calibration process is performed on each quantized convolutional layer according to the channel smoothing coefficient of each quantized convolutional layer to determine the input activation value range of each quantized convolutional layer, which can be implemented using S131 to S134.

[0057] S131, divide the input activation value of each quantized convolutional layer in each batch of calibration processing by the channel smoothing coefficient of each quantized convolutional layer in each input channel, and determine the channel smoothed input activation value of each quantized convolutional layer in each batch of calibration processing.

[0058] Specifically, the first The quantized convolutional layer at the ... The activation value entered during each batch of calibration processing is: ; in, Indicates the first The quantized convolutional layer at the ... Enter the activation value during each batch of calibration processing. Indicates the first The number of input channels for the input activation values ​​of a quantized convolutional layer. Indicates the first The length of the input activation value feature map of a quantized convolutional layer. Indicates the first The width of the feature map of the input activation values ​​of a quantized convolutional layer, where R represents the real number field.

[0059] The first The quantized convolutional layer at the ... The channel-smoothed input activation value in each batch of calibration processing is: ; in, Indicates the first The quantized convolutional layer at the ... The channel-smoothed input activation values ​​in each batch of calibration processing. Indicates the first The quantized convolutional layer at the ... Enter the activation value during each batch of calibration processing. This indicates element-wise division. Indicates the first The channel smoothing coefficient of the input channel c of the quantized convolutional layer.

[0060] S132, for each quantized convolutional layer, a preset number of elements are randomly sampled from the channel-smoothed input activation values ​​in each batch of calibration processing to determine the random sampling elements.

[0061] For example, the preset number of elements can be 1e4.

[0062] S133, based on the random sampling elements, determine the maximum and minimum values ​​of the input activation values ​​for each quantized convolutional layer in each batch of calibration processing.

[0063] Based on the above steps S132 to S133, A preset number of elements are randomly sampled, and the maximum value of the input activation value of each quantized convolutional layer in each batch of calibration processing is calculated. and the minimum value of the input activation value : Compare the values ​​of each of the randomly sampled, preset number of elements one by one, and take the element with the largest value as the maximum value of the batch of input activation values, and take the element with the smallest value as the minimum value of the batch of input activation values.

[0064] S134, average the maximum and minimum input activation values ​​of each quantized convolutional layer in each batch of calibration processing over the batch to determine the maximum and minimum input activation values ​​of each quantized convolutional layer, and determine the input activation value range of each quantized convolutional layer.

[0065] Specifically, the maximum value of the input activation value for each quantized convolutional layer in each batch of calibration processing. and the minimum value of the input activation value The average value of the batch is calculated to obtain the input activation value range of each quantized convolutional layer, i.e., the sampling estimation range of the input activation value of each quantized convolutional layer. ; in, Indicates the first The maximum value of the input activation values ​​of each quantized convolutional layer. Indicates the first The minimum value of the input activation values ​​of a quantized convolutional layer. Indicates the first The quantized convolutional layer at the ... The maximum value of the input activation value in each batch of calibration processing. Indicates the first The quantized convolutional layer at the ... The minimum value of the input activation value in each batch of calibration processing.

[0066] In the above embodiments of this application, based on the channel smoothing coefficient of each quantized convolutional layer, channel smoothing is performed on the activation value of each channel of each quantized convolutional layer, and a preset number of elements are sampled to estimate the input activation value range. This can suppress the influence of outliers in activation values, and also estimate the distribution range of non-outliers in activation values, thereby reducing the impact of errors in activation value quantization.

[0067] To achieve pseudo-quantization of the input activation values, in some specific embodiments of this application, for S14, pseudo-quantization is performed on the input activation values ​​of each quantized convolutional layer according to the input activation value range of each quantized convolutional layer to determine the activation values ​​after pseudo-quantization. This can be achieved by: S141, the input activation value range of each quantized convolutional layer is used as the quantization range of each quantized convolutional layer.

[0068] Specifically, the input activation value range of each quantized convolutional layer, obtained by random sampling through steps S131 to S134 above, is used as the quantization range of each quantized convolutional layer.

[0069] For example, (1, u) is used as the quantization interval.

[0070] S142, determine the quantization step size and zero point based on the quantization interval and quantization bit width of each quantized convolutional layer.

[0071] Specifically, the quantization step size is calculated based on the quantization bit width b: ; Where s represents the quantization step size, u represents the maximum value of the input activation value, and b represents the quantization bit width.

[0072] Calculate the zero point: ; Where z represents the zero point, b represents the quantization bit width, x represents the input activation value, lb represents the left boundary of quantization, and ub represents the right boundary of quantization.

[0073] S143, based on the quantization step size and zero point, convert the input activation value of each quantized convolutional layer into an integer representation, and determine the integer representation of the input activation value of each quantized convolutional layer.

[0074] Specifically, the input activation values ​​of each quantized convolutional layer are converted into integer representations using the following formula: ; Where x_z represents the input activation value in integer form, and x represents the input activation value.

[0075] S144, dequantize the integer representation of the input activation value of each quantized convolutional layer to determine the activation value after pseudo-quantization.

[0076] Specifically, the dequantization process uses the following formula: ; Where x_q represents the activation value after pseudo-quantization, x_z represents the input activation value in integer form, z represents the zero point, and s represents the quantization step size.

[0077] In the embodiments described above, pseudo-quantization is performed on the input activation values ​​based on the input activation value range of each quantized convolutional layer to obtain low-bit quantized activation values, thereby reducing computation and storage, and thus reducing model size and bandwidth.

[0078] To achieve mixed precision quantization of weights, in some specific embodiments of this application, for S15, the weights of each quantized convolutional layer are subjected to mixed precision quantization processing based on the channel smoothing coefficient of each quantized convolutional layer to determine the mixed precision weights, which can be implemented using S151 to S155.

[0079] S151, divide the weight of each input channel of each quantized convolutional layer in each batch of calibration processing by the channel smoothing coefficient of each quantized convolutional layer in each input channel, and determine the weight of each quantized convolutional layer after channel smoothing scaling.

[0080] Specifically, the weights of each quantized convolutional layer after channel smoothing scaling are: ; in, Indicates the first The element values ​​of the weights of each convolutional layer after channel-smoothing scaling at output channel o, input channel c, and kernel position (h, w). Indicates the first The element values ​​of the convolutional layer weight tensor at output channel o, input channel c, and kernel position (h, w). Indicates the first The channel smoothing coefficient of the input channel c of the quantized convolutional layer.

[0081] By employing a channel smoothing coefficient to perform channel smoothing scaling on the weights of each quantized convolutional layer, scale matching between the weights and the input activation values ​​is achieved.

[0082] S152, based on the weights of each quantized convolutional layer after channel smoothing and scaling, and the preset first quantile threshold and the preset second quantile threshold, determine the outlier and non-outlier portions of each quantized convolutional layer after channel smoothing and scaling.

[0083] For example, the preset first quantile threshold is set to 0.5%, and the preset second quantile threshold is set to 99.5%. The portion of the weights of each quantized convolutional layer after channel smoothing and scaling that is less than the preset first quantile threshold and greater than the preset second quantile threshold is regarded as the outlier portion, and the portion of the weights of each quantized convolutional layer after channel smoothing and scaling that is not less than the preset first quantile threshold and not greater than the preset second quantile threshold is regarded as the non-outlier portion.

[0084] The preset first quantile threshold and the preset second quantile threshold can be set according to specific needs.

[0085] S153 maintains full-precision floating-point representation for the outlier portion and determines the weight of the outlier portion.

[0086] S154 uses the MinMax method to perform channel-by-channel quantization on the non-outlier part to determine the quantization weight of the non-outlier part.

[0087] S155, determine the mixed precision weights based on the weights of the outlier components and the quantization weights of the non-outlier components.

[0088] The embodiments described above in this application achieve channel-smooth scaling of the weights of each quantized convolutional layer in each input channel by using the channel smoothing coefficient of each quantized convolutional layer in each input channel. This achieves matching of weights with input activation scale. The outlier portion of the weights of each quantized convolutional layer after channel-smooth scaling is maintained in full-precision floating-point representation. The non-outlier portion of the weights of each quantized convolutional layer after channel-smooth scaling is quantized channel-by-channel using the MinMax method. This approach can significantly compress the model and accelerate inference speed while maintaining the model's accuracy, preventing severe performance degradation caused by low-bit quantization of all weights.

[0089] Based on the pseudo-quantized activation values ​​and mixed precision weights obtained above, the demoired convolutional neural network after training and quantization is determined.

[0090] In some specific embodiments of this application, during the quantization inference stage, the moiré image is input into the trained and quantized demoiré convolutional neural network. The input activation value is divided by the channel smoothing coefficient corresponding to the convolutional layer. Based on the input activation value range determined in step S13 above, the input activation value of the convolutional layer is pseudo-quantized to obtain the low-bit quantized activation value. The low-bit quantized activation value is then convolved with the mixed precision weights to obtain the demoiré image.

[0091] During the quantization process, the first and last modules of the trained demoired convolutional neural network are kept in full precision to stabilize edge structure and texture details.

[0092] Specifically, in a demoiroid convolutional neural network, the first module refers to the first feature extraction convolutional layer when the input image is input into the neural network, and sometimes it also includes a preprocessing convolutional block; the tail module refers to the last reconstruction convolutional layer in the output phase of the neural network, which is used to restore the intermediate features to the final image.

[0093] It should be noted that if the neural network structure contains activation functions or normalization operations in the first or last module, these are also considered as part of the first or last module.

[0094] For example, the first module of the Efficient Scale-Diverse Network model includes an initial convolutional layer and a feature transformation unit at the input end; the tail module includes a convolutional reconstruction layer at the output end, which is used to map the feature generated by the model to the image after demoiring.

[0095] To ensure the stability of edge and texture details, the first and last modules are kept at full precision during quantization to avoid amplifying quantization errors and affecting the overall image quality.

[0096] This application provides a post-quantization method for demoired convolutional neural networks. The entire process only requires forward propagation calibration, without labeled data, backpropagation, or retraining. It only requires a small amount of calibration and fine-tuning, which can significantly reduce memory usage and inference time at 4 bits or lower, while mitigating the accuracy loss caused by quantization. It maximizes the reconstruction quality of the demoired task, adapts to time- or computationally-constrained scenarios, and achieves fast, efficient, and accurate post-quantization of demoired convolutional neural networks.

[0097] The preferred features in the above embodiments can be used individually in any embodiment, or in any combination thereof, provided they do not conflict with each other. Furthermore, parts not described in detail in the embodiments can be implemented using existing technologies.

[0098] The following examples and comparative examples will be used to further illustrate this application in order to better understand the above-mentioned technical solutions. It should be understood that the following are only some examples and are not intended to limit this application.

[0099] In some specific embodiments of this application, the following methods are adopted: (1) the MinMax method; (2) the method corresponding to the first stage calibration process of this application, namely amplitude statistics and channel smoothing; (3) the method corresponding to the first stage calibration process and the second stage calibration process of this application, namely amplitude statistics and channel smoothing, and input activation value interval sampling estimation; (4) a post-quantization method for demoired convolutional neural networks provided in this application, namely the method corresponding to the first stage calibration process, the second stage calibration process, and the MinMax method. The demoired convolutional neural network ESDNet trained by the above four post-quantization methods is quantized in 4 bits. The first and last layers of the demoired convolutional neural network are fully quantized, and the peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and perceptual similarity index (LPIPS) of the inference results of the quantized demoired convolutional neural network on the UHDM dataset are calculated.

[0100]

[0101] Referring to Table 1, ↑ indicates that the larger the indicator, the better the image quality, and ↓ indicates that the smaller the indicator, the better the image quality; the bolded result indicates the best result for that indicator.

[0102] The results in Table 1 show that method (2) has a significant advantage over method (1) in all three metrics, which proves that the strategy of estimating the activation value quantization range by sampling outperforms the strategy of determining the activation value quantization range by the MinMax method in ESDNet quantization; method (3) has a significant advantage over method (2) in all three metrics, which proves that the channel smoothing operation significantly improves the performance of the quantization model; method (4) has a significant advantage over method (3) in all three metrics, which proves that weighted extreme value mixed precision weighted quantization has a certain improvement on the performance of the quantization model.

[0103] Therefore, the post-quantization method for demoired convolutional neural networks provided in this application can perform inference at 4 bits (or lower), reducing model size and bandwidth, and lowering computation and storage costs. Under the condition of relying only on a small number of unlabeled samples for forward calibration, it alleviates the accuracy loss caused by quantization, minimizes the loss of indicators such as PSNR / SSIM / LPIPS, and maintains reconstruction quality.

[0104] This application provides a post-quantization method for demoiring convolutional neural networks, which can be applied to image demoiring tasks, such as to a trained Efficient Scale-Diverse Network model. The image demoiring method includes: Step 1: Obtain the moiré image to be processed and the trained Efficient Scale-Diverse Network model.

[0105] Step 2: Perform post-quantization processing on the trained Efficient Scale-Diverse Network model using the post-quantization method of the demoired convolutional neural network in any of the above embodiments to determine the post-quantized Efficient Scale-Diverse Network model.

[0106] For details, please refer to steps S11 to S16 above, which will not be repeated here.

[0107] Step 3: Input the moiré image to be processed into the post-quantized Efficient Scale-Diverse Network model to determine the demoiré image.

[0108] Specifically, the moiré image is input into the trained and quantized Efficient Scale-Diverse Network model. The input activation value is divided by the channel smoothing coefficient corresponding to the convolutional layer. Based on the input activation value range determined in step S13 above, the input activation value of the convolutional layer is pseudo-quantized to obtain the low-bit quantized activation value. The low-bit quantized activation value is then convolved with the mixed precision weights to obtain the demoiré image.

[0109] The embodiments described above employ a post-quantization method for demoiring convolutional neural networks provided in this application to perform post-quantization processing on the trained Efficient Scale-Diverse Network model. This method combines amplitude statistics, channel smoothing, input activation value interval sampling estimation, and the MinMax method to effectively achieve low-bit quantization, reduce model size and bandwidth, reduce computation and storage, effectively alleviate the accuracy degradation caused by quantization, and maintain demoiring quality.

[0110] The post-quantization method for demoired convolutional neural networks provided in this application can also be applied to the post-quantization of video super-resolution models, such as Upscale-A-Video, DOVE, and STAR models.

[0111] A video super-resolution processing method includes: Step 1: Obtain the video to be processed and the trained video super-resolution model.

[0112] Step 2: Use the post-quantization method of the demoired convolutional neural network in any of the above embodiments to perform post-quantization processing on the trained video super-resolution model, and determine the post-quantized video super-resolution model.

[0113] Specifically, in the process of post-quantization processing of the trained video super-resolution model using the post-quantization method of the demoired convolutional neural network provided in this application, an unlabeled low-resolution video calibration dataset is used to perform the first-stage calibration processing on each quantized convolutional layer, and the final result is the trained and post-quantized video super-resolution model.

[0114] A post-quantization method for a video super-resolution model, comprising: Replace the convolutional layers in the trained video super-resolution model with quantized convolutional layers; The first-stage calibration process for each quantized convolutional layer is performed using an unlabeled, low-resolution video calibration dataset to determine the channel smoothing coefficients of each quantized convolutional layer. Based on the channel smoothing coefficient of each quantized convolutional layer, a second-stage calibration process is performed on each quantized convolutional layer to determine the input activation value range of each quantized convolutional layer. Based on the input activation value range of each quantized convolutional layer, pseudo-quantization is performed on the input activation value of each quantized convolutional layer to determine the activation value after pseudo-quantization. Based on the channel smoothing coefficient of each quantized convolutional layer, the weights of each quantized convolutional layer are subjected to mixed precision quantization to determine the mixed precision weights. Based on the pseudo-quantized activation values ​​and mixed precision weights, the trained and quantized video super-resolution model is determined.

[0115] Step 3: Input the video to be processed into the post-quantized video super-resolution model to determine the video that has undergone super-resolution processing.

[0116] The above embodiments of this application employ a post-quantization method for a demoired convolutional neural network provided in this application, which is adaptively modified to a post-quantization method for a video super-resolution model. Post-quantization processing is performed on the trained video super-resolution model to effectively achieve low-bit quantization, reduce model size and bandwidth, reduce computation and storage, effectively alleviate the decrease in accuracy caused by quantization, and maintain the quality of the super-resolution video.

[0117] Figure 2 This is a schematic diagram of the structure of a post-quantization system for a demoired convolutional neural network according to an exemplary embodiment.

[0118] Reference Figure 2As shown in one embodiment of this application, a post-quantization system 100 for demoired convolutional neural networks is provided, including a quantization convolutional layer replacement module 110, a first-stage calibration processing module 120, a second-stage calibration processing module 130, an activation value pseudo-quantization module 140, a weighted mixed precision quantization module 150, and a post-quantization model determination module 160.

[0119] The quantized convolutional layer replacement module 110 is used to replace the convolutional layers in the trained demoired convolutional neural network with quantized convolutional layers. The first-stage calibration processing module 120 is used to perform first-stage calibration processing on each quantized convolutional layer using an unlabeled moiré image calibration dataset to determine the channel smoothing coefficient of each quantized convolutional layer. The second-stage calibration processing module 130 is used to perform a second-stage calibration process on each quantized convolutional layer based on the channel smoothing coefficient of each quantized convolutional layer, and to determine the input activation value range of each quantized convolutional layer. The activation value pseudo-quantization module 140 is used to perform pseudo-quantization on the input activation value of each quantized convolutional layer according to the input activation value range of each quantized convolutional layer, and determine the activation value after pseudo-quantization. The weighted mixed precision quantization module 150 is used to perform mixed precision quantization on the weights of each quantized convolutional layer according to the channel smoothing coefficient of each quantized convolutional layer, and to determine the mixed precision weights. The post-quantization model determination module 160 is used to determine the trained post-quantized demoired convolutional neural network based on the pseudo-quantized activation values ​​and mixed precision weights.

[0120] In the embodiments described above, an unlabeled moiré image calibration dataset is used to perform a first-stage calibration process on each quantized convolutional layer. The channel smoothing coefficient of each quantized convolutional layer is obtained, and the amplitude distribution difference of the activation value and weight of each quantized convolutional layer between different input channels is reduced based on the channel smoothing coefficient, thereby reducing the post-quantization error. Combined with the input activation value range and mixed precision weight of each quantized convolutional layer, the quality of the demoiré task is effectively improved. Moreover, the entire process only requires forward propagation calibration, without the need for labeled data or backpropagation and retraining, achieving fast, efficient and accurate low-bit post-quantization of demoiré convolutional neural networks, which can be adapted to scenarios with limited time or computing power.

[0121] Regarding the embodiments of the above system, the specific ways in which each module performs operations have been described in detail in the embodiments of the method, and will not be elaborated here.

[0122] Based on the same technical concept, in some specific embodiments of this application, a terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and a method that the processor can use to execute when executing the program.

[0123] Based on the same technical concept, in some specific embodiments of this application, a computer-readable storage medium is provided on which a computer program is stored, which can be used to execute a method when the program is executed by a processor.

[0124] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs and functional modules that implement the above methods), computer instructions, etc., and the aforementioned computer programs and computer instructions can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0125] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0126] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0127] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0128] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0130] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0131] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0132] The foregoing has described some specific embodiments of this application. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this application. The above-described preferred features can be used in any combination without conflict.

Claims

1. A post-quantization method for demoired convolutional neural networks, characterized in that, include: Replace the convolutional layers in the trained demoired convolutional neural network with quantized convolutional layers; The first-stage calibration process is performed on each of the quantized convolutional layers using an unlabeled moiré image calibration dataset to determine the channel smoothing coefficient of each of the quantized convolutional layers. Based on the channel smoothing coefficient of each quantized convolutional layer, a second-stage calibration process is performed on each quantized convolutional layer to determine the input activation value range of each quantized convolutional layer. Based on the input activation value range of each quantized convolutional layer, pseudo-quantization is performed on the input activation value of each quantized convolutional layer to determine the activation value after pseudo-quantization. Based on the channel smoothing coefficient of each quantized convolutional layer, the weights of each quantized convolutional layer are subjected to mixed precision quantization to determine the mixed precision weights. Based on the pseudo-quantized activation values ​​and the mixed precision weights, a trained and quantized demoired convolutional neural network is determined. The first-stage calibration process, which uses an unlabeled moiré image calibration dataset to perform a first-stage calibration on each quantized convolutional layer to determine the channel smoothing coefficients of each quantized convolutional layer, includes: The unlabeled moiré image calibration dataset is used to perform a preset batch of calibration processing on each of the quantized convolutional layers, and the maximum absolute activation value of each input channel of each quantized convolutional layer and the maximum absolute weight value of each input channel of each quantized convolutional layer in each batch of calibration processing are determined. For each input channel of the quantized convolutional layer in each batch of calibration processing, the maximum absolute activation value is averaged for each input channel to determine the upper bound of the activation value amplitude of each input channel of the quantized convolutional layer. For each input channel of the quantized convolutional layer in each batch of calibration processing, the average value of the maximum absolute weight is calculated for each input channel to determine the upper bound of the weight magnitude of each input channel of the quantized convolutional layer. The channel smoothing coefficient of each quantized convolutional layer in each input channel is determined by the power average of the upper bound of the activation value magnitude of each quantized convolutional layer in each input channel and the upper bound of the weight magnitude of each quantized convolutional layer in each input channel. The step of performing a second-stage calibration process on each quantized convolutional layer based on the channel smoothing coefficient of each quantized convolutional layer to determine the input activation value range of each quantized convolutional layer includes: The input activation value of each quantized convolutional layer in each batch of calibration is divided channel by channel smoothing coefficient of each quantized convolutional layer in each input channel to determine the channel smoothed input activation value of each quantized convolutional layer in each batch of calibration. For each quantized convolutional layer, a preset number of elements are randomly sampled from the channel-smoothed input activation values ​​in each batch of calibration processing to determine the random sampling elements; Based on the random sampling elements, determine the maximum and minimum values ​​of the input activation values ​​for each quantized convolutional layer in each batch of calibration processing; The maximum and minimum input activation values ​​of each quantized convolutional layer in each batch of calibration processing are averaged over the batch to determine the maximum and minimum input activation values ​​of each quantized convolutional layer, and to determine the input activation value range of each quantized convolutional layer. The step of performing mixed-precision quantization on the weights of each quantized convolutional layer based on the channel smoothing coefficient of each quantized convolutional layer to determine the mixed-precision weights includes: The weight of each input channel of each quantized convolutional layer in each batch of calibration processing is divided channel by channel smoothing coefficient of each quantized convolutional layer in each input channel to determine the weight of each quantized convolutional layer after channel smoothing scaling. Based on the weights of each quantized convolutional layer after channel smoothing and scaling, as well as the preset first quantile threshold and the preset second quantile threshold, the outlier and non-outlier portions of each quantized convolutional layer after channel smoothing and scaling are determined. The outlier portion is represented using full-precision floating-point representation, and the weight of the outlier portion is determined. The MinMax method is used to perform channel-by-channel quantization on the non-outlier portion to determine the quantization weight of the non-outlier portion; The mixed precision weight is determined based on the weights of the outlier components and the quantization weights of the non-outlier components.

2. The post-quantization method for demoired convolutional neural networks according to claim 1, characterized in that, The step of performing pseudo-quantization on the input activation values ​​of each quantized convolutional layer based on the input activation value range of each quantized convolutional layer, and determining the activation values ​​after pseudo-quantization, includes: The input activation value range of each quantized convolutional layer is used as the quantization range of each quantized convolutional layer. The quantization stride and zero point are determined based on the quantization interval and quantization bit width of each quantized convolutional layer. Based on the quantization step size and the zero point, the input activation value of each quantized convolutional layer is converted into an integer representation, and the integer representation of the input activation value of each quantized convolutional layer is determined. The integer representation of the input activation value of each quantized convolutional layer is dequantized to determine the pseudo-quantized activation value.

3. A post-quantization system for a demoired convolutional neural network, implemented using the post-quantization method for a demoired convolutional neural network as described in any one of claims 1 to 2, characterized in that, include: The quantized convolutional layer replacement module is used to replace the convolutional layers in a trained demoired convolutional neural network with quantized convolutional layers. The first-stage calibration processing module is used to perform first-stage calibration processing on each of the quantized convolutional layers using an unlabeled moiré image calibration dataset to determine the channel smoothing coefficient of each of the quantized convolutional layers. The second-stage calibration processing module is used to perform a second-stage calibration process on each quantized convolutional layer according to the channel smoothing coefficient of each quantized convolutional layer, and to determine the input activation value range of each quantized convolutional layer. The activation value pseudo-quantization module is used to perform pseudo-quantization on the input activation value of each quantized convolutional layer according to the input activation value range of each quantized convolutional layer, and determine the activation value after pseudo-quantization. The weighted mixed precision quantization module is used to perform mixed precision quantization on the weights of each quantized convolutional layer according to the channel smoothing coefficient of each quantized convolutional layer, and determine the mixed precision weights. The post-quantization model determination module is used to determine the trained post-quantized demoired convolutional neural network based on the activation values ​​after pseudo-quantization and the mixed precision weights.

4. A method for removing moiré patterns from images, characterized in that, include: Obtain the moiré image to be processed and the trained Efficient Scale-Diverse Network model; The trained Efficient Scale-Diverse Network model is post-quantized using the post-quantization method of the demoired convolutional neural network as described in any one of claims 1 to 2, thereby determining the post-quantized Efficient Scale-Diverse Network model. The moiré image to be processed is input into the post-quantized Efficient Scale-Diverse Network model to determine the demoiré image.

5. A video super-resolution processing method, characterized in that, include: Acquire the video to be processed and the trained video super-resolution model; The trained video super-resolution model is post-quantized using the post-quantization method of the demoired convolutional neural network as described in any one of claims 1 to 2, thereby determining the post-quantized video super-resolution model. The video to be processed is input into the post-quantized video super-resolution model to determine the video that has undergone super-resolution processing.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-2.

7. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-2.

Images