Post-training quantization method and device, and electronic equipment

Abstract

This application discloses a post-training quantization method, apparatus, and electronic device, relating to the field of computer technology. The method includes: acquiring an image to be calibrated, wherein the image to be calibrated is a face image; inputting the image to be calibrated into a floating-point network and optimizing the network parameters of the floating-point network using the image to be calibrated to obtain target parameters, wherein the network parameters of the floating-point network are the network parameters corresponding to the normalization layer of the floating-point network; acquiring an affine matrix corresponding to the target parameters and using this affine matrix as the affine matrix corresponding to the image to be calibrated, wherein the affine matrix includes multiple face calibration parameters; calibrating the image to be calibrated using the affine matrix to obtain a target image; and quantizing the floating-point network based on the target image. Thus, this application can achieve face calibration simply and effectively by acquiring the affine matrix, and can reduce the cost required for quantization.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a post-training quantization method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, when developing neural networks on a computer, 32-bit floating-point numbers are generally used for numerical calculations and storing network weights. Furthermore, graphics processing units (GPUs) with numerous floating-point units can be used on the computer to accelerate the neural network. However, in the mobile internet era, more and more artificial intelligence algorithms need to be deployed on mobile devices. The efficiency of existing neural network models running on personal computers becomes a significant bottleneck on resource-constrained mobile devices. Therefore, how to better quantize floating-point networks is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] This application proposes a post-training quantization method, apparatus, electronic device, and storage medium that can improve the above-mentioned problems.

[0004] On one hand, embodiments of this application provide a post-training quantization method, which includes: acquiring an image to be calibrated, wherein the image to be calibrated is a face image; inputting the image to be calibrated into a floating-point network, and optimizing the network parameters of the floating-point network using the image to be calibrated to obtain target parameters, wherein the network parameters of the floating-point network are the network parameters corresponding to the normalization layer of the floating-point network; acquiring an affine matrix corresponding to the target parameters, and using the affine matrix as the affine matrix corresponding to the image to be calibrated, wherein the affine matrix includes multiple face calibration parameters; calibrating the image to be calibrated using the affine matrix to obtain a target image; and quantizing the floating-point network based on the target image.

[0005] On the other hand, embodiments of this application provide a post-training quantization device, which includes: a first acquisition module, an input module, a second acquisition module, a calibration module, and a quantization module. The first acquisition module is used to acquire an image to be calibrated, wherein the image to be calibrated is a face image. The input module is used to input the image to be calibrated into a floating-point network and optimize the network parameters of the floating-point network using the image to be calibrated to obtain target parameters, wherein the network parameters of the floating-point network are the network parameters corresponding to the normalization layer of the floating-point network. The second acquisition module is used to acquire an affine matrix corresponding to the target parameters and use this affine matrix as the affine matrix corresponding to the image to be calibrated, wherein the affine matrix includes multiple face calibration parameters. The calibration module is used to calibrate the image to be calibrated using the affine matrix to obtain a target image. The quantization module is used to quantize the floating-point network according to the target image, and the quantized floating-point network is used to perform face recognition operations on the image to be calibrated.

[0006] Optionally, the input module may include an initialization unit, a recognition unit, and an optimization unit. The initialization unit initializes the network parameters corresponding to the normalization layer of the floating-point network to obtain first parameters. The recognition unit uses the initialized floating-point network to recognize the image to be calibrated to obtain a recognition result. The optimization unit optimizes the first parameters based on the recognition result to obtain target parameters.

[0007] Optionally, the optimization unit is further configured to obtain the network parameters corresponding to the normalized layer of the floating-point network before initialization, and use the parameters as the second parameter; when the optimized first parameter and the second parameter satisfy a preset condition, the optimized first parameter is used as the target parameter.

[0008] Optionally, the optimization unit is further configured to obtain the loss value between the optimized first parameter and the second parameter; when the loss value is less than the loss threshold, the optimized first parameter is used as the target parameter.

[0009] Optionally, the formula for calculating the loss value is: Wherein, loss refers to the loss value between the first parameter and the second parameter, and x(θ) refers to the image to be calibrated. and This refers to the mean and variance in the first parameter. and This represents the mean and variance in the second parameter.

[0010] Optionally, the input module is further configured to determine the position of the face image in the image to be calibrated, and crop the image to be calibrated according to the position to obtain a target face image; and input the target face image into the floating-point network.

[0011] Optionally, the calibration module is further configured to determine at least one of the rotation parameter, scale parameter, and displacement parameter corresponding to the affine matrix; and to rotate, scale, or translate the image to be calibrated according to the affine matrix to obtain the target image.

[0012] Optionally, before inputting the image to be calibrated into the floating-point network to obtain the affine matrix, the post-training quantization device is also used to obtain the face calibration network and multiple sample affine matrices; the face calibration network is trained based on the multiple sample affine matrices to obtain the floating-point network.

[0013] On the other hand, embodiments of this application provide an electronic device, including: one or more processors; a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the methods described above.

[0014] On the other hand, embodiments of this application provide a computer-readable storage medium having program code stored thereon, the program code being invoked by a processor to execute the above-described method.

[0015] The solution provided in this application first obtains an image to be calibrated, which may be a face image. Then, the obtained image to be calibrated is input into a floating-point network, and the parameters of the floating-point network are optimized using the image to obtain target parameters. The network parameters of the floating-point network are the network parameters corresponding to the normalization layer of the floating-point network. Next, an affine matrix corresponding to the target parameters is obtained and used as the affine matrix corresponding to the image to be calibrated. The affine matrix may include multiple face calibration parameters. Finally, the image to be calibrated is calibrated according to the obtained affine matrix to obtain the target image, and the floating-point network is quantized based on the target image. Thus, this application can reduce the cost of quantization and achieve near-zero-sample quantization training.

[0016] These or other aspects of this application will become more apparent from the description of the following embodiments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of a post-training quantization method provided in an embodiment of this application is shown.

[0019] Figure 2 This illustration shows a schematic diagram of an image to be calibrated in a post-training quantization method according to an embodiment of this application.

[0020] Figure 3 This illustration shows a schematic diagram of an image after affine transformation of the image to be calibrated in a post-training quantization method provided in an embodiment of this application.

[0021] Figure 4 A flowchart of a post-training quantization method provided in another embodiment of this application is shown.

[0022] Figure 5 A flowchart of step S240 in a post-training quantization method provided in another embodiment of this application is shown.

[0023] Figure 6 A flowchart of a post-training quantization method provided in another embodiment of this application is shown.

[0024] Figure 7 This illustration shows a schematic diagram of facial key points in a post-training quantization method provided in another embodiment of this application.

[0025] Figure 8 A flowchart of step S340 in a post-training quantization method provided in another embodiment of this application is shown.

[0026] Figure 9 This illustration shows a cropped image of the image to be calibrated in a post-training quantization method according to another embodiment of this application.

[0027] Figure 10 This illustration shows a schematic diagram of an image after calibration of a cropped image in a post-training quantization method provided in another embodiment of this application.

[0028] Figure 11 A flowchart of step S360 in a post-training quantization method provided in another embodiment of this application is shown.

[0029] Figure 12 A block diagram of a post-training quantization device provided in an embodiment of this application is shown.

[0030] Figure 13 A block diagram of the input module in a post-training quantization device provided in an embodiment of this application is shown.

[0031] Figure 14 This is a block diagram of an electronic device for performing a post-training quantization method according to an embodiment of this application.

[0032] Figure 15 This is a storage unit in this application embodiment for storing or carrying program code that implements the post-training quantization method according to this application embodiment. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0034] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence.

[0035] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, interactive operating systems, and mechatronics. Software AI technologies typically include computer vision, natural language processing, and machine learning / deep learning. With the development and advancement of AI, it is being researched and applied in numerous fields, such as smart homes, intelligent customer service, virtual assistants, smart speakers, intelligent marketing, autonomous driving, robotics, and smart healthcare. It is believed that with further technological advancements, AI will be applied in even more areas, playing an increasingly important role.

[0036] Machine learning is a multidisciplinary field that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and many other disciplines. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.

[0037] Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence. Deep learning, in turn, is the core of machine learning and a technology for realizing machine learning. Machine learning typically includes techniques such as deep learning, reinforcement learning, transfer learning, and inductive learning. Deep learning includes techniques such as Convolutional Neural Networks (CNNs), deep belief networks, recurrent neural networks, autoencoders, and generative adversarial networks.

[0038] Computer vision is a comprehensive discipline that integrates computer science, signal processing, physics, applied mathematics, statistics, neurophysiology, and other fields. It is also a challenging and important research direction in the scientific community. This discipline uses various imaging systems to replace visual organs as input methods, with computers replacing the brain to complete the processing and interpretation, enabling computers to observe and understand the world through vision, just like humans.

[0039] Currently, when developing neural networks on a computer, 32-bit floating-point numbers are generally used for numerical calculations and storing network weights. Furthermore, GPUs with a large number of floating-point units can be used to accelerate neural networks on the computer. However, in the mobile internet era, more and more AI algorithms need to be deployed on mobile devices, and the efficiency of neural network models that originally ran on PCs becomes a significant bottleneck on resource-constrained mobile devices.

[0040] To address the aforementioned issues, the concept of quantization was proposed. Currently, two commonly used model quantization techniques are post-training quantization and quantization-aware training. Quantization-aware training requires the entire training process and large-scale data, which leads to excessively long training times. Furthermore, training data and code are often confidential, preventing quantization algorithm developers from accessing the complete training process and hindering quantization-aware training. In contrast, post-training quantization typically quantizes the model more quickly and requires only partial calibration data. Specifically, post-training quantization applies post-training quantization to both weights and numerical activations. While post-training quantization of network weights does not require samples, post-training quantization of numerical activations still requires samples to calibrate the numerical ranges.

[0041] Please refer to Figure 1 , Figure 1 The diagram shows a flowchart of a post-training quantization method according to an embodiment of this application, which may include steps S110 to S150.

[0042] Step S110: Obtain the image to be calibrated, wherein the image to be calibrated is a face image.

[0043] The post-training quantization method provided in this application can be applied to electronic devices, such as desktop computers, smartphones, tablets, e-readers, and other electronic devices capable of running applications. Furthermore, the image to be calibrated can be an image acquired in real-time by the electronic device; that is, the image to be calibrated can be an image acquired by the electronic device through its built-in image acquisition device, an image acquired by another electronic device and then sent to the electronic device, or an image obtained through a search by a specified application.

[0044] In some implementations, the image to be calibrated can be a face image, meaning that the image to be calibrated may contain a face image. Alternatively, the image to be calibrated can be a standard face image or a non-standard face image. A standard face image is an image where the face is unobstructed, not tilted, and directly facing the image acquisition device. When the electronic device receives a standard face image, it can perform face recognition and face detection more easily and effectively.

[0045] As another approach, due to the influence of photography technology and shooting environment, most of the acquired images to be calibrated are non-standard face images. These non-standard face images can be images where the face is tilted, rotated, or occluded. To clearly understand non-standard face images, embodiments of this application provide, for example... Figure 2 The diagram shown is from Figure 2 It can be seen that the face of person 101 in the image to be calibrated is tilted, that is, the face of person 101 is tilted to the left.

[0046] Step S120: Input the image to be calibrated into the floating-point network, and use the image to be calibrated to optimize the network parameters of the floating-point network to obtain the target parameters.

[0047] In some implementations, after acquiring the image to be calibrated, this embodiment can input the image to be calibrated into a floating-point network, and then use the image to optimize the network parameters of the floating-point network to obtain target parameters. The network parameters of the floating-point network are the network parameters corresponding to the normalization layer of the floating-point network. Alternatively, before inputting the image to be calibrated into the floating-point network, this embodiment can first determine whether the image to be calibrated contains a face image. If it contains a face image, it is further determined whether the face image is a standard face image. If it is a standard face image, it can be directly used to quantize the floating-point network; if the face image is not a standard face image, the image to be calibrated can be input into the floating-point network.

[0048] Optionally, the floating-point network can be a network with face recognition functionality. The precision of the network parameters corresponding to this floating-point network can be either regular precision or mixed precision. The regular precision can use FP32 (32-bit floating-point, single precision) to store the network parameters; the mixed precision can use FP32 and FP16 to store the network parameters. FP16 is half-precision floating-point, which can reduce the memory size, but some parameters or operators must use FP31 format to maintain accuracy.

[0049] In this embodiment, the floating-point network can be a pre-trained face recognition model, meaning it can be trained using a large amount of face data. This data can be transmitted to the electronic device via wired or wireless means from other electronic devices, or it can be a general-purpose face recognition model downloaded directly from the cloud. As can be seen, the floating-point network can be a face recognition network trained on a large amount of data. However, with the continuous development of technology and the increasing awareness of data protection, the training data and code corresponding to the floating-point network have a certain degree of privacy. Therefore, although the floating-point network can be obtained, the corresponding training data is usually unavailable unless the floating-point network was trained by the user themselves.

[0050] In some implementations, after obtaining the image to be calibrated and the floating-point network, this embodiment can use the image to be calibrated to optimize the network parameters of the floating-point network. As described above, the network parameters are the network parameters corresponding to the normalization layer of the floating-point network. Specifically, these network parameters can be the mean and variance corresponding to batch normalization (BN), an affine matrix, or weight parameters or loss functions constituting the floating-point network. Therefore, after inputting the image to be calibrated into the floating-point network, this embodiment can continuously optimize the parameters of the floating-point network using the image to be calibrated, and then obtain the optimized network parameters.

[0051] In summary, different images to be calibrated will result in different target parameters for optimization; that is, there can be a one-to-one correspondence between the images to be calibrated and the target parameters. For example, when image 1 to be calibrated is input into a floating-point network, the parameters obtained through continuous optimization of the network parameters can be target parameter 1. However, when image 2 to be calibrated is input into a floating-point network, the parameters obtained through continuous optimization of the network parameters can be target parameter 2. Target parameter 1 and target parameter 2 are not the same. Therefore, different images to be calibrated will have different target parameters, and each image to be calibrated has its own optimal target parameters.

[0052] Step S130: Obtain the affine matrix corresponding to the target parameters, and use the affine matrix as the affine matrix corresponding to the image to be calibrated. The affine matrix includes multiple face calibration parameters.

[0053] In some implementations, after obtaining the target parameters using the image to be calibrated, embodiments of this application can obtain the affine matrix corresponding to the target parameters and use this affine matrix as the affine matrix of the image to be calibrated. The affine matrix may include multiple face calibration parameters. Through these face calibration parameters, embodiments of this application can perform operations such as translation, scaling, or rotation on the face image in the image to be calibrated.

[0054] In other embodiments, the affine matrix in this application can be used to perform an affine transformation on the image to be calibrated. This affine transformation, also known as an affine mapping, refers to the geometric transformation of a vector space into another vector space by performing a linear transformation followed by a translation. An affine transformation can be geometrically defined as an affine transformation between two vector spaces, or an affine mapping consisting of a non-singular linear transformation followed by a translation transformation.

[0055] In this embodiment, the affine matrix can correspond to four variables, which can be a, b, T, etc. x And T y Where a and b can be parameters related to rotation, scaling, etc., while T x With T y It can be a parameter related to translation, i.e., T x This can be a translation of the horizontal axis, i.e., a horizontal translation. When the value is positive, the image to be calibrated is translated horizontally to the right; when the value is negative, the image to be calibrated is translated horizontally to the left. Additionally, T... y This can be a translation of the ordinate, i.e., a vertical translation. When the value is positive, the image to be calibrated is translated vertically upwards; when it is negative, the image is translated vertically downwards. The affine matrix can be constructed using these four variables.

[0056] In one specific implementation, an affine matrix can be represented as Where θ can be the rotation angle required to convert the image to be calibrated into a standard face image (average face image), and T x It can be the horizontal coordinate translation information of a face in the image to be calibrated into a standard face image, T y It can be the vertical coordinate translation information of the face in the image to be calibrated into the standard face image.

[0057] It should be noted that, in the embodiments of this application, when optimizing the network parameters of the floating-point network, the electronic device can continuously optimize the mean and variance of each layer of batch normalization (BN) in the floating-point network. That is, the floating-point network may include multiple layers of batch normalization (BN). In the embodiments of this application, when optimizing the network parameters of the floating-point network, it can continuously optimize the mean and variance corresponding to each layer of batch normalization.

[0058] Furthermore, during the optimization of the mean and variance corresponding to batch normalization, this embodiment can also continuously optimize the affine matrix corresponding to the mean and variance. It is evident that there is a one-to-one correspondence between the mean, variance, and affine matrix; each mean and variance can correspond to one affine matrix, and the relationship between the two can be positively correlated. That is, when the mean and variance increase, the affine matrix can also increase accordingly; when the mean and variance decrease, the affine matrix can also decrease accordingly. In other words, when the obtained mean and variance are optimal, the corresponding affine matrix can also be optimal. Therefore, this embodiment can obtain the affine matrix using the mean and variance corresponding to the floating-point network.

[0059] Step S140: The image to be calibrated is calibrated using the affine matrix to obtain the target image.

[0060] In some implementations, after obtaining the affine matrix corresponding to the target parameters, the embodiments of this application can use the affine matrix to calibrate the image to be calibrated, thereby obtaining the target image. Specifically, the embodiments of this application can first obtain the coordinate matrix corresponding to the key points of the face to be calibrated, then obtain the affine matrix corresponding to the face image to be calibrated, and then use the affine matrix to calibrate the coordinate matrix of the face to be calibrated.

[0061] For example, the coordinate matrix corresponding to the facial key points to be calibrated is: And the coordinate matrix of the obtained affine matrix is: The coordinate matrix corresponding to the calibrated facial key points can be obtained through matrix transformations and matrix multiplication.

[0062] To better understand the image calibration process, embodiments of this application provide... Figure 2 The diagram after calibration is shown in detail below. Figure 3 As shown, from Figure 3 It can be seen that after obtaining the affine matrix and using it to calibrate the face image corresponding to person 101, the calibrated face image is a standard face image, and the head tilt problem is no longer present. Therefore, in... Figure 2 When calibrating a face image, this embodiment of the application can rotate the face to the right.

[0063] Step S150: Quantize the floating-point network based on the target image.

[0064] In some implementations, an affine matrix is ​​used to calibrate the image to be calibrated. After obtaining the target image, embodiments of this application can quantize the floating-point network based on the target image. Quantization can convert the floating-point algorithm of the neural network into a fixed-point algorithm, thus enabling real-time operation of the neural network on mobile terminals. Furthermore, quantization of the neural network also facilitates cloud computing deployment. Quantization is one of the most widely used compression methods, and it has several similar terms, with low precision being the most common concept. Quantization is usually tied to acceleration instructions and hardware, and engineering users developing quantized networks often have limited access to business data due to the confidentiality of their business processes.

[0065] In some implementations, quantization can be INT8, but depending on the number of bits required to store the neural network parameters, quantization can also include binary neural networks, ternary weight networks, and XNOR networks. Among them, a binary neural network can be a neural network in which the weights and activations take only two values ​​(e.g., +1, -1) at runtime, and the gradient of the parameters is calculated during training. Unlike binary neural networks, a ternary weight network can be a neural network in which the weight network is constrained to +1, 0, and -1. The inputs to the filters and convolutional layers of an XNOR network can be binary, that is, an XNOR network can mainly approximate convolution through binary operations.

[0066] In this embodiment, the quantization process can be mainly divided into two parts: the first part is to convert the floating-point network from FP32 to INT8, and the second part is to use INT8 to infer the network parameters. Furthermore, quantizing the floating-point network based on the target image yields a quantized neural network, and running the quantized neural network produces the same effect as the original network.

[0067] In one specific implementation, the quantization process can be as follows: first, a floating-point network is obtained, then Fake-Quant nodes are inserted to train min / max, then the network is quantized using a specific quantization tool, and finally the quantized neural network obtained by quantization can be deployed on a mobile terminal.

[0068] It should be noted that, in the embodiments of this application, the images are typically preprocessed by subtracting the mean and dividing by the variance during training of the floating-point network, thus normalizing the pixel values ​​of the image. To optimize the mean and variance of the floating-point network training, the embodiments of this application can assume that the mean and variance of the batch normalization layer are similar for each image channel. Therefore, when the electronic device uses the image to be calibrated to optimize the floating-point network, the main degrees of freedom to be optimized are six. These six degrees of freedom correspond to the four variables of the affine matrix, and the mean and variance corresponding to the batch normalization. In other words, the main parameters optimized in the embodiments of this application are the parameters a, b, and T corresponding to the affine matrix. x And T y And the mean and variance corresponding to batch normalization.

[0069] The post-training quantization method provided in this embodiment of the invention first acquires an image to be calibrated, which may be a face image. Then, the acquired image to be calibrated is input into a floating-point network, and the parameters of the floating-point network are optimized using the image to obtain target parameters. The network parameters of the floating-point network are the network parameters corresponding to the normalization layer of the floating-point network. Next, an affine matrix corresponding to the target parameters is acquired and used as the affine matrix corresponding to the image to be calibrated. The affine matrix may include multiple face calibration parameters. Finally, the image to be calibrated is calibrated according to the acquired affine matrix to obtain the target image, and the floating-point network is quantized based on the target image. Thus, this application can reduce the cost of quantization and achieve near-zero-sample quantization training.

[0070] Please refer to Figure 4 , Figure 4 The diagram shows a flowchart of a post-training quantization method according to another embodiment of this application, which may include steps S210 to S270.

[0071] Step S210: Obtain the image to be calibrated, wherein the image to be calibrated is a face image.

[0072] Step S210 has been described in detail in the above embodiments, and will not be repeated here.

[0073] Step S220: Input the image to be calibrated into the floating-point network, initialize the network parameters corresponding to the normalization layer of the floating-point network, and obtain the first parameter.

[0074] In one approach, after acquiring the image to be calibrated, this embodiment can first acquire a standard face image, also known as an average face image, which can be obtained by averaging multiple face images. After acquiring the standard face image, the electronic device can align the image to be calibrated onto the standard face image. Specifically, face image alignment can be achieved through normalized rotation, scaling, and displacement. Furthermore, the parameters corresponding to rotation, scaling, and displacement can be obtained based on the mean and variance of batch normalization, i.e., by calculating the loss function corresponding to the mean and variance and iterating through the search scale and displacement parameters.

[0075] In some implementations, after obtaining the floating-point network, the parameters and weight parameters corresponding to all batch normalization (BN) in the floating-point network can be fixed in advance, and the mean and variance of all batch normalization layers in the floating-point network can be summarized. The mean and variance can be collectively referred to as the second parameter.

[0076] In other embodiments, the image to be calibrated is input to a floating-point network. The electronic device can first initialize the network parameters corresponding to the floating-point network to obtain a first parameter, which may include an affine matrix, the mean and variance of the image preprocessing, etc. In a specific embodiment, the affine matrix corresponding to the floating-point network can be initialized to an identity matrix, which can be an identity matrix, such as an affine matrix initialized to... At the same time, the mean and variance of the batch normalization layer (BN layer) of the floating-point network can be initialized to 0 and 1, respectively, that is, the mean is initialized to 0 and the variance is initialized to 1.

[0077] Step S230: Use the initialized floating-point network to identify the image to be calibrated and obtain the identification result.

[0078] In some implementations, after initializing the parameters corresponding to the floating-point network, the embodiments of this application can use the initialized floating-point network to identify the image to be calibrated and obtain the identification result. Then, the electronic device can optimize the first parameter based on the result to obtain the target parameter, i.e., proceed to step S240.

[0079] Step S240: Optimize the first parameter based on the recognition result to obtain the target parameter.

[0080] In this embodiment, after obtaining the recognition result, the electronic device can optimize the first parameter based on the recognition result to obtain the target parameter. Specifically, this embodiment can use the gradient descent method to continuously optimize the first parameter so that the first parameter can more accurately recognize the face.

[0081] In one approach, this embodiment utilizes the mean and variance of the batch normalization (BN) of the floating-point network with respect to the input obtained through forward propagation. Then, it optimizes the mean, variance, and affine matrix parameters of the floating-point network by calculating the loss value and performing backpropagation. Finally, different affine matrices and image preprocessing parameters can be optimized for different images to be calibrated. These image preprocessing parameters may include the mean and variance. The standardized face image can be obtained using this affine matrix, and this standardized face image can then be used in the subsequent quantization process.

[0082] As another way, please refer to Figure 5 Step S240 may include steps S241 to S242.

[0083] Step S241: Obtain the network parameters corresponding to the normalized layer of the floating-point network before initialization, and use these parameters as the second parameter.

[0084] As explained above, after initializing a floating-point network, the network parameters can be considered the first parameter. The network parameters before initialization can be called the second parameter. This second parameter can include the mean and variance of each batch normalization layer of the floating-point network before initialization. Additionally, the second parameter can also include the affine matrix of the floating-point network before initialization, which can be obtained through training with a large number of samples.

[0085] In this embodiment, the second parameter can be the network parameters corresponding to the floating-point network when the electronic device acquires the floating-point network. Therefore, the second parameter can be a superior mean and variance obtained through training with a large amount of data. By using this mean and variance as a standard and combining it with the image to be calibrated, this embodiment can continuously optimize the first parameter to obtain the optimal first parameter.

[0086] Step S242: When the optimized first parameter and the second parameter meet the preset conditions, the optimized first parameter is used as the target parameter.

[0087] In one approach, embodiments of this application can obtain the loss value between the optimized first parameter and the second parameter, and then determine whether the loss value is less than a loss threshold. If the loss value is determined to be less than the loss threshold, the optimized first parameter is used as the target parameter. If the loss value is greater than or equal to the loss threshold, the first parameter is optimized until the loss value between the optimized first parameter and the second parameter is less than the loss threshold, at which point the optimization stops.

[0088] In one specific implementation, the formula for calculating the loss value can be:

[0089]

[0090] Wherein, loss refers to the loss value between the first parameter and the second parameter, and x(θ) refers to the image to be calibrated. and This refers to the mean and variance in the first parameter. and This represents the mean and variance in the second parameter, while i refers to the i-th layer of the floating-point network.

[0091] Step S250: Obtain the affine matrix corresponding to the target parameters, and use the affine matrix as the affine matrix corresponding to the image to be calibrated. The affine matrix includes multiple face calibration parameters.

[0092] Step S260: The image to be calibrated is calibrated using the affine matrix to obtain the target image.

[0093] In one approach, after obtaining the affine matrix, this embodiment of the application can use the affine matrix to perform a differentiable affine transformation on the image to be calibrated. Specifically, the electronic device can use PyTorch's affine_grid and grad_sample to implement the affine transformation. The input of the affine transformation is the affine matrix and the image to be calibrated, and the output is the affine image corresponding to the image to be calibrated.

[0094] It should be noted that, according to the specification, the embodiments of this application can obtain the optimal affine matrix by utilizing the mean and variance corresponding to batch normalization, and the calibration of the image to be calibrated can be achieved through this affine transformation. Furthermore, by utilizing the mean and variance corresponding to batch normalization, the embodiments of this application can also obtain the optimal image samples, thus enabling near-zero sample quantization training.

[0095] Step S270: Quantize the floating-point network based on the target image.

[0096] The post-training quantization method provided in this invention first acquires an image to be calibrated, which may be a face image. Then, the acquired image to be calibrated is input into a floating-point network, and the parameters of the floating-point network are optimized using the image to obtain target parameters. The network parameters of the floating-point network are the network parameters corresponding to the normalization layer of the floating-point network. Next, an affine matrix corresponding to the target parameters is obtained and used as the affine matrix corresponding to the image to be calibrated. The affine matrix may include multiple face calibration parameters. Finally, the image to be calibrated is calibrated according to the obtained affine matrix to obtain the target image, and the floating-point network is quantized based on the target image. Thus, this application can reduce the cost of quantization and achieve near-zero-sample quantization training. Furthermore, this application embodiment obtains the corresponding affine matrix by continuously optimizing parameters such as mean and variance. This not only reduces the difficulty of parameter optimization but also generates more realistic face data. Moreover, the optimization freedom information in this application embodiment is small, which can simply and effectively solve the pain point of lacking calibration data in human post-quantization training.

[0097] Please refer to Figure 6 , Figure 6 The diagram shows a flowchart of a post-training quantization method according to another embodiment of this application, which may include steps S310 to S370.

[0098] Step S310: Obtain the face calibration network and obtain multiple sample affine matrices.

[0099] In this embodiment of the application, the multiple sample affine matrices can be obtained by training multiple face images, wherein each face image can include 5 facial key points, and the 5 facial key points can be the two eyes, the tip of the nose, and the two corners of the mouth. Figure 7 Keypoints 201 and 202 shown can be referred to as keypoints corresponding to the left and right eyes, keypoint 203 can be the keypoint of the tip of the nose, and keypoints 204 and 205 can be the keypoints corresponding to the left and right corners of the mouth. Therefore, each face image can include 5 facial keypoints.

[0100] In this embodiment, the multiple sample affine matrices can be obtained through multiple face image data, and the storage method for the multiple face image data can be cloud storage. Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to aggregate a large number of storage devices (also called storage nodes) of various types in the network through application software or application interfaces to work together and jointly provide data storage and business access functions to the outside world.

[0101] Currently, the storage method of storage systems is as follows: Logical volumes are created. During the creation of a logical volume, physical storage space is allocated to each logical volume. This physical storage space may consist of a single storage device or the disks of several storage devices. Clients store data on a logical volume, which means storing the data on the file system. The file system divides the data into many parts, each part being an object. Each object contains not only the data but also additional information such as a data identifier (ID, ID entity). The file system writes each object to the physical storage space of that logical volume and records the storage location information of each object. Therefore, when a client requests access to data, the file system can allow the client to access the data based on the storage location information of each object.

[0102] The process by which a storage system allocates physical storage space to a logical volume is as follows: the physical storage space is pre-divided into strips according to the capacity estimate of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of Redundant Array of Independent Disks (RAID). A logical volume can be understood as a strip, thus allocating physical storage space to the logical volume.

[0103] In some implementations, multiple facial image data can be acquired through big data methods. Big data refers to data sets that cannot be captured, managed, and processed within a certain timeframe using conventional software tools. It represents massive, rapidly growing, and diverse information assets that require new processing models to achieve stronger decision-making, insight discovery, and process optimization capabilities. With the advent of the cloud era, big data has attracted increasing attention. Big data requires specialized technologies to effectively process large amounts of data within a tolerable timeframe. Technologies suitable for big data include massively parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems.

[0104] As a method, when performing tasks such as facial landmark detection, face recognition, and face classification, the position, angle, and scale of the face in the acquired face image are arbitrary. In order to better achieve operations such as face detection and face recognition, it is necessary to first standardize the position, angle, and scale of the face. This allows the final network model to focus more directly on the recognition task, while also improving the recognition accuracy.

[0105] As another approach, standardizing acquired face images can be achieved by calculating the affine matrix between five facial landmarks and the average face. Specifically, firstly, the left and right eyes, nose tip, and left and right corners of the mouth in the face image are manually labeled, resulting in a total of five landmarks. Multiple labeled face datasets are obtained by standardizing multiple face datasets. Then, a subset of frontal face images are selected, and the bounding box and center of the face are calculated based on the five facial landmarks. The images are then normalized to a 256x256 image size by using the two corners of the eyes as references. Finally, the average of the five standardized facial landmarks is calculated to obtain the average face coordinates. Furthermore, for the most recently acquired face image, its five facial landmarks are known; the affine matrix between these five landmarks and the average face can be optimized using the least squares method.

[0106] Step S320: Train the face calibration network based on the multiple sample affine matrices to obtain the floating-point network.

[0107] As described above, this embodiment of the application can obtain an average face from multiple face images, and then use the least squares method to obtain the sample affine matrix between each face image and the average face. Finally, the face calibration network can be trained using the multiple sample affine matrices, thereby obtaining a floating-point network.

[0108] Step S330: Obtain the image to be calibrated, wherein the image to be calibrated is a face image.

[0109] Step S340: Input the image to be calibrated into the floating-point network, and use the image to be calibrated to optimize the network parameters of the floating-point network to obtain the target parameters.

[0110] In other implementations, please refer to Figure 8 Step S340 may include steps S341 to S342.

[0111] Step S341: Determine the position of the face image in the image to be calibrated, and crop the image to be calibrated according to the position to obtain the target face image.

[0112] As described above, each image to be calibrated contains a face image. To achieve more accurate face recognition, this embodiment of the application, upon acquiring the face image to be calibrated, can determine the position of the face image within the image, and then crop the image to be calibrated based on the determined position to obtain the target face image. This embodiment of the application can use an open-source face detection algorithm to detect the image to be calibrated and crop out the corresponding face image. For example, for... Figure 7 By detecting and cropping the faces in the image to be calibrated, the following result can be obtained: Figure 9 The image shown is a human face.

[0113] Step S342: Input the target face image into the floating-point network, and use the target image to optimize the network parameters of the floating-point network to obtain the target parameters.

[0114] Step S350: Obtain the affine matrix corresponding to the target parameters, and use the affine matrix as the affine matrix corresponding to the image to be calibrated. The affine matrix includes multiple face calibration parameters.

[0115] Step S360: The image to be calibrated is calibrated using the affine matrix to obtain the target image.

[0116] In this embodiment of the application, the image to be calibrated is cropped to obtain the following... Figure 9 After obtaining the target image shown, this embodiment of the application can obtain the affine matrix corresponding to the target face image, and then use the affine matrix to calibrate the target face image to obtain the target image. For example, for Figure 9 By calibrating the face image shown, you can obtain the following: Figure 10 The example image shown. By comparison Figure 9 and Figure 10 It can be observed that in the embodiments of this application, the orientation of the face image is changed when performing affine transformation on the target face image using an affine matrix.

[0117] As one way, please refer to Figure 11 Step S360 may include steps S361 to S362.

[0118] Step S361: Determine at least one of the rotation parameter, scale parameter, and displacement parameter corresponding to the affine matrix.

[0119] In some implementations, after obtaining the affine matrix, the embodiments of this application can determine what parameters correspond to the affine matrix, and which parameters are valid parameters, wherein valid parameters can be non-zero parameters. As described above, the parameters of the affine matrix mainly include a, b, and T. x And T yWhere a and b are related to the rotation and scaling of the face image, while T x And T y This relates to the translation of the face image. When a and b are 0, no rotation or scaling is needed, while when T... x And T y When a is 0, no translation is needed. Additionally, when a, b, and T... x And T y When none of the values ​​are 0, the embodiments of this application can perform operations such as rotation, scaling, or translation based on the corresponding values.

[0120] Step S362: Rotate, scale, or translate the image to be calibrated according to the affine matrix to obtain the target image.

[0121] In some implementations, before rotating, scaling, or translating the image to be calibrated according to the affine matrix, this embodiment can first determine each parameter corresponding to the affine matrix, and then determine whether each parameter is valid data. If it is valid data, it is determined whether each data is less than a preset value. If it is less than the preset value, then there is no need to rotate, scale, or translate the image to be calibrated. If it is greater than or equal to the preset value, this embodiment can perform operations such as rotation, scaling, or translation on the image to be calibrated according to the corresponding value. The preset values ​​are different for different parameters in the affine matrix. For example, the preset value for 'a' can be A, while the preset value for 'b' can be B. The specific value of the preset value is not explicitly limited here and can be set according to the actual situation.

[0122] Step S370: Quantize the floating-point network based on the target image.

[0123] The post-training quantization method provided in this invention first acquires an image to be calibrated, which may be a face image. Then, the acquired image to be calibrated is input into a floating-point network, and the parameters of the floating-point network are optimized using the image to obtain target parameters. The network parameters of the floating-point network are the network parameters corresponding to the normalization layer of the floating-point network. Next, an affine matrix corresponding to the target parameters is acquired and used as the affine matrix corresponding to the image to be calibrated. The affine matrix may include multiple face calibration parameters. Finally, the image to be calibrated is calibrated according to the acquired affine matrix to obtain the target image, and the floating-point network is quantized based on the target image. Thus, this application can reduce the cost of quantization, achieving near-zero-sample quantization training. Furthermore, by acquiring the affine matrix, this application reduces the dependence of post-training quantization on training data, and the image to be calibrated can be calibrated according to the actual situation of the affine matrix, which can improve quantization efficiency to a certain extent. Moreover, in post-training quantization with numerical activation, this application eliminates the need to use samples to calibrate numerical ranges by introducing the affine matrix.

[0124] Please see Figure 12 The diagram illustrates a structural block diagram of a post-training quantization device provided in an embodiment of this application. The device 400 may include: a first acquisition module 410, an input module 420, a second acquisition module 430, a calibration module 440, and a quantization module 450.

[0125] The first acquisition module 410 is used to acquire the image to be calibrated, which is a face image.

[0126] The input module 420 is used to input the image to be calibrated into the floating-point network and optimize the network parameters of the floating-point network using the image to be calibrated to obtain target parameters. The network parameters of the floating-point network are the network parameters corresponding to the normalization layer of the floating-point network.

[0127] Please see Figure 13 The input module 420 may include an initialization unit 421, a recognition unit 422, and an optimization unit 423.

[0128] The initialization unit 421 is used to initialize the network parameters corresponding to the normalization layer of the floating-point network to obtain the first parameter.

[0129] The recognition unit 422 is used to recognize the image to be calibrated using the initialized floating-point network to obtain the recognition result.

[0130] The optimization unit 423 is used to optimize the first parameter based on the recognition result to obtain the target parameter.

[0131] Furthermore, the optimization unit 423 is also used to obtain the network parameters corresponding to the normalized layer of the floating-point network before initialization, and use the parameters as the second parameter; when the optimized first parameter and the second parameter satisfy a preset condition, the optimized first parameter is used as the target parameter.

[0132] Furthermore, the optimization unit 423 is also used to obtain the loss value between the optimized first parameter and the second parameter; when the loss value is less than the loss threshold, the optimized first parameter is used as the target parameter.

[0133] Optionally, the formula for calculating the loss value is: Wherein, loss refers to the loss value between the first parameter and the second parameter, and x(θ) refers to the image to be calibrated. and This refers to the mean and variance in the first parameter. and This represents the mean and variance in the second parameter.

[0134] Furthermore, the input module 420 is also used to determine the position of the face image in the image to be calibrated, and to crop the image to be calibrated according to the position to obtain a target face image; and to input the target face image into the floating-point network.

[0135] The second acquisition module 430 is used to acquire the affine matrix corresponding to the target parameters and use the affine matrix as the affine matrix corresponding to the image to be calibrated. The affine matrix includes multiple face calibration parameters.

[0136] The calibration module 440 is used to calibrate the image to be calibrated using the affine matrix to obtain the target image.

[0137] Furthermore, the calibration module 440 is also used to determine at least one of the rotation parameter, scale parameter, and displacement parameter corresponding to the affine matrix; and to rotate, scale, or translate the image to be calibrated according to the affine matrix to obtain the target image.

[0138] The quantization module 450 is used to quantize the floating-point network according to the target image, and the quantized floating-point network is used to perform face recognition operation on the image to be calibrated.

[0139] Furthermore, before inputting the image to be calibrated into the floating-point network to obtain the affine matrix, the post-training quantization device 400 is also used to acquire the face calibration network and acquire multiple sample affine matrices; the face calibration network is trained according to the multiple sample affine matrices to obtain the floating-point network.

[0140] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0141] In the several embodiments provided in this application, the coupling or direct coupling or communication connection between the modules shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be electrical, mechanical or other forms.

[0142] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0143] Please refer to Figure 14 This diagram illustrates a structural block diagram of an electronic device 900 provided in an embodiment of this application. The electronic device 900 can be the aforementioned alarm subsystem. The electronic device 900 in this application may include one or more of the following components: a processor 510, a memory 520, and one or more programs, wherein the one or more programs can be stored in the memory 520 and configured to be executed by one or more processors 510, and the one or more programs are configured to perform the methods described in the foregoing method embodiments.

[0144] Processor 510 may include one or more processing cores. Processor 510 connects to various parts within the electronic device 500 using various interfaces and lines, and performs various functions and processes data of the electronic device 500 by running or executing instructions, programs, code sets, or instruction sets stored in memory 520, and by calling data stored in memory 520. Optionally, processor 510 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 510 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 510 and may be implemented separately using a communication chip.

[0145] The memory 520 may include random access memory (RAM) or read-only memory (ROM). The memory 520 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 520 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the electronic device 500 during use (such as images to be recognized, encoded information, etc.).

[0146] Please refer to Figure 15 This diagram illustrates a structural block diagram of a computer-readable storage medium 1000 provided in an embodiment of this application. The computer-readable medium 600 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0147] The computer-readable storage medium 600 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 600 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 600 has storage space for program code 610 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 610 may be compressed, for example, in a suitable form.

[0148] Furthermore, embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium (e.g., 600). A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the post-training quantization method provided in the above embodiments of this application.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A post-training quantization method, characterized in that, The method includes: Acquire an image to be calibrated, wherein the image to be calibrated is a face image; The image to be calibrated is input into a floating-point network, and the network parameters of the floating-point network are optimized using the image to obtain target parameters. The network parameters of the floating-point network are the network parameters corresponding to the normalization layer of the floating-point network. Obtain the affine matrix corresponding to the target parameters, and use the affine matrix as the affine matrix corresponding to the image to be calibrated. The affine matrix includes multiple face calibration parameters. The image to be calibrated is calibrated using the affine matrix to obtain the target image; The floating-point network is quantized based on the target image.

2. The method according to claim 1, characterized in that, The optimization of the floating-point network parameters using the image to be calibrated to obtain target parameters includes: The network parameters corresponding to the normalization layer of the floating-point network are initialized to obtain the first parameter; The image to be calibrated is identified using the initialized floating-point network to obtain the identification result; The first parameter is optimized based on the recognition result to obtain the target parameter.

3. The method according to claim 2, characterized in that, The step of optimizing the first parameter based on the recognition result to obtain the target parameter includes: Obtain the network parameters corresponding to the normalized layer of the floating-point network before initialization, and use these parameters as the second parameter; When the optimized first parameter and the second parameter meet the preset conditions, the optimized first parameter is used as the target parameter.

4. The method according to claim 3, characterized in that, When the optimized first parameter and the second parameter satisfy a preset condition, the optimized first parameter is used as the target parameter, including: Obtain the loss value between the optimized first parameter and the second parameter; When the loss value is less than the loss threshold, the optimized first parameter is used as the target parameter.

5. The method according to claim 4, characterized in that, The formula for calculating the loss value is: Wherein, loss refers to the loss value between the first parameter and the second parameter, and x(θ) refers to the image to be calibrated. and This refers to the mean and variance in the first parameter. and This represents the mean and variance in the second parameter.

6. The method according to any one of claims 1 to 5, characterized in that, Before inputting the image to be calibrated into a floating-point network to obtain the affine matrix, the following steps are included: Obtain the face calibration network and multiple sample affine matrices; The face calibration network is trained based on the multiple sample affine matrices to obtain the floating-point network.

7. The method according to any one of claims 1 to 5, characterized in that, The step of calibrating the image to be calibrated using the affine matrix to obtain the target image includes: Determine at least one of the rotation parameter, scale parameter, and displacement parameter corresponding to the affine matrix; The target image is obtained by rotating, scaling, or translating the image to be calibrated according to the affine matrix.

8. The method according to any one of claims 1 to 5, characterized in that, The step of inputting the image to be calibrated into a floating-point network includes: The position of the face image in the image to be calibrated is determined, and the image to be calibrated is cropped according to the position to obtain the target face image; The target face image is input into the floating-point network.

9. A post-training quantization device, characterized in that, The device includes: The first acquisition module is used to acquire the image to be calibrated, wherein the image to be calibrated is a face image; The input module is used to input the image to be calibrated into the floating-point network and optimize the network parameters of the floating-point network using the image to be calibrated to obtain target parameters. The network parameters of the floating-point network are the network parameters corresponding to the normalization layer of the floating-point network. The second acquisition module is used to acquire the affine matrix corresponding to the target parameters and use the affine matrix as the affine matrix corresponding to the image to be calibrated. The affine matrix includes multiple face calibration parameters. A calibration module is used to calibrate the image to be calibrated using the affine matrix to obtain a target image; The quantization module is used to quantize the floating-point network based on the target image, and the quantized floating-point network is used to perform face recognition operations on the image to be calibrated.

10. An electronic device, characterized in that, include: One or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to execute the method as described in any one of claims 1-8.

Images