Model quantification method and device based on diffusion model generation data
By using diffusion models to generate pseudo-training data in scenarios such as coal mines, combined with UNet network structure and multi-head attention mechanism, the problem that cannot be quantified in the existing technology is solved, and an efficient model on embedded devices is realized Quantization ensures the real-time operation of the model.
Patent Information
- Application Number
- CN202411124196.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-26
AI Technical Summary
In scenarios such as coal mines that lack training data, existing model quantization methods cannot effectively quantify models, resulting in AI models being unable to be deployed and run in real time on embedded devices.
Using a method of generating data based on diffusion model, pseudo-training data is generated through UNet network structure and multi-head attention mechanism, which is used to quantify the model and realize model quantization without training data.
The generated pseudo-training data distribution is basically consistent with the original data, and the quantization accuracy is basically consistent with the floating-point accuracy, realizing model quantization without training data, and is suitable for embedded devices with limited resources.
Smart Images

Figure CN119026645A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of model quantization, and in particular to a model quantization method, device, electronic device and storage medium based on diffusion model-based data generation. Background Art
[0002] As the needs of different coal scenarios become more diverse, more and more sensor devices and scenarios need to make decisions and operations with the closest low latency to the data collection point; therefore, AI models need to deploy high-performance neural network models on mobile devices and run them in real time in real scenarios, and be able to process data collected from the site in real time. However, the characteristics of mobile / embedded devices are small memory resources, low processor performance, and limited power consumption, which makes it impossible to deploy the most accurate models on these devices and achieve real-time operation. In addition, the network transmission bandwidth underground in coal mines is low, which brings great challenges to data security and privacy. Embedded device computing is small in size, low in power consumption, and low in cost, making lightweight computing underground in coal mines a major trend in the future.
[0003] Low-bit model quantization is the main method to accelerate AI models in embedded computing, which converts the trained floating-point network model into fixed-point operations, and then cooperates with specific computing instructions such as SIMD (single instruction multiple data streams) to increase the computing speed and reduce memory usage. Therefore, model quantization is an effective way to increase the model running speed and reduce energy consumption by quantizing floating-point values to low precision (such as 4 bits) to reduce the model size.
[0004] However, in the process of model quantization, some training data is often required for quantitative calibration. However, in coal mines, training data is often difficult to obtain due to the scarcity of underground data or for some privacy reasons. Due to the lack of training data, existing quantitative methods are no longer applicable to model quantization in coal mine scenarios or other scenarios where training data is lacking. Therefore, it is of great practical value to achieve model quantization without training data. Summary of the invention
[0005] The present application aims to solve one of the technical problems in the related art at least to some extent.
[0006] To this end, the first purpose of this application is to propose a model quantization method based on diffusion model data generation to achieve model quantization without training data, and to solve the problem in related technologies that due to the lack of training data, the existing quantization methods are no longer applicable to model quantization in coal mine scenarios or other scenarios where there is a lack of training data.
[0007] The second objective of the present application is to provide a model quantization device for generating data based on a diffusion model.
[0008] The third objective of the present application is to provide an electronic device.
[0009] A fourth objective of the present application is to provide a computer-readable storage medium.
[0010] A fifth object of the present application is to provide a computer program product.
[0011] To achieve the above-mentioned purpose, the first embodiment of the present application proposes a model quantization method based on diffusion model generated data, including:
[0012] Get a sample of the Gaussian distributed data set;
[0013] Based on the Gaussian distributed data set sampling samples and the model to be quantized, pseudo training data is generated through a diffusion model; the diffusion model adopts a UNet network structure as a core architecture, and a multi-head attention mechanism is added to the encoder backend of the UNet network structure, and the multi-head attention mechanism is used to establish the association relationship between the elements in the feature map, so that the receptive field of the UNet network structure is the entire image;
[0014] The model to be quantized is subjected to model quantization processing using the pseudo training data to obtain a low-bit quantization model.
[0015] To achieve the above-mentioned purpose, the second embodiment of the present application proposes a model quantization device based on diffusion model-based data generation, including:
[0016] A data sampling module is used to obtain a sample of a Gaussian distributed data set;
[0017] A data generation module is used to generate pseudo training data through a diffusion model based on the Gaussian distributed data set sampling samples and the model to be quantized; the diffusion model adopts a UNet network structure as the core architecture, and a multi-head attention mechanism is added to the encoder backend of the UNet network structure, and the multi-head attention mechanism is used to establish the association relationship between the elements in the feature map so that the receptive field of the UNet network structure is the entire image;
[0018] The model quantization module is used to perform model quantization processing on the model to be quantized using the pseudo training data to obtain a low-bit quantization model.
[0019] To achieve the above-mentioned purpose, the third aspect embodiment of the present application proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method described in the first aspect.
[0020] To achieve the above-mentioned purpose, the fourth aspect of the present application proposes a computer-readable storage medium, in which computer-readable storage medium is stored computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.
[0021] To achieve the above-mentioned purpose, the fifth aspect of the present application proposes a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.
[0022] The model quantization method, device, electronic device and storage medium provided in the present application based on the diffusion model to generate data, generate pseudo training data that can effectively quantize the model through the diffusion model, the distribution of the generated pseudo training data can be basically consistent with the original data, and the quantization accuracy is basically consistent with the original floating point accuracy of the model to be quantized; the model is quantized by the generated pseudo training data, thereby realizing the quantization of the model without training data. In addition, the Unet underlying network is modified in the diffusion model, and after the multi-head attention mechanism is added to the encoder at the end, the receptive field of the Unet network structure is the entire image. After the multi-head cross attention mechanism is used in the jump connection in the decoder, it combines the high-level semantically richer feature map with the high-resolution map from the jump connection to make the data generated by the diffusion model closer to the original data distribution.
[0023] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0025] Figure 1 A schematic flow chart of a model quantification method for generating data based on a diffusion model provided in an embodiment of the present application;
[0026] Figure 2 A schematic diagram of the UNet network structure provided in the embodiment of the present application;
[0027] Figure 3 A diagram of the network structure of the multi-head attention mechanism provided in the embodiment of the present application;
[0028] Figure 4 A block diagram of a model quantization device for generating data based on a diffusion model provided in an embodiment of the present application;
[0029] Figure 5 A block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0030] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0031] The following describes a model quantization method, device, and apparatus for generating data based on a diffusion model according to an embodiment of the present application with reference to the accompanying drawings.
[0032] Figure 1 A schematic flow chart of a model quantization method for generating data based on a diffusion model provided in an embodiment of the present application.
[0033] It should be noted that the executor of the model quantization method based on the diffusion model to generate data in the embodiment of the present application is the model quantization device based on the diffusion model to generate data in the embodiment of the present application. The model quantization device based on the diffusion model to generate data can be configured in an electronic device so that the electronic device can perform the model quantization function based on the diffusion model to generate data.
[0034] like Figure 1 As shown, the model quantification method based on the diffusion model to generate data includes the following steps:
[0035] Step 101, obtaining a Gaussian distributed data set sampling sample.
[0036] As an implementation method, the implementation method of obtaining a Gaussian distributed data set sampling sample includes: simulating an image of a standard Gaussian distribution and generating an image sample that conforms to the Gaussian distribution.
[0037] Step 102, based on the Gaussian distributed data set sampling samples and the model to be quantized, pseudo training data is generated through a diffusion model; the diffusion model uses a UNet network structure as the core architecture, and a multi-head attention mechanism is added to the encoder backend of the UNet network structure, and the multi-head attention mechanism is used to establish the association relationship between the elements in the feature map so that the receptive field of the UNet network structure is the entire image. The UNet network structure is a convolutional neural network structure commonly used for image segmentation.
[0038] In some embodiments, the sampling of the data set based on the Gaussian distribution and the model to be quantized, generating pseudo training data through a diffusion model, comprises:
[0039] In the diffusion model training stage, the diffusion model is trained based on the Gaussian distributed data set sampling samples and the model to be quantized to obtain a trained diffusion model;
[0040] In the diffusion model resampling stage, the pseudo training data is obtained based on the trained diffusion model.
[0041] Furthermore, in some embodiments, the sampling of the data set based on the Gaussian distribution and the model to be quantized, training the diffusion model to obtain a trained diffusion model, comprises:
[0042] Inputting the Gaussian distribution data set sampling samples into the model to be quantized to obtain Gaussian distribution data of the BN layer of the diffusion model;
[0043] Generate Gaussian noise based on Gaussian distribution data of the diffusion model BN layer;
[0044] Iterate the following steps until the number of iterations is reached and a trained diffusion model is obtained;
[0045] Based on the Gaussian noise, calling the diffusion model to obtain estimated noise;
[0046] Calculating a noise loss between the Gaussian noise and the estimated noise;
[0047] Based on the noise loss, the diffusion model is trained by back propagation.
[0048] As a specific implementation method, in the diffusion model training stage, the following training process is constructed based on the graph entity of the knowledge base system:
[0049] 1) Generate a Gaussian distributed data set sample x0 and randomly select the time step;
[0050] 2) Send the sampled samples to the full-precision model (M) to obtain the ∈ l =N(u l , σ 2 l I); wherein the full-precision model (M) is the model to be quantized.
[0051] 3) Generate Gaussian noise ∈ t =N(0,I);
[0052] 4) Call the diffusion model to estimate noise
[0053] 5) The noise loss MSE Loss between Gaussian noise and estimated noise is calculated by the following formula, and the diffusion model is trained using back propagation.
[0054]
[0055] Through the above implementation, in the absence of original data, data classification boundary knowledge and distribution information are mined from the full-precision model (M) to train the diffusion model to generate pseudo training data that can effectively quantize the model (model to be quantized Q).
[0056] As a specific implementation method, in the diffusion model resampling stage, the following process is implemented:
[0057] 1) Sample X from a Gaussian distribution T
[0058] 2) Perform the following iterations in the order of T, ...1:
[0059] a. If t = 1, let z = 0. If t> 1, sample z~N(0, I) from the Gaussian distribution;
[0060] b. Learn the mean And calculate the value of mean square error
[0061] c. Through the re-parameter sampling technique x t-1 =μ θ (x t , t)+σ t z.
[0062] Through the iteration of the above process, we finally get the pseudo data sample x0, that is, pseudo training data.
[0063] It should be noted that the UNet network structure is usually used for training in the diffusion model. It is a model structure that includes downsampling and upsampling. Since UNet cannot model long-range relationships and spatial dependencies, the embodiment of the present application adds a multi-head attention mechanism (MHSA) at the bottom after UNet completes the encoder part. The modified UNet network is as follows: Figure 2 The main function of the multi-head attention mechanism is to associate each element in the feature map. The corresponding receptive field is the entire image. Therefore, the classification of the current pixel depends on all input pixels. The MHSA network structure is as follows: Figure 3 As shown, the calculation formula of multi-head attention is as follows:
[0064]
[0065] This embodiment modifies the Unet underlying network in the diffusion model. After adding a multi-head attention mechanism at the end of the encoder, the receptive field of the Unet network structure is the entire image. After the multi-head cross-attention mechanism is used in the jump connection in the decoder, it combines the high-level semantically richer feature map with the high-resolution map from the jump connection to make the data generated by the diffusion model closer to the original data distribution.
[0066] Step 103: Perform model quantization processing on the model to be quantized using the pseudo training data to obtain a low-bit quantization model.
[0067] After the pseudo training data is generated by the diffusion model in the previous step, this step quantizes the model using the pseudo training data.
[0068] In some embodiments, the model to be quantized is subjected to model quantization processing by using the pseudo training data to obtain a low-bit quantization model; including:
[0069] Through the pseudo training data, the Pearson cross-correlation coefficient between the quantized model and the inverse quantized model obtained by inverse quantization of the quantized model is maximized as the optimization goal, and an exhaustive search algorithm is used to perform model quantization processing on the model to be quantized to obtain a low-bit quantization model.
[0070] As an implementation method, the basic principles of model quantization and model dequantization are as follows:
[0071]
[0072] Among them, X is the input tensor tensor, S is the scaling factor, Clip is the clipping process, and Round is the rounding process; n and p are the minimum and maximum values that can be represented by fixed-point numbers, respectively.
[0073] In order to measure the size of the quantization loss, the quantized fixed-point number Q can be L Dequantize back to floating point Then calculate Q L and The difference between S and Q is the loss caused by quantization. The goal of the quantization algorithm is to make the quantization loss as small as possible. In this embodiment, an iterative optimization strategy is used to find the best S. In order to measure Q L and The difference between the two, this embodiment uses the Pearson correlation coefficient to evaluate the loss size, and the Pearson correlation coefficient is expressed as follows:
[0074]
[0075] Among them, Q L =(X1, X2, ...X i ), is (X1, X2, ...X i ), is (Y1, Y2, ...Y i ), the Pearson correlation coefficient is used to measure the mean of two variables Q L and The linear correlation between them is between -1 and 1. Some quantization methods in related technologies use cosine similarity. Since cosine similarity is only related to the direction of the vector, it will be affected by the translation of the vector. In the angle cosine formula, if X is translated to X+1, the cosine value will change. The Pearson Correlation coefficient will not be affected by this situation.
[0076] Therefore, in the quantization process, an exhaustive search algorithm is used to maximize the Pearson cross-correlation coefficient between the quantized true value and the dequantized value as the optimization goal to search for the optimal scaling factor S. Experiments show that the Pearson cross-correlation coefficient has good quantization convergence and can quickly approach the floating-point numerical precision model.
[0077] The model quantization method based on the diffusion model generation data of the embodiment of the present application generates pseudo training data that can effectively quantize the model through the diffusion model, and the distribution of the generated pseudo training data can be basically consistent with the original data, and the quantization accuracy is also basically consistent with the original floating-point accuracy of the model to be quantized; the model is quantized by the generated pseudo training data, thereby realizing the quantization of the model without training data. In addition, the Unet underlying network is modified in the diffusion model, and after the multi-head attention mechanism is added to the encoder at the end, the receptive field of the Unet network structure is the entire image. After the multi-head cross attention mechanism is used in the jump connection in the decoder, he combines the feature map with richer high-level semantics with the high-resolution map from the jump connection, so that the data generated by the diffusion model is closer to the original data distribution. In addition, in the quantization process, an exhaustive search algorithm is used to maximize the Pearson correlation coefficient between the quantized true value and the dequantized value as the optimization goal, and the optimal scaling factor is searched, which has good quantization convergence and can quickly approach the floating-point numerical precision model.
[0078] In order to implement the above embodiment, the present application also proposes a model quantization device for generating data based on a diffusion model. Figure 4 A block diagram of a model quantization device based on a diffusion model to generate data provided by an embodiment of the present application. Figure 4 As shown, the model quantization device for generating data based on the diffusion model may include: a data sampling module 201 , a data generating module 202 and a model quantization module 203 .
[0079] The data sampling module 201 is used to obtain a sample of a data set with a Gaussian distribution;
[0080] A data generation module 202 is used to generate pseudo training data through a diffusion model based on the Gaussian distributed data set sampling samples and the model to be quantized; the diffusion model uses a UNet network structure as a core architecture, and a multi-head attention mechanism is added to the encoder backend of the UNet network structure. The multi-head attention mechanism is used to establish the association relationship between the elements in the feature map so that the receptive field of the UNet network structure is the entire image;
[0081] The model quantization module 203 is used to perform model quantization processing on the model to be quantized using the pseudo training data to obtain a low-bit quantization model.
[0082] In some implementations, the data generation module 202 is specifically configured to:
[0083] In the diffusion model training stage, the diffusion model is trained based on the Gaussian distributed data set sampling samples and the model to be quantized to obtain a trained diffusion model;
[0084] In the diffusion model resampling stage, the pseudo training data is obtained based on the trained diffusion model.
[0085] In some implementations, the data generation module 202 trains the diffusion model based on the Gaussian distributed data set sampling samples and the model to be quantized to obtain the trained diffusion model; specifically, it is used to:
[0086] Inputting the Gaussian distribution data set sampling samples into the model to be quantized to obtain Gaussian distribution data of the BN layer of the diffusion model;
[0087] Generate Gaussian noise based on Gaussian distribution data of the diffusion model BN layer;
[0088] Iterate the following steps until the number of iterations is reached and a trained diffusion model is obtained;
[0089] Based on the Gaussian noise, calling the diffusion model to obtain estimated noise;
[0090] Calculating a noise loss between the Gaussian noise and the estimated noise;
[0091] Based on the noise loss, the diffusion model is trained by back propagation.
[0092] In some implementations, the model quantization module 203 is specifically configured to:
[0093] Through the pseudo training data, the Pearson cross-correlation coefficient between the quantized model and the inverse quantized model obtained by inverse quantization of the quantized model is maximized as the optimization goal, and an exhaustive search algorithm is used to perform model quantization processing on the model to be quantized to obtain a low-bit quantization model.
[0094] In some implementations, the formula for the model quantization process is expressed as follows:
[0095] It is expressed as follows:
[0096]
[0097] Among them, X is the input tensor tensor, S is the scaling factor, Clip is the clipping process, and Round is the rounding process; n and p are the minimum and maximum values that can be represented by fixed-point numbers, respectively.
[0098] It should be noted that the above explanation of the embodiment of the model quantization method for generating data based on a diffusion model is also applicable to the model quantization device for generating data based on a diffusion model in this embodiment, and will not be repeated here.
[0099] The model quantization device based on the diffusion model data generation in the embodiment of the present application generates pseudo training data that can effectively quantize the model through the diffusion model. The distribution of the generated pseudo training data can be basically consistent with the original data, and the quantization accuracy is also basically consistent with the original floating-point accuracy of the model to be quantized; the model is quantized by the generated pseudo training data, thereby realizing the quantization of the model without training data. In addition, the Unet underlying network is modified in the diffusion model. After the multi-head attention mechanism is added to the encoder at the end, the receptive field of the Unet network structure is the entire image. After the multi-head cross attention mechanism is used in the jump connection in the decoder, it combines the high-level semantically richer feature map with the high-resolution map from the jump connection to make the data generated by the diffusion model closer to the original data distribution.
[0100] In order to implement the above embodiment, the present application also proposes an electronic device. Figure 5 , Figure 5 is a block diagram of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 500 includes: a processor 501, and a memory 502 communicatively connected to the processor 501; the memory 502 stores computer-executable instructions; the processor 501 executes the computer-executable instructions stored in the memory to implement the method provided in the aforementioned embodiment.
[0101] In order to implement the above embodiments, the present application also proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the above embodiments.
[0102] In order to implement the above embodiments, the present application also proposes a computer program product, including a computer program, which implements the methods provided by the above embodiments when executed by a processor.
[0103] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this application are in compliance with relevant laws and regulations and do not violate public order and good morals.
[0104] It should be noted that personal information from users should be collected for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign the agreement / authorization including authorization of relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others who have access to personal information data comply with its privacy policy and procedures.
[0105] The present application is expected to provide an implementation scheme for users to selectively block the use or access of personal information data. That is, the present disclosure is expected to provide hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, the risk can be minimized by limiting data collection and deleting the data. In addition, when applicable, such personal information is de-identified to protect the privacy of the user.
[0106] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0107] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0108] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0109] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.
[0110] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0111] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0112] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0113] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.
Claims
1. A model quantization method based on diffusion model generated data, characterized in that: The following steps are involved: Get a sample of the Gaussian distributed data set; Based on the Gaussian distributed data set sampling samples and the model to be quantized, pseudo training data is generated through a diffusion model; the diffusion model adopts a UNet network structure as a core architecture, and a multi-head attention mechanism is added to the encoder backend of the UNet network structure, and the multi-head attention mechanism is used to establish the association relationship between the elements in the feature map, so that the receptive field of the UNet network structure is the entire image; The model to be quantized is subjected to model quantization processing using the pseudo training data to obtain a low-bit quantization model.
2. The method according to claim 1, characterized in that The data set sampling samples and the model to be quantized based on the Gaussian distribution generate pseudo training data through a diffusion model; including: In the diffusion model training stage, the diffusion model is trained based on the Gaussian distributed data set sampling samples and the model to be quantized to obtain a trained diffusion model; In the diffusion model resampling stage, the pseudo training data is obtained based on the trained diffusion model.
3. The method according to claim 2, characterized in that The sampling of the data set based on the Gaussian distribution and the model to be quantized, training the diffusion model to obtain a trained diffusion model, comprises: Inputting the Gaussian distribution data set sampling samples into the model to be quantized to obtain Gaussian distribution data of the BN layer of the diffusion model; Generate Gaussian noise based on Gaussian distribution data of the diffusion model BN layer; Iterate the following steps until the number of iterations is reached and a trained diffusion model is obtained; Based on the Gaussian noise, calling the diffusion model to obtain estimated noise; Calculating a noise loss between the Gaussian noise and the estimated noise; Based on the noise loss, the diffusion model is trained by back propagation.
4. The method according to claim 1, characterized in that: The method of performing model quantization processing on the model to be quantized by using the pseudo training data to obtain a low-bit quantization model comprises: Through the pseudo training data, the Pearson cross-correlation coefficient between the quantized model and the inverse quantized model obtained by inverse quantization of the quantized model is maximized as the optimization goal, and an exhaustive search algorithm is used to perform model quantization processing on the model to be quantized to obtain a low-bit quantization model.
5. The method according to claim 4, characterized in that The formula for the model quantization process is as follows: It is expressed as follows: Among them, X is the input tensor tensor, S is the scaling factor, Clip is the clipping process, and Round is the rounding process; n and p are the minimum and maximum values that can be represented by fixed-point numbers, respectively.
6. A model quantization device for generating data based on a diffusion model, characterized in that: include: A data sampling module is used to obtain a sample of a Gaussian distributed data set; A data generation module is used to generate pseudo training data through a diffusion model based on the Gaussian distributed data set sampling samples and the model to be quantized; the diffusion model adopts a UNet network structure as the core architecture, and a multi-head attention mechanism is added to the encoder backend of the UNet network structure, and the multi-head attention mechanism is used to establish the association relationship between the elements in the feature map so that the receptive field of the UNet network structure is the entire image; The model quantization module is used to perform model quantization processing on the model to be quantized using the pseudo training data to obtain a low-bit quantization model.
7. The device according to claim 6, characterized in that The model quantization module is specifically used for: Through the pseudo training data, the Pearson cross-correlation coefficient between the quantized model and the inverse quantized model obtained by inverse quantization of the quantized model is maximized as the optimization goal, and an exhaustive search algorithm is used to perform model quantization processing on the model to be quantized to obtain a low-bit quantization model.
8. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 5.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 5 when executed by a processor.
10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 5 when being executed by a processor.
Citation Information
Cited By
Time step self-adaptive diffusion model post-training quantification method
CN119722506A
