Data processing method

Abstract

The invention provides a data processing method. The data processing method comprises the following steps: loading a first group of models; quantizing the first group of models according to a unified quantization parameter to obtain a first group of quantized models; wherein the first group of models comprises at least one model, the first group of quantized models comprises at least one quantized model, and the unified quantization parameter is obtained according to a plurality of quantization parameters of the second group of models.

Description

Technical Field

[0001] The present invention relates to a data processing mechanism, and more particularly, to a data processing method for quantization in artificial intelligent (AI) generation tasks. Background Art

[0002] Artificial intelligent (AI) technology can be widely applied to various applications, including generation tasks such as generating images from text. In image generation tasks, various model adaptations can be used to obtain different styles and characters of images, so as to achieve customization / personalization to meet user needs.

[0003] Various model adaptations may involve different post-adaptation models. The neural network structures of these models have multiple weights and biases (i.e., for operations) and multiple activations (i.e., for input / output (I / O)). These weights, biases, and activations usually have floating-point values. For the purpose of inference efficiency, before being compiled and inferred, the model must be quantized to have fixed-point values or integer values.

[0004] In addition, before being quantized, the model must be calibrated to evaluate its value range. However, the calibration process is time-consuming, including collecting calibration data, analyzing each node and layer in the model using the calibration data, etc. When using a large number of post-adaptation models, each model must undergo the calibration process, which will result in a huge calibration time, thus reducing the model efficiency.

[0005] In view of the above problems, it is necessary to have an improved data processing mechanism that can improve model efficiency. Summary of the Invention

[0006] The present invention provides a method for data processing to effectively perform quantization.

[0007] According to an embodiment of the present invention, a data processing method is provided. The data processing method includes the following steps: loading a first set of models; and quantizing the first set of models according to unified quantization parameters to obtain a first set of quantized models; wherein the first set of models includes at least one model, the first set of quantized models includes at least one quantized model, and the unified quantization parameters are obtained according to multiple quantization parameters of a second set of models.

[0008] According to another embodiment of the present invention, a data processing method is provided. The data processing method includes the following steps: loading a network including a plurality of operation units; and quantifying the network according to unified quantization parameters to obtain a quantized network; wherein the unified quantization parameters are obtained according to a plurality of quantization parameters of a set of modified networks, and the set of modified networks is obtained by inputting multiple sets of weights into the network. Description of the Drawings

[0009] Figure 1 is a flowchart of a data processing method according to an embodiment of the present invention.

[0010] Figure 2 is a diagram showing Figure 1 a schematic diagram of the data processing method.

[0011] Figure 3 is a flowchart of a pre-calibration process for obtaining unified quantization parameters according to an embodiment of the present invention.

[0012] Figure 4 is a diagram showing Figure 3 a schematic diagram of the pre-calibration process.

[0013] Figure 5 is a schematic diagram for explaining the quantization parameters of the model.

[0014] Figure 6 is a flowchart of a data processing method according to another embodiment of the present invention.

[0015] Figure 7A is a schematic diagram showing an example structure of the network.

[0016] Figure 7B is a schematic diagram showing weighted model adaptation.

[0017] Figure 8 is a diagram showing Figure 6 a schematic diagram of the data processing method.

[0018] Figure 9 is in Figure 6 the embodiment, a flowchart of a pre-calibration process for obtaining unified quantization parameters.

[0019] Figure 10A is a diagram showing Figure 9 a schematic diagram of the pre-calibration process.

[0020] Figure 10B is a schematic diagram showing a modified network formed based on calibration data.

[0021] In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the disclosed embodiments. It will be apparent, however, that one or more embodiments may be practiced without these specific details. In other instances, well-known structures and devices are shown in schematic form in order to simplify the drawings. Detailed Description

[0022] The data processing method of the present invention is used to process computational models in artificial intelligence (AI) applications. More specifically, the data processing method provides a quantization mechanism for effectively quantizing computational models.

[0023] Figure 1 is a flowchart of a data processing method according to an embodiment of the present invention. Figure 2 shows Figure 1 a schematic diagram of the data processing method in. Referring to Figure 1 and Figure 2 , the data processing method starts from step S100: performing a quantization process. During the quantization process, a first set of models is quantized according to unified quantization parameters to obtain a first set of quantized models, where the first set of models includes at least one model. For example, in step S100, multiple models md1(1) to md1(N1) are processed. More specifically, each of the models md1(1) to md1(N1) is quantized according to the unified quantization parameter u_qt, thereby forming multiple quantized models md1_q(1) to md1_q(N1). The unified quantization parameter u_qt is predetermined. In some embodiments, the unified quantization parameter u_qt is obtained according to a pre-calibration process of a set of models md2(0) to md2(N2) (which may be referred to as the "second set of models"), and such a pre-calibration process will be discussed later with reference to Figures 3 - 5 . Still referring to Figure 1 and Figure 2 , during the quantization process, the quantization of the models md1(1) to md1(N1) may be performed by the quantization unit 100, where the quantization unit 100 may be a hardware circuit or a software module. These models md1(1) to md1(N1) have floating-point values (e.g., floating-point values with 32 bits (FP32)) and are respectively quantized to form the quantized models md1_q(1) to md1_q(N1). The quantized models md1_q(1) to md1_q(N1) may have fixed-point values or integer values (e.g., integer values with 16 bits (INT16)). Although not shown, before step S100, the data processing method of the present invention may further include loading at least one model into a quantization module (such as the quantization unit 100).

[0024] Next, the data processing method proceeds to step S102: perform a compilation process. During the compilation process, the first set of quantized models is compiled to form a first set of executable models, where the first set of executable models includes at least one executable model. For example, in step S102, the quantized models md1_q(1) to md1_q(N1) are compiled and transformed to form a plurality of executable models md1_e(1) to md1_e(N1). The compilation may be performed by a compilation unit 200, which may be a hardware circuit or a software module.

[0025] Next, the data processing method proceeds to step S104: perform an inference process. During the inference process, the first set of executable models is inferred. For example, in step S104, the executable models md1_e(1) to md1_e(N1) are inferred / deployed on an execution module 300 in an application domain (such as an execution module of a smartphone, a tablet computer, a laptop computer, etc.). On the execution module 300, the executable models md1_e(1) to md1_e(N1) may be inferred to process images of interest with various styles and / or roles.

[0026] In the present invention, the models md1(1) to md1(N1) are quantized directly based on a predetermined unified quantization parameter u_qt (i.e., without calibration), and without calibration, so that calibration time can be saved, thereby improving the model efficiency.

[0027] Now, refer to Figure 3 、 4 and 5 to discuss the pre-calibration process of the aforementioned second set of models (i.e., models md2(1) to md2(N2)). More specifically, Figure 3 is a flowchart of a pre-calibration process for obtaining a unified quantization parameter u_qt according to an embodiment of the present invention, Figure 4 is a schematic diagram showing the pre-calibration process for obtaining a unified quantization parameter u_qt. As Figure 3 and Figure 4As shown, first, the pre-calibration process starts from step S300: Execute the model preparation process. In the model preparation process, multiple models are obtained, such as the second set of models mentioned previously. For example, prepare / obtain multiple models md2(0) to md2(N2) (i.e., the aforementioned second set of models). These models md2(0) to md2(N2) have a total of (N2 + 1) models. Each of the models md1(0) to md1(N1) and models md2(0) to md2(N2) is a computational model for AI generation tasks. In one example, each of the models md1(0) to md1(N1) and models md2(0) to md2(N2) is a Stable Diffusion model for text-to-image generation of images of interest. Among these models md2(0) to md2(N2), the model md2(0) is a "base model" (e.g., the SD1.5 model). On the other hand, each of the other models md2(1) to md2(N2) is an "adapted model" adapted from the base model md2(0) for customization and personalization. For example, each of the models md2(1) to md2(N2) may perform model adaptation (MA) based on the model md2(0).

[0028] Model adaptation may be performed by adapters. These adapters may provide customization for specific tasks or specific domains. For example, the adapter may be a small Neural Network (NN) module that can be inserted into the base model md2(0) to enable the base model md2(0) to adapt to a predefined task or a predefined domain. In one example, the adapter may perform model adaptation such as Low-Rank Adaptation (LoRA) or other adaptation methods. LoRA is related to various styles and / or characters of the image of interest. More specifically, taking three models md2(1) to md2(3) as an example: The model md2(1) performs LoRA adaptation based on the model md2(0) to form a "cyberpunk style" of the image of interest. In addition, the model md2(2) performs LoRA adaptation based on the model md2(0) to form a "MoXin style" of the image of interest. Furthermore, the model md2(3) performs LoRA adaptation based on the model md2(0) to form a "crayon style" of the image of interest. When the image of interest is a portrait, the models md2(1), md2(2), and md2(3) may be adapted to fit the role of a specific person.

[0029] In another example, each of the models md1(0) to md1(N1) and models md2(0) to md2(N2) is a Large Language Model (LLM) for performing natural language processing tasks (e.g., language generation). The large language model may learn the statistical relationships of the text during self-supervised and semi-supervised training. Similarly, the large language model can be model-adapted through an adapter to achieve personalization or customization.

[0030] Next, the pre-calibration process proceeds to step S302: perform the calibration process. During the calibration process, a set of calibration data is used to calibrate the second set of models to obtain a plurality of quantization parameters. For example, a set of calibration data ca(0) to ca(N2) is used to calibrate the models md2(0) to md2(N2) to correspondingly obtain the corresponding quantization parameters qt(0) to qt(N2) of the models md2(0) to md2(N2). More specifically, the calibration data ca(0) to ca(N2) is correspondingly input into the models md2(0) to md2(N2), and the quantization parameters qt(0) to qt(N2) of the models md2(0) to md2(N2) are evaluated through calibration. The quantization parameters qt(0) to qt(N2) may reflect the value ranges of the weights of the nodes and layers in the models md2(0) to md2(N2) (i.e., the ranges of floating-point values). Taking three models md2(0), md2(1), and md2(2) as an example and referring to Figure 5 , this figure illustrates the quantization parameters qt(0), qt(1), and qt(2) of these three models. The calibration data ca(0) is input into the model md2(0) to evaluate the quantization parameter qt(0) of the model md2(0). The value range reflected by the quantization parameter qt(0) is (-48, 50), with its minimum value being "-48" and its maximum value being "50". In addition, another calibration data ca(1) is input into the corresponding model md2(1) to evaluate the quantization parameter qt(1) of the model md2(1), which reflects a value range of (-46, 55), with its minimum value being "-46" and its maximum value being "55". Similarly, another calibration data ca(2) is correspondingly input into the model md2(2) to evaluate its quantization parameter qt(2), which reflects a value range of (-44, 54), with its minimum value being "-44" and its maximum value being "54".

[0031] Now, please refer to Figure 3 and Figure 4, Next, the pre-calibration process proceeds to step S304: obtaining a unified quantization parameter based on multiple quantization parameters. For example, a unified quantization parameter u_qt is obtained based on the quantization parameters qt(0) to qt(N2) of models md2(0) to md2(N2). Such a unified quantization parameter u_qt can be referred to as "Post-Training-Quantization (PTQ)". In one example, the unified quantization parameter u_qt is the union of all value ranges reflected by the quantization parameters qt(0) to qt(N2). That is, the global minimum of all value ranges reflected by the quantization parameters qt(0) to qt(N2) is defined as the lower limit of the unified quantization parameter u_qt. On the other hand, the global maximum of all value ranges reflected by the quantization parameters qt(0) to qt(N2) is defined as the upper limit of the unified quantization parameter u_qt. In another example, after obtaining the quantization parameters qt(0) to qt(N2), more value ranges (possibly called "weighted ranges") are obtained by performing a weighting operation on these quantization parameters qt(0) to qt(N2). Therefore, the unified quantization parameter u_qt is obtained not only from the value ranges of the quantization parameters qt(0) to qt(N2), but also from the weighted ranges.

[0032] Please refer again to Figure 5 , which also shows the value range reflected by the unified quantization parameter u_qt. The minimum of the value ranges reflected by the quantization parameters qt(0) to qt(N2) of all models md2(0) to md2(N2) is considered to determine the lower limit of the unified quantization parameter u_qt. For example, the quantization parameter qt(8) of model md2(8) reaches the global minimum of "-50", so the lower limit of the unified quantization parameter u_qt is determined to be "-50". Similarly, the maximum of the value ranges reflected by all quantization parameters qt(0) to qt(N2) is considered to determine the upper limit of the unified quantization parameter u_qt. For example, the quantization parameter qt(12) of model md2(12) reaches the global maximum of "56", so the upper limit of the unified quantization parameter u_qt is determined to be "56".

[0033] From step S300 to step S304, models md2(0) to md2(N2) (i.e., the second group of models) are obtained in step S300, calibrated in step S302, and then the unified quantization parameter u_qt is obtained in step S304. In Figures 1 - 5 the example, models md1(1) to md1(N1) (i.e., the first group of models) are quantized, compiled, and then deployed / inferred. On the other hand, models md2(0) to md2(N2) (i.e., the second group of models) are only used to obtain the unified quantization parameter u_qt, and they are not quantized, compiled, and deployed.

[0034] In other embodiments, in addition to obtaining the unified quantization parameter u_qt, the models md2(0) to md2(N2) may also be quantized, compiled, and deployed / inferred. In another example, only one model may be selected from the models md2(0) to md2(N2), and only this selected model may be quantized, compiled, and deployed.

[0035] Figure 6 is a flowchart of a data processing method according to another embodiment of the present invention. Referring to Figure 6 , the data processing method starts from step S600: loading a network, such as NT1. Referring also to Figure 7A , which shows an example structure of the network NT1. The network NT1 has multiple branches, such as three branches B1, B2, and B3. Branch B1 is associated with operation units op1 and op2, branch B2 is associated with operation units op3 to op6, and branch B3 is associated with operation units op7 to op10. In addition, these three branches B1, B2, and B3 are superimposed by operation unit op11, where operation unit op11 may perform the function of an adder to perform this superimposition.

[0036] Like operation unit op11, each of the operation units op1 to op10 may perform corresponding operations, such as 2D convolution (i.e., "2D conv"), 3D convolution (i.e., "3D conv"), or "reshape", etc. Branch B1 with operation units op1 and op2 forms the base model md0. In addition, branch B2 with operation units op3 and op4 forms the model adaptation (MA) ma11, and branch B2 with operation units op5 and op6 forms the model adaptation ma12. Similarly, branch B3 with operation units op7 and op8 forms the model adaptation ma21, and branch B2 with operation units op9 and op10 forms the model adaptation ma22. As mentioned in the example of Figures 3 - 5 , Figure 7A the model adaptations ma11 to ma22 may be small neural network modules for performing predefined tasks or predefined domains, such as LoRA related to various styles and / or characters of an image of interest. The model adaptations ma11 to ma22 can be inserted into the network NT1 to cooperate with the base model md0 to enable the network NT1 to adapt to perform predefined tasks or predefined domains. It should be noted that Figure 7A the network NT1 in Figure 7AThe number of branches or operation units shown is for illustrative purposes only. In other embodiments, the number of branches or operations may be other values. For example, in addition to the branches of the base model, the network of the present invention may include at least one other branch, and each other branch represents a model adaptation. In some embodiments, the weights of the base model are typically loaded together with the network, but the weights of the model adaptation, such as the weights of the second branch B2 and the third branch B3, are not loaded together with the network and can be input after the network is loaded.

[0037] Next, the data processing method proceeds to step S602: quantize the network according to unified quantization parameters to obtain a quantized network. For example, in step S602, the network NT1 is quantized according to the unified quantization parameter u_qt, thereby obtaining the quantized network NT1_q. Refer to Figure 8 , which shows Figure 6 a schematic diagram of the data processing method. A quantization unit 100 (which may be a hardware circuit or a software module), such as Figure 2 the quantization unit 100 of Figure 3 and Figure 4 can be configured to perform quantization. In one example, the network NT1 originally has 32-bit floating-point values (i.e., "FP32"), and such a network NT1 can be quantized into a quantized network NT1_q with fixed-point values or integer values (e.g., with 16-bit integer values ("INT16")). Similar to Figure 9 the unified quantization parameter u_qt in steps S302 and S304 of

[0038] the unified quantization parameter u_qt for quantizing the network NT1 is predetermined and can be obtained according to a pre-calibration process. In the pre-calibration process, the network NT1 is modified with multiple sets of weights to obtain multiple modified networks NT1(1), NT1(2),..., NT1(N3), and the unified quantization parameter u_qt is obtained using the modified networks NT1(1), NT1(2),..., NT1(N3), which will be discussed later with reference to Figure 9 and FIG. 10.

[0038] Next, the data processing method proceeds to step S604: perform a compilation process. In the compilation process, the quantized network is compiled to form an executable network. For example, in step S604, the quantized network NT1_q is compiled into an executable network NT1_e by a hardware / software-based compilation unit 200 (such as Figure 2 the compilation unit in

[0039] Next, the data processing method proceeds to step S606: input at least one set of weights into an executable network, such as NT1_e. For example, in step S606, four sets of weights x1, x2, y1, and y2 are input into the executable network NT1_e, where the first set of weights x1 is provided to the model adaptation ma11. In addition, the second set of weights x2 is provided to the model adaptation ma12. Similarly, the third set of weights y1 is provided to the model adaptation ma21, and the fourth set of weights y2 is provided to the model adaptation ma22.

[0040] Next, the data processing method proceeds to step S608: quantize at least one set of weights according to the unified quantization parameters of the executable network to obtain at least one set of quantized weights. For example, in step S608, the weight sets x1, x2, y1, and y2 are quantized according to the unified quantization parameter u_qt. In one example, each set of weights x1, x2, y1, and y2 can be quantized using the quantization parameter u_qt respectively, thereby forming the quantized weight sets x1_q, x2_q, y1_q, and y2_q. In another example, the weight sets x1, x2, y1, and y2 are first combined into a total weight set wm, and then the combined weight set wm is quantized using the quantization parameter u_qt to form the quantized combined weight set wm_q.

[0041] Next, the data processing method proceeds to step S610: modify the executable network with at least one set of quantized weights to obtain an executable model. For example, in step S610, the executable network NT1_e is modified with the quantized weight sets x1_q, x2_q, y1_q, and y2_q (or, with the quantized combined weight set wm_q). Specifically, the model adaptations ma11, ma12, ma21, and ma22 are weighted by each set of quantized weights x1_q, x2_q, y1_q, and y2_q respectively, to form the weighted model adaptations ma11_w, ma12_w, ma21_w, and ma22_w, as Figure 7B shown (this is a schematic diagram showing the weighted model adaptations). The weighted model adaptation ma11_w is expressed as (ma11 * x1_q). Similarly, the other weighted model adaptations ma12_w, ma21_w, and ma22_w are expressed as (ma12 * x2_q), (ma21 * y1_q), and (ma22 * y2_q) respectively. Then, the operation unit op11 can perform an operation to superimpose the base model md0 with the weighted model adaptations ma11_w, ma12_w, ma21_w, and ma22_w to form the executable model NT1_w.

[0042] Next, the data processing method proceeds to step S612: Infer the executable model. For example, in step S612, the inference process deploys the executable model NT1_w on the execution module 300. Thus, the executable model NT1_w can achieve multiple model adaptations according to the weighted model adaptations ma11_w, ma12_w, ma21_w, and ma22_w.

[0043] In some alternative embodiments, after step S612, the data processing method can return to step S606 to provide a new set of weights as input. Thus, the data processing method will repeatedly go through steps S606 - S612. In each repetition, in step S610, the executable model NT1_w can achieve a new model adaptation according to the new set of weights, based on the weighted model adaptations ma11_w, ma12_w, ma21_w, and ma22_w. That is to say, in the present invention, the MA of the executable model can be adjusted in real time.

[0044] In some alternative embodiments, after obtaining the executable model in step S610, step S612 can be repeatedly executed according to different application requirements.

[0045] As discussed in the previous paragraph, Figure 6 the unified quantization parameter u_qt of step S602 in the middle embodiment is pre - determined and obtained according to a pre - calibration process. Now, refer to Figure 9 which is the flowchart of the pre - calibration process for obtaining Figure 6 the unified quantization parameter u_qt, and at the same time refer to Figure 10A which is the schematic diagram showing Figure 9 the pre - calibration process. The pre - calibration process starts from step S900: Load the network containing multiple operation units. Step S900 is the same as step S600. Next, the pre - calibration process proceeds to step S902. After loading the network, multiple sets of weights are input into the network to obtain a modified network group. In some embodiments, multiple sets of weights are input into the network separately, and a corresponding modified network can be obtained according to each set of weights in the multiple sets of weights. In other embodiments, some combinations of multiple sets of weights (e.g., the weighted sum of multiple sets of weights) are input into the network separately, and a corresponding modified network can be obtained according to each combination. In Figure 10AIn the example, the first combination of weight groups x1(1), x2(1), y1(1), and y2(1) can be input into network NT1 to obtain the modified network NT1(1), the second combination of weight groups x1(2), x2(2), y1(2), and y2(2) can be input into network NT1 to obtain the modified network NT1(2), and the third combination of weight groups x1(N3), x2(N3), y1(N3), and y2(N3) can be input into network NT1 to obtain the modified network NT1(N3). Additionally, as mentioned before, network NT1 may include a base model and at least one model adaptation (MA). In this case, each of the multiple groups of weights is associated with an MA, or a combination of multiple groups of weights is associated with an MA. For example, in some embodiments, the weight groups x1(1), x2(1), y1(1), and y2(1), x1(2), x2(2), y1(2), and y2(2), etc. may be associated with an MA. In some other embodiments, the combination of weight groups x1(1), x2(1), y1(1), and y2(1) may be associated with MA_A, and the combination of weight groups x1(2), x2(2), y1(2), and y2(2) may be associated with MA_B. In some embodiments, each of MA_A and MA_B is equivalent to multiple MAs.

[0046] Reference Figure 10B , which is a schematic diagram showing the modified networks NT1(1) and NT1(2) formed based on the weight groups. The weight groups x1(1), x2(1), y1(1), and y2(1) are respectively provided to the model adaptations ma11, ma12, ma21, and ma22, thereby forming the weighted model adaptations ma11(1), ma12(1), ma21(1), and ma22(1). Then, these weighted model adaptations ma11(1), ma12(1), ma21(1), and ma22(1) are superimposed with the base model md(0) to form the modified network NT1(1). Such a modified network NT1(1) can be used as a complete model.

[0047] Similarly, the weight groups x1(2), x2(2), y1(2), and y2(2) are also respectively provided to the model adaptations ma11, ma12, ma21, and ma22 to form the weighted model adaptations ma11(2), ma12(2), ma21(2), and ma22(2), which are superimposed with the base model md(0) to form the modified network NT1(2). Such a modified network NT1(2) can be used as a complete model.

[0048] Similarly, the weight groups x1(N3), x2(N3), y1(N3), and y2(N3) are also provided to the model adaptations ma11, ma12, ma21, and ma22 respectively to form the weighted model adaptations ma11(N3), ma12(N3), ma21(N3), and ma22(N3), which are superimposed with the base model md(0) to form the modified network NT1(N3). Such a modified network NT1(N3) can be used as a complete model.

[0049] Next, the pre-calibration process proceeds to step S904, where a set of calibration data is used to calibrate the modified network group to obtain a plurality of quantization parameters. For example, in Figure 10A , a set of calibration data ca(1), ca(2), …, and ca(N3) are used to calibrate the modified networks NT1(1), NT1(2), …, and NT1(N3) respectively to obtain a plurality of quantization parameters qt(1), qt(2), …, and qt(N3). The quantization parameter qt(1) represents the value range (i.e., the range of floating-point values) of the weights of the nodes and layers in the modified network NT1(1). The quantization parameter qt(2) represents the value range (i.e., the range of floating-point values) of the weights of the nodes and layers in the modified network NT1(2). The quantization parameter qt(N3) represents the value range (i.e., the range of floating-point values) of the weights of the nodes and layers in the modified network NT1(N3).

[0050] Then, step S906 is executed: a unified quantization parameter is obtained based on the plurality of quantization parameters of the modified network group. For example, in step S906, a unified quantization parameter u_qt is obtained based on the quantization parameters qt(1), qt(2), …, and qt(N3). For example, the unified quantization parameter u_qt is formed to have a value range that covers all the values of each of the quantization parameters qt(1), qt(2), …, and qt(N3). For example, the global minimum of all the value ranges reflected by the quantization parameters qt(1), qt(2), …, and qt(N3) is defined as the lower limit of the unified quantization parameter, and the global maximum of all the value ranges reflected by the quantization parameters qt(1), qt(2), …, and qt(N3) is defined as the upper limit of the unified quantization parameter. The resulting unified quantization parameter u_qt is used to quantize the network NT1 in Figure 6 step S602. In some embodiments, after obtaining the quantization parameters qt(1), qt(2), …, and qt(N3), more value ranges (possibly called "weighted ranges") are further obtained by performing a weighting operation on these quantization parameters qt(1), qt(2), …, and qt(N3). Therefore, the unified quantization parameter u_qt is obtained not only from the value ranges of the quantization parameters qt(1), qt(2), …, and qt(N3), but also from the weighted ranges.

[0051] Those skilled in the art may make various modifications and variations to the disclosed embodiments. It is intended that the specification and examples be considered as exemplary only, with the true scope of the disclosure being indicated by the following claims and their equivalents.

Claims

1. A data processing method, characterized in that, Including: Loading a first set of models; And Quantizing the first set of models according to unified quantization parameters to obtain a first set of quantized models; Wherein, the first set of models includes at least one model, the first set of quantized models includes at least one quantized model, and the unified quantization parameters are obtained according to multiple quantization parameters of a second set of models.

2. The data processing method according to claim 1, characterized in that Further including: Compiling the first set of quantized models to form a first set of executable models, wherein the first set of executable models includes at least one executable model.

3. The data processing method according to claim 2, wherein Further including: Inferring the first set of executable models.

4. The data processing method according to claim 1, characterized in that The unified quantization parameters are obtained in a pre-calibration process, and the pre-calibration process includes: Obtaining the second set of models, where the second set of models includes multiple models; Calibrating the second set of models using a set of calibration data to obtain the multiple quantization parameters; and Obtaining the unified quantization parameters based on the multiple quantization parameters.

5. The data processing method according to claim 4, wherein The global minimum of the value ranges reflected by all the quantization parameters among the multiple quantization parameters is defined as the lower limit of the unified quantization parameters, and the global maximum of the value ranges reflected by all the quantization parameters among the multiple quantization parameters is defined as the upper limit of the unified quantization parameters.

6. The data processing method according to claim 5, wherein, The value ranges reflected by all the quantization parameters include at least one value range obtained by performing a weighting operation on the multiple quantization parameters.

7. The data processing method according to claim 4, characterized in that The first set of models includes at least one adapted model, and the second set of models includes a base model and multiple adapted models, and each adapted model is obtained by performing model adaptation MA on the base model.

8. The data processing method according to claim 7, characterized in that, Each model in the first set of models and the second set of models is a Stable Diffusion model for text-to-image generation of multiple images; or, Each model in the first set of models and the second set of models is a large language model LLM for performing natural language processing tasks.

9. The data processing method according to claim 7, characterized in that, The model adaptation includes at least one low-rank adaptation LoRA corresponding to various styles or various roles of images.

10. A data processing method, characterized in that, Including: Loading a network including multiple operation units; And Quantizing the network according to unified quantization parameters to obtain a quantized network; Wherein, the unified quantization parameters are obtained according to multiple quantization parameters of a set of modified networks, and the set of modified networks is obtained by inputting multiple sets of weights into the network.

11. The data processing method according to claim 10, characterized in that, The network includes a base model and at least one model adaptation MA, and the base model and each MA include at least one of the multiple operation units.

12. The data processing method according to claim 11, wherein Each set of the multiple sets of weights is associated with an MA, or a combination of the multiple sets of weights is associated with an MA.

13. The data processing method according to claim 10, characterized in that, The unified quantization parameters are obtained in a pre-calibration process, and the pre-calibration process includes: After loading the network, inputting the multiple sets of weights into the network to obtain the set of modified networks; Calibrating the set of modified networks using a set of calibration data to obtain the multiple quantization parameters; and Obtaining the unified quantization parameters based on the multiple quantization parameters.

14. The data processing method according to claim 13, characterized in that, The global minimum value of the value ranges reflected by all the quantization parameters among the multiple quantization parameters is defined as the lower limit of the unified quantization parameter, and the global maximum value of the value ranges reflected by all the quantization parameters among the multiple quantization parameters is defined as the upper limit of the unified quantization parameter.

15. The data processing method according to claim 14, wherein The value ranges reflected by all the quantization parameters include at least one value range obtained by performing a weighting operation on the multiple quantization parameters.

16. The data processing method according to claim 14, wherein Each of the set of modified networks is obtained by inputting a set of weights into the network; or, Each of the set of modified networks is obtained by inputting a combination of multiple sets of weights into the network.

17. The data processing method according to claim 10, wherein Further comprising: Compiling the quantized network to form an executable network.

18. The data processing method according to claim 17, characterized in that, Further comprising: Inputting at least one set of new weights into the executable network; Quantizing the at least one set of new weights based on the unified quantization parameter of the executable network to obtain at least one set of quantized new weights; Modifying the executable network with the at least one set of quantized new weights to obtain an executable model; and Performing inference on the executable model.

19. The data processing method according to claim 18, wherein Further comprising: Returning to the step of inputting at least one set of new weights after the inference step.

20. The data processing method according to claim 18, wherein The executable network includes a base model and at least one model adaptation MA, and the base model and each MA include at least one operation unit among the multiple operation units; Wherein each set of new weights in the at least one set of new weights is associated with one MA among the at least one MA.

Images