Data processing method and electronic device

Abstract

The present disclosure provides a data processing method and an electronic device, which are applied to the field of artificial intelligence technology. The data processing method comprises: obtaining a corpus with domain characteristics; inputting the corpus into a to-be-trained large model to determine the sensitivity score and the correlation score of each layer in the to-be-trained large model, wherein the sensitivity score represents the degree of sensitivity of each layer to weight quantization, the correlation score represents the degree of correlation between the activation value in each layer and the domain characteristics, and the sensitivity scores and the correlation scores of different layers in the model are the same or different; determining the quantization strategy of each layer according to the sensitivity score and the correlation score; and adjusting the gradient of the back propagation of each layer of the to-be-trained large model according to the quantization strategy of each layer and the correlation score of each layer to obtain a trained target large model, and deploying the target large model on a computing device to perform inference on input data with domain characteristics.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a data processing method and an electronic device. Background Technology

[0002] Large language models face significant resource bottlenecks and accuracy challenges when deployed in vertical domains. Existing model quantization solutions mainly employ compression methods with uniform granularity or uniform precision, which cannot adapt to the differences in structure and numerical distribution among different layers of the model.

[0003] In professional scenarios such as healthcare, law, and finance, the accuracy of key information is just as important as the model's response efficiency. Existing solutions struggle to maintain processing accuracy for vertical tasks while ensuring inference speed, which hinders the effective deployment of large language models on resource-constrained devices. Summary of the Invention

[0004] In view of this, the present disclosure provides a data processing method and an electronic device.

[0005] According to a first aspect of this disclosure, a data processing method is provided, comprising: acquiring corpus with domain features; inputting the corpus into a large model to be trained, determining the sensitivity score and relevance score of each layer in the large model to be trained, wherein the sensitivity score characterizes the degree of sensitivity of each layer to weight quantization, and the relevance score characterizes the degree of correlation between the activation value in each layer and the domain features, wherein the sensitivity score and relevance score of different layers in the model are the same or different; determining the quantization strategy of each layer based on the sensitivity score and the relevance score; adjusting the gradient of the backpropagation of each layer of the large model to be trained based on the quantization strategy and the relevance score of each layer, thereby obtaining a trained target large model, wherein the target large model is deployed on a computing device to perform inference on input data with domain features.

[0006] According to embodiments of this disclosure, the corpus is input into a large model to be trained, and multiple first activation value vectors and multiple second activation value vectors under pseudo-quantization are obtained for each layer. A sensitivity score is determined based on the difference between the multiple first activation value vectors and the multiple second activation value vectors. The first activation value vectors are the original activation values ​​generated by the model under unquantized conditions, and the second activation value vectors are the noisy activation values ​​generated by the model under quantized and dequantized conditions. A correlation score is determined based on the multiple first activation value vectors and a preset activation value mapping library.

[0007] According to embodiments of this disclosure, a sum of squares of the differences between each first activation value vector and the corresponding second activation value vector is calculated; the average of the sums of squares is normalized to obtain a sensitivity score.

[0008] According to embodiments of this disclosure, multiple similarities between multiple first activation value vectors and multiple feature vectors in a preset activation value mapping library are calculated. The preset activation value mapping library consists of activation value vectors output by each layer after the domain information labeled in the corpus is input into the large model to be trained. The maximum value of the multiple similarities is used as the relevance score.

[0009] According to embodiments of this disclosure, layers that satisfy a preset first classification condition are determined as first-class layers. The preset first classification condition includes a sensitivity score greater than or equal to a first threshold and a correlation score greater than or equal to a second threshold. The activation value of the first-class layer is quantized using a first precision method, and the weight quantization precision is lower than the activation value quantization precision. Layers that satisfy a preset second classification condition are determined as second-class layers. The preset second classification condition includes a sensitivity score greater than or equal to a third threshold and less than the first threshold, or / and a correlation score greater than or equal to a fourth threshold and less than the second threshold. The activation value of the second-class layer is quantized using a second precision method, and the weight quantization precision is lower than the activation value quantization precision. Layers that satisfy a preset third classification condition are determined as third-class layers. The preset third classification condition includes a sensitivity score less than the third threshold and a correlation score less than the fourth threshold. The activation value of the third-class layer is quantized using a third precision method, and the weight quantization precision is lower than or equal to the activation value quantization precision. The preset first, second, and third classification conditions are used to filter layers of different importance levels. The precision of the first precision quantization method is higher than that of the second precision quantization method, and the precision of the second precision quantization method is higher than that of the third precision quantization method.

[0010] According to embodiments of this disclosure, the weight quantization parameters and activation value quantization parameters of each layer are determined according to the quantization strategy of each layer; the gradients of backpropagation of each layer of the large model to be trained are adjusted according to the activation value quantization parameters, activation value quantization parameters and correlation scores of each layer, so as to obtain the trained target large model.

[0011] According to embodiments of this disclosure, based on the weight quantization parameters and activation value quantization parameters of each layer, pseudo-quantization is performed on the weights and activation values ​​of each layer of the large model to be trained, resulting in pseudo-quantized weights and activation values. The corpus is input into the large model to be trained, and forward propagation is performed using the pseudo-quantized weights and activation values ​​to obtain prediction results. A loss function is calculated based on the prediction results. Backpropagation is performed based on the loss function to determine the gradient of backpropagation, and the gradient of backpropagation is weighted and adjusted according to the relevance scores of each layer to obtain the adjusted gradient. The original weights of the large model to be trained are updated using the adjusted gradient to obtain the target large model.

[0012] According to embodiments of this disclosure, a weight truncation threshold for each layer's weight is determined based on the numerical distribution of weights in each layer of the corpus. The weights of each layer are then restricted within the weight truncation threshold range. The restricted weights are then quantized and dequantized according to the weight quantization parameters of each layer to obtain pseudo-quantized weights. Similarly, an activation value truncation threshold for each layer's activation value is determined based on the numerical distribution of activation values ​​in each layer of the corpus. The activation values ​​of each layer are then restricted within the activation value truncation threshold range. The restricted activation values ​​are then quantized and dequantized according to the activation value quantization parameters of each layer to obtain pseudo-quantized activation values.

[0013] According to embodiments of this disclosure, dividing the restricted weight by the weight quantization parameter and rounding it down yields the quantized integer weight; multiplying the quantized integer weight by the weight quantization parameter yields the pseudo-quantized weight; quantizing and then dequantizing the restricted activation value according to the activation value quantization parameter of each layer to obtain the pseudo-quantized activation value includes: dividing the restricted activation value by the activation value quantization parameter and rounding it down to obtain the quantized integer activation value; multiplying the quantized integer activation value by the activation value quantization parameter to obtain the pseudo-quantized activation value.

[0014] A second aspect of this disclosure provides an electronic device, including a memory and a processor; wherein the processor is configured to: acquire corpus with domain features; input the corpus into a large model to be trained; determine a sensitivity score representing the sensitivity to weight quantization and a correlation score representing the correlation between activation values ​​and domain features in each layer of the large model to be trained; determine a quantization strategy for each layer based on the sensitivity score and the correlation score; adjust the gradients of backpropagation of each layer of the large model to be trained based on the quantization strategy and the correlation score of each layer to obtain a trained target large model; and deploy the target large model on the electronic device to perform inference on input data with domain features.

[0015] A third aspect of this disclosure provides a data processing apparatus, comprising: a corpus acquisition module for acquiring corpus with domain features; a model analysis module for inputting the corpus into a large model to be trained, determining the sensitivity score and relevance score of each layer in the large model to be trained, wherein the sensitivity score characterizes the degree of sensitivity of each layer to weight quantization, and the relevance score characterizes the degree of correlation between the activation values ​​in each layer and the domain features, and the sensitivity score and relevance score of different layers in the model are the same or different; a policy determination module for determining the quantization policy of each layer based on the sensitivity score and relevance score; and a gradient adjustment module for adjusting the gradient of the backpropagation of each layer of the large model to be trained based on the quantization policy and the relevance score of each layer, to obtain a trained target large model, wherein the target large model is deployed on a computing device to perform inference on input data with domain features.

[0016] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0017] The above and other objects, features, and advantages of this disclosure will become clearer from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0018] Figure 1 The diagram illustrates an application scenario of the data processing method and electronic device according to embodiments of the present disclosure.

[0019] Figure 2 A flowchart illustrating a data processing method according to an embodiment of the present disclosure is shown schematically.

[0020] Figure 3 A flowchart illustrating the determination of a relevance score according to an embodiment of this disclosure is shown schematically.

[0021] Figure 4 A schematic diagram illustrating the system architecture of the data processing method according to an embodiment of the present disclosure is shown.

[0022] Figure 5 A schematic block diagram of a data processing apparatus according to an embodiment of the present disclosure is shown.

[0023] Figure 6 A block diagram of an electronic device for data processing according to embodiments of the present disclosure is shown schematically. Detailed Implementation

[0024] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely examples and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0027] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "having at least one of A, B and C" should include, but is not limited to, having only A, having only B, having only C, having A and B, having A and C, having B and C, and / or having A, B and C, etc.).

[0028] Before introducing the technical solutions provided in the embodiments of this disclosure, the relevant technologies involved in this disclosure will be explained first.

[0029] In related technologies, large language models face two main challenges when deployed in vertical domains: First, traditional quantization schemes typically use compression methods with uniform granularity or uniform precision, applying the same quantization strategy to different layers, which leads to problems such as loss of domain information, precision degradation, and waste of hardware resources. Second, existing quantization schemes based on general corpus calibration perform poorly in professional scenarios such as medicine, law, and finance, lacking the ability to perceive and retain key information in vertical domains, and failing to effectively distinguish the different levels of demand for quantization and domain features from different layers.

[0030] Specifically, in real-world deployment environments, existing quantization schemes often compress model volume using only single-granularity quantization (such as full-model tensor quantization or full-model channel quantization). This method is insensitive to differences in numerical distribution across different layers and struggles to capture structural differences between highly sensitive and low-sensitive layers, as well as between domain-core and general layers. Furthermore, existing schemes lack fine-grained modeling of inter-layer differences. When there is a significant shift between the vertical domain corpus distribution and the general corpus distribution, key domain information can easily be distorted, resulting in the quantized model's domain task accuracy failing to meet practical application requirements. It is particularly noteworthy that existing schemes typically execute domain fine-tuning and quantization compression as two independent stages, which inherently introduces error accumulation and hinders the deployment of high-precision, high-efficiency vertical domain models.

[0031] In view of this, embodiments of this disclosure provide a data processing method applied to the vertical domain optimization deployment of a large language model. The method includes: acquiring corpus with domain features; inputting the corpus into a large model to be trained; determining the sensitivity score and relevance score of each layer in the large model to be trained, whereby the sensitivity score characterizes the degree of sensitivity of each layer to weight quantization, and the relevance score characterizes the degree of correlation between the activation values ​​in each layer and the domain features; the sensitivity score and relevance score of different layers in the model may be the same or different; determining the quantization strategy of each layer based on the sensitivity score and relevance score; adjusting the gradient of the backpropagation of each layer of the large model to be trained based on the quantization strategy and the relevance score of each layer, to obtain the trained target large model; and deploying the target large model on a computing device to perform inference on input data with domain features.

[0032] This embodiment of the disclosure effectively overcomes the limitations of traditional unified quantization schemes by employing a joint optimization mechanism of layer adaptive quantization and gradient differential adjustment, combined with a domain corpus-driven calibration strategy. The layer adaptive framework used in this method significantly reduces domain information loss, enabling the model to achieve differentiated configurations of quantization strategies between layers. This not only effectively protects the accuracy of highly sensitive layers and core domain layers but also enhances the domain feature preservation capability through gradient adjustment guided by relevance scoring. It achieves an efficient balance between model compression and accuracy preservation in vertical domain scenarios, providing users with high-precision, low-latency professional domain intelligent services.

[0033] The embodiments disclosed herein can be widely applied to the deployment of large language models in vertical domains that require high-precision domain knowledge, such as medical auxiliary diagnosis, legal document processing, and financial risk analysis. They are particularly suitable for model inference scenarios on resource-constrained edge computing devices. Furthermore, this method can also be extended to scenarios such as intelligent customer service, professional question-answering systems, and knowledge graph construction, providing a unified and reliable technical solution for the efficient deployment and accurate inference of large language models in vertical domains.

[0034] Figure 1 The diagram illustrates an application scenario of the data processing method and electronic device according to embodiments of the present disclosure.

[0035] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a computing device 101, a domain data server 102, and a network 103 connecting the various entities.

[0036] Computing device 101 is an intelligent device deployed in vertical application environments for model training and inference on domain data in specialized scenarios. Equipped with high-performance computing and storage units, computing device 101 possesses independent model training and neural network computation capabilities, enabling it to complete the entire process of corpus acquisition, model training, and inference locally.

[0037] Domain data server 102 serves as the corpus management and model service center, establishing a communication connection with computing device 101 via network 103. Domain data server 102 is responsible for the collection and organization of vertical domain corpora, the management and centralized distribution of preset models.

[0038] Network 103 includes, but is not limited to, wireless local area network, mobile communication network or wired network, and is used to establish a data channel between computing device 101 and domain data server 102. This network connection is mainly used for the following scenarios: 1) periodically updating the parameters of the large model to be trained in computing device 101; 2) synchronously updating the domain corpus and quantization strategy configuration information; 3) collecting model training status and inference results.

[0039] It should be noted that although the computing device 101 and the domain data server 102 are connected through the network 103, the computing device 101 can independently complete all processing steps, including corpus acquisition, model analysis and policy execution, when performing a single data processing task, without relying on the real-time performance of the network connection.

[0040] Accordingly, the data processing method provided in this embodiment is mainly implemented by computing device 101, including: acquiring corpus with domain characteristics; inputting the corpus into a large model to be trained, and determining the sensitivity score and relevance score of each layer in the large model to be trained; determining the quantization strategy of each layer based on the sensitivity score and relevance score; and adjusting the gradient of backpropagation of each layer of the large model to be trained based on the quantization strategy and the relevance score of each layer, to obtain the trained target large model.

[0041] It should be understood that Figure 1 The system architecture shown can be adjusted according to actual needs. The computing device 101 can be configured to complete all processing tasks independently, or it can be configured as a terminal device that relies on a server for real-time computing.

[0042] The data processing method of the present disclosure will be described in detail below.

[0043] Figure 2 A flowchart illustrating a data processing method according to an embodiment of the present disclosure is shown schematically.

[0044] like Figure 2 As shown, the data processing method of this embodiment can be applied to the vertical domain deployment scenario of large language models. The data processing method includes operations S210 to S240.

[0045] In operation S210, corpus with domain characteristics is acquired.

[0046] In this embodiment, the corpus refers to text data specific to a vertical domain. Domain features refer to the knowledge expression methods and terminology system unique to the professional field to which the corpus belongs. Corpora from different domains exhibit significant differences in vocabulary distribution, sentence structure, and semantic focus. The more distinct the domain features, the higher the density of professional knowledge carried by the corpus, and the more the performance of large models on tasks within that domain depends on the accurate capture of these features. The strength of domain features directly affects the formulation of subsequent quantization strategies; layers with distinct features require more refined quantization protection.

[0047] In operation S220, the corpus is input into the large model to be trained, and the sensitivity score and relevance score of each layer in the large model to be trained are determined.

[0048] In this embodiment of the disclosure, the sensitivity score characterizes the degree to which each layer is sensitive to weight quantization, and the correlation score characterizes the degree of correlation between the activation value and the domain features in each layer. The sensitivity scores and correlation scores of different layers in the model may be the same or different. The sensitivity score reflects the degree to which the output of a certain layer changes after being quantized. The higher the score, the more sensitive the layer is to the quantization operation, and the greater the accuracy loss caused by quantization.

[0049] The relevance score reflects the degree of matching between the activation values ​​of a certain layer's output and the domain features. A higher score indicates that the layer carries more key information in the domain task. Both scores characterize the importance of each layer from different dimensions, providing a decision-making basis for subsequent layer quantification.

[0050] In operating S230, the quantification strategy for each layer is determined based on the sensitivity score and the correlation score.

[0051] In this embodiment, the quantization strategy is a compression scheme independently formulated for each layer, including the quantization granularity and precision used for that layer. A sensitivity score is used to determine the layer's tolerance for loss of numerical precision, and a relevance score is used to determine the layer's importance in preserving domain knowledge. Combining these two scores, each layer is divided into different importance levels, and layers of different levels receive differentiated quantization configurations, ensuring overall compression efficiency while maximizing the protection of the precision of critical layers.

[0052] In the S240 operation, the gradients of the backpropagation of each layer of the large model to be trained are adjusted according to the quantization strategy and the correlation score of each layer, so as to obtain the target large model after training.

[0053] In this embodiment of the disclosure, the target large model is deployed on a computing device to perform inference on input data with domain characteristics. The target large model is a lightweight model that can be directly deployed, obtained by quantization training based on the original model, in order to maintain domain task accuracy while reducing storage overhead and improving inference speed.

[0054] While maintaining improved inference efficiency, this model preserves its accuracy in handling vertical domain tasks through integrated quantization and fine-tuning training. During training, relevance scores are used to dynamically adjust the gradient update strength during backpropagation, ensuring that layers highly correlated with domain features receive more adequate error compensation during training and preventing domain knowledge degradation caused by quantization operations.

[0055] In this embodiment, domain corpus is acquired and input into the large model to be trained. Sensitivity scores for weight quantization and correlation scores between activation values ​​and domain features are determined for each layer. Based on these, differentiated quantization strategies are formulated for each layer, and the gradients of backpropagation for each layer are adjusted according to the correlation scores during training. Through joint optimization of layered perception and gradient control, the quantized target large model, when deployed to a computing device, can maintain vertical task accuracy while reducing storage overhead and improving inference speed.

[0056] The data processing methods, including operations S210 to S240, are described in detail below.

[0057] In this embodiment, after acquiring domain-specific corpora, the corpora are preprocessed, such as electronic medical records and disease diagnosis guidelines in the medical field, legal provisions and judgments in the legal field, and financial reports and risk assessment reports in the financial field. The corpus size ensures coverage of core domain scenarios. Specifically, the collected corpora are preprocessed to remove invalid medical records, duplicate legal provisions, and other noisy data, and core domain information is labeled, such as disease names and diagnostic conclusions in the medical field, and clause numbers and descriptions of rights and obligations in the legal field, so as to subsequently construct an activation value mapping library based on the core domain information.

[0058] Figure 3 A flowchart illustrating the determination of a relevance score according to an embodiment of this disclosure is shown.

[0059] like Figure 3 As shown, the above operation S220 may further include operations S310 to S320.

[0060] When operating S310, the corpus is input into the large model to be trained, and multiple first activation value vectors and multiple second activation value vectors under full precision and pseudo-quantization are obtained for each layer. The sensitivity score is determined based on the difference between the multiple first activation value vectors and the multiple second activation value vectors.

[0061] In this embodiment, the first activation value vector is the original activation value generated by the model under unquantized conditions, and the second activation value vector is the noisy activation value generated by the model under quantized and dequantized conditions. Full precision refers to the unquantized state, and pseudo-quantization refers to simulating quantization and then restoring the values ​​on the weights and activation values. After the corpus is input into the model, forward propagation is performed under both conditions to collect the corresponding activation value vectors layer by layer.

[0062] For each layer, the difference between the first activation vector and the second activation vector corresponding to the same input is calculated. The greater the difference, the more sensitive the layer is to quantization. The sensitivity score measures the vulnerability of each layer in the quantization process by quantizing the degree of this difference, serving as the basis for subsequent differentiated configuration of quantization accuracy.

[0063] In operation S320, a correlation score is determined based on multiple first activation value vectors and a preset activation value mapping library.

[0064] In this embodiment, the preset activation value mapping library is a reference database pre-built based on domain core information, which stores the activation value response patterns generated by each layer when the domain core information is input into the model. For each layer's first activation value vector generated by the corpus input, the degree of matching between it and the corresponding feature vectors of each layer in the mapping library is calculated.

[0065] If the first activation vector of a certain layer highly matches the core features of the domain in the mapping library, it indicates that this layer plays a key role in domain data processing, and its output carries a significant amount of domain knowledge. The relevance score quantifies the strength of the association between each layer and the domain features based on the degree of matching; a higher score indicates that the layer is more important to the domain task.

[0066] In this embodiment, the sensitivity score is determined by inputting the corpus into the model and collecting the activation value vectors of each layer under both full-precision and pseudo-quantization conditions, comparing the differences in activation values ​​under the two conditions; simultaneously, the relevance score is determined based on the matching of activation values ​​with a preset activation value mapping library. This approach provides two independent evaluation dimensions for subsequent hierarchical quantization strategies, making the formulation of quantization strategies more accurate.

[0067] In this embodiment of the disclosure, the determination of the sensitivity score based on the difference between the plurality of first activation value vectors and the plurality of second activation value vectors in the above-described operation S310 may further include: calculating the sum of squares of the differences between each first activation value vector and the second activation value vector corresponding to each first activation value vector; and normalizing the average of the plurality of sums of squares to obtain the sensitivity score.

[0068] In this embodiment, for each layer of the large model to be trained, the vertical domain corpus is input into the model in batches, and forward propagation is performed under two conditions: unquantized and pseudo-quantized. The unquantized condition means that no compression is performed on the weights and activation values, and the original floating-point precision is retained for calculation. The output of each layer under this condition is the first activation value vector. The pseudo-quantized condition simulates quantization operations on the weights and activation values, compressing the floating-point values ​​to a lower ratio of a specific number of points before restoring them to floating-point format. The output of each layer under this condition is the second activation value vector.

[0069] The only difference between the two conditions is whether quantization noise is introduced; all other conditions are exactly the same. Therefore, the output difference of the same layer under the two conditions is entirely caused by the quantization operation. For each input sample in each batch, the sum of the squares of the differences between the first activation vector and the corresponding second activation vector of the layer's output is calculated.

[0070] The average quantization error of this layer on the vertical domain corpus is obtained by summing the squares of all samples and dividing by the total number of samples. This average error is then normalized to a value between zero and one, yielding a comparable sensitivity score for this layer. The closer the normalized score is to one, the greater the impact of quantization on the layer's output, and the more sensitive the layer is to quantization.

[0071] For example, sensitivity score The calculation formula is as follows:

[0072]

[0073] in, This is the i-th output vector (i.e., the activation value vector) in full precision (FP16 / FP32); This is the i-th output vector under pseudo-quantization (INT8 / INT4); The total number of samples collected; The normalization function maps the error to the interval [0,1].

[0074] Full precision refers to the state where model weights and activation values ​​are stored and calculated in a high-bit-width floating-point format, such as 16-bit or 32-bit floating-point. This can accurately represent subtle changes in values, but it requires a large amount of storage space and is slow to compute. Pseudo-quantization, on the other hand, simulates the quantization effect of low-bit-width integers (such as 8-bit or 4-bit integers) while preserving the floating-point data format. Specifically, it compresses floating-point values ​​to a lower bit-width number and then restores them to floating-point values. This allows the data to carry quantization noise while maintaining the floating-point format, enabling the model to adapt to the precision loss caused by quantization during training and improving the performance of the quantized model.

[0075] It should be noted that i represents the i-th input sample (the i-th data batch), and l refers to the l-th layer of the model. and No additional training or computation is required; simply perform a forward propagation (inference) once, and each layer of the model will naturally output activation values. and These outputs are from the model's forward propagation.

[0076] In this embodiment, the sum of squares of the differences between each first activation value vector and its corresponding second activation value vector is calculated. The sum of squares for all samples is then averaged and normalized to obtain the sensitivity score for each layer. This calculation method unifies the measurement standard for quantification sensitivity across layers, providing a comparable quantitative basis for formulating hierarchical quantification strategies and ensuring that highly sensitive layers receive more adequate accuracy protection.

[0077] In this embodiment of the disclosure, the above operation S320 may further include: calculating multiple similarities between multiple first activation value vectors and each feature vector in a preset activation value mapping library, wherein the preset activation value mapping library is composed of activation value vectors output by each layer after the domain information labeled in the corpus is input into the large model to be trained; and using the maximum value of multiple similarities as a relevance score.

[0078] In this embodiment of the disclosure, for each batch of input corpus, the first activation value vector corresponding to each layer is obtained, and the cosine similarity between the first activation value vector and the feature vector of the corresponding layer in the mapping library is calculated. The maximum value is taken as the matching score of that layer in that batch, which is the relevance score. The higher the relevance score, the more key information that layer carries in the domain task, and it should be given priority protection during the quantization compression process.

[0079] For example, relevance score The formula for calculating is as follows:

[0080]

[0081] in, For the current input in the th The activation vector generated by the layer; Domain activation value mapping library The feature vectors in; Given a pre-built set of features for the vertical domain, different layers in the model are divided into different importance levels based on sensitivity scores and relevance scores.

[0082] It should be noted that when both the sensitivity score and the relevance score are greater than or equal to their respective high thresholds, the layer is identified as a key layer, such as the output layer and the domain-specific layer; when the sensitivity score or the relevance score is between its respective medium and high thresholds, the layer is identified as an important layer, such as the projection layer and the attention computation layer; when both the sensitivity score and the relevance score are less than their respective medium thresholds, the layer is identified as a normal layer, such as the embedding layer and the feedforward network layer; the normalization layer is identified as a quantization-free layer due to its high sensitivity to numerical distribution, and no quantization operation is performed, where the high threshold can be 0.8 and the medium threshold can be 0.4.

[0083] In this embodiment, the similarity between each first activation value vector and each feature vector in a preset activation value mapping library is calculated, and the maximum value is taken as the relevance score. This mapping library consists of activation values ​​after domain information is input into the model. The relevance score quantifies the association strength between each layer and domain features, enabling the core domain layer to be preferentially identified and protected during the quantization and compression process.

[0084] In this embodiment of the disclosure, the above-mentioned operation S230 may further include: determining layers that meet preset first classification conditions as first-class layers, wherein the preset first classification conditions include a sensitivity score greater than or equal to a first threshold and a correlation score greater than or equal to a second threshold, the activation value of the first-class layer adopts a first precision quantization method, and the weight quantization precision is lower than the activation value quantization precision; determining layers that meet preset second classification conditions as second-class layers, wherein the preset second classification conditions include a sensitivity score greater than or equal to a third threshold and less than the first threshold, or / and a correlation score greater than or equal to a fourth threshold and less than the second threshold, the activation value of the second-class layer adopts a second precision quantization method, and the weight quantization precision is lower than the activation value quantization precision; determining layers that meet preset third classification conditions as third-class layers, wherein the preset third classification conditions include a sensitivity score less than a third threshold and a correlation score less than a fourth threshold, the activation value of the third-class layer adopts a third precision quantization method, and the weight quantization precision is lower than or equal to the activation value quantization precision, wherein the preset first classification conditions, preset second classification conditions, and preset third classification conditions are respectively used to filter layers of different importance levels, the precision of the first precision quantization method is higher than that of the second precision quantization method, and the precision of the second precision quantization method is higher than that of the third precision quantization method.

[0085] In this embodiment, the importance level of the layers selected by the preset first classification condition is higher than that of the layers selected by the preset second classification condition, and the importance level of the layers selected by the preset second classification condition is higher than that of the layers selected by the preset third classification condition. The first type of layer is the layer whose sensitivity score and relevance score are both in the high score range. It belongs to the core part of the model that is most sensitive to quantization and most closely related to domain features, and requires the highest level of accuracy protection during the quantization process.

[0086] The second type of layer consists of layers with sensitivity or relevance scores in the medium range. These layers are of some importance in the model but less so than the first type, and require moderate precision protection during quantization. The third type of layer consists of layers with both sensitivity and relevance scores in the low range. These layers are general parts of the model that are not sensitive to quantization and have a weak correlation with domain features. In this case, lower precision can be used to achieve higher compression efficiency.

[0087] For example, the first type of layer includes the output layer and the domain-specific layer, with quantization precision of 8-bit integers for weights, no quantization for core activation values, and 8-bit integers for non-core activation values. The second type of layer includes the attention calculation layer and the projection layer, with quantization precision of 8-bit integers for weights, 8-bit integers for core activation values, and 8-bit integers for non-core activation values. The third type of layer includes the feedforward network layer and the embedding layer, with quantization precision of 4-bit integers for weights, 8-bit integers for core activation values, and 4-bit integers for non-core activation values. Normalization layers, due to their high sensitivity to numerical distribution, retain 16-bit floating-point precision for both weights and activation values, and do not undergo quantization. Within the same type of layer, the distinction between core and non-core activation values ​​is based on the degree of matching between each dimension of the layer's activation value vector and the domain's core activation value mapping library. Dimensions with a matching degree higher than a preset threshold correspond to core activation values, while the rest are non-core activation values. In this way, each layer can obtain differentiated quantization configurations according to its importance in the model structure and domain task, ensuring overall compression efficiency while maximizing the protection of the accuracy of key layers.

[0088] In this embodiment, the layers are divided into three categories based on sensitivity and relevance scores: high-score layers use high-precision activation value quantization, medium-score layers use medium precision, and low-score layers use low precision, with the precision decreasing progressively among the three categories. This hierarchical quantization strategy allows high-importance layers to receive higher precision quantization protection, while low-importance layers use lower precision in exchange for compression efficiency, achieving a differentiated balance between precision and efficiency.

[0089] In this embodiment of the disclosure, the above operation S240 may further include: determining the weight quantization parameters and activation value quantization parameters of each layer according to the quantization strategy of each layer; adjusting the gradient of the backpropagation of each layer of the large model to be trained according to the activation value quantization parameters, activation value quantization parameters and correlation scores of each layer, to obtain the trained target large model.

[0090] In this embodiment, the weight quantization parameter can be a weight scaling factor, and the activation value quantization parameter can be an activation value scaling factor. Both are parameters used to scale the weights and activation values ​​during the quantization operation. Specifically, for the first and second types of layers, i.e., the core layer and the important layer, a 99.9 percentile strategy is used to calculate the weight scaling factor. The formula is as follows:

[0091]

[0092] in, These are the weight values ​​generated from input corpora in the vertical domain. The 99.9th percentile of the absolute value is used as the cutoff threshold; Quantize the target bit width (e.g., 8 for INT8, 4 for INT4).

[0093] In this embodiment, the activation value quantization parameter is processed differently according to the activation value type. The primary activation value retains its original precision and is not quantized; the secondary activation value uses a sliding window calibration method to dynamically calculate the activation value scaling factor.

[0094] Specifically, a fixed-size window is maintained. Each time a new batch of data is input, the minimum and maximum values ​​of the activation values ​​for that batch are calculated and stored in the window. When the window is full, the minimum of all minimum values ​​and the maximum of all maximum values ​​within the window are taken, and the activation value scaling factor is calculated based on this range. The formula is as follows:

[0095]

[0096] in, It is the minimum value among all minimum values ​​within the sliding window. The maximum value among all the maximum values ​​within the sliding window, where b is the target quantization bit width.

[0097] In this embodiment, the weight quantization parameters and activation value quantization parameters of each layer are determined according to the quantization strategy, and then the gradient of backpropagation is adjusted based on the correlation score of each layer. This process connects the quantization strategy with gradient control, ensuring that the calculation of quantization parameters and the adjustment of gradients are based on layer importance, thus ensuring that key domain layers receive more adequate error compensation during training.

[0098] In this embodiment, based on the weight quantization parameters and activation value quantization parameters of each layer, pseudo-quantization is performed on the weights and activation values ​​of each layer of the large model to be trained, resulting in pseudo-quantized weights and activation values. The corpus is input into the large model to be trained, and forward propagation is performed using the pseudo-quantized weights and activation values ​​to obtain prediction results. A loss function is calculated based on the prediction results. Backpropagation is performed based on the loss function to determine the gradient of backpropagation, and the gradient of backpropagation is weighted and adjusted according to the relevance scores of each layer to obtain the adjusted gradient. The original weights of the large model to be trained are updated using the adjusted gradient to obtain the target large model.

[0099] In this embodiment, pseudo-quantization is a process that simulates the effect of low-bit-width integer quantization while preserving the floating-point data format. Specifically, it involves compressing the floating-point value to a low-bit-width integer and then restoring it to a floating-point number. This allows the data to carry quantization noise while maintaining the floating-point format, ensuring that the gradient can be propagated normally during backpropagation. During forward propagation, pseudo-quantization is performed on the weights and activation values ​​of each layer to obtain pseudo-quantized weights and activation values ​​carrying quantization noise. The model performs calculations based on these pseudo-quantized values ​​and propagates them layer by layer, allowing each layer to experience the precision loss caused by quantization in advance during the forward propagation process.

[0100] During backpropagation, a pass-through estimator is used to bypass the rounding operation in pseudo-quantization, allowing gradients to be properly backpropagated to the original weights. Simultaneously, the backpropagated gradients are differentially weighted based on the relevance scores of each layer, with layers with high relevance scores receiving larger gradient updates and layers with low relevance scores receiving correspondingly smaller gradient updates. The adjusted gradients are then used to update the original full-precision weights, ensuring that layers highly correlated with domain features receive more adequate error compensation during training, preventing domain knowledge degradation caused by quantization operations. Layers with lower relevance are updated only moderately, maintaining the overall model's convergence efficiency.

[0101] For example, the adjusted gradient is weighted and adjusted based on the correlation scores of each layer, as shown in the following formula:

[0102]

[0103] in, The adjusted gradient value. The original gradient calculated by backpropagation. For the first Layer relevance score. Update the original weights using the adjusted gradient, as shown in the following formula:

[0104] W new = W old - η × G adj

[0105] Among them, W new For the updated weights, W old The weights before the update are η, the learning rate is η, and G is G. adj This is the adjusted gradient.

[0106] In this embodiment, pseudo-quantization is performed on the weights and activation values ​​of each layer based on the weight quantization parameters and activation value quantization parameters. The pseudo-quantized values ​​are used for forward propagation to calculate the loss. During backpropagation, the gradient weights are adjusted based on the relevance score and then updated with the original weights. Pseudo-quantization allows the model to adapt to quantization noise in advance, while gradient weighting enables more optimization of the domain-critical layers. Together, they improve the domain accuracy of the quantized model.

[0107] In this embodiment, based on the numerical distribution of weights at each layer in the corpus, a weight truncation threshold is determined for each layer's weights. The weights at each layer are then restricted within the weight truncation threshold range. The restricted weights are then quantized and dequantized according to the weight quantization parameters of each layer to obtain pseudo-quantized weights. Similarly, based on the numerical distribution of activation values ​​at each layer in the corpus, an activation value truncation threshold is determined for each layer's activation values. The activation values ​​at each layer are then restricted within the activation value truncation threshold range. The restricted activation values ​​are then quantized and dequantized according to the activation value quantization parameters of each layer to obtain pseudo-quantized activation values.

[0108] In this embodiment, the determination of the truncation threshold depends on the actual numerical distribution of weights and activation values ​​at each layer within the vertical domain corpus. For each layer, the 99.9th percentile of the absolute value of the weights or activation values ​​in the vertical domain corpus input is calculated, and this percentile is used as the truncation threshold for that layer. Weights or activation values ​​exceeding the positive or negative truncation threshold range are directly truncated, limiting the values ​​to a reasonable range and preventing extreme outliers from dominating the entire numerical range during quantization, thus avoiding a loss of quantization accuracy for most values.

[0109] After truncation, quantization and dequantization operations are performed on the restricted weights and activation values ​​according to the quantization parameters of each layer. That is, the values ​​are first compressed to a low ratio of a specific number of points and then restored to floating-point numbers, so that the values ​​carry quantization noise but maintain floating-point format for subsequent gradient calculation. The truncation threshold and quantization parameters of each layer work together to determine the actual effect of pseudo-quantization of that layer, so that the quantization range matches the actual numerical distribution of each layer.

[0110] In this embodiment, truncation thresholds are determined based on the numerical distribution of weights and activation values ​​at each layer of the corpus, limiting the weights and activation values ​​to their respective truncation threshold ranges. Then, quantization and dequantization are performed according to the quantization parameters to obtain pseudo-quantization results. Through distribution-aware truncation processing, outliers are filtered out from interfering with the quantization range, ensuring that the quantization parameters match the actual numerical distribution of each layer, thereby improving quantization accuracy.

[0111] In this embodiment of the disclosure, dividing the restricted weight by the weight quantization parameter and rounding it down yields the quantized integer weight; multiplying the quantized integer weight by the weight quantization parameter yields the pseudo-quantized weight; quantizing and then dequantizing the restricted activation value according to the activation value quantization parameter of each layer to obtain the pseudo-quantized activation value includes: dividing the restricted activation value by the activation value quantization parameter and rounding it down to obtain the quantized integer activation value; multiplying the quantized integer activation value by the activation value quantization parameter to obtain the pseudo-quantized activation value.

[0112] In this embodiment, the truncated weight or activation value is divided by the corresponding quantization parameter to compress the continuous floating-point value into a discrete integer space. The mapping from floating-point to integer is achieved by rounding. This rounding operation causes the value to carry quantization noise, which is the core link in simulating low-bit-width storage. The rounding method includes, but is not limited to, rounding to zero, rounding up, rounding down, or random rounding. Different rounding methods have their own characteristics in terms of quantization error distribution and computational complexity, and can be flexibly selected according to the actual deployment scenario and accuracy requirements.

[0113] During dequantization, the quantized integer is multiplied by the same quantization parameter, expanding the discrete integer state back into a floating-point value. This preserves the quantization noise while restoring the floating-point format for subsequent floating-point calculations and gradient backpropagation. The combination of quantization and dequantization allows the model to experience the effects of quantization noise during forward propagation while maintaining floating-point format to meet the gradient calculation requirements of backpropagation. The pseudo-quantization operations for weights and activations are performed independently, each using its corresponding quantization parameter, ensuring that the accuracy loss during quantization is included in the training optimization scope.

[0114] For example, based on the numerical distribution of weights at each layer within the vertical domain corpus, a weight truncation threshold for each layer is determined. The truncation threshold is determined by the 99.9th percentile of the absolute values ​​of the weights at each layer within the vertical domain corpus, using the following formula:

[0115]

[0116] in, To truncate the threshold, These are the weight values ​​generated from input corpora in the vertical domain.

[0117] The weights of each layer are restricted to a weight truncation threshold range, as shown in the following formula:

[0118]

[0119] in, These are the truncated weight values. The weights of each layer are limited to... Within the specified range, any portion exceeding the range will be truncated.

[0120] Divide the constrained weights by the weight quantization parameter and round to the nearest integer to obtain the quantized integer weights; multiply the quantized integer weights by the weight quantization parameter to obtain the pseudo-quantized weights. The formula for the quantization process is as follows:

[0121]

[0122] in, These are the quantized integer weights. The truncated weights, weight scaling factor , This is the rounding function.

[0123] The formula for the dequantization process is as follows:

[0124]

[0125] in, These are the weights after pseudo-quantization.

[0126] In this embodiment, the restricted weights and activation values ​​are divided by their respective quantization parameters and rounded to obtain quantized integers. These integers are then multiplied by the corresponding quantization parameters to restore them to floating-point numbers, thus completing the quantization and dequantization operations. This process introduces quantization noise while preserving the floating-point data format, allowing the model to experience the effects of quantization in advance during forward propagation, while still being able to calculate gradients normally during backpropagation, achieving a seamless transition between pseudo-quantization and training.

[0127] In this embodiment of the disclosure, the target large model obtained through the above training is deployed on a computing device, which may be a server, a personal computer, an edge computing device, or a mobile terminal, to perform inference operations on input data with domain characteristics and obtain inference results.

[0128] In this embodiment of the disclosure, when the target large model is deployed, its weights and activation values ​​have been converted to low precision according to the quantization strategy determined during the training phase. Compared with the original full-precision model, the storage volume is greatly reduced and the inference speed is significantly improved, which can meet the real-time inference requirements of resource-constrained devices.

[0129] In this embodiment of the disclosure, the target large model is loaded into the processor and memory of the computing device. The processor performs layer-by-layer forward propagation calculations on the input data by calling the deployed target large model to generate inference results corresponding to the input data. These inference results are applied to vertical field scenarios such as medical auxiliary diagnosis, legal document analysis, or financial risk assessment.

[0130] Figure 4 A schematic diagram of the system architecture of the data processing method according to an embodiment of the present disclosure is shown.

[0131] In the embodiments disclosed herein, such as Figure 4 As shown, the system architecture includes an input and analysis phase, a computation and training phase, and a deployment and optimization phase.

[0132] During the input and analysis phase, the vertical domain corpus processing module acquires corpora with domain characteristics and preprocesses them, including cleaning noisy data and labeling core domain information. The layer feature analysis module inputs the corpus into the large-scale model to be trained, determining the sensitivity and relevance scores of each layer. The sensitivity score characterizes the degree to which each layer is sensitive to weight quantization, while the relevance score characterizes the correlation between activation values ​​and domain features in each layer. Different layers in the model may have the same or different sensitivity and relevance scores. This module also provides a stratification basis for subsequent quantization strategy decisions.

[0133] During the computation and training phases, the quantization strategy decision module determines the quantization strategy for each layer based on sensitivity and relevance scores. The domain-adaptive quantization parameter calculation module determines the weight quantization parameters and activation value quantization parameters for each layer based on their respective quantization strategies. The weight quantization parameters use a preset quantile from the weight distribution of each layer within the vertical domain corpus to calculate the weight scaling factor. For activation value quantization, the original precision is preserved for core activation values, while for non-core activation values, a sliding window calibration is used to dynamically calculate the activation value scaling factor. The integrated quantization fine-tuning training module performs pseudo-quantization on the weights and activation values ​​of each layer based on their quantization strategies, weight quantization parameters, activation value quantization parameters, and relevance scores. Forward propagation using the pseudo-quantized weights and activation values ​​yields the prediction results. A loss function is calculated based on the prediction results, and backpropagation is performed using the loss function to determine the gradient. The gradient is then weighted and adjusted based on the relevance scores of each layer to obtain the adjusted gradient. The adjusted gradient is used to update the original weights, resulting in the trained target model.

[0134] During the deployment and optimization phase, the quantization model conversion and deployment module converts the trained target model into a low-precision format and deploys it on computing devices for inference on domain-specific input data. The error assessment and iterative optimization module monitors the performance metrics of the core domain tasks after deployment. If performance degradation exceeds a preset threshold, it feeds back optimization strategies to the quantization strategy decision module, adjusting the quantization granularity or precision range and re-executing the optimization.

[0135] In this embodiment, regarding architectural layer innovation, a dual-dimensional driven hierarchical quantization framework combining sensitivity scoring and relevance scoring is used. Sensitivity scoring measures the degree of sensitivity of each layer to weight quantization, while relevance scoring measures the degree of correlation between the activation values ​​of each layer and domain features. Both jointly guide the formulation of differentiated quantization strategies. Regarding preprocessing layer innovation, quantization parameters are dynamically calculated using the 99.9th percentile of the vertical domain corpus, effectively filtering outliers from interfering with the quantization range and ensuring that the quantization parameters match the actual numerical distribution of each layer.

[0136] In terms of training layer innovation, a collaborative optimization strategy integrating quantization and fine-tuning is adopted. During the training process, pseudo-quantization is performed on the weights and activation values ​​of each layer to allow the model to adapt to quantization noise in advance. Combined with gradient weighted error compensation guided by relevance scoring, the domain key layers can obtain more sufficient error compensation during training, avoiding the degradation of domain knowledge caused by quantization operations, thereby improving the stability of vertical domain tasks.

[0137] It should be noted that, in the embodiments of this disclosure, experimental verification was conducted based on the method disclosed herein. In the medical large-scale model scenario, the storage volume of the traditional quantization scheme was 13GB, while that of this scheme was 2.3GB, a reduction of 82.3%; the inference speed increased from 10 tokens per second to 42 tokens per second, an improvement of 4.2 times; and the question-answering accuracy increased from 88% to 97.5%, an improvement of 9.5 percentage points. In the legal large-scale model scenario, the storage volume was reduced from 12GB to 2.1GB, a reduction of 82.5%; the clause citation accuracy increased from 85% to 96%, an improvement of 11 percentage points; and the response time was shortened from more than 1 second to at least 0.5 seconds, an improvement of 50%. The experimental results show that, after targeted quantization based on corpora from different vertical domains, this scheme effectively reduces the model size while ensuring inference accuracy, verifying the feasibility and effectiveness of this scheme.

[0138] Figure 5 A schematic block diagram of a data processing apparatus according to an embodiment of the present disclosure is shown.

[0139] like Figure 5 As shown, the data processing device 500 of this embodiment includes a corpus acquisition module 510, a model analysis module 520, a strategy determination module 530, and a gradient adjustment module 540.

[0140] The corpus acquisition module 510 is used to acquire corpora with domain features. The model analysis module 520 is used to input the corpus into the large model to be trained, and determine the sensitivity score and relevance score of each layer in the large model to be trained. The sensitivity score represents the degree of sensitivity of each layer to weight quantization, and the relevance score represents the degree of correlation between the activation values ​​in each layer and the domain features. The sensitivity scores and relevance scores of different layers in the model may be the same or different. The policy determination module 530 is used to determine the quantization strategy of each layer based on the sensitivity score and relevance score. The gradient adjustment module 540 is used to adjust the gradient of the backpropagation of each layer of the large model to be trained based on the quantization strategy and the relevance score of each layer, to obtain the trained target large model. The target large model is deployed on a computing device to perform inference on the input data with domain features.

[0141] According to embodiments of this disclosure, any plurality of modules among the corpus acquisition module 510, model analysis module 520, policy determination module 530, and gradient adjustment module 540 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the corpus acquisition module 510, model analysis module 520, policy determination module 530, and gradient adjustment module 540 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any one of the three implementation methods or a suitable combination of any of them. Alternatively, at least one of the corpus acquisition module 510, model analysis module 520, strategy determination module 530, and gradient adjustment module 540 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0142] Figure 6 A block diagram of an electronic device for data processing according to embodiments of the present disclosure is shown schematically.

[0143] like Figure 6 As shown, an electronic device 600 according to an embodiment of this disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this disclosure.

[0144] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 executes various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 602 and / or RAM 603. It should be noted that programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also execute various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.

[0145] According to embodiments of this disclosure, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 605 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0146] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0147] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 602 and / or RAM 603 and / or one or more memories other than ROM 602 and RAM 603 described above.

[0148] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this disclosure.

[0149] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0150] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0151] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0152] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0153] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0154] Those skilled in the art will understand that the features described in the various embodiments of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments of this disclosure can be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0155] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A data processing method applied to a computing device, the method comprising: Obtain corpora with domain-specific features; The corpus is input into the large model to be trained, and the sensitivity score and relevance score of each layer in the large model to be trained are determined. The sensitivity score represents the degree of sensitivity of each layer to weight quantization, and the relevance score represents the degree of correlation between the activation value and the domain feature in each layer. The sensitivity score and relevance score of different layers in the model may be the same or different. The quantification strategy for each layer is determined based on the sensitivity score and the correlation score. Based on the quantization strategy and the correlation score of each layer, the gradient of the backpropagation of each layer of the large model to be trained is adjusted to obtain the trained target large model. The target large model is deployed on the computing device to perform inference on input data with the domain features.

2. The method according to claim 1, wherein inputting the corpus into the large model to be trained and determining the sensitivity score and relevance score of each layer in the large model to be trained includes: The corpus is input into the large model to be trained, and multiple first activation value vectors and multiple second activation value vectors under pseudo-quantization are obtained for each layer. The sensitivity score is determined based on the difference between the multiple first activation value vectors and the multiple second activation value vectors. The first activation value vector is the original activation value generated by the model under unquantized conditions, and the second activation value vector is the noisy activation value generated by the model under quantized and dequantized conditions. The correlation score is determined based on the plurality of first activation value vectors and the preset activation value mapping library.

3. The method according to claim 2, wherein determining the sensitivity score based on the difference between the plurality of first activation value vectors and the plurality of second activation value vectors comprises: Calculate the sum of squares of the differences between each first activation value vector and the corresponding second activation value vector; The sensitivity score is obtained by normalizing the average of the sums of squares.

4. The method according to claim 2, wherein determining the relevance score based on the first activation value vector and a preset activation value mapping library includes: Calculate multiple similarities between the plurality of first activation value vectors and each feature vector in the preset activation value mapping library. The preset activation value mapping library is composed of activation value vectors output by each layer after the domain information labeled in the corpus is input into the large model to be trained. The maximum value of the multiple similarities is used as the relevance score.

5. The method according to claim 1, wherein determining the quantification strategy for each layer based on the sensitivity score and the correlation score comprises: Layers that meet the preset first classification conditions are identified as first-class layers. The preset first classification conditions include that the sensitivity score is greater than or equal to a first threshold and the correlation score is greater than or equal to a second threshold. The activation value of the first-class layer adopts a first precision quantization method, and the weight quantization precision is lower than the activation value quantization precision. The layers that meet the preset second classification conditions are determined as the second type of layer. The preset second classification conditions include the sensitivity score being greater than or equal to the third threshold and less than the first threshold, or / and the correlation score being greater than or equal to the fourth threshold and less than the second threshold. The activation value of the second type of layer adopts the second precision quantization method, and the weight quantization precision is lower than the activation value quantization precision. Layers that meet the preset third classification conditions are identified as third-class layers. The preset third classification conditions include that the sensitivity score is less than a third threshold and the correlation score is less than a fourth threshold. The activation value of the third-class layer adopts a third-precision quantization method, and the weight quantization precision is lower than or equal to the activation value quantization precision. The preset first classification conditions, the preset second classification conditions, and the preset third classification conditions are used to filter layers with different importance levels. The precision of the first precision quantization method is higher than that of the second precision quantization method, and the precision of the second precision quantization method is higher than that of the third precision quantization method.

6. The method according to claim 1, wherein adjusting the gradients of the backpropagation of each layer of the large model to be trained based on the quantization strategy and the correlation score of each layer to obtain the trained target large model comprises: The weight quantization parameters and activation value quantization parameters of each layer are determined according to the quantization strategy of each layer. Based on the activation value quantization parameters of each layer, the correlation scores of each layer, the gradients of the backpropagation of each layer of the large model to be trained are adjusted to obtain the trained target large model.

7. The method according to claim 6, wherein adjusting the gradient of backpropagation of each layer of the large model to be trained based on the activation value quantization parameters of each layer, the activation value quantization parameters, and the correlation score of each layer to obtain the trained target large model includes: Based on the weight quantization parameters and activation value quantization parameters of each layer, pseudo-quantization is performed on the weights and activation values ​​of each layer of the large model to be trained to obtain pseudo-quantized weights and pseudo-quantized activation values. The corpus is input into the large model to be trained, and the pseudo-quantized weights and pseudo-quantized activation values ​​are used for forward propagation to obtain the prediction result. Calculate the loss function based on the prediction results; Backpropagation is performed based on the loss function to determine the backpropagation gradient. The backpropagation gradient is then weighted and adjusted according to the correlation scores of each layer to obtain the adjusted gradient. The original weights of the large model to be trained are updated using the adjusted gradients to obtain the target large model.

8. The method according to claim 7, wherein the step of pseudo-quantizing the weights and activation values ​​of each layer of the large model to be trained based on the weight quantization parameters and activation value quantization parameters of each layer to obtain pseudo-quantized weights and pseudo-quantized activation values ​​includes: Based on the numerical distribution of the weights of each layer in the corpus, the weight cutoff threshold of each layer is determined, and the weights of each layer are restricted within the weight cutoff threshold range. The restricted weights are then quantized and dequantized according to the weight quantization parameters of each layer to obtain pseudo-quantized weights. Based on the numerical distribution of activation values ​​in each layer of the corpus, an activation value truncation threshold is determined for each layer. The activation values ​​of each layer are then restricted to the range of the activation value truncation threshold. The restricted activation values ​​are then quantized and dequantized according to the activation value quantization parameters of each layer to obtain pseudo-quantized activation values.

9. The method according to claim 8, wherein quantizing and then dequantizing the restricted weights to obtain pseudo-quantized weights comprises: Divide the restricted weight by the weight quantization parameter and round down to obtain the quantized integer weight. Multiply the quantized integer weights by the weight quantization parameters to obtain the pseudo-quantized weights; The step of quantizing the restricted activation value according to the activation value quantization parameters of each layer and then dequantizing it to obtain the pseudo-quantized activation value includes: Divide the restricted activation value by the activation value quantization parameter and round down to obtain the quantized integer activation value. The quantized integer activation value is multiplied by the activation value quantization parameter to obtain the pseudo-quantized activation value.

10. An electronic device, comprising: Memory and processor; The processor is configured as follows: Obtain corpora with domain-specific features; The corpus is input into the large model to be trained. Sensitivity scores, which represent the sensitivity to weight quantization, and correlation scores, which represent the correlation between activation values ​​and domain features in each layer of the large model to be trained, are determined. Based on the sensitivity scores and correlation scores, the quantization strategy for each layer is determined. Based on the quantization strategy and correlation scores of each layer, the gradients of the backpropagation of each layer of the large model to be trained are adjusted to obtain the trained target large model. The target large model is deployed on the electronic device to perform inference on input data with the domain features.

Images