A service processing method, apparatus, device and medium

Abstract

A service processing method, device, equipment and medium are disclosed. The method comprises: determining a service processing model to be called according to a received service processing request, and parsing the service processing request to obtain service data; determining a service processing result based on the service data and the service processing model; determining a service response message of the service processing request according to the service processing result; wherein the service processing model is obtained by quantitatively processing network nodes in an original processing model based on preset standard data. The technical solution of the present application can reduce the parameter size and calculation amount of the model, and reduce the calculation cost of processing services.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a business processing method, apparatus, device, and medium. Background Technology

[0002] With the widespread application of neural networks in fields such as computer vision, speech recognition, natural language processing, autonomous driving, smart cockpits, and industrial vision, the parameter scale and inference computation of business models used to process business in various fields continue to increase.

[0003] For low-power computing platforms such as Field Programmable Gate Arrays (FPGAs), Digital Signal Processors (DSPs), Neural Processing Units (NPUs), mobile chips, or automotive processors, directly deploying and calling business models to process corresponding business operations usually leads to problems such as large storage usage, power consumption, and latency, resulting in high computing costs for processing business operations.

[0004] Therefore, there is an urgent need for a method that can reduce the computing costs of processing business. Summary of the Invention

[0005] This invention provides a business processing method, apparatus, equipment, and medium that can reduce the parameter scale and computational load of the model, thereby reducing the computational cost of processing business.

[0006] According to one aspect of the present invention, a business processing method is provided, comprising: Based on the received business processing request, determine the business processing model that needs to be invoked, and parse the business processing request to obtain business data; Based on the business data and the business processing model, the business processing result is determined; Based on the business processing result, determine the business response message for the business processing request; The business processing model is obtained by quantifying the network nodes in the original processing model based on preset standard data.

[0007] According to a second aspect of the present invention, a business processing apparatus is provided, comprising: The acquisition module is used to determine the business processing model to be invoked based on the received business processing request, and to parse the business processing request to obtain business data; The determination module is used to determine the business processing result based on the business data and the business processing model; The response module is used to determine the business response message for the business processing request based on the business processing result. The business processing model is obtained by quantifying the network nodes in the original processing model based on preset standard data.

[0008] According to a third aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the business processing method described in any embodiment of the present invention.

[0009] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the business processing method described in any embodiment of the present invention.

[0010] The technical solution of this invention can determine the business processing model to be invoked based on the received business processing request, parse the business processing request to obtain business data, then determine the business processing result based on the business data and the business processing model, and finally determine the business response message of the business processing request based on the business processing result. The business processing model is obtained by quantizing the network nodes in the original processing model based on preset standard data. Therefore, the technical solution of this invention can pre-quantize the network nodes in the original processing model based on preset standard data, thereby reducing the parameter size and computational load of the model, and thus performing business processing operations through the quantized model, thereby reducing the computational cost of processing business.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a business processing method provided according to Embodiment 1 of the present invention; Figure 2This is a flowchart of a business processing model acquisition method provided in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the structure of a business processing device according to Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the business processing method of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] Example 1 Figure 1 This is a flowchart of a business processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where business processing is implemented through a quantified business processing model. The method can be executed by a business processing device, which can be implemented in hardware and / or software and configured in a device or system used for business processing. Figure 1 As shown, the method includes: S101. Based on the received business processing request, determine the business processing model that needs to be called, and parse the business processing request to obtain business data.

[0017] In this context, a business processing request can be an instruction or message initiated by the user or an upstream system, requesting the execution of a specific artificial intelligence (AI) business task. AI business tasks can be those requiring AI inference and computation in fields such as computer vision, speech recognition, natural language processing, autonomous driving, smart cockpits, and industrial vision, such as image classification, speech recognition, and text understanding.

[0018] A business processing model can be an AI model that has been pre-trained and quantized for optimization and can be used to perform specific AI business tasks. For example, a business processing model can be a model specifically designed for image classification, object detection, speech recognition, or natural language processing.

[0019] Business data can be information extracted from business processing requests that needs to be processed by a business processing model. For example, business data can be images to be classified, speech segments to be recognized, or text sequences to be understood.

[0020] For example, upon receiving a business processing request, the request can first be parsed to identify the task type identifier or model call information contained therein, thereby determining the business processing model that needs to be called.

[0021] At the same time, compressed data can be extracted from business processing requests, and after format conversion or preprocessing, it can be formed into business data that can be directly processed by the business processing model.

[0022] S102. Based on business data and business processing model, determine the business processing result, wherein the business processing model is obtained by quantifying the network nodes in the original processing model based on preset standard data.

[0023] In this embodiment, the original processing model can be a deep neural network model pre-built for performing specific artificial intelligence business tasks, and its internal computational logic can consist of multiple interconnected network nodes. These network nodes can be used to perform specific mathematical operations on their input tensors and generate output tensors. It should be noted that network nodes can be computational units of various types, such as convolution nodes, matrix multiplication nodes, fully connected nodes, activation nodes, or normalization nodes.

[0024] The preset standard data can be a batch of representative sample data used to drive the model quantization process. This sample data can reflect the distribution characteristics of the data that the processing model needs to process in actual deployment scenarios. Quantization can be understood as converting the weights, activation values, or output tensors in network nodes from their original high-precision numerical representations to lower-precision numerical representations, thereby reducing the computational load, storage consumption, and power consumption of the model during inference.

[0025] It should be noted that the original processing model has a large parameter scale and inference computation load. Directly using the original processing model for business processing will cause significant problems for the low-power computing platform, such as increased storage consumption, power consumption, and latency. Therefore, this example can reduce the parameter scale and computation load of the model by quantizing the original processing model first.

[0026] For example, firstly, a pre-built original processing model for performing a specific business task can be obtained, and multiple network nodes in the computation graph of the original processing model can be identified. Then, inference is performed on the original processing model using preset standard data, and the output tensor information generated by each network node under the drive of standard data input is collected. Then, based on the output distribution information, a quantization adaptability evaluation is performed on the network nodes in the original processing model to determine which nodes are suitable for low-precision quantization and which nodes need to retain high precision. Thus, precision conversion is performed on nodes suitable for quantization, and the original precision is retained for nodes unsuitable for quantization, ultimately forming a quantized business processing model.

[0027] S103. Based on the business processing result, determine the business response message for the business processing request.

[0028] Among them, the business response message can be a feedback message containing processing result information sent to the party requesting the business processing.

[0029] For example, after obtaining the business processing result, the result can be processed according to the needs of the business scenario. For instance, the text processing result can be formatted and encapsulated, or the image recognition result can be labeled. Then, the processed data can be encapsulated into a response message of a preset communication protocol as the final business response message of the business processing request and returned to the requester.

[0030] The technical solution of this invention can determine the business processing model to be invoked based on the received business processing request, parse the business processing request to obtain business data, then determine the business processing result based on the business data and the business processing model, and finally determine the business response message of the business processing request based on the business processing result. The business processing model is obtained by quantizing the network nodes in the original processing model based on preset standard data. Therefore, the technical solution of this invention can pre-quantize the network nodes in the original processing model based on preset standard data, thereby reducing the parameter size and computational load of the model, and thus performing business processing operations through the quantized model, thereby reducing the computational cost of processing business.

[0031] Example 2 Figure 2This is a flowchart of a business processing model acquisition method provided in Embodiment 2 of the present invention. This embodiment can further explain the process of obtaining a business processing model by quantifying network nodes in the original processing model based on preset standard data.

[0032] It should be noted that, in order to reduce the computational, storage, and inference latency of the post-deployment processing model, model quantization is required (that is, converting some data from high-precision numerical representation to low-precision numerical representation). However, not all nodes in the processing model are suitable for quantization; therefore, the nodes suitable for quantization need to be identified before model quantization.

[0033] Currently, the existing methods for determining whether each node in a model is suitable for quantization are usually based on setting fixed rules according to the operator type of the node (such as skipping nodes in key structural positions such as attention mechanisms and normalization), or having engineers manually maintain whitelists or blacklists based on experience. This relies on human experience and often requires reconfiguration for different processing models, resulting in low quantization efficiency of the processing model.

[0034] Based on this, the present invention provides a method for obtaining business processing models, which can improve the efficiency and accuracy of model quantification. For example... Figure 2 As shown, the method includes: S201. Determine at least one candidate quantization node for the original processing model.

[0035] Among them, candidate quantization nodes can be computing nodes that can be initially screened from all network nodes of the original processing model and are suitable for quantization processing.

[0036] For example, after obtaining the original processing model, the computation graph of the original processing model can be parsed first to obtain the set of all nodes in the original processing model and the directed connections between the nodes. The computation graph can describe the flow order and connection relationships of data between nodes in the neural network model.

[0037] Then, based on the types of quantization operators supported by the hardware backend that deploys the business processing model, computational nodes capable of performing low-precision calculations can be initially screened from the node set as candidate quantization nodes. For example, for hardware backends that only support low-precision inference for some operators such as convolution and matrix multiplication, candidate nodes can be limited to convolution nodes and matrix multiplication nodes; for hardware backends that support a wider range of low-precision calculations, addition nodes or concatenation nodes can be further included.

[0038] The advantage of this setup is that it can initially screen out nodes with quantization potential from the numerous network nodes of the original processing model, narrowing down the scope for subsequent refined judgment and improving the efficiency of determining quantization targets.

[0039] S202. Based on preset standard data, determine the node feature vector of the candidate quantization node.

[0040] Among them, the node feature vector can be the feature value used to characterize the relevant attributes of the candidate quantized node.

[0041] For example, after determining the candidate quantization nodes, the preset standard data is first input into the original processing model for high-precision inference, and the high-precision output tensor generated by each candidate quantization node under the standard data is collected. Here, high-precision inference can be understood as performing inference calculations using the original numerical precision of the original processing model (such as 32-bit floating-point numbers), and high-precision output tensors can be understood as the output tensors generated by the candidate quantization nodes during the high-precision inference process.

[0042] Then, statistical features can be extracted from the high-precision output tensor to obtain the numerical distribution features of the high-precision output tensor, such as mean, standard deviation, extreme values, quantiles, skewness, kurtosis, or the proportion of zero values, which can be used as the node feature vector of the candidate quantization node.

[0043] S203. Analyze the node feature vectors using a pre-trained node judgment model to obtain the quantization adaptability results of candidate quantization nodes.

[0044] The node judgment model can be a machine learning model that is trained in advance using node sample data of the reference processing model. This node judgment model can perform quantization adaptive prediction on candidate quantization nodes by learning the mapping relationship between node feature vectors and quantization adaptability.

[0045] The quantization adaptability result can be the judgment conclusion output by the node judgment model on whether the candidate quantization node is suitable for low-precision quantization.

[0046] For example, the node feature vectors of each candidate quantization node can be input into a pre-trained node judgment model. The node judgment model then analyzes and processes each candidate quantization node based on the learned mapping relationship between node features and quantization fitness, and outputs the corresponding quantization fitness results. For instance, the node judgment model can output the probability value of each candidate quantization node being suitable for quantization, or output the quantization risk score of each candidate quantization node, or directly classify the candidate quantization nodes as suitable for quantization or unsuitable for quantization.

[0047] S204. Based on the quantization adaptability results, determine the target quantization node from the candidate quantization nodes.

[0048] The target quantization node can be a network node selected from candidate quantization nodes and deemed suitable for low-precision quantization. The quantization adaptability result can include judgment information on whether each candidate quantization node is suitable for quantization, such as probability value, risk level, or classification label.

[0049] For example, the quantization adaptability results of each candidate quantization node can be judged and processed according to preset decision rules. For instance, when the quantization adaptability result is a probability value, candidate quantization nodes with probability values ​​greater than or equal to a preset threshold can be identified as target quantization nodes; when the quantization adaptability result is a risk score, candidate quantization nodes with risk scores lower than a preset threshold can be identified as target quantization nodes; when the quantization adaptability result is a classification label, candidate quantization nodes labeled as suitable for quantization can be directly identified as target quantization nodes. For the remaining candidate quantization nodes that are not identified as target quantization nodes, they can be configured to retain a high-precision representation to avoid accuracy loss due to over-quantization.

[0050] S205. Perform quantization operations on the original processing model based on the target quantization node to obtain the business processing model.

[0051] For example, a low-precision quantization transformation can be performed on the target quantization node, mapping its weights and / or activation outputs to a lower-precision numerical space; for the remaining nodes that are not identified as the target quantization node, their original high-precision numerical representations are preserved.

[0052] Specifically, it should be noted that when the quantization adaptability result of a candidate quantization node does not meet the preset decision conditions, the candidate quantization node can be determined as a non-target quantization node, i.e., a node unsuitable for low-precision quantization. After determining the target and non-target quantization nodes, a node-level quantization strategy configuration file can be generated based on the information of the target and non-target quantization nodes.

[0053] The node-level quantization strategy configuration file can be understood as a structured configuration file used to specify, on a node-by-node basis, which nodes perform low-precision quantization and which nodes skip quantization and retain high precision. This node-level quantization strategy configuration file can be in JSON, YAML, or other structured formats that can be parsed by the deployment framework.

[0054] Then, the node-level quantization strategy configuration file can be parsed using the target deployment framework. Low-precision quantization operations can be performed on target quantization nodes, while the original high precision is retained for non-target quantization nodes, thereby obtaining the business processing model. The target deployment framework can be an inference optimization framework used for model transformation and deployment.

[0055] Understandably, by using differentiated quantization, the overall parameter size and computational load of the model can be reduced while avoiding the decrease in accuracy caused by quantizing key nodes, ultimately resulting in a business processing model suitable for business processing.

[0056] The technical solution of this embodiment can first determine at least one candidate quantization node of the original processing model, then determine the node feature vector of the candidate quantization node based on preset standard data, then analyze the node feature vector through a pre-trained node judgment model to obtain the quantization adaptability result of the candidate quantization node, and finally determine the target quantization node from the candidate quantization nodes according to the quantization adaptability result, and perform quantization operation on the original processing model based on the target quantization node to obtain the business processing model. It can realize automatic judgment of the quantization adaptability of candidate quantization nodes without relying on human experience to set fixed rules or manually maintain blacklists and whitelists, which improves the efficiency and accuracy of model quantization. At the same time, by independently evaluating the adaptability of each candidate quantization node, differentiated quantization strategies can be generated, which can effectively maintain the business processing accuracy of the model.

[0057] Based on the above embodiments, as one possible approach, determining the node feature vector of candidate quantized nodes based on preset standard data may include: Standard data is input into the original processing model for high-precision inference to obtain a high-precision output tensor. Determine the numerical distribution characteristics of the high-precision output tensor, and determine the node structure characteristics of the candidate quantization nodes; By combining numerical distribution features and node structure features, a node feature vector is obtained.

[0058] Among them, numerical distribution features can be attribute information used to characterize the numerical statistical regularity and distribution pattern of the output data of candidate quantization nodes, which may include, but are not limited to, statistical features, distribution features and quantization prior features.

[0059] Statistical features can be quantities that describe the central tendency and dispersion of the output dataset, such as the mean, standard deviation, maximum, minimum, absolute maximum, median, quantile, dynamic range, mean absolute value, and coefficient of variation of the output tensor.

[0060] Distribution characteristics can be quantities that describe the distribution pattern and sparsity of output data, such as skewness, kurtosis, histogram entropy, long tail ratio, zero value ratio, near-zero value ratio, non-zero ratio, positive and negative value ratio, saturation interval ratio, and distribution differences between channels.

[0061] Prior quantization features can be error-related attributes estimated in advance based on the simulated quantization process, such as symmetric quantization estimation error, asymmetric quantization estimation error, error differences under different quantization granularities, simulation errors under different quantization bit widths, truncation threshold sensitivity, quantization scaling factor distribution, and zero point distribution.

[0062] Node structural features can be attribute information describing the contextual positional relationship of candidate quantization nodes in the computation graph, such as node operator type, upstream and downstream node types, input and output tensor dimensions, number of channels, feature map size, node topology depth, number of branches, whether it is located in a residual connection, whether it is located in an attention module, whether it is located near the output head, and whether it has broadcasting, splicing, or normalization operations associated with it. Node feature vectors can be fixed-length feature representations formed by combining the above features to comprehensively characterize the quantization adaptability of candidate quantization nodes.

[0063] For example, after determining the candidate quantization nodes, the preset standard data can be input into the original processing model for high-precision inference, and the high-precision output tensor generated by each candidate quantization node under the standard data driving is collected.

[0064] Then, numerical distribution features can be extracted from the high-precision output tensor, including calculating the statistical features (such as mean, standard deviation, extreme values, quantiles, etc.), distribution features (such as skewness, kurtosis, percentage of zero values, proportion of long tails, etc.), and quantization prior features (such as simulated quantization estimation error, error differences under different bit widths, etc.). It should be noted that the extraction of numerical distribution features can be performed synchronously with the high-precision inference process. To reduce storage overhead, online statistics can be used during inference, without storing the complete high-precision output tensor. Instead, the necessary statistics for features such as mean, variance, maximum value, minimum value, quantile histogram, and percentage of zero values ​​are calculated simultaneously during forward inference, thereby reducing the storage resource consumption of the feature extraction stage.

[0065] Meanwhile, based on the computational graph topology of the original processing model, the node structure features of each candidate quantization node can be extracted, including node type, upstream and downstream connection relationship, topology depth, branch position, and whether it is in a critical structural position.

[0066] Subsequently, the extracted numerical distribution features and node structure features can be combined and processed, for example, by feature concatenation, normalization, standardization or encoding, to form a node feature vector that can comprehensively characterize the quantization adaptability of candidate quantization nodes.

[0067] The advantage of this setup is that by extracting features from two dimensions—the numerical distribution of node output data and the computational graph structural context—it can comprehensively reflect the data distribution characteristics and structural features of candidate quantization nodes, providing rich decision-making basis for subsequent node judgment models, thereby improving the accuracy of quantization adaptability judgment.

[0068] Based on the above embodiments, it should be further explained that the training method of the node judgment model may include... After obtaining the reference processing model, at least one reference node of the reference processing model is determined; Based on the preset standard data, the reference output tensor of the reference node is determined, and the corresponding reference node feature vector is extracted. Simulated quantization is performed on the reference node based on the reference node's feature vector to obtain the simulated output tensor; Calculate the error index between the simulated output tensor and the reference output tensor, and determine the quantization fitness label of the reference node based on the error index; The reference node feature vector and quantized adaptive label are used as training samples to train a pre-defined learning model, resulting in a node judgment model.

[0069] The reference processing model can be a neural network model used to provide training samples, which can be used to process the corresponding business that the business processing model needs to handle. It can have the same or different network structure as the original processing model. The reference node can be a network node in the reference processing model that needs to be quantitatively evaluated for adaptability.

[0070] The reference output tensor can be the high-precision output data generated by the reference node during inference computation using the original numerical precision. The reference node eigenvector can be the numerical distribution characteristics derived from the reference output tensor. The simulated output tensor can be the output data obtained from the reference node after simulated quantization processing.

[0071] Error metrics can be quantities used to measure the degree of difference between an analog output tensor and a reference output tensor. These can include one or more of the following: mean square error, mean absolute error, maximum absolute error, cosine similarity, KL divergence, relative error, signal-to-noise ratio, and mission-level accuracy variation.

[0072] Quantization adaptability labels can be annotation information used to characterize whether a reference node is suitable for low-precision quantization, and they can be determined based on the comparison results of error indicators and preset thresholds.

[0073] Training samples can be data pairs consisting of reference node feature vectors and corresponding quantized adaptive labels. The pre-defined learning model can be a machine learning model capable of learning the mapping relationship between node feature vectors and quantized adaptive labels, such as random forests, gradient boosting trees, support vector machines, logistic regression, multilayer perceptrons, graph neural networks, or ensemble learning models.

[0074] For example, after obtaining the reference processing model, its computational graph structure is first parsed to determine at least one reference node. Then, preset standard data can be input into the reference processing model for high-precision inference, the reference output tensor of each reference node is collected, and feature extraction is performed on the reference output tensor to obtain the feature vector of the reference node.

[0075] Next, simulated quantization is performed on each reference node to obtain a simulated output tensor. Then, the error index between the simulated output tensor and the reference output tensor is calculated, and the quantization adaptability label for each reference node is determined according to preset labeling rules. For example, the error index can be compared with a preset threshold, and binary classification labels can be generated based on the comparison result; alternatively, labels can be determined by combining task-level accuracy changes. Finally, the feature vectors of the reference nodes and their corresponding quantization adaptability labels are combined to form training samples. These training samples are used to train a preset learning model, enabling the model to learn the mapping relationship between node features and quantization adaptability, thus obtaining a node judgment model.

[0076] During training, the imbalance of sample numbers for different label categories can be addressed through methods such as cross-validation, class weight adjustment, oversampling, or undersampling. Furthermore, the training and validation sets can be divided according to the model architecture to evaluate the model's cross-architecture generalization ability.

[0077] The advantage of this setup is that by automatically constructing training samples using the node data of the reference processing model, the node judgment model can learn the complex mapping relationship between node features and quantization error. Thus, when facing the processing model, it can automatically predict the quantization adaptability of each node without relying on human experience.

[0078] Based on the above description, performing simulated quantization on the reference node according to the reference node's feature vector to obtain the simulated output tensor can include: The quantization scaling factor and zero point are determined based on the eigenvectors of the reference nodes. Based on the quantization scaling factor and zero point, a pseudo-quantization operation is performed on the reference output tensor to obtain the analog output tensor.

[0079] The quantization scaling factor can be a quantization coefficient used to map high-precision floating-point values ​​to low-precision integer value space. The zero point can be an offset used to align floating-point values ​​with integer value spaces. The analog output tensor can be the analog output used to simulate low-precision inference.

[0080] For example, the quantization scaling factor and zeros can be calculated based on the maximum, minimum, or quantile threshold of the node output tensor in the reference node feature vector. Then, based on the determined quantization scaling factor and zeros, the values ​​of the reference output tensor are scaled according to the quantization scaling factor and the zero offset is added to map them to the target low-precision integer space. The mapped integer values ​​are then rounded and truncated to ensure they fall within the range of the target low-precision numerical representation. Finally, the processed integer values ​​are back-mapped back to the floating-point space according to the quantization scaling factor and zeros to obtain the analog output tensor.

[0081] Based on the above description, calculating the error index between the simulated output tensor and the reference output tensor, and determining the quantization fitness label of the reference node based on the error index, may include: Calculate at least two error metrics between the simulated output tensor and the reference output tensor; Each error index is normalized separately, and then the normalized error indices are weighted and summed according to the preset weight coefficients to obtain the comprehensive error result. The comprehensive error result is compared with a preset threshold, and the quantization adaptive label is determined based on the comparison result.

[0082] The error metrics may include one or more of the following: mean squared error, mean absolute error, maximum absolute error, cosine similarity, KL divergence, relative error, signal-to-noise ratio, and task-level accuracy variation. Normalization can be an operation used to eliminate differences in dimensions and numerical ranges between different error metrics, such as min-max normalization or standardization.

[0083] The comprehensive error results can be used to comprehensively assess the quantization risk of the reference node (i.e., the risk of reduced accuracy after quantization). Quantization adaptability labels can be annotation information used to characterize whether the reference node is suitable for low-precision quantization, and can include binary classification labels, multi-level risk labels, or continuous risk scores.

[0084] For example, at least two error metrics, such as mean squared error and cosine similarity, are calculated between the simulated output tensor and the reference output tensor. Then, each error metric is normalized to eliminate differences in their numerical ranges. Next, the normalized error metrics are weighted and summed according to preset weighting coefficients to obtain a comprehensive error result. Finally, the comprehensive error result is compared with a preset threshold, and a quantization suitability label is determined based on the comparison result. For example, when the comprehensive error result is lower than the preset threshold, the reference node is labeled as suitable for quantization; when the comprehensive error result is higher than or equal to the preset threshold, the reference node is labeled as unsuitable for quantization.

[0085] Furthermore, different levels of quantization adaptability labels can be generated based on the different numerical ranges of the comprehensive error results to support differentiated quantization strategy configurations. For example, low risk suggests direct INT8 quantization; medium risk suggests per-channel quantization or FP16 retention; high risk suggests skipping quantization; and very high risk suggests QAT or structural adjustments.

[0086] The advantage of this setup is that by integrating multiple error indicators and assigning different weights, it is possible to comprehensively evaluate the error characteristics of nodes after quantization, avoiding the one-sidedness of evaluating a single error indicator. At the same time, it can also improve the accuracy of quantization adaptive label generation.

[0087] Based on the above embodiments, the method may further include: Obtain the accuracy feedback data of the business processing model; The node judgment model is updated based on the accuracy feedback data.

[0088] Among them, the accuracy feedback data can be information that reflects the accuracy of the business processing model during the actual deployment and operation of the business processing model. It can include, but is not limited to, one or more of the following: changes in task-level accuracy indicators, node-level error data, and hardware execution effects.

[0089] Task-level accuracy metrics can be interpreted as the change in processing accuracy of a business processing model relative to the original model when executing a specific business task. Node-level error data can be the quantized output error information of each node in the business processing model. Hardware execution performance can be the effect of computing resources and storage space when the business processing model is executed on hardware.

[0090] For example, after deploying the business processing model to the target hardware platform and executing the business processing task, accuracy feedback data of the business processing model can be collected, such as recording changes in task-level accuracy metrics and the actual error performance of each node. Then, based on the collected accuracy feedback data, the corresponding node feature vectors and quantization adaptability labels are extracted and added to the training sample set as new training samples. Finally, based on the updated training sample set, the node judgment model is trained and updated, enabling the node judgment model to learn the correspondence between new node features and quantization adaptability, thereby continuously optimizing its ability to predict the quantization adaptability of subsequent new models.

[0091] The advantage of this setup is that it allows the node judgment model to continuously accumulate quantitative experience under different model architectures, business tasks, and hardware backends, gradually improving prediction accuracy and generalization ability.

[0092] Example 3 Figure 3 This is a schematic diagram of a business processing device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: The acquisition module 301 can be used to determine the business processing model to be invoked based on the received business processing request, and to parse the business processing request to obtain business data. The determination module 302 can be used to determine the business processing result based on business data and business processing model; The response module 303 can be used to determine the business response message of the business processing request based on the business processing result. The business processing model is obtained by quantifying the network nodes in the original processing model based on preset standard data.

[0093] The technical solution of this invention can determine the business processing model to be invoked based on the received business processing request, parse the business processing request to obtain business data, then determine the business processing result based on the business data and the business processing model, and finally determine the business response message of the business processing request based on the business processing result. The business processing model is obtained by quantizing the network nodes in the original processing model based on preset standard data. Therefore, the technical solution of this invention can pre-quantize the network nodes in the original processing model based on preset standard data, thereby reducing the parameter size and computational load of the model, and thus performing business processing operations through the quantized model, thereby reducing the computational cost of processing business.

[0094] Optionally, the business processing model can be obtained by quantizing the network nodes in the original processing model based on preset standard data, which may include: Identify at least one candidate quantization node for the original processing model; Based on preset standard data, determine the node feature vector of candidate quantized nodes; The quantization adaptability results of candidate quantization nodes are obtained by analyzing the node feature vectors through a pre-trained node judgment model. Based on the quantization adaptability results, the target quantization node is determined from the candidate quantization nodes; The original processing model is quantized based on the target quantization node to obtain the business processing model.

[0095] Optionally, determining the node feature vectors of candidate quantized nodes based on preset standard data may include: Standard data is input into the original processing model for high-precision inference to obtain a high-precision output tensor. Determine the numerical distribution characteristics of the high-precision output tensor, and determine the node structure characteristics of the candidate quantization nodes; By combining numerical distribution features and node structure features, a node feature vector is obtained.

[0096] Optionally, the training method for the node determination model may include: After obtaining the reference processing model, at least one reference node of the reference processing model is determined; Based on the preset standard data, the reference output tensor of the reference node is determined, and the corresponding reference node feature vector is extracted. Simulated quantization is performed on the reference node based on the reference node's feature vector to obtain the simulated output tensor; Calculate the error index between the simulated output tensor and the reference output tensor, and determine the quantization fitness label of the reference node based on the error index; The reference node feature vector and quantized adaptive label are used as training samples to train a pre-defined learning model, resulting in a node judgment model.

[0097] Optionally, performing simulated quantization on the reference node based on its eigenvector to obtain a simulated output tensor may include: The quantization scaling factor and zero point are determined based on the eigenvectors of the reference nodes. Based on the quantization scaling factor and zero point, a pseudo-quantization operation is performed on the reference output tensor to obtain the analog output tensor.

[0098] Optionally, calculate the error index between the simulated output tensor and the reference output tensor, and determine the quantization fitness label of the reference node based on the error index, including: Calculate at least two error metrics between the simulated output tensor and the reference output tensor; Each error index is normalized separately, and then the normalized error indices are weighted and summed according to the preset weight coefficients to obtain the comprehensive error result. The comprehensive error result is compared with a preset threshold, and the quantization adaptive label is determined based on the comparison result.

[0099] Optionally, the device may also include: an update module, which can be used to obtain accuracy feedback data of the business processing model; and update the node judgment model based on the accuracy feedback data.

[0100] The business processing apparatus provided in the embodiments of the present invention can execute the business processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0101] Example 4 Figure 4 A schematic diagram of an electronic device 400 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0102] like Figure 4 As shown, the electronic device 400 includes at least one processor 401 and a memory, such as a read-only memory (ROM) 402 or a random access memory (RAM) 403, communicatively connected to the at least one processor 401. The memory stores computer programs executable by the at least one processor. The processor 401 can perform various appropriate actions and processes based on the computer program stored in the ROM 402 or loaded into the RAM 403 from storage unit 408. The RAM 403 may also store various programs and data required for the operation of the electronic device 400. The processor 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0103] Multiple components in electronic device 400 are connected to I / O interface 405, including: input unit 406, such as keyboard, mouse, etc.; output unit 407, such as various types of displays, speakers, etc.; storage unit 408, such as disk, optical disk, etc.; and communication unit 409, such as network card, modem, wireless transceiver, etc. Communication unit 409 allows electronic device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0104] Processor 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 401 performs the various methods and processes described above, such as business processing methods.

[0105] In some embodiments, the business processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by processor 401, one or more steps of the business processing method described above may be performed. Alternatively, in other embodiments, processor 401 may be configured to perform the business processing method by any other suitable means (e.g., by means of firmware).

[0106] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0107] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0108] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0109] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0110] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0111] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0112] In one embodiment, the present invention further includes a computer program product, which includes a computer program that, when executed by a processor, implements the business processing method of any embodiment of the present invention.

[0113] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0114] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0115] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A business processing method, characterized in that, include: Based on the received business processing request, determine the business processing model that needs to be invoked, and parse the business processing request to obtain business data; Based on the business data and the business processing model, the business processing result is determined; Based on the business processing result, determine the business response message for the business processing request; The business processing model is obtained by quantifying the network nodes in the original processing model based on preset standard data.

2. The method according to claim 1, characterized in that, The business processing model is obtained by quantifying the network nodes in the original processing model based on preset standard data, including: Identify at least one candidate quantization node for the original processing model; Based on the preset standard data, the node feature vector of the candidate quantized node is determined; The quantization adaptability results of the candidate quantization nodes are obtained by analyzing the node feature vectors through a pre-trained node judgment model. Based on the quantization adaptability results, a target quantization node is determined from the candidate quantization nodes; The original processing model is quantized based on the target quantization node to obtain the business processing model.

3. The method according to claim 2, characterized in that, The step of determining the node feature vector of the candidate quantized node based on the preset standard data includes: The standard data is input into the original processing model for high-precision inference to obtain a high-precision output tensor. Determine the numerical distribution characteristics of the high-precision output tensor, and determine the node structure characteristics of the candidate quantization nodes; The node feature vector is obtained by combining the numerical distribution features and the node structure features.

4. The method according to claim 2, characterized in that, The training method for the node determination model includes: After obtaining the reference processing model, at least one reference node of the reference processing model is determined; Based on the preset standard data, the reference output tensor of the reference node is determined, and the corresponding reference node feature vector is extracted. Simulated quantization is performed on the reference node based on the feature vector of the reference node to obtain the simulated output tensor; Calculate the error index between the simulated output tensor and the reference output tensor, and determine the quantization fitness label of the reference node based on the error index; The reference node feature vector and the quantized adaptive label are used as training samples to train a preset learning model, thereby obtaining the node judgment model.

5. The method according to claim 4, characterized in that, The step of performing simulated quantization on the reference node based on the reference node feature vector to obtain a simulated output tensor includes: The quantization scaling factor and zero point are determined based on the feature vector of the reference node. Based on the quantization scaling factor and zero point, a pseudo-quantization operation is performed on the reference output tensor to obtain the analog output tensor.

6. The method according to claim 4, characterized in that, The step of calculating the error index between the simulated output tensor and the reference output tensor, and determining the quantization fitness label of the reference node based on the error index, includes: Calculate at least two error metrics between the simulated output tensor and the reference output tensor; Each error index is normalized, and the normalized error indices are weighted and summed according to preset weighting coefficients to obtain a comprehensive error result. The comprehensive error result is compared with a preset threshold, and the quantization adaptive label is determined based on the comparison result.

7. The method according to claim 2, characterized in that, The method further includes: Obtain the accuracy feedback data of the business processing model; The node judgment model is updated based on the accuracy feedback data.

8. A business processing apparatus, characterized in that, include: The acquisition module is used to determine the business processing model to be invoked based on the received business processing request, and to parse the business processing request to obtain business data; The determination module is used to determine the business processing result based on the business data and the business processing model; The response module is used to determine the business response message for the business processing request based on the business processing result. The business processing model is obtained by quantifying the network nodes in the original processing model based on preset standard data.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the business processing method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the business processing method according to any one of claims 1-7.

Images