Data processing method, text data processing method, computing device and storage medium

Abstract

Disclosed in the embodiments of the present disclosure are a data processing method, a text data processing method, a computing device and a storage medium. The data processing method comprises: determining data to be processed and a data processing model for processing the data to be processed; acquiring from a data storage unit compressed weight data corresponding to the data processing model and decompression data corresponding to the compressed weight data; using the decompression data to perform data decompression processing on the compressed weight data, so as to obtain decompressed weight data; and on the basis of the decompressed weight data, using the data processing model to perform data processing on the data to be processed, so as to obtain a data processing result. Therefore, the data processing efficiency of a data processing model is improved, a data processing result is quickly obtained, and the data efficient processing requirement of a user is thus met.

Description

[0001] The present disclosure claims priority to Chinese Patent Application No. 202410702795.4, filed on May 31, 2024, with the Chinese Patent Office, entitled "Data processing method, text data processing method, computing device, and storage medium", the entire contents of which are incorporated herein by reference. TECHNICAL FIELD The embodiments of the present disclosure relate to the technical field of computer technology, and in particular to a data processing method. One or more embodiments of the present disclosure also relate to a text data processing method, another data processing method, a data processing apparatus, a text data processing apparatus, another data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product. BACKGROUND TECHNIQUE With the continuous development of computer technology and artificial intelligence technology, in the data processing scene, a neural network model can be used to perform data processing operations on to-be-processed data, thereby obtaining a corresponding data processing result. However, in the process of using the neural network model for data processing, the neural network model has the problem of low data processing efficiency, which leads to the inability to quickly obtain the data processing result, and thus cannot meet the user's data efficient processing demand. Therefore, how to improve the data processing efficiency of the neural network model has become a problem to be solved. SUMMARY In view of the above, the embodiments of the present disclosure provide a data processing method. One or more embodiments of the present disclosure also relate to a text data processing method, another data processing method, a data processing apparatus, a text data processing apparatus, another data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, to solve the technical defects of low data processing efficiency of the neural network model in the prior art. According to a first aspect of the embodiments of the present disclosure, a data processing method is provided, including: determining to-be-processed data and a data processing model for processing the to-be-processed data; obtaining, from a data storage unit, compressed weight data corresponding to the data processing model and decompression data corresponding to the compressed weight data; performing data decompression processing on the compressed weight data using the decompression data to obtain decompressed weight data; and performing data processing on the to-be-processed data using the data processing model according to the decompressed weight data to obtain a data processing result.According to a second aspect of the embodiments of the present disclosure, a data processing apparatus is provided, including: a first data determining module configured to determine to-be-processed data and a data processing model for processing the to-be-processed data; a second data determining module configured to acquire, from a data storage unit, compressed weight data corresponding to the data processing model and decompression data corresponding to the compressed weight data; a data decompression module configured to perform data decompression processing on the compressed weight data by using the decompression data to obtain decompressed weight data; and a data processing module configured to perform data processing on the to-be-processed data by using the data processing model according to the decompressed weight data to obtain a data processing result. According to a third aspect of the embodiments of the present disclosure, a text data processing method is provided, including: determining to-be-processed question text and a data processing model for processing the to-be-processed question text; acquiring, from a data storage unit, compressed weight data corresponding to the data processing model and decompression data corresponding to the compressed weight data; performing data decompression processing on the compressed weight data by using the decompression data to obtain decompressed weight data; and performing data processing on the to-be-processed question text by using the data processing model according to the decompressed weight data to obtain an answer text. According to a fourth aspect of the embodiments of the present disclosure, a text data processing apparatus is provided, including: a first data determining module configured to determine to-be-processed question text and a data processing model for processing the to-be-processed question text; a second data determining module configured to acquire, from a data storage unit, compressed weight data corresponding to the data processing model and decompression data corresponding to the compressed weight data; a data decompression module configured to perform data decompression processing on the compressed weight data by using the decompression data to obtain decompressed weight data; and a data processing module configured to perform data processing on the to-be-processed question text by using the data processing model according to the decompressed weight data to obtain an answer text.According to a fifth aspect of the embodiments of the present disclosure, a data processing method is provided, including: receiving a weight obtaining request sent by a data processing unit, wherein the weight obtaining request is sent by the data processing unit according to a data processing model used for processing to-be-processed data, and the data processing model is a model used for processing to-be-processed data; determining compressed weight data corresponding to the data processing model and decompression data corresponding to the compressed weight data; and sending the compressed weight data and the decompression data corresponding to the compressed weight data to the data processing unit, so that the data processing unit obtains decompressed weight data by using the decompression data, and performs data processing on the to-be-processed data by using the data processing model according to the decompressed weight data, to obtain a data processing result. According to a sixth aspect of the embodiments of the present disclosure, a data processing apparatus is provided, including: a request receiving module configured to receive a weight obtaining request sent by a data processing unit, wherein the weight obtaining request is sent by the data processing unit according to a data processing model used for processing to-be-processed data, and the data processing model is a model used for processing to-be-processed data; a data determining module configured to determine compressed weight data corresponding to the data processing model and decompression data corresponding to the compressed weight data; and a data sending module configured to send the compressed weight data and the decompression data corresponding to the compressed weight data to the data processing unit, so that the data processing unit obtains decompressed weight data by using the decompression data, and performs data processing on the to-be-processed data by using the data processing model according to the decompressed weight data, to obtain a data processing result. According to a seventh aspect of the embodiments of the present disclosure, a computing device is provided, including: a memory and a processor; the memory is used for storing computer programs / instructions, and the processor is used for executing the computer programs / instructions, which realize steps of the above data processing method, a text data processing method, or another data processing method when executed by the processor. According to an eighth aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores computer programs / instructions, which realize steps of the above data processing method, a text data processing method, or another data processing method when executed by the processor. According to a ninth aspect of the embodiments of the present disclosure, a computer program product is provided, including computer programs / instructions, which realize steps of the above data processing method, a text data processing method, or another data processing method when executed by the processor.The data processing method in one or more embodiments of the present disclosure can obtain the compressed weight data corresponding to the data processing model and the decompression data corresponding to the compressed weight data from the data storage unit after determining the to-be-processed data and the data processing model for processing the to-be-processed data. Since the data quantity of the compressed weight data is smaller than that of the uncompressed weight data, the compressed weight data and the decompression data with small data quantity can be obtained from the data storage unit in a small data transmission time, thereby saving the data transmission time of the weight data in the data processing process, and then the to-be-processed data can be quickly processed by using the data processing model according to the weight data, so as to obtain the data processing result, thereby improving the data processing efficiency of the data processing model, realizing the quick obtaining of the data processing result, and meeting the user's data efficient processing demand. BRIEF DESCRIPTION OF DRAWINGS FIG. 1 is an application schematic diagram of a data processing method according to an embodiment of the present disclosure; FIG. 2 is a flowchart of a data processing method according to an embodiment of the present disclosure; FIG. 3 is a process flowchart of a data processing method according to an embodiment of the present disclosure; FIG. 4 is a structural schematic diagram of a Huffman tree according to an embodiment of the present disclosure; FIG. 5 is a flowchart of a text data processing method according to an embodiment of the present disclosure; FIG. 6 is a flowchart of another data processing method according to an embodiment of the present disclosure; FIG. 7 is a structural schematic diagram of a data processing apparatus according to an embodiment of the present disclosure; FIG. 8 is a structural schematic diagram of a text data processing apparatus according to an embodiment of the present disclosure; FIG. 9 is a structural schematic diagram of another data processing apparatus according to an embodiment of the present disclosure; and FIG. 10 is a structural block diagram of a computing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, the present disclosure can be practiced without the specific details. Those skilled in the art, in light of the disclosure, can implement modifications and changes without departing from the scope of the present disclosure. Therefore, the present disclosure is not limited to the specific implementations disclosed below. The terms used in one or more embodiments of the present disclosure are merely for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present disclosure. In one or more embodiments of the present disclosure and the appended claims, the singular forms "a", "said" and "the" are intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present disclosure means and includes any or all possible combinations of one or more associated listed items. It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present disclosure, these information should not be limited to these terms.These terms are used only to distinguish one type of information from another type of information. For example, the first can also be referred to as the second without departing from the scope of the embodiments of the present disclosure, and similarly, the second can also be referred to as the first. Depending on the context, the word "if' as used herein can be interpreted to mean "when" or "in response to determining" or "in response to ascertaining." In addition, it needs to be explained that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the embodiments of the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation interfaces for the user to choose authorization or refusal. In the embodiments of the present disclosure, a large model refers to a deep learning model with a large number of model parameters, usually containing hundreds of millions, tens of billions, hundreds of billions, thousands of billions or even tens of billions of model parameters. The large model can also be called a foundation model, which is pre-trained through a large amount of unlabeled corpus to output a pre-training model with hundreds of millions of parameters. Such a model can adapt to a wide range of downstream tasks, and the model has good generalization ability, such as a large language model (LLM) and a multi-modal pre-training model. In actual application, the pre-training model only needs to be fine-tuned with a small amount of samples to be applied to different tasks. The large model can be widely used in natural language processing (NLP) and computer vision fields, and can be applied to visual question answering (VQA), image captioning (IC), image generation, sentiment classification based on text, text summary generation, machine translation and other natural language processing tasks. The main application scenarios of the large model include digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc. First, the noun terms involved in the embodiments of the present disclosure are explained.

[0002] LLM: Large Language Model, a deep learning algorithm that can perform various natural language processing tasks.

[0003] Per-tensor quantization: A quantization operation applied uniformly to an entire tensor (such as weight parameters or activation values), meaning all elements use the same scaling factor and quantization parameter. In Per-tensor quantization, all elements share the same quantization range and precision, which is suitable for cases where elements in different channels or positions have little difference.

[0004] Per-channel quantization: A quantization operation applied separately to each channel (such as input channels or output channels of a convolutional layer) in a tensor, meaning each channel has an independent scaling factor and quantization parameter. In Per-channel quantization, elements in different channels can have different quantization ranges and precisions, which is suitable for cases where elements in different channels have significant differences.

[0005] INT8 (8-bit integer): A low-bit quantization precision that uses 8-bit integers to represent values or information, providing lower storage requirements and computational complexity. INT8 is suitable for the inference stage of deep learning models and reduces the demand for computational resources to some extent.

[0006] INT4 (4-bit integer): refers to a low-bit quantization precision represented by a 4-bit integer, and INT4 provides lower storage requirements and computational complexity. token: Chinese name for token, character, etc., which is usually used to represent the smallest unit in text or sequence data. With the continuous development of computer technology and artificial intelligence technology, in the data processing scene, the neural network model can be used to realize the data processing operation on the to-be-processed data, so as to obtain the corresponding data processing result. For example, LLM has shown excellent effect in many fields, and is a high-potential path to general artificial intelligence (AGI). However, at the same time, LLM faces great challenges in inference deployment. On the one hand, the parameter quantity of LLM is large, which needs to occupy a large memory space; on the other hand, the calculation quantity and memory access quantity of LLM are also very amazing, which needs to consume a lot of computing power and memory bandwidth resources; and because LLM belongs to the autoregressive model, that is, the output token is generated one by one through loop iteration in the inference process; this calculation load characteristic determines that the inference process of LLM is limited by the memory bandwidth; among them, the memory bandwidth mainly consists of two parts, namely the activation memory bandwidth and the weight memory bandwidth, the activation memory bandwidth can be the access bandwidth of reading the activation value in the memory, and the weight memory bandwidth can refer to the access bandwidth of reading the weight parameter in the memory; and the activation memory bandwidth accounts for a very small proportion compared with the weight memory bandwidth; therefore, if the weight can be effectively compressed, on the one hand, the memory occupation can be reduced, and on the other hand, the weight memory bandwidth can be reduced, thereby improving the LLM inference performance. In view of the above problems, a scheme provided by the present disclosure is to quantize the weight data to INT8 or INT4 in a low-bit quantization manner, and then perform LLM inference based on the low-bit quantized weight. However, this scheme has a big defect, specifically, this scheme does not fully utilize the sparsification characteristics of LLM weight data, that is, the low-bit weight data after quantization is not uniformly distributed in the low-bit representation range, but there are a large number of same values or a large number of empty values; this actually wastes the representation range of low-bit quantization formats such as INT8 or INT4, and does not realize more efficient and sufficient compression. Based on this, in the present disclosure, a data processing method is provided, and one or more embodiments of the present disclosure also relate to a text data processing method, another data processing method, a data processing device, a text data processing device, another data processing device, a computing device, a computer readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.Referring to FIG. 1, FIG. 1 shows an application diagram of a data processing method according to one embodiment of the present disclosure. Based on FIG. 1, a user sends a question text to a server 104 through a terminal 102. The server 104 performs language inference based on a data processing model. In the process of language inference, the compressed weight data corresponding to the data processing model and the Huffman coding table are obtained from the video memory. The compressed weight data is decompressed based on the Huffman coding table to obtain decompressed weight data. The data processing model processes the question text based on the weight data to obtain an answer text. The server 104 sends the answer text to the terminal 102, so that the terminal 102 shows the answer text to the user. Thus, the LLM inference performance is improved. Referring to FIG. 2, FIG. 2 shows a flowchart of a data processing method according to one embodiment of the present disclosure. The method includes the following steps. Step 202: determining to-be-processed data and a data processing model for processing the to-be-processed data. The to-be-processed data can be understood as data that needs to be processed by the data processing model. The to-be-processed data can be any one of text data, document data, image data, voice data, video data, point cloud data, or code data, or a combination of multiple data. The data processing model can be understood as a model that can process input to-be-processed data and output a data processing result corresponding to the to-be-processed data. In different scenarios, the data processing model is different. In the text data processing scenario, the data processing model can be a data processing model for processing to-be-processed question text. The to-be-processed question text can be a question in text form that needs to be answered by the data processing model. The to-be-processed question text is processed by the data processing model to obtain an answer text corresponding to the to-be-processed question text. The answer text can be understood as an answer to the to-be-processed question text. For example, the to-be-processed question text can be “Labor Day is on which day in which month”. Based on this, the to-be-processed question text is input into the data processing model, and the to-be-processed question text is processed based on the data processing model and the decompressed weight data to obtain the answer text “Labor Day is on May 1st every year”. In this way, the user's question is answered in the actual scenario, and the need to answer the user's question by using the model in the actual scenario is met.In the case where the data processing method provided in one or more embodiments of the present disclosure is applied in a picture-text question and answer scenario, the data processing model can be a picture-text question and answer model, the to-be-processed data can be picture-text question and answer data containing an image and text, and the data processing result can be an answer text for the picture-text question and answer data. For example, the picture-text question and answer data can be picture-text question and answer data containing an image of an office table and a text question "What is placed on the office table in the image?", based on which the picture-text question and answer data can be input into the picture-text question and answer model for question and answer processing, and an answer text "A computer, a mouse, a keyboard, and a water cup are placed on the office table" can be output. Thus, the data processing method can be used to answer a picture-text question raised by a user by using a data processing model, so as to reply to a picture-text question raised by a user in an actual scenario, and the demand for picture-text question and answer by using a model in an actual scenario can be met. In the case where the data processing method provided in one or more embodiments of the present disclosure is applied in a user communication scenario, the data processing model can be a text communication model, and correspondingly, the to-be-processed data can be communication text, and the data processing result can be reply data (for example, a reply text or a reply audio) corresponding to the communication text. For example, the communication text can be communication text data "Is the weather good today?", based on which the communication text can be input into the text communication model for communication processing, and a reply text "The weather is very good today" or a reply audio can be output. Thus, the data processing method can be used to communicate with a user in an actual scenario, and the demand for communication with a user by using a model in an actual scenario can be met. In the case where the data processing method provided in one or more embodiments of the present disclosure is applied in a speech recognition scenario, the data processing model can be a speech recognition model, and correspondingly, the to-be-processed data can be to-be-recognized speech data, and the data processing result can be a recognition result (for example, speech text) corresponding to the to-be-recognized speech data. For example, the to-be-recognized speech data can be a recording, based on which the recording can be input into the speech recognition model for recognition processing, and speech text data corresponding to the recording can be output, and the demand for speech recognition by using a model in an actual scenario can be met. It should be noted that the data processing model can be a large model, for example, the data processing model can be an LLM model. The data processing model can run in a data processing node, and the data processing node can be a CPU, a GPU, or a worker node. Correspondingly, the data storage unit can be a memory accessible by a CPU, a video memory accessible by a GPU, or a parameter server accessible by a worker node.In one or more embodiments of the present disclosure, before obtaining the compressed weight data corresponding to the data processing model and the decompression data corresponding to the compressed weight data from the data storage unit, the method further comprises: determining the weight data corresponding to the data processing model, wherein the weight data is obtained by model training of the data processing model; performing data compression on the weight data to obtain the compressed weight data and the decompression data corresponding to the compressed weight data; and storing the compressed weight data and the decompression data into the data storage unit. The weight data can be understood as parameters required by the data processing model in the process of processing the to-be-processed data. For example, the weight data can be parameters in a neural network. These weight parameters are used to adjust and learn the behavior of the model so as to effectively map the input data and extract useful features. The weight parameters can be represented in the form of a matrix or a vector. The compressed weight data can be understood as compressed data obtained by compressing the weight data. The compressed weight data can be represented in an encoded form, for example, the compressed weight data can be weight encoding. The data quantity of the compressed weight data is less than that of the weight data. The compression of the weight data can be understood as compression of the weight data by a two-stage data compression method to obtain compressed data and decompression data. The decompression data can be understood as data used for decompression processing of the compressed weight data. The decompression data is generated in the process of compressing the weight data, for example, the decompression data can be a decompression key, a data decompression encoding table, a Huffman encoding table, etc. Specifically, in order to avoid the problem of low data processing efficiency of the neural network model, the data processing method provided by the present disclosure determines the weight data corresponding to the data processing model after the training of the data processing model is completed, performs data compression on the weight data to obtain the compressed weight data and the decompression data corresponding to the compressed weight data, and stores the compressed weight data and the decompression data into the data storage unit. This facilitates subsequent processing of to-be-processed data by the data processing model, that is, the operation of data compression on the weight data is completed offline in advance, thereby not occupying inference and not affecting the inference performance of the model.In addition, it needs to be explained that, since the data quantity of the compressed weight data is less than the data quantity of the uncompressed weight data, the compressed weight data with a smaller data quantity can be obtained from the data storage unit in a smaller data transmission time, and the decompressed data is obtained, so as to save the weight data transmission time in the data processing process, and then the data processing model can be used to quickly process the to-be-processed data according to the weight data, to obtain a data processing result, thereby improving the data processing efficiency of the data processing model. It needs to be explained that the data processing method provided by the present disclosure can be applied to a data processing node, and in one or more embodiments provided by the present disclosure, the step of compressing the weight data to obtain the compressed weight data and the decompressed data corresponding to the compressed weight data can be executed by the data processing node, so as to realize the data compression of the weight data. The data storage unit can be a storage unit deployed in the data processing node and used for storing the compressed weight data and the decompressed data corresponding to the compressed weight data. The data processing node can be understood as a node for processing the to-be-processed data, for example, the data processing node can be a client, a server, a virtual machine, etc. The data storage unit can be a memory, a video memory, a local disk, etc. Alternatively, the data storage unit can be a database connected to the data processing node and used for storing the compressed weight data and the decompressed data corresponding to the compressed weight data. In one or more embodiments provided by the present disclosure, the step of compressing the weight data to obtain the compressed weight data and the decompressed data corresponding to the compressed weight data can be executed by a data compression node other than the data processing node, so as to realize the data compression of the weight data by the data compression node. The data compression node can be a node for compressing the weight data, which can be a client, a server, a virtual machine, etc., without specific limitation here. After the data compression node completes the weight data compression, the compressed weight data and the decompressed data corresponding to the compressed weight data can be sent to the data processing node. In one or more embodiments provided by the present disclosure, the step of compressing the weight data to obtain the compressed weight data and the decompressed data corresponding to the compressed weight data includes steps one and two: step one: performing first-stage data compression on the weight data to obtain initial compressed weight data.The initial compressed weight data can be understood as compressed data obtained after the first-stage data compression operation. In the case of different compression modes in the first-stage data compression operation, the initial compressed weight data can also be different. For example, the initial compressed weight data can be low-bit weight data in the int8 form or low-bit weight data in the int4 form. In one or more embodiments provided in the present disclosure, the first-stage data compression of the weight data to obtain the initial compressed weight data includes: performing the first-stage data compression by pruning the weight data to obtain the initial compressed weight data. The pruning process can be understood as removing weights that contribute less to the model output. These weights can be close to zero, so they can be set to zero or directly deleted. In one or more embodiments provided in the present disclosure, the first-stage data compression of the weight data to obtain the initial compressed weight data includes: performing the first-stage data compression by sparse representation of the weight data to obtain the initial compressed weight data. The sparse representation process can be understood as storing only non-zero weight parameters. By using a sparse matrix storage format to store the weight data, the storage requirement is reduced. In one or more embodiments provided in the present disclosure, the first-stage data compression can be understood as lossy compression. In one or more embodiments provided in the present disclosure, the first-stage data compression of the weight data to obtain the initial compressed weight data includes: determining a data range corresponding to the weight data, and determining a quantization parameter corresponding to the weight data based on the data range; performing low-bit quantization on the weight data based on the quantization parameter and a preset quantization strategy to obtain low-bit weight data, and taking the low-bit weight data as the initial compressed weight data. The data range can be understood as a numerical range corresponding to the weight data. It should be noted that the weight data can be floating-point data. The quantization parameter can be understood as a parameter used for quantization of the weight data. The quantization parameter can be the maximum value (i.e., the maximum weight parameter), the minimum value (i.e., the minimum weight parameter), and / or the intermediate value (i.e., the intermediate value of the weight data) in the weight data. The low-bit quantization can be understood as a quantization operation for representing the weight data (which occupies a large storage space) by using a small number of binary bits (i.e., low bits), for example, converting the floating-point weight data into INT8 or INT4 for representation.The low-bit weight data can be understood as weight data represented by a small number of binary bits (i.e., INT8 or INT4). The preset quantization strategy can be understood as a strategy for low-bit quantization of the weight data, and the preset quantization strategy can be set according to an actual application scenario. In one or more embodiments provided in the present disclosure, the preset quantization strategy can be a per-tensor quantization strategy or a per-channel quantization strategy. In the scenario of improving the running efficiency of the LLM model by using the data processing method provided in the present disclosure, the data processing model is the LLM model, and the LLM model runs on the GPU. After the GPU running the LLM model receives the problem text that needs to be processed by the LLM model, the GPU needs to obtain the compressed weight data corresponding to the LLM model and the decompression data corresponding to the compressed weight data based on the LLM. Before the GPU obtains the compressed weight data and the decompression data, the weight data needs to be compressed in two stages. The first-stage data compression can be a low-bit quantization processing stage, which specifically includes the following modes: 1. determining the trained LLM weight data; 2. counting the weight value range (i.e., the data range); specifically, in the process of weight data compression, the value range of the floating-point weight data needs to be counted to determine the maximum value and the minimum value in the floating-point weight data; 3. determining the quantization parameter; specifically, the maximum value and the minimum value in the floating-point weight data are used as the quantization parameter; 4. quantizing the weight data; specifically, after the quantization parameter required for quantization is determined, the weight parameter can be low-bit quantized by using the quantization parameter and the quantization strategy to obtain a low-bit representation of the weight parameter. The low-bit representation can be in the INT8 format or the INT4 format; wherein, the quantization strategy can be selected as the per-tensor quantization strategy or the per-channel quantization strategy, and the low-bit quantization precision can be selected as INT8 or INT4; then, the weight is quantized according to the quantization parameter and the quantization strategy to obtain the low-bit representation (INT8 or INT4) format weight. In the above embodiment, in the process of first-stage data compression of the weight data to obtain the initial compressed weight data, the weight data can be first-stage data compressed by using the low-bit quantization mode, so as to obtain the low-bit representation of the initial compressed weight data, so that the second-stage data compression can be efficiently completed based on the low-bit weight data, and the efficiency of data compression is improved.Step two: performing second-stage data compression on the initial compressed weight data to obtain the compressed weight data and the decompression data corresponding to the compressed weight data. The second-stage data compression can be understood as a second data compression process on the weight data, and can be implemented by using a Huffman coding mode. That is, performing second-stage data compression on the initial compressed weight data can be understood as performing data compression on the initial compressed weight data by using the Huffman coding mode. In one or more embodiments provided by the present disclosure, the second-stage data compression on the initial compressed weight data to obtain the compressed weight data and the decompression data corresponding to the compressed weight data includes: performing second-stage data compression by performing run-length coding processing on the initial compressed weight data to obtain the compressed weight data and the decompression data corresponding to the compressed weight data. The run-length coding processing can be understood as storing only the weight value and the number of repetitions, instead of each individual value, for the weight value (i.e., the weight parameter) that is continuously repeated in the weight data. In one or more embodiments provided by the present disclosure, the second-stage data compression can be lossless compression. In one or more embodiments provided by the present disclosure, the second-stage data compression on the initial compressed weight data to obtain the compressed weight data and the decompression data corresponding to the compressed weight data includes: determining a plurality of weight parameters included in the initial compressed weight data, and constructing a weight parameter sequence table based on the plurality of weight parameters; determining a leaf node based on the weight parameter in the weight parameter sequence table, and constructing a data compression tree structure based on the leaf node; performing data compression on the initial compressed weight data based on the data compression tree structure to obtain the compressed weight data, and determining the weight parameter sequence table as the decompression data corresponding to the compressed weight data. It should be noted that the weight data corresponding to the data processing model can be a matrix or a vector, and the matrix or the vector includes a plurality of weight parameters, for example, the matrix elements of the matrix can be weight parameters, or the vector elements of a group of vectors can be weight parameters. Correspondingly, the plurality of weight parameters included in the initial compressed weight data can be understood as the plurality of compressed weight parameters included in the initial compressed weight data.The weight parameter sequence list can be understood as a sequence list composed of multiple weight parameters arranged in order, for example, a sequence list composed of multiple weight parameters arranged in order from large to small, or a sequence list composed of multiple weight parameters arranged in order from small to large; the leaf node can be understood as a leaf node constituting a data compression tree structure, which is constituted according to multiple leaf nodes and a root node; the data compression tree structure can be understood as a tree structure for data compression of weight data, for example, the data compression tree structure can be a binary tree, a Huffman tree, etc.; the weight parameter sequence list is determined as the decompression data corresponding to the compressed weight data can be understood as determining the weight parameter sequence list as the decompression data corresponding to the compressed weight data. Specifically, in one or more embodiments provided by the present disclosure, the manner of performing second-stage data compression on the initial compressed weight data to obtain the compressed weight data and the decompression data corresponding to the compressed weight data can be: first, determining multiple weight parameters contained in the initial compressed weight data, and sorting the multiple weight parameters to obtain multiple sorted weight parameters, then constructing a weight parameter sequence list based on the multiple sorted weight parameters; second, determining the weight parameters in the weight parameter sequence list as leaf nodes, and determining a root node based on the parameter quantity of the weight parameters in the weight parameter sequence list, and constructing a data compression tree structure based on the leaf nodes and the root node; finally, performing data compression on the initial compressed weight data based on the data compression tree structure to obtain the compressed weight data, and taking the weight parameter sequence list as the decompression data corresponding to the compressed weight data. In the above embodiment, the weight parameter sequence list is constructed based on the multiple weight parameters contained in the initial compressed weight data, and the data compression tree structure is constructed based on the weight parameters, and the initial compressed weight data is compressed based on the data compression tree structure to obtain the compressed weight data, and finally the weight parameter sequence list is taken as the decompression data corresponding to the compressed weight data; thereby realizing the second-stage data compression of the weight data, obtaining the compressed weight data and the decompression data, facilitating subsequent improvement of the processing efficiency of the data processing model based on the compressed weight data and the decompression data. In one or more embodiments provided by the present disclosure, the determination of the multiple weight parameters contained in the initial compressed weight data and the construction of the weight parameter sequence list based on the multiple weight parameters include: determining the multiple weight parameters contained in the initial compressed weight data and the parameter quantity of the multiple weight parameters; sorting each weight parameter based on the parameter quantity to obtain a weight parameter sequence, and constructing the weight parameter sequence list based on the weight parameter sequence.The parameter quantity can be understood as the number of weight parameters. In one or more embodiments provided by the present disclosure, multiple same weight parameters can exist in the plurality of weight parameters, for example, 10 weight parameters are all 255. In this case, the frequency of occurrence (for example, 10 times) of the weight parameter is the parameter quantity of the weight parameter. In the above example, after the weight quantization process is completed and low-bit weight data is obtained, Huffman coding compression is performed. The basic principle of Huffman coding is that a value with a higher frequency of occurrence in the weight data is represented by a shorter bit, and a value with a lower frequency of occurrence is represented by a longer bit, which can reduce the average length after coding, thereby realizing lossless data compression. Therefore, when performing Huffman coding compression, frequency statistics of weight values are required. For convenience, the INT8 format of the low-bit representation of the weight parameter is analyzed and represented as UI NT8, so the representation range is 0-255. The frequency of occurrence (i.e., the parameter quantity) of each value of 0-255 in the weight data is counted, and a frequency table (i.e., a weight parameter sequence table) is generated according to the frequency size. The frequency table is shown in Table 1 below. By constructing the frequency table, subsequent second-stage data compression can efficiently perform data compression, thereby improving the efficiency of data compression. In one or more embodiments provided by the present disclosure, the leaf node is determined based on the weight parameter in the weight parameter sequence table, and the data compression tree structure is constructed based on the leaf node, including: determining a target weight parameter in the weight parameter sequence table and an associated parameter quantity corresponding to the target weight parameter, wherein the associated parameter quantity is determined by the parameter quantities of at least two initial weight parameters, the target weight parameter is any one of the plurality of weight parameters in the weight parameter sequence table except the at least two initial weight parameters, and the at least two initial weight parameters are weight parameters smaller than the target weight parameter in the weight parameter sequence table; determining a total parameter quantity based on the parameter quantities of the weight parameters in the weight parameter sequence table; determining the total parameter quantity as a root node and the weight parameters and the associated parameter quantity as leaf nodes; and constructing the data compression tree structure by using the size relationship between the parameter quantities of the weight parameters and the associated parameter quantity.Wherein, the at least two initial weight parameters can be understood as the two smallest weight parameters in the weight parameter sequence list, the target weight parameter can be understood as a weight parameter greater than the at least two initial weight parameters; the associated parameter quantity can be understood as the parameter quantity corresponding to each initial weight parameter; and the total parameter quantity can be understood as the sum of the parameter quantities of each weight parameter in the weight parameter sequence list. Following the above example, after the frequency table is constructed, the table 1 can be used to establish a Huffman tree, which can be seen from Fig. 4. The specific execution mode of constructing the Huffman tree is as follows.

[0007] 1. Take out the two U I NT8 values with the smallest frequencies from the frequency table, i.e. 210 and 113, which are terminal nodes (leaf nodes), i.e. the dashed line nodes in Fig. 3 (i.e. the nodes drawn with dashed lines), and their corresponding frequencies are added, i.e. 2+3=5, which is an intermediate node, i.e. the solid line node in Fig. 3 (i.e. the node drawn with a solid line).

[0008] 2. Continue to take out the U I NT8 value with the smallest frequency from the remaining values in the frequency table, i.e. 88, which is a terminal node, and its frequency 4 is less than that of the intermediate node 5, so the terminal node 88 is placed on the left side of the intermediate node 5. The frequency 4 corresponding to the terminal node 88 is added to the frequency 5 of the intermediate node to obtain an intermediate node 9 O

[0009] 3. Continue to take out the U I NT8 value with the smallest frequency from the remaining values in the frequency table, i.e. 22, which is a terminal node, and its frequency 5 is less than that of the intermediate node 9, so the terminal node 22 is placed on the left side of the intermediate node 9. The frequency 5 corresponding to the terminal node 22 is added to the frequency 9 of the intermediate node to obtain an intermediate node 14 o

[0010] 4. Continue to take out the U I NT8 value with the smallest frequency from the remaining values in the frequency table, i.e. 255, which is a terminal node, and its frequency 15 is greater than that of the intermediate node 14, so the terminal node 255 is placed on the right side of the intermediate node 14. The frequency 15 corresponding to the terminal node 255 is added to the frequency 14 of the intermediate node to obtain an intermediate node 29 o

[0011] 5、 Finally, the value 0 is taken out from the frequency table as a terminal node, and the frequency 20 of the terminal node 0 is less than the intermediate node 29, so the terminal node 0 is placed on the left side of the intermediate node 29. The frequency 20 corresponding to the terminal node 0 is added to the intermediate node 29 to obtain an intermediate node 49. Based on the above steps, it can be known that the manner of constructing the Huffman tree can be that two terminal nodes are determined as the smallest two weight parameters in the weight parameter sequence list; based on the number of parameters corresponding to the two terminal nodes, an intermediate node corresponding to the two terminal nodes is determined; based on the intermediate node and the two terminal nodes, a node branch is formed; then, the weight parameter greater than the two terminal nodes is taken out from the weight parameter sequence list, the weight parameter and the intermediate node in the node branch are taken as terminal nodes again, and the step of “based on the number of parameters corresponding to the two terminal nodes, an intermediate node corresponding to the two terminal nodes is determined” is repeatedly executed until all weight parameters in the weight parameter sequence list are taken as a node in the node branch; after the plurality of node branches are constructed through all weight parameters, the plurality of node branches are used to construct the Huffman tree. Through the above manner, the construction of the Huffman tree is completed, which is convenient for subsequent lossless compression of weight data based on the Huffman tree, so as to ensure the accuracy of the weight data and facilitate subsequent improvement of the processing efficiency of the data processing model based on the compressed weight data. In one or more embodiments of the present disclosure, the data compression based on the data compression tree structure and the initial compressed weight data to obtain the compressed weight data comprises: determining a plurality of node branches in the data compression tree structure, wherein the node branch is formed by arranging two leaf nodes on the left and right sides of an intermediate node according to the size relationship of the number of parameters corresponding to the two leaf nodes; encoding the left leaf node in each node branch as first bit data, and encoding the right leaf node in each node branch as second bit data; traversing a target leaf node in the data compression tree structure to determine a traversal path node sequence corresponding to the target leaf node, wherein the target leaf node is any one leaf node in the data compression tree structure; determining a weight compression code corresponding to the target leaf node according to the first bit data of the left leaf node and the second bit data of the right leaf node in the traversal path node sequence, and determining the weight compression code as the compressed weight data.The node branch can be understood as a plurality of branch structures constituting the data compression tree structure, which can be sorted in a top-down manner, and each node branch can be a layer of the data compression tree structure. The node branch can be composed of an intermediate node and leaf nodes distributed on the left and right sides of the intermediate node. The intermediate node in the node branch can be a leaf node of the node branch of the previous layer, and the two leaf nodes in the node branch can be intermediate nodes of the node branch of the next layer. For details, refer to FIG. 4. The first bit data can be understood as binary bit data identifying the left leaf node. For example, the encoding of the left leaf node on the left side of the branch can be 0. The second bit data can be understood as binary bit data identifying the right leaf node. For example, the encoding of the left leaf node on the right side of the branch can be 1. In the above example, after the Huffman construction is completed, the weight data can be encoded and compressed based on the Huffman. The specific method is as follows: all left branches in the Huffman tree are encoded as 0, and right branches are encoded as 1: thus, traversing the Huffman binary tree from the top node (i.e., the 49 node) can obtain the encoding of all terminal nodes. For example, the 0 terminal node corresponds to the encoding 0, the 255 terminal node corresponds to the encoding 1 1, the 22 terminal node corresponds to the encoding 100, the 210 terminal node corresponds to the encoding 101 10, and so on. After encoding, the encoding data (i.e., compressed weight data) corresponding to the weight data is stored in the Huffman encoding table, which can be seen in Table 2 below. After constructing the encoding table, the original weight value sequence can be encoded; for example, the sequence 0, 255, 22, 88, 113, 210 becomes 01 1 1001010101 1 1 101 10 (i.e., compressed weight data) after encoding. Based on this, it can be known that the weight data occupies 48 bits of space before encoding, and 20 bits of space after encoding. In the above example, after the Huffman tree is constructed, the weight data is encoded based on the Huffman tree to realize lossless compression of the weight data, thereby ensuring the accuracy of the weight data while facilitating subsequent improvement of the processing efficiency of the data processing model based on the compressed weight data. Step 204: obtaining the compressed weight data corresponding to the data processing model and the decompression data corresponding to the compressed weight data from the data storage unit.In one or more embodiments of the present disclosure, the obtaining, from the data storage unit, the compressed weight data corresponding to the data processing model and the decompression data corresponding to the compressed weight data comprises: generating, by a data processing node, a weight obtaining request based on a data processing network layer of the data processing model, and sending the weight obtaining request to the data storage unit; and receiving the compressed weight data and the decompression data corresponding to the compressed weight data sent by the data storage unit in response to the weight obtaining request. The data storage unit can be understood as a unit for storing compressed weight data and decompression data corresponding to the compressed weight data. The data processing network layer can be understood as a network layer in the data processing model for processing the to-be-processed data. For example, the data processing network layer can be a convolutional layer, a feature extraction layer, an encoding layer, an output layer, an input layer, etc., without specific limitation here. It should be noted that each data processing network layer can have corresponding weight data, which is used to process the to-be-processed data. Therefore, in the process of processing the to-be-processed data by using one or more data processing network layers of the data processing model, the compressed weight data corresponding to one or more data processing network layers and the decompression data corresponding to the compressed weight data can be obtained from the data storage unit. That is, each data processing network layer has corresponding compressed weight data and decompression data corresponding to the compressed weight data. The weight obtaining request can be understood as a request for obtaining the compressed weight data and the decompression data corresponding to the compressed weight data. The weight obtaining request can carry an identifier of the data processing model or the data processing network layer, so as to accurately obtain the corresponding compressed weight data and the decompression data corresponding to the compressed weight data from the data storage unit. Specifically, the manner of obtaining, from the data storage unit, the compressed weight data corresponding to the data processing model and the decompression data corresponding to the compressed weight data can be that a data processing node generates a weight obtaining request based on a data processing network layer of the data processing model, and sends the weight obtaining request to the data storage unit. The weight obtaining request can carry an identifier of the data processing model or the data processing network layer, or the weight obtaining request can carry a weight data identifier of the compressed weight data. The identifier of the data processing model or the data processing network layer can be data for uniquely identifying the data processing model or the data processing network layer, such as a number, an ID, etc., without specific limitation here. The weight data identifier can be understood as information for uniquely identifying a compressed weight data, such as a number, an ID, a storage address, etc., without specific limitation here.In a case where the data storage unit receives the weight acquisition request, the data storage unit determines an identifier of a data processing model or a data processing network layer carried in the weight acquisition request, and based on the identifier of the data processing model or the data processing network layer, determines, from locally stored data, compressed weight data corresponding to the data processing model or the data processing network layer and decompression data corresponding to the compressed weight data, and sends the compressed weight data and the decompression data corresponding to the compressed weight data to a data processing node running the data processing model. In a case where the data storage unit receives the weight acquisition request, the data storage unit determines a weight data identifier of the compressed weight data carried in the weight acquisition request, and based on the weight data identifier, determines, from locally stored data, compressed weight data corresponding to the data processing model or the data processing network layer and decompression data corresponding to the compressed weight data, and sends the compressed weight data and the decompression data corresponding to the compressed weight data to a data processing node running the data processing model. The data processing node can receive the compressed weight data and the decompression data corresponding to the compressed weight data sent by the data storage unit in response to the weight acquisition request. In the above embodiment, the compressed weight data corresponding to the data processing model and the decompression data corresponding to the compressed weight data can be quickly and accurately obtained from the data storage unit based on the weight acquisition request generated by the data processing network layer of the data processing model, so as to facilitate subsequent data processing of the to-be-processed data based on the weight data, obtain a data processing result, and thus improve the data processing efficiency of the data processing model. Step 206: performing data decompression processing on the compressed weight data using the decompression data to obtain decompressed weight data. In one or more embodiments of the present disclosure, the decompression data is a weight parameter sequence table; the data decompression processing on the compressed weight data using the decompression data to obtain decompressed weight data includes: determining decompression parameters corresponding to the compressed weight data from the weight parameter sequence table, and performing first-stage data decompression on the compressed weight data based on the decompression parameters to obtain initial decompressed weight data; determining quantization parameters and a quantization strategy corresponding to the compressed weight data, and performing second-stage data decompression on the initial weight data based on the quantization parameters and the quantization strategy to obtain the decompressed weight data. In the above example, after the weight data is quantized at a low bit and compressed by Huffman coding, the compressed weight parameters can be used for operation in the actual model inference process, specifically including the following manners.

[0012] 1. Receive a question text (to-be-processed data) sent by a user, and an LLM large model (i.e., a data processing model) for language inference on the question text; wherein the LLM large model can run on a CPU or a GPU (i.e., a data processing node).

[0013] 2. In LLM online inference, the compressed saved weight data is obtained from the video memory or the memory.

[0014] 3、 decode using the huffman tree coding table. The specific way to decompress the compressed weight data is: first, decode the compressed weight using the huffman coding table (i.e. the first stage of data decompression); that is, parse the compressed bits one by one, and compare them with the coding table to obtain the original value before compression. For example, parse the compressed bits 0111001010111110110 one by one, 0 can be restored to the original value 0 through the huffman coding table, 1 has no corresponding value in the huffman coding table, so it is stacked to see 111 in the huffman coding table, which can be restored to 255, and so on, that is, the original value of the compressed bit stream can be restored. Secondly, after huffman decoding, the data is dequantized using the quantization parameter and the specific quantization strategy (i.e. the second stage of data decompression), to obtain the weight data in the original floating point format. In the above example, the first stage of data decompression and the second stage of data decompression are used to realize lossless decompression of the compressed weight data, avoiding the problem that the data processing model cannot process the to-be-processed data due to decompression error, improving the data processing efficiency of the data processing model, realizing fast data processing result, and meeting the user's data efficient processing demand. Step 208, according to the decompressed weight data, using the data processing model to process the to-be-processed data, and obtaining the data processing result. In the above example, after obtaining the decompressed weight data, the original floating point format weight data can be operated with the activation value to realize the inference of LLM; for example, in the case of text data processing scene, the data processing model can be a data processing model for processing the to-be-processed problem text, and correspondingly, the to-be-processed problem text can be a problem in the form of text proposed by the user. By processing the to-be-processed problem text through the data processing model, the answer text corresponding to the to-be-processed problem text can be obtained, for example, the to-be-processed problem text can be "labor day is in which month and which day", based on this, the to-be-processed problem text can be input into the data processing model, and the to-be-processed problem text can be processed based on the data processing model and the decompressed weight data, to obtain the answer text "labor day is on may 1st every year"; through the above way, the user's question in the actual scene is answered, meeting the demand of answering the user's question by using the model in the actual scene.The data processing method in one or more embodiments of the present disclosure can obtain the compressed weight data and the decompression data corresponding to the compressed weight data from the data storage unit after determining the to-be-processed data and the data processing model for processing the to-be-processed data. Since the data quantity of the compressed weight data is smaller than that of the uncompressed weight data, the compressed weight data and the decompression data with a smaller data quantity can be obtained from the data storage unit in a shorter data transmission time, thereby saving the data transmission time of the weight data in the data processing process, and then the to-be-processed data can be quickly processed by using the data processing model according to the weight data, so as to obtain the data processing result, thereby improving the data processing efficiency of the data processing model, realizing the quick obtaining of the data processing result, and meeting the user's data efficient processing demand. It should be noted that the data processing method in one or more embodiments of the present disclosure considers that the uncompressed weight data occupies too much data transmission resource (for example, bandwidth) and has a large data volume, and a longer data transmission time is required in the data transmission process of the uncompressed weight data. Therefore, by compressing the weight data, the volume of the weight data is reduced on the one hand, and the data transmission time is reduced in the data transmission process of the compressed weight data on the other hand. Moreover, although the decompression time of the compressed weight data is increased, the total time of the decompression time of the compressed weight data and the data transmission time of the compressed weight data is less than the data transmission time of the uncompressed weight data. That is to say, after the weight data is compressed, although the decompression time is increased, the total time is less than the time consumption corresponding to the uncompressed weight data. The data processing method provided by the present disclosure is described further in the following with reference to FIG. 3. FIG. 3 shows a processing process flowchart of a data processing method provided by one embodiment of the present disclosure. The data processing method provided by the present disclosure proposes an effective weight compression scheme. First, the weight of the floating-point type is quantized by a low bit, and then lossless compression is performed by Huffman coding, so as to realize the efficient compression of the weight data, thereby achieving the purpose of improving the LLM inference performance by reducing the memory occupation and reducing the memory access overhead. It should be noted that for the LLM inference model, the trained weight data is fixed and unchanged. Therefore, the quantization and Huffman coding of the weight data in steps 302 to 326 described below can be completed offline in advance, and will not occupy and affect the performance of the LLM inference. Based on this, the data processing method in the present disclosure specifically includes the following steps.Step 302: determining the trained LLM weight data. Step 304: counting the weight value range. Specifically, in the process of weight data compression, the numerical range of floating-point weight data needs to be counted, so as to determine the maximum and minimum values in the floating-point weight data. Step 306: determining the quantization parameter. Specifically, the maximum and minimum values in the floating-point weight data are taken as the quantization parameter. Step 308: quantizing the weight data. Specifically, after determining the quantization parameter required for quantization, the quantization parameter and the quantization strategy can be used to quantize the weight parameter to obtain a low-bit representation of the weight parameter; the low-bit representation can be in INT8 format or INT4 format. Here, the quantization strategy can be per-tensor quantization strategy or per-channel quantization strategy, and the low-bit quantization precision can be INT8 or INT4. Then, according to the quantization parameter and the quantization strategy, the weight is quantized to obtain the weight in low-bit representation (INT8 or INT4) format. It should be noted that the process of low-bit quantization of weight data can be understood as the first stage of data compression of weight data, so that the weight data (with large memory occupation) is represented by fewer binary bits (low bits); the Huffman coding compression process performed in the following steps can be understood as the second stage of data compression of weight data. Step 310: counting the value frequency. Specifically, after the weight quantization process is completed, Huffman coding compression is started; the basic principle of Huffman coding compression is that the values with higher frequency in the weight data are represented by shorter bits, and the values with lower frequency are represented by longer bits, which can reduce the average length after coding, thereby realizing lossless data compression. Therefore, when performing Huffman coding compression, the frequency of the weight values needs to be counted; taking the low-bit representation of the weight parameter as INT8 format as an example, for convenience, the INT8 values are parsed and represented as UINT8, so the representation range is 0-255; the frequency of each value from 0 to 255 in the weight data is counted, and a frequency table is generated according to the frequency size, which is shown in Table 1 below. Table 1. Based on the above table 1, it is assumed that there are 6 values in the weight data; in addition, when more values appear in the weight, the example can be analogized. Step 312: Constructing a Huffman tree. Specifically, a Huffman tree can be established based on the table 1, which can be seen from FIG. 4, which is a structural schematic diagram of a Huffman tree of a data processing method provided by an embodiment of the present disclosure; the specific way of establishing the Huffman tree is as follows:

[0015] 1. Take out the two U I NT8 values with the minimum frequency from the frequency table, that is, 210 and 113, which are terminal nodes (leaf nodes), that is, the dashed line nodes in FIG. 3, and their corresponding frequencies are added, that is, 2+3=5, which is an intermediate node, that is, the solid line node in FIG. 3.

[0016] 2. Continue to take out the U I NT8 value with the minimum frequency from the remaining values in the frequency table, that is, 88, which is a terminal node, and its frequency 4 is less than that of the intermediate node 5, so the terminal node 88 is placed on the left side of the intermediate node 5. The frequency 4 corresponding to the terminal node 88 is added to the frequency 5 of the intermediate node to obtain an intermediate node 9 O

[0017] 3. Continue to take out the U I NT8 value with the minimum frequency from the remaining values in the frequency table, that is, 22, which is a terminal node, and its frequency 5 is less than that of the intermediate node 9, so the terminal node 22 is placed on the left side of the intermediate node 9. The frequency 5 corresponding to the terminal node 22 is added to the frequency 9 of the intermediate node to obtain an intermediate node 14 o

[0018] 4. Continue to take out the U I NT8 value with the minimum frequency from the remaining values in the frequency table, that is, 255, which is a terminal node, and its frequency 15 is greater than that of the intermediate node 14, so the terminal node 255 is placed on the right side of the intermediate node 14. The frequency 15 corresponding to the terminal node 255 is added to the frequency 14 of the intermediate node to obtain an intermediate node 29 o

[0019] 5、 Finally, the value 0 is taken out from the frequency table as a terminal node, and its frequency 20 is less than the intermediate node 29, so the terminal node 0 is placed at the left side of the intermediate node 29. The frequency 20 corresponding to the terminal node 0 is added to the intermediate node 29 to obtain an intermediate node 49. Step 314: Create a Huffman coding table. Specifically, after the Huffman construction is completed, the weight data can be encoded and compressed in the following manner: all left branches in the Huffman tree are coded as 0, and the right branches are coded as 1; thus, all terminal nodes can be obtained by traversing the Huffman binary tree from the top node (i.e., the 49 node). For example, the 0 terminal node corresponds to the code 0, the 255 terminal node corresponds to the code 11, the 22 terminal node corresponds to the code 100, the 210 terminal node corresponds to the code 10110, and so on. After encoding, the encoded data corresponding to the weight data is saved in the Huffman coding table, which can be seen in Table 2 below. Table 2 The data in the "Encoding" row in Table 2 above is the encoding corresponding to the weight data. Step 316: Determine the weight encoding and save. Specifically, after the encoding table is constructed, the original weight value sequence can be encoded; for example, the sequence 0, 255, 22, 88, 113, 210 becomes 01110010101011110110o after encoding. Based on this, it can be seen that the weight data occupies 48 bits of space before encoding, and 20 bits of space after encoding. After low-bit quantization and Huffman encoding compression of the weight data, in the actual model inference process, the compressed weight parameters can be used for operation, and the specific model operation process can be performed in steps 318 to 326 described below. Step 318: Receive the question text sent by the user, and the LLM large model for language inference of the question text. The LLM large model can run on a CPU or a GPU. Step 320: Obtain the compressed and saved weight data. Specifically, in the LLM online inference process, the compressed and saved weight data is obtained from the video memory or the memory. Step 322: Decode using the Huffman tree encoding table. Specifically, in the LLM online inference process, the decompression method for the compressed weight data is as follows: First, decode the compressed weight using the Huffman encoding table; that is, analyze the compressed bits one by one, and compare them with the encoding table to obtain the original value before compression. For example, analyze the compressed bits 01110010101011110110 one by one, 0 can be restored to the original value 0 through the Huffman encoding table, 1 has no corresponding value in the Huffman encoding table, so it is stacked to 11, which can be restored to 255 in the Huffman encoding table, and so on. In this way, the original value of the compressed bit stream can be restored. Step 324: Use the quantization parameter to perform dequantization. Specifically, after Huffman decoding, the data is dequantized using the quantization parameter and the specific quantization strategy to obtain the original floating-point format weight. Step 326: Perform operations based on the weight data and the activation value. Specifically, the original floating-point format weight is finally operated with the activation to implement the inference of the LLM. It should be noted that this process needs to be completed inside the CPU kernel or the GPU kernel, because the decompressed and dequantized weight data needs to be kept in the cache to avoid writing to the global memory; if it is written to the global memory and then read, additional memory access overhead will be generated.Based on the above steps, the data processing method provided by the present disclosure provides a scheme for improving the running efficiency of the LLM model. By quantizing and Huffman coding the weight data, efficient compression of the weight data is achieved, which can reduce the LLM inference memory occupancy and reduce the memory bandwidth, thereby improving the LLM inference performance. Moreover, due to the model structure characteristics of the LLM, the inference performance is generally limited by the memory bandwidth overhead; most of the memory overhead comes from the memory access of the weight data; and by quantizing and Huffman coding the weight data, the weight size can be reduced by 50%-90%. Therefore, the memory overhead can be greatly reduced, thereby improving the inference performance of the model. In addition, it should be noted that for the way of compressing the weight data only by quantization and saving the quantized data in the form of low-bit (INT8 or INT4) data format, considering that this way does not fully utilize the sparsity characteristics of the quantized weight data, because the quantized weight data is not uniformly distributed, there are a large number of same values (such as zero values) or a large number of missing values. Based on this, the data processing method provided by the present disclosure utilizes this feature to introduce Huffman coding; by Huffman coding, the non-uniformity of the data can be utilized for lossless compression of the data; therefore, by applying Huffman coding, the weight size can be further compressed on the basis of maintaining the accuracy of the quantized weight, thereby reducing the memory bandwidth of the LLM inference process and improving the performance of the LLM inference. Referring to FIG. 5, FIG. 5 shows a flowchart of a text data processing method according to an embodiment of the present disclosure, which specifically includes the following steps. Step 502: determining a to-be-processed problem text and determining a data processing model for processing the to-be-processed problem text. Step 504: obtaining compressed weight data corresponding to the data processing model and decompression data corresponding to the compressed weight data from a data storage unit. Step 506: performing data decompression processing on the compressed weight data by using the decompression data to obtain decompressed weight data. Step 508: performing data processing on the to-be-processed problem text by using the data processing model according to the decompressed weight data to obtain an answer text. In one or more embodiments provided by the present disclosure, the determination of the to-be-processed problem text includes: receiving a to-be-processed problem text sent by a client, wherein the to-be-processed problem text is sent by a user through a problem text processing interface in the client; and after the data processing on the to-be-processed problem text by using the data processing model according to the decompressed weight data to obtain an answer text, the method further includes: sending the answer text to the client to enable the client to display the answer text to the user through the problem text processing interface.The problem text processing interface can be understood as an interface for a user to process problem text. The problem text processing interface can be an application interface, a webpage, or the like. The user can interact with the client through the problem text processing interface. Specifically, the user can send the to-be-processed problem text to the client through the problem text processing interface, so that the client sends the to-be-processed problem text to the data processing node running the data processing model for processing. After the data processing node processes the to-be-processed problem text by using the decompressed weight data and the data processing model, the data processing node can send the answer text to the client, so that the client displays the answer text to the user through the problem text processing interface. In the above embodiment, the client and the problem text processing interface can interact with the user. The user can input the to-be-processed problem text through the problem text processing interface, and the answer text can be displayed to the user through the problem text processing interface, thereby meeting the actual demand of replying to the question raised by the user. In one or more embodiments of the present disclosure, after the to-be-processed problem text and the data processing model for processing the to-be-processed problem text are determined, the compressed weight data corresponding to the data processing model and the decompression data corresponding to the compressed weight data can be obtained from the data storage unit. Since the data quantity of the compressed weight data is smaller than that of the uncompressed weight data, the data quantity of the compressed weight data and the decompression data can be obtained from the data storage unit in a smaller data transmission time, thereby saving the data transmission time of the weight data in the data processing process, and the to-be-processed problem text can be quickly processed by using the data processing model according to the weight data to obtain the answer text, thereby improving the data processing efficiency of the data processing model, quickly obtaining the answer text, and meeting the data efficient processing demand of the user. Referring to FIG. 6, FIG. 6 shows a flowchart of another data processing method according to one embodiment of the present disclosure, specifically including the following steps. Step 602: receiving a weight obtaining request sent by a data processing unit, wherein the weight obtaining request is sent by the data processing unit according to a data processing model for processing to-be-processed data, and the data processing model is a model for processing to-be-processed data. Step 604: determining compressed weight data corresponding to the data processing model and decompression data corresponding to the compressed weight data.Step 606: send the compressed weight data and the decompression data corresponding to the compressed weight data to the data processing unit, so that the data processing unit obtains decompressed weight data by using the decompression data, and performs data processing on the to-be-processed data by using the data processing model according to the decompressed weight data, to obtain a data processing result. Another data processing method in one or more embodiments of the present disclosure can determine the compressed weight data corresponding to the data processing model and the decompression data corresponding to the compressed weight data from the data stored this time after receiving the weight obtaining request sent by the data processing unit, and send the compressed weight data and the decompression data corresponding to the compressed weight data to the data processing unit. Since the data quantity of the compressed weight data is smaller than that of the uncompressed weight data, the weight data can be sent to the data processing unit in a smaller data transmission time, thereby saving the data transmission time of the weight data in the data processing process, so that the data processing unit can quickly perform data processing on the to-be-processed data by using the data processing model according to the weight data, to obtain a data processing result, thereby improving the data processing efficiency of the data processing model, realizing fast obtaining of the data processing result, and further meeting the user's data efficient processing demand. Corresponding to the method embodiments described above, the present disclosure further provides a data processing device embodiment. FIG. 7 shows a structural schematic diagram of a data processing device according to an embodiment of the present disclosure. As shown in FIG. 7, the device comprises: a first data determination module 702 configured to determine to-be-processed data and a data processing model for processing the to-be-processed data; a second data determination module 704 configured to obtain compressed weight data corresponding to the data processing model and decompression data corresponding to the compressed weight data from a data storage unit; a data decompression module 706 configured to perform data decompression processing on the compressed weight data by using the decompression data, to obtain decompressed weight data; and a data processing module 708 configured to perform data processing on the to-be-processed data by using the data processing model according to the decompressed weight data, to obtain a data processing result. Optionally, the data processing device further comprises a data compression module configured to: determine the weight data corresponding to the data processing model, wherein the weight data is obtained by model training on the data processing model; perform data compression on the weight data, to obtain the compressed weight data and the decompression data corresponding to the compressed weight data; and store the compressed weight data and the decompression data into the data storage unit.Optionally, the data compression module is further configured to: perform first-stage data compression on the weight data to obtain initial compressed weight data; and perform second-stage data compression on the initial compressed weight data to obtain the compressed weight data and the decompression data corresponding to the compressed weight data. Optionally, the data compression module is further configured to: determine a data range corresponding to the weight data, and determine a quantization parameter corresponding to the weight data based on the data range; perform low-bit quantization on the weight data based on the quantization parameter and a preset quantization strategy to obtain low-bit weight data, and take the low-bit weight data as the initial compressed weight data. Optionally, the data compression module is further configured to: determine a plurality of weight parameters included in the initial compressed weight data, and construct a weight parameter sequence list based on the plurality of weight parameters; determine a leaf node based on a weight parameter in the weight parameter sequence list, and construct a data compression tree structure based on the leaf node; perform data compression on the initial compressed weight data based on the data compression tree structure to obtain the compressed weight data, and determine the weight parameter sequence list as the decompression data corresponding to the compressed weight data. Optionally, the data compression module is further configured to: determine a plurality of weight parameters included in the initial compressed weight data, and a parameter quantity of the plurality of weight parameters; sort each weight parameter based on the parameter quantity to obtain a weight parameter sequence, and construct the weight parameter sequence list based on the weight parameter sequence. Optionally, the data compression module is further configured to: determine a target weight parameter in the weight parameter sequence list and an associated parameter quantity corresponding to the target weight parameter, wherein the associated parameter quantity is determined by parameter quantities of at least two initial weight parameters, the target weight parameter is any one of the plurality of weight parameters in the weight parameter sequence list other than the at least two initial weight parameters, and the at least two initial weight parameters are weight parameters smaller than the target weight parameter in the weight parameter sequence list; determine a total parameter quantity based on the parameter quantities of each weight parameter in the weight parameter sequence list; determine the total parameter quantity as a root node, and determine each weight parameter and the associated parameter quantity as a leaf node; and construct the data compression tree structure based on the root node and the leaf node by using a size relationship between the parameter quantities of each weight parameter and the associated parameter quantity.Optionally, the data compression module is further configured to: determine a plurality of node branches in the data compression tree structure, wherein the node branch is formed by setting two leaf nodes on the left and right sides of an intermediate node according to the size relationship of the number of parameters corresponding to the two leaf nodes; encode the left leaf node in each node branch as first bit data, and encode the right leaf node in each node branch as second bit data; traverse a target leaf node in the data compression tree structure, determine a traversal path node sequence of the target leaf node, wherein the target leaf node is any one leaf node in the data compression tree structure; determine a weight compression code corresponding to the target leaf node according to the first bit data of the left leaf node and the second bit data of the right leaf node in the traversal path node sequence, and determine the weight compression code as the compressed weight data. Optionally, the decompressed data is a weight parameter sequence table; the data decompression module 706 is further configured to: determine a decompressed parameter corresponding to the compressed weight data from the weight parameter sequence table, and perform first-stage data decompression on the compressed weight data based on the decompressed parameter to obtain initial weight data after decompression; determine a quantization parameter and a quantization strategy corresponding to the compressed weight data, and perform second-stage data decompression on the initial weight data based on the quantization parameter and the quantization strategy to obtain the decompressed weight data. Optionally, the second data determination module 704 is further configured to: generate a weight acquisition request based on the data processing network layer of the data processing model, and send the weight acquisition request to the data storage unit; receive the compressed weight data and the decompressed data corresponding to the compressed weight data sent by the data storage unit in response to the weight acquisition request. In one of the one or more embodiments of the present disclosure, after determining the data to be processed and the data processing model for processing the data to be processed, the data processing device can obtain the compressed weight data corresponding to the data processing model and the decompressed data corresponding to the compressed weight data from the data storage unit. Since the data quantity of the compressed weight data is less than that of the uncompressed weight data, the data processing device can obtain the compressed weight data and the decompressed data with smaller data quantity from the data storage unit in a smaller data transmission time, thereby saving the data transmission time of the weight data in the data processing process, and quickly processing the data to be processed based on the weight data using the data processing model to obtain the data processing result, thereby improving the data processing efficiency of the data processing model, quickly obtaining the data processing result, and meeting the user's demand for efficient data processing. The above is a schematic scheme of a data processing device of the present embodiment.It should be noted that the technical scheme of the data processing apparatus belongs to the same concept as the technical scheme of the data processing method described above, and the details of the technical scheme of the data processing apparatus that are not described in detail can be referred to the description of the technical scheme of the data processing method. Corresponding to the method embodiments described above, the present disclosure also provides text data processing apparatus embodiments, and FIG. 8 shows a structural schematic diagram of a text data processing apparatus according to an embodiment of the present disclosure. As shown in FIG. 8, the apparatus comprises: a first data determination module 802 configured to determine a to-be-processed question text and determine a data processing model for processing the to-be-processed question text; a second data determination module 804 configured to obtain, from a data storage unit, compressed weight data corresponding to the data processing model and decompression data corresponding to the compressed weight data; a data decompression module 806 configured to perform data decompression processing on the compressed weight data by using the decompression data to obtain decompressed weight data; and a data processing module 808 configured to perform data processing on the to-be-processed question text by using the data processing model according to the decompressed weight data to obtain an answer text. Optionally, the first data determination module 802 is further configured to: receive a to-be-processed question text sent by a client, wherein the to-be-processed question text is sent by a user through a question text processing interface in the client; and the text data processing apparatus further comprises a text display module configured to: send the answer text to the client so that the client displays the answer text to the user through the question text processing interface. The text data processing apparatus in one or more embodiments of the present disclosure can obtain, after determining a to-be-processed question text and a data processing model for processing the to-be-processed question text, compressed weight data corresponding to the data processing model and decompression data corresponding to the compressed weight data from a data storage unit. Since the data quantity of the compressed weight data is smaller than that of the uncompressed weight data, the compressed weight data and the decompression data with smaller data quantity can be obtained from the data storage unit in a shorter data transmission time, thereby saving the data transmission time of the weight data in the data processing process, and the to-be-processed question text can be quickly processed by using the data processing model according to the weight data to obtain an answer text, thereby improving the data processing efficiency of the data processing model, realizing quick obtaining of the answer text, and meeting the user's requirement for efficient data processing. The above is a schematic scheme of a text data processing apparatus according to the present embodiment.It should be noted that the technical scheme of the text data processing apparatus belongs to the same concept as the technical scheme of the text data processing method described above, and the details of the technical scheme of the text data processing apparatus that are not described in detail can be referred to the description of the technical scheme of the text data processing method. Corresponding to the method embodiments described above, the disclosure also provides another data processing apparatus embodiment, and FIG. 9 shows a structural schematic diagram of another data processing apparatus provided by an embodiment of the disclosure. As shown in FIG. 9, the apparatus comprises: a request receiving module 902 configured to receive a weight obtaining request sent by a data processing unit, wherein the weight obtaining request is sent by the data processing unit according to a data processing model for processing the to-be-processed data, and the data processing model is a model for processing the to-be-processed data; a data determining module 904 configured to determine compressed weight data corresponding to the data processing model and decompression data corresponding to the compressed weight data; and a data sending module 906 configured to send the compressed weight data and the decompression data corresponding to the compressed weight data to the data processing unit, so that the data processing unit obtains decompressed weight data by using the decompression data, and performs data processing on the to-be-processed data by using the data processing model according to the decompressed weight data, to obtain a data processing result. The another data processing apparatus in one or more embodiments of the disclosure can determine the compressed weight data corresponding to the data processing model and the decompression data corresponding to the compressed weight data from the data stored this time after receiving the weight obtaining request sent by the data processing unit, and send the compressed weight data and the decompression data corresponding to the compressed weight data to the data processing unit. Since the data quantity of the compressed weight data is smaller than the data quantity of the uncompressed weight data, the weight data can be sent to the data processing unit by using a smaller data transmission time, thereby saving the data transmission time of the weight data in the data processing process, so that the data processing unit can quickly perform data processing on the to-be-processed data by using the data processing model according to the weight data, to obtain a data processing result, thereby improving the data processing efficiency of the data processing model, realizing fast obtaining of the data processing result, and further meeting the user's data efficient processing demand. The above is a schematic scheme of the another data processing apparatus of the embodiment. It should be noted that the technical scheme of the another data processing apparatus belongs to the same concept as the technical scheme of the another data processing method described above, and the details of the technical scheme of the another data processing apparatus that are not described in detail can be referred to the description of the technical scheme of the another data processing method. FIG. 10 shows a structural block diagram of a computing device 1000 according to an embodiment of the disclosure.The components of computing device 1000 include, but are not limited to, a memory 1010, a processor 1020, and a database 1050 for storing data. The processor 1020 is connected to the memory 1010 via a bus 1030. The computing device 1000 also includes an access device 1040 that enables the computing device 1000 to communicate via one or more networks 1060. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or combinations of such networks, such as the Internet. The access device 1040 can include one or more of any type of network interface (e.g., a network interface controller (NIC)), wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface. In one embodiment of the present disclosure, the aforementioned components of the computing device 1000, as well as other components not shown in FIG. 10, can also be connected to each other, such as via a bus. It should be appreciated that the computing device structure diagram shown in FIG. 10 is merely for the purpose of example, and is not a limitation on the scope of the present disclosure. Those skilled in the art can add or replace other components as needed.The computing device 1000 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 1000 can also be a mobile or stationary server. The processor 1020 is configured to execute instructions / computer programs to implement the steps of the above-described data processing method, the text data processing method, or another data processing method. Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, the computing device embodiment is described simply because it is basically similar to the above-described data processing method, the text data processing method, or another data processing method embodiment. The relevant parts can be referred to the description of the above-described data processing method, the text data processing method, or another data processing method embodiment. An embodiment of the present disclosure further provides a computer-readable storage medium storing computer programs / instructions, which are executed by a processor to implement the steps of the above-described data processing method, the text data processing method, or another data processing method. Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, the computer-readable storage medium embodiment is described simply because it is basically similar to the above-described data processing method, the text data processing method, or another data processing method embodiment. The relevant parts can be referred to the description of the above-described data processing method, the text data processing method, or another data processing method embodiment. An embodiment of the present disclosure further provides a computer program product including computer programs / instructions, which are executed by a processor to implement the steps of the above-described data processing method, the text data processing method, or another data processing method. The above is a schematic scheme of the computer program product of the embodiment.It should be noted that the technical solution of the computer program product belongs to the same concept as the technical solutions of the above-described data processing method, the text data processing method, or another data processing method. The technical solution of the computer program product is not described in detail, and can be referred to the description of the technical solution of the above-described data processing method, the text data processing method, or another data processing method. The specific embodiments of the present disclosure are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps in a claim can be performed in an order different from embodiments and still achieve the desired result. Also, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous. The computer instructions include computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer storage, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content included in the computer readable medium can be appropriately added or reduced according to the requirements of patent practice. For example, in some regions, according to patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals. It should be noted that for the foregoing method embodiments, in order to facilitate description, they are all expressed as a combination of a series of actions, but those skilled in the art should know that the embodiments of the present disclosure are not limited by the order of the described actions, because according to the embodiments of the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the present disclosure are all preferred embodiments, and the actions and modules involved are not necessarily necessary for the embodiments of the present disclosure. In the above embodiments, the description of each embodiment is focused on, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments. The preferred embodiments of the present disclosure disclosed above are only used to help explain the present disclosure. The alternative embodiments do not describe all the details and do not limit the present disclosure to the specific embodiments described. Obviously, according to the content of the embodiments of the present disclosure, many modifications and changes can be made.The present disclosure selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of the present disclosure, so that those skilled in the art can well understand and utilize the present disclosure. The present disclosure is limited only by the claims and their full scope and equivalents.

Claims

22 CLAIM 1. A data processing method, comprising: determining to-be-processed data and a data processing model for processing the to-be-processed data; obtaining, from a data storage unit, compressed weight data corresponding to the data processing model and decompression data corresponding to the compressed weight data, performing data decompression processing on the compressed weight data by using the decompression data to obtain decompressed weight data, and performing data processing on the to-be-processed data by using the data processing model according to the decompressed weight data to obtain a data processing result.

2. The data processing method of claim 1, before the obtaining, from the data storage unit, the compressed weight data corresponding to the data processing model and the decompression data corresponding to the compressed weight data, the method further comprises: determining weight data corresponding to the data processing model, wherein the weight data is obtained by model training on the data processing model, performing data compression on the weight data to obtain the compressed weight data and decompression data corresponding to the compressed weight data, and storing the compressed weight data and the decompression data in a data storage unit.

3. The data processing method of claim 2, wherein the data compression on the weight data to obtain the compressed weight data and the decompression data corresponding to the compressed weight data comprises: performing first-stage data compression on the weight data to obtain initial compressed weight data. performing second-stage data compression on the initial compressed weight data to obtain the compressed weight data and the decompression data corresponding to the compressed weight data.

4. The data processing method according to claim 3, wherein the first-stage data compression on the weight data to obtain initial compressed weight data comprises: determining a data range corresponding to the weight data and determining a quantization parameter corresponding to the weight data based on the data range. performing low-bit quantization on the weight data based on the quantization parameter and a preset quantization strategy to obtain low-bit weight data, and taking the low-bit weight data as the initial compressed weight data.

5. The data processing method of claim 3 or 4, wherein the second-stage data compression on the initial compressed weight data to obtain the compressed weight data and the decompression data corresponding to the compressed weight data comprises: determining a plurality of weight parameters included in the initial compressed weight data and constructing a weight parameter sequence list based on the plurality of weight parameters; determining a leaf node based on a weight parameter in the weight parameter sequence list and constructing a data compression tree structure based on the leaf node; and performing data compression on the initial compressed weight data based on the data compression tree structure to obtain the compressed weight data, and determining the weight parameter sequence list as the decompression data corresponding to the compressed weight data.

6. The data processing method of claim 5, wherein the plurality of weight parameters contained in the initial compression weight data are determined, and a weight parameter sequence table is constructed based on the plurality of weight parameters, comprising: determining a plurality of weight parameters included in the initial compressed weight data and a parameter quantity of the plurality of weight parameters; sorting each weight parameter based on the parameter quantity to obtain a weight parameter sequence and constructing the weight parameter sequence list based on the weight parameter sequence.

7. The data processing method of claim 5 or 6, wherein the determining a leaf node based on a weight parameter in the weight parameter sequence list and constructing a data compression tree structure based on the leaf node comprises: determine a target weight parameter in the weight parameter list, and a quantity of associated parameters corresponding to the target weight parameter, wherein the quantity of associated parameters is determined by a quantity of parameters of at least two initial weight parameters, the target weight parameter is any one weight parameter in the weight parameter list except the at least two initial weight parameters, and the at least two initial weight parameters are weight parameters smaller than the target weight parameter in the weight parameter list; determine a total quantity of parameters based on the quantity of parameters of each weight parameter in the weight parameter list; determine the total quantity of parameters as a root node, and determine each weight parameter and the quantity of associated parameters as a leaf node; and construct the data compression tree structure by using the quantity of parameters corresponding to each weight parameter and the quantity of associated parameters based on a size relationship therebetween.

8. The data processing method of any one of claims 5-7, wherein the data compression based on the data compression tree structure on the initial compressed weight data to obtain the compressed weight data comprises: determine a plurality of node branches in the data compression tree structure, wherein the node branch is formed by arranging two leaf nodes at left and right sides of an intermediate node according to a size relationship of the quantity of parameters corresponding to the two leaf nodes; encode a left leaf node in each node branch as first bit data, and encode a right leaf node in each node branch as second bit data; traverse a target leaf node in the data compression tree structure to determine a sequence of traversal path nodes corresponding to the target leaf node, wherein the target leaf node is any one leaf node in the data compression tree structure; determine a weight compression code corresponding to the target leaf node based on the first bit data of the left leaf node and the second bit data of the right leaf node in the sequence of traversal path nodes, and determine the weight compression code as the compressed weight data.

9. The data processing method according to any one of claims 1-8, wherein the decompression data is a weight parameter sequence table; and the data decompression processing of the compressed weight data by using the decompression data to obtain decompressed weight data comprises: determine a decompression parameter corresponding to the compressed weight data from the weight parameter list, and perform first-stage data decompression on the compressed weight data based on the decompression parameter to obtain initial weight data after decompression; determine a quantization parameter and a quantization strategy corresponding to the compressed weight data, and perform second-stage data decompression on the initial weight data based on the quantization parameter and the quantization strategy to obtain the weight data after decompression.

10. The data processing method according to any one of claims 1-9, wherein the compressed weight data corresponding to the data processing model and the decompression data corresponding to the compressed weight data are obtained from a data storage unit, and the obtaining comprises: generate a weight acquisition request based on a data processing network layer of the data processing model, and send the weight acquisition request to the data storage unit; receive the compressed weight data and decompression data corresponding to the compressed weight data sent by the data storage unit in response to the weight acquisition request.

11. A method of processing text data, comprising: determine a to-be-processed problem text, and determine a data processing model for processing the to-be-processed problem text; The data compression weight data corresponding to the data processing model and the decompression data corresponding to the data compression weight data are obtained from the data storage unit; the data compression weight data is decompressed by using the decompression data to obtain decompressed weight data; and the data processing model is used to process the to-be-processed problem text according to the decompressed weight data, to obtain an answer text.

12. The text data processing method of claim 11, wherein the to-be-processed problem text is determined by: receiving the to-be-processed problem text sent by a client, wherein the to-be-processed problem text is sent by a user through a problem text processing interface in the client; and after the data processing model is used to process the to-be-processed problem text according to the decompressed weight data to obtain an answer text, the answer text is sent to the client, so that the client displays the answer text to the user through the problem text processing interface.

13. A data processing method, comprising: receiving a weight obtaining request sent by a data processing unit, wherein the weight obtaining request is sent by the data processing unit according to a data processing model used to process to-be-processed data; determining the data compression weight data corresponding to the data processing model and the decompression data corresponding to the data compression weight data; and sending the data compression weight data and the decompression data corresponding to the data compression weight data to the data processing unit, so that the data processing unit obtains decompressed weight data by using the decompression data, and processes the to-be-processed data according to the decompressed weight data by using the data processing model to obtain a data processing result.

14. A computing device comprising: a memory and a processor; the memory is configured to store computer programs / instructions, and the processor is configured to execute the computer programs / instructions, so as to implement the steps of the data processing method of any one of claims 1 to 10, the text data processing method of any one of claims 11 to 12, or the data processing method of claim 13.

15. A computer readable storage medium, which stores computer programs / instructions, and the computer programs / instructions are executed by a processor to implement the steps of the data processing method of any one of claims 1 to 10, the text data processing method of any one of claims 11 to 12, or the data processing method of claim 13.

16. A computer program product, which comprises computer programs / instructions, and the computer programs / instructions are executed by a processor to implement the steps of the data processing method of any one of claims 1 to 10, the text data processing method of any one of claims 11 to 12, or the data processing method of claim 13.

Images