Data compression method, system and device, computer equipment and readable storage medium

Abstract

The invention relates to a data compression method, system and device, computer equipment and a readable storage medium. The method comprises the following steps: inputting a byte sequence corresponding to data to be compressed into a pre-trained probability prediction model to obtain conditional probability distribution of the data to be compressed; inputting the conditional probability distribution of the data to be compressed into an entropy encoder ANS to obtain a compression result; wherein the sample byte sequences are input into the initial probability prediction model, and prediction results corresponding to the sample byte sequences are obtained; determining an initial loss value according to the prediction result and the distribution label corresponding to each sample byte sequence; according to the initial loss value and the regular penalty term, determining a target loss value and a to-be-pruned network in an initial probability prediction model; and pruning the network to be pruned in the initial probability prediction model to update the initial probability prediction model, and performing parameter adjustment on the updated initial probability prediction model based on the target loss value. The data compression speed can be improved.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a data compression method, system, apparatus, computer equipment, and readable storage medium. Background Technology

[0002] As the scale of enterprise business data continues to expand, enterprises need to process more and more data, leading to the emergence of digital operation systems.

[0003] Traditional technologies, which rely on compression algorithms based on static or limited context modeling, offer advantages in versatility and low computational power. However, when processing semantically complex and formatted operational data, the compression rate drops significantly and the data becomes susceptible to disturbances.

[0004] In recent years, deep learning-based autoregressive compressors have made significant breakthroughs in compression ratio, but their inference speed still needs to be improved. Summary of the Invention

[0005] Therefore, it is necessary to provide a data compression method, system, apparatus, computer equipment, and readable storage medium to address the aforementioned technical problems, which can improve data compression speed while ensuring compression ratio.

[0006] Firstly, this application provides a data compression method, including:

[0007] Obtain the data to be compressed and perform serialization processing on the data to be compressed to obtain the byte sequence corresponding to the data to be compressed;

[0008] The byte sequence is input into a pre-trained probability prediction model to obtain the conditional probability distribution of the data to be compressed;

[0009] The conditional probability distribution of the data to be compressed is input into the entropy encoder (Asymmetric Numeral Systems, ANS) to obtain the compression result of the data to be compressed.

[0010] The pre-trained probability prediction model is trained in the following way:

[0011] Each sample byte sequence is input into the initial probability prediction model to obtain the prediction result corresponding to each sample byte sequence;

[0012] The initial loss value is determined based on the prediction results and distribution labels corresponding to the byte sequences of each sample;

[0013] Based on the initial loss value and the regularization penalty term, determine the target loss value and the network to be pruned in the initial probability prediction model;

[0014] The network to be pruned in the initial probabilistic prediction model is pruned to update the initial probabilistic prediction model, and the parameters of the updated initial probabilistic prediction model are adjusted based on the target loss value.

[0015] In one embodiment, determining the target loss value and the network to be pruned in the initial probability prediction model based on the initial loss value and the regularization penalty term includes:

[0016] Based on the regularization penalty term, the initial loss value is corrected to obtain the target loss value and the weight coefficients corresponding to each network group in the probabilistic prediction model; each network group includes a preset number of network layers.

[0017] Based on the relationship between the weight coefficients of each network group and the preset weight threshold, the networks to be pruned in the initial probability prediction model are determined.

[0018] In one embodiment, the initial loss value is corrected based on a regularization penalty term to obtain the target loss value and the weight coefficients corresponding to each network group in the probabilistic prediction model, including:

[0019] A target function is constructed using the weight coefficients and initial loss values ​​corresponding to each network group as independent variables and the target loss value as the dependent variable.

[0020] The objective function is solved with the goal of minimizing the target loss value and having a network layer whose weight coefficients approach zero. This yields the target loss value and the corresponding weight coefficients for each network group.

[0021] In one embodiment, the network to be pruned in the initial probability prediction model is determined based on the relationship between the weight coefficients corresponding to each network group and a preset weight threshold, including:

[0022] Network groups with weight coefficients less than a preset weight threshold are used as the networks to be pruned in the initial probabilistic prediction model.

[0023] In one embodiment, the pre-trained probabilistic prediction model includes a byte embedding layer, a fully connected layer, and an output layer, with the fully connected layer connected to both the byte embedding layer and the output layer. Correspondingly, the byte sequence is input into the pre-trained probabilistic prediction model to obtain the compression result of the data to be compressed, including:

[0024] The byte sequence is input into the byte embedding layer to obtain the vectorized data corresponding to the data to be compressed;

[0025] Vectorized data is input into a fully connected layer to obtain context information corresponding to the data to be compressed;

[0026] The context information is input to the output layer to obtain the conditional probability distribution of the data to be compressed.

[0027] In one embodiment, the fully connected layer includes a first fully connected layer and a second fully connected layer, with the second fully connected layer connected to both the first fully connected layer and the output layer; correspondingly, the byte sequence is input into a pre-trained probability prediction model to obtain the conditional probability distribution of the data to be compressed, including:

[0028] Vectorized data is input into the first fully connected layer to obtain nonlinear feature data corresponding to the data to be compressed;

[0029] The nonlinear feature data is input into the second fully connected layer to obtain the context information corresponding to the data to be compressed; the context information is the result of the second fully connected layer performing high-order semantic abstraction processing on the nonlinear feature data.

[0030] Secondly, this application also provides a data compression system, which includes a graphics processing unit (GPU) and an entropy encoder (ANS). The GPU is equipped with a probabilistic prediction model, which is the aforementioned pre-trained probabilistic prediction model.

[0031] Thirdly, this application also provides a data compression apparatus, comprising:

[0032] The acquisition module is used to acquire the data to be compressed and to perform serialization processing on the data to be compressed to obtain the byte sequence corresponding to the data to be compressed.

[0033] The first determining module is used to input the byte sequence into a pre-trained probability prediction model to obtain the conditional probability distribution of the data to be compressed;

[0034] The compression module is used to input the conditional probability distribution of the data to be compressed into the entropy encoder ANS to obtain the compression result of the data to be compressed.

[0035] The pre-trained probability prediction model is trained in the following way:

[0036] The prediction module is used to input the byte sequence of each sample into the initial probability prediction model and obtain the prediction result corresponding to each byte sequence of the sample.

[0037] The second determination module is used to determine the initial loss value based on the prediction results and distribution labels corresponding to the byte sequences of each sample;

[0038] The third determination module is used to determine the target loss value and the network to be pruned in the initial probability prediction model based on the initial loss value and the regularization penalty term.

[0039] The update module is used to prune the network to be pruned in the initial probabilistic prediction model to update the initial probabilistic prediction model, and to adjust the parameters of the updated initial probabilistic prediction model based on the target loss value.

[0040] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0041] Obtain the data to be compressed and perform serialization processing on the data to be compressed to obtain the byte sequence corresponding to the data to be compressed;

[0042] The byte sequence is input into a pre-trained probability prediction model to obtain the conditional probability distribution of the data to be compressed;

[0043] The conditional probability distribution of the data to be compressed is input into the entropy encoder ANS to obtain the compression result of the data to be compressed.

[0044] The pre-trained probability prediction model is trained in the following way:

[0045] Each sample byte sequence is input into the initial probability prediction model to obtain the prediction result corresponding to each sample byte sequence;

[0046] The initial loss value is determined based on the prediction results and distribution labels corresponding to the byte sequences of each sample;

[0047] Based on the initial loss value and the regularization penalty term, determine the target loss value and the network to be pruned in the initial probability prediction model;

[0048] The network to be pruned in the initial probabilistic prediction model is pruned to update the initial probabilistic prediction model, and the parameters of the updated initial probabilistic prediction model are adjusted based on the target loss value.

[0049] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0050] Obtain the data to be compressed and perform serialization processing on the data to be compressed to obtain the byte sequence corresponding to the data to be compressed;

[0051] The byte sequence is input into a pre-trained probability prediction model to obtain the conditional probability distribution of the data to be compressed;

[0052] The conditional probability distribution of the data to be compressed is input into the entropy encoder ANS to obtain the compression result of the data to be compressed.

[0053] The pre-trained probability prediction model is trained in the following way:

[0054] Each sample byte sequence is input into the initial probability prediction model to obtain the prediction result corresponding to each sample byte sequence;

[0055] The initial loss value is determined based on the prediction results and distribution labels corresponding to the byte sequences of each sample;

[0056] Based on the initial loss value and the regularization penalty term, determine the target loss value and the network to be pruned in the initial probability prediction model;

[0057] The network to be pruned in the initial probabilistic prediction model is pruned to update the initial probabilistic prediction model, and the parameters of the updated initial probabilistic prediction model are adjusted based on the target loss value.

[0058] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0059] Obtain the data to be compressed and perform serialization processing on the data to be compressed to obtain the byte sequence corresponding to the data to be compressed;

[0060] The byte sequence is input into a pre-trained probability prediction model to obtain the conditional probability distribution of the data to be compressed;

[0061] The conditional probability distribution of the data to be compressed is input into the entropy encoder ANS to obtain the compression result of the data to be compressed.

[0062] The pre-trained probability prediction model is trained in the following way:

[0063] Each sample byte sequence is input into the initial probability prediction model to obtain the prediction result corresponding to each sample byte sequence;

[0064] The initial loss value is determined based on the prediction results and distribution labels corresponding to the byte sequences of each sample;

[0065] Based on the initial loss value and the regularization penalty term, determine the target loss value and the network to be pruned in the initial probability prediction model;

[0066] The network to be pruned in the initial probabilistic prediction model is pruned to update the initial probabilistic prediction model, and the parameters of the updated initial probabilistic prediction model are adjusted based on the target loss value.

[0067] The aforementioned data compression method, system, apparatus, computer equipment, and readable storage medium determine the target loss value and the pruned networks in the initial probabilistic prediction model using an initial loss value and a regularization penalty term. The pruned networks in the initial probabilistic prediction model are then pruned to update the model. The updated initial probabilistic prediction model's parameters are adjusted based on the target loss value to obtain a pre-trained probabilistic prediction model. The compressed data is then processed based on this pre-trained model to obtain the compressed result. In this process, because some networks with limited contribution to probability prediction are removed from the pre-trained probabilistic prediction model, its structural complexity is lower, thus improving compression efficiency. Therefore, the aforementioned data compression method can improve compression efficiency while maintaining compression effectiveness. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 This is a flowchart illustrating a data compression method in one embodiment;

[0070] Figure 2 This is a flowchart illustrating the steps for determining the network to be pruned in one embodiment;

[0071] Figure 3 This is a block diagram of a pre-trained probability prediction model in one embodiment.

[0072] Figure 4 This is a flowchart illustrating the steps for determining the compression result in one embodiment;

[0073] Figure 5 Here is a block diagram of a pre-trained probability prediction model in another embodiment;

[0074] Figure 6 This is a flowchart illustrating a data compression method in another embodiment;

[0075] Figure 7 This is a block diagram of a data compression system in one embodiment;

[0076] Figure 8 This is a structural block diagram of a data compression device in one embodiment;

[0077] Figure 9A block diagram of a device for implementing a convenient model training device;

[0078] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0079] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0080] Before introducing the embodiments of this application, it should be noted that in the current digital operation system, as the scale of enterprise business data continues to expand, traditional data compression technologies are gradually revealing significant shortcomings in terms of compression efficiency, computing resource consumption, and real-time performance. Especially with "data-driven operation" becoming a core development strategy, operation platforms need to collect and process various types of structured and unstructured data at high frequency, posing a severe challenge to traditional compression technologies.

[0081] Existing widely used traditional compression algorithms (such as gzip, bzip2, and LZ77 series) are based on static or limited context modeling. While they offer good versatility and low computational barriers, their compression ratios are significantly insufficient when dealing with semantically complex and diverse operational data. Taking typical data such as company operational indicator logs, user behavior text, and sensor time-series streams as examples, traditional algorithms struggle to accurately capture long-range dependencies and local patterns, resulting in unstable compression performance that is easily affected by data disturbances, failing to support the precision requirements of "letting the data speak for itself."

[0082] While deep learning-based autoregressive compressors have achieved significant breakthroughs in compression ratios in recent years, their practical applications are limited by extremely low inference speeds. These models often use complex recurrent neural networks with parameter counts exceeding one million, yet their inference speeds are only at the KB / s level. When enterprise digital platforms require concurrent compression of thousands of data sources and support for multi-tenant, multi-period data management, this throughput is far from meeting practical needs, let alone enabling collaboration with modules for real-time visualization and automated analysis.

[0083] Existing neural network compression methods mostly focus on distillation, quantization, and pruning of image models, while a complete solution for byte sequences, multimodal structures, and entropy coding compatibility has yet to be developed in the field of data compression. This limits many deep learning compressors to academic validation rather than seamless integration into industrial-grade operational platforms. Furthermore, redundant model structures and complex computational graphs hinder their deployment on edge devices, lightweight terminals, or embedded systems, failing to meet the needs of heterogeneous computing power across the entire digital operational platform chain.

[0084] Based on this, the embodiments of this application aim to solve a number of key technical bottlenecks faced by traditional compression algorithms and existing neural network compressors in digital operation environments. In particular, in response to the problems of insufficient compression capacity and untimely response exposed by the data processing, transmission and analysis needs in the process of building the "Cloudscape Digital Operation Management and Control Platform", a data compression method, system, device, computer equipment and readable storage medium are proposed.

[0085] Existing compressors cannot reconcile the trade-off between high compression ratios and high inference speeds. In real-world operational platforms, data is updated frequently, has multiple dimensions, and comes from complex sources, placing stringent demands on compressor throughput. While current mainstream deep learning compressors have achieved good compression ratios on certain text or genomic datasets, their compression speeds are often as low as KB / s due to their complex recurrent structures and massive networks, making it difficult to support the real-time requirements of the platform. The first problem this patent aims to solve is how to significantly improve model inference speed through structural optimization and sparse modeling while maintaining compression ratios, enabling it to achieve industrial-grade compression throughput.

[0086] Current models often contain a large number of redundant nodes and invalid connections, leading to significant resource waste. Traditional network designs often prioritize extreme compression while neglecting structural simplicity, resulting in excessive unnecessary computation and model storage overhead, hindering the widespread application of these models on resource-constrained devices. This patent addresses this issue by using dynamic training and regular pruning mechanisms to automatically identify and remove invalid nodes, forming a minimal expressible structure. This significantly reduces the size of model parameters and memory usage, enabling lightweight deployment.

[0087] Existing compression methods lack adaptability across data types. Enterprise digital platforms contain diverse data, including time-series monitoring values, business reports, natural language descriptions, and log structures, exhibiting significant multimodal and highly dynamic characteristics. However, most deep learning models are only trained and optimized for specific data formats, resulting in poor portability and scalability, and failing to effectively handle data streams from different business lines. This application proposes a byte-based modeling unit structure, combined with hierarchical pruning and curve analysis, to achieve minimal structure selection and compression quality control for the compressor across multiple data types.

[0088] In one exemplary embodiment, such as Figure 1 As shown, a data compression method is provided. Taking the application of this method to a graphics processing unit (GPU) as an example, the method includes the following steps:

[0089] S110: Obtain the data to be compressed and perform serialization processing on the data to be compressed to obtain the byte sequence corresponding to the data to be compressed.

[0090] The data to be compressed can be any type of data in the enterprise's digital platform. For example, the data to be compressed can include time-series monitoring values, business reports, natural language descriptions, and log structures.

[0091] Specifically, in this embodiment, the data to be compressed can be format normalized and uniformly encoded into a standard byte sequence to obtain the byte sequence corresponding to the data to be compressed.

[0092] This process not only eliminates structural differences between raw inputs such as text formats, log records, and tabular data, but also ensures that subsequent probabilistic prediction models can handle multi-source heterogeneous data without discrimination. Through this preprocessing step, integrated fusion of enterprise-wide operational data is achieved before compression, laying a unified foundation for subsequent intelligent modeling and coding processing.

[0093] S120: Input the byte sequence into the pre-trained probability prediction model to obtain the conditional probability distribution of the data to be compressed.

[0094] Among them, the probability prediction model is used to process byte sequences and predict the probability distribution corresponding to each byte.

[0095] The specific processing logic of the probabilistic prediction model for byte sequences is described in the following embodiments and will not be repeated here.

[0096] S130, input the conditional probability distribution of the data to be compressed into the entropy encoder ANS to obtain the compression result of the data to be compressed.

[0097] Specifically, in this embodiment, the conditional probability distribution of the data to be compressed is input into the ANS. The ANS uses the statistical characteristics of the probability distribution to dynamically adjust the encoding interval and generate a bit stream without redundancy, which is the compression result.

[0098] The pre-trained probability prediction model is trained in the following way:

[0099] Each sample byte sequence is input into the initial probabilistic prediction model to obtain the prediction result corresponding to each sample byte sequence; the initial loss value is determined based on the prediction result and distribution label corresponding to each sample byte sequence; the target loss value and the network to be pruned in the initial probabilistic prediction model are determined based on the initial loss value and the regularization penalty term; the network to be pruned in the initial probabilistic prediction model is pruned to update the initial probabilistic prediction model, and the parameters of the updated initial probabilistic prediction model are adjusted based on the target loss value.

[0100] The initial probability prediction model is a traditional neural network predictor. The prediction result corresponding to each sample byte sequence is the prediction result corresponding to the sample data, i.e., the conditional probability distribution. The regularization penalty term can be a preset regularization penalty term with a corresponding preset regularization factor. The network to be pruned can be understood as the network in the initial probability prediction model that does not play a significant role in probability prediction; therefore, it can be removed to reduce the complexity of the model structure.

[0101] In the aforementioned data compression method, an initial loss value and a regularization penalty term are used to determine the target loss value and the networks to be pruned in the initial probabilistic prediction model. The networks to be pruned in the initial probabilistic prediction model are then pruned to update the model. Based on the target loss value, the parameters of the updated model are adjusted to obtain a pre-trained probabilistic prediction model. The compressed data is then processed based on this pre-trained model to obtain the compressed result. In this process, because some networks with limited contribution to probability prediction are removed from the pre-trained model, its structural complexity is lower, thus improving compression efficiency. Therefore, the aforementioned data compression method can improve compression efficiency while maintaining compression effectiveness.

[0102] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment. In this optional embodiment, the process of determining the target loss value and the network to be pruned in the initial probability prediction model based on the initial loss value and the regularization penalty term is refined.

[0103] See Figure 2 The steps for determining the network to be pruned, as shown, include:

[0104] S210, based on the regularization penalty term, corrects the initial loss value to obtain the target loss value and the weight coefficients corresponding to each network group in the probabilistic prediction model.

[0105] Each network group includes a predetermined number of network layers. The target loss value is used to adjust the parameters of the probabilistic prediction model.

[0106] To minimize redundant structure in the model and improve inference efficiency while ensuring compression accuracy, this scheme introduces sparsity constraints based on group Lasso regularization terms, also known as regularization penalty terms.

[0107] In an optional embodiment, an objective function can be constructed with the weight coefficients and initial loss values ​​corresponding to each network group as independent variables and the target loss value as the dependent variable; the objective function is solved with the goal of minimizing the target loss value and having a network layer whose weight coefficients approach zero, to obtain the target loss value and the weight coefficients corresponding to each network group.

[0108] The division of network groups can be determined based on human experience; for example, it can be done by row or column. For example, the objective function can be as follows:

[0109] ;

[0110] In the formula, L represents the target loss; L represents the initial loss. This indicates the grouping of weight parameters (e.g., dividing a fully connected layer by columns). Regularization factor represents the regularization factor in the regularization penalty term. Each group Describes a neural network structure whose deletion can significantly accelerate the model; w represents a network group; in The surrounding area has a continuous and non-zero gradient, which can induce a local minimum to approach zero.

[0111] S220, Based on the relationship between the weight coefficients of each network group and the preset weight threshold, determine the network to be pruned in the initial probability prediction model.

[0112] The preset weight threshold can be determined based on human experience, and this application does not impose any restrictions on it.

[0113] For example, the network group corresponding to the weight coefficients that are less than a preset weight threshold can be used as the network to be pruned in the initial probability prediction model.

[0114] In another alternative embodiment, the root mean square (RMS) of the weights for each network group can be determined, and the RMS is compared with a preset RMS threshold. The network groups with RMS values ​​less than the preset RMS threshold are selected as the networks to be pruned in the initial probabilistic prediction model. The preset RMS threshold can be set based on the stable noise level of the Adam optimizer to ensure that the pruning process does not impair the network's primary learning ability.

[0115] For example, the root mean square weights of each network group can be determined based on the following formula:

[0116] ;

[0117] In the formula, denoted by , where represents the root mean square of the weights of network group g; w represents a specific network group.

[0118] If a certain network group Less than a preset root mean square threshold, for example, 10 -3 If the network group has a very low contribution to the overall prediction effect, it can be considered as a network to be pruned in the initial probabilistic prediction model and removed from the initial probabilistic prediction model.

[0119] The above embodiments provide a specific process for determining the networks to be pruned in the initial probability prediction model. Network groups that contribute very little to the prediction effect are selected as the networks to be pruned, so that the network structure is simplified without compromising the network's prediction ability.

[0120] Furthermore, the network structure is significantly simplified after pruning, retaining only a few critical paths. To verify the predictive ability of the pruned probabilistic prediction model, a fine-tuning training strategy with small rounds can be used based on the target loss value to fine-tune the probabilistic prediction model and restore any lost predictive ability.

[0121] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment. In this optional embodiment, such as... Figure 3 As shown, the pre-trained probability prediction model includes a byte embedding layer, a fully connected layer, and an output layer. The fully connected layer is connected to the byte embedding layer and the output layer, respectively. Correspondingly, the process of inputting the byte sequence into the pre-trained probability prediction model to obtain the conditional probability distribution of the data to be compressed is described in detail.

[0122] See Figure 4 The steps for determining the compression result shown include:

[0123] S410, input the byte sequence to the byte embedding layer to obtain the vectorized data corresponding to the data to be compressed.

[0124] The byte embedding layer is used to map high-dimensional sparse discrete bytes into low-dimensional dense vectors, i.e. vectorized data, to alleviate sparsity and improve context representation capabilities.

[0125] S420 inputs vectorized data into the fully connected layer to obtain context information corresponding to the data to be compressed.

[0126] In one alternative embodiment, such as Figure 5 As shown, the fully connected layer includes a first fully connected layer and a second fully connected layer. The second fully connected layer is connected to the first fully connected layer and the output layer, respectively. Accordingly, vectorized data can be input into the first fully connected layer to obtain nonlinear feature data corresponding to the data to be compressed. The nonlinear feature data can be input into the second fully connected layer to obtain context information corresponding to the data to be compressed. The context information is the result of the second fully connected layer performing high-order semantic abstraction processing on the nonlinear feature data.

[0127] Specifically, the two fully connected layers are denoted as FC1 (the first fully connected layer) and FC2 (the second fully connected layer). FC1 receives vectorized data as input, extracts basic nonlinear features, and forms the first-level representation. FC2 further models higher-order contextual information based on the output of FC1, providing support for the final output layer to generate semantic predictions.

[0128] S430 inputs context information to the output layer to obtain the conditional probability distribution of the data to be compressed.

[0129] In this embodiment, the probabilistic prediction model can use an autoregressive method to decompose the joint probability into a product of conditional probabilities:

[0130] ;

[0131] In the formula, i=1 represents the prior probability; the probability prediction model needs to output N distributions. Each by Represent each of the positive real numbers and normalize them to 1. This is given as long as all distributions are known. Given a real sequence x, the entropy encoder can encode x into a bitstream z, whose shortest code length is L, which can be determined based on the following formula:

[0132] ;

[0133] In the formula, N is the number of distributions output by the probability prediction model; Let i be the probability distribution; machine learning uses learnable parameters Let the above conditional probability be such that: .

[0134] Many predictors (such as feedforward neural networks, PAQ, CMIX) truncate the input to the first k symbols:

[0135] ;

[0136] In the formula, k is an adjustment parameter that represents the relevant length of the data.

[0137] The probabilistic prediction model provided in this embodiment can efficiently capture the context structure in the data to be compressed, forming a probabilistic predictor oriented towards optimal information entropy, which is significantly better than the traditional compression prediction method based on Markov chains.

[0138] In the above embodiments, the structure based on the autoregressive compressor serves as a structural reference for the probabilistic prediction model, making the conditional distribution probability output by the prediction model more accurate and laying the foundation for subsequent data compression.

[0139] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment. In this optional embodiment, a probability prediction model such as... Figure 5 Taking the example shown, the process of training the probability prediction model in this application will be described in detail.

[0140] See Figure 6 The steps for determining the pre-trained probability prediction model shown include:

[0141] S601, serialize the sample data to obtain the sample byte sequence;

[0142] S602, input the byte sequence of each sample into the initial probability prediction model to obtain the vectorized data corresponding to the sample data;

[0143] S603, the vectorized data is input into the first fully connected layer to obtain the nonlinear feature data corresponding to the sample data;

[0144] S604, input the nonlinear feature data into the second fully connected layer to obtain the context information corresponding to the sample data;

[0145] S605, input the context information to the output layer to obtain the conditional probability distribution of the sample data;

[0146] S606, determine the initial loss value based on the prediction results and distribution labels corresponding to the byte sequences of each sample;

[0147] S607, construct the objective function with the weight coefficients and initial loss values ​​corresponding to each network group as independent variables and the target loss value as the dependent variable;

[0148] S608: With the goal of minimizing the target loss value and having a network layer whose weight coefficients approach zero, solve the objective function to obtain the target loss value and the corresponding weight coefficients of each network group.

[0149] S609, take the network group corresponding to the weight coefficients that are less than the preset weight threshold as the network to be pruned in the initial probability prediction model;

[0150] S610, prune the network to be pruned in the initial probability prediction model to update the initial probability prediction model, and adjust the parameters of the updated initial probability prediction model based on the target loss value.

[0151] It should be noted that after data compression using the method provided in this embodiment, during the data decoding process, the decoding end can select a suitable probability prediction model for inference calculation based on device performance. For example, only the main path structure (i.e., the structure remaining after pruning) can be loaded on the edge device; while the complete inference path (i.e., the unpruned structure) can be enabled on the server side, thereby achieving the platform scheduling capability of "on-demand inference and elastic decoding". This ensures that compression and decoding operations have good controllability and system adaptability in different business scenarios such as cloud management, edge analysis, and edge decision-making.

[0152] Based on the technical solutions of the above embodiments, this application also provides a data compression system, such as... Figure 7 As shown, the data compression system includes a graphics processing unit (GPU710) and an entropy encoder (ANS720). A probability prediction model is deployed in the GPU, which is the probability prediction model provided in this application.

[0153] In this embodiment, a GPU is used for high-concurrency batch inference, and the predicted probability distribution is transformed into the final compressed bitstream (i.e., the compressed result) through an ANS (Asymmetric Numeral System) encoder. This entropy coding method is easier to implement in parallel than traditional arithmetic coding, and has higher speed and coding efficiency, achieving better compression ratio and throughput performance on multiple public datasets.

[0154] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0155] Based on the same inventive concept, this application also provides a data compression apparatus for implementing the data compression method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more data compression apparatus embodiments provided below can be found in the limitations of the data compression method described above, and will not be repeated here.

[0156] In one exemplary embodiment, such as Figure 8 As shown, a data compression device is provided, including: an acquisition module 810, a first determination module 820, and a compression module 830, wherein:

[0157] The acquisition module 810 is used to acquire the data to be compressed and perform serialization processing on the data to be compressed to obtain the byte sequence corresponding to the data to be compressed.

[0158] The first determining module 820 is used to input the byte sequence into a pre-trained probability prediction model to obtain the conditional probability distribution of the data to be compressed;

[0159] The compression module 830 is used to input the conditional probability distribution of the data to be compressed into the entropy encoder ANS to obtain the compression result of the data to be compressed.

[0160] In one exemplary embodiment, such as Figure 9 As shown, a model training apparatus is provided, comprising: a prediction module 910, a second determination module 920, a third determination module 930, and an update module 940, wherein:

[0161] The prediction module 910 is used to input the byte sequence of each sample into the initial probability prediction model to obtain the prediction result corresponding to each byte sequence of the sample.

[0162] The second determining module 920 is used to determine the initial loss value based on the prediction results and distribution labels corresponding to the byte sequences of each sample;

[0163] The third determination module 930 is used to determine the target loss value and the network to be pruned in the initial probability prediction model based on the initial loss value and the regularization penalty term.

[0164] The update module 940 is used to prune the network to be pruned in the initial probability prediction model in order to update the initial probability prediction model, and to adjust the parameters of the updated initial probability prediction model based on the target loss value.

[0165] In one embodiment, the third determining module 930 includes a first determining unit, used to correct the initial loss value based on the regularization penalty term to obtain the target loss value and the weight coefficients corresponding to each network group in the probabilistic prediction model; each network group includes a preset number of network layers; and a second determining unit, used to determine the network to be pruned in the initial probabilistic prediction model according to the relationship between the weight coefficients corresponding to each network group and the preset weight threshold.

[0166] In one embodiment, the first determining unit includes a first determining subunit, used to construct an objective function with the weight coefficients and initial loss values ​​corresponding to each network group as independent variables and the target loss value as the dependent variable; and a second determining subunit, used to solve the objective function with the goal of minimizing the target loss value and having a network layer with weight coefficients approaching zero, to obtain the target loss value and the weight coefficients corresponding to each network group.

[0167] In one embodiment, the second determining unit is specifically used to select the network group corresponding to the weight coefficients that are less than a preset weight threshold as the network to be pruned in the initial probability prediction model.

[0168] In one embodiment, the pre-trained probability prediction model includes a byte embedding layer, a fully connected layer, and an output layer, with the fully connected layer connected to both the byte embedding layer and the output layer. Correspondingly, the first determining module 820 includes a first processing unit for inputting a byte sequence into the byte embedding layer to obtain vectorized data corresponding to the data to be compressed; a second processing unit for inputting the vectorized data into the fully connected layer to obtain context information corresponding to the data to be compressed; and a third processing unit for inputting the context information into the output layer to obtain the conditional probability distribution of the data to be compressed.

[0169] In one embodiment, the fully connected layer includes a first fully connected layer and a second fully connected layer, with the second fully connected layer connected to both the first fully connected layer and the output layer. Correspondingly, the second processing unit includes a first processing subunit for inputting vectorized data into the first fully connected layer to obtain nonlinear feature data corresponding to the data to be compressed; and a second processing subunit for inputting the nonlinear feature data into the second fully connected layer to obtain context information corresponding to the data to be compressed. The context information is the result of the second fully connected layer performing high-order semantic abstraction processing on the nonlinear feature data.

[0170] Each module in the aforementioned data compression device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0171] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a data compression method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0172] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0173] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0174] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0175] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0176] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0177] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0178] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this application. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of this application. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data compression method, characterized in that, The method includes: Obtain the data to be compressed and perform serialization processing on the data to be compressed to obtain the byte sequence corresponding to the data to be compressed; The byte sequence is input into a pre-trained probability prediction model to obtain the conditional probability distribution of the data to be compressed; The conditional probability distribution of the data to be compressed is input into the entropy encoder ANS to obtain the compression result of the data to be compressed. The pre-trained probability prediction model is trained in the following way: Each sample byte sequence is input into the initial probability prediction model to obtain the prediction result corresponding to each sample byte sequence; The initial loss value is determined based on the prediction results and distribution labels corresponding to each sample byte sequence; Based on the initial loss value and the regularization penalty term, determine the target loss value and the network to be pruned in the initial probability prediction model; The network to be pruned in the initial probability prediction model is pruned to update the initial probability prediction model, and the parameters of the updated initial probability prediction model are adjusted based on the target loss value.

2. The method according to claim 1, characterized in that, The step of determining the target loss value and the network to be pruned in the initial probability prediction model based on the initial loss value and the regularization penalty term includes: Based on the regularization penalty term, the initial loss value is corrected to obtain the target loss value and the weight coefficients corresponding to each network group in the probability prediction model; each network group includes a preset number of network layers. Based on the relationship between the weight coefficients of each network group and the preset weight threshold, the networks to be pruned in the initial probability prediction model are determined.

3. The method according to claim 2, characterized in that, The step of correcting the initial loss value based on the regularization penalty term to obtain the target loss value and the weight coefficients corresponding to each network group in the probabilistic prediction model includes: A target function is constructed using the weight coefficients corresponding to each network group and the initial loss value as independent variables, and the target loss value as the dependent variable. The objective function is solved with the goal of minimizing the target loss value and finding a network layer whose weight coefficients approach zero, thereby obtaining the target loss value and the weight coefficients corresponding to each network group.

4. The method according to claim 2, characterized in that, The step of determining the network to be pruned in the initial probability prediction model based on the relationship between the weight coefficients corresponding to each network group and the preset weight threshold includes: The network groups corresponding to weight coefficients that are less than the preset weight threshold are used as the networks to be pruned in the initial probability prediction model.

5. The method according to any one of claims 1-4, characterized in that, The pre-trained probability prediction model includes a byte embedding layer, a fully connected layer, and an output layer, wherein the fully connected layer is connected to both the byte embedding layer and the output layer; correspondingly, the step of inputting the byte sequence into the pre-trained probability prediction model to obtain the conditional probability distribution of the data to be compressed includes: The byte sequence is input into the byte embedding layer to obtain the vectorized data corresponding to the data to be compressed; The vectorized data is input into the fully connected layer to obtain the context information corresponding to the data to be compressed; The context information is input into the output layer to obtain the conditional probability distribution of the data to be compressed.

6. The method according to claim 5, characterized in that, The fully connected layer includes a first fully connected layer and a second fully connected layer, wherein the second fully connected layer is connected to the first fully connected layer and the output layer, respectively; correspondingly, the step of inputting the vectorized data into the fully connected layer to obtain the context information corresponding to the data to be compressed includes: The vectorized data is input into the first fully connected layer to obtain the nonlinear feature data corresponding to the data to be compressed; The nonlinear feature data is input into the second fully connected layer to obtain the context information corresponding to the data to be compressed; the context information is the result of the second fully connected layer performing high-order semantic abstraction processing on the nonlinear feature data.

7. A data compression system, characterized in that, The data compression system includes a graphics processing unit (GPU) and an entropy encoder (ANS). The GPU is equipped with a probabilistic prediction model, which is a pre-trained probabilistic prediction model as described in any one of claims 1-6.

8. A data compression device, characterized in that, The device includes: An acquisition module is used to acquire the data to be compressed and to perform serialization processing on the data to be compressed to obtain the byte sequence corresponding to the data to be compressed. The first determining module is used to input the byte sequence into a pre-trained probability prediction model to obtain the conditional probability distribution of the data to be compressed; The compression module is used to input the conditional probability distribution of the data to be compressed into the entropy encoder ANS to obtain the compression result of the data to be compressed. The pre-trained probability prediction model is trained in the following way: The prediction module is used to input each sample byte sequence into the initial probability prediction model to obtain the prediction result corresponding to each sample byte sequence; The second determining module is used to determine the initial loss value based on the prediction results and distribution labels corresponding to each sample byte sequence; The third determining module is used to determine the target loss value and the network to be pruned in the initial probability prediction model based on the initial loss value and the regularization penalty term. The update module is used to prune the network to be pruned in the initial probability prediction model to update the initial probability prediction model, and to adjust the parameters of the updated initial probability prediction model based on the target loss value.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-6.

Images