Huffman coding data compression method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a Huffman coded data compression method and system, electronic equipment and a storage medium, and belongs to the technical field of data compression. According to the method, chaotic mapping is carried out on target compressed data to obtain a chaotic initial value sequence, feature extraction is carried out on the chaotic initial value sequence to obtain a data sequence feature vector, and then the optimal compression ratio of the target compressed data is predicted according to the data sequence feature vector. The character code length distribution in the Huffman coding table is dynamically adjusted according to the optimal compression ratio so as to adapt to data changes, the target compressed data is coded according to the adjusted Huffman coding table so as to be compressed, and the compression efficiency of the Huffman coding data is improved.

Description

Technical Field

[0001] The present application relates to the technical field of data compression, and in particular to a Huffman coding data compression method, system, electronic device and storage medium. Background Art

[0002] In large data transmission scenarios, data streams are usually dynamically changing, such as video streams, real-time communication data, etc. Although the traditional Huffman coding method can effectively compress static data sets, it has obvious limitations when processing dynamically changing data streams, resulting in low compression efficiency. Specifically, the Huffman coding method establishes a coding table based on the probability distribution of the initial data at the beginning of the encoding, and remains unchanged throughout the compression process. When the data stream distribution changes, the original coding table is no longer applicable, and the probability distribution needs to be recalculated and a new Huffman code needs to be established, resulting in a decrease in compression efficiency. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a Huffman coding data compression method, system, electronic device and storage medium, aiming to improve the Huffman coding data compression efficiency when processing dynamically changing data streams.

[0004] To achieve the above object, an embodiment of the present application provides a Huffman coding data compression method, comprising the following steps:

[0005] Get target compressed data;

[0006] Performing chaotic mapping on the target compressed data to obtain a chaotic initial value sequence;

[0007] Extracting features from the chaotic initial value sequence to obtain a data sequence feature vector;

[0008] Predicting the optimal compression ratio of the target compressed data according to the data sequence feature vector;

[0009] Adjusting the character code length allocation in the Huffman coding table according to the optimal compression ratio;

[0010] The target compressed data is encoded according to the adjusted Huffman coding table to compress the target compressed data.

[0011] In some embodiments, performing chaotic mapping on the target compressed data to obtain a chaotic initial value sequence comprises the following steps:

[0012] Performing chaotic mapping on the target compressed data by using a unimodal mapping function to obtain a chaotic sequence;

[0013] A segment of data is selected from the chaotic sequence as the chaotic initial value sequence.

[0014] In some embodiments, the extracting features of the chaotic initial value sequence to obtain a data sequence feature vector comprises the following steps:

[0015] Performing statistical analysis on the data distribution of the chaotic initial value sequence to obtain data distribution characteristics, wherein the data distribution characteristics include data mean, data variance and data entropy;

[0016] A data sequence feature vector is constructed according to the data distribution characteristics, data type and data amount.

[0017] In some embodiments, predicting the optimal compression ratio of the target compressed data according to the data sequence feature vector comprises the following steps:

[0018] Inputting the data sequence feature vector into an intelligent agent model based on deep reinforcement learning to obtain an optimal compression ratio of the target compressed data;

[0019] The agent model is obtained by the following steps:

[0020] Acquire a training data set, wherein the training data set includes a plurality of data samples, each of the data samples includes a plurality of compressed action data, each of the compressed action data includes a state, an action and a reward value, the state represents a data sequence feature vector, the action represents a character code length adjustment amount, and the reward value represents a compression ratio;

[0021] The training data set is used to train a deep Q network, and the trained deep Q network is used as an intelligent agent model.

[0022] In some embodiments, the compression ratio is obtained by the following steps:

[0023] Determining a first data volume of original data;

[0024] Calculate a second data amount of data obtained by compressing the original data based on the character code length adjustment amount;

[0025] Multiplying the difference between the first data amount and the second data amount by a balance coefficient to obtain a third data amount;

[0026] The third data amount is added to the inverse of the second data amount to obtain a compression ratio.

[0027] In some embodiments, the step of training a deep Q network using the training data set and using the trained deep Q network as an agent model comprises the following steps:

[0028] Initializing a target network, wherein the target network is used to calculate a predicted reward value for the next state;

[0029] Inputting the state of the sample data in the training data set into the target network to obtain a predicted reward value;

[0030] Update the target network according to the predicted reward value and the true reward value of the data sample in the training data set;

[0031] With the goal of maximizing the cumulative reward value, the policy network is updated according to the updated target network, and the updated policy network group is used as the agent model, wherein the policy network is used to select the next action.

[0032] In some embodiments, adjusting the character code length allocation in the Huffman coding table according to the optimal compression ratio comprises the following steps:

[0033] Determine a compression ratio difference according to the optimal compression ratio and the current compression ratio;

[0034] Determine the character code length adjustment amount according to the compression ratio difference and the character occurrence frequency;

[0035] The character code length allocation in the Huffman coding table is adjusted according to the character code length adjustment amount.

[0036] To achieve the above object, another aspect of the embodiment of the present application provides a Huffman coding data compression system, comprising:

[0037] The first module is used to obtain target compressed data;

[0038] The second module is used to perform chaotic mapping on the target compressed data to obtain a chaotic initial value sequence;

[0039] The third module is used to extract features from the chaotic initial value sequence to obtain a data sequence feature vector;

[0040] A fourth module is used to predict the optimal compression ratio of the target compressed data according to the data sequence feature vector;

[0041] A fifth module, used for adjusting the character code length distribution in the Huffman coding table according to the optimal compression ratio;

[0042] The sixth module is used to encode the target compressed data according to the adjusted Huffman coding table to compress the target compressed data.

[0043] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the Huffman coding data compression method described in the above embodiment is realized.

[0044] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium used for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the Huffman coding data compression method described in the above embodiment.

[0045] The Huffman coding data compression method, system, electronic device and storage medium proposed in the present application perform chaotic mapping on target compressed data to obtain a chaotic initial value sequence, perform feature extraction on the chaotic initial value sequence to obtain a data sequence feature vector, and then predict the optimal compression ratio of the target compressed data based on the data sequence feature vector, dynamically adjust the character code length allocation in the Huffman coding table based on the optimal compression ratio to adapt to data changes, encode the target compressed data according to the adjusted Huffman coding table to compress the target compressed data, and improve the Huffman coding data compression efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flow chart of a Huffman coding data compression method provided by an embodiment of the present application;

[0047] Figure 2 yes Figure 1 Flow chart of step S102 in FIG.

[0048] Figure 3 yes Figure 1 Flow chart of step S103 in FIG.

[0049] Figure 4 is a flow chart of the agent model training method provided in an embodiment of the present application;

[0050] Figure 5 yes Figure 4 Flow chart of step S401 in FIG.

[0051] Figure 6 yes Figure 4 Flow chart of step S402 in FIG.

[0052] Figure 7 yes Figure 1 Flow chart of step S105 in FIG.

[0053] Figure 8 It is a schematic diagram of a Huffman coding data compression system provided in an embodiment of the present application;

[0054] Fig. 9 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0055] Fig.10 It is a schematic diagram of the overall concept of dynamic coding compression provided by an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] It should be noted that, although the functional modules are divided in the system and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0059] First, some nouns involved in this application are analyzed:

[0060] DRL (Deep Reinforcement Learning) is a technology that combines deep learning and reinforcement learning. Deep reinforcement learning introduces neural networks to represent states, actions, and rewards, thereby achieving better model performance and generalization capabilities. The core idea of ​​deep reinforcement learning is to use deep neural networks to approximate value functions or policy functions, combined with reinforcement learning algorithms for decision-making and learning. The deep learning part uses multi-layer neural networks to learn data representation and abstract machine learning models, which can process large amounts of data and learn complex nonlinear relationships. The reinforcement learning part allows the agent to learn the optimal behavior strategy through interaction with the environment. The machine learning method usually includes an agent, an environment, a reward system, and a policy.

[0061] Huffman coding, also known as Huffman coding, is an entropy coding (weight coding) algorithm for lossless data compression, proposed by David Huffman in 1952. Huffman coding is a variable length coding (VLC), which constructs the codeword with the shortest average length of different letter heads based entirely on the probability of character occurrence, sometimes called the best code. Its basic idea is: characters with high frequency of occurrence use shorter codes, and conversely, characters with low frequency of occurrence use longer codes, so as to achieve the purpose of compressing data.

[0062] DQN (Deep Q-Network) is an algorithm that combines deep learning and reinforcement learning. This algorithm is mainly used to solve optimization problems in the decision-making process, especially in an environment with a large number of possible states and actions. The core concept of DQN is to combine deep neural networks (DNN) with Q-learning algorithms. In Q-learning, the agent learns a Q-value function by interacting with the environment. This function estimates the future rewards that can be obtained by taking an action in a given state.

[0063] Chaotic mapping: It is a random motion state directly obtained from a deterministic equation. This deterministic equation is called a chaotic mapping. Although chaotic motion evolves from a deterministic equation, its characteristics show a high degree of randomness and unpredictability.

[0064] Logistic mapping, also known as Logistic function or Logistic equation, also known as parabola mapping or unimodal mapping, is a quadratic polynomial mapping (recursive relation), specifically in the form of x(t+1)=μx(t)(1-x(t)), where x(t) represents the state value at time step t, usually between [0,1]; μ is an adjustable parameter (also known as control parameter or branch parameter), in order to ensure that the mapped x(t) is always within [0,1], μ∈[0,4]; t is the iteration time step. Logistic mapping can perform nonlinear mapping on the input, introduce nonlinear factors, and better simulate complex relationships and behaviors. Generally, when 3.5699456<μ≤4, the system enters a chaotic state, showing the characteristics of non-periodicity, non-convergence, and extreme sensitivity to initial conditions. Therefore, by adjusting the parameter μ and the initial value x0, the transition from a stable point to a chaotic state can be observed, which makes Logistic mapping an important tool for studying nonlinear dynamics and chaos theory.

[0065] In the scenario of large data transmission, data streams are usually dynamically changing, such as video streams, real-time communication data, etc. Although the traditional Huffman coding method can effectively compress static data sets, it has obvious limitations when processing dynamically changing data streams, resulting in low compression efficiency. Specifically, first, Huffman coding is a static coding table. At present, the data compression method of Huffman coding will establish a coding table according to the probability distribution of the initial data at the beginning of coding, and remain unchanged throughout the compression process. When the data stream changes, the original coding table is no longer applicable. If the coding table continues to be used for compression, data loss will occur. Since it is currently impossible to adapt to changes in data distribution, in order to reduce data loss, it is necessary to re-establish a new Huffman coding table based on the changed data, resulting in slower compression efficiency. In general, the data compression method of Huffman coding in the related art has poor compression effect when facing dynamically changing data streams, and lacks a real-time coding table adjustment mechanism, resulting in an inability to respond in time when the data stream changes.

[0066] Based on this, the embodiments of the present application provide a Huffman coding data compression method, system, electronic device and storage medium, which aim to improve the Huffman coding data compression efficiency when processing dynamically changing data streams.

[0067] The Huffman coding data compression method, system, electronic device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the Huffman coding data compression method in the embodiments of the present application is described.

[0068] The Huffman coding data compression method provided in the embodiment of the present application relates to the field of data compression technology. The Huffman coding data compression method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, or can be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the Huffman coding data compression method, etc., but is not limited to the above forms.

[0069] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0070] Figure 1 is an optional flow chart of the Huffman coding data compression method provided in the embodiment of the present application, Figure 1 The method may include but is not limited to steps S101 to S106.

[0071] Step S101, obtaining target compressed data;

[0072] Step S102, performing chaotic mapping on the target compressed data to obtain a chaotic initial value sequence;

[0073] Step S103, extracting features from the chaotic initial value sequence to obtain a data sequence feature vector;

[0074] Step S104, predicting the optimal compression ratio of the target compressed data according to the data sequence feature vector;

[0075] Step S105, adjusting the character code length allocation in the Huffman coding table according to the optimal compression ratio;

[0076] Step S106, encoding the target compressed data according to the adjusted Huffman coding table to compress the target compressed data.

[0077] In steps S101 to S106 shown in the embodiment of the present application, a chaotic initial value sequence is obtained by performing chaotic mapping on the target compressed data, and features are extracted from the chaotic initial value sequence to obtain a data sequence feature vector. Then, the optimal compression ratio of the target compressed data is predicted based on the data sequence feature vector, and the character code length allocation in the Huffman coding table is dynamically adjusted based on the optimal compression ratio to adapt to data changes. The target compressed data is encoded according to the adjusted Huffman coding table to compress the target compressed data, thereby improving the Huffman coding data compression efficiency.

[0078] In step S101 of some embodiments, the target compressed data refers to one of the data (such as a frame of data or a data packet in a video) in a data stream (such as a video stream or a network transmission data packet) that needs to be compressed. The type of the target compressed data can be text data or image data in a video. Taking the target compressed data as image data in a video as an example, when compressing video data, it is necessary to compress each frame of the image in the video. Most images often contain large-area blocks of monochrome, and some colors appear more frequently than other colors. The Huffman coding method is to first scan the image data, calculate the probability of occurrence of various pixels, and specify unique code words of different lengths according to the size of the probability, thereby obtaining a Huffman code encoding table for the image. The encoded image data records the code word of each pixel, and the corresponding relationship between the code word and the actual pixel value is recorded in the Huffman code encoding table. Encoding the image data based on the Huffman code encoding table can obtain a larger image compression ratio. Since the image content in the video will change, after the initial Huffman coding table of the video is established, the embodiment of the present application needs to adjust the established Huffman coding table based on the changed image data.

[0079] In step S102 of some embodiments, a chaotic map is used to generate a chaotic sequence, and a chaotic map is a random sequence generated by a simple deterministic system. The chaotic map has the characteristics of initial value sensitivity, ergodicity and pseudo-randomness. Through the chaotic map, the initial deterministic data can be converted into a chaotic sequence with randomness. The increase of this randomness helps to break the correlation between the data, making the data more difficult to predict, thereby improving the data compression efficiency. The embodiment of the present application can use the method of Chebyshev chaotic map, Gauss / mouse chaotic map, Bernoulli chaotic map or Tent-Logistic-Cosine chaotic map to perform chaotic mapping on the target compressed data, and a chaotic initial value sequence can be obtained based on the target compressed data after chaotic mapping.

[0080] See also Figure 2 In some embodiments, step S102 may include but is not limited to steps S201 to S202:

[0081] Step S201, using a unimodal mapping function to perform chaotic mapping on target compressed data to obtain a chaotic sequence;

[0082] Step S202, selecting a segment of data from the chaotic sequence as a chaotic initial value sequence.

[0083] In this embodiment, a chaotic map is used to perform a preliminary transformation on the original data to obtain a chaotic sequence. The chaotic mapping method can be determined according to the characteristics of the data that need to be chaotically processed. For example, the embodiment of the present application can use Logistic mapping to perform chaotic processing on the image data in the video. When the image data is chaotically mapped using a unimodal mapping function, the image data can be regarded as a distribution of population density, where the gray value or color intensity of each pixel represents the population density at that position. Then, the image data can be iterated by adjusting the value of the μ parameter and applying the unimodal mapping function to produce a chaotic mapping effect. After a frame of image data (i.e., target compressed data) is chaotically mapped using a unimodal mapping function, a chaotic sequence composed of data with different mapping effects can be obtained, and then a segment of data is randomly selected from the chaotic sequence as a chaotic initial value sequence.

[0084] In some examples, during the compression process of the video stream, chaotic mapping can be performed on each frame of image data in the video stream to generate a corresponding chaotic initial value sequence, so that the encoding table of each frame of image data can be dynamically adjusted.

[0085] In step S103 of some embodiments, after obtaining the chaotic initial value sequence of the target compressed data, feature extraction can be performed on the chaotic initial value sequence, specifically, the mean μ, variance σ 2 Or statistical features such as data entropy H, or other features that may affect the compression effect, such as data type data type, data size data size, etc. The features extracted from the chaotic initial value sequence are combined to obtain the data sequence feature vector. For example, the data sequence feature vector is represented by X, X = [μ, σ 2 ,H,data type,data size,…].

[0086] See also Figure 3 In some embodiments, step S103 may include but is not limited to steps S301 to S302:

[0087] Step S301, statistically analyzing the data distribution of the chaotic initial value sequence to obtain data distribution characteristics, wherein the data distribution characteristics include data mean, data variance and data entropy;

[0088] Step S302, constructing a data sequence feature vector according to data distribution characteristics, data type and data volume.

[0089] In this embodiment, the mean, variance, data entropy, etc. in the data sequence can be determined by statistical analysis. Taking image data as an example, all pixel values ​​in the image data are added and then divided by the number of pixels in the image data to obtain the data mean; the deviation of each pixel value in the image data from the mean is calculated, and each deviation is squared to obtain the square value of the deviation; the average values ​​of all deviations are added and divided by the number of pixel values ​​to obtain the data variance; the probability of each pixel value appearing in the image data is counted, and then according to the formula H(X)=-∑p(x i )log(p(x i )) Calculate the data entropy H(X), where p(x i ) is the probability of the i-th gray value in the image.

[0090] In this embodiment, the data type may be one of the data precision types such as uint8 (unsigned 8-bit integer), uint16 (unsigned 16-bit integer), float32 (32-bit floating point number) or double (double-precision floating point type). The data sequence feature vector may be obtained by combining features such as data distribution features, data types and data volume.

[0091] In step S104 of some embodiments, the constructed compression ratio prediction model can be used to predict the optimal compression ratio corresponding to the feature vector of the data sequence, wherein the compression ratio prediction model can characterize the mapping relationship between the feature vector of the data sequence and the optimal compression ratio of the data. The compression ratio prediction model can be obtained by training a neural network using a deep learning algorithm, and the neural network can be a convolutional neural network, a recursive neural network, a long short-term memory network, or a residual network. Further, the compression ratio prediction model can also be a deep Q network, which is further trained using reinforcement learning on the basis of deep learning. In Q learning, the agent learns a Q value function by interacting with the environment. This function can estimate the future return (i.e., reward value) that can be obtained by taking a certain action in a given state. Based on the calculated reward value, the agent can be guided to select an action to obtain the optimal reward value, thereby obtaining the compression ratio represented by the optimal action, which is the optimal compression ratio.

[0092] In some embodiments, step S104 may include but is not limited to the following steps:

[0093] The data sequence feature vector is input into the intelligent agent model based on deep reinforcement learning to obtain the optimal compression ratio of the target compressed data.

[0094] See also Figure 4 , the above-mentioned agent model is obtained through but not limited to step S401 to step S402:

[0095] Step S401, obtaining a training data set, wherein the training data set includes a plurality of data samples, each data sample includes a plurality of compressed action data, each compressed action data includes a state, an action and a reward value, the state represents a data sequence feature vector, the action represents a character code length adjustment amount, and the reward value represents a compression ratio;

[0096] Step S402: Use the training data set to train the deep Q network, and use the trained deep Q network as the intelligent agent model.

[0097] In this embodiment, the compression ratio prediction model may be an intelligent agent model based on deep reinforcement learning, for example, a model obtained by deep Q network training. The training process of the intelligent agent model is as follows:

[0098] First, define the state space and action space. The state space S represents the set of all possible states, which includes the statistical characteristics of the data, the results of the pre-order compression, and other information. t}, where X t Represents the data sequence feature vector at time t. State s belongs to a specific state in the state space S, that is, the data sequence feature vector X describes a set of numerical features of state s. Action space A represents the set of all possible actions, which includes operations such as changing the code length allocation in the Huffman coding table. Action a belongs to a specific action in action space A. Action a can be a vector, representing the code length adjustment amount of different data symbols.

[0099] A reward function R is defined. The reward function R is used to calculate the reward value obtained after performing a certain action. In this embodiment, the ratio of the compressed data size to the original data size is used as the reward value, and the optimization goal is to minimize the ratio.

[0100] Then construct a training data set. The training data set of this embodiment may include two parts, one part is historical sample data, and the other part is real-time sample data obtained from the environment. The historical sample data is used to train the agent model offline using a deep learning algorithm, and the real-time sample data is used to update the agent model online using a reinforcement learning algorithm to improve the adaptability of the agent model in processing different data streams and improve the prediction accuracy of the optimal compression ratio. The agent continuously learns by interacting with the environment and adjusts its strategy to maximize the cumulative reward. The training data set contains a large number of data samples. Each data sample includes multiple compressed action data, and the format of each action compressed data can be state-action-reward value, that is, each data sample contains a series of state-action-reward sequences, and the data sample format can be expressed as: {(s1, a1, r1), (s2, a2, r2)...}, where r belongs to a specific reward sequence in the reward function R.

[0101] See also Figure 5In some embodiments, the compression ratio of step S401 may be obtained by, but not limited to, steps S501 to S504:

[0102] Step S501, determining a first data volume of original data;

[0103] Step S502, calculating a second data amount of data obtained by compressing the original data based on the character code length adjustment amount;

[0104] Step S503, multiplying the difference between the first data volume and the second data volume by the balance coefficient to obtain a third data volume;

[0105] Step S504: Add the third data amount to the inverse of the second data amount to obtain a compression ratio.

[0106] In this embodiment, the reward value in the agent model represents the compression ratio, which can be calculated using a defined reward function. The reward function R defines the instantaneous reward obtained under state s and action a. t ,a t ) can be expressed as:

[0107]

[0108] Among them, h b Indicates the first data volume of the original data, h a represents the second data volume of the data obtained after compressing the original data, α is a balance coefficient, which is used to adjust the weight between the two objectives, and is determined based on experiments or experience. For example, the balance coefficient may be 0.5. The compression ratio of this implementation can be calculated using the above formula.

[0109] See also Figure 6 In some embodiments, step S402 includes but is not limited to steps S601 to S604:

[0110] Step S601, initializing a target network, wherein the target network is used to calculate a predicted reward value for the next state;

[0111] Step S602, inputting the state of the sample data in the training data set into the target network to obtain a predicted reward value;

[0112] Step S603, updating the target network according to the predicted reward value and the actual reward value of the data sample in the training data set;

[0113] Step S604, with the goal of maximizing the cumulative reward value, the policy network is updated according to the updated target network, and the updated policy network is used as the agent model, wherein the policy network is used to select the next action.

[0114] In this embodiment, the embodiment of the present application defines the physical meaning of the reward value representation by defining a reward function as the compression ratio before and after data compression. The purpose of this implementation of training the agent model is to enable the agent model to determine the next action a based on the feedback of the reward function R in the previous step. If the value of R(s,a) is high, it means that the compression effect is good, then the agent may continue to take similar actions, or make slight adjustments to explore better results. If the value of R(s,a) is low, it means that the compression effect is not good, and the agent will try other different actions to improve the compression effect. For example, if the value of R(s,a) is low because the compressed data size is too large, the agent may choose to increase the code length of certain symbols in the hope of further reducing the compressed file size. The agent continuously adjusts its strategy based on the feedback of R(s,a) to maximize the cumulative reward, that is, to maximize the compression effect. In other words, the role of the reward function R is to guide the agent on how to choose the next action a in order to achieve the optimal compression effect. Based on this, an embodiment of the present application constructs a target network, which can predict the amount of data obtained by performing a certain compression action, and then predict the predicted reward value (i.e., the predicted compression ratio) after performing a certain compression action in combination with the reward function. Furthermore, the policy network is updated according to the action evaluation feedback of the target network, so that the policy network can select a series of actions that meet the goal of maximizing the cumulative reward value. The optimal compression ratio can be calculated by simulating the compression operation of the data through this action.

[0115] Specifically, the agent learns how to use the current state s in the reinforcement learning process t The action a is performed t The reward value R(s) t ,a t ) to update its strategy and choose the next action a t+1 , specifically expressed as follows:

[0116]

[0117] Among them, a t is the action taken by the agent at time step t, ω is the learning rate, which is a hyperparameter used to control the speed at which the agent updates its strategy based on the reward. A smaller value means a more conservative update, and a larger value means a more aggressive update.

[0118] The reward threshold θ is a pre-set value used to compare with the actual reward value R(s t ,a t ) to decide whether to update the strategy. If R(s t ,a t ) is greater than θ, the agent may continue with the current strategy, otherwise it may need to explore new strategies.

[0119] Gradient of the reward function It is the partial derivative of the reward function with respect to action a, which indicates the rate of change of the reward when a small change is made to the action under the current state. This gradient guides the agent on how to adjust the action to increase the reward. In practical applications, this is usually approximated by numerical methods or automatic differentiation techniques.

[0120] The exploration rate ∈ is another hyperparameter that controls the balance between exploring new actions and exploiting known strategies. When ∈ is close to 0, the agent is more inclined to exploit known strategies; when ∈ is larger, the agent is more inclined to explore. ∈ is usually gradually reduced as the time step t increases to reduce exploration and increase the proportion of exploiting known strategies.

[0121] The random exploration term ν is a random value drawn from a probability distribution (e.g., a normal distribution) that represents the randomness introduced by the agent when updating its policy to promote exploration. The variance of this random value usually decreases as the learning process progresses.

[0122] This embodiment introduces a reward function that takes into account the impact of code length adjustment, thereby improving the intelligence level of the compression algorithm.

[0123] In step S105 of some embodiments, after predicting the optimal compression ratio of the target compressed data, the character code length allocation in the Huffman coding table can be dynamically adjusted according to the optimal compression ratio. In this embodiment, the code length allocation in the Huffman coding table is dynamically adjusted through deep reinforcement learning, so that the compression strategy can adapt to the changes in the data stream and improve the compression efficiency.

[0124] See also Figure 7 In some embodiments, step S105 may include but is not limited to steps S701 to S703:

[0125] Step S701, determining a compression ratio difference according to the optimal compression ratio and the current compression ratio;

[0126] Step S702, determining the character code length adjustment amount according to the compression ratio difference and the character occurrence frequency;

[0127] Step S703: adjusting the character code length distribution in the Huffman coding table according to the character code length adjustment amount.

[0128] In this embodiment, the character code length adjustment amount Δl of the i-th character l is i It can be calculated by the following formula:

[0129] Δl i =β·(CD)·f(e);

[0130] Among them, β is an adjustment coefficient, which is determined according to experiments or experience, C is the optimal compression ratio of the predicted target compressed data, D is the compression ratio of the target compressed data corresponding to the current Huffman coding table, and f(e) is a non-linear function of the character occurrence frequency. Exemplarily, f(e) = log(1 + e).

[0131] Update the code length of the corresponding character in the Huffman coding table according to the character code length adjustment amount. The adjusted character code length is expressed as:

[0132]

[0133] Among them, represents the adjusted code length of the i-th character l, represents the code length of the i-th character l before adjustment, and Δl i represents the character code length adjustment amount of the i-th character l. The character code length adjustment amount may be a positive value (i.e., C > D) or a negative value (i.e., C < D). If it is a positive value, it means that the code length needs to be increased. If it is a negative value, it means that the code length needs to be shortened. Exemplarily, for image data, the character can be a certain pixel value in the image.

[0134] In some embodiments of step S106, the updated Huffman coding table is used to encode the data to generate the compressed data. The compression process can be expressed as: Compressed data = Huffman coding( Target compressed data).

[0135] According to some embodiments of the present application, please refer to Fig.10 , the embodiments of the present application illustrate the dynamic coding compression concept of video streams as follows:

[0136] First, data preprocessing, that is, chaotic sequence generation and data mapping.

[0137] Specifically, the Logistic mapping function can be selected to perform chaotic mapping on each frame of image data to generate the corresponding chaotic initial value sequence. Then, through feature extraction on the chaotic initial value sequence, a feature vector X is constructed, X = [μ, σ 2 , H, data type, data size,...].

[0138] Second, the construction of a deep reinforcement learning model. The deep reinforcement learning model can select a deep Q network, and define the state space, action space, and reward function of the deep Q network.

[0139] Third, based on the deep reinforcement learning model defined above, the sample data is processed. The input in the sample data can be processed using the above preprocessing method. A large amount of sample data is used to train the intelligent agent model and build a simulated compression environment. The model can predict the compression ratio of a given chaotic initial value sequence, that is, the intelligent agent model can predict the optimal compression ratio C based on the feature vector X.

[0140] Fourth, dynamic compression ratio prediction and coding table adjustment. The image data in the video is also preprocessed, that is, a section is selected from the chaotic sequence generated by the chaotic mapping as the chaotic initial value sequence, and the eigenvector of the sequence is calculated. The eigenvector of the image data is input into the trained intelligent agent model to obtain the optimal compression ratio C. Assume that the trained model predicts that the compression ratio of a frame of image data is C = 0.5. Dynamically adjust the code length allocation of each pixel character in the Huffman coding table, for example, the code length adjustment amount Δl i =0.01·(0.5-D)·log(1+e). Update the code length in the Huffman coding table to reflect Δl i The updated character code length is

[0141] Fifth, encoding and decoding. Use the adjusted Huffman coding table to encode and compress the data. Specifically, for the next frame of image data, incremental compression encoding is performed according to the adjusted coding table to generate compressed data, thereby achieving effective data compression. At the same time, the compression process is monitored and the coding strategy is adjusted in real time according to environmental feedback. The same coding table is used for decoding at the receiving end.

[0142] Sixth, evaluation and optimization. Performance evaluation indicators include compression ratio, compression speed, etc. The compression effect can be evaluated by combining performance evaluation indicators. According to the evaluation results, the intelligent agent model and encoding strategy are continuously adjusted until satisfactory compression performance is achieved.

[0143] The embodiment of the present application provides an adaptive Huffman incremental compression optimization method based on chaos theory and deep reinforcement learning, which is used to solve the problem of insufficient compression efficiency of traditional Huffman coding when processing dynamically changing data streams. The method generates a chaotic sequence by preprocessing data and extracts feature vectors from it, and then uses a deep reinforcement learning agent to solve the optimal coding strategy according to the characteristics of the real-time data stream. Specifically, the embodiment of the present application first uses chaotic mapping to increase the randomness of the data, and then constructs a deep reinforcement learning model, including defining the state space, action space, and designing a reward function, with the goal of minimizing the compressed file size. The agent learns the adjustment method of the optimal coding table by interacting with the environment, and dynamically adjusts the code length allocation in the Huffman coding table based on the predicted compression ratio. Therefore, it can effectively cope with changes in data streams and improve compression efficiency and adaptability.

[0144] See also Figure 8 The embodiment of the present application also provides a Huffman coding data compression system, which can implement the above-mentioned Huffman coding data compression method, including:

[0145] The first module is used to obtain target compressed data;

[0146] The second module is used to perform chaotic mapping on the target compressed data to obtain a chaotic initial value sequence;

[0147] The third module is used to extract features from the chaotic initial value sequence to obtain a feature vector of the data sequence;

[0148] The fourth module is used to predict the optimal compression ratio of the target compressed data according to the data sequence feature vector;

[0149] A fifth module is used for adjusting the character code length allocation in the Huffman coding table according to the optimal compression ratio;

[0150] The sixth module is used to encode the target compressed data according to the adjusted Huffman coding table to compress the target compressed data.

[0151] It can be understood that the contents of the above-mentioned Huffman coding data compression method embodiment are all applicable to the present system embodiment, the functions specifically implemented by the present system embodiment are the same as those in the above-mentioned Huffman coding data compression method embodiment, and the beneficial effects achieved are also the same as those achieved by the above-mentioned Huffman coding data compression method embodiment.

[0152] The embodiment of the present application also provides an electronic device, the electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the above-mentioned Huffman coding data compression method. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0153] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0154] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0155] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the Huffman coding data compression method of the embodiment of this application;

[0156] Input / output interface 903, used to implement information input and output;

[0157] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0158] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0159] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0160] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned Huffman coding data compression method.

[0161] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0162] The Huffman coding data compression method, device, electronic device and storage medium provided in the embodiment of the present application perform chaotic mapping on the target compressed data to obtain a chaotic initial value sequence, extract features from the chaotic initial value sequence to obtain a data sequence feature vector, and then predict the optimal compression ratio of the target compressed data according to the data sequence feature vector, dynamically adjust the character code length allocation in the Huffman coding table according to the optimal compression ratio to adapt to data changes, encode the target compressed data according to the adjusted Huffman coding table to compress the target compressed data, and improve the Huffman coding data compression efficiency. .

[0163] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0164] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0165] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0166] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0167] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0168] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0169] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0170] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0171] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0172] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0173] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. A Huffman coding data compression method, characterized in that: The following steps are involved: Get target compressed data; Performing chaotic mapping on the target compressed data to obtain a chaotic initial value sequence; Extracting features from the chaotic initial value sequence to obtain a data sequence feature vector; Predicting the optimal compression ratio of the target compressed data according to the data sequence feature vector; Adjusting the character code length allocation in the Huffman coding table according to the optimal compression ratio; The target compressed data is encoded according to the adjusted Huffman coding table to compress the target compressed data.

2. The Huffman coding data compression method according to claim 1, characterized in that: The method of performing chaotic mapping on the target compressed data to obtain a chaotic initial value sequence comprises the following steps: Performing chaotic mapping on the target compressed data by using a unimodal mapping function to obtain a chaotic sequence; A segment of data is selected from the chaotic sequence as the chaotic initial value sequence.

3. The Huffman coding data compression method according to claim 1, characterized in that: The feature extraction of the chaotic initial value sequence to obtain a data sequence feature vector comprises the following steps: Performing statistical analysis on the data distribution of the chaotic initial value sequence to obtain data distribution characteristics, wherein the data distribution characteristics include data mean, data variance and data entropy; A data sequence feature vector is constructed according to the data distribution characteristics, data type and data amount.

4. The Huffman coding data compression method according to claim 1, characterized in that: The step of predicting the optimal compression ratio of the target compressed data according to the data sequence feature vector comprises the following steps: Inputting the data sequence feature vector into an intelligent agent model based on deep reinforcement learning to obtain an optimal compression ratio of the target compressed data; The agent model is obtained by the following steps: Acquire a training data set, wherein the training data set includes a plurality of data samples, each of the data samples includes a plurality of compressed action data, each of the compressed action data includes a state, an action and a reward value, the state represents a data sequence feature vector, the action represents a character code length adjustment amount, and the reward value represents a compression ratio; The training data set is used to train a deep Q network, and the trained deep Q network is used as an intelligent agent model.

5. The Huffman coding data compression method according to claim 4, characterized in that: The compression ratio is obtained by the following steps: Determining a first data volume of original data; Calculate a second data amount of data obtained by compressing the original data based on the character code length adjustment amount; Multiplying the difference between the first data amount and the second data amount by a balance coefficient to obtain a third data amount; The third data amount is added to the inverse of the second data amount to obtain a compression ratio.

6. The Huffman coding data compression method according to claim 4, characterized in that: The method of training a deep Q network using the training data set and using the trained deep Q network as an intelligent agent model comprises the following steps: Initializing a target network, wherein the target network is used to calculate a predicted reward value for the next state; Inputting the state of the sample data in the training data set into the target network to obtain a predicted reward value; Update the target network according to the predicted reward value and the true reward value of the data sample in the training data set; With the goal of maximizing the cumulative reward value, the policy network is updated according to the updated target network, and the updated policy network group is used as the agent model, wherein the policy network is used to select the next action.

7. The Huffman coding data compression method according to claim 1, characterized in that: The step of adjusting the character code length distribution in the Huffman coding table according to the optimal compression ratio comprises the following steps: Determine a compression ratio difference according to the optimal compression ratio and the current compression ratio; Determine the character code length adjustment amount according to the compression ratio difference and the character occurrence frequency; The character code length allocation in the Huffman coding table is adjusted according to the character code length adjustment amount.

8. A Huffman coding data compression system, characterized in that: include: The first module is used to obtain target compressed data; The second module is used to perform chaotic mapping on the target compressed data to obtain a chaotic initial value sequence; The third module is used to extract features from the chaotic initial value sequence to obtain a data sequence feature vector; A fourth module is used to predict the optimal compression ratio of the target compressed data according to the data sequence feature vector; A fifth module, used for adjusting the character code length distribution in the Huffman coding table according to the optimal compression ratio; The sixth module is used to encode the target compressed data according to the adjusted Huffman coding table to compress the target compressed data.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the Huffman coding data compression method as described in any one of claims 1 to 7 are realized.

10. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the Huffman coding data compression method according to any one of claims 1 to 7.

Images