A data intelligent compression method for twin systems

Abstract

The present invention discloses a data intelligent compression method for a twin system, which collects video, audio and text data for preprocessing; constructs a Bootstrap model to extract features and preliminarily compress the input data; constructs a Combined model composed of a Bootstrap model and a Supporter model, and combines the probability distribution output by the Bootstrap model with the probability distribution output by the Supporter model; uses an arithmetic coding method to finally compress the output probability distribution to reduce the volume of data transmission, and transmits the data to a target device; the target device decodes and reconstructs the received data to restore the original data before compression; the present invention improves the data compression ratio and compression time efficiency, thereby effectively reducing the delay of data transmission, adapting to various data types in the twin system, and ensuring real-time processing of data.

Description

Technical Field

[0001] The present invention relates to the technical field of computers and electronic information engineering, and in particular to a data intelligent compression method for twin systems. Background Art

[0002] In the digital twin system, the twin model realizes accurate monitoring, simulation and optimization of the real world by mapping the physical object in real time. Digital twin systems are widely used in many fields such as manufacturing, energy, smart cities, transportation, etc., which can improve operational efficiency, reduce failure rate, and provide higher flexibility and automation. Digital twin systems rely on the transmission and processing of large amounts of data to ensure real-time interaction between virtual models and physical entities. Data includes multiple formats such as video, audio, sensor data and text information, and requires extremely low data transmission latency. With the continuous increase in the amount of data, how to efficiently transmit this data while ensuring real-time performance has become one of the key challenges facing digital twin systems. In the process of meeting this challenge, data compression technology has been proven to be a reliable and effective solution. Data compression technology can effectively reduce the amount of data by reducing redundant information, thereby increasing transmission speed and reducing bandwidth usage. It is particularly important for digital twin systems that process diverse data and cope with complex environments. Especially in scenarios with low latency requirements, data compression technology can not only effectively meet these challenges, but also ensure the real-time and high efficiency of digital twin systems. Therefore, data compression technology is urgently needed to achieve low latency and high-efficiency data transmission. However, existing data compression methods are usually optimized for specific types of data. They are effective when processing a single type of data, but have poor adaptability when facing multiple data types, and the compression effect is significantly reduced, making it difficult to meet the environment's requirements for low-latency transmission.

[0003] In view of the above problems and current situation, the present invention proposes a universal data compression method based on neural network and arithmetic coding. By designing Bootstrap model and Combined model, combined with multi-head attention mechanism, position encoding and residual network, it aims to improve data compression ratio and compression speed, thereby effectively reducing data transmission delay, adapting to various data types in twin system, and providing an efficient data compression transmission solution for twin system construction. Summary of the invention

[0004] Purpose of the invention: In a twin system, data transmission faces the challenges of low latency and diversity, which seriously affects the real-time interaction of data between the twin system and the physical entity. Existing compression methods usually compress specific types of data, which are effective when processing a single type of data, but have poor adaptability when facing multiple data types, and the compression effect is significantly reduced. In addition, traditional compression methods are difficult to achieve efficient compression of diversified data in a rapidly changing complex environment. In response to the above problems, the present invention proposes a data intelligent compression method for twin systems, which aims to improve the data compression ratio and compression time efficiency by designing a Bootstrap model and a Combined model, combining a multi-head attention mechanism, position encoding and a residual network, thereby effectively reducing the delay of data transmission, adapting to a variety of data types in the twin system, and ensuring real-time processing of data.

[0005] To achieve the above functions, the present invention designs a data intelligent compression method for twin systems, and executes the following steps S1 to S5 to complete data compression and transmission:

[0006] Step S1: collecting video data, audio data and text data as data to be compressed, and performing preprocessing on the data to be compressed, including data standardization and format conversion, to obtain preprocessed data;

[0007] Step S2: Build and train the Bootstrap model, taking the preprocessed data as input, extracting features and preliminarily compressing the preprocessed data based on the position encoding module, bidirectional gated recurrent unit, multi-head attention mechanism, residual module and functional module, and outputting the probability distribution logits b ;

[0008] Step S3: Build and train the Combined model. The Combined model consists of the Bootstrap model and the Supporter model. The Supporter model is based on the position encoding module, multi-head attention mechanism, residual module and functional module, and outputs the probability distribution logits s , the Combined model outputs the probability distribution of the Bootstrap model logits b , and the probability distribution output by the Supporter model logits s Combined, strengthen the sequence feature representation of data and further compress it to generate probability distribution logits c ;

[0009] Step S4: finally compressing the probability distribution outputted in step S3 using an arithmetic coding method to reduce the volume of data transmission, and transmitting the data to the target device;

[0010] Step S5: On the target device, the received data is decoded and reconstructed using the pre-trained Bootstrap model, Combined model and arithmetic decoding method to restore the original data before compression.

[0011] Beneficial effects: Compared with the prior art, the advantages of the present invention include:

[0012] The present invention proposes a data intelligent compression method for twin systems, which is applicable to various types of data including text, video, audio and sensor data. First, through specific data preprocessing steps and general compression models, consistent and efficient compression is achieved between different types of data. Secondly, using Bootstrap and Combined models, combined with multi-head attention mechanism and residual block, efficient compression of diversified data is achieved, which has significant advantages in compression ratio and compression speed over traditional methods. At the same time, combined with the probability model of arithmetic coding, the high reliability of data compression is ensured, efficient lossless compression is achieved at a lower bit rate, and the loss and bit error rate of data during transmission are reduced. Finally, the method of the present invention adopts a standard neural network architecture and arithmetic coding algorithm, has good feasibility and scalability, can be easily integrated into the existing twin system, and can be customized and expanded according to specific needs. In summary, the general data compression method based on neural network and arithmetic coding of the present invention can not only adapt to a variety of data types, but also has significant advantages in compression efficiency, thereby significantly reducing the delay of data transmission, especially in complex environments, can ensure the rapid transmission and real-time processing of twin system data, and provide an efficient data compression transmission solution for twin system construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a flow chart of a data intelligent compression method for a twin system provided according to an embodiment of the present invention;

[0014] Figure 2 is a schematic diagram of a training process of a Bootstrap model provided according to an embodiment of the present invention;

[0015] Figure 3 is a structural diagram of a Combined model provided according to an embodiment of the present invention;

[0016] Figure 4 is a graph showing changes in training and test losses of the Bootstrap model provided by an embodiment of the present invention as a function of the number of iterations;

[0017] Figure 5 is a diagram showing changes in loss during the training process of the Combined model provided in an embodiment of the present invention;

[0018] Figure 6 It is a performance comparison chart of the method designed by the present invention and a series of compression algorithms on diverse data sets provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0020] An embodiment of the present invention provides a data intelligent compression method for a twin system, referring to Figure 1 , execute the following steps S1 to S5 to complete data compression and transmission:

[0021] Step S1: collecting video data, audio data and text data as data to be compressed, and preprocessing the data to be compressed to ensure that different types of data can be effectively processed, the preprocessing including data standardization and format conversion to obtain preprocessed data;

[0022] The method for preprocessing the data to be compressed in step S1 is as follows:

[0023] Inter-frame sampling is performed on the data frames of video data and audio data, and the sampling rate is reduced by the following formula:

[0024] ;

[0025] in, is the sampled data frame, S i Represents the first i frame, n is the sampling interval, i.e. n Take one frame at a time;

[0026] For the sampled video data and audio data, normalization processing is performed to unify the data scale, as shown in the following formula:

[0027] ;

[0028] in, x norm is the normalized data value, x is the original data value, min( x ) and max( x ) are the minimum and maximum values ​​in the data respectively;

[0029] For text data, characters are mapped to integers using the following formula:

[0030] ;

[0031] in, C int is the integer value after mapping, c is the original character, ord is a function that converts a character to its corresponding ASCII integer value.

[0032] Step S2: Build and train the Bootstrap model, taking the preprocessed data as input, and extracting and preliminarily compressing the preprocessed data based on the position encoding module, bidirectional gated recurrent unit, multi-head attention mechanism, residual module and functional module to optimize the data representation and improve the compression efficiency; the Bootstrap model outputs the probability distribution logits b ;

[0033] The Bootstrap model described in step S2 includes a position encoding module, a bidirectional gated recurrent unit, a multi-head attention mechanism, a residual module and a functional module in sequence;

[0034] The Bootstrap model inputs the preprocessed data, adds position information to the data through the position encoding module, and then uses the bidirectional gated recurrent unit (BiGRU) to output embedded features to capture the bidirectional dependencies of the data, further improving the model's ability to understand sequence data. After that, it undergoes a flattening operation and uses a multi-head attention mechanism to output attention scores to capture the complex relationships between the data. Through the residual module, the gradient vanishing problem in the deep network is solved. Finally, through two parallel functional modules, an unscaled probability distribution is generated. logits b , to complete the preliminary prediction, where the two parallel functional modules are the linear layer and the fully connected layer. The linear layer is first used to perform a preliminary transformation on the extracted features, and then the features are mapped to the output space through the fully connected layer.

[0035] Bootstrap model training process reference Figure 2. The Bootstrap model is trained by traversing the input data multiple times, optimizing parameters to minimize the cross entropy loss, and generating a high-quality symbolic probability prediction model. The Bootstrap model adapts to the needs of different data sets by automatically selecting hyperparameters, and uses multi-head self-attention mechanisms, position encoding, and residual modules to capture long-term dependencies and complex patterns of data. After training, the model parameters are saved as part of the compressed file, thereby providing an efficient prediction basis for the compression process of the data intelligent compression method for twin systems designed by the present invention.

[0036] Step S3: Build and train the Combined model. The Combined model consists of the Bootstrap model and the Supporter model. The Supporter model is based on the position encoding module, multi-head attention mechanism, residual module and functional module, and outputs the probability distribution logits s , the Combined model outputs the probability distribution of the Bootstrap model logits b , and the probability distribution output by the Supporter model logits s Combined, strengthen the sequence feature representation of data and further compress it to generate probability distribution logits c ;

[0037] In the Combined model described in step S3, the Supporter model inputs the preprocessed data, adds position information to the data through the position encoding module, splices the data with the added position information with the embedded features output by the bidirectional gated recurrent unit of the Bootstrap model, and outputs the attention score of the spliced ​​features through the multi-head attention mechanism to capture the complex relationship between the data. It passes through multiple residual modules in sequence to enhance the stability and training efficiency of the model, and finally passes through three parallel functional modules. The three parallel functional modules are linear modules, dense modules and residual modules. Each module acts as an independent predictor for learning features of different complexities. The output vector of each module is reduced to a dimension that matches the size of the vocabulary through linear transformation, and is weighted and summed at the end to generate a probability distribution. logits s ; The Supporter model can extract basic patterns, complex features and deep information respectively, thereby improving the accuracy of the overall prediction.

[0038] The Combined model combines the probability distributions output by the Bootstrap model and the Supporter model, and generates the final probability distribution through convex sum logits c ;

[0039] ;

[0040] in, λ is a learnable parameter constrained to the range of [0,1] by the sigmoid activation function. Through this combination, the Combined model significantly improves compression efficiency and prediction accuracy, ensuring the excellent performance of the model on various data sets.

[0041] Combined model structure reference Figure 3 In the Combined model, the Bootstrap model is combined with the randomly initialized Supporter model to generate the final symbol probability prediction through convex sum. At the beginning of the training phase of the Combined model, the Bootstrap model keeps its parameters trained in step S2 unchanged, while the Supporter model is adaptively updated as needed. The input data sequence is divided into multiple equal-sized parts and batch predictions are performed in parallel. The prediction results of the Combined model are converted into probability distributions through the softmax activation function and then input into the arithmetic encoder for symbol encoding. During the encoding process, the Supporter model continuously updates its parameters according to the actual symbol predictions and optimizes its prediction ability by minimizing the cross entropy loss. The multi-head self-attention mechanism, position encoding, and residual module further improve the adaptability and prediction accuracy of the model, ensuring that the model can quickly adapt to the non-stationary statistical characteristics of the sequence. Finally, the encoded symbols are input into the arithmetic encoder together with the probability predictions generated by the Combined model to generate an efficiently compressed file, thereby achieving a higher compression ratio and faster encoding speed.

[0042] The position encoding module processes data as follows:

[0043] ;

[0044] ;

[0045] in, , is the position-encoded data, POS is the sequence position, i is the dimension index, d is the model dimension.

[0046] The multi-head attention mechanism processes data as follows:

[0047] ;

[0048] Among them, Attention(Q,K,V) is the attention score, Q, K, V represent the query, key and value matrix respectively, d k It is the dimension of the key vector, which is used to scale the dot product to prevent the result after the dot product from being too large and affecting the gradient of softmax. Softmax is a normalization function used to convert the attention score into a probability distribution.

[0049] Step S4: finally compressing the probability distribution outputted in step S3 using an arithmetic coding method, where the arithmetic coding method is an efficient coding method based on the probability distribution of data to reduce the volume of data transmission and transmit the data to the target device;

[0050] The arithmetic coding method in step S4 is as follows:

[0051] ;

[0052] Among them, new_interval is the new encoding interval, low and high are the lower and upper bounds of the current interval, and probability is the probability of occurrence of the current symbol.

[0053] Step S5: On the target device, the received data is decoded and reconstructed using the pre-trained Bootstrap model, Combined model, and arithmetic decoding method to restore the original data before compression. The arithmetic decoding method and model reasoning work together in the decoding process to ensure the integrity and accuracy of the data during transmission and recovery.

[0054] Figure 4 The data intelligent compression method designed by the present invention for twin systems is demonstrated in application, and the training and test losses of the Bootstrap model in step S2 change with the number of iterations. With the increase of the number of iterations, the training loss dropped significantly from 1.35 to close to 0.98, and the test loss dropped from 1.20 to close to 0.98, indicating that the performance of the Bootstrap model on the training and validation data sets continued to improve and eventually stabilized. Throughout the process, the test loss was always slightly lower than the training loss, indicating that the Bootstrap model was not overfitting and had good generalization ability. Overall, the Bootstrap model successfully converged during the training process and achieved the expected compression effect.

[0055] Figure 5The change of loss during the training of the Combined model in step S3 is shown. This part of the training is the result of further combining and optimizing the model structure based on the model training in step S2. It can be seen that the loss value gradually decreases with the progress of training, and the error of the model gradually decreases during the continuous optimization process, indicating that the performance of the model has been further improved during the training of the Combined model. By comparing the training results of step S2, it can be seen that the training of the Combined model is based on the performance of the initial model, and the loss value continues to be reduced in further combination and optimization. This shows that the Combined model has successfully inherited the advantages of the previous training part and further improved the generalization ability and compression effect of the model. From the overall trend, Figure 5 This shows that the optimization of the model at this stage is effective, and the test loss has reached a relatively stable level, proving the rationality and effectiveness of the model design and training strategy.

[0056] Figure 6 The performance comparison of the method designed by the present invention and a series of compression algorithms on diverse data sets is demonstrated. These algorithms include traditional compressors 7-Zip, BSC, Zip, Bzip, Gzip and Tar, as well as neural network-based compressors CMIX and Dzip. The test data sets cover a wide range of fields, including text (such as Webster and Text8), audio (Audio), genomic data (such as ill-quality), floating-point data (such as Num-control), human chromosome data (such as h.chr1 and h.chr20), image data (such as Kodim02), as well as mixed data sets XOR60 and HMM60, and specific data sets such as Np-bases and Enwiki9. The experimental results show that the method designed by the present invention performs well on multiple data sets, especially on text and genomic data sets, significantly outperforming other traditional compressors and neural network compressors. Compared with the original compression algorithm, the method designed by the present invention has significant improvements in bit rate per character (BPC) and overall compression efficiency. In addition, compared with other neural network compressors, the method designed by the present invention not only performs well in compression efficiency, but also shows significant advantages in processing speed. For example, although its compression effect on text data is slightly lower than that of the CMIX method, its encoding and decoding efficiency is three to four times that of CMIX, which fully demonstrates its performance advantage. The excellent performance of the method designed by the present invention is due to the multi-head self-attention mechanism, position encoding and residual block architecture. These technologies greatly improve its adaptability and generalization ability on different data types, demonstrating its wide application potential and technical advantages in the field of modern data compression.

[0057] In summary, by comparing and analyzing the bits per character (BPC) of multiple compressors on different types of real data sets, the method designed by the present invention shows significant advantages. By introducing a multi-head self-attention mechanism, position encoding, and residual modules, the compression effect of the present invention on text, audio, and genomic data sets is significantly better than that of traditional compressors and other neural network compressors. These improvements not only improve compression efficiency and prediction accuracy, but also enhance the adaptability and generalization ability of the model. Therefore, the excellent performance of the method designed by the present invention on a variety of data types proves its feasibility and effectiveness as an efficient and reliable compression method.

[0058] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A data intelligent compression method for twin systems, characterized in that: Execute the following steps S1 to S5 to complete data compression and transmission: Step S1: collecting video data, audio data and text data as data to be compressed, and performing preprocessing on the data to be compressed, including data standardization and format conversion, to obtain preprocessed data; Step S2: Build and train the Bootstrap model, taking the preprocessed data as input, extracting features and preliminarily compressing the preprocessed data based on the position encoding module, bidirectional gated recurrent unit, multi-head attention mechanism, residual module and functional module, and outputting the probability distribution logits b ; Step S3: Build and train the Combined model. The Combined model consists of the Bootstrap model and the Supporter model. The Supporter model is based on the position encoding module, multi-head attention mechanism, residual module and functional module, and outputs the probability distribution logits s , the Combined model outputs the probability distribution of the Bootstrap model logits b , and the probability distribution output by the Supporter model logits s Combined, strengthen the sequence feature representation of data and further compress it to generate probability distribution logits c ; Step S4: finally compressing the probability distribution outputted in step S3 using an arithmetic coding method to reduce the volume of data transmission, and transmitting the data to the target device; Step S5: On the target device, the received data is decoded and reconstructed using the pre-trained Bootstrap model, Combined model and arithmetic decoding method to restore the original data before compression.

2. According to claim 1, a data intelligent compression method for twin systems is characterized in that: The method for preprocessing the data to be compressed in step S1 is as follows: Inter-frame sampling is performed on the data frames of video data and audio data, and the sampling rate is reduced by the following formula: ； in, is the sampled data frame, S i Represents the first i frame, n is the sampling interval; For the sampled video data and audio data, normalization is performed as follows: ； in, x norm is the normalized data value, x is the original data value, min( x ) and max( x ) are the minimum and maximum values ​​in the data respectively; For text data, characters are mapped to integers using the following formula: ； in, C int is the integer value after mapping, c is the original character, ord is a function that converts a character to its corresponding ASCII integer value.

3. The method for intelligent data compression for twin systems according to claim 1, characterized in that: The Bootstrap model described in step S2 includes a position encoding module, a bidirectional gated recurrent unit, a multi-head attention mechanism, a residual module and a functional module in sequence; The Bootstrap model inputs the preprocessed data, adds position information to the data through the position encoding module, and then uses a bidirectional gated recurrent unit to output embedded features to capture the bidirectional dependencies of the data. After that, it flattens and uses a multi-head attention mechanism to output attention scores to capture the complex relationships between the data. The unscaled probability distribution is generated through the residual module and two parallel functional modules. logits b , to complete the preliminary prediction, where the two parallel functional modules are the linear layer and the fully connected layer. The linear layer is first used to perform a preliminary transformation on the extracted features, and then the features are mapped to the output space through the fully connected layer.

4. The method for intelligent data compression for twin systems according to claim 1, characterized in that: In the Combined model described in step S3, the Supporter model inputs the preprocessed data, adds position information to the data through the position encoding module, concatenates the data with the added position information with the embedded features output by the bidirectional gated recurrent unit of the Bootstrap model, and outputs the attention score through the multi-head attention mechanism to capture the complex relationship between the data, passes through multiple residual modules in turn, and finally passes through three parallel functional modules, which are linear modules, dense modules and residual modules. The output vector of each module is reduced to a dimension matching the vocabulary size through linear transformation, and is weighted and summed at the end to generate a probability distribution. logits s ; The Combined model combines the probability distributions output by the Bootstrap model and the Supporter model, and generates the final probability distribution through convex sum logits c ; ； in, λ is a learnable parameter constrained to the range [0,1] by the sigmoid activation function.

5. A data intelligent compression method for twin systems according to claim 3 or 4, characterized in that: The position encoding module processes data as follows: ； ； in, , is the position-encoded data, POS is the sequence position, i is the dimension index, d is the model dimension.

6. The method for intelligent data compression for twin systems according to claim 3 or 4, characterized in that: The multi-head attention mechanism processes data as follows: ； Among them, Attention(Q,K,V) is the attention score, Q, K, V represent query, key and value matrices respectively, d k is the dimension of the key vector and softmax is the normalization function.

7. The method for intelligent data compression for twin systems according to claim 1, characterized in that: The arithmetic coding method in step S4 is as follows: ； Among them, new_interval is the new encoding interval, low and high are the lower and upper bounds of the current interval, and probability is the probability of occurrence of the current symbol.

Images