Data compression method and system of chip

Abstract

The invention relates to the field of data compression, and discloses a data compression method and system for a chip, and the method comprises the steps: obtaining data needing to be stored in the chip; the data is preprocessed; and encoding the preprocessed data to realize compression of the data. More flexible, more interactive and more user-friendly data compression is realized. A user not only can select and adjust coding combinations and parameters, but also can carry out deep analysis and optimization, so that an optimal compression effect is realized.

Description

Technical Field

[0001] This invention relates to the field of data compression technology, specifically to a data compression method and system for chips. Background Technology

[0002] With the advent of the big data era, the volume of data has exploded, bringing enormous challenges to data storage and transmission. Traditional data compression techniques, such as Huffman coding and Run-Length coding, while performing well in certain scenarios, may have limited effectiveness when faced with complex and ever-changing modern data. For example, data generated by social media, IoT devices, and high-definition video streams is highly diverse and complex, requiring more advanced and flexible compression techniques to handle it.

[0003] Furthermore, with the improvement of computing power and the deepening of algorithm research, new data compression technologies and methods are constantly emerging. These technologies often combine various principles of mathematics, statistics, and information theory, aiming to provide optimal compression results for different data characteristics. However, these advanced technologies usually require professional knowledge and experience to apply correctly, which can be a challenge for ordinary users.

[0004] With the continuous growth of data volume, efficient data compression technology is becoming increasingly important. This invention provides an interactive data compression system that combines multiple complex encoding algorithms, aiming to provide users with the best compression results. Summary of the Invention

[0005] In view of the above-mentioned problems, the present invention is proposed.

[0006] Therefore, the technical problem solved by the present invention is that existing data compression methods are not flexible enough, have high operational complexity, and cannot automatically select or adjust the optimal compression strategy according to the characteristics of the data.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a data compression method and system for a chip, comprising:

[0008] Retrieve the data that needs to be stored in the chip;

[0009] The data is preprocessed;

[0010] The preprocessed data is encoded to achieve data compression.

[0011] In a preferred embodiment of the data compression method for the chip described in this invention, the preprocessing includes: identifying and removing redundant information in the data; sorting the data to cluster data with the same encoding characteristics together to form a basic classification for data encoding.

[0012] In a preferred embodiment of the data compression method for the chip described in this invention, the encoding includes conversion encoding, variable-length encoding, and entropy encoding; the user can choose a single encoding method or a combination of encoding methods; when the user does not make an encoding selection or the user authorizes the system to automatically select an encoding, the system performs encoding by analyzing the data.

[0013] If the data contains repeating patterns, then use conversion encoding;

[0014] If the frequency distribution of characters in the data is uneven, then variable-length encoding should be used;

[0015] If the probability distribution of characters in the data is uneven, then entropy coding is used;

[0016] When there are a large number of repetitive patterns in the data and the character frequency distribution is uneven, first use conversion encoding to process the repetitive patterns, and then use variable length encoding to optimize the character frequency.

[0017] When the frequency and probability of characters in the data are unevenly distributed, variable-length encoding is used to process the frequency of characters first, and then entropy encoding is used to optimize the probability of characters.

[0018] When there are repetitive patterns in the data and the character probability distribution is uneven, first use conversion encoding to process the repetitive patterns, and then use entropy encoding to optimize the character probability.

[0019] When there are many repetitive patterns in the data, and the frequency and probability of characters are unevenly distributed, we use transformation encoding to process the repetitive patterns, then use variable-length encoding to optimize the character frequency, and finally use entropy encoding to optimize the character probability.

[0020] In a preferred embodiment of the data compression method for the chip described in this invention, the variable-length encoding includes a frequency adjustment function expressed as follows:

[0021]

[0022] Where f(x) represents the frequency of character x in the original data; α represents an adjustable parameter used to adjust the amplitude of the frequency function; log(f(x)+1) represents the logarithmic function used to smooth the frequency data; β represents the integral of the Gaussian function, and γ is an adjustable parameter used to adjust the width of the Gaussian function; γ is an adjustable parameter used to adjust the weight of the second derivative. The second derivative of f(x) is used to capture the curvature of f(x).

[0023] In a preferred embodiment of the data compression method for the chip described in this invention, the conversion encoding includes a representation function.

[0024]

[0025] Where M represents a 2×2 adjustable matrix used to transform the position and length of the data block; P(x) represents the position of data block x; L(x) represents the length of data block x; ζ represents an adjustable parameter used to adjust the weight of the derivative of the position; This represents the derivative of P(x), used to capture the rate of change of position.

[0026] In a preferred embodiment of the data compression method for the chip described in this invention, the entropy encoding includes a probability adjustment function.

[0027]

[0028] Where p(x) represents the probability of character x in the original data; δ represents an adjustable parameter used to adjust the amplitude of the tangent function; ∈ represents an adjustable parameter used to adjust the weight of the third derivative; The third derivative of log(p(x)+1) is used to capture the curvature change of probability.

[0029] As a preferred embodiment of the data compression method for the chip described in this invention, the compression further includes the ability of the user to adjust compression parameters according to the characteristics and needs of the data;

[0030] The compression effect is verified by decompressing the data. The decompressed data is compared with the original data. If there are missing data in the decompressed data, the missing content is analyzed.

[0031] When the missing data is information that appears excessively in the data, it is assumed that the compression process ensures the integrity of the data. When the missing data is not information that appears excessively in the data, the data preprocessing process is traced back. If the missing data was removed due to over-processing during preprocessing, a report is generated for the over-processed original data and the processed data, and technical personnel are scheduled to adjust the preprocessing. If the missing data was not removed due to over-processing during preprocessing, the compressed data is regenerated and verified.

[0032] If the regenerated compressed data does not contain missing data, the data is saved; if the regenerated compressed data still contains missing data, the missing information is marked, and the corresponding encoding algorithm is found based on the marked missing information. The technical staff then schedules the corresponding encoding algorithm to be updated.

[0033] A data compression system for a chip employing any of the methods described in this invention, characterized in that it comprises:

[0034] The data acquisition module acquires the data that needs to be stored in the chip and transmits the acquired information to the data preprocessing module.

[0035] The data preprocessing module preprocesses the data and transmits the processed data to the data compression module;

[0036] The data compression module encodes the preprocessed data to compress it.

[0037] A computer device includes: a memory and a processor; the memory stores a computer program, characterized in that: when the processor executes the computer program, it implements the steps of the method described in any one of the present invention.

[0038] A computer-readable storage medium having a computer program stored thereon, characterized in that: when the computer program is executed by a processor, it implements the steps of the method described in any one of the present invention.

[0039] The beneficial effects of this invention are as follows: The chip data compression method provided by this invention achieves more flexible, interactive, and user-friendly data compression. Users can not only select and adjust encoding combinations and parameters, but also perform in-depth analysis and optimization to achieve the best compression effect. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0041] Figure 1 A flowchart of a chip data compression method and system provided in the first embodiment of the present invention;

[0042] Figure 2 The image shows the compression effect of a chip data compression method provided in the second embodiment of the present invention compared with a traditional method. Detailed Implementation

[0043] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0044] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0045] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0046] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0047] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0048] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0049] Example 1

[0050] Reference Figure 1 As an embodiment of the present invention, a data compression method and system for a chip are provided, comprising:

[0051] S1: Obtain the data that needs to be stored in the chip.

[0052] Furthermore, when users store data, the system first needs to determine the data source, such as sensors, databases, or user input. To ensure data integrity and accuracy, the system performs data validation, such as checking for missing or outlier values.

[0053] It's worth mentioning that the data also needs to be converted to a suitable storage format, which may involve data type conversion or encoding. Depending on storage needs and space constraints, data compression may be required. For security reasons, data encryption may be necessary.

[0054] S2: Preprocess the data.

[0055] Furthermore, preprocessing includes identifying and removing redundant information from the data; sorting the data to cluster data with the same coding characteristics together to form a basic classification of the data encoding.

[0056] It's important to understand that redundant information in data refers to information that doesn't carry useful information or appears repeatedly. Many datasets, especially unoptimized ones, may contain a large amount of duplicate or unnecessary data. Removing this data can make the dataset smaller and more concise.

[0057] Furthermore, sorting or classifying data helps group similar data together, making it easier for data compression algorithms to identify and utilize these patterns for efficient compression. The specific steps are as follows: a. Feature extraction: Extract key features from the data; these features are used to distinguish data points. b. Data classification: Based on the extracted features, use classification algorithms to divide the data into several groups or clusters. Data within each group or cluster has similar coded features. c. Data sorting: Within each category or cluster, further sort the data to ensure data continuity. This is usually based on some metric, such as data size or frequency.

[0058] It's also important to know that this forms the basic classification for data encoding. Once the data is sorted and classified, it can be used as input to data compression algorithms. Through preprocessing, we ensure that the data has an optimal structure to achieve the highest compression efficiency. This preprocessing method is to ensure that the data has been optimized before compression, thereby improving the efficiency and compression ratio of the compression algorithm.

[0059] S3: Encode the preprocessed data to compress it.

[0060] Furthermore, the encoding includes transformation encoding, variable-length encoding, and entropy encoding; users can choose a single encoding method or a combination of encodings; when the user does not make an encoding selection or the user authorizes the system to select one automatically, the system performs encoding by analyzing the data.

[0061] If the data contains repeating patterns, use transformation encoding. If the character frequency distribution in the data is uneven, use variable-length encoding. If the character probability distribution in the data is uneven, use entropy encoding.

[0062] When there are many repetitive patterns in the data and the character frequency distribution is uneven, first use conversion encoding to process the repetitive patterns, and then use variable-length encoding to optimize the character frequency; when both the character frequency and character probability in the data are unevenly distributed, first use variable-length encoding to process the character frequency, and then use entropy encoding to optimize the character probability; when there are many repetitive patterns in the data and the character probability distribution is uneven, first use conversion encoding to process the repetitive patterns, and then use entropy encoding to optimize the character probability; when there are many repetitive patterns in the data and both the character frequency and character probability are unevenly distributed, use conversion encoding to process the repetitive patterns, then use variable-length encoding to optimize the character frequency, and finally use entropy encoding to optimize the character probability.

[0063] It's important to understand that when we refer to repetitive patterns, we're referring to substrings, characters, or bit patterns that appear repeatedly in data. For example, a document might contain repeating phrases, an image might contain repeating pixel patterns, or an audio file might contain repeating sound wave patterns. We need to identify and effectively compress these repetitive patterns. Algorithms achieve compression by replacing repetitive patterns with shorter codes. Uneven character frequency distribution refers to a situation where some characters appear far more frequently than others in a piece of data. For example, in English text, the letter "e" appears much more frequently than "z". When character frequency distribution is uneven, we can use variable-length coding to assign shorter codes to frequently occurring characters and longer codes to infrequently occurring characters. This method can improve compression efficiency. Uneven character probability distribution, similar to uneven character frequency distribution, refers to a situation where some characters have a much higher probability of appearing than others. Probabilities are usually obtained through statistical analysis or other predictive methods. When we know or can predict the probability distribution of characters, entropy coding can assign each character a code length inversely correlated with its probability. Simply put, characters with high probability receive shorter codes, while characters with low probability receive longer codes. This is based on a fundamental concept in information theory: less likely events contain more information and therefore require longer codes; while more likely events contain less information and therefore can use shorter codes.

[0064] It's also worth mentioning that, based on data analysis results, the system automatically recommends one or more encoding combinations. Users can choose to accept the system's recommended encoding combinations or select other combinations themselves. The system provides users with a simple explanation describing the applicability and expected effects of each encoding combination. Users can adjust the parameters of each encoding, such as the frequency function of variable-length encoding and the representation function of conversion encoding. The system provides users with a real-time preview function, displaying the expected compression effect after parameter adjustments.

[0065] It's important to know that the system provides users with a visual interface that displays data statistics, recommended encoding strategies, and expected compression results. Users can adjust encoding parameters using drag-and-drop or sliders and view the expected compression effect in real time. After the user confirms their selection, the system begins compressing the data and provides the user with the compressed data and compression ratio.

[0066] Variable-length coding includes a frequency adjustment function, expressed as:

[0067]

[0068] Where f(x) represents the frequency of character x in the original data; α represents an adjustable parameter used to adjust the amplitude of the frequency function; log(f(x)+1) represents the logarithmic function used to smooth the frequency data; β represents the integral of the Gaussian function, and γ is an adjustable parameter used to adjust the width of the Gaussian function; γ is an adjustable parameter used to adjust the weight of the second derivative. The second derivative is used to capture the curvature of f(x). Character frequencies in the data may vary significantly. Traditional variable-length encoding methods may not fully utilize these differences to achieve optimal compression. By adjusting the frequency function, we can assign shorter codes to lower-frequency characters, thus achieving better compression. This invention can adjust the frequency function according to the characteristics of the data to achieve optimal compression. It provides a flexible and customizable encoding method.

[0069] Transformation encoding includes representation functions:

[0070]

[0071] Where M represents a 2×2 adjustable matrix used to transform the position and length of the data block; P(x) represents the position of data block x; L(x) represents the length of data block x; ζ represents an adjustable parameter used to adjust the weight of the derivative of the position; The derivative of P(x) is used to capture the rate of change of position. The data may contain multiple repeating patterns or strings. Traditional transformational encoding methods may not fully utilize these patterns to achieve optimal compression. By using matrix operations and other advanced techniques, we can assign different codes to different patterns, thereby achieving better compression. This invention provides a new method to optimize transformational encoding. The representation function can be adjusted according to the characteristics of the data to achieve optimal compression. A flexible and customizable encoding method is provided.

[0072] Entropy coding includes probability adjustment functions:

[0073]

[0074] Where p(x) represents the probability of character x in the original data; δ represents an adjustable parameter used to adjust the amplitude of the tangent function; ∈ represents an adjustable parameter used to adjust the weight of the third derivative; The third derivative of log(p(x)+1) is used to capture the curvature change of probability. Character probabilities in the data may vary significantly. Traditional entropy coding methods may not fully utilize these differences to achieve optimal compression. By adjusting the probability function, we can assign shorter codes to characters with lower probabilities, thus achieving better compression. Our invention provides a new method to optimize entropy coding. The probability function can be adjusted according to the characteristics of the data to achieve optimal compression. It offers a flexible and customizable coding method.

[0075] Furthermore, users can adjust compression parameters according to the characteristics and needs of the data.

[0076] The compression effect is verified by decompressing the data. The decompressed data is compared with the original data. If there are missing data in the decompressed data, the missing content is analyzed.

[0077] When the missing data is information that appears excessively in the data, the compression process is considered to have ensured the integrity of the data. When the missing data is not information that appears excessively in the data, the data preprocessing process is traced back. If the missing data was removed due to over-processing during preprocessing, a report is generated of the over-processed original data and the processed data, and technical personnel are scheduled to adjust the preprocessing. If the missing data was not removed due to over-processing during preprocessing, the compressed data is regenerated and verified.

[0078] If the regenerated compressed data does not contain missing data, the data is saved; if the regenerated compressed data still contains missing data, the missing information is marked, and the corresponding encoding algorithm is found based on the marked missing information. The technical staff then schedules the corresponding encoding algorithm to be updated.

[0079] It's important to note that after data compression, decompression is necessary to verify its effectiveness and integrity. The decompressed data is then compared one-to-one with the original data. This comparison ensures the data integrity was maintained during compression and decompression. If discrepancies or missing portions are found in the decompressed data during the comparison, these portions require further analysis.

[0080] Furthermore, first, check if the missing data contains information that frequently appears in the original data. This information might be redundant, such as repeating patterns or strings. If the missing data is indeed such excessive information, then the compression process can be considered successful because it has successfully removed unnecessary redundancy. If the missing data is not excessive information, then the data preprocessing process needs to be traced back. Preprocessing is the initial processing of data before formal compression, such as noise reduction, filtering, or other transformations. If it is found that the missing data was overprocessed and removed during preprocessing, then a detailed report needs to be generated. This report should include the overprocessed original data and the processed data, as well as the preprocessing parameters and methods. Then, technicians should adjust the preprocessing process based on this report to ensure that important data is not removed again. If the missing data was not removed due to overprocessing, then the compressed data needs to be regenerated and validated. This is to ensure that the compression process is correct and there are no other problems.

[0081] It's also important to decompress and verify the regenerated compressed data to ensure its integrity. If the regenerated compressed data still contains missing data, this missing information needs to be marked. Based on the marked missing information, the corresponding encoding algorithm is identified. This is to determine which encoding algorithm caused the data loss. Then, the technical staff should update this encoding algorithm to ensure that this problem does not recur in future compression processes.

[0082] A data compression system for a chip employing the method described in this invention, characterized in that it comprises:

[0083] The data acquisition module acquires the data to be stored in the chip and transmits the acquired information to the data preprocessing module; the data preprocessing module preprocesses the data and transmits the processed data to the data compression module; the data compression module encodes the preprocessed data to compress it.

[0084] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory, magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory, magnetic variable memory, ferroelectric memory, phase change memory, graphene memory, etc. Volatile memory can include random access memory or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory or dynamic random access memory, etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include blockchain-based distributed databases, etc., and are not limited thereto.

[0085] The processors involved in the various embodiments provided in this application may be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited thereto.

[0086] Example 2

[0087] Reference Figure 2 As an embodiment of the present invention, a data compression method and system for a chip are provided. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0088] Through simulation experiments, the following datasets were set up: a text dataset (100MB in size), an image dataset (1000 high-resolution images, 1GB in size), an audio dataset (2 hours of lossless audio, 500MB in size), and a video dataset (1 hour of 1080p video, 2GB in size). The experimental results are shown in Table 1.

[0089] Table 1 Simulation Data of Compression Results

[0090]

[0091]

[0092] The experimental data demonstrates that the method of this invention performs excellently on all datasets, achieving a higher compression ratio than other methods, with shorter compression and decompression times. Furthermore, the method achieves 100% data integrity, meaning it not only provides efficient compression but also ensures data integrity. These simulation data strongly support the method of this invention, proving its superiority and practicality.

[0093] To verify whether the method of this invention truly provides users with a more flexible, interactive, and user-friendly data compression experience, the following experiment was designed. Each group of users was provided with the same dataset and asked to compress the data using both the method of this invention and a traditional compression tool. The experimental results are shown in Table 2.

[0094] Select two groups of users:

[0095] Expert group: Technical personnel with backgrounds in data compression.

[0096] Regular user group: Regular users without data compression background.

[0097] Table 2 Experimental Data

[0098] User Group tool Operation time min User satisfaction Compression ratio expert group Method of the present invention 10 9 92％ expert group Traditional tools 15 7 89％ Ordinary user group Method of the present invention 12 8.5 91％ Ordinary user group Traditional tools 20 6 87％

[0099] Whether for expert or ordinary users, the operation time using the method of this invention is shorter than that using traditional tools, indicating that the method of this invention is more efficient. In terms of user satisfaction, the method of this invention also received a higher score, indicating that it provides a better interactive experience. In terms of compression effect, the method of this invention is also slightly superior, demonstrating that it is not only user-friendly but also produces excellent compression results. From this experimental and simulation data, we can conclude that the method of this invention truly provides users with a more flexible, interactive, and user-friendly data compression experience. Both experts and ordinary users can easily use it and obtain optimal compression results.

[0100] Figure 2 This diagram compares the compression effects of the present invention and conventional inventions when applying compression to different data types. It shows that the present invention can automatically make the optimal compression selection under different environments, resulting in a more stable and higher compression ratio. In contrast, conventional inventions use only a single compression method, which cannot adapt to changing data environments and lacks stability.

[0101] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A data compression method for a chip, characterized in that, include: Retrieve the data that needs to be stored in the chip; The data is preprocessed; The preprocessed data is encoded to achieve data compression.

2. The data compression method for a chip as described in claim 1, characterized in that: The preprocessing includes identifying and removing redundant information from the data; sorting the data to cluster data with the same coding characteristics together to form a basic classification of the data encoding.

3. The chip data compression method as described in claim 2, characterized in that: The encoding includes transformation encoding, variable-length encoding, and entropy encoding; users can choose a single encoding method or a combination of encodings; if the user does not make an encoding selection or the user authorizes the system to select automatically, the system performs encoding by analyzing the data. If the data contains repeating patterns, then use conversion encoding; If the frequency distribution of characters in the data is uneven, then variable-length encoding should be used; If the probability distribution of characters in the data is uneven, then entropy coding is used; When there are a large number of repetitive patterns in the data and the character frequency distribution is uneven, first use conversion encoding to process the repetitive patterns, and then use variable length encoding to optimize the character frequency. When the frequency and probability of characters in the data are unevenly distributed, variable-length encoding is used to process the frequency of characters first, and then entropy encoding is used to optimize the probability of characters. When there are repetitive patterns in the data and the character probability distribution is uneven, first use conversion encoding to process the repetitive patterns, and then use entropy encoding to optimize the character probability. When there are many repetitive patterns in the data, and the frequency and probability of characters are unevenly distributed, we use conversion encoding to process the repetitive patterns, then use variable-length encoding to optimize the character frequency, and finally use entropy encoding to optimize the character probability.

4. The chip data compression method as described in claim 3, characterized in that: The variable-length encoding includes a frequency adjustment function, expressed as follows: Where f(x) represents the frequency of character x in the original data; α represents an adjustable parameter used to adjust the amplitude of the frequency function; log(f(x)+1) represents the logarithmic function used to smooth the frequency data; β represents the integral of the Gaussian function, and γ is an adjustable parameter used to adjust the width of the Gaussian function; γ is an adjustable parameter used to adjust the weight of the second derivative. The second derivative of f(x) is used to capture the curvature of f(x).

5. The chip data compression method as described in claim 4, characterized in that: The conversion encoding includes a representation function. Where M represents a 2×2 adjustable matrix used to transform the position and length of the data block; P(x) represents the position of data block x; L(x) represents the length of data block x; ζ represents an adjustable parameter used to adjust the weight of the derivative of the position; This represents the derivative of P(x), used to capture the rate of change of position.

6. The chip data compression method as described in claim 5, characterized in that: The entropy encoding includes a probability adjustment function. Where p(x) represents the probability of character x in the original data; δ represents an adjustable parameter used to adjust the amplitude of the tangent function; ∈ represents an adjustable parameter used to adjust the weight of the third derivative; The third derivative of log(p(x)+1) is used to capture the curvature change of probability.

7. The chip data compression method as described in claim 6, characterized in that: The compression also includes the ability for users to adjust compression parameters according to the characteristics and needs of the data; The compression effect is verified by decompressing the data. The decompressed data is compared with the original data. If there are missing data in the decompressed data, the missing content is analyzed. When the missing data is information that appears excessively in the data, it is assumed that the compression process ensures the integrity of the data; when the missing data is not information that appears excessively in the data, the data preprocessing process is traced back. If the missing data was removed due to over-processing during preprocessing, a report is generated for the over-processed original data and the processed data, and the scheduling technicians adjust the preprocessing. If the missing data was not removed due to overprocessing during preprocessing, the compressed data is regenerated and validated. If the regenerated compressed data does not contain missing data, the data is saved; if the regenerated compressed data still contains missing data, the missing information is marked, and the corresponding encoding algorithm is found based on the marked missing information. The scheduling technicians then update the corresponding encoding algorithm.

8. A data compression system for a chip employing the method described in any one of claims 1-7, characterized in that: include, The data acquisition module acquires the data that needs to be stored in the chip and transmits the acquired information to the data preprocessing module. The data preprocessing module preprocesses the data and transmits the processed data to the data compression module; The data compression module encodes the preprocessed data to compress it.

Images