Data compression method for low-level embedded systems

Abstract

The present invention discloses a data compression method suitable for low-level embedded systems, including the following steps: S1, processing target data to obtain the required data format, and confirming the minimum effective data length; S2, presetting the fault tolerance range according to the effective data length; S3, collecting a reference frame as a reference data reference table, and subsequent data compression and decompression are based on this reference data reference table, and update synchronization is performed on the basis of this reference data reference table; S41, compressing the data vertically line by line through the reference frame, comparing whether the changes before and after each section of effective data are within the preset fault tolerance range, and judging whether the effective data needs to be transmitted. The present invention can provide a higher data compression rate to a certain extent, is more suitable for low-level embedded systems, saves costs, replaces the existing complex compression transmission method with a simple and easy-to-implement compression method, makes the project development faster, and the entire compression method is easier to maintain and transplant.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a data compression method suitable for low-order embedded systems. Background Art

[0002] Data compression refers to a technical method that reduces the amount of data to reduce storage space, improves its transmission, storage and processing efficiency, or reorganizes data according to a certain algorithm to reduce data redundancy and storage space without losing useful information.

[0003] At present, the H.264 compression technology is widely used in high-definition image data compression based on real-time transmission of high-definition images. The H.264 compression technology mainly uses the following methods to compress video data: 1. Intra-frame prediction compression, which solves the problem of spatial data redundancy; 2. Inter-frame prediction compression (motion estimation and compensation), which solves the problem of temporal data redundancy; 3. Integer discrete cosine transform (DCT), which converts spatial correlation into irrelevant data in the frequency domain and then quantizes it; 4. The compressed frames are divided into I frames, P frames and B frames.

[0004] The main purpose of H.264 compression technology is to ensure fast transmission speed and high image quality. However, its compression logic is complex, which makes its software algorithm bulky and not suitable for integrating the software into embedded systems with low-level configuration and small-capacity Flash. In addition, its algorithm divides one frame of data into three frames of data. Because each frame of data needs to be packaged and unpacked before and after transmission, the workload of both parties is increased and unnecessary system overhead is increased. For some low-configuration embedded systems, the already scarce processor computing resources are even more incapable, and the system solution has to upgrade the CPU and Flash, resulting in increased costs. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a data compression method suitable for low-end embedded systems for low-configuration embedded products and systems that have certain fault tolerance requirements for the accuracy of images, audio and other data streams, to ensure that data can be transmitted quickly within a certain fault tolerance range and its correctness is guaranteed.

[0006] The technical solution adopted by the present invention to solve the technical problem is: a data compression method suitable for low-level embedded systems, comprising the following steps:

[0007] S1, process the target data to obtain the required data format and confirm the minimum valid data length;

[0008] S2, preset the error tolerance range according to the effective data length;

[0009] S3, collecting a reference frame as a reference data reference table. Subsequent data compression and decompression are based on this reference data reference table, and update synchronization is performed based on this reference data reference table;

[0010] S41, compressing the data line by line vertically through the reference frame, comparing whether the changes before and after each section of valid data are within a preset error tolerance range, and determining whether the valid data needs to be transmitted.

[0011] As a further improvement of the present invention, in the S41, first, the first several bytes of each row of data to be sent are defined as a longitudinal position map; secondly, the current position valid data of the subsequent frame data is compared with the corresponding position valid data of the reference frame, and the difference is calculated. If the difference is within the fault tolerance range, it is not transmitted, and the corresponding position is marked as "0" in the longitudinal position map; if the difference exceeds the fault tolerance range, the difference is retained, and the corresponding position is marked as "1" in the longitudinal position map, and the longitudinal position information of the longitudinal position "1" and its corresponding difference information are sent.

[0012] As a further improvement of the present invention, a step S40 is provided between S3 and S41, in which the rear frame data is compared with the benchmark data reference table to determine the rate of change between the two. When the rate of change between the rear frame data and the benchmark data reference table is in a set second rate of change interval, S41 is executed.

[0013] As a further improvement of the present invention, the change rate is divided into three intervals by setting a first threshold and a second threshold, wherein the first threshold is smaller than the second threshold, and the interval less than the first threshold is defined as the first change rate interval, the interval between the first threshold and the second threshold is defined as the second change rate interval, and the interval greater than the second threshold is defined as the third change rate interval.

[0014] As a further improvement of the present invention, when the change rate between the post-frame data and the benchmark data reference table is in the first change rate interval, step S42 is executed to compare the current position valid data of the post-frame data with the corresponding position valid data of the benchmark frame to calculate the difference, and record the position information of all the differences, and send the position information whose difference is not "0" and its corresponding difference information.

[0015] As a further improvement of the present invention, when the change rate between the post-frame data and the benchmark data reference table is in the third change rate interval, step S43 is executed to compare the current position valid data of the post-frame data with the corresponding position valid data of the benchmark frame to calculate the difference and send all the difference information.

[0016] As a further improvement of the present invention, different data headers are added to the data sent in steps S41, S42, and S43, respectively, and the receiving end decompresses the data according to different processing logics by identifying the data headers.

[0017] As a further improvement of the present invention, CRC check is performed on the data sent in steps S41, S42, and S43, and information files of "header frame", "data frame", and "tail frame" are added to the data according to the data type.

[0018] The beneficial effects of the present invention are as follows: the present invention provides a data compression method suitable for low-level embedded systems, which can provide a higher data compression rate to a certain extent, and is more suitable for low-level embedded systems. By sacrificing the allowable error of part of the data, the hardware requirements of the corresponding functions are reduced by several levels, saving costs and reducing power consumption; at the same time, the entire data presentation and application meet relevant requirements, and the final data application does not generate or generates some errors caused by data compression that are more acceptable than increasing hardware costs. By replacing the existing complex compression transmission method with a simple and easy-to-implement compression method, the project development speed is faster, the entire compression method is easier to maintain and transplant, and the simple compression method can reduce the occurrence of anomalies and locate problems faster. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a block diagram of the steps of the data compression method applicable to low-level embedded systems of the present invention;

[0020] Figure 2 The figure is a flow chart of a data compression method applicable to a low-level embedded system according to the present invention. DETAILED DESCRIPTION

[0021] A preferred embodiment of the present invention is described in detail below in conjunction with the accompanying drawings.

[0022] See also Figure 1 and Figure 2 The present invention provides a data compression method suitable for low-order embedded systems, including the following steps S1 to S4.

[0023] S1, process the target data to obtain the required data format and confirm the effective data length.

[0024] The target data to be compressed may be in the form of images, audio or other data streams, for example. The target data is first converted into binary data through existing application software, and then encoded to obtain octal or hexadecimal data.

[0025] S2, preset the error tolerance range according to the effective data length.

[0026] Since the effective data length determines the effective area of ​​the fault tolerance range, high-bit data cannot be fault-tolerant to avoid data distortion due to large changes before and after the data. Therefore, before compressing the data, the minimum effective data length must be confirmed. For example, for image data, due to the display, low-end system displays often cannot display complete pixel primary colors, so when the color value has a certain deviation before and after, the display cannot show the difference, especially some edge transition colors, so for 8-bit chroma, the deviation is 6 to 249, and the display effect difference is acceptable; but if it is 16-bit chroma, both bytes are out of 5, and the deviation of the difference before and after is very large, which is no longer in the same order of magnitude, so before compressing the data, the minimum effective data bit length must be determined, and then the fault tolerance range must be confirmed.

[0027] Among them, the fault tolerance range can be set according to needs. For example, if the image data requires a smaller compression volume and the clarity requirement is not high, an appropriately larger fault tolerance range can be set; if there are very high requirements for the display effect and high image clarity is required, the fault tolerance range should be smaller.

[0028] S3, collect the reference frame as the reference data reference table, the subsequent data compression and decompression are based on this reference data reference table, and update synchronization is performed on the basis of this reference data reference table.

[0029] S4, data compression, specifically includes the following steps S40-S41.

[0030] S40, comparing the subsequent frame data with the reference table of the benchmark data, determining the change rate between the two, and executing corresponding steps according to the interval in which the change rate is located.

[0031] Specifically, the change rate is divided into three intervals from 0% to 100% by setting the first threshold and the second threshold, wherein the first threshold is less than the second threshold, and the interval less than the first threshold is defined as the first change rate interval, the interval between the first threshold and the second threshold is defined as the second change rate interval, and the interval greater than the second threshold is defined as the third change rate interval.

[0032] When the change rate between the subsequent frame data and the reference table of the benchmark data is within the set second change rate interval, the following step S41 is executed.

[0033] S41, compressing the data line by line vertically through the reference frame, comparing whether the changes before and after each section of valid data are within a preset error tolerance range, and determining whether the valid data needs to be transmitted.

[0034] Specifically, first, define the first several bytes (for example, 8 bytes or 64 bytes) of each line of data to be sent as a longitudinal position map; secondly, compare the current position valid data of the subsequent frame data with the corresponding position valid data of the reference frame, and calculate the difference. If the difference is within the fault tolerance range, it will not be transmitted, and the corresponding position will be marked as "0" in the longitudinal position map; if the difference exceeds the fault tolerance range, the difference will be retained, and the corresponding position will be marked as "1" in the longitudinal position map, and the longitudinal position information with a longitudinal position of "1" and its corresponding difference information will be sent. By setting a fault tolerance range, the present application ignores some details that do not change very much, which can reflect the required information and achieve the purpose of reducing the compression volume.

[0035] When the change rate between the subsequent frame data and the reference table of the benchmark data is in the first change rate interval, the following step S42 is executed.

[0036] S42, compare the current position valid data of the post-frame data with the corresponding position valid data of the reference frame, calculate the difference, and record the position information of all the differences, send the position information whose difference is not "0" and its corresponding difference information, to ensure that the receiving end can correctly match the difference information to the destination according to its position information. The change rate in the first change rate interval indicates that most of the 16-byte position map is vacant, the change is relatively small, and the same data is mostly, so it is easier to extract the common points. At this time, it is necessary to add the position information of this difference before each difference, mark the row and column information, and for large data volume, which leads to too long row and column information, the data can be appropriately divided into different pages, and each page has fixed length data, so that only one byte of position information for recording the change value can meet the demand.

[0037] When the change rate between the subsequent frame data and the reference table of the benchmark data is in the third change rate interval, the following step S43 is executed.

[0038] S43, compare the valid data of the current position of the post-frame data with the valid data of the corresponding position of the reference frame, calculate the difference, and send all the difference information. The change rate in the third change rate interval proves that most of the data has changed. If the data is compressed and then the head, tail, and position map are inserted, the data transmission content will increase, so the difference of all positions of the entire map is directly sent. Among them, if the difference exceeds 2 bytes, it can be negotiated to convert it by 10 times first.

[0039] Furthermore, different data headers are added to the data sent in steps S41, S42, and S43, respectively, and the receiving end decompresses the data according to different processing logics by identifying the data headers during compression.

[0040] In addition, CRC check is performed on the data sent in steps S41, S42, and S43, and information files of "header frame", "data frame", and "tail frame" are added to the data according to the data type so that the receiving end can correctly parse the received data.

[0041] Similarly, the data parsing step at the receiving end is the reverse operation of compression. After receiving the data, the receiving end performs verification processing, and then caches the header frame according to the data tag. According to the graph information and the content of the header frame, each bit of data is updated to parse out the complete valid data.

[0042] To further illustrate the present application, the first threshold is specifically set to 30%, and the second threshold is specifically set to 60%. Taking an 8*8 pixel, 16-bit color picture as an example, the first frame picture data is as follows:

[0043] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0044] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0045] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0046] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0047] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0048] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0049] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0050] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0051] It can be confirmed that the effective data length is 2 bytes, and the error tolerance range is set to 200 accordingly.

[0052] Taking the first frame image as the reference table of benchmark data, the subsequent frame image data is divided into the following three cases:

[0053] 1. If the next frame image data is:

[0054] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0055] 0X00,0X20,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0XC0,0X00,0X00,

[0056] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00, 0X00,0X00,0XAA,0X55,0XAA,0X55,0XAA,0X55,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0057] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0XAA,0X00,0X00,0X00,0X00,

[0058] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0059] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0060] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0061] Then the change rate between the latter frame image data and the first frame image data is less than 30%, and the difference between the two is:

[0062] 0,0,0,0,0,0,0,0,

[0063] 32,0,0,0,0,0,192,0,

[0064] 0,0,0,0,0,0,0,0,

[0065] 0,43650,43650,43650,0,0,0,0,

[0066] 0,0,0,0,0,170,0,0,

[0067] 0,0,0,0,0,0,0,0,

[0068] 0,0,0,0,0,0,0,0,

[0069] 0,0,0,0,0,0,0,0,

[0070] The first difference position is recorded as 0x00, the second difference position is recorded as 0x01, and so on, and the position can be recorded up to 0xFF; if there is too much data, the first difference position is recorded as 0x0000, the second difference position is recorded as 0x0001, and so on, and the position can be recorded up to 0xFFFF.

[0071] Then send data

[0072] 0xA1, 0x09, 0x20, 0x0F, 0xC0,…CRC

[0073] Data header, position information, difference information, position information, difference information, ... CRC check.

[0074] 2. If the next frame image data is:

[0075] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0076] 0X00,0X20,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0XC0,0X00,0X00,

[0077] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0078] 0X00,0X00,0XAA,0X55,0XAA,0X55,0XAA,0X55,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0079] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0XAA,0X00,0X00,0X00,0X00,0XAA,0X00,0XAA,0X00,0XAA,0X00,0XAA,0X00, 0XAA,0X00,0XAA,0X00,0XAA,0X00,0XAA,0X00,0X00,0XC0,0X00,0XC0, 0X00,0XC0,0X00,0XC0,0X00,0XC0,0X00,0XC0,0X00,0XC0,0X00,0XC0,

[0080] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0081] Then the change rate between the latter frame image data and the first frame image data is between 30% and 60%, and the difference between the two is:

[0082] 0,0,0,0,0,0,0,0,

[0083] 32,0,0,0,0,0,192,0,

[0084] 0,0,0,0,0,0,0,0,

[0085] 0,43650,43650,43650,0,0,0,0,

[0086] 0,0,0,0,0,170,0,0,

[0087] 43650,43650,43650,43650,43650,43650,43650,43650,

[0088] 192,192,192,192,192,192,192,192,

[0089] 0,0,0,0,0,0,0,0,

[0090] Define the first 8 bytes of each row of data to be sent as the vertical position map. The vertical position map of the next frame image data and the first frame image data is:

[0091] 0,0,0,0,0,0,0,0,

[0092] 0,0,0,0,0,0,0,0,

[0093] 0,0,0,0,0,0,0,0,

[0094] 0,1,1,1,0,0,0,0,

[0095] 0,0,0,0,0,0,0,0,

[0096] 1,1,1,1,1,1,1,1,

[0097] 0,0,0,0,0,0,0,0,

[0098] 0,0,0,0,0,0,0,0,

[0099] The data sent is:

[0100] 0xA2, 0x00, 0x00, CRC

[0101] 0xA2, 0x01, 0x00, CRC

[0102] 0xA2, 0x02, 0x00, CRC

[0103] 0xA2, 0x03, 0x70, 0xAA, 0x55, 0xAA, 0x55, 0xAA, 0x55, CRC

[0104] …

[0105] Data header, row number information, vertical position information, difference information, CRC check.

[0106] 3. If the next frame image data is:

[0107] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0108] 0X00,0X20,0X00,0X20,0X00,0X20,0X00,0X20,0X00,0X20,0X00,0X20,0X00,0X20,0X00,0X20,

[0109] 0X00,0X20,0X00,0X20,0X00,0X20,0X00,0X20,0X00,0X20,0X00,0X20,0X00,0X20,0X00,0X20,0X00,0X00,0XAA,0X55, 0XAA,0X55,0XAA,0X55,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0XAA,0X00,0XAA,0X00,0XAA,0X00,0XAA, 0X00,0XAA,0X00,0XAA,0X00,0XAA,0X00,0XAA,0XAA,0X00,0XAA,0X00,0XAA,0X00,0XAA,0X00,0XAA,0X00,0XAA,0X00, 0XAA,0X00,0XAA,0X00,0X00,0XC0,0X00,0XC0,0X00,0XC0,0X00,0XC0,0X00,0XC0,0X00,0XC0,0X00,0XC0,0X00,0XC0,

[0110] 0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,0X00,

[0111] The change rate between the latter frame image data and the first frame image data is greater than 60%, and the difference between the two is:

[0112] 0,0,0,0,0,0,0,0,

[0113] 32,32,32,32,32,32,32,32,

[0114] 32,32,32,32,32,32,32,32,

[0115] 0,43650,43650,43650,0,0,0,0,

[0116] 170,170,170,170,170,170,170,170,

[0117] 43650,43650,43650,43650,43650,43650,43650,43650,

[0118] 192,192,192,192,192,192,192,192,

[0119] 0,0,0,0,0,0,0,0,

[0120] The data sent is:

[0121] 0xA3, 0x00, 0x00, 0x00, 0x00,…0x00, 0xC0, 0x00, 0xC0,… CRC

[0122] Data header, difference information, difference information, difference information...CRC check.

[0123] It can be seen that the data compression method suitable for low-level embedded systems of the present invention can provide a higher data compression rate to a certain extent, and is more suitable for low-level embedded systems, such as warning prompts, voice prompts, real-time driving image recording, matrix sensor data storage, etc. in current advanced driver assistance. By sacrificing the allowable error of part of the data, the hardware requirements of the corresponding functions are reduced by several levels, saving costs and reducing power consumption; at the same time, the entire data performance and application meet the relevant requirements, and the final data application does not generate or generates some errors caused by data compression that are more acceptable than increasing hardware costs. By replacing the existing complex compression transmission method with a simple and easy-to-implement compression method, the project development speed is faster, the entire compression method is easier to maintain and transplant, and the simple compression method can reduce the occurrence of anomalies and locate problems faster.

[0124] Many specific details are described in the above description to facilitate a full understanding of the present invention. However, the above description is only a preferred embodiment of the present invention. The present invention can be implemented in many other ways different from those described herein, so the present invention is not limited to the specific implementation disclosed above. At the same time, any person familiar with the art can make many possible changes and modifications to the technical solution of the present invention using the methods and technical contents disclosed above without departing from the scope of the technical solution of the present invention, or modify it into an equivalent embodiment of equivalent changes. Any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of protection of the technical solution of the present invention.

Claims

1. A data compression method suitable for low-level embedded systems, characterized in that: The following steps are involved: S1, process the target data to obtain the required data format and confirm the minimum valid data length; S2, preset the error tolerance range according to the effective data length; S3, collecting a reference frame as a reference data reference table. Subsequent data compression and decompression are based on this reference data reference table, and update synchronization is performed based on this reference data reference table; S40, comparing the subsequent frame data with the reference table of the benchmark data to determine the change rate between the two, and when the change rate between the subsequent frame data and the reference table of the benchmark data is in a set second change rate interval, executing S41; wherein the change rate is divided into three intervals by setting a first threshold and a second threshold, the first threshold is less than the second threshold, and the interval less than the first threshold is defined as the first change rate interval, the interval between the first threshold and the second threshold is defined as the second change rate interval, and the interval greater than the second threshold is defined as the third change rate interval; S41, compressing the data line by line vertically through the reference frame, comparing whether the changes before and after each section of valid data are within a preset error tolerance range, and determining whether the valid data needs to be transmitted; In the S41, first, the first several bytes of each row of data to be sent are defined as a longitudinal position map; secondly, the current position valid data of the subsequent frame data is compared with the corresponding position valid data of the reference frame, and the difference is calculated. If the difference is within the fault tolerance range, it is not transmitted, and the corresponding position is marked as "0" in the longitudinal position map; if the difference exceeds the fault tolerance range, the difference is retained, and the corresponding position is marked as "1" in the longitudinal position map, and the longitudinal position information of the longitudinal position "1" and its corresponding difference information are sent.

2. The data compression method suitable for low-level embedded systems according to claim 1, characterized in that: When the change rate between the post-frame data and the reference table of the benchmark data is in the first change rate range, execute step S42, compare the current position valid data of the post-frame data with the corresponding position valid data of the benchmark frame to calculate the difference, and record the position information of all the differences, and send the position information whose difference is not "0" and its corresponding difference information.

3. The data compression method suitable for low-level embedded systems according to claim 2, characterized in that: When the change rate between the subsequent frame data and the reference table of the benchmark data is in the third change rate interval, step S43 is executed to compare the current position valid data of the subsequent frame data with the corresponding position valid data of the benchmark frame to calculate the difference and send all the difference information.

4. The data compression method suitable for low-level embedded systems according to claim 3, characterized in that: Different data headers are added to the data sent in steps S41, S42, and S43 respectively, and the receiving end decompresses the data according to different processing logics by identifying the data headers.

5. The data compression method suitable for low-level embedded systems according to claim 3, characterized in that: The data sent in steps S41, S42, and S43 are all subjected to CRC check, and information files of "header frame", "data frame", and "tail frame" are added to the data according to the data type.

Images