Method, transmitter and system for continuously optimizing data compression

Abstract

A method is provided for continuously optimizing data compression using a frequency-based dictionary lookup table for communication between a transmitter and a receiver in a vehicle. The method includes the following steps: providing a first and a second frequency-based dictionary lookup table in a transmitter. The transmitter receives a data block and compresses the data block using the first table. The second table is updated using the data block, and a difference between the compression efficiency of the first table and the compression efficiency of the second table is calculated for the data block. The compressed data block is transmitted to the receiver. When the difference is greater than a predefined threshold, the contents of the first table in the transmitter are replaced with the updated contents of the second table. The updated contents of the second table are then transmitted to the receiver.

Description

Background Art

[0001] Modern vehicles can use V2X communications to transmit data to other vehicles, central servers, and / or infrastructure entities along the road (for example, traffic lights). Data within the vehicle can be transmitted between sensors and ECUs, infotainment systems, etc. The bandwidth required between the transmitter and receiver can be large. Because more bandwidth is needed to exchange large amounts of data between systems, systems have emerged that can process gigabytes of data in seconds.

[0002] Information exchanged between systems can be sent in a symbolic manner that reduces the required bandwidth, i.e., information representing the data is transmitted without transmitting the data itself. This is the basis of many compression techniques.

[0003] However, if data blocks are compressed and decompressed, the real-time nature of data transmission may be lost when the data blocks are too large, and compression may not be efficient when the data blocks are too small.

[0004] A different approach is to apply probability theory to find the maximum to minimum frequency of occurrence of data entities and give the symbol or codeword assigned to each data entity so that the data entity with the maximum frequency of occurrence has the symbol or codeword of the minimum size, and the data entity with the minimum frequency of occurrence has the symbol or codeword of the desired maximum size. The relationship between the symbol or codeword and the data entity can be found in a dictionary lookup table. This compression technique is used in many lossless data compression algorithms such as LZW and Huffman coding.

[0005] However, the frequency of occurrence of data entities may change over time, which may render the dictionary lookup table ineffective or inefficient.

[0006] Purpose of the Invention

[0007] It is an object of the present invention to optimize data compression using a frequency based dictionary lookup table in a vehicle. Summary of the invention

[0008] In the present invention, a method is provided for continuously optimizing data compression using a frequency-based dictionary lookup table for communication between a transmitter and a receiver in a vehicle, the method comprising the following steps:

[0009] - providing first and second frequency-based dictionary look-up tables in said transmitter;

[0010] - receiving a data block in said transmitter;

[0011] - compressing said data block in said transmitter using said first table;

[0012] - using the data block to update the content of the second table, and calculating the difference between the compression efficiency of the first table and the compression efficiency of the second table for the data block;

[0013] - transmitting the compressed data block to the receiver; and,

[0014] - When the difference is greater than a predefined threshold, replacing the content of the first table in the transmitter with the content of the updated second table, and transmitting the content of the updated second table to the receiver.

[0015] The transmitter may be a sensor or part of an electronic control unit (ECU) in a vehicle. The data may be sensor data or / and control data. The receiver may be part of another sensor or another ECU in the same vehicle. Thus, the receiver may be located in the same vehicle as the transmitter. In one or more embodiments, the receiver may be located outside the vehicle where the transmitter is located.

[0016] The communication between the transmitter and the receiver can be based on a wired communication protocol (such as Ethernet or CAN), or it can be based on a wireless communication protocol (such as 4G / 5G or Wifi or V2X communication).

[0017] The vehicle can be a car, motorcycle, van or truck. The vehicle can also be a bicycle or scooter.

[0018] The phrase "continuously optimizing data compression" indicates that while a data block is continuously compressed and transmitted, a dictionary lookup table is simultaneously updated using the same data block to make compression of the next data block more efficient.

[0019] Thus, the step of compressing the data block in the transmitter and / or the step of transmitting the compressed data block to the receiver may be performed simultaneously with the step of updating the content of the second table and / or the step of calculating the difference.

[0020] A frequency-based dictionary lookup table is a table that links data entities to codewords. In this case, the length or size of the codeword for the data entity is related to the frequency of occurrence of the data entity in the data. The table may be based on Huffman coding. Two tables are provided, namely, a first and a second frequency-based dictionary lookup table.

[0021] The received data block may include data entities listed in the first table, and may include data entities not listed in the first table.

[0022] When compressing a data block, the data entities listed in the first table may be replaced with the code words corresponding to these data entities. Data entities not listed in the first table are not replaced and are transmitted in their original form. Thus, a compressed data block may include code words and data entities that have not been replaced.

[0023] The received data block is not only compressed, but also used to update the second table. Based on the frequency of occurrence of the data entity in the received data block, the second table is adjusted or updated. This may include adding or removing one or more data entities from the second table and / or changing the corresponding codewords of one or more data entities in the second table.

[0024] The compression efficiency of a table indicates to what extent a data block has been compressed using the table, or how much smaller the bit size of a data block compressed using the table is relative to an uncompressed data block. Thus, the compression efficiency may be defined as the difference in bit size between a compressed data block and an uncompressed data block divided by the bit size of the uncompressed data block.

[0025] It may be the case that the compression efficiency of the second table is higher than that of the first table for a received data block. In this case, the content of the first table is replaced by the content of the second table. In other words, the first table becomes identical to the second table. With this first table, the next received data block will be compressed, while the second table will be updated with the next received data block. The updated content of the second table is also transmitted to the receiver, since a copy of the table used for compression is required for decompression in the receiver.

[0026] Since the second table is generated based on the received data, it can be expected that the compression efficiency of the second table is indeed higher than that of the first table for the received data block. When the content of the first table is replaced by the content of the updated second table, compression of the data block cannot be performed. Therefore, the replacement content has a negative impact on the amount of data transmitted per unit time. In addition, transmitting the content of the updated second table to the receiver will also require some bandwidth, which will also have a negative impact on the amount of data transmitted per unit time.

[0027] Therefore, only when the difference between the compression efficiency of the first table and the second table is greater than a predefined threshold, the content of the first table is replaced and the updated content of the second table is transmitted to the receiver. This limits the occurrence of the above two negative effects.

[0028] Replacing the contents of the first table with the updated contents of the second table may improve compression efficiency and thus have a positive impact on the amount of data transferred per unit time.

[0029] The predefined threshold may be selected such that the above two negative effects do not offset the above positive effect.

[0030] In one or more embodiments, the method further comprises the following steps:

[0031] - providing a copy of said first table in said receiver;

[0032] - receiving said compressed data block in said receiver;

[0033] - decompressing the received compressed data block in the receiver using the copy;

[0034] - receiving in said receiver the contents of said updated second table; and,

[0035] - replacing the contents of said copy in said receiver with the contents of the received updated second table.

[0036] In one or more embodiments, the data block includes data entities listed in the first table and / or data entities not listed in the first table, and wherein the compressed data block includes data entities not listed in the first table and / or codewords replacing these data entities listed in the first table.

[0037] In one or more embodiments, the compression and / or decompression is based on Huffman data compression.

[0038] A computer program is provided, wherein the computer program is adapted to perform any of the methods described in this document when the computer program is executed on a computer.

[0039] A computer readable storage medium is provided that stores a set of controller executable codes that, when executed in a device, a processing system, or a digital signal processor, performs any of the methods described in this document.

[0040] A transmitter is provided for continuously optimizing data compression using a frequency-based dictionary lookup table for communication between the transmitter and a receiver in a vehicle, the transmitter comprising

[0041] - a memory unit having first and second frequency-based dictionary lookup tables;

[0042] - a compressor unit for (i) receiving a data block, and (ii) compressing the data block in the transmitter using the first table;

[0043] - a learning unit configured to (i) receive the data block; (ii) update the content of the second table using the data block; (iii) calculate the difference between the compression efficiency of the first table and the compression efficiency of the second table for the data block; and (iv) when the difference is greater than a predefined threshold, replace the content of the first table with the updated content of the second table; and,

[0044] - an output unit for (i) transmitting the compressed data block to the receiver, and (ii) transmitting the updated content of the second table to the receiver when the difference is greater than the predefined threshold.

[0045] In one or more embodiments of the transmitter, the data block includes data entities listed in the first table and / or data entities not listed in the first table, and wherein the compressed data block includes data entities not listed in the first table and / or codewords replacing these data entities listed in the first table.

[0046] A system is provided for continuously optimizing data compression using a frequency-based dictionary lookup table for communication between a transmitter and a receiver in a vehicle, the system comprising a transmitter according to any embodiment described in this document, and a receiver, wherein the receiver comprises:

[0047] - a memory unit having a copy of said first frequency-based dictionary lookup table;

[0048] - a decompressor unit for (i) receiving the compressed data block, and (ii) decompressing the received compressed data block using the copy;

[0049] - an updating unit configured to (i) receive the updated content of the second table, and (ii) replace the content of the replica with the received updated content of the second table.

[0050] Furthermore, the vehicle is provided with a transmitter according to any embodiment of the transmitter described in this document or with a system according to any embodiment of the system described in this document.

[0051] It is understood that: (i) transmitters, systems and / or operation of the systems, (ii) related advantages, and (iii) further embodiments of transmitters, systems and / or vehicles according to corresponding method steps as described in this document. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] For a more complete understanding of the present invention, reference will be made to the following description taken in conjunction with the accompanying drawings, in which:

[0053] Figure 1 Schematically illustrates a transmitter, a receiver and a system according to one or more embodiments of the present invention;

[0054] Figure 2 schematically illustrates a vehicle according to one or more embodiments of the present invention; and,

[0055] Figure 3 Methods according to one or more embodiments of the invention are schematically illustrated. DETAILED DESCRIPTION

[0056] Figure 1 Schematically illustrates a transmitter 110, a receiver 120, and a system 100 according to one or more embodiments of the present invention. In one or more embodiments, the transmitter 110 may include a memory unit 160, a compressor unit 140, a learning unit 140, and / or an output unit 150. In the memory unit 160, in one or more embodiments, a first and a second frequency-based dictionary lookup table may be stored. Examples of such tables are discussed below.

[0057] The compressor unit 140 may be arranged to receive a data block (e.g. from a data generator using the data communication line 105). The data generator may be a camera or a sensor. The transmitter and the data generator may be combined in one device. The compressor unit 140 is arranged to compress the received data block using a first frequency-based dictionary lookup table.

[0058] Although the receiver may receive data in a continuous manner, compression and decompression will occur sequentially on the data blocks. Compression and decompression may occur in real time, which may indicate that both the data generator and the data user (e.g., a CPU processing decompressed data) may generate and / or receive data blocks at, for example, a rate of 100 Mb / s without any delay caused by compression and decompression.

[0059] In one or more embodiments, the data block includes data entities listed in the first table and / or data entities not listed in the first table. The data entity may be a string of a certain length. For those data entities listed in the first table, the frequency of occurrence of these data entities in previous data blocks and the codewords that will replace these data entities during compression may have been established. Those data entities not listed in the first table may be new data entities that are not included in previous data blocks, or the frequency of occurrence of these data entities is so low compared to other data entities that replacement with codewords does not improve compression efficiency. And therefore, these data entities are not listed in the first table and are not compressed.

[0060] Thus, a compressed data block may include compressed data entities and uncompressed data entities. The uncompressed data entities may be marked with a unique identifier so that the receiver 120 or the decompressor unit 190 will not attempt to decompress these data entities.

[0061] The learning unit 140 may be arranged to receive the same data block and use the data block to update a second frequency-based dictionary lookup table. The updated second table is optimized for compressing the received data block. The learning unit may compare the compression efficiency of the first and second frequency-based dictionary lookup tables, and when the difference is greater than a predefined threshold, replace the contents of the first table (which may be stored in the memory unit 160) with the contents of the updated second table (which may also be stored in the memory unit 160). In one or more embodiments, the predefined threshold may be an increase of 10% or 25%. In one or more embodiments, the learning unit 140 may be arranged to send the contents of the updated second table to the output unit 150.

[0062] In one or more embodiments, the compression efficiency of the first table and the second table is determined based on more than one received data block or based on the number of data blocks received within a predefined time period (eg, 1 second).

[0063] When the contents of a table are replaced, the table cannot be used for compression. This will have a negative impact on the amount of data transmitted per unit time. Therefore, in one or more embodiments, replacing the contents of the first table with the contents of the updated second table can be performed as follows: a temporary first frequency-based dictionary lookup table can be stored in the memory unit 160. The learning unit 140 replaces the contents of the temporary first table with the contents of the updated second table. The learning unit 140 then changes (or renames) (i) the temporary first table to the first table, and changes (or renames) (ii) the first table to the temporary first table. Since replacing the contents of a table may take more time than changing or renaming a table, the time period during which the table is unavailable for compression is reduced.

[0064] The output unit 150 may be arranged to transmit the compressed data block to the receiver 120 using the data communication connection 115. Such a data communication connection may be wired or wireless, it may be based on Ethernet, Wifi or 5G standards. The output unit 150 may also be arranged to transmit the content of the updated second table to the receiver 120 when the compression efficiency difference is greater than the predefined threshold.

[0065] In one or more embodiments, the output unit 150 is further arranged to combine the compressed data block with the updated content of the second table. The updated content of the second table can be marked with a unique identifier so that the receiver 120 or the decompression unit 190 does not attempt to decompress the content, and the update unit 170 identifies the identifier as the updated content of the second table.

[0066] In one or more embodiments, the receiver 120 may include a memory unit 180 having a copy of the first frequency-based dictionary lookup table, a decompressor unit 190, and an update unit 170. The decompressor unit 190 may be arranged to receive the compressed data block via the data communication connection 115 and decompress the received compressed data block using the copy of the first table stored on the memory unit 180.

[0067] The decompressor unit 190 may further be arranged to transmit the decompressed data blocks using a data communication connection 125 which may be wired or wireless. The connection 125 is based on Ethernet, Wifi or 5G standards. The decompressed data blocks transmitted via the connection 125 are identical to the data blocks received via the connection 105.

[0068] In one or more embodiments, the update unit 170 may be arranged for receiving the updated contents of the second table, preferably via the connection 115, and for replacing the contents of the copy of the first table stored on the memory unit 180 with the received updated contents of the second table.

[0069] In one or more embodiments, replacing the contents of the replica with the contents of the updated second table may be performed as follows: a temporary replica may be stored in the memory unit 160. The update unit 170 replaces the contents of the temporary replica with the contents of the updated second table. The update unit 170 then changes (or renames) (i) the temporary replica to the replica, and changes (or renames) (ii) the replica to the temporary replica. Since replacing the contents of the table may take more time than changing or renaming the contents, the period of time during which the replica is unavailable for decompression is reduced.

[0070] exist Figure 1 , a system 100 for continuously optimizing data compression using a frequency-based dictionary lookup table for communication between a transmitter 110 and a receiver 120 in a vehicle is schematically depicted according to one or more embodiments. The receiver 120 may be located inside or outside the vehicle.

[0071] exist Figure 2, a vehicle 200 according to one or more embodiments of the present invention is schematically depicted. In one or more embodiments, the vehicle 200 may include a transmitter 110 according to one or more embodiments described in this document and / or a receiver 120 according to one or more embodiments described in this document.

[0072] Figure 3 A method 300 according to one or more embodiments of the present invention is schematically illustrated. The method shows the following steps:

[0073] Step 310: Providing first and second frequency-based dictionary lookup tables in the transmitter;

[0074] Step 315: receiving a data block in the transmitter;

[0075] Step 320: compressing the data block in the transmitter using the first table;

[0076] Step 325: Use the data block to update the content of the second table, and calculate the difference between the compression efficiency of the first table and the compression efficiency of the second table for the data block;

[0077] Step 330: Transmitting the compressed data block to the receiver; and,

[0078] Step 335: When the difference is greater than a predefined threshold, the content of the first table in the transmitter is replaced with the content of the updated second table, and the content of the updated second table is transmitted to the receiver.

[0079] In one or more embodiments, the method 300 may further include the following steps:

[0080] Step 340: Providing a copy of the first table in the receiver;

[0081] Step 345: receiving the compressed data block in the receiver;

[0082] Step 350: Decompressing the received compressed data block in the receiver using the replica;

[0083] Step 355: receiving the updated content of the second table in the receiver; and,

[0084] Step 360: Replace the content of the copy in the receiver with the received updated content of the second table.

[0085] In one or more embodiments, the number of different data blocks may be limited, for example, when the data block involves the number of repetitive steps or between the minimum and maximum values ​​of a given system. For example: a CAN signal carrying the engine RPM value may only have values ​​between 0 and 8000RPM with a step size of 10RPM. 2. Or in another example: a 12-bit analog-to-digital converter (ADC) with an input range of 0 to 10V may only produce a digital output with a step size of 0.0024 from 0 to 4095. Since the possible data values ​​in the data block are fixed and these data values ​​are continuous and can follow a normal distribution, effective data compression can be performed.

[0086] For example, consider two blocks of data:

[0087] - Data block 1 = {3A 3A B1 26 26F0 F0 26 17 3A AB AB AB AB AB AB AB AB AB 9C11}*50 repetitions; and,

[0088] - Data block 2 = {3A 3A AB 26 26F0 F0 26 17 3A B1 B1 B1 B1 B1 B1 B1 B1 B1 9C11}*50 repetitions.

[0089] Each data block has 1050 data entities. In this example, the frequency of occurrence of each entity is carefully selected so that the most occurring entity in data block 1 is "AB", and the most occurring entity in data block 2 is "B1".

[0090] For data block 1, the probability distribution is as follows

[0091] symbol Symbol Count Probability B1 50 0.047619048 17 50 0.047619048 3A 250 0.238095238 26 150 0.142857143 AB 450 0.428571429 9C 50 0.047619048 11 50 0.047619048

[0092] Based on the probability distribution, the following frequency-based dictionary lookup table can be generated:

[0093]

[0094]

[0095] The table can be used as the first table for compressing and decompressing data blocks. The table can be optimized for data block 1. When assuming that data block 2 is the next data block received by the transmitter, the learning unit 140 can generate the following probability distribution for data block 2:

[0096] symbol Symbol Count Probability B1 450 0.428571429 17 50 0.047619048 3A 250 0.238095238 26 150 0.142857143 AB 50 0.047619048 9C 50 0.047619048 11 50 0.047619048

[0097] Based on the probability distribution, the following frequency-based dictionary lookup table can be generated:

[0098] symbol Huffman Coding B1 1 17 00001 3A 01 26 001 AB 00000 9C 00011 11 00010

[0099] This table can be used as the second table. The learning unit 140 can now calculate the difference between the compression efficiency of the first table and the compression efficiency of the second table for data block 2:

[0100] First table Second table Original data size 1150 bytes 1150 bytes Compressed data size 500 bytes 300 bytes Compression efficiency 56.5％ 73.9％

[0101] The calculated difference is 17.4%. When the predefined threshold for replacing the contents of the first table with the contents of the second table is 20%, the contents of the first table in the transmitter (and the copy in the receiver) will not be replaced or updated.

[0102] In addition, one or more embodiments may be described as follows: A method is provided for continuously optimizing data compression using a frequency-based dictionary lookup table for communication between a transmitter and a receiver in a vehicle. The method includes the following steps: providing a first and a second frequency-based dictionary lookup table in a transmitter. The transmitter receives a data block and compresses the data block using the first table. The second table is updated using the data block, and the difference between the compression efficiency of the first table and the compression efficiency of the second table is calculated for the data block. The compressed data block is transmitted to the receiver. When the difference is greater than a predefined threshold, the content of the first table in the transmitter is replaced with the updated content of the second table. The updated content of the second table is then transmitted to the receiver.

[0103] As will be appreciated by those skilled in the art, aspects of the present invention may be implemented as a system, device, method or computer program product. Accordingly, aspects of the present invention may take the form of a complete hardware embodiment, a complete software embodiment (including firmware, resident software, microcode, etc.) or a combination of software and hardware aspects generally referred to herein as a "unit", "module", "system", "device" or "element".

[0104] The functions or steps described in this document may be implemented as an algorithm executed by a microprocessor of a computer. In addition, aspects of the present invention may also take the form of a computer program product implemented in one or more computer-readable media, the computer-readable media having a computer-readable program code implemented (e.g., stored) thereon.

[0105] It should be understood that the above description is included to illustrate the operation of the preferred embodiment and is not intended to limit the scope of the invention. Based on the above discussion, many variations will be apparent to those skilled in the art, and these variations will still be included within the scope of the invention.

Claims

1. A method for continuously optimizing data compression based on Huffman data compression using a frequency-based dictionary lookup table for communication between a transmitter and a receiver in a vehicle, in, "Continue optimizing data compression" indicates that while continuously compressing and transmitting a data block, the dictionary lookup table is updated with the same data block to make the compression of the next data block more efficient. The compression efficiency is defined as the difference in bit size between the compressed and uncompressed data blocks divided by the bit size of the uncompressed data block. Wherein, the frequency-based dictionary lookup table is a table that links data entities to codewords, wherein the length or size of the codeword of the data entity is related to the frequency of occurrence of the data entity in the data, wherein the table is based on Huffman coding, The method comprises the following steps: providing first and second frequency-based dictionary lookup tables in said transmitter and providing a copy of said first table in said receiver; receiving a data block in the transmitter; compressing the data block in the transmitter using the first table; updating the contents of the second table using the data block and calculating the difference between the compression efficiency of the first table and the compression efficiency of the second table for the data block; transmitting the compressed data block to the receiver; and decompressing the received compressed data block in the receiver using the copy of the first table; When the difference is greater than a predefined threshold, replacing the content of the first table in the transmitter with the updated content of the second table, and transmitting the updated content of the second table to the receiver; receiving, in the receiver, the updated contents of the second table; and, The contents of the copy in the receiver are replaced with the contents of the received updated second table.

2. A method according to any one of the preceding claims, wherein: The data block includes data entities listed in the first table and / or data entities not listed in the first table, and wherein the compressed data block includes data entities not listed in the first table and / or codewords replacing data entities listed in the first table.

3. A computer program product, wherein: The computer program is adapted to perform the method according to any of the preceding claims when the computer program is executed on a computer.

4. A computer readable storage medium storing a set of controller executable codes which, when executed in a device, on a processing system or in a digital signal processor, perform the steps of the method according to any one of the preceding claims.

5. A transmitter for continuously optimizing data compression based on Huffman data compression using a frequency-based dictionary lookup table for communication between said transmitter and a receiver in a vehicle, in, "Continue optimizing data compression" indicates that while continuously compressing and transmitting a data block, the dictionary lookup table is updated with the same data block to make the compression of the next data block more efficient. The compression efficiency is defined as the difference in bit size between the compressed and uncompressed data blocks divided by the bit size of the uncompressed data block. Wherein, the frequency-based dictionary lookup table is a table that links data entities to codewords, wherein the length or size of the codeword of the data entity is related to the frequency of occurrence of the data entity in the data, wherein the table is based on Huffman coding, The transmitter includes: a memory unit having first and second frequency-based dictionary lookup tables; a compressor unit for (i) receiving a data block, and (ii) compressing the data block in the transmitter using the first table; a learning unit, the learning unit being configured to (i) receive the data block; (ii) update the content of the second table using the data block; (iii) calculate the difference between the compression efficiency of the first table and the compression efficiency of the second table for the data block; and (iv) when the difference is greater than a predefined threshold, replace the content of the first table with the updated content of the second table; and, An output unit is configured to (i) transmit the compressed data block to the receiver, and (ii) transmit the updated content of the second table to the receiver when the difference is greater than the predefined threshold.

6. The transmitter according to claim 5, wherein: The data block includes data entities listed in the first table and / or data entities not listed in the first table, and wherein the compressed data block includes data entities not listed in the first table and / or codewords replacing data entities listed in the first table.

7. A system for continuously optimizing data compression using a frequency-based dictionary lookup table for communication between a transmitter and a receiver in a vehicle, the system comprising a transmitter according to claim 5 or 6, and a receiver, wherein: The receiver comprises: a memory unit having a copy of the first frequency-based dictionary lookup table; a decompressor unit for (i) receiving the compressed data block, and (ii) decompressing the received compressed data block using the replica; An updating unit is configured to (i) receive the updated content of the second table, and (ii) replace the content of the copy with the received updated content of the second table.

8. A vehicle having a transmitter according to claim 5 or 6 or a system according to claim 7.

Images