Plain text data compression and restoration method based on multi-mapping and computer system

Abstract

The present invention discloses a method and computer system for compressing and restoring plain text data based on multiple mappings. The present invention is dedicated to providing a solution for compressing, identifying, reading and restoring large amounts of plain text information sets. It adopts user-defined multiple data mapping tables to segment, extract, compare, replace and restore plain text information. The data mapping table is open to users and can be customized by users. Users can construct data mapping tables according to their industry fields. The data compression process is actually also an encryption process, which greatly ensures the security of the data. The present invention is implemented in the form of middleware, which is simple to call, easy to use, and highly compatible. It is suitable for application programs developed for Windows, Linux, Android, IOS, and Hongmeng operating systems.

Description

Technical Field

[0001] The present invention relates to the technical field of data compression and restoration, and in particular to a method and computer device for compressing and restoring plain text data based on multiple mappings, which can be used in application scenarios such as plain text transmission, QR code label printing when the data volume is large, and RFID electronic tag data storage. Background Art

[0002] With the development of society, the progress of the times, the rise of mobile Internet and the Internet of Things, and the increasingly high requirements for real-time information transmission and large-scale data transmission, the information transmitted and stored is limited when the communication bandwidth is small (such as Beidou short message), the size of the QR code label is fixed, and the capacity of the RFID electronic tag is limited. In many scenarios, it cannot meet user needs. To meet this demand, an algorithm has been developed that uses a user-defined data mapping table to segment, extract, compare, replace, and restore plain text data to solve problems in the fields of large-scale plain text transmission, large-scale QR code label printing, and RFID electronic tag data storage. Summary of the Invention

[0003] In view of this, one of the objects of the present invention is to provide a plain text data compression method based on multiple mapping, which can solve the problems raised in the background technology; the second object of the present invention is to provide a plain text data restoration method based on multiple mapping on the basis of the first object.

[0004] One of the purposes of the present invention is achieved through the following technical solutions:.

[0005] A plain text data compression method based on multiple mappings is characterized in that the method comprises the following steps:

[0006] Set a data mapping table, which can be set to N levels as needed, where N is greater than 1 and a positive integer. The elements included in the first level data mapping table include "sequence number", "mapping value", "word", "special character replacement value" and "mapping type". The field types of the "mapping value", "word" and "special character replacement value" are all character types, where "mapping value" represents the data mapping replacement value, the "word" is used to search for characters existing in the plain text, and the "special character replacement value" is used to replace the special character that is the same as the mapping value with the special character replacement value when it appears; the field type of "mapping type" is a numeric type, which is used to identify the multiplicity of the mapping value. If the setting value is greater than 1, multiple data mappings are required. If the setting value is N, N data mappings are required; the elements included in the second to Nth level data mapping tables include "sequence number", "mapping value", "word" and "mapping type";

[0007] Use the middleware to receive the original data that needs to be compressed. The original data is in plain text format, and then set the data encoding format of the plain text data.

[0008] To escape special characters, first automatically build a special character set through the "mapping value" of a data mapping table, and automatically build a special character replacement value through the "special character replacement value" of the data mapping table. If special characters exist in the original data, the special characters in the original data are replaced with the "special character replacement value" according to the replacement value.

[0009] The escaped original data is segmented by a word segmentation algorithm, and then the segmented data is stored and the number of word occurrences is calculated;

[0010] Escape data replacement, cyclically replace the data list according to the number of word occurrences from high to low, and record the plain text information after the escape data replacement.

[0011] Multiple data replacement, that is, multiple data conversions are performed according to the "mapping type" of the data mapping table, the result of the previous layer is used as the "data input" of the next layer, and the aforementioned escape, word segmentation and replacement steps are repeated to perform data mapping and replacement until the set value of the mapping type is reached;

[0012] The data results after multiple data replacement are output as plain text, and the compressed data can be used.

[0013] Furthermore, the data encoding format of the plain text data includes UTF-8, ASCII, ANSI, GBK, GB2312, UTF-8, GB18030 or UNICODE.

[0014] Generally, the word segmentation algorithm used to segment the escaped original data includes word segmentation through knowledge understanding, word segmentation through dictionary and word segmentation based on statistics.

[0015] A second object of the present invention is to provide a method for restoring plain text data based on multiple mappings, for restoring the data results obtained by the aforementioned compression method, the method comprising:

[0016] Read the compressed data and set the data encoding format. The data encoding format must be consistent with the compression method. Read the data mapping table to obtain the "mapping type".

[0017] Perform reverse loop analysis based on "mapping type". First, loop from largest to smallest according to "mapping type". Replace "mapping value" with "word" according to the data mapping table. This is equivalent to the reverse operation of data compression.

[0018] When "Mapping Type" is "1", first replace the "Mapping Value" and "Word". After the replacement is completed, replace the "Special Character Replacement Value" with the special characters in the original data.

[0019] Restore to original data and output.

[0020] A third object of the present invention is to provide a computer system comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the compression method and / or restoration method as described above when executing the computer program.

[0021] A fourth object of the present invention is to provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the compression method and / or restoration method as described above.

[0022] The beneficial effects of the present invention are:

[0023] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description and the preceding claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0025] Figure 1 This is a general flow chart of the compression and restoration method of the present invention. DETAILED DESCRIPTION

[0026] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments are only for illustrating the present invention, and are not intended to limit the scope of protection of the present invention.

[0027] Figure 1It is a general flow chart including a compression method and a restoration method. The present invention first provides a plain text data compression method based on multiple mappings. The method first sets a data mapping table. The data mapping table can be set to N levels as needed, where N is greater than 1 and is a positive integer. The elements contained in the first level data mapping table include "sequence number", "mapping value", "word", "special character replacement value" and "mapping type". The field types of the "mapping value", "word" and "special character replacement value" are all character types, wherein the "mapping value" represents the data mapping replacement value, the "word" is used to search for characters existing in the plain text, and the "special character replacement value" is used to replace the special character that is the same as the mapping value with the special character replacement value when it appears; the field type of the "mapping type" is a numeric type, which is used to identify the number of mapping values. If the setting value is greater than 1, multiple data mappings are required. If the setting value is N, N data mappings are required; the elements contained in the second to Nth level data mapping tables include "sequence number", "mapping value", "word" and "mapping type"; as shown in the following table:

[0028] Table 1 Description of establishing data mapping table

[0029]

[0030] The specific steps of this multi-mapping based plain text data compression method are as follows: Figure 1 As shown:

[0031] Step S1: using the middleware to receive the original data that needs to be compressed, the original data is in plain text form, and then setting the data encoding format of the plain text data;

[0032] Step S2: Escape special characters. First, a special character set is automatically constructed using the "mapping value" of a data mapping table. Special character replacement values ​​are automatically constructed using the "special character replacement value" of the data mapping table. If special characters exist in the original data, the special characters in the original data are replaced with the "special character replacement value" according to the replacement value.

[0033] Step S3: Segment the escaped original data using a word segmentation algorithm, then store the segmented data and calculate the number of times the word appears;

[0034] Step S4: replace the escaped data, cyclically replace the data list according to the number of word occurrences from high to low, and record the plain text information after the escaped data is replaced.

[0035] Step S5: Multiple data replacement, i.e., performing multiple data conversions according to the "mapping type" of the data mapping table, using the result of the previous layer as the "data input" of the next layer, and repeating the aforementioned escape, word segmentation, and replacement steps to perform data mapping and replacement until the set value of the mapping type is reached;

[0036] Step S6: Output the data results after multiple data replacement as plain text. At this time, the compressed data can be used.

[0037] In order to facilitate the restoration of the data results obtained by the aforementioned compression method, the present invention also provides a method for restoring plain text data based on multiple mappings, which includes:

[0038] Step S7: Read the compressed data and set the data encoding format of the data. The data encoding format must be consistent with the compression method. Read the data mapping table to obtain the "mapping type";

[0039] Step S8: Perform reverse loop analysis based on the "mapping type". First, loop from the largest to the smallest "mapping type". Replace the "mapping value" with the "word" according to the data mapping table. This is equivalent to the reverse operation of data compression.

[0040] Step S9: When the "mapping type" is "1", the "mapping value" and "word" are replaced first. After the replacement is completed, the "special character replacement value" is replaced with the special characters in the original data;

[0041] Step S10: Restore to original data and output.

[0042] The above compression method and restoration method will be further described below through a specific embodiment:

[0043] Example 1

[0044] Taking the term "computer aspect" as an example, two data mapping tables are constructed on the data compression side and the data restoration side as follows:

[0045] Table 2 First-level data mapping table

[0046]

[0047]

[0048] Table 3 Dual data mapping table

[0049] Serial number Mapping Values Words Mapping Type 1 A A1 2 2 B A2 2 3 C A3 2 4 D A4 2 5 E A5 2 6 F A6 2 7 G A7 2 8 H A8 2 9 I A9 2 10 J A10 2 11 K A11 2 12 L A12 2 13 M A13 2 14 N A14 2 15 O A15 2 16 ... ... 17 ... ... 18 ... ...

[0050] The specific process of data compression using the data compression method is as follows:

[0051] (1) Original data

[0052] a) The preferred middleware receives the original data that needs to be compressed, and the original data is in plain text form.

[0053] b) Set the data encoding format of the plain text data. In this embodiment, UTF-8 is used.

[0054] (2) Special character escape

[0055] a) Define special character sets and automatically construct special character sets through the "mapping value" of a data mapping table;

[0056] b) Set the replacement value and automatically construct the special character replacement value through the "special character replacement value" in the data mapping table;

[0057] c) Replace special character information: replace the special characters in the original data with the "special character replacement value" according to the replacement value in "b)".

[0058] (3) Escaped data segmentation

[0059] a) Segmenting the escaped original data using a word segmentation algorithm (implemented by Pangu word segmentation in this embodiment);

[0060] b) Word counting: store the segmented data and calculate the number of times the word appears.

[0061] (4) Escape data replacement

[0062] a) Replace the data list cyclically according to the number of word occurrences from high to low;

[0063] b) Record the plain text information after the escaped data is replaced.

[0064] (5) Multiple data replacement

[0065] a) Based on step "4-b)", perform multiple data conversions according to the "mapping type" of the data mapping table, use the result of the previous layer as the "data input" of the next layer, and repeat the above steps "3, 4, 5" to perform data mapping and replacement.

[0066] (6)Data output.

[0067] a) Output the data results after multiple data replacement as plain text.

[0068] The compressed data can now be used for various purposes, such as data transmission, QR code printing, or RFID tag information writing.

[0069] The specific process of data restoration using the data restoration method is as follows:

[0070] (1) Data reading

[0071] a) Read compressed data.

[0072] b) Set the data encoding format (must be consistent with the compression format).

[0073] c) Read the data mapping table and obtain the "mapping type".

[0074] (2) Data analysis

[0075] a) The results of steps (1)-c) are analyzed in reverse order according to the mapping type. First, the mapping type is looped from large to small, and the mapping value is replaced with the word according to the data mapping table (equivalent to the reverse operation during data compression).

[0076] b) When the "mapping type" is "1", first replace the "mapping value" and "word" according to the steps of "(2)-a)". After the replacement is completed, replace the escaped word.

[0077] (3) Data restoration

[0078] Restore data output.

[0079] Example 2

[0080] The present invention is called between different application systems, different operating platforms, and different terminals through middleware. Users can call according to the calling method. The following is a specific embodiment of the calling method, including the following steps:

[0081] 1. Set the character encoding "CHARSET" and customize it according to user needs (such as: UTF-8).

[0082] 2. Set up the data mapping table. The establishment of the data mapping table is shown in Table 1;

[0083] 3. Get the data "mapping type" value (as the number of loop data mapping and parsing);

[0084] 4. Data compression function call, the function definition is "string strCompress DataCompress(stringData, int cs, DataTabledtYsb)".

[0085] Parameter Description:

[0086] strCompress: compressed return value;

[0087] Data: plain text data to be compressed;

[0088] Cs: the number of cycles of the mapping table;

[0089] dtYsb: data mapping table;

[0090] 5. Data decompression function call, the function definition is "string strDeCompress DataDeCompress(string deData, int cs, DataTabledtYsb)".

[0091] strDeCompress: return value after decompression;

[0092] Data: plain text data to be decompressed;

[0093] Cs: the number of cycles of the mapping table;

[0094] dtYsb: data mapping table;

[0095] 6. The data compression party calls the compression function "DataCompress", and the data decompression party calls the decompression function "DataDeCompress".

[0096] The present invention provides a solution for compressing, identifying, reading, and restoring large amounts of plain text information. This solution is implemented by adopting a middleware method, is simple to call, easy to use, and highly compatible, and is applicable to applications developed for Windows, Linux, Android, IOS, and Hongmeng operating systems. First, a user-defined multiple data mapping table is used to segment, extract, compare, replace, and restore plain text information. The data mapping table is open to the user and can be customized by the user. The user can build a data mapping table according to the industry to which they belong. In theory, the more detailed the data mapping table is established by the user, the more multiple data mapping tables there are, and the higher the information compression efficiency is. The information compression rate is completely under the control of the user. To restore data losslessly, it is necessary to know all the data mapping tables used in the corresponding data compression. The data compression process is actually also an encryption process, which greatly guarantees the security of the data. It can be used in fields such as the Internet of Things, warehousing and logistics, material management, and military industry, which have high security and privacy requirements.

[0097] It should be appreciated that embodiments of the present invention can be implemented or practiced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The methods can be implemented in a computer program using standard programming techniques, including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner, according to the methods and figures described in the specific embodiments. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, the program can be run on a programmed application-specific integrated circuit for this purpose.

[0098] Furthermore, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by the context. The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. The computer program includes a plurality of instructions that can be executed by one or more processors.

[0099] Further, the methods can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, RAM, ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein. In addition, the machine-readable code, or portions thereof, can be transmitted over a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A plain text data compression method based on multiple mappings, characterized in that: The method comprises the following steps Set a data mapping table, which is set to N layers as needed, where N is a positive integer greater than 1, and includes a first to an Nth data mapping table. The first data mapping table includes elements such as "sequence number", "mapping value", "word", "special character replacement value" and "mapping type". The field types of "mapping value", "word" and "special character replacement value" are all character types, where "mapping value" represents the data mapping replacement value, the "word" is used to search for characters existing in the plain text, and the "special character replacement value" is used to replace special characters that are the same as the mapping value with the special character replacement value when they appear; the field type of "mapping type" is a numeric type, which is used to identify the number of mapping values. If the setting value is greater than 1, multiple data mappings are required. If the setting value is N, N data mappings are required; the second to Nth data mapping tables include elements such as "sequence number", "mapping value", "word" and "mapping type"; Use the middleware to receive the original data that needs to be compressed. The original data is in plain text format, and then set the data encoding format of the plain text data. To escape special characters, first automatically construct a special character set using the "Mapping Value" in the first data mapping table, and then automatically construct a special character replacement value using the "Special Character Replacement Value" in the data mapping table. If special characters exist in the original data, the special characters in the original data are replaced with the "Special Character Replacement Value" according to the replacement value. The escaped original data is segmented by a word segmentation algorithm, and then the segmented data is stored and the number of word occurrences is calculated; Escape data replacement: replace the data list cyclically according to the number of word occurrences from high to low, and record the plain text information after the escape data replacement; Multiple data replacement, that is, multiple data conversions are performed according to the "mapping type" of the data mapping table, the result of the previous layer is used as the "data input" of the next layer, and the aforementioned escape, word segmentation and replacement steps are repeated to perform data mapping and replacement until the set value of the mapping type is reached; The data results after multiple data replacement are output as plain text, and the compressed data can be used.

2. The method for compressing plain text data based on multiple mappings according to claim 1, wherein: The data encoding format of the plain text data includes UTF-8, ASCII, ANSI, GBK, GB2312 or GB18030.

3. The method for compressing plain text data based on multiple mappings according to claim 1 or 2, characterized in that: The word segmentation algorithm used to segment the escaped original data includes word segmentation through knowledge understanding, word segmentation through dictionary and word segmentation based on statistics.

4. A method for restoring plain text data based on multiple mappings, for restoring the data result obtained by the compression method according to any one of claims 1 to 3, characterized in that: The reduction method comprises: Read compressed data and set the data encoding format. The data encoding format must be consistent with the compression method. Read the data mapping table to obtain the "mapping type". Perform reverse loop parsing based on "Mapping Type". First, loop from largest to smallest "Mapping Type". Replace "Mapping Value" with "Word" according to the data mapping table. This is equivalent to the reverse operation of data compression. When "Mapping Type" is "1", the "Mapping Value" and "Word" are replaced first. After the replacement is completed, the "Special Character Replacement Value" is replaced with the special characters in the original data. Restore to original data and output.

5. A computer system comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the computer program, the compression and / or restoration method according to any one of claims 1 to 4 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the compression and / or restoration method according to any one of claims 1 to 4 is implemented.

Images