Pure text data compression and restoration method based on multiple mapping and computer system

Abstract

The invention discloses a plain text data compression and restoration method based on multiple mapping and a computer system, and aims to provide a solution integrating data compression, recognition, reading and restoration for plain text information with a large data volume. Pure text information is subjected to word segmentation, extraction, comparison, replacement and restoration, a data mapping table is opened to a user and can be customized by the user, the user can construct the data mapping table according to the belonging industry field, the data compression process is also an encryption process actually, the data security is greatly guaranteed, the method is implemented in a middleware mode, and the data compression efficiency is greatly improved. The method is simple to call, convenient to use and high in compatibility, and is suitable for application programs researched and developed by Windows, Linux, Android, IOS and Kwan operating systems.

Description

Technical Field

[0001] The present invention relates to the technical field of data compression and restoration, and in particular to a method and a computer device for compressing and restoring plain text data based on multiple mappings, which can be used in application scenarios such as plain text transmission, two-dimensional code identification printing when the data volume is large, and RFID electronic tag data storage. Background Art

[0002] With the development of society, the progress of the times, the rise of mobile Internet and the Internet of Things, and the increasing requirements for real-time information transmission and large data volume transmission, the information transmitted and stored is limited when the communication bandwidth is small (such as Beidou short message), the size of the QR code label is fixed, and the capacity of the RFID electronic tag is limited. In many scenarios, it cannot meet user needs. To meet this demand, an algorithm has been developed to segment, extract, compare, replace, and restore plain text data through a user-defined data mapping table to solve problems in the fields of large data plain text transmission, large data volume QR code label printing, and RFID electronic tag data storage. Summary of the invention

[0003] In view of this, one of the objects of the present invention is to provide a plain text data compression method based on multiple mapping, which can solve the problems raised in the background technology; the second object of the present invention is to provide a plain text data restoration method based on multiple mapping on the basis of the first object.

[0004] One of the purposes of the present invention is achieved through the following technical solution:.

[0005] A plain text data compression method based on multiple mappings, characterized in that the method comprises the following steps:

[0006] A data mapping table is set, and the data mapping table can be set to N layers as needed, where N is greater than 1 and is a positive integer. The elements contained in the first layer of the data mapping table include "sequence number", "mapping value", "word", "special character replacement value" and "mapping type". The field types of the "mapping value", "word" and "special character replacement value" are all character types, wherein the "mapping value" represents the data mapping replacement value, the "word" is used to find the characters existing in the plain text, and the "special character replacement value" is used to replace the special character that is the same as the mapping value with the special character replacement value when it appears; the field type of the "mapping type" is a numeric type, which is used to identify the multiplicity of the mapping value. If the setting value is greater than 1, multiple data mappings are required, and if the setting value is N, N data mappings are required; the elements contained in the second to Nth layer of the data mapping table include "sequence number", "mapping value", "word" and "mapping type";

[0007] The middleware is used to receive the original data that needs to be compressed, the original data is in plain text form, and then the data encoding format of the plain text data is set;

[0008] To escape special characters, firstly, a special character set is automatically constructed through the "mapping value" of a data mapping table, and then a special character replacement value is automatically constructed through the "special character replacement value" of the data mapping table. If there are special characters in the original data, the special characters in the original data are replaced with the "special character replacement value" according to the replacement value.

[0009] The escaped original data is segmented by a word segmentation algorithm, and then the segmented data is stored and the number of word occurrences is calculated;

[0010] Escape data replacement, cyclically replace the data list according to the number of word occurrences from high to low, and record the plain text information after the escape data replacement.

[0011] Multiple data replacement, that is, multiple data conversion is performed according to the "mapping type" of the data mapping table, the result of the previous layer is used as the "data input" of the next layer, and the aforementioned escape, word segmentation and replacement steps are repeated to perform data mapping and replacement until the set value of the mapping type is reached;

[0012] The data results after multiple data replacement are output as plain text, and the compressed data can be used at this time.

[0013] Furthermore, the data encoding format of the plain text data includes UTF-8, ASCII, ANSI, GBK, GB2312, UTF-8, GB18030 or UNICODE.

[0014] Generally, the word segmentation algorithm used to segment the escaped original data includes word segmentation through knowledge understanding, word segmentation through dictionary and word segmentation based on statistics.

[0015] The second object of the present invention is to provide a method for restoring plain text data based on multiple mappings, which is used to restore the data results obtained by the above-mentioned compression method, and the restoration method comprises:

[0016] Read the compressed data and set the data encoding format, which must be consistent with the compression method, read the data mapping table, and obtain the "mapping type";

[0017] Perform reverse loop analysis according to "mapping type". First, loop from large to small according to "mapping type". Replace "mapping value" with "word" according to the data mapping table, which is equivalent to the reverse operation of data compression.

[0018] When "Mapping Type" is "1", first replace "Mapping Value" and "Word". After the replacement is completed, replace "Special Character Replacement Value" with the special characters in the original data.

[0019] Restore to original data and output.

[0020] The third object of the present invention is to provide a computer system, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the compression method and / or restoration method as described above when executing the computer program.

[0021] A fourth object of the present invention is to provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the compression method and / or restoration method as described above.

[0022] The beneficial effects of the present invention are:

[0023] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description and the preceding claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0025] Figure 1 The figure is a general flow chart of the compression and restoration method of the present invention. DETAILED DESCRIPTION

[0026] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments are only for illustrating the present invention, rather than for limiting the protection scope of the present invention.

[0027] Figure 1The invention is a general flow chart including a compression method and a restoration method. The invention first provides a plain text data compression method based on multiple mapping. The method first sets a data mapping table. The data mapping table can be set to N layers as needed, where N is greater than 1 and is a positive integer. The elements contained in the first layer of the data mapping table include "sequence number", "mapping value", "word", "special character replacement value" and "mapping type". The field types of the "mapping value", "word" and "special character replacement value" are all character types, wherein the "mapping value" represents the data mapping replacement value, and the "word" is used to find the characters existing in the plain text. The "special character replacement value" is used to replace the special character that is the same as the mapping value with the special character replacement value when it appears; the field type of the "mapping type" is a numeric type, which is used to identify the multiplicity of the mapping value. If the setting value is greater than 1, multiple data mappings need to be performed. If the setting value is N, N data mappings need to be performed; the elements contained in the second to Nth layer of the data mapping table include "sequence number", "mapping value", "word" and "mapping type"; as shown in the following table:

[0028] Table 1 Description of establishing data mapping table

[0029]

[0030] The specific steps of this method of plain text data compression based on multi-mapping are as follows: Figure 1 As shown:

[0031] Step S1: using the middleware to receive the original data that needs to be compressed, the original data is in plain text form, and then setting the data encoding format of the plain text data;

[0032] Step S2: perform special character escape, first automatically construct a special character set through the "mapping value" of a data mapping table, and automatically construct a special character replacement value through the "special character replacement value" of the data mapping table. If there are special characters in the original data, the special characters in the original data are replaced with the "special character replacement value" according to the replacement value;

[0033] Step S3: segmenting the escaped original data using a segmentation algorithm, then storing the segmented data and calculating the number of occurrences of the words;

[0034] Step S4: replace the escaped data, cyclically replace the data list according to the number of word occurrences from high to low, and record the plain text information after the escaped data is replaced.

[0035] Step S5: multiple data replacement, i.e., multiple data conversion is performed according to the "mapping type" of the data mapping table, the result of the previous layer is used as the "data input" of the next layer, and the aforementioned escape, word segmentation and replacement steps are repeated to perform data mapping and replacement until the set value of the mapping type is reached;

[0036] Step S6: Output the data results after multiple data replacement as plain text, and the compressed data can be used at this time.

[0037] In order to facilitate the restoration of the data results obtained by the aforementioned compression method, the present invention also provides a method for restoring plain text data based on multiple mappings, the method comprising:

[0038] Step S7: read the compressed data and set the data encoding format of the data, which must be consistent with the compression method, read the data mapping table, and obtain the "mapping type";

[0039] Step S8: Perform reverse loop analysis according to the "mapping type", first loop from large to small according to the "mapping type", and replace the "mapping value" with the "word" according to the data mapping table, which is equivalent to the reverse operation during data compression;

[0040] Step S9: When the "mapping type" is "1", the "mapping value" and "word" are replaced first. After the replacement is completed, the "special character replacement value" is replaced with the special characters in the original data;

[0041] Step S10: Restore to original data and output.

[0042] The above compression method and restoration method will be further described below through a specific embodiment:

[0043] Embodiment 1

[0044] Taking the word "computer aspect" as an example, two data mapping tables are constructed on the data compression side and the data restoration side as follows:

[0045] Table 2 First-order data mapping table

[0046]

[0047]

[0048] Table 3 Dual data mapping table

[0049] Serial number Mapping Values Words Mapping Type 1 A A1 2 2 B A2 2 3 C A3 2 4 D A4 2 5 E A5 2 6 F A6 2 7 G A7 2 8 H A8 2 9 I A9 2 10 J A10 2 11 K A11 2 12 L A12 2 13 M A13 2 14 N A14 2 15 O A15 2 16 ... ... 17 ... ... 18 ... ...

[0050] The specific process of data compression using the data compression method is as follows:

[0051] (1) Raw data

[0052] a) The preferred middleware receives the original data that needs to be compressed, and the original data is in plain text.

[0053] b) Set the data encoding format of the plain text data, UTF-8 is used in this embodiment.

[0054] (2) Special character escape

[0055] a) Define special character sets and automatically construct special character sets through the "mapping value" of a data mapping table;

[0056] b) Set the replacement value and automatically construct the special character replacement value through the "special character replacement value" of the data mapping table;

[0057] c) Replace special character information: replace the special characters in the original data with the "special character replacement value" according to the replacement value in "b)".

[0058] (3) Escaped data segmentation

[0059] a) Segmenting the escaped original data through a word segmentation algorithm (implemented by Pangu word segmentation in this embodiment);

[0060] b) Word counting: store the segmented data and calculate the number of times the word appears.

[0061] (4) Escape data replacement

[0062] a) Cycle through the data list according to the number of occurrences of the words from high to low;

[0063] b) Record the plain text information after the escaped data is replaced.

[0064] (5) Multiple data replacement

[0065] a) Based on step "4-b)", perform multiple data conversions according to the "mapping type" of the data mapping table, use the result of the previous layer as the "data input" of the next layer, and repeat the above steps "3, 4, 5" to map and replace the data.

[0066] (6)Data output.

[0067] a) Output the data results after multiple data replacement as plain text.

[0068] At this time, the compressed data can be used, such as data transmission, QR code printing, or RFID electronic tag information writing.

[0069] The specific process of data restoration using the data restoration method is as follows:

[0070] (1) Data reading

[0071] a) Read compressed data.

[0072] b) Set the data encoding format (must be consistent with compression).

[0073] c) Read the data mapping table and obtain the "mapping type".

[0074] (2) Data analysis

[0075] a) The results of steps "(1)-c)" are analyzed in reverse order according to the "mapping type". First, the "mapping type" is looped from large to small, and the "mapping value" is replaced with the "word" according to the data mapping table (equivalent to the reverse operation during data compression).

[0076] b) When the "mapping type" is "1", first replace the "mapping value" and "word" according to the steps of "(2)-a)". After the replacement is completed, replace the escaped word.

[0077] (3) Data restoration

[0078] Restore data output.

[0079] Embodiment 2

[0080] The present invention is called between different application systems, different operating platforms, and different terminals through middleware, and the user can call according to the calling method. The following is a specific embodiment of the calling method, including the following steps:

[0081] 1. Set the character encoding "CHARSET" and customize it according to user needs (such as: UTF-8).

[0082] 2. Set the data mapping table. The establishment of the data mapping table is shown in Table 1;

[0083] 3. Get the data "mapping type" value (as the number of loop data mapping and parsing);

[0084] 4. Data compression function call, the function definition is "string strCompress DataCompress(stringData, int cs, DataTabledtYsb)".

[0085] Parameter Description:

[0086] strCompress: compressed return value;

[0087] Data: Plain text data to be compressed;

[0088] Cs: the number of cycles of the mapping table;

[0089] dtYsb: data mapping table;

[0090] 5. Data decompression function call, the function definition is "string strDeCompress DataDeCompress(string deData, int cs, DataTabledtYsb)".

[0091] strDeCompress: return value after decompression;

[0092] Data: Plain text data to be decompressed;

[0093] Cs: the number of cycles of the mapping table;

[0094] dtYsb: data mapping table;

[0095] 6. The data compression party calls the compression function "DataCompress", and the data decompression party calls the decompression function "DataDeCompress".

[0096] The present invention provides a solution for data compression, identification, reading and restoration of large amounts of plain text information. It is implemented by adopting middleware, is easy to call, easy to use, and has strong compatibility. It is suitable for applications developed for Windows, Linux, Android, IOS, and Hongmeng operating systems. First, a user-defined multiple data mapping table is used to segment, extract, compare, replace, and restore plain text information. The data mapping table is open to users and can be customized by users. Users can build data mapping tables according to their industry fields. In theory, the more refined the data mapping table is established by the user, the more data mapping tables are multiplied, the higher the information compression efficiency is, and the information compression rate is completely in hand; to restore data losslessly, it is necessary to know all the data mapping tables used in the corresponding data compression. The data compression process is actually an encryption process, which greatly guarantees the security of the data. It can be used in the fields of Internet of Things, warehousing and logistics, material management, military industry, etc. with high security and privacy requirements.

[0097] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in an assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed ASIC for this purpose.

[0098] Furthermore, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. The computer program includes a plurality of instructions that may be executed by one or more processors.

[0099] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, a RAM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or portions thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A plain text data compression method based on multiple mappings, characterized in that: The method comprises the following steps A data mapping table is set, and the data mapping table can be set to N layers as needed, where N is greater than 1 and is a positive integer. The elements contained in the first layer of the data mapping table include "sequence number", "mapping value", "word", "special character replacement value" and "mapping type". The field types of the "mapping value", "word" and "special character replacement value" are all character types, wherein the "mapping value" represents the data mapping replacement value, the "word" is used to find the characters existing in the plain text, and the "special character replacement value" is used to replace the special characters that are the same as the mapping value with the special character replacement value when they appear; the field type of the "mapping type" is a numeric type, which is used to identify the multiplicity of the mapping value. If the setting value is greater than 1, multiple data mappings are required, and if the setting value is N, N data mappings are required; the elements contained in the second to Nth layer of the data mapping table include "sequence number", "mapping value", "word" and "mapping type"; The middleware is used to receive the original data that needs to be compressed, the original data is in plain text form, and then the data encoding format of the plain text data is set; To escape special characters, firstly, a special character set is automatically constructed through the "mapping value" of a data mapping table, and then a special character replacement value is automatically constructed through the "special character replacement value" of the data mapping table. If special characters exist in the original data, the special characters in the original data are replaced with the "special character replacement value" according to the replacement value. The escaped original data is segmented by a word segmentation algorithm, and then the segmented data is stored and the number of word occurrences is calculated; Escape data replacement, cyclically replace the data list according to the number of word occurrences from high to low, and record the plain text information after the escape data replacement. Multiple data replacement, that is, multiple data conversion is performed according to the "mapping type" of the data mapping table, the result of the previous layer is used as the "data input" of the next layer, and the aforementioned escape, word segmentation and replacement steps are repeated to perform data mapping and replacement until the set value of the mapping type is reached; The data results after multiple data replacement are output as plain text, and the compressed data can be used at this time.

2. The method for compressing plain text data based on multiple mappings according to claim 1, characterized in that: The data encoding format of the plain text data includes UTF-8, ASCII, ANSI, GBK, GB2312, UTF-8, GB18030 or UNICODE.

3. The method for compressing plain text data based on multiple mapping according to claim 1 or 2, characterized in that: The word segmentation algorithm used to segment the escaped original data includes word segmentation through knowledge understanding, word segmentation through dictionary and word segmentation based on statistics.

4. A method for restoring plain text data based on multiple mappings, used to restore the data results obtained by the compression method according to any one of claims 1 to 3, characterized in that: The reduction method comprises: Read the compressed data and set the data encoding format, which must be consistent with the compression method. Read the data mapping table to obtain the "mapping type"; Perform reverse loop analysis according to "mapping type". First, loop from large to small according to "mapping type". Replace "mapping value" with "word" according to the data mapping table, which is equivalent to the reverse operation of data compression. When "Mapping Type" is "1", first replace "Mapping Value" and "Word". After the replacement is completed, replace "Special Character Replacement Value" with the special characters in the original data. Restore to original data and output.

5. A computer system comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the computer program, the compression and / or restoration method according to any one of claims 1 to 4 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the compression and / or restoration method according to any one of claims 1 to 4 is implemented.

Images