A data compression storage method and device, a terminal device and a readable storage medium

Abstract

The application is suitable for the technical field of data processing, and provides a data compression storage method and device, terminal equipment and readable storage medium. The method comprises the following steps: preprocessing the data to be stored to obtain a sequence result, determining a plurality of data partitions according to the sequence result, and merging the data partitions to obtain a compression storage result of the data to be stored. According to the application, the data to be stored is preprocessed to reduce the data repetition rate and the amount of redundant data, the sequence result is obtained, the plurality of data partitions are determined based on the sequence result, the corresponding compression storage result is obtained by merging the data partitions, the data is classified and stored in the storage process, and the data storage efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of data processing, and particularly relates to a data compression storage method and device, a terminal device and a readable storage medium. BACKGROUND

[0002] In the process of data storage, if there are multiple data structures pointing to the same object, and the entire object needs to be saved to a file, the object is usually saved by directly writing. However, this method may cause repeated writing of the same content, resulting in data redundancy and large storage files, thereby reducing data storage efficiency. SUMMARY

[0003] The embodiments of the present application provide a data compression storage method, device, terminal device and readable storage medium, which can solve the problems of data redundancy and large storage files caused by repeated writing of data in related data storage methods.

[0004] In a first aspect, the embodiments of the present application provide a data compression storage method, comprising:

[0005] Preprocessing the to-be-stored data to obtain a sequence result;

[0006] Determining a plurality of data partitions according to the sequence result, and merging the data partitions to obtain a compression storage result of the to-be-stored data.

[0007] In one embodiment, the preprocessing the to-be-stored data to obtain a sequence result comprises:

[0008] Scanning all data objects in the to-be-stored data structure to obtain the to-be-stored data;

[0009] De-duplicating the to-be-stored data to obtain a sequence result.

[0010] In one embodiment, the de-duplicating the to-be-stored data to obtain a sequence result comprises:

[0011] Dividing the preprocessed to-be-stored data into a plurality of comparison objects according to the order of data storage;

[0012] De-duplicating the plurality of comparison objects based on a predefined data type to obtain a sequence result; wherein the sequence result comprises a plurality of different instances containing the predefined data type.

[0013] In one embodiment, the data partition comprises a file header, an index area and a data segment.

[0014] The method comprises the following steps:

[0015] According to the sequence result, the first length of the predefined data type and the second length of each basic data type in each instance are determined.

[0016] Based on the first length and the second length, the index area and the data segment are determined.

[0017] Based on the data segment and the index area, the file header is determined.

[0018] The file header, the index area and the data segment are combined to obtain the compressed storage result of the to-be-stored data.

[0019] In one embodiment, the method of determining the index area and the data segment based on the first length and the second length comprises:

[0020] According to the first length and the second length of each instance, the total length of each instance is determined.

[0021] The relative offset of each instance is determined, and the index area is determined based on the total length and the relative offset of each instance.

[0022] According to the total length of each instance and the field information contained in each instance, the data segment is determined; wherein the data segment comprises all instances.

[0023] In one embodiment, the method of determining the data segment according to the total length of each instance and the field information contained in each instance comprises:

[0024] According to the total length of each instance, the total length of the data segment is determined.

[0025] According to the total length of the data segment and the field information contained in each instance, the data segment is determined.

[0026] In one embodiment, the method of combining the file header, the index area and the data segment to obtain the compressed storage result of the to-be-stored data comprises:

[0027] The file header, the index area and the data segment are combined in a preset combination order and written into a file to obtain the compressed storage result of the to-be-stored data.

[0028] In a second aspect, the embodiments of the present application provide a data compression storage device, comprising:

[0029] A preprocessing module is configured to preprocess the to-be-stored data to obtain a sequence result.

[0030] The compression storage module is configured to determine a plurality of data partitions according to the ordered result, and combine the data partitions to obtain a compression storage result of the to-be-stored data.

[0031] In one embodiment, the preprocessing module comprises:

[0032] The scanning sub-module is configured to scan all data objects in the to-be-stored data structure as the to-be-stored data.

[0033] The deduplication processing sub-module is configured to perform deduplication processing on the to-be-stored data to obtain the ordered result.

[0034] In one embodiment, the deduplication processing sub-module comprises:

[0035] The division unit is configured to divide the preprocessed to-be-stored data into a plurality of comparison objects according to the order of data storage.

[0036] The deduplication processing unit is configured to perform deduplication processing on the plurality of comparison objects based on a predefined data type to obtain the ordered result, wherein the ordered result comprises a plurality of different instances containing the predefined data type.

[0037] In one embodiment, the data partition comprises a file header, an index area, and a data segment.

[0038] The compression storage module comprises:

[0039] The first determination sub-module is configured to determine, according to the ordered result, a first length of the predefined data type in each instance and a second length of each basic data type.

[0040] The second determination sub-module is configured to determine the index area and the data segment based on the first length and the second length.

[0041] The third determination sub-module is configured to determine the file header based on the data segment and the index area.

[0042] The combining sub-module is configured to combine the file header, the index area, and the data segment to obtain the compression storage result of the to-be-stored data.

[0043] In one embodiment, the second determination sub-module comprises:

[0044] The first determination unit is configured to determine, according to the first length and the second length of each instance, a total length of each instance.

[0045] The second determination unit is configured to determine a relative offset of each instance, and determine the index area based on the total length and the relative offset of each instance.

[0046] The third determining unit is configured to determine the data segment according to the total length of each instance and the field information contained in each instance, wherein the data segment comprises all instances.

[0047] In one embodiment, the third determining unit comprises:

[0048] The first determining sub-unit is configured to determine the total length of the data segment according to the total length of each instance.

[0049] The second determining sub-unit is configured to determine the data segment according to the total length of the data segment and the field information contained in each instance.

[0050] In one embodiment, the merging sub-module is specifically configured to merge the file header, the index area and the data segment in a preset combination order, and write into a file to obtain the compressed storage result of the to-be-stored data.

[0051] In a third aspect, the embodiments of the present application provide a terminal device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the data compression storage method according to any one of the first aspect.

[0052] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the data compression storage method according to any one of the first aspect.

[0053] In a fifth aspect, the embodiments of the present application provide a computer program product, which, when executed on a terminal device, causes the terminal device to execute the data compression storage method according to any one of the first aspect.

[0054] Compared with the prior art, the embodiments of the present application have the beneficial effects that: the data repetition rate is reduced by preprocessing the to-be-stored data, the amount of redundant data is reduced, the ordered result is obtained, the multiple data partitions are determined according to the ordered result, the data partitions are merged to obtain the compressed storage result of the to-be-stored data, and the data is classified and stored in the storage process, thereby improving the data storage efficiency.

[0055] It can be understood that the beneficial effects of the above-mentioned second aspect to fifth aspect can be referred to the related description in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative effort.

[0057] Figure 1 is a flowchart of a data compression storage method provided by an embodiment of the present application;

[0058] Figure 2 is a schematic diagram of a file header provided by an embodiment of the present application;

[0059] Figure 3 is a schematic diagram of an index area provided by an embodiment of the present application;

[0060] Figure 4 is a schematic diagram of a data segment provided by an embodiment of the present application;

[0061] Figure 5 is a schematic diagram of an instance provided by an embodiment of the present application;

[0062] Figure 6 is a structural schematic diagram of a data compression storage device provided by an embodiment of the present application;

[0063] Figure 7 is a structural schematic diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0064] In the following description, specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, persons skilled in the art will understand that the present application can be practiced without these specific details. In other instances, well-known systems, methods, circuits, and devices are not shown or described in detail in order to avoid obscuring the description of the present application.

[0065] It should be understood that, when used in the present application specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or sets thereof.

[0066] It should also be understood that the term "and / or" used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0067] As used in the specification and the appended claims, the term "if' can be interpreted as meaning "when" or "upon" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [the described condition or event] is detected" can be interpreted as meaning "upon determining" or "in response to determining" or "upon detecting [the described condition or event]" or "in response to detecting [the described condition or event]" depending on the context.

[0068] In addition, in the description of the present application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0069] In the present application, the reference "one embodiment" or "some embodiments" and the like means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in further some embodiments" and the like appearing in the present specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0070] The data compression storage method provided by the embodiments of the present application can be applied to terminal devices such as mobile phones, tablet computers, notebook computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA) and the like. The embodiments of the present application do not make any limitation on the specific type of terminal device.

[0071] Embodiment one

[0072] Figure 1 The schematic flowchart of the data compression storage method provided by the present application is shown, which is used as an example but not limited, and the method can be applied to the notebook computer described above.

[0073] S101, pre-processing the data to be stored to obtain a sequence result.

[0074] Specifically, for any plurality of data structures containing a plurality of fields of the same data type, when needing to be stored into the same storage file, each data structure can be instantiated and saved in a recursive manner. Before saving, the to-be-stored data needs to be preprocessed to reduce the repetition rate of the to-be-stored data. The preprocessing manner includes but is not limited to any one or more of scanning, splitting, and deduplication. The ordered sequence result includes a plurality of different instances containing a predefined data type, and the instance is the smallest data field containing the predefined data type. The predefined data type refers to a specific data type that is set by a user according to actual needs and is different from a basic data type, which is used to distinguish a plurality of different data fields into a plurality of different instances, so as to facilitate segmented storage of the to-be-stored data.

[0075] It can be understood that each instance is composed of a predefined data type and a basic data type. For example, the predefined data type is T0, and the basic data type includes T1, T2, and T3; according to the order of data storage in the data structure, the predefined data type processes the to-be-stored data to obtain an ordered sequence result including a first instance (composed of T0 and T2), a second instance (composed of T0 and T1), and a third instance (composed of T0 and T3).

[0076] S102, determining a plurality of data partitions according to the ordered sequence result, and merging the data partitions to obtain a compressed storage result of the to-be-stored data.

[0077] Specifically, according to the ordered sequence result, the specific field information of the plurality of preset data partitions is determined, and all the data partitions are merged and written into the storage file to obtain the compressed storage result of the to-be-stored data.

[0078] In this embodiment, the data partition includes but is not limited to an index area, a data segment, and a file header.

[0079] The file header is used to store data format information, such as a mark bit (indicating that the file belongs to what type of file or stores what content), a version number (used to distinguish which way to parse the format of the file, or used to represent the way of upgrading or downgrading the file), the memory size of the file, and the starting offset and field length of the index area, the data segment, and other data partitions (such as Figure 2 , an exemplary schematic diagram of a file header is provided). The index area is used to store the total length of each instance and the relative offset of each instance (such as Figure 3 , an exemplary schematic diagram of an index area is provided). The data segment is used to store the total length of all instances and the field information of each instance (such as Figure 4 , an exemplary schematic diagram of a data segment is provided).

[0080] In one embodiment, the pre-processing of the data to be stored comprises:

[0081] Scanning all data objects in the data structure to be stored as the data to be stored;

[0082] De-duplication processing of the data to be stored to obtain a sequence result.

[0083] Specifically, scanning the data structure requiring data storage to determine all data objects in the data to be stored as the data to be stored; de-duplication processing of the data to be stored to filter out duplicate data, reduce the duplication rate of the data to be stored, and obtain a sequence result.

[0084] In one embodiment, the de-duplication processing of the data to be stored to obtain a sequence result comprises:

[0085] Dividing the pre-processed data to be stored into multiple comparison objects according to the order of data storage;

[0086] De-duplication processing of the multiple comparison objects based on a predefined data type to obtain a sequence result; wherein the sequence result comprises multiple different instances containing the predefined data type.

[0087] Specifically, the size of the comparison object is determined, the pre-processed data to be stored is divided into multiple comparison objects according to the order of data storage in the data structure to be stored, and de-duplication processing of the multiple comparison objects based on a predefined data type is performed to obtain a sequence result. The size of the comparison object can be specifically set according to user requirements. For example, the size of the comparison object can be set as a field or a bit.

[0088] For example, the size of the comparison object is set as a field, at this time the pre-processed data to be stored is divided into multiple fields according to the order of data storage in the data structure to be stored, and de-duplication processing is performed by comparing each field with a predefined data type to obtain multiple different instances containing the predefined data type (i.e. a sequence result).

[0089] For example, the predefined data type is TT, at this time the multiple comparison objects divided by the pre-processed data to be stored according to the fields include: TT0; TT1; TT2; TT3; TT4; assuming that TT2 is equal to TT1 and TT3 and TT4 are equal to TT1, the corresponding sequence result can be: TT0; TT1.

[0090] As an example but not limitation, the field information in each instance includes: first field information of the predefined data type and second field information of the basic data type.

[0091] In one embodiment, the instances further include, but are not limited to, a total length of each instance, a first length of a predefined data type, and a second length of a basic data type.

[0092] In one embodiment, the data partition includes a file header, an index area, and a data segment.

[0093] The data partition is determined according to the ordered result, and the data partition is combined to obtain a compressed storage result of the to-be-stored data.

[0094] The first length of the predefined data type and the second length of each basic data type in each instance are determined according to the ordered result.

[0095] The index area and the data segment are determined based on the first length and the second length.

[0096] The file header is determined based on the data segment and the index area.

[0097] The file header, the index area, and the data segment are combined to obtain a compressed storage result of the to-be-stored data.

[0098] Specifically, the first length of the predefined data type and the second length of the basic data type contained in each instance are determined according to the ordered result, the total length of each instance is determined based on the first length and the second length of each instance, and the field information contained in the index area and the data segment is determined based on the total length of each instance and the field information of each instance. The first total length and the first starting offset of the index area are determined based on the field information contained in the index area, the second total length and the second starting offset of the data segment are determined according to the field information contained in the data segment, and the total length of the to-be-stored data is determined, so as to determine the file header. The file header, the index area, and the data segment are combined and written into a storage file to obtain a compressed storage result of the to-be-stored data.

[0099] In one embodiment, the index area and the data segment are determined based on the first length and the second length, including:

[0100] The total length of each instance is determined according to the first length and the second length of each instance.

[0101] The relative offset of each instance is determined, and the index area is determined based on the total length and the relative offset of each instance.

[0102] The data segment is determined according to the total length of each instance and the field information contained in each instance, and the data segment includes all instances.

[0103] Specifically, the total length of each instance is calculated according to the first length of the predefined data type and the second length of the basic data type contained in each instance, the relative offset of each instance is determined, the field information of the index area is determined according to the total length of each instance and the relative offset of each instance, the field information of the data segment is determined according to the total length of each instance and the field information contained in each instance, and the field information of the predefined data type in the data segment is re-corrected by traversing all fields in the data segment according to the field information of the index area.

[0104] In one embodiment, the determining the data segment according to the total length of each instance and the field information contained in each instance comprises:

[0105] determining the total length of the data segment according to the total length of each instance;

[0106] determining the data segment according to the total length of the data segment and the field information contained in each instance.

[0107] Specifically, the total length of the data segment is obtained by summing up the total length of each instance, and the data segment is obtained according to the field information contained in each instance (i.e. the first field information of the predefined data type and the second field information of the basic data type contained in each instance) and the arrangement order of each instance in the ordered result.

[0108] It can be understood that each instance can contain multiple predefined data types and multiple basic data types. Each basic data field includes the data type ID and data information of the field. Each predefined data type includes the relative offset of the field and the information whether to reference the object (bool).

[0109] Figure 5 An exemplary schematic diagram of an instance is provided.

[0110] According to Figure 5 It can be seen that the instance contains the total length of the instance, the total length of the basic data type, the field information of each basic data type, the total length of the predefined data type, and the field information of each predefined data type.

[0111] In one embodiment, the combining the file header, the index area and the data segment to obtain the compressed storage result of the to-be-stored data comprises:

[0112] The file header, the index area and the data segment are combined in a preset combination order and written into a file to obtain the compressed storage result of the to-be-stored data.

[0113] Specifically, a storage order of each data partition is determined to obtain a preset combination order, the file header, the index area and the data segment are merged according to the preset combination order, and are written into a file to obtain a compression storage result of the to-be-stored data.

[0114] The preset combination order can be specifically set according to actual requirements, and is determined to be the file header, the index area and the data segment stored in the file in sequence based on data contents stored in the file header, the index area and the data segment.

[0115] The embodiment pre-processes the to-be-stored data to reduce a data repetition rate and reduce an amount of redundant data, obtains a sequence result, determines a plurality of data partitions according to the sequence result, merges the data partitions to obtain a compression storage result of the to-be-stored data, and realizes classified storage of data in a storage process to improve data storage efficiency.

[0116] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0117] Embodiment Two

[0118] According to the data compression storage method described in the above embodiment, Figure 6 A structural block diagram of a data compression storage device provided by the embodiment of the application is shown, and only parts related to the embodiment of the application are shown for ease of illustration.

[0119] Referring to Figure 6 The data compression storage device 100 includes:

[0120] The preprocessing module 101 is configured to pre-process the to-be-stored data to obtain a sequence result.

[0121] The compression storage module 102 is configured to determine a plurality of data partitions according to the sequence result, and merge the data partitions to obtain a compression storage result of the to-be-stored data.

[0122] In one embodiment, the preprocessing module includes:

[0123] The scanning sub-module is configured to scan all data objects in a to-be-stored data structure as the to-be-stored data.

[0124] The deduplication processing sub-module is configured to perform deduplication processing on the to-be-stored data to obtain a sequence result.

[0125] In one embodiment, the deduplication processing sub-module includes:

[0126] The division unit is configured to divide the preprocessed data to be stored into a plurality of comparison objects according to the storage order of the data.

[0127] The deduplication processing unit is configured to perform deduplication processing on the plurality of comparison objects based on a predefined data type to obtain an ordered result; wherein the ordered result comprises a plurality of different instances containing the predefined data type.

[0128] In one embodiment, the data partition includes a file header, an index area and a data segment.

[0129] The compressed storage module comprises:

[0130] The first determination sub-module is configured to determine, according to the ordered result, a first length of the predefined data type in each instance and a second length of each basic data type.

[0131] The second determination sub-module is configured to determine, based on the first length and the second length, the index area and the data segment.

[0132] The third determination sub-module is configured to determine, based on the data segment and the index area, the file header.

[0133] The merging sub-module is configured to merge the file header, the index area and the data segment to obtain a compressed storage result of the data to be stored.

[0134] In one embodiment, the second determination sub-module comprises:

[0135] The first determination unit is configured to determine, according to the first length and the second length of each instance, a total length of each instance.

[0136] The second determination unit is configured to determine a relative offset of each instance, and determine the index area based on the total length and the relative offset of each instance.

[0137] The third determination unit is configured to determine the data segment according to the total length of each instance and field information contained in each instance; wherein the data segment comprises all instances.

[0138] In one embodiment, the third determination unit comprises:

[0139] The first determination sub-unit is configured to determine, according to the total length of each instance, a total length of the data segment.

[0140] The second determination sub-unit is configured to determine, according to the total length of the data segment and the field information contained in each instance, the data segment.

[0141] In one embodiment, the merging submodule is specifically used to merge the file header, the index area, and the data segment according to a preset combination order, and write them into a file to obtain the compressed storage result of the data to be stored.

[0142] This embodiment preprocesses the data to be stored to reduce data duplication and redundant data, resulting in a sequential result. Based on the sequential result, multiple data partitions are determined, and the data partitions are merged to obtain a compressed storage result of the data to be stored. This achieves data classification and storage during the storage process, thereby improving data storage efficiency.

[0143] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0144] Example 3

[0145] Figure 7 This is a schematic diagram of the structure of the terminal device provided in this embodiment. Figure 7 As shown, the terminal device 7 of this embodiment includes: at least one processor 70 ( Figure 7 (Only one is shown) a processor, a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70, wherein the processor 70 executes the computer program 72 to implement the steps in any of the above-described data compression and storage method embodiments.

[0146] The terminal device 7 can be a desktop computer, laptop, handheld computer, or cloud server, etc. This terminal device may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that... Figure 7 The example of terminal device 7 is merely an illustration and does not constitute a limitation on terminal device 7. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0147] The processor 70 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0148] The memory 71 can be an internal storage unit of the terminal device 7 in some embodiments, such as a hard disk or a memory of the terminal device 7. The memory 71 can also be an external storage device of the terminal device 7 in other embodiments, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD), a flash card, etc. equipped on the terminal device 7. Further, the memory 71 can include both the internal storage unit and the external storage device of the terminal device 7. The memory 71 is used to store an operating system, an application program, a boot loader, data, and other programs, such as program codes of the computer program, etc. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0149] It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, only the division of the above functional units and modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the above-described functions. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of mutual distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0150] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the steps in the above-mentioned various method embodiments.

[0151] The embodiment of the present application provides a computer program product. When the computer program product is run on a mobile terminal, the mobile terminal is caused to execute the steps in the above-mentioned various method embodiments.

[0152] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application can implement all or part of the processes in the above-mentioned embodiments by a computer program to instruct related hardware to complete, and the computer program can be stored in a computer readable storage medium. The computer program is executed by a processor to implement the steps in the above-mentioned various method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to a photographing device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal and a software distribution medium. For example, a U disk, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunications signal.

[0153] In the above-mentioned embodiments, the description of each embodiment has its own focus. The parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0154] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0155] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other manners. For example, the embodiments of the apparatus / network device described above are merely illustrative. For example, the division of the modules or units is merely logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0156] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0157] The above-described embodiments are merely used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalent replacements; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A data compression storage method characterized by, The method comprises the following steps: preprocessing the to-be-stored data to obtain a sequenced result; determining a first length of a predefined data type and a second length of each basic data type in each instance according to the sequenced result; determining an index area and a data segment based on the first length and the second length; determining a file header based on the data segment and the index area; merging the file header, the index area and the data segment to obtain a compressed storage result of the to-be-stored data.

2. The data compression storage method of claim 1, wherein, The preprocessing the to-be-stored data to obtain a sequenced result comprises the following steps: scanning all data objects in a to-be-stored data structure to obtain the to-be-stored data; performing deduplication processing on the to-be-stored data to obtain a sequenced result.

3. The data compression storage method of claim 2, wherein, The preprocessing the to-be-stored data to obtain a sequenced result comprises the following steps: dividing the preprocessed to-be-stored data into a plurality of comparison objects according to the order of data storage; performing deduplication processing on the plurality of comparison objects based on a predefined data type to obtain a sequenced result; wherein the sequenced result comprises a plurality of different instances containing the predefined data type.

4. The data compression storage method of claim 1, wherein, The determining an index area and a data segment based on the first length and the second length comprises the following steps: determining a total length of each instance according to the first length and the second length of each instance; determining a relative offset of each instance, and determining an index area based on the total length and the relative offset of each instance; determining a data segment according to the total length of each instance and field information contained in each instance; wherein the data segment comprises all instances.

5. The data compression storage method of claim 1, wherein, The determining a data segment according to the total length of each instance and field information contained in each instance comprises the following steps: determining a total length of the data segment according to the total length of each instance; determining the data segment according to the total length of the data segment and the field information contained in each instance.

6. The data compression storage method of claim 1, wherein, The merging the file header, the index area and the data segment to obtain a compressed storage result of the to-be-stored data comprises the following steps: merging the file header, the index area and the data segment according to a preset combination order, and writing them into a file to obtain a compressed storage result of the to-be-stored data.

7. A data compression storage device, characterized by, The method comprises the following steps: a preprocessing module configured to preprocess to-be-stored data to obtain a sequenced result; a compression storage module configured to determine a first length of a predefined data type and a second length of each basic data type in each instance according to the sequenced result, and determine an index area and a data segment based on the first length and the second length; determine a file header based on the data segment and the index area, and merge the file header, the index area and the data segment to obtain a compressed storage result of the to-be-stored data.

8. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the method of any one of claims 1 to 6.

Images