Storage method and device of non-redundant data, storage medium and electronic equipment

Abstract

The invention relates to a non-redundant data storage method and device, a storage medium and electronic equipment. The method comprises the steps that each data object in each data set in a first data set is converted into a corresponding data address, a second data set is obtained, in the first data set, one data object exists in at least one data set, and one data object in one data set corresponds to one data category; the data volume of each data address is obtained, and the data volume is the number of the data addresses in the second data set; forming a third data set according to each data address and the corresponding data object and data volume; and storing the second data set to the user equipment, and storing the third data set to the data space. The technical problem that efficient utilization of time and space is difficult to consider in both compressed storage and redundant storage is solved.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a method, device, storage medium and electronic device for storing non-redundant data. Background Art

[0002] In existing data storage methods, there is often a contradiction between time cost and space efficiency, which is usually solved by compressed storage or redundant storage, but each has its own shortcomings. Compressed storage is to compress data during storage to save space, but it needs to be decompressed when reading, which increases CPU (Central Processing Unit) consumption and increases time cost. For example, the row format ROW_FORMAT=COMPRESSED of the MySQL (an open source relational database management system used to store and manage data. It organizes data in a structured manner and supports data query and operation using SQL (Structured Query Language)) table space reduces storage space by compressing data, but requires additional compression and decompression operations. Redundant storage is to store data in an uncompressed manner, which is suitable for fast reading and writing scenarios, but has shortcomings in space utilization. For large objects (such as very large text, images, audio, video, etc.), if there are multiple references, multiple copies may need to be redundantly stored, resulting in space waste. Even for small objects (such as strings), repeated storage of a large amount of the same content will also bring significant space overhead. For example, although MySQL's COMPACT and DYNAMIC row formats do not perform compression, they will save a copy of the data for each table row, further wasting storage space. In short, the prior art is difficult to balance the efficient use of time and space in both compressed storage and redundant storage. Summary of the invention

[0003] The present application provides a method, device, storage medium and electronic device for storing non-redundant data to solve the technical problem that it is difficult to take into account the efficient use of time and space in both compressed storage and redundant storage.

[0004] In a first aspect, the present application provides a method for preserving non-redundant data, comprising: converting each data object in each data group in a first data set into a corresponding data address to obtain a second data set, wherein, in the above-mentioned first data set, a data object exists in at least one data group, and a data object in a data group corresponds to a data category; obtaining the data volume of each of the above-mentioned data addresses, wherein the above-mentioned data volume is the number of the above-mentioned data addresses in the above-mentioned second data set; forming a third data set according to each of the above-mentioned data addresses and their corresponding data objects and data volumes; saving the above-mentioned second data set to a user device, and saving the above-mentioned third data set to a data space.

[0005] In a second aspect, the present application provides a device for storing data without redundant data, comprising: a first conversion module, used to convert each data object in each data group in a first data set into a corresponding data address to obtain a second data set, wherein, in the above-mentioned first data set, a data object exists in at least one data group, and a data object in a data group corresponds to a data category; a first acquisition module, used to obtain the data volume of each of the above-mentioned data addresses, wherein the above-mentioned data volume is the number of the above-mentioned data addresses in the above-mentioned second data set; a first processing module, used to form a third data set according to each of the above-mentioned data addresses and their corresponding data objects and data volumes; a first storage module, used to save the above-mentioned second data set to a user device, and save the above-mentioned third data set to a data space.

[0006] As an optional example, the above-mentioned device also includes: a second acquisition module, which is used to obtain the query data object and query data category carried by the above-mentioned query instruction after saving the above-mentioned second data set to the user device and saving the above-mentioned third data set to the data space, upon receiving a query instruction issued by the above-mentioned user device, wherein the above-mentioned query instruction is used to query the data object corresponding to the above-mentioned query data category in the data group where the above-mentioned query data object is located; a second conversion module, which is used to convert the above-mentioned query data object into a corresponding first data address; a first search module, which is used to search for the second data address corresponding to the above-mentioned query data category in the data group where the above-mentioned first data address is located in the above-mentioned second data set according to the above-mentioned first data address; a third acquisition module, which is used to search for the data object corresponding to the above-mentioned second data address in the above-mentioned third data set according to the above-mentioned second data address, and return it to the above-mentioned user device.

[0007] As an optional example, the above-mentioned device also includes: a fourth acquisition module, which is used to obtain each insertion data object and its data category carried by the above-mentioned insertion instruction after saving the above-mentioned second data set to the user device and saving the above-mentioned third data set to the data space, upon receiving the insertion instruction issued by the above-mentioned user device, wherein the above-mentioned insertion instruction is used to insert the first target data group into the above-mentioned second data set, and the above-mentioned first target data group includes all the insertion data objects carried by the above-mentioned insertion instruction; a creation module, which is used to create the above-mentioned first target data group in the above-mentioned second data set; a third conversion module, which is used to convert each of the above-mentioned insertion data objects into a corresponding data address; a second saving module, which is used to save the data address corresponding to each of the above-mentioned insertion data objects to the above-mentioned first target data group according to the data category of each of the above-mentioned insertion data objects.

[0008] As an optional example, the above-mentioned device also includes: a second processing module, which is used to determine each inserted data object carried by the above-mentioned insertion instruction as a current data object after saving the data address corresponding to each of the above-mentioned inserted data objects to the above-mentioned first target data group according to the data category of each of the above-mentioned inserted data objects, and perform the following operations on the above-mentioned current data object: searching for the current data address corresponding to the above-mentioned current data object in the above-mentioned third data set; when the above-mentioned current data address exists in the above-mentioned third data set, adding 1 to the data volume of the above-mentioned current data address in the above-mentioned third data set; when the above-mentioned current data address does not exist in the above-mentioned third data set, saving the above-mentioned current data address and the current data object to the above-mentioned third data set, and setting the data volume of the above-mentioned current data address in the above-mentioned third data set to 1.

[0009] As an optional example, the above-mentioned device also includes: a fifth acquisition module, which is used to obtain each deletion data object carried by the above-mentioned deletion instruction after saving the above-mentioned second data set to the user device and saving the above-mentioned third data set to the data space, when receiving the deletion instruction issued by the above-mentioned user device, wherein the above-mentioned deletion instruction is used to convert the second target data group in the above-mentioned second data set, and the above-mentioned second target data group includes all the deletion data objects carried by the above-mentioned deletion instruction; a fourth conversion module, which is used to convert each of the above-mentioned deletion data objects into a corresponding data address; a first deletion module, which is used to determine the above-mentioned second target data group from the above-mentioned second data set according to the data address corresponding to each of the above-mentioned deletion data objects, and delete the above-mentioned second target data group from the above-mentioned second data set; a third processing module, which is used to determine each deletion data object carried by the above-mentioned deletion instruction as a current data object, and perform the following operations on the above-mentioned current data object: reduce the data amount of the data address corresponding to the above-mentioned current data object in the above-mentioned third data set by 1.

[0010] As an optional example, the apparatus further includes: a sixth acquisition module, which is used to, after saving the second data set to the user device and saving the third data set to the data space, acquire the modified data object, modified data category and target data object carried by the modification instruction upon receiving the modification instruction issued by the user device, wherein the modification instruction is used to modify the data object corresponding to the modified data category in the data group where the modified data object is located in the second data set to the target data object; a fifth conversion module, which is used to convert the modified data object into a corresponding modified data address; a sixth conversion module, which is used to convert the target data object into a corresponding target data address; a second search module, which is used to search the second data set for the modified data address according to the modified data address. The original data address corresponding to the above-mentioned modified data category in the data group where the modified data address is located; a first modification module, used to modify the original data address in the above-mentioned second data set to the above-mentioned target data address; a fourth processing module, used to reduce the data volume of the above-mentioned original data address in the above-mentioned third data set by 1; a third search module, used to search for the above-mentioned target data address in the above-mentioned third data set; a fifth processing module, used to increase the data volume of the above-mentioned target data address in the above-mentioned third data set by 1 when the above-mentioned target data address exists in the above-mentioned third data set; a third saving module, used to save the above-mentioned target data address and the target data object to the above-mentioned third data set when the above-mentioned target data address does not exist in the above-mentioned third data set, and set the data volume of the above-mentioned target data address in the above-mentioned third data set to 1.

[0011] As an optional example, the above-mentioned device also includes: a second modification module, which is used to delete the data group containing the above-mentioned target data volume from the above-mentioned third data volume when it is detected that the target data volume is 0 after saving the above-mentioned second data set to the user device and saving the above-mentioned third data set to the data space, wherein the above-mentioned target data volume is the data volume of any data address in the above-mentioned third data set; or when it is detected that the above-mentioned target data volume is 0 and reaches a preset time point, the data group containing the above-mentioned target data volume is deleted from the above-mentioned third data set.

[0012] In a third aspect, the present application provides a storage medium storing a computer program, wherein the computer program executes the above-mentioned method for saving non-redundant data when executed by a processor.

[0013] In a fourth aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-mentioned method for saving non-redundant data through the computer program.

[0014] The above technical solution provided by the embodiment of the present application has the following advantages compared with the related art:

[0015] The present application adopts a method of converting each data object in each data group in a first data set into a corresponding data address to obtain a second data set, wherein, in the above-mentioned first data set, a data object exists in at least one data group, and a data object in a data group corresponds to a data category; obtaining the data volume of each of the above-mentioned data addresses, wherein the above-mentioned data volume is the number of the above-mentioned data addresses in the above-mentioned second data set; forming a third data set according to each of the above-mentioned data addresses and their corresponding data objects and data volumes; saving the above-mentioned second data set to a user device, and saving the above-mentioned third data set to a data space. Since in the above-mentioned method, each data object in the first data set is converted into a corresponding data address to obtain the second data set, the second data set only needs to store one data address of the same data object, and the data space can be directly accessed through the data address, thereby achieving the purpose of saving storage space, accelerating access speed, and taking into account both time and space costs, thereby solving the technical problem that it is difficult to take into account efficient utilization of time and space in both compressed storage and redundant storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0019] Figure 1 is a flowchart of an optional method for storing non-redundant data according to an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of a first data set according to an optional method for storing non-redundant data in an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a second data set according to an optional method for storing non-redundant data in an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of a third data set according to an optional method for storing non-redundant data in an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of a query result data set of an optional method for storing non-redundant data according to an embodiment of the present application;

[0024] Figure 6 is a schematic diagram of an insertion result data set of an optional method for storing non-redundant data according to an embodiment of the present application;

[0025] Figure 7 is a schematic diagram of an insertion result data set of an optional method for storing non-redundant data according to an embodiment of the present application;

[0026] Figure 8 is a schematic diagram of a deletion result data set of an optional method for storing non-redundant data according to an embodiment of the present application;

[0027] Fig. 9 is a schematic diagram of an insertion result data set of an optional method for storing non-redundant data according to an embodiment of the present application;

[0028] Fig.10 is a schematic diagram of a modified result data set of an optional method for storing non-redundant data according to an embodiment of the present application;

[0029] Fig.11 is a schematic diagram of a modified result data set of an optional method for storing non-redundant data according to an embodiment of the present application;

[0030] Fig.12 is a schematic structural diagram of an optional non-redundant data storage device according to an embodiment of the present application;

[0031] Fig.13 is a schematic diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0033] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0034] According to a first aspect of an embodiment of the present application, a method for storing non-redundant data is provided. Optionally, as follows: Figure 1 As shown, the above method includes:

[0035] S102, converting each data object in each data group in the first data set into a corresponding data address to obtain a second data set, wherein in the first data set, a data object exists in at least one data group, and a data object in a data group corresponds to a data category;

[0036] S104, obtaining the data volume of each data address, wherein the data volume is the number of data addresses in the second data set;

[0037] S105, forming a third data set according to each data address and its corresponding data object and data amount;

[0038] S106, saving the second data set to the user device, and saving the third data set to the data space.

[0039] Optionally, in this embodiment, the data set may be a data table. Since each row and column in a single table may contain duplicate data, it may be expanded to the entire library or the entire data instance. The database system maintains the following data storage spaces:

[0040] 1. Table data space: If you specify the table data space when creating a table, the table data space is used to store specific data addresses, data objects, and data quantities. The table data space file is named with the table name plus a fixed suffix, for example "student information table t_student_info.data".

[0041] 2. Library data space: If the table data space file is not specified, if the enable parameter is "per_table_data_space=ON", the table data space is used. If the enable parameter is "per_table_data_space=OFF", the library data space is used according to "per_database_data_space=ON". The library data space file is named with the library name plus a fixed suffix, for example "student library d_student.data" and "teacher library d_teacher.data".

[0042] 3. Instance data space: If the table data space is not specified, and the enable parameters are "per_table_data_space=OFF" and "per_database_data_space=OFF", the instance-level data space is used. The instance-level data space file is named with the implementation example name plus a fixed suffix, such as "instance.data".

[0043] Optionally, in this embodiment, in the first data set, each data group includes one or more data objects, and each data object corresponds to a data category. Figure 2 A schematic diagram of a first data set is shown, wherein the first data set contains three groups of data, a row of data constitutes a group of data, a group of data includes multiple data objects "Zhang San", "Male", "1989-10-11", etc., and the corresponding data categories are "Name", "Gender", and "Birth", respectively. It can be seen that there can be multiple data objects "Zhang San" in the first data set.

[0044] Optionally, in this embodiment, each data object in the first data set is converted into a unique data address to form a second data set, such as Figure 3 A schematic diagram of a second data set is shown, in which the data address "0x4D68" corresponding to the data object "Zhang San" is calculated through the address function f(x). The address function can use a hash algorithm or an information digest algorithm. Finally, the data objects of the first data set are replaced by the corresponding data addresses. The first data set contains multiple identical data objects "Zhang San". In multiple data groups, after conversion, "0x4D68" is only stored once, and other references only retain its unique address, avoiding repeated storage. The second data set is traversed, the number of times each data address appears is counted, and the amount of data at the address is recorded. The amount of data reflects the frequency of use of the data object, providing support for subsequent optimization and query. According to each data address, the corresponding data object and its data amount, a third data set is created to record the mapping relationship and frequency information between the data address and the original data object. The data address "0x4D68" and the data object "Zhang San" and the data amount "2" are stored in the third data set for easy and quick search and management, and the following is obtained: Figure 4 The schematic diagram of a third data set is shown, the first column is the data address, the second column is the actual data object, and the last column is the data amount. Finally, the second data set (containing only the data address) is saved to the user device for efficient reference, and the third data set (data object and its meta information) is saved to the data space for storing and managing the complete data content and its usage information.

[0045] Optionally, in this embodiment, by only needing to store one data address of the same data object, redundant storage is greatly reduced, and in scenarios where the same data needs to be referenced multiple times (such as repeated strings or the same image fragments), space savings are significant. Direct access to the data space through the data address has low time complexity and is more efficient than traditional linear search or repeated loading of data. Comprehensive optimization saves storage space and speeds up access, taking into account both time and space costs.

[0046] As an optional example, after the second data set is saved to the user device and the third data set is saved to the data space, the method further includes:

[0047] Upon receiving a query instruction sent by a user device, obtaining a query data object and a query data category carried in the query instruction, wherein the query instruction is used to query a data object corresponding to the query data category in a data group where the query data object is located;

[0048] Convert the query data object into a corresponding first data address;

[0049] According to the first data address, searching in the second data set for a second data address corresponding to the query data category in the data group where the first data address is located;

[0050] According to the second data address, the data object corresponding to the second data address is searched in the third data set and obtained, and returned to the user device.

[0051] Optionally, in this embodiment, a query instruction sent by a user device is received, the query instruction includes a data object to be queried (referred to as a query data object) and a data category to be queried (referred to as a query data category), the purpose of the query instruction is that the user wishes to query a data object of a specified category in a data group where a certain data object is located, and the query instruction may include one or more query data categories, for example, based on Figure 2 The first data set shown, Figure 3 The second data set shown and Figure 4As shown in the third data set, the query instruction issued by the user based on the user device is "query the student number, gender, birth, school name and school address of Li Si", and the data object "Li Si" queried by the user is converted into its unique storage address (called the first data address) "0x5E79" through the address function. The first data address is used to locate the data group in the second data set. In the data group, the data address of the data object corresponding to the category is found according to the query data category (called the second data address), which are "0x3C57", "0x7091", "0x3366", "0xABCD", and "0x3248" respectively. According to the second data address, the corresponding actual data objects are found in the third data set, which are "xxyyzzYYYYMMDD0003", "female", "1991-11-23", "Beijing No. 999 Middle School", and "No. 123456789, Shuncheng Street, Baima Road, Mapo Town, Shunyi District, Beijing". These data objects are formed as shown in the figure. Figure 5 The query result data set is returned to the user device to complete the query.

[0052] Optionally, in this embodiment, by directly locating the data address and hierarchical search, the query complexity is greatly reduced, and even if the data scale is large, the user's query needs can be quickly responded to. By using the address mapping relationship between the second data set and the third data set, there is no need to repeatedly load the entire data set, and only the target object needs to be searched, saving memory and processing time.

[0053] As an optional example, after the second data set is saved to the user device and the third data set is saved to the data space, the method further includes:

[0054] When receiving an insert instruction sent by a user device, obtaining each insert data object and its data category carried by the insert instruction, wherein the insert instruction is used to insert a first target data group into the second data set, and the first target data group includes all the insert data objects carried by the insert instruction;

[0055] creating a first target data set in the second data set;

[0056] Convert each inserted data object into a corresponding data address;

[0057] According to the data category of each inserted data object, a data address corresponding to each inserted data object is saved in the first target data group.

[0058] As an optional example, after saving the data address corresponding to each inserted data object to the first target data group according to the data category of each inserted data object, the method further includes:

[0059] Determine each inserted data object carried by the insert instruction as the current data object, and perform the following operations on the current data object;

[0060] Searching for a current data address corresponding to the current data object in the third data set;

[0061] If the current data address exists in the third data set, the data amount of the current data address in the third data set is increased by 1;

[0062] When the current data address does not exist in the third data set, the current data address and the current data object are saved in the third data set, and the data amount of the current data address in the third data set is set to 1.

[0063] Optionally, in this embodiment, the user device sends an insert instruction, which carries each inserted data object, that is, the new data to be inserted, and the data category corresponding to each inserted data object, in order to add the inserted data object to the new data group of the second data set according to the category. Figure 2 The first data set shown, Figure 3 The second data set shown and Figure 4 In the third data set shown, the user issues an insert instruction of "Insert a piece of data: Student ID: xxyyzzYYYYMMDD0004; Name: Wang Wu; Gender: Male; Birthdate: 1991-11-23; School name: Hangzhou No. 888 Middle School; School address: No. 9876543210, Hongning Road, Xiaoshan District, Hangzhou". A new empty data group (called the first target data group) is created in the second data set to store the data address of the inserted data object. For each inserted data object: convert it into a unique data address through an address function, such as the data address "0x53E0" corresponding to the data object "Wang Wu", and obtain the data address of each inserted data object. According to the category of the inserted data object, save the data address of each inserted data object to the first target data group, and then Figure 3 The second data set shown is updated as follows Figure 6The insertion result data set shown is returned to the user device to complete the insertion. After the insertion is completed, the third data set must be updated according to the inserted data group. For each inserted data object (called the current data object), the following operations are performed: search in the third data set to see if the data address of the data object already exists. If the data address is found in the third data set, increase the data volume corresponding to the data address by "increase by 1". For example, if the data address "0x6F8A" of the data object "male" already exists, then the data volume "1" of the data address "0x6F8A" is modified to "2". If the data address is not found in the third data set, add the current data address and data object to the third data set, and set the data volume corresponding to the data address to 1. For example, if the data address "0x53E0" of the data object "Wang Wu" does not exist, then add a new data group to the third data set, and add the data address "0x53E0" of the data object "Wang Wu" and its data volume "1" to the data group. Figure 4 The third data set shown is updated as follows Figure 7 The inserted result dataset is shown.

[0064] Optionally, in this embodiment, the newly added data object is directly converted into a data address and stored, avoiding repeated storage of the same data object, significantly reducing storage space waste, and data volume statistics are completed through simple address lookup and increment operations, with low time complexity and high performance. New target data groups can be dynamically created and inserted as needed, and the system has good scalability and is suitable for high-concurrency data operation scenarios. The mapping relationship between data objects and data addresses is uniformly maintained in the third data set to ensure reference consistency and prevent data duplication and reference disorder.

[0065] As an optional example, after the second data set is saved to the user device and the third data set is saved to the data space, the method further includes:

[0066] Upon receiving a deletion instruction sent by a user device, obtaining each deletion data object carried by the deletion instruction, wherein the deletion instruction is used to delete a second target data group in the second data set, the second target data group including all the deletion data objects carried by the deletion instruction;

[0067] Convert each deleted data object into a corresponding data address;

[0068] Determine a second target data group from the second data set according to the data address corresponding to each deleted data object, and delete the second target data group from the second data set;

[0069] Determine each deleted data object carried by the delete instruction as the current data object, and perform the following operations on the current data object;

[0070] The data amount of the data address corresponding to the current data object in the third data set is reduced by 1.

[0071] Optionally, in this embodiment, the user device sends a deletion instruction, which carries a deletion data object, that is, a specific data object to be deleted, and the deletion instruction is intended to remove these data objects from the target data group of the second data set. Figure 2 The first data set shown, Figure 3 The second data set shown and Figure 4 For the third data set shown, the user issues a deletion instruction of "delete a piece of data: student number: xxyxxyyzzYYYYMMDD0002; name: Zhang San; gender: female; birth: 1989-10-11; school name: Hangzhou No. 888 Middle School; school address: No. 9876543210, Hongning Road, Xiaoshan District, Hangzhou". For each deleted data object, it is converted into a corresponding data address through an address function, such as the data object "Zhang San" is converted into the data address "0x4D68". According to the data addresses of all the deleted data objects, the target data group to which they belong (called the second target data group) is determined from the second data set, and finally the second target data group is deleted from the second data set. Figure 3 The second data set shown is updated as follows Figure 8 After the deletion is completed, the third data set is updated according to the deleted data group. According to the data address of each deleted data object, the corresponding record in the third data set is found, and the data volume of the data address is reduced by 1. For example, the data volume "2" of the data address "0x4D68" is modified to "1". Figure 4 The third data set shown is updated as follows Fig. 9 The inserted result dataset is shown.

[0072] Optionally, in this embodiment, it is ensured that the deletion operation only removes the specified data object, the deletion of the target data group is performed synchronously with the update of the third data set, data management is more efficient, and the target data group in the second data set is deleted to directly release storage space. The reduction of the reference count in the third data set can further trigger the data cleanup mechanism.

[0073] As an optional example, after the second data set is saved to the user device and the third data set is saved to the data space, the method further includes:

[0074] When receiving a modification instruction sent by a user device, obtaining a modification data object, a modification data category, and a target data object carried in the modification instruction, wherein the modification instruction is used to modify a data object corresponding to the modification data category in a data group where the modification data object in the second data set is located into a target data object;

[0075] Convert the modified data object into the corresponding modified data address;

[0076] Convert the target data object into the corresponding target data address;

[0077] According to the modified data address, searching in the second data set for the original data address corresponding to the modified data category in the data group where the modified data address is located;

[0078] Modify the original data address in the second data set to the target data address;

[0079] Subtract 1 from the data amount of the original data address in the third data set;

[0080] Find the target data address in the third data set;

[0081] If the target data address exists in the third data set, the data amount of the target data address in the third data set is increased by 1;

[0082] When the target data address does not exist in the third data set, the target data address and the target data object are saved in the third data set, and the data amount of the target data address in the third data set is set to 1.

[0083] Optionally, in this embodiment, the user device issues a modification instruction, which carries a modification data object: used to locate the data group where the data object to be modified belongs, a modification data category: the data category to which the data object to be modified belongs, and a target data object: a new data object used to replace the data object to be modified. Figure 2 The first data set shown, Figure 3 The second data set shown and Figure 4In the third data set shown, the user issues a modification instruction of "Name: Li Si's school name: Beijing No. 999 Middle School; School address: No. 123456789, Shuncheng Street, Baima Road, Mapo Town, Shunyi District, Beijing; Change to school name: Nanjing No. 777 Middle School; School address: No. 666, Machou East Road, Jiangning District, Nanjing". The essence of modifying data is to delete the original record and add a new record. Therefore, the deletion and insertion can be performed first through the deletion instruction and insertion instruction recorded above, or the data object to be modified can be located by modifying the data object and modifying the data category, and the data object to be modified can be modified directly. Specifically, the modified data object "Li Si" is converted into the corresponding data address "0x5E79" through the address function, and the data address "0x5E79" is located in the second data set to obtain the data group to be modified as the third group of data, and the data address "0xABCD" to be modified is located according to the modified data category "school name". The target data object "Nanjing No. 777 Middle School" is converted to the corresponding data address "0x53E2" through the address function, and the data address "0xABCD" in the third group of data in the second data set is directly modified to "0x53E2", and the school address is deduced in the same way. Figure 3 The second data set shown is updated as follows Fig.10 The modified result data set shown is returned to the user device to complete the modification. After the modification is completed, the third data set must be updated according to the modified data set. According to the original data address "0xABCD", the data volume "2" of the data address "0xABCD" in the third data set is modified to "1". The target data address "0x53E2" is searched in the third data set. If it exists, the data volume of the data address "0x53E2" in the third data set is increased by 1. If it does not exist, the target data address and the corresponding data object are saved to the third data set, and the data volume is initialized to 1. Finally, Figure 4 The third data set shown is updated as follows Fig.11 The modified result dataset is shown.

[0084] Optionally, in this embodiment, the data mapping relationship is directly modified by address replacement, without copying or deleting redundant data, thereby reducing the redundancy of storage operations. While modifying the data, the data volume is dynamically updated to ensure that the actual storage and reference status of each data object are always consistent, avoiding redundancy or omission. The operation time complexity of address replacement and count update is low, reducing direct operations on data objects, and optimizing storage costs and access performance.

[0085] As an optional example, after the second data set is saved to the user device and the third data set is saved to the data space, the method further includes:

[0086] When it is detected that the target data volume is 0, the data group where the target data volume is located is deleted from the third data set, wherein the target data volume is the data volume of any data address in the third data set; or

[0087] When it is detected that the target data volume is 0 and the preset time point is reached, the data group containing the target data volume is deleted from the third data set.

[0088] Optionally, in this embodiment, the data volume in the third data set is monitored in real time, and when the data volume drops to 0, the third data set is managed, thereby further optimizing the storage space and improving the system efficiency. Specifically, when it is detected that the data volume of a certain data address drops to 0, it is managed based on the following multiple strategies. Strategy 1: When it is detected that the data volume of a certain data address drops to 0, the data group where the data volume is located is immediately deleted from the third data set; Strategy 2: Delayed deletion of the data group, when it is detected that the data volume of a certain data address drops to 0, the data group where the data volume is located is temporarily retained in the third data set, and when the preset time point is reached, it is checked whether the data volume is still 0, and the data group where the data volume is located is deleted. If the data volume is restored (for example, it is re-referenced), the data group where the data volume is located continues to be retained. The preset time point can be a time point set according to the first detection of the data volume as 0, or it can be directly set to 00:00 every day; Strategy 3: The timing of the deletion operation of the data group with a data volume of 0 is determined by the database software and executed when the system is idle.

[0089] Optionally, in this embodiment, by combining immediate deletion and delayed deletion, the third data set is efficiently managed when the data volume is 0, which not only optimizes the storage space but also takes into account the system performance and flexibility, and is suitable for a variety of dynamic scenarios.

[0090] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0091] According to another aspect of the embodiment of the present application, a device for storing non-redundant data is also provided, such as Fig.12 As shown, including:

[0092] A first conversion module 1202 is used to convert each data object in each data group in the first data set into a corresponding data address to obtain a second data set, wherein in the first data set, a data object exists in at least one data group, and a data object in a data group corresponds to a data category;

[0093] A first acquisition module 1204 is used to acquire the data volume of each data address, wherein the data volume is the number of data addresses in the second data set;

[0094] A first processing module 1206, configured to form a third data set according to each data address and its corresponding data object and data amount;

[0095] The first saving module 1208 is used to save the second data set to the user device and save the third data set to the data space.

[0096] It should be noted that the first conversion module 1202 in this embodiment can be used to execute step S102 in the embodiment of the present application, the first acquisition module 1204 in this embodiment can be used to execute step S104 in the embodiment of the present application, the first processing module 1206 in this embodiment can be used to execute step S106 in the embodiment of the present application, and the first saving module 1208 in this embodiment can be used to execute step S108 in the embodiment of the present application.

[0097] As an optional example, the above device further includes:

[0098] A second acquisition module is used to, after saving the second data set to the user device and saving the third data set to the data space, acquire the query data object and the query data category carried by the query instruction when receiving the query instruction sent by the user device, wherein the query instruction is used to query the data object corresponding to the query data category in the data group where the query data object is located;

[0099] A second conversion module, used to convert the query data object into a corresponding first data address;

[0100] A first search module, used to search, in the second data set, according to the first data address, for a second data address corresponding to the query data category in the data group where the first data address is located;

[0101] The third acquisition module is used to search for and acquire the data object corresponding to the second data address in the third data set according to the second data address, and return the data object to the user device.

[0102] As an optional example, the above device further includes:

[0103] a fourth acquisition module, configured to, after saving the second data set to the user device and saving the third data set to the data space, acquire each insertion data object and its data category carried by the insertion instruction upon receiving the insertion instruction issued by the user device, wherein the insertion instruction is used to insert the first target data group into the second data set, and the first target data group includes all the insertion data objects carried by the insertion instruction;

[0104] A creation module, used for creating a first target data set in a second data set;

[0105] A third conversion module, used to convert each inserted data object into a corresponding data address;

[0106] The second saving module is used to save the data address corresponding to each inserted data object to the first target data group according to the data category of each inserted data object.

[0107] As an optional example, the above device further includes:

[0108] A second processing module is used to determine each inserted data object carried by the insertion instruction as a current data object after saving the data address corresponding to each inserted data object to the first target data group according to the data category of each inserted data object, and perform the following operations on the current data object;

[0109] Searching for a current data address corresponding to the current data object in the third data set;

[0110] If the current data address exists in the third data set, the data amount of the current data address in the third data set is increased by 1;

[0111] When the current data address does not exist in the third data set, the current data address and the current data object are saved in the third data set, and the data amount of the current data address in the third data set is set to 1.

[0112] As an optional example, the above device further includes:

[0113] a fifth acquisition module, configured to, after saving the second data set to the user device and saving the third data set to the data space, acquire each deleted data object carried by the deletion instruction upon receiving a deletion instruction issued by the user device, wherein the deletion instruction is used to delete a second target data group in the second data set, and the second target data group includes all deleted data objects carried by the deletion instruction;

[0114] A fourth conversion module, used to convert each deleted data object into a corresponding data address;

[0115] A first deletion module, configured to determine a second target data group from the second data set according to a data address corresponding to each deleted data object, and delete the second target data group from the second data set;

[0116] A third processing module is used to determine each deleted data object carried by the deletion instruction as a current data object, and perform the following operations on the current data object;

[0117] The data amount of the data address corresponding to the current data object in the third data set is reduced by 1.

[0118] As an optional example, the above device further includes:

[0119] a sixth acquisition module, configured to, after saving the second data set to the user device and saving the third data set to the data space, acquire the modified data object, the modified data category, and the target data object carried by the modification instruction upon receiving the modification instruction issued by the user device, wherein the modification instruction is used to modify the data object corresponding to the modified data category in the data group where the modified data object in the second data set is located into the target data object;

[0120] A fifth conversion module, used to convert the modified data object into a corresponding modified data address;

[0121] A sixth conversion module, used to convert the target data object into a corresponding target data address;

[0122] A second search module is used to search, according to the modified data address, in the second data set for an original data address corresponding to the modified data category in the data group where the modified data address is located;

[0123] A first modification module, used for modifying the original data address in the second data set into the target data address;

[0124] A fourth processing module, used for reducing the data amount of the original data address in the third data set by 1;

[0125] A third search module, used to search for a target data address in a third data set;

[0126] A fifth processing module, configured to increase the data amount of the target data address in the third data set by 1 if the target data address exists in the third data set;

[0127] The third saving module is used to save the target data address and the target data object to the third data set if the target data address does not exist in the third data set, and set the data amount of the target data address in the third data set to 1.

[0128] As an optional example, the above device further includes:

[0129] A second modification module is used for, after saving the second data set to the user device and saving the third data set to the data space, deleting the data group containing the target data amount from the third data set when detecting that the target data amount is 0, wherein the target data amount is the data amount of any data address in the third data set; or

[0130] When it is detected that the target data volume is 0 and the preset time point is reached, the data group containing the target data volume is deleted from the third data set.

[0131] For other examples of this embodiment, please refer to the above examples and will not be repeated here.

[0132] Fig.13 is a schematic diagram of an optional electronic device according to an embodiment of the present application, such as Fig.13 As shown, it includes a processor 1302, a communication interface 1304, a memory 1306 and a communication bus 1308, wherein the processor 1302, the communication interface 1304 and the memory 1306 communicate with each other through the communication bus 1308, wherein,

[0133] Memory 1306, used for storing computer programs;

[0134] The processor 1302 is used to execute the computer program stored in the memory 1306 to implement the following steps:

[0135] Convert each data object in each data group in the first data set into a corresponding data address to obtain a second data set, wherein in the first data set, a data object exists in at least one data group, and a data object in a data group corresponds to a data category;

[0136] Obtaining the data volume of each data address, wherein the data volume is the number of data addresses in the second data set;

[0137] forming a third data set according to each data address and its corresponding data object and data amount;

[0138] The second data set is saved to the user device, and the third data set is saved to the data space.

[0139] Optionally, in this embodiment, the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.13 The communication interface is used for communication between the above electronic device and other devices.

[0140] The storage device may include a RAM, or may include a non-volatile memory, such as at least one disk storage device. Optionally, the storage device may also be at least one storage device located away from the aforementioned processor.

[0141] As an example, the memory 1306 may include, but is not limited to, the first conversion module 1202, the first acquisition module 1204, the first processing module 1206, and the first storage module 1208 in the non-redundant data storage device. In addition, other module units in the non-redundant data storage device may also be included but are not limited to, which will not be repeated in this example.

[0142] The above-mentioned processor can be a general-purpose processor, which can include but not be limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0143] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0144] It can be understood by those skilled in the art that Fig.13 The structure shown is for illustration only. The device for implementing the above-mentioned method for saving non-redundant data may be a terminal device, which may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Fig.13 The structure of the electronic device is not limited. For example, the electronic device may also include Fig.13 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Fig.13 Different configurations shown.

[0145] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0146] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program executes the steps in the above-mentioned method for storing non-redundant data when executed by a processor.

[0147] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0148] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0149] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0150] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0151] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0152] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0153] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0154] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for storing non-redundant data, characterized in that: include: Convert each data object in each data group in the first data set into a corresponding data address to obtain a second data set, wherein in the first data set, a data object exists in at least one data group, and a data object in a data group corresponds to a data category; Acquire the data volume of each of the data addresses, wherein the data volume is the number of the data addresses in the second data set; According to each of the data addresses and the corresponding data objects and data amounts, a third data set is formed; The second data set is saved to the user device, and the third data set is saved to the data space.

2. The method according to claim 1, characterized in that After saving the second data set to the user device and saving the third data set to the data space, the method further includes: Upon receiving a query instruction sent by the user equipment, obtaining a query data object and a query data category carried by the query instruction, wherein the query instruction is used to query a data object corresponding to the query data category in the data group where the query data object is located; Converting the query data object into a corresponding first data address; According to the first data address, searching in the second data set for a second data address corresponding to the query data category in the data group where the first data address is located; According to the second data address, the data object corresponding to the second data address is searched in the third data set and returned to the user equipment.

3. The method according to claim 1, characterized in that After saving the second data set to the user device and saving the third data set to the data space, the method further includes: Upon receiving an insert instruction sent by the user equipment, acquiring each insert data object and its data category carried by the insert instruction, wherein the insert instruction is used to insert a first target data group into the second data set, and the first target data group includes all the insert data objects carried by the insert instruction; creating the first target data set in the second data set; Convert each of the inserted data objects into a corresponding data address; According to the data category of each of the inserted data objects, a data address corresponding to each of the inserted data objects is saved in the first target data group.

4. The method according to claim 3, characterized in that After saving the data address corresponding to each of the inserted data objects to the first target data group according to the data category of each of the inserted data objects, the method further includes: Determine each inserted data object carried by the insert instruction as a current data object, and perform the following operations on the current data object; Searching the third data set for a current data address corresponding to the current data object; In the case where the current data address exists in the third data set, adding 1 to the data amount of the current data address in the third data set; In the case that the current data address does not exist in the third data set, the current data address and the current data object are saved in the third data set, and the data amount of the current data address in the third data set is set to 1.

5. The method according to claim 1, characterized in that After saving the second data set to the user device and saving the third data set to the data space, the method further includes: Upon receiving a deletion instruction sent by the user equipment, obtaining each deletion data object carried by the deletion instruction, wherein the deletion instruction is used to delete a second target data group in the second data set, the second target data group including all the deletion data objects carried by the deletion instruction; Convert each of the deleted data objects into a corresponding data address; Determine the second target data group from the second data set according to the data address corresponding to each of the deleted data objects, and delete the second target data group from the second data set; Determine each deleted data object carried by the deletion instruction as a current data object, and perform the following operations on the current data object; The data amount of the data address corresponding to the current data object in the third data set is reduced by 1.

6. The method according to claim 1, characterized in that After saving the second data set to the user device and saving the third data set to the data space, the method further includes: Upon receiving a modification instruction issued by the user equipment, obtaining a modification data object, a modification data category, and a target data object carried by the modification instruction, wherein the modification instruction is used to modify a data object corresponding to the modification data category in a data group where the modification data object in the second data set is located into the target data object; Converting the modified data object into a corresponding modified data address; Converting the target data object into a corresponding target data address; According to the modified data address, searching in the second data set for an original data address corresponding to the modified data category in the data group where the modified data address is located; Modify the original data address in the second data set to the target data address; Subtract 1 from the data amount of the original data address in the third data set; Searching for the target data address in the third data set; In the case where the target data address exists in the third data set, adding 1 to the data amount of the target data address in the third data set; In the case that the target data address does not exist in the third data set, the target data address and the target data object are saved in the third data set, and the data amount of the target data address in the third data set is set to 1.

7. The method according to any one of claims 1 to 6, characterized in that: After saving the second data set to the user device and saving the third data set to the data space, the method further includes: When it is detected that the target data volume is 0, the data group to which the target data volume belongs is deleted from the third data set, wherein the target data volume is the data volume of any data address in the third data set; or When it is detected that the target data volume is 0 and a preset time point is reached, the data group containing the target data volume is deleted from the third data set.

8. A device for storing non-redundant data, characterized in that: include: A first conversion module is used to convert each data object in each data group in the first data set into a corresponding data address to obtain a second data set, wherein in the first data set, a data object exists in at least one data group, and a data object in a data group corresponds to a data category; A first acquisition module, configured to acquire a data volume of each of the data addresses, wherein the data volume is the number of the data addresses in the second data set; A first processing module, used for forming a third data set according to each of the data addresses and the corresponding data objects and data amounts; The first saving module is used to save the second data set to the user equipment and save the third data set to the data space.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images