Structural body data processing method and device, electronic equipment and storage medium

Abstract

The application relates to a structural data processing method and device, electronic equipment and a storage medium. The method comprises the following steps: in the case of storing structural data, storing the structural data to a first position area of a data storage area, and storing query information corresponding to the structural data to a second position area of a query information storage area; in the case of querying the structural data, obtaining the query information stored in the second position area of the query information storage area, and querying the structural data from the first position area of the data storage area according to the query information, wherein the data storage area and the query information storage area are two independent storage areas on the same device. In this way, the structural data can be quickly stored and queried, thereby solving the problem that the traditional structural data processing mode is time-consuming in the storage and query process.

Description

Methods, apparatuses, electronic devices and storage media for processing structured data Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, electronic device and storage medium for processing structured data. Background Technology

[0002] Because structured data can combine different types of data, and its data structure is "highly cohesive and loosely coupled," it is more conducive to program reading, understanding, and portability. Therefore, structured data plays a very important role in practical applications.

[0003] Currently, the traditional method for processing structured data typically involves storing each column or row of data as a unit during storage and then querying each column or row as a unit during retrieval. Therefore, this traditional method is time-consuming in both storage and retrieval processes.

[0004] Therefore, how to quickly store and query structured data has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for processing structured data, in order to solve the problem that traditional methods of processing structured data are time-consuming in the storage and retrieval process.

[0006] In a first aspect, embodiments of this application provide a method for processing structure data, the method comprising:

[0007] When storing structure data, the structure data is stored in the first location area of ​​the data storage area, and the query information corresponding to the structure data is stored in the second location area of ​​the query information storage area.

[0008] When querying the structure data, the query information stored in the second location area of ​​the query information storage area is obtained, and the structure data is retrieved from the first location area of ​​the data storage area based on the query information.

[0009] The data storage area and the query information storage area are two independent storage areas on the same device. The query information includes the hash value obtained based on the target field in the structure data and the location information corresponding to the first location area of ​​the data storage area. The target field is a field in the structure data whose value does not repeat the same field in other structure data.

[0010] Optionally, storing the structure data in a first location area of ​​the data storage area and storing the query information corresponding to the structure data in a second location area of ​​the query information storage area includes:

[0011] Calculate a first hash value and at least one second hash value corresponding to the target field, wherein the first hash value is calculated based on a preset hash algorithm to obtain the value of the target field, and the at least one second hash value is calculated based on a preset hash algorithm to obtain at least one intermediate value, wherein the at least one intermediate value is obtained by processing the value of the target field using at least one preset algorithm;

[0012] Based on the first hash value, determine the second location area of ​​the query information storage area where the query information is stored;

[0013] The structure data is stored in the first location area of ​​the data storage area, and the location information of the first location area of ​​the data storage area and the at least one second hash value are used as query information and stored in the second location area of ​​the query information storage area.

[0014] Optionally, determining the second location region of the query information storage area where the query information is stored based on the first hash value includes:

[0015] The first hash value and the total length of the query information storage area are moduloed to obtain the modulo result. The total length of the query information storage area is obtained by rounding up the product of the number of query information that the query information storage area is allowed to store and a preset threshold. The preset threshold is any value greater than 1.

[0016] The remainder result is used as the storage sequence of the query information in the query information storage area, and each storage sequence corresponds to one query information.

[0017] Based on the storage sequence, determine the second location area of ​​the query information storage area.

[0018] Optionally, after performing a modulo operation on the first hash value and the total length of the query information storage area to obtain the modulo result, the method further includes:

[0019] If the remainder result is the same as the remainder result calculated previously, the other storage sequences after the storage sequence are traversed sequentially until the first storage sequence that has not yet stored the query information is reached, and the second location area of ​​the query information storage area is determined according to the traversed storage sequence.

[0020] Optionally, the location information of the first location area of ​​the data storage area includes the offset of the starting position of the first location area of ​​the data storage area relative to the starting position of the data storage area and the data length of the structure data.

[0021] Optionally, obtaining the query information stored in the second location area of ​​the query information storage area, and querying the structure data from the first location area of ​​the data storage area based on the query information, includes:

[0022] Calculate a first hash value and at least one second hash value corresponding to the target field, wherein the first hash value is calculated based on a preset hash algorithm to obtain the value of the target field, and the at least one second hash value is calculated based on a preset hash algorithm to obtain at least one intermediate value, wherein the at least one intermediate value is obtained by processing the value of the target field using at least one preset algorithm;

[0023] The second location region of the query information storage area is determined based on the first hash value and the at least one second hash value;

[0024] Obtain the location information of the first location area of ​​the data storage area stored in the second location area of ​​the query information storage area;

[0025] The structure data is retrieved from the data storage area based on the location information of the first location region of the data storage area.

[0026] Optionally, determining the second location region of the query information storage area based on the first hash value and the at least one second hash value includes:

[0027] The first hash value and the total length of the query information storage area are moduloed to obtain the modulo result. The total length of the query information storage area is obtained by rounding up the product of the number of query information that the query information storage area is allowed to store and a preset threshold. The preset threshold is any value greater than 1.

[0028] The remainder result is used as the storage sequence of the query information in the query information storage area;

[0029] The at least one second hash value is compared with the at least one second hash value stored in the storage sequence;

[0030] If at least one second hash value matches at least one second hash value stored in the storage sequence, the second location area of ​​the query information storage area is determined according to the storage sequence;

[0031] If the at least one second hash value is inconsistent with the at least one second hash value stored in the storage sequence, the other storage sequences located after the storage sequence are traversed sequentially until at least one second hash value in the storage sequence is consistent with the at least one second hash value, and the second location area of ​​the query information storage area is determined according to the traversed storage sequence.

[0032] Optionally, retrieving the structure data from the data storage area based on the location information of the first location region of the data storage area includes:

[0033] Based on the location information of the first location area of ​​the data storage area, determine the offset of the starting position of the first location area of ​​the data storage area relative to the starting position of the data storage area and the data length of the structure data;

[0034] The structure data is obtained by querying the data storage area based on the offset of the starting position of the first location region of the data storage area relative to the starting position of the data storage area and the data length of the structure data.

[0035] Optionally, the method further includes:

[0036] When other devices need to use the structure data, the data storage area and the query information storage area are packaged and transmitted to the other devices.

[0037] Secondly, embodiments of this application also provide a structure data processing apparatus, the apparatus comprising:

[0038] The storage module is used to store the structure data in a first location area of ​​the data storage area and store the query information corresponding to the structure data in a second location area of ​​the query information storage area when storing structure data.

[0039] The query module is used to retrieve the query information stored in the second location area of ​​the query information storage area when querying the structure data, and to retrieve the structure data from the first location area of ​​the data storage area based on the query information.

[0040] The data storage area and the query information storage area are two independent storage areas on the same device. The query information includes the hash value obtained based on the target field in the structure data and the location information corresponding to the first location area of ​​the data storage area. The target field is a field in the structure data whose value does not repeat the same field in other structure data.

[0041] Thirdly, embodiments of this application also provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0042] Memory, used to store computer programs;

[0043] When a processor executes a program stored in memory, it implements the steps of the method for processing structure data as described in any embodiment of the first aspect.

[0044] Fourthly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method for processing structure data as described in any embodiment of the first aspect.

[0045] In this embodiment, when storing structure data, the structure data is stored in a first location area of ​​a data storage area, and the query information corresponding to the structure data is stored in a second location area of ​​a query information storage area. When querying the structure data, the query information stored in the second location area of ​​the query information storage area is obtained, and the structure data is retrieved from the first location area of ​​the data storage area based on the query information. The data storage area and the query information storage area are two independent storage areas on the same device. The query information includes a hash value obtained based on a target field in the structure data and location information corresponding to the first location area of ​​the data storage area. The target field is a field in the structure data whose value does not repeat the same field as other fields in the same structure data. In this way, the structure data and the query information corresponding to it can be stored separately in two independent storage areas. Therefore, when storing structure data, it is possible to...

[0046] This allows for the direct storage of the entire structure data in the data storage area, enabling fast storage of structure data. Furthermore, when querying the structure data, the location information of the structure data within the data storage area can be retrieved from the query information storage area, thus achieving structure...

[0047] This enables fast querying of structured data, thus solving the problem of long processing times in the storage and retrieval processes of traditional structured data processing methods. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 is a flowchart illustrating a method for processing structure data according to an embodiment of this application;

[0051] Figure 2 is a schematic diagram of the structure of a data storage area and a query information storage area provided in an embodiment of this application;

[0052] Figure 3 is a schematic diagram of a structure data processing device provided in an embodiment of this application;

[0053] Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] Referring to Figure 1, Figure 1 is a flowchart illustrating a method for processing structure data according to an embodiment of this application. As shown in Figure 1, the method for processing structure data may include the following steps:

[0056] Step 101: When storing structure data, store the structure data in the first data storage area.

[0057] The location area is determined, and the query information corresponding to the structure data is stored in the second location area of ​​the query information storage area.

[0058] Specifically, the aforementioned data storage area and the aforementioned query information storage area are two independent storage regions on the same device, such as two different partitions in memory. The data storage area is used to store structure data, and the query information storage area is used to store...

[0059] The query information corresponding to the structure data. This query information may include, but is not limited to, the hash value obtained based on the target field 0 in the structure data and the location information corresponding to the first location area of ​​the data storage area. The target field is the structure...

[0060] A field in a data structure whose value is unique among identical fields in other data structures. For example, if multiple data structures to be stored all include an ID number field, and the value of that field in one data structure is unique among the values ​​of that field in other data structures, then that field can be used as the target field for hash calculation to obtain the hash value of each data structure.

[0061] 5. The length of the first location region mentioned above is consistent with the length of the stored structure data, and the starting position of the first location region is...

[0062] The starting position of the second position area relative to the data storage area and the length of the structure data can be stored as query information in the query information storage area. The length of the second position area is consistent with the length of the query information. The starting position of the second position area relative to the starting position of the query information storage area is determined based on the hash value calculated from the target field in the structure data, and other hash values ​​calculated based on this hash value can be stored as query information in the query information storage area.

[0063] In this step, when storing the structure data, the structure data can be stored in the first location area of ​​the data storage area, and the query information corresponding to the structure data can be stored in the second location area of ​​the query information storage area. That is, the structure data and the query information corresponding to the structure data are stored separately in two different storage areas.

[0064] Step 102: When querying the structure data, obtain the query information stored in the second location area of ​​the query information storage area, and retrieve the structure data from the first location area of ​​the data storage area based on the query information.

[0065] In this step, when querying the structure data, the query information stored in the second location area of ​​the query information storage area can be obtained, and then the corresponding structure data can be retrieved from the first location area of ​​the data storage area based on the query information.

[0066] In this embodiment, the structure data and the corresponding query information can be stored separately in two independent storage areas. In this way, when storing structure data, the entire structure data can be directly stored in the data storage area, achieving fast storage of structure data. When querying the structure data, the location information of the structure data in the data storage area can be obtained from the query information storage area, achieving fast query of structure data. This solves the problem of long time consumption in the storage and querying process of traditional structure data processing methods.

[0067] Furthermore, step 101 above, storing the structure data in the first location area of ​​the data storage area and storing the query information corresponding to the structure data in the second location area of ​​the query information storage area, specifically includes the following steps:

[0068] Calculate a first hash value and at least one second hash value corresponding to the target field, wherein the first hash value is calculated based on a preset hash algorithm to obtain the value of the target field, and at least one second hash value is calculated based on a preset hash algorithm to obtain at least one intermediate value, and at least one intermediate value is obtained by processing the value of the target field using at least one preset algorithm;

[0069] Based on the first hash value, determine the second location area of ​​the query information storage area where the query information is stored;

[0070] The structure data is stored in the first location area of ​​the data storage area, and the location information of the first location area of ​​the data storage area and at least one second hash value are used as query information and stored in the second location area of ​​the query information storage area.

[0071] In one embodiment, when storing structure data, a first hash value and at least one second hash value corresponding to the target field of the structure data can be calculated first. In this way, the second location area of ​​the query information storage area for storing query information of the structure data can be determined according to the first hash value. Then, after storing the structure data in the first location area of ​​the data storage area, the location information of the first location area of ​​the data storage area and at least one second hash value are used as query information and stored in the second location area of ​​the query information storage area.

[0072] It should be noted that both the first hash value and at least one second hash value are calculated using a preset hash algorithm. Compared to using Cyclic Redundancy Check (CRC) or other algorithms, hash values ​​calculated using the preset hash algorithm have greater randomness, thus significantly reducing the probability that different structured data will yield the same value. Specifically, when calculating the first hash value, the preset hash algorithm is directly used to calculate the value of the target field. When calculating the second hash value, the preset algorithm is first used to calculate the value of the target field, such as adding, subtracting, multiplying, or dividing the target field value by a certain value to obtain at least one intermediate value. Then, the preset hash algorithm is used to calculate at least one intermediate value to obtain at least one second hash value. The number of these second hash values ​​can be any integer, such as 1, 2, or 3. These second hash values ​​can be stored as query information in the second location area of ​​the query information storage area to verify the accuracy of the storage location of the query information corresponding to the structured data. Furthermore, when there are multiple values ​​corresponding to the target field of the same structured data, and they are not unique, the value at the default position of the target field can be selected for hash calculation, such as selecting the first or second value of the target field. In this way, when performing hash calculation on the target field of multiple structured data, a unique value can be selected for hash calculation.

[0073] In this embodiment, by storing the structure data and the corresponding query information in two separate storage areas, the entire structure data can be directly stored in the data storage area, achieving fast storage of the structure data.

[0074] Further, the above steps, including determining the second location area of ​​the query information storage area where the query information is stored based on the first hash value, include:

[0075] The first hash value and the total length of the query information storage area are moduloed to obtain the modulo result. The total length of the query information storage area is obtained by rounding up the product of the number of query information that the query information storage area is allowed to store and a preset threshold. The preset threshold is any value greater than 1.

[0076] The remainder result is used as the storage sequence of query information in the query information storage area, and each storage sequence corresponds to one query information.

[0077] Based on the storage sequence, determine the second location area of ​​the query information storage area.

[0078] It should be noted that the total length of the query information storage area is obtained by rounding up the product of the number of query information items allowed to be stored in the query information storage area and a preset threshold. This preset threshold can be any value greater than 1. In an optional embodiment, the preset threshold can be 1.25. Assuming that the number of query information items allowed to be stored is 3, the result of 3 * 1.25 rounded up is 4. Therefore, the total length of the query information storage area is the sum of the lengths of the 4 query information items, meaning there are 4 storage sequences in the query information storage area, and each storage sequence is used to store one query information item. In other words, before storing query information, the total length of the query information storage area needs to be determined based on the number of query information items to be stored and the preset threshold. This allows the location area of ​​each query information item in the query information storage area to be determined based on its length.

[0079] In one embodiment, when determining the second location area of ​​the query information storage area where the query information is stored based on the first hash value, it is necessary to perform a modulo operation on the first hash value and the total length of the query information storage area to obtain the modulo result. This modulo result is then used as the storage sequence of the query information in the query information storage area, and the second location area of ​​the query information storage area is determined based on the storage sequence. For example, assuming the modulo result of the first hash value and the total length of the query information storage area is 2, the query information corresponding to the structure data can be stored in the second storage sequence within the query information storage area; assuming the modulo result of the first hash value and the total length of the query information storage area is 5, the query information corresponding to the structure data needs to be stored in the fifth storage sequence within the query information storage area. Through this method, the storage location of the query information corresponding to each structure data in the query information storage area can be determined.

[0080] Furthermore, after performing a modulo operation on the first hash value and the total length of the query information storage area as described above, and obtaining the modulo result, the method further includes:

[0081] If the remainder result is the same as the remainder result calculated previously, the other storage sequences after the storage sequence are traversed in turn until the first storage sequence that has not yet stored the query information is traversed, and the second location area of ​​the query information storage area is determined according to the traversed storage sequence.

[0082] While using a pre-defined hash algorithm can effectively reduce the probability of different data structures having the same first hash value, there is still a very small chance that the first hash value will overlap with the remainder of the total length of the query information storage area. For example, suppose the first hash value of the target field of structure data 1 is 3, the first hash value of the target field of structure data 2 is 8, and the total length of the query information storage area is 5. In this case, the remainder of both structure data 1 and structure data 2 will be 3.

[0083] In one embodiment, when the remainder result obtained in the current calculation is the same as the remainder result obtained in the previous calculation, it indicates that the storage sequence corresponding to the remainder result has already stored the query information. At this time, other storage sequences located after this storage sequence can be traversed sequentially until the first storage sequence that has not yet stored the query information is reached. The second location area of ​​the query information storage area is then determined based on the traversed storage sequence. For example, if the remainder result corresponding to a certain structure data is 2, and the second storage sequence of the query information storage area has already stored the query information of the previous structure data, then the traversal can proceed to the third storage sequence. If the third storage sequence is found to have already stored the query information, then the traversal can continue to the fourth storage sequence. If the fourth storage sequence is found to have not yet stored the query information, then the query information corresponding to the structure data can be stored in the fourth storage sequence. In this way, the problem of overwriting previously stored query information when the storage areas of the calculated query information overlap can be effectively avoided.

[0084] Furthermore, the location information of the first location area of ​​the data storage area includes the offset of the starting position of the first location area of ​​the data storage area relative to the starting position of the data storage area and the data length of the structure data.

[0085] In one embodiment, the location information of the first location area of ​​the data storage area may include the offset of the starting position of the first location area of ​​the data storage area relative to the starting position of the data storage area and the data length of the structure data. Since the length of each structure data is not fixed, the offset of each structure data in the data storage area and the data length of each structure data can be stored as query information, so that when querying a certain structure data in the future, the structure data can be quickly retrieved from the data storage area.

[0086] Further, step 102 above, obtaining the query information stored in the second location area of ​​the query information storage area, and querying the structure data from the first location area of ​​the data storage area based on the query information, specifically includes:

[0087] Calculate a first hash value and at least one second hash value corresponding to the target field, wherein the first hash value is calculated based on a preset hash algorithm to obtain the value of the target field, and at least one second hash value is calculated based on a preset hash algorithm to obtain at least one intermediate value, and at least one intermediate value is obtained by processing the value of the target field using at least one preset algorithm;

[0088] The second location region of the query information storage area is determined based on the first hash value and at least one second hash value;

[0089] Retrieve the location information of the first location area of ​​the data storage area stored in the second location area of ​​the query information storage area;

[0090] Based on the location information of the first location area of ​​the data storage area, the structure data is retrieved from the data storage area.

[0091] In one embodiment, when querying structure data, a first hash value and at least one second hash value corresponding to the target field of the structure data can be calculated first. In this way, the query information corresponding to the structure data can be determined in the second location area of ​​the query information storage area based on the first hash value and at least one second hash value. Then, the location information of the first location area of ​​the data storage area where the second location area of ​​the query information storage area is stored can be obtained. Finally, the structure data can be retrieved from the data storage area based on the location information of the first location area of ​​the data storage area.

[0092] It should be noted that both the first hash value and at least one second hash value are calculated using a preset hash algorithm. Compared to using Cyclic Redundancy Check (CRC) or other algorithms, hash values ​​calculated using the preset hash algorithm have greater randomness, thus significantly reducing the probability that different structured data will yield the same value. Specifically, when calculating the first hash value, the preset hash algorithm is directly used to calculate the value of the target field. When calculating the second hash value, the preset algorithm is first used to calculate the value of the target field, such as adding, subtracting, multiplying, or dividing the target field value by a certain value to obtain at least one intermediate value. Then, the preset hash algorithm is used to calculate at least one intermediate value to obtain at least one second hash value. The number of these second hash values ​​can be any integer, such as 1, 2, or 3. These second hash values ​​can be stored as query information in the second location area of ​​the query information storage area to verify the accuracy of the storage location of the query information corresponding to the structured data. Furthermore, when there are multiple values ​​corresponding to the target field of the same structured data, and they are not unique, the value at the default position of the target field can be selected for hash calculation, such as selecting the first or second value of the target field. In this way, when performing hash calculation on the target field of multiple structured data, a unique value can be selected for hash calculation.

[0093] In this embodiment, when querying the structure data, the location information of the structure data in the first location area of ​​the data storage area can be obtained from the query information storage area, so as to realize the fast query of the structure data and solve the problem that the traditional structure data processing method takes a long time during the query process.

[0094] Further, the above steps, including determining the second location region of the query information storage area based on the first hash value and at least one second hash value, include:

[0095] The first hash value and the total length of the query information storage area are moduloed to obtain the modulo result. The total length of the query information storage area is obtained by rounding up the product of the number of query information that the query information storage area is allowed to store and a preset threshold. The preset threshold is any value greater than 1.

[0096] The remainder result is used as the storage sequence for query information in the query information storage area;

[0097] Compare at least one second hash value with at least one second hash value stored in the storage sequence;

[0098] If at least one second hash value matches at least one second hash value stored in the storage sequence, the second location area of ​​the query information storage area is determined according to the storage sequence;

[0099] If at least one second hash value is inconsistent with at least one second hash value stored in the storage sequence, the other storage sequences located after the storage sequence are traversed sequentially until at least one second hash value in the storage sequence is consistent with at least one second hash value, and the second location area of ​​the query information storage area is determined according to the traversed storage sequence.

[0100] It should be noted that the total length of the query information storage area is obtained by rounding up the product of the number of query information items allowed to be stored in the query information storage area and a preset threshold. This preset threshold can be any value greater than 1. In an optional embodiment, the preset threshold can be 1.25. Assuming that the number of query information items allowed to be stored is 3, the result of 3 * 1.25 rounded up is 4. Therefore, the total length of the query information storage area is the sum of the lengths of the 4 query information items, meaning there are 4 storage sequences in the query information storage area, and each storage sequence is used to store one query information item. In other words, before storing query information, the total length of the query information storage area needs to be determined based on the number of query information items to be stored and the preset threshold. This allows the location area of ​​each query information item in the query information storage area to be determined based on its length.

[0101] In one embodiment, when determining the second location region of the query information storage area based on the first hash value and at least one second hash value, a modulo operation can be performed on the first hash value and the total length of the query information storage area to obtain the modulo result. This modulo result is then used as the storage sequence of the query information in the query information storage area. The calculated at least one second hash value is then compared with at least one second hash value stored in this storage sequence. If the calculated at least one second hash value matches the stored at least one second hash value, it indicates that the query information stored in this storage sequence is indeed the query information for the structure data. Therefore, the second location region of the query information for the structure data in the query information storage area can be determined based on this storage sequence. If the calculated at least one second hash value does not match the stored at least one second hash value, it indicates that the query information stored in this storage sequence is not the query information for the structure data. In this case, it is necessary to traverse the other storage sequences following this one until at least one second hash value in the storage sequence matches the calculated at least one second hash value. The second location region of the query information storage area is then determined based on the traversed storage sequence. Using the above method, the storage location of the query information corresponding to each structure data in the query information storage area can be accurately and quickly determined based on the first hash value and at least one second hash value.

[0102] Furthermore, based on the location information of the first location area of ​​the data storage area, the structure data is retrieved from the data storage area, including:

[0103] Based on the location information of the first location area of ​​the data storage area, determine the offset of the starting position of the first location area of ​​the data storage area relative to the starting position of the data storage area and the data length of the structure data;

[0104] The structure data is retrieved from the data storage area by using the offset of the starting position of the first location area relative to the starting position of the data storage area and the data length of the structure data.

[0105] In one embodiment, after obtaining the location information of the first location area of ​​the data storage area from the query information, the offset of the starting position of the first location area of ​​the data storage area relative to the starting position of the data storage area and the data length of the structure data can be determined based on the location information of the first location area of ​​the data storage area, so that the structure data can be quickly retrieved from the data storage area.

[0106] Furthermore, the method also includes:

[0107] When other devices need to use the structure data, the data storage area and query information storage area are packaged and transmitted to the other devices.

[0108] In one embodiment, when other devices need to use the structure data stored on this device, the data storage area containing the structure data and the query information storage area containing the query information can be packaged together and transmitted to the other devices. This allows the other devices to parse the data storage area and the query information storage area, find and use the structure data stored in the data storage area, thereby simplifying the transmission method of the structure data.

[0109] In one example, the method for processing structure data provided in this application embodiment is based on a hash algorithm, which requires the creation of two memory areas: one for storing query information (equivalent to the query information storage area mentioned above), and the other for storing structure data (equivalent to the data storage area mentioned above).

[0110] When storing structured data, hash algorithms can effectively reduce key collisions. A value from the target field can be found within the structured data as the key. A pre-defined hash algorithm is then used to calculate the first hash value (hash1) of the key. Next, a modulo operation is performed between hash1 and the total length of the query information storage area. Based on the modulo result, the location of the query information for the structured data in the second position area of ​​the query information storage area is determined. Furthermore, the key is added to and subtracted by a certain value, and then hashed again to obtain two second hash values, hash2 and hash3. Finally, hash2, hash3, the offset of the structured data in the data storage area, and the length of the structured data are used as the query information and stored in the second position area of ​​the query information storage area.

[0111] Specifically, when storing structure data in the data storage area, its relative offset and data length within the data storage area can be obtained. Then, the three hash values ​​hash1, hash2, and hash3 of the target field in the structure data are calculated. The modulo operation of hash1 with the total length of the query information storage area determines the second location area of ​​the query information storage area where the query information is stored. Finally, the calculated hash2, hash3, the offset of the structure data in the data storage area, and the data length of the structure data are stored in this second location area, as shown in Figure 2.

[0112] When querying structure data, the three hash values ​​hash1, hash2, and hash3 corresponding to the target field in the structure data are calculated. The modulo operation of hash1 with the total length of the query information storage area is used to obtain the storage sequence number in the query information storage area. The hash2 and hash3 are then compared with the hash2 and hash3 corresponding to the storage sequence number in the query information storage area. If they match, it means that the query information stored in the storage sequence number in the query information storage area is correct. The offset of the structure data in the data storage area and the data length of the structure data can be determined from the query information, thus allowing the structure data to be retrieved from the data storage area.

[0113] Referring to Figure 3, which is a schematic diagram of a structure data processing device provided in an embodiment of this application, the device 300 includes:

[0114] The storage module 301 is used to store the structure data in the first location area of ​​the data storage area and store the query information corresponding to the structure data in the second location area of ​​the query information storage area when storing structure data.

[0115] The query module 302 is used to obtain the query information stored in the second location area of ​​the query information storage area when querying the structure data, and to retrieve the structure data from the first location area of ​​the data storage area based on the query information.

[0116] The data storage area and the query information storage area are two independent storage areas on the same device. The query information includes the hash value obtained based on the target field in the structure data and the location information corresponding to the first location area of ​​the data storage area. The target field is a field in the structure data whose value does not repeat the same field in other structure data.

[0117] Furthermore, the storage module 301 includes:

[0118] The first calculation submodule is used to calculate a first hash value and at least one second hash value corresponding to the target field. The first hash value is calculated based on a preset hash algorithm to obtain the value of the target field. The at least one second hash value is calculated based on a preset hash algorithm to obtain at least one intermediate value. The at least one intermediate value is obtained by processing the value of the target field using at least one preset algorithm.

[0119] The first determining submodule is used to determine the second location area of ​​the query information storage area where the query information is stored, based on the first hash value;

[0120] The storage submodule is used to store the structure data in the first location area of ​​the data storage area, and store the location information of the first location area of ​​the data storage area and at least one second hash value as query information in the second location area of ​​the query information storage area.

[0121] Furthermore, the first determining submodule includes:

[0122] The first modulo operation unit is used to perform a modulo operation on the first hash value and the total length of the query information storage area to obtain the modulo result. The total length of the query information storage area is obtained by rounding up the product of the number of query information that the query information storage area is allowed to store and a preset threshold. The preset threshold is any value greater than 1.

[0123] The first unit is used to store the remainder result as a sequence of query information in the query information storage area, with each storage sequence corresponding to one query information.

[0124] The first determining unit is used to determine the second location area of ​​the query information storage area based on the storage sequence.

[0125] Furthermore, the first determining submodule also includes:

[0126] The first traversal unit is used to traverse other storage sequences located after the storage sequence in turn when the remainder result is the same as the remainder result calculated previously, until the first storage sequence that has not yet stored the query information is traversed, and to determine the second location area of ​​the query information storage area based on the traversed storage sequence.

[0127] Furthermore, the location information of the first location area of ​​the data storage area includes the offset of the starting position of the first location area of ​​the data storage area relative to the starting position of the data storage area and the data length of the structure data.

[0128] Furthermore, the query module 302 includes:

[0129] The second calculation submodule is used to calculate the first hash value and at least one second hash value corresponding to the target field. The first hash value is calculated based on the value of the target field using a preset hash algorithm, and the at least one second hash value is calculated based on at least one intermediate value using a preset hash algorithm. The at least one intermediate value is obtained by processing the value of the target field using at least one preset algorithm.

[0130] The second determining submodule is used to determine the second location area of ​​the query information storage area based on the first hash value and at least one second hash value;

[0131] The acquisition submodule is used to acquire the location information of the first location area of ​​the data storage area stored in the second location area of ​​the query information storage area;

[0132] The query submodule is used to retrieve structure data from the data storage area based on the location information of the first location area of ​​the data storage area.

[0133] Furthermore, the second determining submodule includes:

[0134] The second modulo operation unit is used to perform a modulo operation on the first hash value and the total length of the query information storage area to obtain the modulo result. The total length of the query information storage area is obtained by rounding up the product of the number of query information that the query information storage area is allowed to store and a preset threshold. The preset threshold is any value greater than 1.

[0135] The second unit is used to store the remainder result as the sequence of query information in the query information storage area.

[0136] A comparison unit is used to compare at least one second hash value with at least one second hash value stored in the storage sequence;

[0137] The second determining unit is used to determine the second location area of ​​the query information storage area according to the storage sequence when at least one second hash value is consistent with at least one second hash value stored in the storage sequence.

[0138] The second traversal unit is used to traverse other storage sequences located after the storage sequence in turn when at least one second hash value is inconsistent with at least one second hash value stored in the storage sequence, until at least one second hash value in the storage sequence is consistent with at least one second hash value, and to determine the second location area of ​​the query information storage area based on the traversed storage sequence.

[0139] Furthermore, the query submodule includes:

[0140] The third determining unit is used to determine the offset of the starting position of the first location area of ​​the data storage area relative to the starting position of the data storage area and the data length of the structure data based on the position information of the first location area of ​​the data storage area.

[0141] The query unit is used to retrieve the structure data from the data storage area by using the offset of the starting position of the first location area of ​​the data storage area relative to the starting position of the data storage area and the data length of the structure data.

[0142] Furthermore, the device 300 also includes:

[0143] The transmission module is used to package and transmit the data storage area and query information storage area to other devices when other devices need to use the structure data.

[0144] It should be noted that the structure data processing device 300 can implement the steps of the structure data processing method provided in any of the aforementioned method embodiments, and can achieve the same technical effect, which will not be described in detail here.

[0145] As shown in Figure 4, this application embodiment also provides an electronic device, including a processor 411, a communication interface 412, a memory 413, and a communication bus 414, wherein the processor 411, the communication interface 412, and the memory 413 communicate with each other through the communication bus 414.

[0146] Memory 413 is used to store computer programs;

[0147] In one embodiment of this application, when the processor 411 executes a program stored in the memory 413, it implements the method for processing structure data provided in any of the foregoing method embodiments, including:

[0148] When storing structure data, the structure data is stored in the first location area of ​​the data storage area, and the query information corresponding to the structure data is stored in the second location area of ​​the query information storage area.

[0149] When querying structure data, the query information stored in the second location area of ​​the query information storage area is obtained, and the structure data is retrieved from the first location area of ​​the data storage area based on the query information.

[0150] The data storage area and the query information storage area are two independent storage areas on the same device. The query information includes the hash value obtained based on the target field in the structure data and the location information corresponding to the first location area of ​​the data storage area. The target field is a field in the structure data whose value does not repeat the same field in other structure data.

[0151] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the structure data processing method provided in any of the foregoing method embodiments.

[0152] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0153] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for processing structured data, characterized in that, The method includes: when storing structure data, storing the structure data in a first location area of ​​a data storage area, and storing query information corresponding to the structure data in a second location area of ​​a query information storage area; when querying the structure data, obtaining the query information stored in the second location area of ​​the query information storage area, and retrieving the structure data from the first location area of ​​the data storage area based on the query information; wherein the data storage area and the query information storage area are two independent storage areas on the same device, and the query information includes a hash value obtained based on a target field in the structure data and location information corresponding to the first location area of ​​the data storage area, wherein the target field is a field in the structure data whose value does not repeat with the same field in other structure data; wherein storing the structure data in a first location area of ​​a data storage area... The first location area of ​​the data storage area, and storing the query information corresponding to the structure data in the second location area of ​​the query information storage area, includes: calculating a first hash value and at least one second hash value corresponding to the target field, wherein the first hash value is calculated based on a preset hash algorithm to calculate the value of the target field, and the at least one second hash value is calculated based on a preset hash algorithm to calculate at least one intermediate value, and the at least one intermediate value is obtained by processing the value of the target field using at least one preset algorithm; determining the second location area of ​​the query information storage area for storing the query information based on the first hash value; storing the structure data in the first location area of ​​the data storage area, and storing the location information of the first location area of ​​the data storage area and the at least one second hash value as the query information in the second location area of ​​the query information storage area.

2. The method according to claim 1, characterized in that, The step of determining the second location region of the query information storage area for storing the query information based on the first hash value includes: performing a modulo operation on the first hash value and the total length of the query information storage area to obtain a modulo result, wherein the total length of the query information storage area is determined by rounding up the product of the number of query information to be stored and a preset threshold, and the length of each query information, wherein the preset threshold is any value greater than 1; using the modulo result as the storage sequence of the query information in the query information storage area, wherein each storage sequence corresponds to one query information; and determining the second location region of the query information storage area based on the storage sequence.

3. The method according to claim 2, characterized in that, After performing a modulo operation on the first hash value and the total length of the query information storage area to obtain the modulo result, the method further includes: if the modulo result is the same as the previously calculated modulo result, sequentially traversing other storage sequences located after the storage sequence until the first storage sequence that has not yet stored query information is traversed, and determining the second location area of ​​the query information storage area based on the traversed storage sequence.

4. The method according to claim 1, characterized in that, The location information of the first location area of ​​the data storage area includes the offset of the starting position of the first location area of ​​the data storage area relative to the starting position of the data storage area and the data length of the structure data.

5. The method according to claim 1, characterized in that, The step of obtaining the query information stored in the second location area of ​​the query information storage area and retrieving the structure data from the first location area of ​​the data storage area based on the query information includes: calculating a first hash value and at least one second hash value corresponding to the target field, wherein the first hash value is calculated based on a preset hash algorithm to calculate the value of the target field, and the at least one second hash value is calculated based on a preset hash algorithm to calculate at least one intermediate value, and the at least one intermediate value is obtained by processing the value of the target field using at least one preset algorithm; determining the second location area of ​​the query information storage area based on the first hash value and the at least one second hash value; obtaining the location information of the first location area of ​​the data storage area stored in the second location area of ​​the query information storage area; and retrieving the structure data from the data storage area based on the location information of the first location area of ​​the data storage area.

6. The method according to claim 5, characterized in that, The step of determining the second location region of the query information storage area based on the first hash value and the at least one second hash value includes: performing a modulo operation on the first hash value and the total length of the query information storage area to obtain a modulo result, wherein the total length of the query information storage area is determined by rounding up the product of the number of query information to be stored and a preset threshold, and the length of each query information, wherein the preset threshold is any value greater than 1; using the modulo result as the storage sequence of the query information in the query information storage area; comparing the at least one second hash value with at least one second hash value stored in the storage sequence; if the at least one second hash value matches the at least one second hash value stored in the storage sequence, determining the second location region of the query information storage area based on the storage sequence; if the at least one second hash value does not match the at least one second hash value stored in the storage sequence, sequentially traversing other storage sequences following the storage sequence until at least one second hash value in the storage sequence matches the at least one second hash value, and determining the second location region of the query information storage area based on the traversed storage sequence.

7. The method according to claim 5, characterized in that, The step of retrieving the structure data from the data storage area based on the location information of the first location area of ​​the data storage area includes: determining the offset of the starting position of the first location area of ​​the data storage area relative to the starting position of the data storage area and the data length of the structure data based on the location information of the first location area of ​​the data storage area; and retrieving the structure data from the data storage area based on the offset of the starting position of the first location area of ​​the data storage area relative to the starting position of the data storage area and the data length of the structure data.

8. The method according to claim 1, characterized in that, The method further includes: when other devices need to use the structure data, packaging and transmitting the data storage area and the query information storage area to the other devices.

9. A device for processing structured data, characterized in that, The device includes: a storage module, configured to, when storing structure data, store the structure data in a first location area of ​​a data storage area and store query information corresponding to the structure data in a second location area of ​​a query information storage area; and a query module, configured to, when querying the structure data, obtain the query information stored in the second location area of ​​the query information storage area and, based on the query information, retrieve the structure data from the first location area of ​​the data storage area; wherein the data storage area and the query information storage area are two independent storage areas on the same device, and the query information includes a hash value obtained based on a target field in the structure data and location information corresponding to the first location area of ​​the data storage area, wherein the target field is a value in the structure data that is not repeated from the value corresponding to the same field in other structure data. The storage module includes: a first calculation submodule, configured to calculate a first hash value and at least one second hash value corresponding to the target field, wherein the first hash value is calculated based on a preset hash algorithm to obtain the value of the target field, and the at least one second hash value is calculated based on a preset hash algorithm to obtain at least one intermediate value, wherein the at least one intermediate value is obtained by processing the value of the target field using at least one preset algorithm; a first determination submodule, configured to determine a second location area of ​​the query information storage area for storing the query information based on the first hash value; and a storage submodule, configured to store the structure data in a first location area of ​​the data storage area, and store the location information of the first location area of ​​the data storage area and the at least one second hash value as the query information in the second location area of ​​the query information storage area.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; and the processor, when executing the program stored in the memory, implements the steps of the method for processing structure data as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method for processing structure data according to any one of claims 1-8.

Images