Hierarchical data storage and loading method based on DAG

Abstract

The invention relates to a hierarchical data storage and loading method based on a DAG, and the method comprises the following steps: S1, constructing a DAG hierarchical data storage model, and storing hierarchical data; s2, traversing data in the DAG hierarchical data storage model, and loading the hierarchical data by adopting a depth-first recursive algorithm; and S3, performing index extraction of information through a relation link between hierarchical data. According to the method, the problem of hierarchical data storage and loading is effectively solved, and the method has the advantages of high data storage efficiency and high loading speed.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage and loading, and in particular to a DAG-based hierarchical data storage and loading method. Background Art

[0002] Under the current circumstances, most data storage and loading methods rely on traditional flat structures. At the same time, there are also some methods that use tree structures or linked list structures to implement data storage and loading. However, when these methods deal with scenarios where a large amount of data is loaded, they usually adopt loop nesting, which to a large extent leads to high occupation of system resources and increased database access pressure. In this context, in order to ensure efficient and stable data processing, it is urgent to propose a more efficient data storage and loading method that can also solve the current problems. Summary of the invention

[0003] In order to solve the technical problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a DAG-based hierarchical data storage and loading method, which is used to solve the problems of high system resource occupancy and high database access pressure in hierarchical data structure loading.

[0004] To achieve the above-mentioned object of the invention, the present invention provides a DAG-based hierarchical data storage and loading method, comprising the following steps:

[0005] Step S1: construct a DAG hierarchical data storage model and store hierarchical data;

[0006] Step S2, traversing the data in the DAG hierarchical data storage model, and using a depth-first recursive algorithm to load the hierarchical data;

[0007] Step S3: extract information indexes through relational links between hierarchical data.

[0008] According to a technical solution of the present invention, in step S1, it specifically includes:

[0009] Step S11, constructing a node to represent the data hierarchy;

[0010] Step S12: constructing edges to connect the nodes according to the hierarchical relationship between the hierarchical data to form the DAG hierarchical data storage model;

[0011] Step S13, storing data attribute information of each node of the DAG hierarchical data storage model;

[0012] Step S14: maintaining the relationship between each node and its corresponding parent node according to the DAG hierarchical data storage model, and establishing a hierarchical data table, wherein the hierarchical data table contains the corresponding relationship between all parent nodes and child nodes in the DAG hierarchical data storage model;

[0013] Step S15: Add, delete, modify, query and store the data attribute information of the DAG hierarchical data storage model.

[0014] According to a technical solution of the present invention, the data attribute information includes data name, data number, data type and description.

[0015] According to a technical solution of the present invention, in the step S2, it specifically includes:

[0016] Step S21, construct a temporary T table, and transfer the hierarchical data table to the temporary T table; the temporary T table includes a parent node column and a child node column, and the child node column contains child nodes corresponding to the nodes in the parent node column;

[0017] Step S22, counting the number of nodes in the temporary T table to obtain data record t1;

[0018] Step S23, determine whether the data record exists t1=0; if so, end, otherwise, execute step S23;

[0019] Step S24, traverse the temporary T table, obtain the hierarchical data corresponding to the nodes in the temporary T table that are in the parent node column but not in the child node column, and insert them into the result set;

[0020] Step S25, deleting the node in the parent node column whose hierarchical data is inserted into the result set and its corresponding child node column;

[0021] Step S26, execute step S22.

[0022] According to a technical solution of the present invention, in the step S24, it also includes:

[0023] The hierarchical data of the node is inserted into a result set, and an index is added to the hierarchical data inserted into the result set.

[0024] According to one aspect of the present invention, an electronic device is provided, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory, and when the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes a DAG-based hierarchical data storage and loading method as described in any one of the above technical solutions.

[0025] According to one aspect of the present invention, a computer-readable storage medium is provided for storing computer instructions. When the computer instructions are executed by a processor, a DAG-based hierarchical data storage and loading method as described in any one of the above technical solutions is implemented.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] The present invention proposes a DAG-based hierarchical data storage and loading method, which realizes the storage of hierarchical data by constructing a DAG model, then loads the data by adopting a depth-first traversal method, and finally extracts the index of information through the relationship links between the hierarchical data, effectively solving the problem of hierarchical data storage and loading, and has the advantages of high data storage efficiency and fast loading speed.

[0028] The present invention can meet the needs of big data processing and has important practical value for efficient storage and loading of data. It can process massive cloud data as well as complex web page data and has excellent practicality and versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 Schematically represents a directed acyclic graph;

[0031] Figure 2 A schematic diagram schematically shows the structure of hierarchical data according to an embodiment of the present invention;

[0032] Figure 3 Schematically represents a directed acyclic graph of hierarchical data according to one embodiment of the present invention;

[0033] Figure 4 The flowchart schematically shows a DAG-based hierarchical data storage and loading method provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0034] The description of the embodiments of this specification should be combined with the corresponding drawings, which should be considered as part of the complete specification. In the drawings, the shape or thickness of the embodiments may be enlarged and indicated for simplification or convenience. Furthermore, the parts of each structure in the drawings will be described separately. It is worth noting that the elements not shown in the drawings or not described in words are in a form known to ordinary technicians in the relevant technical field.

[0035] The description of the embodiments herein and any reference to directions and orientations are only for the convenience of description and are not to be construed as any limitation on the scope of protection of the present invention. The following description of the preferred embodiments may involve combinations of features, which may exist independently or in combination, and the present invention is not particularly limited to the preferred embodiments. The scope of the present invention is defined by the claims.

[0036] like Figure 1 As shown, the present invention provides a DAG-based hierarchical data storage and loading method, comprising the following steps:

[0037] Step S1: construct a DAG hierarchical data storage model and store hierarchical data;

[0038] In step S1, it specifically includes:

[0039] Step S11, constructing a node to represent the data hierarchy;

[0040] Step S12: construct edges to connect nodes according to the hierarchical relationship between hierarchical data to form a DAG hierarchical data storage model;

[0041] Step S13: storing data attribute information of each node of the DAG hierarchical data storage model;

[0042] Step S14: According to the DAG hierarchical data storage model, the relationship between each node and its corresponding parent node is maintained, and a hierarchical data table is established, which contains the corresponding relationship between all parent nodes and child nodes in the DAG hierarchical data storage model;

[0043] Step S15: Add, delete, modify, query and store the data attribute information of the DAG hierarchical data storage model.

[0044] Data attribute information also includes data name, data number, data type and description.

[0045] Step S2, traverse the data in the DAG hierarchical data storage model, and use a depth-first recursive algorithm to load hierarchical data;

[0046] In step S2, it specifically includes:

[0047] Step S21, construct a temporary T table, and transfer the hierarchical data table to the temporary T table; the temporary T table includes a parent node column and a child node column, and the child node column contains child nodes corresponding to the nodes in the parent node column;

[0048] Step S22, counting the number of node names in the temporary T table to obtain data record t1;

[0049] Step S23, determine whether the data record exists t1=0; if so, end, otherwise, execute step S23;

[0050] Step S24, traverse the temporary T table, obtain the hierarchical data corresponding to the nodes in the temporary T table that are in the parent node column but not in the child node column, and insert them into the result set;

[0051] Step S24 also includes: inserting the hierarchical data of the node into the result set, and adding an index to the hierarchical data inserted into the result set.

[0052] Step S25, deleting the node of the hierarchical data insertion result set in the parent node column and its corresponding child node column;

[0053] Step S26, execute step S22.

[0054] Step S3: extract information indexes through relational links between hierarchical data.

[0055] According to the index of the hierarchical data in the result set, information index extraction can be performed according to the relationship links between the hierarchical data, so as to facilitate the acquisition of the corresponding hierarchical data and improve the efficiency of obtaining the hierarchical data.

[0056] like Figure 1 As shown in the figure, in a directed acyclic graph (DAG), numbered nodes represent data, and the direction of arrows represents the relationship and connection between data. Data storage is achieved by constructing such a directed acyclic graph (DAG). When loading, data with fewer or no dependencies is loaded first according to the direction of the arrow, thereby improving the efficiency of data loading.

[0057] The present invention realizes the storage of hierarchical data by constructing a DAG model, and adopts a priority traversal method to load data, which has the advantages of high data storage efficiency and fast loading speed, and can accurately process hierarchical data. It has broad application prospects and can be widely used in big data processing, web page rendering, 3D modeling and other fields.

[0058] The present invention, through the topological sorting idea of ​​graph theory and the improved algorithm of DAG in preventing nesting errors in hierarchical data loading, can be seen that the algorithm avoids the shortcomings of multiple traversals and comparisons of the same node in the commonly used algorithm. Furthermore, in each loop process, the recorded node must be the top-level node, because in each loop, the deleted nodes are all nodes without predecessors, and the number of loops is reduced by one each time, which effectively reduces the complexity of code execution, saves computing time, and saves computer resources.

[0059] According to one aspect of the present invention, an electronic device is provided, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory, and when the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes a DAG-based hierarchical data storage and loading method as any one of the above technical solutions.

[0060] The processor may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0061] According to one aspect of the present invention, a computer-readable storage medium is provided for storing computer instructions. When the computer instructions are executed by a processor, a DAG-based hierarchical data storage and loading method is implemented as described in any one of the above technical solutions.

[0062] Computer-readable storage media may include any medium capable of storing or transmitting information. Examples of computer-readable storage media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments may be downloaded via computer networks such as the Internet, intranets, etc.

[0063] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0064] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0065] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0066] It should also be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0067] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for those skilled in the art, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A DAG-based hierarchical data storage and loading method, characterized in that: The following steps are involved: Step S1: construct a DAG hierarchical data storage model and store hierarchical data; Step S2, traversing the data in the DAG hierarchical data storage model, and using a depth-first recursive algorithm to load the hierarchical data; Step S3: extracting information indexes through the relationship links between the hierarchical data.

2. The DAG-based hierarchical data storage and loading method according to claim 1, characterized in that: In the step S1, it specifically includes: Step S11, constructing a node to represent the data hierarchy; Step S12: constructing edges to connect the nodes according to the hierarchical relationship between the hierarchical data to form the DAG hierarchical data storage model; Step S13, storing data attribute information of each node of the DAG hierarchical data storage model; Step S14: maintaining the relationship between each node and its corresponding parent node according to the DAG hierarchical data storage model, and establishing a hierarchical data table, wherein the hierarchical data table contains the corresponding relationship between all parent nodes and child nodes in the DAG hierarchical data storage model; Step S15: Add, delete, modify, query and store the data attribute information of the DAG hierarchical data storage model.

3. The DAG-based hierarchical data storage and loading method according to claim 2, characterized in that: The data attribute information includes data name, data number, data type and description.

4. The DAG-based hierarchical data storage and loading method according to claim 2, characterized in that: In the step S2, it specifically includes: Step S21, construct a temporary T table, and transfer the hierarchical data table to the temporary T table; the temporary T table includes a parent node column and a child node column, and the child node column contains child nodes corresponding to the nodes in the parent node column; Step S22, counting the number of nodes in the temporary T table to obtain data record t1; Step S23, determine whether the data record exists t1=0; if so, end, otherwise, execute step S23; Step S24, traverse the temporary T table, obtain the hierarchical data corresponding to the nodes in the temporary T table that are in the parent node column but not in the child node column, and insert them into the result set; Step S25, deleting the node in the parent node column whose hierarchical data is inserted into the result set and its corresponding child node column; Step S26, execute step S22.

5. The DAG-based hierarchical data storage and loading method according to claim 4, characterized in that: In the step S24, it also includes: The hierarchical data of the node is inserted into the result set, and an index is added to the hierarchical data inserted into the result set.

6. An electronic device, characterized in that: include: One or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes the DAG-based hierarchical data storage and loading method as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, implement the DAG-based hierarchical data storage and loading method as described in any one of claims 1 to 5.

Images