Cloud edge collaborative data storage and retrieval architecture for industrial scene

Abstract

For scenes of data distributed storage and cross-domain circulation use in each business domain of an industrial scene, the invention provides a cloud-edge collaborative data storage and retrieval architecture aiming at the problems that business systems are many and complex, the data volume is huge and part of data cannot be clouded, and aims to realize unified storage and cross-domain efficient circulation of large-scale data. According to the architecture, a data coding rule based on an RDF graph model and a data multi-level efficient storage strategy based on an S-tree are designed, it is guaranteed that cloud-unavailable data are stored in an edge side server, cloud-available data are stored in a cloud server, and an index tree efficient collaborative retrieval algorithm (CECI-tree) oriented to cloud edge collaborative storage is provided. The efficiency of data retrieval is effectively improved through a concurrent retrieval mechanism and a cloud edge collaborative indexing mechanism.

Description

Technical Field

[0001] The present invention can be applied to the fields of distributed graph data retrieval, intelligent manufacturing, etc., and specifically proposes a cloud-edge collaborative data storage and retrieval architecture for industrial scenarios. This architecture targets the problems of numerous and complex business systems, huge amounts of data, and some data being unable to be uploaded to the cloud. It has great advantages over existing storage methods for collaborative data storage and management. Background Art

[0002] The current mainstream distributed storage and retrieval methods are divided into two types: cloud-based methods and partition-based methods. The cloud-based method uses existing cloud computing platforms, such as Hadoop and Trinity. The core design of Hadoop's architecture is: HDFS and MapReduce. RDF data is stored using HDFS, and MapReduce provides computing for RDF data. Many existing distributed RDF data management methods [7-10] also use HDFS for storage and MapReduce for query processing. The cloud-based method takes into account the elasticity and scalability of the cloud computing platform, as well as the need to improve storage and retrieval efficiency through distributed processing. It realizes the storage and query of large-scale RDF graph data, and has certain scalability and fault tolerance.

[0003] In contrast, the partition-based approach first divides the RDF data graph into several subgraphs, and then distributes these subgraphs to different computing nodes. Each computing node is equipped with a stand-alone RDF data management system to manage the assigned RDF data graph subgraphs. When a SPARQL query is input into these systems, the query is first divided into several subqueries, which are then also distributed to various computing nodes for execution to obtain partial solutions. Finally, all partial solutions are collected and connected to obtain the final solution. The partition-based approach considers the need to divide the RDF graph into multiple subgraphs and use distributed database technology for query processing to achieve distributed storage and query of RDF data, which can better utilize distributed computing resources. However, while meeting data storage requirements, ensuring the effective use of computing resources still faces huge challenges. Summary of the invention

[0004] The core of this invention is to propose a cloud-edge collaborative data storage and retrieval architecture. The following will describe the CECI-tree method based on Hash coding. In order to effectively manage data, cloud-edge collaborative retrieval is also designed.

[0005] In the cloud-edge collaborative data storage and retrieval architecture, the RDF dataset is first represented as an RDF graph and stored using an adjacency list. Each entity and non-entity node in the graph is encoded as a bit string, and the node code and edge code in the adjacency list are used to obtain the node label according to the "OR" operation. Subsequently, according to the structure of the RDF graph, these node labels are connected to form a data label graph G*, where each node corresponds to an entity node in the RDF graph.

[0006] For the incoming SPARQL query, it is also encoded as a query graph Q and converted into a query label graph Q*. However, since the matching of the query graph Q* on the data label graph G* is an NP-hard problem, similar to graph homomorphism, the architecture uses S-tree to store vertex labels to reduce the search complexity of matching. This strategy improves the efficiency of local matching and ensures good performance on large-scale datasets.

[0007] The architecture also takes into account the shortcomings of traditional data storage and retrieval methods, and designs CECI-tree on the cloud side to solve the problem of collaborative storage of cloud-edge data. CECI-tree is a new index structure that combines the storage resources of the cloud side and the edge side, and performs "selective" storage of edge-side indexes. The design of CECI-tree realizes distributed storage and management of data, fully utilizes the computing resources of the cloud side and the edge side, and improves data retrieval efficiency and computing utilization. Compared with traditional distributed architectures, CECI-tree can better adapt to the needs of distributed storage and retrieval, and realizes distributed storage and collaborative management of data. The following is the definition of CECI-tree:

[0008] The height of the index tree on the edge side is h, and the node code set of each layer is represented by C1, C2, ....Ch. For each adjacent two-layer node code Ci and Ci+1, the difference value is defined as Di, which is formally expressed as: Where m represents the number of nodes in each layer, x, x child Represents the node x in Ci and the node x in Ci+1 respectively, and the symbol represents the bitwise XOR operation. The maximum difference value Di between all adjacent layers is stored in the array M = [M1, M2, .... M n-1 ]middle.

[0009] For the cloud-edge collaborative environment, a collaborative algorithm LPM based on graph homomorphism is designed. The following is the specific definition of the algorithm:

[0010] Given a set of m vertices {v1, ..., v m} of a SPARQL query graph Q and a partition Gc / i with n vertices {u1, ..., u n}(n≤m), an LPM fragment of a connected subgraph. An LPM is a local partial matching if and only if there exists a function f: {v1,...,v m}→{u1,...,u n}∪{NULL} holds the following conditions:

[0011] (1)v i Not a parameter, v i =f(v i ); if v i is the parameter f(v i )∈{u1,...u m}; or f(v i )=NULL;

[0012] (2) Existence in Q LPM must meet one of the following conditions:

[0013] a. In LPM, there is an edge with attribute p With edge have the same property p, or is a parameter;

[0014] b. Does not exist in LPM But f(v i ) or f(v j ) belongs to the boundary node;

[0015] cf(v i )=NULLorf(v j )=NULL;

[0016] (3) It contains at least one boundary edge, ensuring that the above empty matching meets the conditions.

[0017] (4) If (internal node), there is an edge in Q Then f(v i )≠NULL, and If the edge The edge attribute is p, then the edge Also has the same property p.

[0018] Compared with the prior art, the advantages of the present invention are as follows:

[0019] (1) In a complex industrial environment, the present invention can isolate and store data, perform collaborative management, and reduce CPU occupancy while improving concurrency.

[0020] (2) The present invention has good scalability and can dynamically adjust data storage in the face of ever-increasing data.

[0021] (3) The application fields of the present invention include but are not limited to the field of distributed graph data storage and the field of intelligent manufacturing, and the application prospects are broad. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. Among them:

[0023] Figure 1 This is a diagram of the cloud-edge collaborative data storage and retrieval architecture in the present invention;

[0024] Figure 2 It is the data storage model graph (RDF graph) in the present invention;

[0025] Figure 3 This is a diagram of a coding technology case in the present invention (coding technology);

[0026] Figure 4 This is the data graph G* case in the present invention (data graph G*).

[0027] Figure 5 This is a node label case diagram obtained by the "OR" operation in the present invention (the "OR" operation of node labels);

[0028] Figure 6 is the CECI-tree case diagram (CECI-tree) in the present invention; DETAILED DESCRIPTION

[0029] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and do not limit the present invention.

[0030] The present invention introduces the specific implementation mode of the present invention based on the cloud-edge collaborative data storage and retrieval architecture case.

[0031] like Figure 1-6 As shown in the figure, the triple table is stored using the RDF data model. After the local index is established locally on the cloud edge, a cloud-edge collaborative index CECI-tree is built on the cloud side. The retrieval request is made from the cloud side. When the CECI-tree is retrieved, a retrieval request is sent to the corresponding edge computing node. The edge side retrieves the corresponding data based on the retrieval request, and finally merges the result with the cloud side data and returns it to the user.

[0032] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A hash encoding method, the characteristics of which include the following: Data encoding method: RDF data is stored in a triple table, using M bits for nodes and N bits for edges. A hash function is used to hash the corresponding attribute values ​​of nodes and edges, and "1" is assigned to the corresponding bit position. Adjacency list: The codes of nodes and edges are stored using an adjacency list. The code obtained by the "OR" operation is called a node label. The edge code between each node label uses the original code. After the "OR" operation on the code, the G* graph composed of the node labels is obtained.

2. According to the coding technology method shown in claim 1, in order to reduce the search space, an S-tree tree is used to store node labels, and the specific features are as follows: According to the coding characteristics of the node signature, the insertion position of the vertex label is determined according to the similarity between them. When inserting a label, an evaluation is performed based on the difference between the encodings of each node. Each time a label is inserted, the node to be inserted is compared with the leaf node, and the number of bits in their encodings at the same position that are the same is recorded, and then the leaf node with the greatest similarity is selected for insertion. Each time a new leaf node is inserted, the entire tree will continue to recurse to modify the parent node encoding. When two identical similarities appear, the proximity principle is followed.

3. According to claim 2, index storage is performed in a distributed environment, and a CECI-tree is constructed, which has the following characteristics: data is selectively stored according to the characteristics of data encoding, a portion of the edge-side index is stored on the cloud side, and the difference values ​​between the encodings of each two layers of nodes and their child nodes in the entire index tree on the edge side are compared, and these difference values ​​are stored in an array. Then, by analyzing the array, it can be determined which two layers in the entire index tree have the largest difference value. The two layers with the largest difference are determined, and the layer with the largest difference and the index tree above it are stored on the cloud side.

4. According to right 3, a candidate set is obtained, and a local connected subgraph LPM is constructed by matching the same nodes in the triples. An LPM pruning method is designed based on the graph homomorphic matching method, and the data results are merged and returned.

Images