Data flow path tracing method and device, electronic equipment and readable storage medium

Abstract

The invention provides a data flow path tracing method and device, electronic equipment and a readable storage medium, and the method comprises the steps: enabling a system identifier of a previous node, a data copy unique identifier of the previous node, a system identifier of a current node and a data copy unique identifier of the current node to form tetrad information in a key value pair form; reporting the data to a distributed storage system, and determining a flow path of the data according to the tetrad information; compared with the prior art, a remarkable effect is achieved in the aspects of data flow path recording and multi-copy identification management, complex operations such as uploading, identification embedding and extraction are avoided, data flow path recording can be achieved only through a simple tetrad storage design, the multi-copy flow relation of the same data can be clearly identified and tracked, and the data flow path recording efficiency is improved. The problem that multiple copies cannot be effectively distinguished through a traditional method is solved, and accurate recording and rapid management of data circulation information in a multi-copy scene are achieved.

Description

Technical Field

[0001] The present invention relates to the field of computer network technology, and in particular to a data flow path tracing method, device, electronic device and readable storage medium. Background Art

[0002] When it comes to recording data flow (i.e., data forwarding) paths, the current mainstream technology tracks and manages data by generating unique identifiers for the data. For example, in network systems, each piece of data is typically assigned a globally unique identifier (GUID) to uniquely identify the data and record the intermediate nodes it passes through. By parsing the GUID, distributed storage systems can track the physical flow path of data, thereby supporting path optimization and fault diagnosis.

[0003] In terms of multi-copy management, distributed storage systems typically generate multiple copies of data to improve reliability and availability. To identify and manage these copies, distributed storage systems typically use hash values as identifiers, which record the uniqueness of the data.

[0004] Existing data provenance methods are costly and complex, unable to effectively distinguish the flow paths of multiple copies of the same data, and unable to distinguish the flow relationships between different copies. Therefore, a data provenance method is urgently needed to solve the problem of multiple copy identification and flow path recording. Summary of the Invention

[0005] The present invention provides a data flow path tracing method, device, electronic device and readable storage medium to solve problems such as multiple copy identification and traceability evidence chain construction in existing data circulation and utilization scenarios, and realize data copy identification and flow path recording and tracing.

[0006] The present invention provides a data flow path tracing method, comprising: forming a four-tuple of information including a system identifier of a previous node, a unique identifier of a data copy of the previous node, a system identifier of a current node, and a unique identifier of a data copy of the current node in the form of a key-value pair, and reporting the information to a distributed storage system; The data flow path is determined according to the quadruple information.

[0007] Optionally, the data flow path tracing method includes: using a distributed key-value storage system to store the quadruple information.

[0008] Optionally, the distributed key-value storage system is a Redis in-memory database.

[0009] Optionally, the data flow path tracing method includes: using a one-way irreversible algorithm to generate a unique identifier for the data copy.

[0010] Optionally, the data flow path tracing method includes: when the unique identifier of the data copy of the previous node is the same as the unique identifier of the data copy of the current node, confirming that the data has not been modified; In a case where the data copy unique identifier of the previous node is different from the data copy unique identifier of the current node, it is confirmed that the data is modified.

[0011] Optionally, the data flow path tracing method includes: the quadruple information reported by the initial node uses a special identifier to identify the previous node of the initial node; the initial node is the first node in which the data flows in the distributed storage system.

[0012] Optionally, the special identifier is null.

[0013] The present invention also provides a data flow path tracing device, comprising the following modules: The processing module forms a four-tuple information consisting of the system identifier of the previous node, the unique identifier of the data copy of the previous node, the system identifier of the current node, and the unique identifier of the data copy of the current node in the form of a key-value pair, and reports the information to the distributed storage system; A determination module determines a data flow path according to the quadruple information.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the data flow path tracing method as described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the data flow path tracing methods described above.

[0016] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the data flow path tracing methods described above.

[0017] The data flow path tracing method, device, electronic device and readable storage medium provided by the present invention form a four-tuple information by combining the system identifier of the previous node, the unique identifier of the data copy of the previous node, the system identifier of the current node and the unique identifier of the data copy of the current node in the form of a key-value pair, and report it to the distributed storage system, and determine the data flow path based on the four-tuple information; it achieves remarkable results in data flow path recording and multi-copy identification management, avoiding complex uploading, identification embedding and extraction operations, and can realize data flow path recording only through a simple four-tuple storage design. The key-value pair form simplifies the query logic, so that in large-scale multi-copy scenarios, a complete chain of evidence can still be quickly constructed; it can clearly identify and track the flow relationship of multiple copies of the same data, solves the problem that traditional methods cannot effectively distinguish multiple copies, and realizes accurate recording and rapid management of data flow information in multi-copy scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is a flow chart of a data flow path tracing method provided by the present invention.

[0020] Figure 2 A data flow diagram provided by the present invention.

[0021] Figure 3 A data flow diagram provided by the present invention.

[0022] Figure 4 A structural diagram of a data flow path tracing device provided by the present invention.

[0023] Figure 5 The following is a schematic diagram of the physical structure of an electronic device.

[0024] Reference numerals: Data flow path tracing device 40; processing module 401, determination module 402; processor 510; communication interface 520; memory 530; communication bus 540. DETAILED DESCRIPTION

[0025] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0026] A distributed storage system is a system consisting of multiple computers (nodes) connected via a network. These nodes work together to complete a common task. The design goals of a distributed storage system are to improve performance, reliability, and scalability.

[0027] A distributed storage system distributes data to multiple physical nodes through a network. Each node independently stores part of the data, together forming a complete storage system.

[0028] A globally unique identifier (GUID) is a unique identifier generated by an algorithm and is commonly used to ensure the uniqueness of data in distributed systems.

[0029] Existing data traceability methods generate a GUID for each piece of data in a virtual network environment, allowing the system to record and track the data's flow path. Specific implementations include: assigning a unique namespace to each tenant and embedding it into the data's GUID to ensure the GUID's global uniqueness. During data flow, network devices parse the GUID to record the data's path information to support dynamic path optimization. However, in a multi-tenant environment, GUID embedding requires assigning a unique namespace to each tenant and embedding it into the data. This process increases the complexity of the data header, and the client needs to support parsing and generating the embedded GUID, which increases implementation costs and poses the risk of GUID tampering. Furthermore, using only GUIDs to identify data cannot effectively distinguish the flow paths of multiple copies of the same data. For example, when multiple copies of the same data are generated on different network nodes, the GUID cannot record its specific flow process.

[0030] Existing data traceability methods also leverage the immutability of blockchain to store the data's GUID, hash value, and flow metadata on-chain, enabling data preservation and flow management. The system generates a GUID for each piece of data, combines it with its hash value, and writes it to the blockchain, ensuring data integrity and credibility. In multi-copy scenarios, the chain records the data's storage nodes and flow information, providing support for multi-copy management. This solution supports multi-copy storage, but the GUID only identifies the data itself and provides no mechanism to distinguish the flow relationships between different copies. This makes it difficult to accurately record the dynamic flow of copies, limiting the integrity of the traceability chain.

[0031] Based on the above reasons, this paper proposes a technology that effectively identifies multiple copies of data and records their transfer paths, aiming to address these issues. This technology offers advantages such as low overhead and high traceability efficiency, effectively solving the problems of identifying multiple copies and establishing traceability evidence chains in data circulation and utilization scenarios.

[0032] Figure 1 A flow chart of a data flow path tracing method provided by the present invention, such as Figure 1 As shown, the data flow path tracing method is used in devices such as servers, desktops, and laptops, and includes the following steps.

[0033] In step 101, the system identifier of the previous node, the unique identifier of the data copy of the previous node, the system identifier of the current node and the unique identifier of the data copy of the current node are combined into a four-tuple information in the form of a key-value pair and reported to the distributed storage system.

[0034] The quadruple storage design based on the key-value structure is used to store data flow information. Key represents the key, value represents the value, and key-value represents the key-value pair, which forms quadruple information in the form of key-value.

[0035] It should be noted that the data here includes structured and unstructured data, and supports multimodal data, including but not limited to: text, database tables, forms, pictures, images, voice, video, etc.

[0036] Exemplarily, pre_sysID represents the system identifier of the previous node; pre_dataID represents the unique identifier of the data copy of the previous node; self_sysID represents the system identifier of the current node; and self_dataID represents the unique identifier of the data copy of the current node. Among them, the system identifier is the unique identifier of the node, and the data copy unique identifier (dataID) is the unique identifier of the data copy. The data copy unique identifier can be, but is not limited to, a GUID. In one embodiment, a one-way irreversible algorithm is used to generate a unique identifier of the data copy. The one-way irreversible algorithm includes, but is not limited to, SHA-256, SM3, etc., and the present invention does not impose any restrictions on this. A one-way irreversible algorithm is used to transform the data to generate a unique identifier of the data copy, thereby ensuring the uniqueness and security of the unique identifier of the data copy, preventing the unique identifier of the data copy from being tampered with, and providing technical support for the trusted management of data flow.

[0037] The data is stored as a key-value quadruple of pre_sysID#pre_dataID: self_sysID#self_dataID and reported to a centralized log system or distributed storage system for subsequent query and tracing, which can clearly express the flow relationship of data between nodes.

[0038] In one implementation, the quad-tuple information is stored in a distributed key-value store, typically a Redis in-memory database. Redis is an open-source, high-performance in-memory key-value store widely used in scenarios such as caching, message queuing, and real-time analytics. It supports a variety of data structures, such as strings, hashes, lists, sets, and ordered sets, and provides a rich set of operations.

[0039] Redis's efficient key-value storage features ensure fast data writing, retrieval, and path construction. Compared with traditional relational databases, the storage structure is simpler and the retrieval performance is significantly improved. It is especially suitable for large-scale data flow information management in multi-copy scenarios.

[0040] In the Redis in-memory database, quadruple information is stored in the database in key-value format, which facilitates rapid tracing of data flow paths and can efficiently build data traceability evidence chains in multi-node and multi-copy scenarios. This solves the problem of the inability to accurately record multi-copy flow paths in existing technologies, and provides support for the efficiency and integrity of data management.

[0041] In step 102, the data flow path is determined according to the quadruple information.

[0042] The following two examples illustrate the process of determining the data flow path based on quadruple information.

[0043] Example 1: If pre_dataID and self_dataID are the same, it means that the content of the data copy has not been modified, the information of all data copies is exactly the same, and the dataID (data copy unique identifier) generated by the single irreversible algorithm is the same. Figure 2 As shown, Figure 2 A data flow diagram provided by the present invention, the distributed storage system includes four nodes A, B, C, and D, A is the initial node, B and C are intermediate nodes, and D is the tail node.

[0044] The initial dataID is "2134dsagy#jas@!s." Data is forwarded from node A to node B, from node B to node C, and from node C to node D. As data flows, each node reports a quad-tuple of information to the distributed storage system. For the initial node A, the special identifier "null" is used to identify A's previous hop when reporting the quad-tuple. For the tail node D, even if data is no longer flowing, a quad-tuple of flow information is still reported, indicating the node from which the data flowed, allowing for complete data flow tracing.

[0045] The complete data flow path is shown below: Data is forwarded from node A to node B: the four-tuple information reported by node A is: (null,2134dsagy#jas@!s,A, 2134dsagy#jas@!s); Data is forwarded from node B to node C: the four-tuple information reported by node B is: (A,2134dsagy#jas@!s,B,2134dsagy#jas@!s); Data is forwarded from node C to node D: The four-tuple information reported by node C is: (B,2134dsagy#jas@!s,C,2134dsagy#jas@!s); Node D: The four-tuple information reported by node D is: (C, 2134dsagy#jas@!s, D, 2134dsagy#jas@!s).

[0046] Among them, for the tail node D, the data no longer flows, but it needs to report the flow information once, indicating that the data flows from node C.

[0047] Example 2: If pre_dataID and self_dataID are different, it means that the content of the data copy has been modified, the information of all data copies is not completely consistent, and the unique identifier generated by the dataID based on the single irreversible algorithm is not completely the same. Figure 3 As shown, Figure 3 A data flow diagram provided by the present invention, the distributed storage system includes four nodes A, B, C, and D, A is the initial node, B and C are intermediate nodes, and D is the tail node.

[0048] The initial dataID is: "2134dsagy#jas@!s". The data is forwarded from node A to node B, from node B to node C, and from node C to node D. When the data is flowing, each node reports the quadruple information to the distributed storage system.

[0049] For node A, the special identifier "null" is used to identify A's previous hop node when reporting the quadruple information; for the tail node D, if the data is no longer flowing, it is also necessary to report a flow information quadruple to indicate which node the data is flowing from, so as to trace the complete flow path of the data.

[0050] The complete data flow path is shown below: Data is forwarded from node A to node B: the four-tuple information reported by node A is: (null,2134dsagy#jas@!s,A, 2134dsagy#jas@!s); Data is forwarded from node B to node C: The four-tuple information reported by node B is: (A,2134dsagy#jas@!s,B,aoisudf43918@$#); Data is forwarded from node C to node D: The four-tuple information reported by node C is: (B,aoisudf43918@$#,C,2134dsagy#jas@!s); Node D: The four-tuple information reported by node D is: (C, 2134dsagy#jas@!s, D, 2134dsagy#jas@!s).

[0051] When the data flows to node B, the data copy is modified from "2134dsagy#jas@!s" to "aoisudf43918@$#". When the data flows to node C, the data copy is changed from "aoisudf43918@$#" to "2134dsagy#jas@!s". For tail node D, the data is no longer transferred, but it needs to report transfer information once, indicating that the data was transferred from node C.

[0052] This paper designs a four-tuple information storage method based on a key-value structure and combines it with the high efficiency of the Redis in-memory database to achieve remarkable results in data flow path recording and multi-copy identification management, as shown in the following: 1. Lightweight: This invention avoids complex operations such as uploading, identifier embedding, and extraction. Instead, it records the data flow path through a simple four-tuple information storage design. The storage structure is clear and concise, reducing management complexity and system resource usage compared to traditional relational databases and blockchain storage solutions.

[0053] 2. Efficient retrieval: Leveraging the efficient query features of the Redis in-memory database, the speed of data traceability and flow path queries is greatly improved. Key-vaule simplifies the query logic, enabling the rapid construction of a complete chain of evidence in large-scale multi-copy scenarios.

[0054] 3. Security: A one-way irreversible algorithm is used to generate a unique identifier for a data copy, ensuring its uniqueness and non-tamperability. This provides a reliable data identification mechanism and offers security for multi-copy management and flow path recording.

[0055] 4. Support for multi-copy management: The quadruple information structure can clearly identify and track the flow relationships of multiple copies of the same data, solving the problem of traditional methods being unable to effectively distinguish multiple copies. It also enables accurate recording and rapid management of data flow information in multi-copy scenarios.

[0056] The data flow path tracing method, device, electronic device and readable storage medium provided by the present invention form a four-tuple information by combining the system identifier of the previous node, the unique identifier of the data copy of the previous node, the system identifier of the current node and the unique identifier of the data copy of the current node in the form of a key-value pair, and report it to the distributed storage system, and determine the data flow path based on the four-tuple information; it achieves remarkable results in data flow path recording and multi-copy identification management, avoiding complex uploading, identification embedding and extraction operations, and can realize data flow path recording only through a simple four-tuple storage design. The key-value pair form simplifies the query logic, so that in large-scale multi-copy scenarios, a complete chain of evidence can still be quickly constructed; it can clearly identify and track the flow relationship of multiple copies of the same data, solves the problem that traditional methods cannot effectively distinguish multiple copies, and realizes accurate recording and rapid management of data flow information in multi-copy scenarios.

[0057] The data flow path tracing device provided by the present invention is described below. The data flow path tracing device described below and the data flow path tracing method described above can be referenced to each other.

[0058] Figure 4 This is a structural diagram of a data flow path tracing device provided by the present invention. The data flow path tracing device is used in devices such as servers, desktops, and laptops. Figure 4 The data flow path tracing device 40 includes a processing module 401 and a determination module 402.

[0059] The processing module 401 is configured to form a four-tuple information consisting of the system identifier of the previous node, the unique identifier of the data copy of the previous node, the system identifier of the current node, and the unique identifier of the data copy of the current node in the form of a key-value pair, and report it to the distributed storage system; The determination module 402 is configured to determine a data flow path according to the quadruple information.

[0060] Figure 5 An example of a physical structure diagram of an electronic device is shown below. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute a data flow path tracing method, which includes: forming a four-tuple information in the form of a key-value pair, including the system identifier of the previous node, the unique identifier of the data copy of the previous node, the system identifier of the current node, and the unique identifier of the data copy of the current node, and reporting it to the distributed storage system; and determining the data flow path based on the four-tuple information.

[0061] Furthermore, the logic instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0062] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data flow path tracing method provided by the above methods, which includes: the system identifier of the previous node, the unique identifier of the data copy of the previous node, the system identifier of the current node, and the unique identifier of the data copy of the current node are combined into a four-tuple information in the form of a key-value pair, and reported to the distributed storage system; determine the data flow path based on the four-tuple information.

[0063] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the data flow path tracing method provided by the above-mentioned methods. The method includes: the system identifier of the previous node, the unique identifier of the data copy of the previous node, the system identifier of the current node, and the unique identifier of the data copy of the current node are combined into a four-tuple information in the form of a key-value pair, and reported to the distributed storage system; the data flow path is determined based on the four-tuple information.

[0064] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0065] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data flow path tracing method, characterized in that: include: The system identifier of the previous node, the unique identifier of the data copy of the previous node, the system identifier of the current node, and the unique identifier of the data copy of the current node are combined into a four-tuple information in the form of a key-value pair and reported to the distributed storage system; The data flow path is determined according to the quadruple information.

2. The method according to claim 1, characterized in that include: A distributed key-value storage system is used to store the quadruple information.

3. The method according to claim 2, characterized in that The distributed key-value storage system is a Redis in-memory database.

4. The method according to claim 1, wherein include: A one-way irreversible algorithm is used to generate a unique identifier for the data copy.

5. The method according to claim 1, wherein include: If the unique identifier of the data copy of the previous node is the same as the unique identifier of the data copy of the current node, confirming that the data has not been modified; In a case where the data copy unique identifier of the previous node is different from the data copy unique identifier of the current node, it is confirmed that the data is modified.

6. The method according to claim 1, characterized in that include: The four-tuple information reported by the initial node uses a special identifier to identify the previous node of the initial node; The initial node is the first node through which the data flows in the distributed storage system.

7. The method according to claim 6, characterized in that The special flag is null.

8. A data flow path tracing device, characterized in that: include: The processing module forms a four-tuple information consisting of the system identifier of the previous node, the unique identifier of the data copy of the previous node, the system identifier of the current node, and the unique identifier of the data copy of the current node in the form of a key-value pair, and reports the information to the distributed storage system; A determination module determines a data flow path according to the quadruple information.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the data flow path tracing method as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data flow path tracing method as described in any one of claims 1 to 7 is implemented.

Images