Method for efficiently executing data segmentation and circulation of low-code data processing platform

Abstract

The invention discloses a method for efficiently executing data segmentation and circulation of a low-code data processing platform. The method comprises the following steps that S1, a user creates a data processing component through dragging operation; s2, creating a node object for each node; s3, the system constructs a complete node network diagram according to the front-back relation between the nodes; s4, identifying and segmenting sub-links which can be executed independently of other nodes; s5, performing further stage division on the sub-links identified and segmented in the step S4; s6, performing parallel scheduling and execution on the sub-links in each stage; s7, integrating the output results of the sub-links by the system; and S8, the system ends the whole data processing flow. The method has the following beneficial effects: 1, the data processing and analysis capability is enhanced; 2, the improved data is visualized; 3, the capability of coping with complex data circulation; 4, efficient data segmentation and circulation; 5, the data processing speed and efficiency are improved; 6, optimized resource utilization; and 7, simplified maintenance and upgrading.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method for efficiently executing data segmentation and circulation on a low-code data processing platform. Background Art

[0002] With the rapid development of information technology, data processing has become an indispensable part of the daily operations of enterprises and organizations. In modern enterprises, there is an increasing demand for data processing platforms, which need to process and analyze large amounts of data to support decision-making, business process optimization, and customer experience improvement. Traditional data processing platforms usually require professional programming knowledge and complex configuration, which limits the use of non-technical users and increases the cost of development and maintenance.

[0003] In the field of low-code development, there are more and more low-code development platforms for enterprises. They use a visual method of application development to enable developers with different experience levels to create web pages and mobile applications through a graphical user interface, using drag-and-drop components and model-driven logic. These platforms establish data storage and reading drivers by studying basic processes, forms, etc., so as to realize the display of data in visual applications.

[0004] For example, patent CN115826971A provides a method for pushing and receiving device data through a separate data processing engine, solving the underlying IoT data processing dependency and improving the flexibility of developing and deploying device visualization applications. Patent CN117555935B discloses a data processing method and device for a low-code platform, which aims to improve the efficiency and flexibility of data processing and is suitable for the management and analysis of multiple data sources. Patent CN118012963B provides a low-code data exploration device that can realize lightweight, fast, customized exploration and analysis of multiple data sources, and is suitable for various application scenarios such as refined operations and business reports.

[0005] However, the above data processing method has the following defects: (1) Limitations of data source access and simple analysis: Many existing low-code platforms can only perform simple access and analysis of data sources, and lack the ability to process and analyze complex data. These platforms are often unable to perform advanced processing and analysis of data, resulting in low efficiency when processing large or complex data sets, and are unable to meet the needs of enterprises for deep data mining.

[0006] (2) Insufficient data visualization: Current technology makes it difficult to visualize the data of each node in the low-code platform. This limits users’ monitoring and understanding of the data flow process, affecting decision-making and business process optimization.

[0007] (3) Inadequate response to the complexity of data flow: Most existing low-code platforms support simple one-way data flow, but are often unable to effectively respond to complex data flow forms such as one-to-many or many-to-one. This requires the platform to have more advanced data routing and processing capabilities to adapt to changing business needs.

[0008] (4) Lack of efficient data segmentation and flow methods: The current low-code platform does not clearly provide an efficient data segmentation method and data flow form. When processing large-scale data, data segmentation and intelligent routing are the key to improving data processing efficiency. The lack of such a mechanism will lead to data processing bottlenecks and affect the performance of the overall system.

[0009] Therefore, further improvements are necessary. Summary of the invention

[0010] The purpose of the present invention is to provide a method for efficiently executing data segmentation and circulation on a low-code data processing platform with high data processing efficiency, high degree of data visualization, strong ability to cope with complex data flow, efficient data segmentation and circulation, high data processing speed and efficiency, high resource utilization, and simple maintenance and upgrade, so as to overcome the shortcomings of the prior art.

[0011] A method for efficiently executing data segmentation and circulation on a low-code data processing platform designed for this purpose is characterized by comprising the following steps: S1. According to the data processing components selected and placed by dragging and dropping and the connection relationship between the components, the user creates the data processing components by dragging and dropping. After the dragging and dropping is completed, the system obtains the parameters and related information of all nodes; S2: The system traverses all nodes involved in data processing and creates a node object for each node; S3: The system builds a complete node network graph based on the previous and next relationships between nodes; S4: The system uses a breadth-first search strategy to traverse the constructed node network graph and identify and segment sub-links that can be executed independently of other nodes; S5: further divide the sub-links identified and segmented in step S4 into different stages. The system assigns the sub-links to different execution stages according to their dependencies. S6: According to the new stage division result, the system adopts a parallel scheduling algorithm to parallel schedule and execute the sub-links in each stage in the order of the stages; S7: After the sub-links of each stage complete their respective execution tasks, the system will integrate the output results of each sub-link and pass the results to the sub-link of the next stage according to the dependency relationship; S8: After all sub-links have been executed and data transfer is completed, the system will end the entire data processing process and verify and post-process the final results.

[0012] In step S1, the relevant information of the node includes the node ID, the node name, the node pointing ID, the node pointed ID and the node execution method.

[0013] In step S2, the steps of creating a node object are as follows: S21, traverse all nodes for the first time and complete the node object class initialization; S22. According to the connection pointing relationship of the nodes, the node type is determined and the node type attribute is assigned.

[0014] In step S21, the node object includes the following attributes: node ID, node name, child node ID, parent node ID, node execution method, node type, child node object list, parent node object list and node execution result.

[0015] In step S3, the steps for constructing the node network graph are as follows: S31, establishing a connection relationship between nodes according to their preceding node IDs and succeeding node IDs to construct a node network graph; S32. After constructing the node network graph, the system will verify the graph to ensure that there are no loops.

[0016] In step S32, the system performs loop detection when verifying the node network graph. The loop detection method is as follows: the system traverses the node network through algorithms such as depth-first search, checks the predecessor and successor relationships of each node, and ensures that there is no repeated access to nodes in the graph; if a loop is detected, the system will issue a warning and ask the user to check the connection relationship; if no loop is detected, the system proceeds to the next step.

[0017] In step S4, the steps of identifying and segmenting the sub-links are as follows: S41, find all nodes of type "one-way pointing" as the starting point of the search; S42, traverse all starting points and retrieve simple sub-links; S43, traverse the entire network graph to complete sub-link cutting.

[0018] In step S42, the steps for retrieving a simple sub-link are as follows: S421, start searching from the starting point: for each starting point node found, the system will start a breadth-first search and traverse its subsequent nodes until a complete sub-link is found; S422, during the retrieval process, the system will record each complete sub-link; S423. When a complete sub-link is found, the last node of the sub-link is used as a new starting point to continue looking for the next simple sub-link.

[0019] In step S5, the sub-link stage division steps are as follows: S51, the system will traverse all sub-links, find out those links whose end nodes have no subsequent nodes, and mark them as "end links"; S52: Taking the end link defined in step S51 as the starting point, reversely search for the parent link and divide into new stages.

[0020] In step S8, when the system ends the entire data processing flow, the system will make detailed log records of the execution process of the entire data processing flow, including the execution time of each node, data flow status, resource consumption, etc., and the system will inform the user that the data processing has been completed through a message or notification mechanism, and provide access methods and download links for the final results.

[0021] The low-code data processing platform of the present invention has the following beneficial effects: (1) Enhanced data processing and analysis capabilities: By building node objects and parsing the connections and dependencies between nodes, this solution provides a mechanism that enables low-code platforms to handle more complex data sets and analysis tasks. This overcomes the problem of low efficiency of existing platforms in processing large-scale or complex data sets and meets the needs of enterprises for deep data mining. (2) Improved data visualization: This solution enables users to monitor and understand the data flow process more clearly through efficient data flow. This improved data visualization capability helps users make better decisions and optimize business processes, thus solving the shortcomings of existing technologies in data visualization. (3) Ability to cope with complex data flows: By dividing the stages and executing the sub-links of the same stage in parallel, this solution provides an effective method to deal with complex data flows such as one-to-many and many-to-one. This advanced data routing and processing capability enables the platform to adapt to changing business needs and solves the shortcomings of existing platforms in dealing with complex data flows. (4) Efficient data segmentation and circulation: This solution provides an efficient data segmentation method and data flow form through stage segmentation and intelligent routing mechanism, which helps to improve the efficiency of large-scale data processing and solves the bottleneck problem of data segmentation and flow in the existing platform, thereby improving the performance of the overall system; (5) Improve data processing speed and efficiency: By executing sub-links of the same stage in parallel, this scheme significantly improves the speed and efficiency of data processing. This method allows multiple sub-links to be processed simultaneously, reducing waiting time and thus speeding up the overall data processing process. (6) Optimized resource utilization: By identifying sub-links that can be executed independently, this solution allows resources to be more efficiently allocated and utilized among multiple sub-links. This optimized resource utilization helps improve the performance and responsiveness of the entire platform.

[0022] (7) Simplified maintenance and upgrades: Since this solution provides clear dependencies between nodes and sub-links, it makes platform maintenance and upgrades simpler, and maintenance personnel can more easily identify and solve potential problems, thereby improving the stability and reliability of the platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 The present invention is a flowchart of a method for data segmentation and circulation in one embodiment of the present invention.

[0024] Figure 2 FIG. 1 is a schematic diagram of node connection in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0026] See also Figure 1-Figure 2 , the low-code data processing platform's efficient execution of data segmentation and circulation method includes the following steps: S1. According to the data processing components selected and placed by dragging and dropping and the connection relationship between the components, the user creates the data processing components by dragging and dropping. The data processing components are as follows: Figure 1 As shown in the figure, after dragging is completed, the system obtains the parameters and related information of all nodes; a brief example of specific node information is as follows: { "nodeList": [ { "name": "Taxi Data", "nodeType": "jdbc", "nodeId": "211ukci8w1694071821570", "preNodeId": [], "nextNodeId": ["fzztb5hwdo1694072985930"] }, { "name": "District", "nodeType": "jdbc", "nodeId": "cz9qpo3qoa1694071996912", "preNodeId": [], "nextNodeId": ["uzhrhitl0n1694074170505"] }, { "name": "Date Formatter", "nodeType": "date_format", "nodeId": "fzztb5hwdo1694072985930", "preNodeId": ["211ukci8w1694071821570"], "nextNodeId": ["9fgte19s381694073011362"] }, { "name": "Deduplication Component", "nodeType": "drop_duplicates", "nodeId": "9fgte19s381694073011362", "preNodeId": ["fzztb5hwdo1694072985930"], "nextNodeId": ["p4tjgcsmee1694073021049"] }, { "name": "Aggregation Component", "nodeType": "aggregate", "nodeId": "p4tjgcsmee1694073021049", "preNodeId": ["9fgte19s381694073011362"], "nextNodeId": ["qksblwe7361694074224254"] }, { "name": "Database", "nodeType": "jdbc", "nodeId": "fno7yccvhb1694073885056", "preNodeId": [], "nextNodeId": ["umrebv4s91694074155487"] }, { "name": "Field rename", "nodeType": "rename_columns", "nodeId": "umrebv4s91694074155487", "preNodeId": ["fno7yccvhb1694073885056"], "nextNodeId": ["uzhrhitl0n1694074170505"] }, { "name": "Merge Component", "nodeType": "row_concat", "nodeId": "uzhrhitl0n1694074170505", "preNodeId": ["cz9qpo3qoa1694071996912", "umrebv4s91694074155487"], "nextNodeId": ["41v7qx4p691694074203079"] }, { "name": "Complete Division", "nodeType": "drop_duplicates", "nodeId": "41v7qx4p691694074203079", "preNodeId": ["uzhrhitl0n1694074170505"], "nextNodeId": ["qksblwe7361694074224254"], "params": { "input": "uzhrhitl0n1694074170505", "keep": "first", "subset": [] } }, { "name": "Statistical results", "nodeType": "inner_join", "nodeId": "qksblwe7361694074224254", "preNodeId": ["p4tjgcsmee1694073021049","41v7qx4p691694074203079"], "nextNodeId": [] } ] } The specific information of the above code is as follows: (1) Node ID: Each node has a unique identification code. For example, the ID of the taxi data node is 211ukci8w1694071821570. (2) Node name: Each node has a corresponding name, such as "Taxi Data" or "Date Formatting"; (3) Node pointing ID: describes the connection relationship between a node and other nodes. For example, the "Taxi Data" node points to the "Date Formatting" node through a connection; (4) Node pointing ID: describes the connection relationship between a node and other nodes. For example, the "Date Formatting" node is pointed to by the "Taxi Data" node; (5) Node execution method: the specific functions performed by each node, such as JDBC connection, deduplication, aggregation, etc.

[0027] S2. After obtaining the node information in step S1, the system traverses all nodes involved in data processing and creates a node object for each node. The steps are as follows: S21. Traverse all nodes for the first time and complete the node object class initialization; the node object includes the following attributes: (1) Node ID: Each node has a unique identification code (for example, UUID4 format), such as the ID of the “taxi data” node is 211ukci8w1694071821570.

[0028] (2) Node name: a name that identifies the node, such as "region" or "deduplication component".

[0029] (3) Child node ID: The ID of the other child node pointed to by this node. For example, the child node of the "Taxi Data" node is the "Date Formatting" node.

[0030] (4) Parent node ID: points to the parent node ID of the node. For example, the parent node of the "Duplicate Component Removal" node is the "Date Formatting" node.

[0031] (5) Node execution method: the corresponding data processing method, such as JDBC reading, field renaming, deduplication or aggregation.

[0032] In addition, in addition to the information provided by the node, the system will automatically generate additional attributes to facilitate the construction of a complete graph network structure. During the initialization process, the following attributes will be left blank and will be supplemented in the next step.

[0033] (6) Node type: such as bidirectional type, unidirectional pointing type or unidirectional pointing type.

[0034] (7) Subnode object list: A subnode object list constructed based on the subnode ID. For example, the subnode pointed to by the “aggregation component” is “statistical result”.

[0035] (8) Parent node object list: a parent node object list constructed based on the parent node ID.

[0036] (9) Node execution result: When the node is executed, if there is an output result, it is stored in this attribute.

[0037] S22. According to the connection relationship of the nodes, the node type is determined and the node type attribute is assigned. For example: The "Taxi Data" node has no dependent nodes. As the source of data, it is the head node of the entire link. This node is a one-way pointing type. The "Date Format" node relies on the "Taxi Data" node to provide data as its input. At the same time, the processing result of this node also needs to be used as the input of other nodes. Therefore, this node is a bidirectional type. The "Statistical Results" node is at the end. It depends on the results of multiple parent nodes, but its results do not need to be passed to other nodes. Therefore, this node is a one-way input type. After judgment, the system completes the judgment on each node type and marks the node type.

[0038] S3: In this step, the system will traverse each node dragged and configured from the user interface, and build a complete node network diagram based on the previous and next relationships between the nodes; the following is a detailed description of the specific steps: S31. Establish connection relationships between nodes according to their preNodeId (previous node ID) and nextNodeId (next node ID) to construct a node network graph; for example: For the "Date Formatting" node, whose predecessor node is "Taxi Data", the system will add the node object of "Date Formatting" to the child node object list of "Taxi Data"; similarly, the system will also add the ID of each node to the parent node object list of its successor node to form a complete node linked list structure.

[0039] Through this process, the final node network diagram will show the dependency relationship and execution order between each node.

[0040] S32. After the node network diagram is constructed, the system will verify the diagram to ensure that there is no loop. The specific operations include: (1) Loop detection: The system traverses the node network through algorithms such as depth-first search (DFS), checks the predecessor and successor relationships of each node, and ensures that there is no repeated access to nodes in the graph; this step is key to ensuring a smooth and effective data processing flow and avoiding potential infinite loop problems.

[0041] (2) Result confirmation: If a loop is detected, the system will issue a warning and ask the user to check the connection relationship; if the network diagram is valid (that is, no loop is detected), proceed to the next step.

[0042] S4: The system uses the breadth-first search (BFS) strategy to traverse the constructed node network graph, identify and split sub-links that can be executed independently of other nodes. These sub-links are simple sub-links. These sub-links can be processed in parallel without pre-dependencies, thereby accelerating the overall data processing flow; a simple sub-link is defined as: a unidirectional link with a unique direction, in which only the starting point of the link is allowed to have multiple parent nodes and the end point to have multiple child nodes.

[0043] The steps for identifying and segmenting sub-links are as follows: S41, find all nodes of the type "one-way pointing" as the starting point of the search; in the example, for example, the "taxi data" and "district" nodes are all nodes of the one-way pointing type; S42, traverse all starting points and retrieve simple sub-links; the specific steps are as follows: S43, traverse the entire network graph to complete sub-link cutting.

[0044] As the system traverses the entire node network graph, it ensures that each starting point is searched until all possible sub-links are identified; finally, the system will summarize all the simple sub-links identified and prepare for subsequent stage division; for example, the sub-links that may be identified include: Sublink 1: ["Taxi data", "Date formatting", "Deduplication component", "Aggregation component"] Sublink 2: ["Zoning"] Sublink 3: ["Database", "Field Rename"] Sublink 4: ["Merge components", "Complete partitioning"] Sublink 5: ["Statistical Results"] In step S42, the steps for retrieving a simple sub-link are as follows: S421, start searching from the starting point: for each starting point node found, the system will start a breadth-first search (BFS) and traverse its subsequent nodes until a complete sub-link is found; S422, record sub-links: During the retrieval process, the system will record each complete sub-link; for example, starting from the "Taxi Data" node, through the "Date Formatting" and "Deduplication Components" nodes in sequence, a complete sub-link is formed: Sublink example: ["Taxi data", "Date formatting", "Duplicate removal component", "Aggregation component"] S423, update the search starting point: when a complete sub-link is found, the last node of the sub-link will be used as a new starting point to continue searching for the next simple sub-link; in this way, the system will continue to search for other potential sub-links.

[0045] S5: Further stage division is performed on the sub-links identified and segmented in step S4. The system allocates them to different execution stages according to the dependencies of the sub-links. The specific steps are as follows: S51, the system will traverse all sub-links, find out those links whose end nodes have no subsequent nodes, and mark them as "end links"; according to the sub-link division in step S4, the "statistical result" is the end node, so it belongs to stage 1, indicating that it will be the final executed sub-link; S52, starting from the end link defined in step S51, searching the parent link in reverse order to divide into new stages; the specific steps are as follows: (1) Reverse search: Starting from the "Statistical Results" node, search forward to its parent node and find the sub-links that depend on it; according to the data, "Statistical Results" depends on the "Aggregation Component" and "Complete Division" nodes, and the sub-links corresponding to these two nodes are divided into the second stage. That is, sub-link 1: ["Taxi Data", "Date Formatting", "Deduplication Component", "Aggregation Component"] and sub-link 4: ["Merge Component", "Complete Division"] are the second stage; (2) Level-by-level division: The system continues to search the parent node of the sub-link in reverse order, finds the predecessor node in turn, and continues to divide the new stage; the final stage is as follows: Phase 1: Sublink 5 Phase 2: Sublink 1, Sublink 4 Phase 3: Sublink 2, Sublink 3 S53: Reverse the order of the stages divided in S52 to complete the final stage division. The final stage division is: Phase 1: Sublink 2, Sublink 3 Phase 2: Sublink 1, Sublink 4 Phase 3: Sublink 5 Through step S5, the system completes the stage division of all sub-links; the sub-links in each stage are independent of each other and can be executed in parallel, while the stages are executed serially according to the dependency relationship.

[0046] S6: According to the new stage division result, the system adopts a parallel scheduling algorithm to parallel schedule and execute the sub-links in each stage in the order of the stages; (1) Phase 1: ["Zoning"] and ["Database", "Field Renaming"] In the first stage, the "zoning" sub-link and the "database" sub-link can be executed in parallel; since they have no pre-dependencies, the system will allocate computing resources to both sub-links for parallel processing.

[0047] (2) Phase 2: ["Taxi data", "Date formatting", "Deduplication component", "Aggregation component"] and ["Merge component", "Complete division"] In the second stage, the "taxi data" link and the "merged component" link depend on the completion of the sub-links in the first stage; when the first stage is completed, the system will schedule these two links in parallel and dynamically optimize the allocation based on resource conditions.

[0048] (3) Stage 3: ["Statistical Results"] The last stage, "statistical results", is the end point of the process and depends on the output of the second stage; when all the preceding links are completed, the system will schedule and execute the link.

[0049] S7: After the sub-links of each stage complete their respective execution tasks, the system will integrate the output results of each sub-link and pass the results to the sub-links of the next stage according to the dependency relationship to ensure the consistency and accuracy of the entire data processing process.

[0050] (1) Phase 1 Integration: After sub-link 2 (["Zoning"]) and sub-link 3 (["Database", "Field Rename"]) are executed, the system will pass their output results to the sub-link of stage 2. The output of sub-link 2 will be passed to sub-link 4 ["Merge Components", "Complete Zoning"].

[0051] (2) Phase 2 Integration: After sublink 1 (["Taxi data", "Date formatting", "Duplicate removal component", "Aggregation component"]) and sublink 4 (["Merge component", "Complete districting"]) are completed, the system will integrate the results of the two sublinks. At this time, the processing results of the taxi data will be associated with the data of the complete districting in the next stage of sublink 5 (["Statistical results"]).

[0052] S8: After all sub-links are executed and data flow is completed, the system will end the entire data processing process and verify and post-process the final results to ensure the quality and availability of the results.

[0053] When sub-link 5 (["Statistical Results"]) of stage 3 is completed, the system marks the entire data processing flow as completed; the execution status of all data processing tasks will be updated to "Completed" for subsequent operations or user queries.

[0054] When the system completes the entire data processing flow, the system will record the execution process of the entire data processing flow in detail, including the execution time of each node, data flow, resource consumption, etc. These logs can be used for subsequent performance optimization, debugging, and tracking of data processing paths.

[0055] Finally, the system will inform the user through a message or notification mechanism that the data processing is complete, and provide access methods and download links for the final results.

[0056] The above is a preferred embodiment of the present invention, which shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for illustrating the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention, and these changes and improvements fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for efficiently executing data segmentation and circulation on a low-code data processing platform, characterized in that: The following steps are involved: S1. According to the data processing components selected and placed by dragging and dropping and the connection relationship between the components, the user creates the data processing components by dragging and dropping. After the dragging and dropping is completed, the system obtains the parameters and related information of all nodes; S2, the system traverses all nodes involved in data processing and creates a node object for each node; S3, the system builds a complete node network diagram based on the previous and next relationships between nodes; S4, the system uses a breadth-first search strategy to traverse the constructed node network graph and identify and segment sub-links that can be executed independently of other nodes; S5, further dividing the sub-links identified and segmented in step S4 into different stages, and the system assigns them to different execution stages according to the dependencies of the sub-links; S6. According to the new stage division result, the system adopts a parallel scheduling algorithm to perform parallel scheduling and execution of sub-links in each stage in the order of stages. S7, after the sub-links of each stage complete their respective execution tasks, the system will integrate the output results of each sub-link and pass the results to the sub-link of the next stage according to the dependency relationship; S8. After all sub-links are executed and data flow is completed, the system will end the entire data processing process and verify and post-process the final results.

2. The method for efficiently executing data segmentation and circulation of a low-code data processing platform according to claim 1 is characterized in that: In step S1, the relevant information of the node includes the node ID, the node name, the node pointing ID, the node pointed ID and the node execution method.

3. The method for efficiently executing data segmentation and circulation of a low-code data processing platform according to claim 1 is characterized in that: In step S2, the steps of creating a node object are as follows: S21, traverse all nodes for the first time and complete the node object class initialization; S22. According to the connection pointing relationship of the nodes, the node type is determined and the node type attribute is assigned.

4. The method for efficiently executing data segmentation and circulation of a low-code data processing platform according to claim 3 is characterized in that: In step S21, the node object includes the following attributes: node ID, node name, child node ID, parent node ID, node execution method, node type, child node object list, parent node object list and node execution result.

5. The method for efficiently executing data segmentation and circulation of a low-code data processing platform according to claim 1 is characterized in that: In step S3, the steps for constructing the node network graph are as follows: S31, establishing a connection relationship between nodes according to their preceding node IDs and succeeding node IDs to construct a node network graph; S32. After constructing the node network graph, the system will verify the graph to ensure that there are no loops.

6. The method for efficiently executing data segmentation and circulation of a low-code data processing platform according to claim 5 is characterized in that: In step S32, the system performs loop detection when verifying the node network graph. The loop detection method is as follows: the system traverses the node network through algorithms such as depth-first search, checks the predecessor and successor relationships of each node, and ensures that there is no repeated access to nodes in the graph; if a loop is detected, the system will issue a warning and ask the user to check the connection relationship; if no loop is detected, the system proceeds to the next step.

7. The method for efficiently executing data segmentation and circulation of a low-code data processing platform according to claim 1 is characterized in that: In step S4, the steps of identifying and segmenting the sub-links are as follows: S41, find all nodes of type "one-way pointing" as the starting point of the search; S42, traverse all starting points and retrieve simple sub-links; S43, traverse the entire network graph to complete sub-link cutting.

8. The method for efficiently executing data segmentation and circulation of a low-code data processing platform according to claim 7 is characterized in that: In step S42, the steps for retrieving a simple sub-link are as follows: S421, start searching from the starting point: for each starting point node found, the system will start a breadth-first search and traverse its subsequent nodes until a complete sub-link is found; S422, during the retrieval process, the system will record each complete sub-link; S423. When a complete sub-link is found, the last node of the sub-link is used as a new starting point to continue looking for the next simple sub-link.

9. The method for efficiently executing data segmentation and circulation of a low-code data processing platform according to claim 1 is characterized in that: In step S5, the sub-link stage division steps are as follows: S51, the system will traverse all sub-links, find out those links whose end nodes have no subsequent nodes, and mark them as "end links"; S52: Taking the end link defined in step S51 as the starting point, reversely search for the parent link and divide into new stages.

10. The method for efficiently executing data segmentation and circulation of a low-code data processing platform according to claim 1 is characterized in that: In step S8, when the system ends the entire data processing flow, the system will make detailed log records of the execution process of the entire data processing flow, including the execution time of each node, data flow status, resource consumption, etc., and the system will inform the user that the data processing has been completed through a message or notification mechanism, and provide access methods and download links for the final results.

Images