Workflow building type spatio-temporal dynamic perception data aggregation system and method
Through a workflow-built system, the graphical interface and modular design are used to solve the problem of insufficient scalability and flexibility of data aggregation systems in the existing technology, and the flexible processing and efficient aggregation of space-time dynamically perceived data is realized to adapt to complex Requirements of the IoT environment.
Patent Information
- Application Number
- CN202411562147.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-05
- Publication Date
- 2024-12-03
AI Technical Summary
The existing space-time dynamic perceptual data aggregation system has shortcomings in terms of scalability and flexibility, and cannot flexibly configure and adjust workflows, resulting in limited performance in the face of complex and changing IoT environments.
Using a workflow-building system, the tool library module, visual drag-and-drop editing module, workflow management module, scheduled task supervision module and log management module provide a graphical interface and flexible configuration method, allowing users to build and adjust data aggregation work. Streaming, implements data source adaptation, processing and storage, and supports long-term automated access and real-time monitoring.
It improves the flexibility and adaptability of the system, lowers the threshold for use, ensures the accuracy and stability of data processing, adapts to a variety of data sources and processing needs, and achieves efficient aggregation of complex data.
Smart Images

Figure CN119065775A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing, and in particular to a workflow-based spatiotemporal dynamic perception data aggregation system and method. Background Art
[0002] With the widespread application of Internet of Things technology, the number of smart sensor devices connected has increased dramatically, generating massive and real-time transmission of spatiotemporal dynamic perception data. This type of data has the characteristics of high frequency, time series and spatiality, and has become the core support for modern intelligent analysis and decision-making. Although technological advances have greatly facilitated data collection, the complex and diverse communication protocols and access scenarios have also brought major challenges to the real-time aggregation and sharing and exchange of data.
[0003] At present, the mainstream spatiotemporal dynamic perception data aggregation systems are mostly based on static data processing architectures, and usually rely on preset workflows and models for data integration and processing. The system receives spatiotemporal dynamic data from a variety of data sources (such as sensor networks, API interfaces, IoT devices, etc.) through standardized transmission protocols (such as HTTP, MQTT, CoAP, etc.), and performs a series of immediate processing operations such as data cleaning, format conversion, and outlier removal, and then stores it in a relational database or spatiotemporal database in a predetermined format for subsequent query, analysis, and application. This static architecture ensures the stability of data processing and long-term continuous access capabilities, but has significant limitations in flexibility and configurability. Specifically, the workflow of existing systems is usually pre-defined, and users lack intuitive ways to flexibly configure and adjust the workflow. When new data sources or processing requirements are introduced, complex modifications to the system are often required, which limits the applicability and scalability of the system.
[0004] Therefore, facing the complex and ever-changing IoT environment, the lack of scalability and flexibility of the existing data aggregation system has become an important factor restricting its effectiveness. Exploring a spatiotemporal dynamic data aggregation system with higher configurability and flexibility has become an urgent problem to be solved. Summary of the invention
[0005] In view of the problems of insufficient scalability, flexibility and configurability of data aggregation systems in the prior art, the purpose of the present invention is to introduce visual process management technology to provide a system that enables users to flexibly build and adjust customized data aggregation workflows in a graphical manner to dynamically adapt to complex and diverse data source types and data processing requirements, thereby improving the system's adaptability and ability to cope with complex data processing challenges, and further serving the Internet of Things application scenarios more accurately and efficiently. A workflow-based spatiotemporal dynamic perception data aggregation system and method.
[0006] To achieve the above purpose, the technical solution adopted by the present invention is: a workflow-based spatiotemporal dynamic perception data aggregation system, comprising: Tool library module: includes predefined and reusable tools for data reception, processing and storage, and is used to perform adaptation access and conversion processing operations on data streams of different protocols and formats. The tool carries the functional logic of the operation and is implemented as an executable script code, which is intuitively displayed through graphical interface elements; Visual drag-and-drop editing module: a visual interactive interface for building and editing data aggregation workflows, connecting multiple tools in series in the form of graphical interface elements to form a logical diagram of the execution process, representing the complete aggregation process from data reception, parsing to processing and storage. The output of one tool is used as the input of the next tool, thereby achieving continuous data processing; Workflow management module: used to perform specific operation management on each created logical diagram, including deletion, renaming, editing, import / export, and scheduled tasks. The scheduled tasks support the logical diagram to automatically execute its processing according to the predetermined plan and perform long-term access to spatiotemporal dynamic perception data; Planned task supervision module: used to supervise and manage the planned tasks of the logic diagram, monitor the execution status of the planned tasks, track the execution status of the planned tasks in real time, and monitor the key operating indicators of each task execution; Log management module: comprehensively record the execution process of scheduled tasks, generate detailed execution logs, capture key data of scheduled tasks, and accurately filter execution logs according to query parameters of scheduled tasks.
[0007] The above-mentioned workflow-based spatiotemporal dynamic perception data aggregation system, the tool library module includes: Data definition toolset: provides tools for describing the data receiving structure, including specifying the field content and type of the received data, setting the date and time conversion method, and constructing the spatial geometry field; Data receiving toolset: It brings together targeted receiving and adapting tools, supports mainstream IoT transmission protocols and data formats, and enables accurate reception and analysis and conversion of diverse real-time data streams from different sensors and devices, including HTTP tools, RabbitMQ tools, and MQTT tools. Data processing tool set: Provides specialized tools for immediate processing of received real-time data, including data filtering tools and data conversion tools; Data storage toolset: used to configure data storage strategies to save accessed data information as structured data sets efficiently and orderly.
[0008] The workflow-based spatiotemporal dynamic perception data aggregation system mentioned above, wherein the data filtering tool focuses on accurately filtering out target information from massive and continuously updated data streams based on spatial location coordinates and attribute information; The data conversion tools are used to perform complex aggregation, splitting, and reconstruction operations, including field value calculation, spatial transformation, and processing.
[0009] The above-mentioned workflow-built spatiotemporal dynamic perception data aggregation system, the visual drag editing module includes three elements: tool graphics, connecting lines, and execution parameters. The tool graphics are graphical elements added to the visual drag editing module from the tool library through drag operations, and are freely laid out and arranged in the visual drag editing module; the connecting lines are used to connect the tool graphics in series, establish connection relationships between each other to determine the data flow direction and execution order; the execution parameters are parameter conditions set for the tool graphics or connecting lines in the logical diagram.
[0010] The above-mentioned workflow-built spatiotemporal dynamic perception data aggregation system, the visual drag-and-drop editing module also includes an execution button element and a save button element. The execution button element is used to trigger the execution process of the entire logical diagram once the logical diagram is created and configured, and to display the execution status of each tool, data transmission status and possible error messages in real time to help verify the validity of the current workflow configuration and the accuracy of data aggregation; the save button element is used to save all configuration contents of the current logical diagram, and store it in the workflow management module with a custom name to facilitate subsequent editing or reuse.
[0011] The above-mentioned workflow-built spatiotemporal dynamic perception data aggregation system, in the workflow management module, enters into the visual drag editing module through the editing jump, and at the same time loads the configuration details of the current logical diagram to be edited, and displays its corresponding process diagram. Users can adjust or modify the tools, parameter content or execution process in the logical diagram based on the graphical operation interface of the visual drag editing module; export the configuration information of the created logical diagram into a text file, and load the specific configuration into the visual drag editing module page through import; when the scheduled task setting of the logical diagram is completed, the scheduled task information will be saved to the scheduled task supervision module for management.
[0012] The above-mentioned workflow-based spatiotemporal dynamic perception data aggregation system, the planned task supervision module includes task management and task monitoring, the task management includes starting tasks, pausing tasks, and deleting tasks. After the planned task is started, its execution process will not be affected by the changes in the original logical schematic configuration. When the planned task is restarted, the system will reload the latest logical schematic configuration and apply it to subsequent task executions.
[0013] The above-mentioned workflow-based spatiotemporal dynamic perception data aggregation system, the key operating indicators include task progress, execution time, access data volume and current execution status.
[0014] A workflow-based spatiotemporal dynamic perception data aggregation method is implemented based on the workflow-based spatiotemporal dynamic perception data aggregation system as described in any one of the above items, and includes the following steps: Step 1: According to the transmission protocol and data format characteristics of the access data source and the specific storage requirements, select the appropriate receiving, processing and storage tools from the system tool library, and drag and drop the tool graphic elements corresponding to the tools into the visual drag-and-drop editing module through visual operations; Step 2: Using the visual drag-and-drop editing module, the selected tool graphics are connected in series and parameterized to construct a workflow for completing the data aggregation process; Step 3: After the workflow is created, verify and save it in the visual drag-and-drop editing module, and execute the entire workflow once; Step 4: Set a scheduled task for the workflow in the workflow management module, define the trigger parameters, start the scheduled task after the task is set, and start periodic aggregation of long-term data; Step 5: Manage each scheduled task through the scheduled task supervision module. The scheduled task information of the workflow is synchronously stored and displayed in the scheduled task supervision module. The scheduled task supervision module monitors the execution of each task and grasps the task execution status.
[0015] The above-mentioned workflow-based spatiotemporal dynamic perception data aggregation method also includes step 6: when a task execution fails, enter the log management module to read the detailed log, analyze the task input and output parameters and abnormal conditions, and use the visual drag and drop editing module to adaptively adjust the workflow configuration for problems caused by changes in external conditions of the data source and storage environment. After verification, save it, and restart the planned task in the workflow management module to restore the normal execution of the data aggregation process.
[0016] The beneficial effects of the workflow-based spatiotemporal dynamic perception data aggregation system and method of the present invention are: The system has improved user-friendliness by introducing a visual drag-and-drop editing module. Users can flexibly build and edit data aggregation processes through an intuitive graphical interface, which greatly reduces the system's usage threshold and improves user experience.
[0017] Through the comprehensive use of various modules, the system can flexibly adapt to diverse data sources and complex processing requirements, allowing users to adjust the data processing process in real time to ensure the high adaptability of the system in different application scenarios.
[0018] Through the planned task scheduling function, the system can realize the automatic access and processing of long time series data. Users can flexibly set task execution plans and adjust execution time and trigger conditions according to actual business needs.
[0019] By using the log management module for real-time recording and feedback, the system has a complete log management function, which records in detail the key indicators and abnormal situations during the task execution. Through log feedback, users can quickly discover and solve problems, adjust the aggregation process configuration, and ensure continuous and stable data access. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 This is a schematic diagram of the system structure in Example 1 of the present invention; Figure 2 This is a schematic diagram of the method flow in Example 2 of the present invention. DETAILED DESCRIPTION
[0021] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention is described below in conjunction with specific implementation methods and drawings.
[0022] Example 1 like Figure 1 As shown, a workflow-based spatiotemporal dynamic perception data aggregation system includes a tool library module, a visual drag editing module, a workflow management module, a planned task supervision module and a log management module. Each module and the modules are connected by signals or linear data.
[0023] Tool library module: includes predefined and reusable tools for data reception, processing and storage, which are used to perform adaptation access and conversion processing operations on data streams of different protocols and formats. The tools carry the functional logic of specific operations and are implemented as an executable script code, which is intuitively displayed through graphical interface elements.
[0024] The tool library module contains various predefined and reusable tools for data reception, processing and storage. These tools are key components for building data aggregation workflows and are used to perform adaptation, access and conversion processing operations on data streams of different protocols and formats.
[0025] Each tool carries specific functional logic and is implemented as an executable script code, and is intuitively displayed through graphical interface elements to facilitate user interaction.
[0026] The tool library module consists of four parts: data definition toolset, data reception toolset, data processing toolset and data storage toolset.
[0027] The Data Definition toolset provides tools for describing the structure of data reception, such as specifying the field content and type of the received data, setting the date and time conversion method, and constructing spatial geometry fields.
[0028] The data receiving toolset brings together a series of targeted receiving adaptation tools, supports mainstream IoT transmission protocols and data formats, and realizes accurate reception and parsing and conversion of diverse real-time data streams from different sensors and devices, such as Http tools, RabbitMQ tools, MQTT tools, etc.
[0029] The data processing toolset provides a series of specialized tools for real-time processing of received real-time data. These tools are divided into two categories according to their functions: data filtering and data conversion. Among them, the filtering tool focuses on accurately filtering out target information, such as geographic fences, from massive and constantly updated data streams based on spatial location coordinates and specific attribute information; the conversion tool is used to perform complex aggregation, splitting or reconstruction operations, including but not limited to field value calculation, spatial transformation and processing, etc., to meet the needs of subsequent analysis or services.
[0030] The data storage toolset is used to configure data storage strategies to save the accessed data information as structured data sets efficiently and orderly.
[0031] Visual drag-and-drop editing module: a visual interactive interface for building and editing data aggregation workflows. It connects multiple tools in the form of graphical interface elements to form a logical diagram of the execution process, namely the workflow. The workflow represents the complete aggregation process from data reception, parsing to processing and storage. The output of one tool is used as the input of the next tool, thereby realizing continuous data processing.
[0032] The visual drag-and-drop editing module contains three elements: tool graphics, connection lines, and execution parameters.
[0033] Tool graphics are graphical elements that are added to the editing module from the tool library by dragging and dropping, and can be freely laid out and arranged in the editing module.
[0034] Connecting lines are used to connect tool graphics in series and establish connection relationships between them to determine data flow and execution order.
[0035] Execution parameters refer to the parameter conditions set for tools or connectors in the workflow, which directly affect the accuracy of data aggregation results. Each tool involves some mandatory or optional parameters related to its function, such as input and output content, operating environment, trigger conditions and restrictions, etc., while the parameters of connectors usually include transmission conditions and exception handling mechanisms.
[0036] In addition, the visual drag-and-drop editing module also has an execute button and a save button. The function of the execute button is to trigger the execution process of the entire workflow once the workflow is created and configured, and to display the execution status of each tool, data transmission status, and possible error messages in real time to help verify the validity of the current workflow configuration and the accuracy of data aggregation. The save button is responsible for saving all configuration contents of the current workflow, including tools, parameter settings, flowchart structure, and overall schematic layout, and storing them in the workflow management module with a custom name for subsequent editing or reuse.
[0037] Workflow management module: used to perform specific operations on each created logical diagram, including deletion, renaming, editing, import / export, and scheduled tasks. Scheduled tasks support logical diagrams to automatically execute their processing according to a predetermined plan and perform long-term access to spatiotemporal dynamic perception data.
[0038] Among them, the editing function can jump into the visual drag-and-drop editing module, and at the same time load the configuration details of the current workflow to be edited and display its corresponding process diagram. Users can adjust or modify the tools, parameter content or execution process in the workflow based on the graphical operation interface of the editing module.
[0039] The import / export function provides the ability to reuse and share workflows. It supports exporting the configuration information of created workflows as text files, so that when they need to be reused, the specific configuration can be loaded into the editing module page through the import function, avoiding the tedious operation of repeatedly building workflows.
[0040] The scheduled task function supports the workflow to automatically execute its processing according to the predetermined plan, thereby realizing long-term access to spatiotemporal dynamic perception data. In addition to setting the start time and end time of the task, this function also provides a variety of task triggers, such as execution when monitoring changes occur or automatic execution at specified time intervals. Users can select the appropriate trigger type based on data flow characteristics and business needs. When the workflow's scheduled task settings are completed, its task information will be saved to the scheduled task supervision module for management.
[0041] Delete and rename the tools used to adjust the workflow and the names of scheduled tasks to make them easier for users to understand.
[0042] Planned task supervision module: includes task management and task monitoring, which is used to monitor and manage the planned tasks of the logic diagram, monitor the execution status of the planned tasks, track the execution status of the planned tasks in real time, and monitor the key operating indicators of each task execution.
[0043] The management functions of task management include starting, pausing and deleting task operations. It should be noted that when a scheduled task is started, its execution process will not be affected by changes to the original workflow configuration. Only when the task is restarted will the system reload the latest workflow configuration and apply it to subsequent task execution. In addition, this module also supports editing the settings of scheduled tasks. Users can flexibly modify parameters such as trigger type, loop interval, expiration time, etc. to meet the task scheduling needs in different scenarios.
[0044] In terms of task monitoring, this module tracks the execution status of scheduled tasks in real time and monitors the key operating indicators of each task execution, including task progress, execution time, access data volume, and current execution status. After each task is completed, the system records the execution results and related operating indicators in historical data, and updates the execution overview of scheduled tasks in real time, providing statistics on successful, failed, and ongoing tasks. In addition, this module also provides the most recent completion times of scheduled tasks and the next estimated start time, helping users to fully understand the progress of task execution.
[0045] Log management module: comprehensively record the execution process of scheduled tasks, generate detailed execution logs, capture key data of scheduled tasks, and accurately filter execution logs according to query parameters of scheduled tasks.
[0046] This module can record key data such as the task start time, end time, execution status, possible abnormal situations during execution, and captured task input and output parameters. It also provides multi-dimensional filtering functions, supporting accurate screening of logs based on time range, task type, execution results and other conditions to improve the availability of logs. In addition, this module supports log browsing, allowing users to view the details of the log and quickly locate the required information through paging or search functions. This module also has a log deletion function, and users can set log retention policies or manually delete specific log records to optimize system storage resources and keep the log library tidy.
[0047] Example 2 This embodiment mainly explains the method of data aggregation using the system in Embodiment 1.
[0048] A workflow-based spatiotemporal dynamic perception data aggregation method is implemented based on the workflow-based spatiotemporal dynamic perception data aggregation system in 1, and includes the following steps.
[0049] Step 1: According to the transmission protocol, data format and other characteristics of the access data source, combined with specific storage requirements, select appropriate receiving, processing and storage tools from the tool library of the system tool module, and through visual operations, drag and drop the tool graphic elements corresponding to the tool into the visual drag and drop editing module.
[0050] Step 2: Using the visual drag-and-drop editing module, the selected tool graphics are connected in series and parameterized to construct a workflow for completing the data aggregation process.
[0051] Manually adjust the overall layout and execution order of tool graphics in the editor, then use connecting lines to connect tool graphics in series, establish connection relationships between tools to determine the data flow, and finally select tool graphics and connecting lines, and enter execution parameters or conditions related to their functions.
[0052] Step 3: After the workflow is created, verify and save it in the visual drag-and-drop editing module, and execute the entire workflow once.
[0053] Specifically, the whole process is first executed once to verify the validity of the workflow configuration. If the execution is successful, the current configuration is saved and stored and displayed in the workflow management module. If the execution fails, the workflow is adjusted according to the error message until it passes the verification and then saved.
[0054] Step 4: Set up a scheduled task for the workflow in the workflow management module, define the trigger parameters, and start the scheduled task after it is set up to start periodic aggregation of long-term data.
[0055] The trigger parameters include trigger type, cycle interval, expiration time and other parameters. During the long-term periodic aggregation of data, the workflow's scheduled task information will also be synchronously stored and displayed in the scheduled task supervision module.
[0056] Step 5: Manage each scheduled task through the scheduled task supervision module. Since the scheduled task information of the workflow in step 4 is synchronously stored and displayed in the scheduled task supervision module, the scheduled task supervision module monitors the execution of each task and grasps the task execution status. In response to changing needs, the execution time or triggering conditions of the scheduled task can be adjusted in this module.
[0057] In order to facilitate users to view the data of related workflows in a timely manner, step 6 is also included: when a task fails to execute, enter the log management module to read the detailed log, analyze the task input and output parameters and abnormal situations, and use the visual drag and drop editing module to adaptively adjust the workflow configuration for problems caused by changes in external conditions of the data source and storage environment. After verification, save it, and restart the planned task in the workflow management module to restore the normal execution of the data aggregation process.
[0058] The above embodiments are only for illustrating the inventive concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.
Claims
1. A workflow-based spatiotemporal dynamic perception data aggregation system, characterized by: include, Tool library module: includes predefined and reusable tools for data reception, processing and storage, and is used to perform adaptation access and conversion processing operations on data streams of different protocols and formats. The tool carries the functional logic of the operation and is implemented as an executable script code, which is intuitively displayed through graphical interface elements; Visual drag-and-drop editing module: a visual interactive interface for building and editing data aggregation workflows, connecting multiple tools in series in the form of graphical interface elements to form a logical diagram of the execution process, representing the complete aggregation process from data reception, parsing to processing and storage. The output of one tool is used as the input of the next tool, thereby achieving continuous data processing; Workflow management module: used to perform specific operation management on each created logical diagram, including deletion, renaming, editing, import / export, and scheduled tasks. The scheduled tasks support the logical diagram to automatically execute its processing according to the predetermined plan and perform long-term access to spatiotemporal dynamic perception data; Planned task supervision module: used to supervise and manage the planned tasks of the logic diagram, monitor the execution status of the planned tasks, track the execution status of the planned tasks in real time, and monitor the key operating indicators of each task execution; Log management module: comprehensively record the execution process of scheduled tasks, generate detailed execution logs, capture key data of scheduled tasks, and accurately filter execution logs according to query parameters of scheduled tasks.
2. The workflow-based spatiotemporal dynamic perception data aggregation system according to claim 1 is characterized in that: The tool library module includes: Data definition toolset: provides tools for describing the data receiving structure, including specifying the field content and type of the received data, setting the date and time conversion method, and constructing the spatial geometry field; Data receiving toolset: It brings together targeted receiving and adapting tools, supports mainstream IoT transmission protocols and data formats, and enables accurate reception and analysis and conversion of diverse real-time data streams from different sensors and devices, including HTTP tools, RabbitMQ tools, and MQTT tools. Data processing tool set: Provides specialized tools for immediate processing of received real-time data, including data filtering tools and data conversion tools; Data storage toolset: used to configure data storage strategies to save accessed data information as structured data sets efficiently and orderly.
3. The workflow-based spatiotemporal dynamic perception data aggregation system according to claim 2 is characterized in that: The data filtering tool focuses on accurately filtering out target information from massive and constantly updated data streams based on spatial location coordinates and attribute information; The data conversion tools are used to perform complex aggregation, splitting, and reconstruction operations, including field value calculation, spatial transformation, and processing.
4. The workflow-based spatiotemporal dynamic perception data aggregation system according to claim 1 is characterized in that: The visual drag-and-drop editing module includes three elements: tool graphics, connecting lines, and execution parameters. The tool graphics are graphical elements added to the visual drag-and-drop editing module from the tool library through drag-and-drop operations, and are freely laid out and arranged in the visual drag-and-drop editing module; the connecting lines are used to connect the tool graphics in series and establish connection relationships between them to determine data flow and execution order; the execution parameters are parameter conditions set for tool graphics or connecting lines in the logic diagram.
5. The workflow-based spatiotemporal dynamic perception data aggregation system according to claim 4 is characterized in that: The visual drag-and-drop editing module also includes an execution button element and a save button element. The execution button element is used to trigger the execution process of the entire logic diagram once the logic diagram is created and configured, and to display the execution status of each tool, data transmission status, and possible error messages in real time to help verify the validity of the current logic diagram configuration and the accuracy of data aggregation; the save button element is used to save all configuration contents of the current logic diagram and store it in the workflow management module with a custom name to facilitate subsequent editing or reuse.
6. The workflow-based spatiotemporal dynamic perception data aggregation system according to claim 1 is characterized in that: In the workflow management module, the editing jump is used to enter the visual drag-and-drop editing module, and the configuration details of the current logic diagram to be edited are loaded at the same time, and its corresponding process diagram is displayed. The user can adjust or modify the tools, parameter content or execution process in the logic diagram based on the graphical operation interface of the visual drag-and-drop editing module; export the configuration information of the created logic diagram as a text file, and load the specific configuration into the visual drag-and-drop editing module page through import; when the scheduled task setting of the logic diagram is completed, the scheduled task information will be saved to the scheduled task supervision module for management.
7. The workflow-based spatiotemporal dynamic perception data aggregation system according to claim 1 is characterized in that: The planned task supervision module includes task management and task monitoring. The task management includes starting tasks, pausing tasks, and deleting tasks. After the planned task is started, its execution process will not be affected by changes to the original logical diagram configuration. When the planned task is restarted, the system will reload the latest logical diagram configuration and apply it to subsequent task executions.
8. The workflow-based spatiotemporal dynamic perception data aggregation system according to claim 1 is characterized in that: The key operating indicators include task progress, execution time, access data volume and current execution status.
9. A workflow-based spatiotemporal dynamic perception data aggregation method, characterized in that: The workflow-based spatiotemporal dynamic perception data aggregation system according to any one of claims 1 to 8 is implemented, comprising the following steps: Step 1: According to the transmission protocol and data format characteristics of the access data source and the specific storage requirements, select the appropriate receiving, processing and storage tools from the system tool library, and drag and drop the tool graphic elements corresponding to the tools into the visual drag-and-drop editing module through visual operations; Step 2: Using the visual drag-and-drop editing module, the selected tool graphics are connected in series and parameterized to construct a logical schematic diagram for completing the data aggregation process; Step 3: After the logic diagram is created, verify and save it in the visual drag-and-drop editing module, and execute the logic diagram as a whole once; Step 4: Set a scheduled task for the logic diagram in the workflow management module, define the trigger parameters, start the scheduled task after the setting is completed, and start the long-term periodic aggregation of data; Step 5: Manage each planned task through the planned task supervision module. The planned task information of the logic diagram is synchronously stored and displayed in the planned task supervision module. The planned task supervision module monitors the execution of each task and grasps the task execution status.
10. The workflow-building spatiotemporal dynamic perception data aggregation method according to claim 9, characterized in that: It also includes step 6: When a task fails to execute, enter the log management module to read the detailed log, analyze the task input and output parameters and abnormal situations, and use the visual drag and drop editing module to adaptively adjust the logic diagram configuration for problems caused by changes in external conditions of the data source and storage environment. After verification, save it, and restart the planned task in the workflow management module to restore the normal execution of the data aggregation process.
Citation Information
Patent Citations
Workflow engine method suitable for foundation enhanced network operation and maintenance scene and workflow engine system thereof
CN106952024A
Multi-objective optimization platform based on low codes and establishment method thereof
CN115328465A
Big data hyper-fusion visual management method
CN117708223A
Big data platform task management method and system
CN118034887A
