Data analysis flow construction method and device

Abstract

The application relates to the technical field of data processing, and provides a data analysis process construction method and device. The method comprises the following steps: in response to an operation signal of a node, a plurality of target nodes for constructing a data analysis process are acquired, each target node comprises a node selected from a node database or a node selected from a graph network stored in a graph network database, at least one model node is included in each target node, and the model node is used for calling a corresponding model service instance to perform inference analysis on input data of the input model node; a target graph network representing a data analysis process is obtained according to a dependency relationship between the target nodes, and the target graph network is stored in a graph network database; and a data analysis process is generated according to the target graph network. The data analysis process construction method provided in the application embodiment can decouple the construction of the data analysis process from business application, can cross-business reuse the analysis process, and can quickly respond to changes in data analysis requirements.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a method and apparatus for constructing a data analysis workflow. Background Technology

[0002] In complex analysis scenarios such as power grid equipment condition monitoring or industrial fault diagnosis, a single artificial intelligence model is often insufficient to meet business needs. It is usually necessary to combine multiple AI models with different functions to build a collaborative analysis process to complete the full data analysis task from data collection and feature extraction to comprehensive judgment.

[0003] In related technologies, the construction of data analysis workflows typically involves directly writing the collaborative logic between multiple models into the code of the business application. This includes hard-coding the model call order, data flow paths, and intermediate processing logic at the application layer to construct the data analysis workflow. However, this approach results in a highly coupled workflow with the business application, making it unusable across different business areas. When similar multi-model collaborative analysis tasks need to be performed on different data, a complete set of model call and collaboration logic code must be independently developed for each data analysis task to construct the corresponding data analysis workflow. This leads to long response cycles for adjusting the data analysis workflow, making it difficult to adapt to rapid changes in data analysis needs. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in related technologies. To this end, this application proposes a data analysis process construction method that decouples the construction of the data analysis process from business applications, enabling the analysis process to be reused across business operations and quickly responding to changes in data analysis needs.

[0005] The data analysis process construction method according to the first aspect of this application includes: In response to operation signals on nodes, acquire multiple target nodes used to construct data analysis workflows; Based on the dependencies between the target nodes, the target graph network representing the data analysis process is obtained and stored in the graph network database; The data analysis process is generated based on the target graph network; Each of the target nodes includes nodes selected from the node database or nodes selected from the graph network stored in the graph network database. Each of the target nodes includes at least one model node, which is used to call the corresponding model service instance to perform inference analysis on the input data input to the model node; The dependency relationship is used to define the execution order and data flow path between the target nodes.

[0006] By responding to operation signals on nodes, multiple target nodes for constructing the data analysis process are retrieved from a node database or a graph network stored in a graph network database. Based on the dependencies between these target nodes, a target graph network representing the data analysis process is obtained and stored in the graph network database. The data analysis process is then generated based on this target graph network. This changes the data analysis process construction method from hard-coding to a configurable definition based on graph networks. When performing similar multi-model collaborative analysis tasks on different business data, existing nodes or graph networks can be directly called or reused to generate the analysis process, eliminating the need to rewrite model calling and collaboration logic code for each task. Furthermore, when analysis requirements change, only the nodes or their dependencies need to be adjusted to redefine the process, without modifying the business application code. This decouples the construction of the data analysis process from business applications, enabling the analysis process to be reused across business lines and quickly responding to changes in data analysis requirements.

[0007] According to one embodiment of this application, each of the target nodes further includes at least one rule node, wherein the rule node is configured with preset judgment conditions; The rule node is used to determine the corresponding output data based on the judgment result of the input data of the rule node according to the judgment condition, and to route the output data to the target node associated with the judgment result.

[0008] According to one embodiment of this application, the rule node is further configured with a callback function; The rule node is used to trigger the corresponding callback function based on the judgment result to process the input data, obtain the corresponding output data, and route the output data to the target node associated with the callback function.

[0009] According to one embodiment of this application, the model node includes a first model node or a second model node; The first model node is used to call the corresponding model service instance to obtain the corresponding feature data based on the input data input to the first model node, and output the output data including the feature data to the downstream target node; The second model node is used to call the corresponding model service instance to generate corresponding analysis results based on the input data input to the second model node and output them to the downstream target node.

[0010] According to one embodiment of this application, the model node is configured with at least one instance of the model service; The model node is used to call the model service instance whose current load is less than or equal to a preset load, based on the current load of each configured model service instance, to perform inference analysis on the input data input to the model node.

[0011] According to one embodiment of this application, it also includes: Detect the registration information of the model service instance configured in the model node; If the registration information of the model service instance stops refreshing within multiple consecutive registration periods, then the model service instance is deleted. When the model service instance is enabled, its registration information is refreshed according to the registration period.

[0012] According to one embodiment of this application, it also includes: According to the data analysis process, a task database is determined for storing the execution records of each target node corresponding to the data analysis process, so as to associate and store the execution records generated by each target node, the record identifiers corresponding to the execution records, and the task identifiers of the data analysis process in the task database.

[0013] The data analysis process construction apparatus according to the second aspect of the present application includes: The node acquisition module is used to acquire multiple target nodes for constructing data analysis processes in response to operation signals on nodes; The network generation module is used to obtain a target graph network representing the data analysis process based on the dependency relationships between the target nodes and store it in the graph network database. The process construction module is used to generate the data analysis process based on the target graph network; Each of the target nodes includes nodes selected from the node database or nodes selected from the graph network stored in the graph network database. Each of the target nodes includes at least one model node, which is used to call the corresponding model service instance to perform inference analysis on the input data input to the model node; The dependency relationship is used to define the execution order and data flow path between the target nodes.

[0014] An electronic device according to a third aspect of this application includes a processor and a memory storing a computer program, wherein the processor executes the computer program to implement the data analysis process construction method described in any of the above embodiments.

[0015] A computer-readable storage medium according to a fourth aspect of this application stores a computer program thereon, which, when executed by a processor, implements the data analysis process construction method described in any of the above embodiments. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A first flowchart illustrating the data analysis workflow construction method provided in this application embodiment; Figure 2 This is a first structural schematic diagram of a graph network provided in an embodiment of this application; Figure 3 This is a schematic diagram of the second structure of a graph network provided in an embodiment of this application; Figure 4 This is a schematic diagram of the third structure of a graph network provided in an embodiment of this application; Figure 5 A second flowchart illustrating the data analysis workflow construction method provided in this application embodiment; Figure 6 This is a schematic diagram of the data analysis process construction device provided in the embodiments of this application; Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0020] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0023] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), and similarly, "multiple items" refers to two or more (including two).

[0024] The data analysis process construction method and apparatus provided in this application will be described in detail and explained through several specific embodiments.

[0025] In complex analytical scenarios such as power grid equipment condition monitoring or industrial fault diagnosis, a single artificial intelligence model is often insufficient to meet business needs. It is typically necessary to combine multiple AI models with different functions to construct a collaborative analysis process, thereby completing the entire data analysis task from data acquisition and feature extraction to comprehensive judgment. For example, in transformer condition assessment, it may be necessary to sequentially call the dissolved gas analysis model in oil, the partial discharge identification model, and the comprehensive fault diagnosis model. These models exchange data and work collaboratively in a preset order to complete the transformer condition assessment.

[0026] In related technologies, the construction of data analysis workflows typically involves directly writing the collaborative logic between multiple models into the code of the business application. This includes hard-coding the model call order, data flow paths, and intermediate processing logic at the application layer to construct the data analysis workflow. However, this approach results in a highly coupled workflow with the business application, making it unusable across different business areas. When similar multi-model collaborative analysis tasks need to be performed on different data, a complete set of model call and collaboration logic code must be independently developed for each data analysis task to construct the corresponding data analysis workflow. This leads to long response cycles for adjusting the data analysis workflow, making it difficult to adapt to rapid changes in data analysis needs.

[0027] In view of this, the embodiments of this application, in response to operation signals on nodes, obtain multiple target nodes for constructing a data analysis process from a node database or a graph network stored in a graph network database, and, based on the dependencies between the target nodes, obtain a target graph network representing the data analysis process and store it in the graph network database, so as to generate the data analysis process based on the target graph network. This changes the construction method of the data analysis process from hard-coding to a configurable definition based on graph networks. This allows for the direct calling or reuse of existing nodes or graph networks to generate the analysis process when similar multi-model collaborative analysis tasks need to be performed on different business data, without the need to rewrite model calling and collaborative logic code for each task. Furthermore, when analysis requirements change, only the nodes or their dependencies need to be adjusted to redefine the process, without modifying the business application code. This decouples the construction of the data analysis process from the business application, enabling the analysis process to be reused across business operations and quickly responding to changes in data analysis requirements.

[0028] According to some embodiments of this application, this application provides a method for constructing a data analysis workflow. This method can be applied to a terminal device to implement the construction of a data analysis workflow. The terminal device can be a mobile terminal or a desktop terminal, such as a laptop or desktop computer. Figure 1 As shown, the data analysis workflow construction method provided in this embodiment includes: S101, in response to the operation signal on the node, acquires multiple target nodes for constructing the data analysis process; S102, Based on the dependencies between the target nodes, the target graph network representing the data analysis process is obtained and stored in the graph network database; S103, Generate the data analysis process based on the target graph network; Each of the target nodes includes nodes selected from the node database or nodes selected from the graph network stored in the graph network database. Each of the target nodes includes at least one model node, which is used to call the corresponding model service instance to perform inference analysis on the input data input to the model node; The dependency relationship is used to define the execution order and data flow path between the target nodes.

[0029] In some embodiments, the operation signal for a node can be a user's selection operation on a node in a node database containing multiple nodes, such as dragging a node from the node database to the editing area of ​​the interface; the dragged node is the target node. Alternatively, the user can select a stored graph network from a graph network database and reuse at least one node from the graph network; the reused node is the target node. The target node includes a model node, which is configured with a model service instance generated from the model configuration file of the AI ​​model. For example, the user can configure the configuration parameters of an AI model, such as DeepSeek or GPT, to generate a model configuration file for that AI model, allowing the AI ​​model to focus on inference and analysis of a specific type of data, such as equipment temperature, voltage waveform, or current waveform. Then, the model configuration file is run to form a corresponding model service instance, such as a service instance for analyzing equipment temperature in the power grid, or a service instance for analyzing the opening and closing of switchgear in the power grid, and the model service instance is configured to the corresponding model node. Each model node is configured with a corresponding task executor; the input data to the model node can refer to the input data to the task executor of that model node. Once the task executor of a model node receives input data, it can call the model service instance corresponding to that model node through the task executor to perform inference analysis on the input data of the input model node.

[0030] In some embodiments, different model nodes can be described using fields in a uniform format, such as using text to describe the basic information of different model nodes, so as to facilitate the identification of each model node.

[0031] In some embodiments, for each target node, the dependency relationship between target nodes can be defined in response to association operations between target nodes, such as connection operations between target nodes; alternatively, the dependency relationship between target nodes can be automatically derived based on the matching relationship of data input and output between target nodes. For example, if a target node A is configured with a model service instance for analyzing the equipment temperature of the power grid, and another target node B is configured with a model service instance for inferring the equipment temperature of the power grid, then target node A depends on target node B. In this case, target node B can be determined as the upstream node of target node A to obtain the dependency relationship between target node B and target node A.

[0032] like Figure 2 As shown, based on the dependencies between target nodes, connection edges representing these dependencies can be generated between them. These edges represent the execution order and data flow paths between the target nodes, thus forming a target graph network that represents the data analysis process. The data flow path indicates that the output data of the upstream target node is used as the input data of the downstream target node, and data is transferred according to the dependencies. If an upstream target node has multiple outputs, the corresponding output data can be selected for transfer based on the input requirements of the downstream target node.

[0033] For example, assuming that target node A depends on target node B, a connection edge can be formed between target node B and target node A to indicate that target node B is executed before target node A, and the output data of target node B is transferred to target node A as input data of target node A.

[0034] After generating the target graph network, it can be stored in a graph network database. When storing the target graph network in the database, it can be stored in the form of a node data table and an edge data table. The node data table stores the node identifier and configuration information of each target node, while the edge data table stores the dependency information between the target nodes.

[0035] When the target graph network is invoked to execute a data analysis task, a corresponding data analysis process is generated to analyze the data. If the upper-layer application transmits the task ID and input data of the data analysis task according to the interface requirements via an interface, the API gateway can route the input data to the task executor of the target node according to the data analysis process generated by the target graph network. After the task executor executes the data processing or analysis task of the target node, the corresponding analysis data is output.

[0036] By responding to operation signals on nodes, multiple target nodes for constructing the data analysis process are retrieved from a node database or a graph network stored in a graph network database. Based on the dependencies between these target nodes, a target graph network representing the data analysis process is obtained and stored in the graph network database. The data analysis process is then generated based on this target graph network. This changes the data analysis process construction method from hard-coding to a configurable definition based on graph networks. When similar multi-model collaborative analysis tasks need to be performed on different business data, existing nodes or graph networks can be directly called or reused to generate the analysis process, without rewriting model calling and collaboration logic code for each task. Furthermore, when analysis requirements change, only nodes or their dependencies need to be adjusted to redefine the process, without modifying the business application code. This decouples the construction of the data analysis process from business applications, enabling the analysis process to be reused across business lines and quickly respond to changes in data analysis requirements.

[0037] To further improve the flexibility of data analysis process construction and enable the constructed data analysis process to adapt to more complex scenarios, in some embodiments, each of the target nodes also includes at least one rule node, and the rule node is configured with preset judgment conditions. The rule node is used to determine the corresponding output data based on the judgment result of the input data of the rule node according to the judgment condition, and to route the output data to the target node associated with the judgment result.

[0038] In some embodiments, users can configure judgment conditions according to actual needs, encapsulating the configured judgment conditions as intermediate functions to form rule nodes. The types of judgment conditions can include: simple comparison conditions, used to compare input data with a preset threshold; composite logic conditions, used to logically combine multiple simple comparison conditions; and script conditions, used to judge input data by executing a preset script. The judgment result of each judgment condition is associated with a corresponding target node. For example, a rule node can be configured with the following judgment condition: if the confidence level of the inference result is >90%, it is passed to downstream node C; otherwise, it is passed to downstream node D.

[0039] Considering the numerous devices and data types involved in power grid data analysis, which often require collaborative analysis of various data types, and the varying acquisition cycles for different data types (e.g., transformer oil chromatography data acquisition cycle is one day, while transformer core grounding current acquisition cycle is typically on the order of minutes), to avoid the inability to utilize multiple data types for collaborative analysis due to differing acquisition cycles, in some embodiments, at least one rule node is configured with a judgment condition to determine whether all types of input data meeting the current data analysis requirements have been received. If such input data is received, the input data is output to the next target node; otherwise, the currently received input data is updated. This rule node can be configured as the starting node of the target graph network to determine whether all types of input data meeting the current data analysis requirements have arrived. This improves the reliability of data analysis processes that require the use of multiple data types.

[0040] By introducing rule nodes that can configure judgment conditions, the data analysis process is equipped with the ability to perform condition judgments and dynamic routing on input data. It can transfer data to different downstream nodes according to preset judgment conditions, thereby improving the flexibility of the data analysis process construction and enabling the constructed data analysis process to adapt to more complex scenarios.

[0041] In order to enable the constructed data analysis process to adapt to more complex scenarios, in some embodiments, the rule node is also configured with a callback function; The rule node is used to trigger the corresponding callback function based on the judgment result to process the input data, obtain the corresponding output data, and route the output data to the target node associated with the callback function.

[0042] In some embodiments, the callback function is a function pre-registered in the rule node configuration. It can be stored as a parameter in the rule node definition and dynamically invoked according to the judgment conditions. Simultaneously, the callback function can be pre-associated with the corresponding node. For example, callback function A could be: to perform a standardized format conversion on the current inference result and then pass it to the downstream node used for fault diagnosis.

[0043] The judgment conditions configured for rule nodes can be pre-configured with corresponding callback functions. For example, the judgment conditions for a rule node can be configured as follows: if the confidence level of the inference result is > 90%, then execute callback function A; otherwise, execute callback function B.

[0044] For example, taking the data analysis process for multi-time-series balance analysis and decision-making in a smart power grid dispatch scenario as an example, rule nodes can be configured with judgment conditions to determine whether the zonal balance margin calculated from the input multi-source data, such as load forecast data, new energy forecast output, and tie-line plans, is lower than a preset safety lower limit threshold. When the judgment result is that the zonal balance margin is insufficient, a pre-configured callback function is triggered to automatically process the input data. For example, based on the margin gap, inter-provincial mutual assistance transaction price signals, and available transmission capacity, the required electricity purchase volume and suggested price range for the inter-provincial spot market are calculated in real time, and the output data containing the processing result is routed to the node associated with the callback function.

[0045] By configuring callback functions in rule nodes, when the judgment conditions are met, the process can not only complete data routing, but also actively trigger specific processing and transformation of the data. This enhances the real-time response and processing capabilities of rule nodes, allowing data routing and data processing actions to be tightly coupled. This enables the construction of more complex data analysis logic, making the constructed data analysis process adaptable to more complex scenarios.

[0046] In some embodiments, a rule node configured with judgment conditions and callback functions can serve as the start node, intermediate node, or end node of the target graph network. When a rule node serves as the start node of the target graph network, it can decompose the input data of multiple downstream target nodes by parsing the input data called through the judgment condition parsing interface. When a rule node serves as an intermediate node of the target graph network, it can be used to receive data output from upstream target nodes for judgment and processing. For example, in a data analysis process for filtering and spectrum analysis of switchgear opening and closing waveform data, a rule node can be deployed as an intermediate node of the target graph network to normalize the partial discharge spectrum data and perform median filtering, cropping, rotation, and mirroring on the image data. When a rule node serves as the end node of the target graph network, it can process multiple input data through judgment conditions and merge them into a single data for output.

[0047] To enable the data analysis process to be built to adapt to more complex scenarios, in some embodiments, the model node includes a first model node or a second model node. The first model node is used to call the corresponding model service instance to obtain the corresponding feature data based on the input data input to the first model node, and output the output data including the feature data to the downstream target node; The second model node is used to call the corresponding model service instance to generate corresponding analysis results based on the input data input to the second model node and output them to the downstream target node.

[0048] In some embodiments, the model service instance configured on the first model node is a model service instance that analyzes the input data to extract its feature data. For example, it may perform quantitative analysis on the input data to obtain features such as indicators, thresholds, trend slopes, and confidence levels. The model service instance configured on the second model node is a model service instance that performs feature analysis on the input data to generate a reviewable final analysis result, such as the comprehensive state level of the output power grid.

[0049] For example, the first model node can invoke a configured model service instance to clean and transform the raw current, voltage, and power data, extracting feature data that characterizes the power grid's operating status, such as harmonic distortion rate, load change trend slope, or voltage sag confidence level, thereby providing high-quality, high-density feature vectors for downstream in-depth analysis. The second model node, acting as a high-level decision analysis layer, receives output data from upstream target nodes, such as multi-dimensional feature data output from the first model node, and invokes more complex model service instances to comprehensively analyze the input data, outputting the final analysis results, such as the overall operating status level of the power grid, future short-term load forecast curves, or fault probability assessment reports for specific equipment.

[0050] For example, such as Figure 3 As shown, taking the intelligent analysis scenario of switchgear comprehensive status as an example, a target graph network can be formed by constructing dependencies between multiple rule nodes, multiple first model nodes (temperature rise identification, partial discharge diagram identification, switchgear opening and closing analysis), and second model nodes (large model). The feature data output by multiple first model nodes is judged by the rule nodes, and corresponding prompts are output to the second model nodes. The second model nodes then analyze the data based on these prompts to obtain the final status monitoring results.

[0051] Or, such as Figure 4 As shown, taking the intelligent analysis scenario of all-round condition monitoring of transformers as an example, the dependencies between multiple rule nodes, multiple first model nodes (DGA identification, partial discharge identification, core grounding current analysis, acoustic vibration analysis), and second model nodes (large model) can be constructed to form a target graph network. Among them, multiple first model nodes and multiple second models are parallel nodes, jointly participating in data analysis, and finally fused and output through rule nodes.

[0052] The data analysis process is constructed by using a first model node that generates feature data and a second model node that generates business analysis results. This decouples the acquisition of data features from the analysis of results at the process construction level, allowing users to flexibly combine different feature extraction and reasoning analysis capabilities. This results in greater reusability and the ability to construct more complex data analysis processes with clearer hierarchies, thereby enabling the constructed data analysis processes to adapt to more complex scenarios.

[0053] To improve the execution efficiency and stability of the constructed data analysis process, in some embodiments, the model node is configured with at least one instance of the model service; The model node is used to call the model service instance whose current load is less than or equal to a preset load, based on the current load of each configured model service instance, to perform inference analysis on the input data input to the model node.

[0054] In some embodiments, considering that model service instances may be repeatedly invoked at the same time, a model node can be configured with multiple model service instances. When a model node needs to process input data, the current load of each model service instance configured for that model node can be detected. From among these model service instances, a model service instance with a current load less than or equal to a preset load can be selected to perform inference analysis on the input data. The preset load can be set according to actual conditions. For example, the preset load can be the minimum current load among the current loads of each model service instance configured for the model node. Therefore, when a model node needs to process input data, the model service instance with the minimum current load is selected from among the model service instances to perform inference analysis on the input data.

[0055] Taking the data analysis process for load forecasting of the power grid as an example, multiple model service instances for load forecasting can be configured in the model node. During critical periods such as peak summer demand and significant fluctuations in renewable energy output, multiple data analysis processes may be triggered in parallel simultaneously, generating a large number of concurrent call requests to the load forecasting service. At this time, the model node automatically routes input data to model service instances with lower current loads by real-time sensing of the current load status of each model service instance. This avoids inference queuing or timeouts caused by single-instance overload, ensuring that the key forecast results required for scheduling decisions can be reliably returned within the specified time.

[0056] By prioritizing the use of model service instances with lower loads to perform inference analysis of input data based on the current load of each model service instance configured in the model node, dynamic load balancing at the model service call level is achieved. This effectively avoids inference delays or failures caused by single instance overload, thereby improving the execution efficiency and stability of the constructed data analysis process.

[0057] To improve the operational reliability of the constructed data analysis process, in some embodiments, the method further includes: Detect the registration information of the model service instance configured in the model node; If the registration information of the model service instance stops refreshing within multiple consecutive registration periods, then the model service instance is deleted. When the model service instance is enabled, its registration information is refreshed according to the registration period.

[0058] In some embodiments, the registration information of a model service instance is generated during the initial startup of the model service instance. After startup, the service instance registers with the task execution layer of the terminal device according to a fixed message format, generating registration information. When the model service instance is enabled, its registration information is refreshed at a fixed registration period, such as 60 seconds. After receiving a refresh message from the model service instance, the terminal device maintains a heartbeat for that model service instance. If no refresh message is received from the model service instance for several consecutive registration periods, such as three consecutive registration periods, it can be determined that the registration information of the model service instance has stopped refreshing. In this case, the model service instance can be considered offline and deleted.

[0059] For example, in the operation scenario of a power grid dispatch automation system, when a model service instance deployed on a terminal device to perform equipment anomaly detection is initialized and started, it registers its address, port, service capabilities, and other metadata information at the task execution layer of the terminal device, generating corresponding registration entries. During normal operation, this instance sends a heartbeat refresh message to the task execution layer every 60 seconds to indicate that it is still available. If the computing node hosting this instance becomes disconnected due to network interruption, process crash, or resource exhaustion, the task execution layer will not receive its heartbeat refresh for three consecutive registration cycles. At this time, the task execution layer determines that the instance is offline and unavailable, automatically deletes its registration information from the service registry, and reclaims related resources. Thus, when subsequent data analysis processes reach a node that needs to call the anomaly detection service, the task scheduling layer will not route the request to the failed instance, but will select other available instances or trigger alarms, thereby avoiding process blocking or missing analysis results caused by calling failed instances, ensuring the real-time performance and reliability of power grid fault early warning and dispatch decisions.

[0060] By periodically checking the registration information of model service instances and automatically cleaning up instances whose registration information has not been refreshed for a long time, it is ensured that the model service instances called by the data analysis process are all in an available state when the process is executed. This avoids process interruption or errors caused by calling invalid instances, thereby ensuring the reliability and robustness of the process operation.

[0061] To facilitate traceability of the data analysis process, in some embodiments, the method further includes: According to the data analysis process, a task database is determined for storing the execution records of each target node corresponding to the data analysis process, so as to associate and store the execution records generated by each target node, the record identifiers corresponding to the execution records, and the task identifiers of the data analysis process in the task database.

[0062] For example, in a power grid data analysis scenario, when a user triggers a data analysis process for a comprehensive assessment of regional power grid stability, the system generates a unique task identifier, such as TASK1, for this analysis task. The data analysis process corresponding to this task includes multiple model nodes, such as a first model node for load data cleaning and a second model node downstream of the first model node for stability level assessment. During process execution, the first model node calls a model service instance to denoise and standardize the input data, generating cleaned feature data as output. It then associates the node's execution record, such as input data, processing time, and output data, along with its corresponding record identifier, such as NODE1, with the task identifier TASK1 and writes it into the task database. Subsequently, the second model node receives the upstream output, calls an ensemble learning model for comprehensive analysis, and arrives at an analysis result of "Level II Warning." It also associates the node's execution record, such as input feature vectors, model version, judgment conclusion, and its record identifier NODE2, with the same task identifier TASK1 for storage.

[0063] In addition, for the execution records of each target node, task snapshots can be generated and displayed on the visualization page for users to query and browse.

[0064] Thus, when a user has doubts about the results of an evaluation or needs an audit, they can quickly retrieve and trace the complete execution chain of each model node involved in the task based on the task identifier in the task database. This includes the model service instance version called by each node, input and output data, and intermediate processing procedures. This solves the technical pain point in traditional power grid data analysis, where the complexity of the process and the lack of intermediate data make it difficult to accurately locate the root cause of the problem. It achieves transparent traceability of the entire analysis and decision-making process, supporting the reliability management requirements of power grid operation data.

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below. In some embodiments, such as Figure 5 As shown, the data analysis workflow construction method includes: S201, in response to operation signals on nodes, acquires multiple target nodes for constructing data analysis processes.

[0066] Each target node includes nodes selected from a node database or nodes selected from a graph network stored in a graph network database. Each target node includes at least one model node and at least one rule node. The model node is used to call the corresponding model service instance to perform inference analysis on the input data of the input model node. The rule node is configured with preset judgment conditions and callback functions. The rule node is used to trigger the corresponding callback function according to the judgment result to process the input data, obtain the corresponding output data, and route the output data to the target node associated with the callback function.

[0067] The model node includes a first model node or a second model node; the first model node is used to call the corresponding model service instance to obtain the corresponding feature data based on the input data input to the first model node, and output the output data including the feature data to the downstream target node; the second model node is used to call the corresponding model service instance to generate the corresponding analysis results based on the input data input to the second model node and output them to the downstream target node.

[0068] The model node is configured with at least one model service instance; the model node is used to call the model service instance whose current load is less than or equal to the preset load according to the current load of each configured model service instance, and to perform inference analysis on the input data of the input model node.

[0069] S202, based on the dependencies between each target node, obtain the target graph network representing the data analysis process and store it in the graph network database.

[0070] Among them, the dependency relationship is used to define the execution order and data flow path between each target node.

[0071] S203, Generate the data analysis process based on the target graph network.

[0072] S204, Detect the registration information of the model service instance configured in the model node.

[0073] When a model service instance is enabled, its registration information is refreshed according to the registration cycle.

[0074] S205, if it is determined that the registration information of the model service instance stops refreshing within multiple consecutive registration periods, the model service instance is deleted.

[0075] S206, Based on the data analysis process, determine a task database for storing the execution records of each target node corresponding to the data analysis process, so as to associate and store the execution records generated by each target node, the record identifiers corresponding to the execution records, and the task identifiers of the data analysis process in the task database.

[0076] The data analysis process construction apparatus provided in this application is described below. The data analysis process construction apparatus described below and the data analysis process construction method described above can be referred to in correspondence.

[0077] In one embodiment, such as Figure 6 As shown, a data analysis workflow construction device is provided, including: The node acquisition module 310 is used to acquire multiple target nodes for constructing a data analysis process in response to operation signals on the nodes. The network generation module 320 is used to obtain a target graph network representing the data analysis process based on the dependency relationships between the target nodes and store it in the graph network database. The process construction module 330 is used to generate the data analysis process based on the target graph network; Each of the target nodes includes nodes selected from the node database or nodes selected from the graph network stored in the graph network database. Each of the target nodes includes at least one model node, which is used to call the corresponding model service instance to perform inference analysis on the input data input to the model node; The dependency relationship is used to define the execution order and data flow path between the target nodes.

[0078] By responding to operation signals on nodes, multiple target nodes for constructing the data analysis process are retrieved from a node database or a graph network stored in a graph network database. Based on the dependencies between these target nodes, a target graph network representing the data analysis process is obtained and stored in the graph network database. The data analysis process is then generated based on this target graph network. This changes the data analysis process construction method from hard-coding to a configurable definition based on graph networks. When performing similar multi-model collaborative analysis tasks on different business data, existing nodes or graph networks can be directly called or reused to generate the analysis process, eliminating the need to rewrite model calling and collaboration logic code for each task. Furthermore, when analysis requirements change, only the nodes or their dependencies need to be adjusted to redefine the process, without modifying the business application code. This decouples the construction of the data analysis process from business applications, enabling the analysis process to be reused across business lines and quickly responding to changes in data analysis requirements.

[0079] In one embodiment, each of the target nodes further includes at least one rule node, wherein the rule node is configured with preset judgment conditions; The rule node is used to determine the corresponding output data based on the judgment result of the input data of the rule node according to the judgment condition, and to route the output data to the target node associated with the judgment result.

[0080] In one embodiment, the rule node is further configured with a callback function; The rule node is used to trigger the corresponding callback function based on the judgment result to process the input data, obtain the corresponding output data, and route the output data to the target node associated with the callback function.

[0081] In one embodiment, the model node includes a first model node or a second model node; The first model node is used to call the corresponding model service instance to obtain the corresponding feature data based on the input data input to the first model node, and output the output data including the feature data to the downstream target node; The second model node is used to call the corresponding model service instance to generate corresponding analysis results based on the input data input to the second model node and output them to the downstream target node.

[0082] In one embodiment, the model node is configured with at least one instance of the model service; The model node is used to call the model service instance whose current load is less than or equal to a preset load, based on the current load of each configured model service instance, to perform inference analysis on the input data input to the model node.

[0083] In one embodiment, the process construction module 330 is further configured to: Detect the registration information of the model service instance configured in the model node; If the registration information of the model service instance stops refreshing within multiple consecutive registration periods, then the model service instance is deleted. When the model service instance is enabled, its registration information is refreshed according to the registration period.

[0084] In one embodiment, the process construction module 330 is further configured to: According to the data analysis process, a task database is determined for storing the execution records of each target node corresponding to the data analysis process, so as to associate and store the execution records generated by each target node, the record identifiers corresponding to the execution records, and the task identifiers of the data analysis process in the task database.

[0085] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call computer programs stored in the memory 830 to execute data analysis process construction methods, such as including: In response to operation signals on nodes, acquire multiple target nodes used to construct data analysis workflows; Based on the dependencies between the target nodes, the target graph network representing the data analysis process is obtained and stored in the graph network database; The data analysis process is generated based on the target graph network; Each of the target nodes includes nodes selected from the node database or nodes selected from the graph network stored in the graph network database. Each of the target nodes includes at least one model node, which is used to call the corresponding model service instance to perform inference analysis on the input data input to the model node; The dependency relationship is used to define the execution order and data flow path between the target nodes.

[0086] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0087] On the other hand, this application also provides a storage medium, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data analysis process construction method provided in the above embodiments.

[0088] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0089] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for constructing a data analysis workflow, characterized in that, include: In response to operation signals on nodes, acquire multiple target nodes used to construct data analysis workflows; Based on the dependencies between the target nodes, the target graph network representing the data analysis process is obtained and stored in the graph network database; The data analysis process is generated based on the target graph network; Each of the target nodes includes nodes selected from the node database or nodes selected from the graph network stored in the graph network database. Each of the target nodes includes at least one model node, which is used to call the corresponding model service instance to perform inference analysis on the input data input to the model node; The dependency relationship is used to define the execution order and data flow path between the target nodes.

2. The data analysis workflow construction method according to claim 1, characterized in that, in, Each of the target nodes also includes at least one rule node, and the rule node is configured with preset judgment conditions; The rule node is used to determine the corresponding output data based on the judgment result of the input data of the rule node according to the judgment condition, and to route the output data to the target node associated with the judgment result.

3. The data analysis workflow construction method according to claim 2, characterized in that, in, The rule node is also configured with a callback function; The rule node is used to trigger the corresponding callback function based on the judgment result to process the input data, obtain the corresponding output data, and route the output data to the target node associated with the callback function.

4. The data analysis workflow construction method according to claim 1, characterized in that, in, The model node includes a first model node or a second model node; The first model node is used to call the corresponding model service instance to obtain the corresponding feature data based on the input data input to the first model node, and output the output data including the feature data to the downstream target node; The second model node is used to call the corresponding model service instance to generate corresponding analysis results based on the input data input to the second model node and output them to the downstream target node.

5. The data analysis workflow construction method according to claim 1 or 4, characterized in that, in, The model node is configured with at least one instance of the model service; The model node is used to call the model service instance whose current load is less than or equal to a preset load, based on the current load of each configured model service instance, to perform inference analysis on the input data input to the model node.

6. The data analysis workflow construction method according to any one of claims 1-4, characterized in that, Also includes: Detect the registration information of the model service instance configured in the model node; If the registration information of the model service instance stops refreshing within multiple consecutive registration periods, then the model service instance is deleted. When the model service instance is enabled, its registration information is refreshed according to the registration period.

7. The data analysis workflow construction method according to claim 1, characterized in that, Also includes: According to the data analysis process, a task database is determined for storing the execution records of each target node corresponding to the data analysis process, so as to associate and store the execution records generated by each target node, the record identifiers corresponding to the execution records, and the task identifiers of the data analysis process in the task database.

8. A data analysis workflow construction device, characterized in that, include: The node acquisition module is used to acquire multiple target nodes for constructing data analysis processes in response to operation signals on nodes; The network generation module is used to obtain a target graph network representing the data analysis process based on the dependency relationships between the target nodes and store it in the graph network database. A process construction module is used to generate the data analysis process based on the target graph network; Each of the target nodes includes nodes selected from the node database or nodes selected from the graph network stored in the graph network database. Each of the target nodes includes at least one model node, which is used to call the corresponding model service instance to perform inference analysis on the input data input to the model node; The dependency relationship is used to define the execution order and data flow path between the target nodes.

9. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the data analysis process construction method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data analysis process construction method according to any one of claims 1 to 7.

Images