Prediction and early warning method and device based on workflow engine technology

Abstract

The embodiment of the invention provides a forecasting and early warning method and device based on a workflow engine technology. The method is applied to the technical field of data processing and comprises the steps that codes are uploaded and verified through an algorithm registration interface, authorization is managed based on permission identification, and an algorithm library is constructed; monitoring a data source according to the algorithm feature index, and generating a DI trigger signal and task description when new data is acquired; analyzing data by using a language model and optimizing resource allocation; arranging task nodes by using a workflow engine; executing pre-verification to generate a task sequence; and analyzing the abnormity through the EI message and feeding back an optimization scheduling scheme. According to the invention, the task execution strategy is automatically adjusted according to the change of the real-time meteorological data, and the timeliness and accuracy of the forecasting and early warning information are ensured.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of data processing, and particularly relates to a forecasting and early warning method and device based on workflow engine technology. BACKGROUND

[0002] In existing forecasting and early warning systems, a variety of technologies have been widely applied to realize the collection, processing and early warning release of meteorological data. These systems usually execute forecasting and early warning tasks in a timing trigger mode, and coordinate the execution of multiple tasks through a simple task queue management mechanism. In terms of data processing, the system uses traditional database technology and batch processing to store and analyze meteorological data, and triggers the generation and release of early warning information based on preset threshold rules. At the same time, some systems also introduce basic workflow management functions to standardize the execution steps of forecasting and early warning tasks through predefined process templates.

[0003] However, the existing technology has the following shortcomings: first, the timing trigger mode lacks flexibility and cannot adjust the task execution strategy in a timely manner according to the changes in actual meteorological data; second, the simple queue management mechanism is inefficient in handling complex task dependency relationships and is difficult to realize parallel optimization of tasks; third, the traditional data processing method fails to fully utilize real-time computing resources, resulting in slow system response and low resource utilization; finally, the predefined workflow template lacks dynamic adaptability and cannot self-optimize and adjust according to actual running conditions. These problems seriously affect the timeliness and accuracy of the forecasting and early warning system. SUMMARY

[0004] The present disclosure provides a forecasting and early warning method and device based on workflow engine technology, which realizes intelligent scheduling of forecasting and early warning tasks and dynamic optimization of resource allocation by introducing a scheduling mechanism based on workflow engine, combining DI / EI message triggering and language model analysis, and significantly improves the execution efficiency and resource utilization of the system. This scheme can automatically adjust the task execution strategy according to the changes in real-time meteorological data, ensuring the timeliness and accuracy of the forecasting and early warning information.

[0005] According to a first aspect of the present disclosure, a prediction and early warning method based on workflow engine technology is provided, comprising: uploading and timeliness verifying a meteorological prediction and early warning algorithm through an algorithm registration interface, authorizing and managing different users via a permission identifier, thereby constructing a meteorological prediction and early warning algorithm library and generating an algorithm feature index; continuously monitoring a meteorological data source through a real-time message queue according to the algorithm feature index, generating a DI trigger signal when new meteorological data is obtained, and generating a real-time task description according to the DI trigger signal; performing time series analysis on historical meteorological data and real-time monitoring data through a language model using the real-time task description, quickly deploying task urgency and computing resources via an adaptive optimization algorithm, and obtaining a real-time scheduling scheme; according to the real-time scheduling scheme, using a workflow engine to parallelly arrange meteorological prediction and early warning task nodes, converting the trigger dependency and data flow conversion between task nodes into a workflow execution strategy; based on the workflow execution strategy, pre-condition verification and data transfer are performed on multiple meteorological prediction and early warning tasks, a priority task sequence is generated, and a task running state is output; according to the task running state, real-time recording and analysis of task delay and exceptions are performed through an EI message, and the analysis results are fed back to the language model for dynamically updating the real-time scheduling scheme.

[0006] According to a second aspect of the present disclosure, a prediction and early warning device based on workflow engine technology is provided, comprising:

[0007] A verification module is configured to upload and verify the code of a meteorological prediction and early warning algorithm through an algorithm registration interface, authorize and manage different users via a permission identifier, thereby constructing a meteorological prediction and early warning algorithm library and generating an algorithm feature index;

[0008] A monitoring module is configured to continuously monitor a meteorological data source through a real-time message queue according to the algorithm feature index, generate a DI trigger signal when new meteorological data is obtained, and generate a real-time task description according to the DI trigger signal;

[0009] An analysis module is configured to perform time series analysis on historical meteorological data and real-time monitoring data through a language model using the real-time task description, quickly deploy task urgency and computing resources via an adaptive optimization algorithm, and obtain a real-time scheduling scheme;

[0010] An arrangement module is configured to use a workflow engine to parallelly arrange meteorological prediction and early warning task nodes according to the real-time scheduling scheme, and convert the trigger dependency and data flow conversion between task nodes into a workflow execution strategy;

[0011] A transfer module is configured to perform pre-condition verification and data transfer on multiple meteorological prediction and early warning tasks based on the workflow execution strategy, generate a priority task sequence, and output a task running state.

[0012] a feedback module configured to record and analyze the task delay and abnormality in real time through the EI message according to the task running state, and feed back the analysis result to the language model for dynamically updating the real-time scheduling scheme.

[0013] The present disclosure realizes the standardized management of the meteorological forecasting and warning algorithm by introducing the algorithm registration interface in the forecasting and warning system, ensures the reliability and safety of the algorithm, effectively controls the access rights of different users to the algorithm through the permission identification mechanism, and improves the security of the system. The real-time message queue is used to continuously monitor the meteorological data source, and the DI triggering mechanism is combined to realize the timely response to new meteorological data, which greatly improves the real-time performance of the system. The language model is used to intelligently analyze the historical meteorological data and real-time monitoring data, and the adaptive optimization algorithm is combined to allocate resources, which improves the accuracy of task scheduling and the resource utilization efficiency. The parallel arrangement of the forecasting and warning task nodes is realized based on the workflow engine technology, the task execution process is optimized, and the overall processing efficiency of the system is significantly improved. In the task execution process, the reliability of task execution is ensured through the precondition verification and data transmission mechanism, and the generation of the priority task sequence guarantees the timely processing of the key tasks. Finally, the EI message is used to realize the real-time monitoring and analysis of the task abnormality, and the dynamic optimization capability of the language model is combined to continuously improve the scheduling scheme and constantly improve the operation quality of the system. The problems of poor real-time performance, low resource utilization rate and complex task management in the traditional system are effectively solved.

[0014] It should be understood that the content described in the summary section is not intended to limit or define key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0015] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail the following embodiments with reference to the attached drawings. The drawings are intended to better understand the present disclosure and do not limit the present disclosure. In the drawings, the same or similar reference numerals refer to the same or similar elements, and

[0016] Figure 1 A flowchart of a forecasting and warning method based on a workflow engine technology according to an embodiment of the present disclosure is shown;

[0017] Figure 2 A block diagram of a forecasting and warning device based on a workflow engine technology according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0018] In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only some of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present disclosure.

[0019] In addition, the term "and / or" herein is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects.

[0020] Figure 1 A flowchart of a forecasting and early warning method 100 based on a workflow engine technology in the embodiments of the present disclosure is shown, as shown in Figure 1 The method 100 includes the following steps:

[0021] S110: uploading code and timeliness verification of the meteorological forecasting and early warning algorithm through an algorithm registration interface, authorizing and managing different users via a permission identifier, thereby constructing a meteorological forecasting and early warning algorithm library and generating an algorithm feature index;

[0022] Optionally, the program code of the meteorological forecasting and early warning algorithm is subjected to syntax checking through a pre-defined format specification to obtain a code compliance test result; the code compliance test result is compared and analyzed with historical running timeliness data to generate an algorithm timeliness evaluation report; the algorithm timeliness evaluation report is used to score and calculate from three dimensions of algorithm execution efficiency, resource consumption and data processing capacity to obtain algorithm scoring data; a permission access matrix is established according to the algorithm scoring data to map user identity information and algorithm operation permissions to generate a permission identifier; a unique identification code is generated for each algorithm according to the permission identifier, and version information, calling interfaces and input and output parameters of the algorithm are recorded to form an algorithm feature description; the algorithm feature description is integrated into the meteorological forecasting and early warning algorithm library to establish an algorithm index table and generate the algorithm feature index.

[0023] In the syntax checking stage, the program code is analyzed using predefined format specifications. The format specifications include code indentation requirements, variable naming rules, function interface definition specifications, comment specifications, and error handling specifications. During the syntax checking process, the code is scanned line by line to identify syntax errors, undefined variables, and interface parameter mismatches, and a code compliance test result is generated. The code compliance test result records the types, locations, and severity of the problems in the code. In the timeliness verification stage, the code compliance test result is compared and analyzed with historical running timeliness data. The historical running timeliness data includes historical execution time, computing resource occupation, and data processing volume of the algorithm. Through comparative analysis, the execution efficiency and resource utilization of the algorithm are evaluated, and an algorithm timeliness evaluation report is generated. The algorithm timeliness evaluation report details the execution performance of the algorithm under different data sizes.

[0024] Based on the algorithm timeliness evaluation report, the algorithm is quantitatively evaluated from three dimensions: algorithm execution efficiency, resource consumption, and data processing capability. The algorithm execution efficiency dimension evaluates the time required by the algorithm to process a unit of data; the resource consumption dimension evaluates the CPU usage, memory occupation, and network bandwidth occupation during the algorithm's running; and the data processing capability dimension evaluates the algorithm's ability and stability to process large-scale data. Through comprehensive scoring calculation of these three dimensions, algorithm scoring data is obtained. The permission management link establishes a permission access matrix based on the algorithm scoring data. The permission access matrix defines the operation permissions of different users on the algorithm, including algorithm calling permission, modification permission, and review permission. By mapping user identity information and algorithm operation permissions, a permission identifier is generated. The permission identifier uses a unique coding method to ensure the accuracy and security of permission management. The algorithm feature management link generates a unique identification code for each algorithm based on the permission identifier. The algorithm identification code uses specific coding rules and contains information such as algorithm type, version number, and creation time. At the same time, the algorithm's version information, calling interface specification, and input / output parameter definition are recorded to form a complete algorithm feature description. The algorithm feature description details the functional characteristics, running environment requirements, and interface specifications of the algorithm.

[0025] Finally, the algorithm feature description is integrated into the meteorological forecast and warning algorithm library to establish an algorithm index table. The algorithm index table uses a relational data structure to record the association and calling dependency between algorithms, facilitating quick retrieval and calling of algorithms. Through the algorithm index table, an algorithm feature index is generated, providing basic support for subsequent algorithm calling and task scheduling.

[0026] Taking the temperature prediction algorithm as an example, when the algorithm developer submits a newly developed temperature prediction algorithm through the algorithm registration interface, first, the algorithm code is checked for syntax. The check finds that there are problems such as non-standard variable naming and mismatching of interface parameter types in the code, and a code compliance inspection result is generated. The algorithm developer modifies the code according to the inspection result and resubmits it. The modified code passes the syntax check and enters the timeliness verification stage. Through comparative analysis, it is found that the algorithm shortens the average execution time by 30% and reduces the memory occupancy by 20% when processing 1-hour resolution temperature data compared with the existing algorithm, and can stably process temperature data containing 100 weather stations, and generate an algorithm timeliness evaluation report.

[0027] Based on the evaluation report, three dimensions are scored: execution efficiency score 0.9 (full score 1.0), resource consumption score 0.8, and data processing capacity score 0.85. The weighted calculation gives a comprehensive score of 0.85. According to the score result, the algorithm is classified into the high-performance algorithm category, and the algorithm developer and system administrator are granted complete algorithm operation permissions. A unique identification code "TEMP PRED V120231201” is generated for the algorithm, recording algorithm version, input parameters (temperature time series data format requirements), output parameters (predicted temperature value and accuracy evaluation index), and other characteristic information. The algorithm characteristic description is added to the algorithm library, an association relationship with other temperature prediction algorithms is established, and an algorithm characteristic index is generated.

[0028] S120: According to the algorithm characteristic index, continuously monitor the meteorological data source through the real-time message queue, generate a DI trigger signal when new meteorological data is obtained, and generate a real-time task description according to the DI trigger signal;

[0029] Optionally, the data structure description of the meteorological data source is obtained according to the algorithm characteristic index, and the data structure description is converted into a message queue listening parameter; the Kafka message topic is established through the message queue listening parameter, and the data update timestamp of the meteorological data source is periodically scanned; when a new timestamp of the meteorological data source appears, the integrity check result of the new meteorological data is obtained, and the data that passes the check is generated to generate a data arrival notification; the data arrival notification is compared with a preset trigger threshold, and the DI trigger signal is generated when the data change amplitude exceeds the threshold; based on the DI trigger signal, the key attribute information of the meteorological data is extracted, including data type, collection time, and geographic location information, to form a task parameter set; the task parameter set is associated and mapped with the algorithm characteristic index to generate the real-time task description.

[0030] The data structure description of the meteorological data source is obtained according to the algorithm feature index, and the data structure description defines the format specification of the meteorological data in detail, including data field type, numerical range, unit standard and the like. In the process of converting the data structure description into the message queue listening parameter, the data field needs to be mapped into a standard message format. The listening parameter includes message topic name, message partition number, message format version number and the like configuration information. These parameters determine how the message queue monitors and distributes the data source.

[0031] When the message topic is established in the Kafka message queue, the storage strategy and distribution rule of the message are configured based on the listening parameter. The message topic adopts a partition storage mechanism, and each partition corresponds to a type of meteorological data source. In the process of periodically scanning the data update timestamp of the meteorological data source, the latest timestamp of each data source is checked in a polling manner. When a new timestamp of the meteorological data source is detected, a data integrity verification process is triggered immediately. The integrity verification includes data format verification, numerical range checking, mandatory field verification and the like. For the data that passes the integrity verification, a standard format data arrival notification is generated, and the notification includes data source identification, update time, data volume statistics and the like information.

[0032] When the data arrival notification is compared and analyzed with the preset trigger threshold, the change amplitude of the data is focused on. The trigger threshold is defined from the data update frequency, numerical change rate, data volume change and the like dimensions. When the data change amplitude exceeds the threshold, it indicates that there is a data change worthy of attention, and a DI trigger signal is generated at this time. The DI trigger signal records the trigger time, trigger reason, data feature and the like information.

[0033] When the key attribute information of the meteorological data is extracted based on the DI trigger signal, three types of information are focused on: data type (such as temperature, humidity, air pressure and the like), collection time (accurate to the second timestamp), and geographic location information (including latitude and longitude coordinates, altitude and the like). These key attribute information collectively constitutes a task parameter set, which provides a basis for subsequent task scheduling. In the process of associating and mapping the task parameter set with the algorithm feature index, the matching of the parameter type and format needs to be ensured. Through the association and mapping, the specific meteorological data is associated with the algorithm suitable for processing the data, and finally a real-time task description is generated. The real-time task description defines the execution condition, processing algorithm, data source and the like information of the task in detail.

[0034] For example, a weather station collects temperature data every minute, and the data structure description defines the temperature value as a floating-point type with a valid range of -50°C to 50°C and a precision of 0.1°C. Convert these specifications into message queue monitoring parameters, create a message topic named "temperature_monitor" in Kafka, and configure 3 message partitions for temperature data storage in different regions.

[0035] The system performs a timestamp scan on the temperature data source every 10 seconds, and performs integrity checking when new data is detected. Integrity checking ensures that temperature data meets format specifications, values are within the valid range, and contains complete time and location information. After passing the check, generate data arrival notification, record the collection time and basic statistical information of the latest temperature data. The trigger threshold of the system is set to a temperature change rate greater than 2°C / min. When the temperature of a certain monitoring point changes more than 2°C within 1 minute, a DI trigger signal is generated. The system extracts the data type (temperature), collection time (specific timestamp), and geographic location information (latitude and longitude coordinates of the monitoring point) from the latest temperature data to form a task parameter set.

[0036] Match the task parameter set with the temperature prediction algorithm in the algorithm feature index to confirm that the data format meets the input requirements of the algorithm. Finally, generate a real-time task description containing the task definition of using the specified temperature prediction algorithm to analyze and warn abnormal temperature changes.

[0037] S130: Use the real-time task description to perform time series analysis on historical weather data and real-time monitoring data through a language model, and quickly allocate task urgency and computing resources via an adaptive optimization algorithm to obtain a real-time scheduling scheme.

[0038] Optionally, associate the real-time task description with historical weather data in the time dimension, and filter out a data feature set in a similar historical scenario; use the data feature set to perform parameter matching with real-time monitoring data, extract time series change features, and generate a time series data matrix; calculate the data change rate and trend indicators based on the time series data matrix to quantitatively evaluate the degree of data anomaly and form a task urgency quantification index; use the task urgency quantification index to construct a task weight coefficient, and perform weighted operation with the computing resource occupancy rate to obtain a resource allocation parameter; dynamically balance the computing node load according to the resource allocation parameter through an adaptive optimization algorithm to output a resource scheduling instruction; generate the real-time scheduling scheme according to the combination relationship between the resource scheduling instruction and the task urgency quantification index.

[0039] Wherein, by time dimension correlation analysis of real-time task description and historical meteorological data, firstly, the corresponding relationship of time series needs to be established, that is, the current time meteorological condition is compared with the historical similar period meteorological condition. The time dimension correlation considers multiple dimensions such as seasonal characteristics, daily variation characteristics, weather system evolution characteristics, etc. By calculating the similarity of time characteristics, the historical scene data most similar to the current situation is selected. In the historical scene data screening process, the key information such as the spatial and temporal distribution characteristics, the change trend characteristics, the extreme value characteristics of meteorological elements are focused on. Through extraction and normalization processing of these characteristics, a standardized data feature set is formed. The data feature set contains numerical characteristics, change rate characteristics, spatial distribution characteristics and other multiple dimension information of meteorological elements.

[0040] When the data feature set is matched with the real-time monitoring data, the sliding window technology is used for segment processing of time series data. For the data in each time window, its time series change characteristics are extracted, including trend change, periodic change, mutation characteristics, etc. Through feature extraction and combination, the time series data is converted into a standard format time series data matrix, each row of which represents a time point and each column represents a feature dimension. Based on the time series data matrix, the change rate of the data is calculated, mainly focusing on the change amount of meteorological elements in unit time. At the same time, the trend index of the data is extracted through time series analysis method, including rising trend, falling trend, oscillation characteristics, etc. Combined with the change rate and trend index, the abnormality degree of the data is quantitatively evaluated, the significance level of the abnormal value is calculated, and finally the task urgency quantitative index is formed.

[0041] When the task weight coefficient is constructed for the task urgency quantitative index, the following calculation formula is used:

[0042]

[0043] Wherein: W is the task weight coefficient; α is the global weight adjustment factor; β i is the weight of the i-th abnormal index; Y i is the normalized value of the i-th abnormal index; δ i is the timeliness factor of the i-th abnormal index; ∈ is the historical similarity weight factor; η j is the weight of the j-th historical feature; θ j is the similarity value of the j-th historical feature; n is the number of abnormal indexes; m is the number of historical features.

[0044] When the task weight coefficient is weighted with the computing resource occupancy rate, the following formula is used:

[0045]

[0046] wherein: R is a resource allocation parameter; λ is a task weight adjustment factor; μ is a resource weight adjustment factor; v k is a weight of the kth resource indicator; ξ k is a current occupancy rate of the kth resource; is an availability factor of the kth resource; p is the number of resource indicators.

[0047] According to the calculated resource allocation parameter, the adaptive optimization algorithm is used to dynamically balance the load of the computing node. According to the current task distribution and node load condition, the adaptive optimization algorithm dynamically adjusts the task allocation strategy to ensure efficient use of computing resources. Finally, the resource scheduling instruction is output, which contains the specific task allocation scheme and resource scheduling strategy. The resource scheduling instruction is combined with the task urgency quantitative indicator for analysis, and the task priority and resource utilization efficiency are considered comprehensively to generate the final real-time scheduling scheme. The real-time scheduling scheme defines the execution order of the task, the resource allocation strategy, the scheduling priority and other information in detail.

[0048] Taking the rainstorm warning task as an example, when receiving real-time monitoring data of rapid increase of precipitation, first, similar rainstorm weather processes are retrieved in historical data. By comparing the similarity of the current precipitation intensity, spatial distribution, pressure field characteristics and historical cases, the most valuable historical rainstorm event data is selected. The historical rainstorm event data and the current real-time monitoring data are matched, and the time evolution characteristics of the precipitation intensity are analyzed. Hourly precipitation, cumulative precipitation, precipitation intensity change rate and other time series characteristics are extracted to construct a time series data matrix reflecting the development process of the rainstorm. By calculating the change rate and development trend of the precipitation intensity, the degree of strengthening or weakening of the rainstorm is evaluated to form the urgency quantitative indicator reflecting the danger degree of the rainstorm.

[0049] Based on the urgency quantitative indicator, the task weight coefficient is calculated, in which the precipitation intensity change rate, cumulative precipitation, rainstorm influence range and other factors are given higher weights. Meanwhile, the current load condition of the computing node is considered, including CPU occupancy rate, memory usage rate, network bandwidth occupancy and other resource indicators, and the final resource allocation parameter is obtained through weighted operation. According to the resource allocation parameter, the optimization algorithm dynamically adjusts the task allocation of each computing node, and the rainstorm warning task is preferentially allocated to the high-performance node with low load. The finally generated real-time scheduling scheme not only ensures the timely processing of the rainstorm warning task, but also realizes the reasonable use of computing resources, providing a strong guarantee for the timely release of rainstorm warning information.

[0050] S140: According to the real-time scheduling scheme, the workflow engine is used to parallel arrange the meteorological forecast and warning task nodes, and the trigger dependency and data flow conversion between the task nodes are converted into a workflow execution strategy;

[0051] Optionally, the task urgency in the real-time scheduling scheme is converted into node priority identification, a dependency graph of meteorological forecast and warning task nodes is established; combinations of task nodes that are not dependent on each other are identified from the dependency graph, data transmission costs between nodes are calculated, and parallel execution groups are generated; input and output interfaces of the task nodes in the parallel execution groups are matched, data flow transfer paths are established, and a node connection relationship table is formed; trigger conditions between task nodes are analyzed according to the node connection relationship table, a node state transition matrix is constructed, and a trigger rule set is output; the trigger rule set is used to sequence the task nodes in time, the node execution order and the data flow transfer path are integrated, and a workflow structure description is generated; and the workflow structure description and the trigger rule set are combined and mapped into the workflow execution strategy.

[0052] In order to realize the parallel arrangement of the meteorological forecast and warning task nodes by the workflow engine, the conversion rule between the task urgency in the real-time scheduling scheme and the node priority identification needs to be determined first. The task urgency is a quantitative index obtained by comprehensively evaluating multiple dimensions such as the rate of change of meteorological data, the range of influence, and the potential harm degree, while the node priority identification is an execution level parameter for the workflow engine scheduling. The task urgency is converted into the node priority identification by establishing a mapping relationship, for example, the urgency is divided into multiple intervals and corresponding to different priority levels, and then the dependency graph of the forecast and warning task nodes is constructed according to these priority identifications. The dependency graph is in the form of a directed acyclic graph (DAG), and the nodes represent specific forecast and warning tasks, and the edges represent the dependency relationship between tasks. For the analysis of the dependency graph, the first task is to identify combinations of task nodes that are not dependent on each other. The connection relationship between nodes is analyzed by graph traversal algorithm, and the node set without direct or indirect dependency relationship is found out. For example, in the meteorological forecast and warning system, the temperature forecast task and the wind speed forecast task may not have a direct dependency relationship, and can be divided into different parallel execution groups. After identifying these parallel executable task groups, the data transmission cost between nodes needs to be calculated. The data transmission cost is determined by factors such as data size, network bandwidth, and transmission distance, and a cost evaluation model is established to quantify the influence of these factors. Based on the calculated data transmission cost, task combinations with low transmission cost and mutual independence are integrated into parallel execution groups. Input and output interface matching of the task nodes in the parallel execution groups is a key step to ensure the reliability of data flow transfer. The input and output interfaces of the task nodes describe the format, type, timestamp, and other attributes of the data, and the data flow transfer path is established by analyzing the compatibility of these attributes. Input and output interface matching includes the definition of data format conversion rules, the setting of data quality control parameters, the selection of data transmission protocols, and other contents. The matching results are recorded in the node connection relationship table, which describes the input source and output destination of each task node in detail.

[0053] Based on the node connection relationship table, the trigger conditions between task nodes are analyzed in depth. The trigger conditions include time trigger conditions (such as timing trigger, periodic trigger) and data trigger conditions (such as data update trigger, threshold trigger). The state transition rules of these trigger conditions are extracted as a state transition matrix, and the elements in the matrix represent the probability or rule of transitioning from one task state to another task state under certain conditions. By analyzing the state transition matrix, a complete set of trigger rules is generated, which contains all possible trigger scenarios and corresponding processing strategies.

[0054] The timing sequence of the task nodes using the trigger rule set is an important step in forming the final workflow. Timing sequence needs to consider the priority, dependency relationship and trigger condition of the node, to ensure that each task node is executed at the appropriate time. The execution order of the sorted nodes is integrated with the previously established data flow path to form a complete workflow structure description. The workflow structure description is a complete model that contains all information about node relationship, execution order, data flow direction, etc.

[0055] The workflow structure description and trigger rule set are combined and mapped to form a workflow execution strategy. The workflow execution strategy is a set of instructions that can be directly understood and executed by the workflow engine, which specifies the scheduling order of tasks, resource allocation scheme, exception handling mechanism, etc. The generation process of the execution strategy needs to consider the resource constraints, performance requirements and reliability targets of the system, and evaluate and select various execution schemes through optimization algorithms.

[0056] For example, when multiple regional temperature, precipitation, wind speed and other meteorological elements need to be forecasted and warned simultaneously, the task urgency is first divided into multiple levels according to the timeliness of data processing, such as three levels of special urgent, urgent and ordinary, which are mapped to node priority identifiers 1, 2 and 3 respectively. Taking temperature prediction and precipitation prediction as examples, these two tasks have no direct dependency and can be executed in parallel, but they require different input data and computing resources. The data transmission cost calculation considers the size of the original observation data, the amount of preprocessed data and the network transmission conditions, and reduces the system resource consumption through optimization of the data flow path. In the trigger rule design, the temperature prediction task uses fixed time interval trigger, while the precipitation prediction task triggers according to the change amplitude of pressure, humidity and other indicators, and the two trigger mechanisms have different forms in the state transition matrix. The final generated workflow execution strategy can flexibly adjust the task execution order and resource allocation according to the real-time monitoring data changes, to ensure the timely and accurate release of the forecast and warning information.

[0057] S150: Based on the workflow execution strategy, pre-condition verification and data transfer are performed on multiple meteorological forecasting and warning tasks to generate a priority task sequence and output the task running status.

[0058] Optionally, the task node precondition is extracted from the workflow execution strategy, the numerical range and constraint rule of each condition are parsed, the condition judgment standard is generated, the input data of the current meteorological forecast and early warning task is verified for validity by using the condition judgment standard, and the data verification result is output; based on the data verification result, the task set satisfying the precondition is filtered, the data dependency strength between tasks is calculated, and a task correlation degree matrix is formed; the task correlation degree matrix and the task node priority are weighted calculated, the task execution priority is constructed, and the priority task sequence is obtained; the tasks in the priority task sequence are marked for resource allocation, the data processing progress and resource occupation are monitored, and the task execution parameters are recorded; the task completion condition is counted according to the task execution parameters, the processing delay and resource utilization data are summarized, and the task running state is generated.

[0059] The precondition information of the task node is extracted from the workflow execution strategy. The precondition contains various requirements that need to be met for each task node to start execution, such as data integrity requirements, data timeliness requirements, and computing resource requirements. For each precondition, the specific numerical range and constraint rule need to be parsed. The numerical range defines the valid value interval of the parameter, and the constraint rule specifies the logical relationship between parameters. By parsing this information, a standardized condition judgment standard is formed to provide a basis for subsequent data verification. The generation process of the condition judgment standard needs to consider the particularity of the meteorological forecast and early warning task. Each precondition is converted into an explicit judgment rule, including threshold setting for numerical conditions, enumeration value list for category conditions, and validity period limit for time conditions. For example, for the temperature forecast task, the temperature sensor data collection time interval should not exceed 5 minutes, and the data missing rate should not exceed 1%, which are all converted into specific judgment rules.

[0060] The generated condition judgment standard is used to conduct comprehensive validity verification on the input data of the current meteorological forecast and early warning task. The verification process includes data format checking, numerical range verification, time series continuity testing, and spatial distribution rationality testing in multiple dimensions. The verification result details the passing status of each test, and the data items that fail the test are marked with specific problem types and severity. The result of data verification directly affects whether the task can start execution. According to the data verification result, the task set satisfying all preconditions is filtered. For these tasks, the data dependency relationship between them needs to be further analyzed. The data dependency strength reflects the coupling degree between tasks, which is quantitatively calculated by analyzing the data flow relationship, data update frequency, data size, and other factors between tasks. The calculation result of the dependency strength is organized as a task correlation degree matrix, and each element in the matrix represents the dependency degree between the corresponding task pairs.

[0061] The task correlation matrix needs to be considered in combination with the priority indicators of the task nodes. By setting reasonable weight coefficients, the dependency strength and priority are weighted and calculated. The calculation process takes into account multiple factors such as the urgency of the task, resource demand, expected execution time, etc., and finally obtains a comprehensive score. Based on the comprehensive score, the tasks are sorted to generate a priority task sequence, and the tasks in the sequence are arranged in order from high to low according to the score. Each task in the priority task sequence is marked with resource allocation. The resource allocation process needs to consider the current system's computing resource state, storage resource state and network resource state. Assign an appropriate amount of resources to each task, and mark the type, quantity and usage period of the resources in the allocation record. At the same time, a real-time monitoring mechanism is established to track the data processing progress and resource occupation of the task.

[0062] During task execution, key execution parameters are continuously recorded. Execution parameters include data processing rate, resource utilization efficiency, processing quality indicators, etc. These parameters change dynamically over time, and a parameter collection and storage mechanism needs to be established to ensure data accuracy and integrity. The time granularity of parameter recording should be reasonably set according to the characteristics of the task, meeting the monitoring needs while avoiding excessive redundant data. Finally, based on the recorded task execution parameters, the completion of the task is statistically analyzed. The analysis includes the deviation between the actual execution time and the expected time, the fluctuation of resource utilization, the compliance rate of data processing quality, etc. At the same time, the processing delay data is summarized, including the time consumption of each link such as data waiting delay, calculation processing delay and result transmission delay. Resource utilization data reflects the efficiency and load balancing of various resources. These statistical analysis results are integrated to form a task running status report, providing decision basis for task scheduling and resource allocation optimization.

[0063] S160: According to the task running state, record and analyze the task delay and abnormality in real time through the EI message, and feed back the analysis results to the language model for dynamically updating the real-time scheduling scheme.

[0064] Optionally, the task execution time, resource usage, data processing volume parameters are extracted from the task running state to construct a performance index matrix; the performance index matrix is compared with the standard running parameters to identify overtime tasks and abnormal tasks, and an EI message record is generated; the EI message record is classified and counted according to the task type and the abnormal type to calculate the occurrence frequency and the influence degree of each type of abnormality, and an abnormal feature vector is formed; the abnormal feature vector is used to construct a task performance evaluation index to quantitatively analyze the system bottleneck point and the resource competition situation, and a performance evaluation report is output; based on the performance evaluation report, the task scheduling rules are dynamically adjusted, the task priority and the resource allocation weight are recalculated, and a scheduling optimization suggestion is generated; the scheduling optimization suggestion is fed back to the language model, associated analysis is performed with historical scheduling data, and the real-time scheduling scheme is dynamically updated.

[0065] Among them, key performance parameters are extracted from the task running state, including task execution time (actual time consumption from task start to completion), resource usage (CPU usage, memory occupancy, network bandwidth utilization, etc.), and data processing volume (input data volume, output data volume, intermediate data volume, etc.). These parameters are organized into a performance index matrix, each row of the matrix representing a task instance, and each column representing a performance index, so as to quantitatively express the task running condition in a structured manner. The performance index matrix needs to be compared with the pre-defined standard running parameters. The standard running parameters are benchmark values determined based on historical running data and business requirements, reflecting the expected performance of the task under normal circumstances. By setting reasonable judgment thresholds, tasks with abnormally long execution time, excessively high resource occupancy, or abnormal data processing are identified. For the identified abnormal conditions, corresponding EI (Event Integration) message records are generated. The EI message records contain detailed information such as the time of abnormal occurrence, the type of abnormality, and the degree of abnormality, providing data support for subsequent analysis and processing.

[0066] The EI message records are systematically classified and statistically analyzed. The message records are classified according to the task type (such as temperature prediction task, precipitation prediction task, etc.) and the abnormal type (such as execution timeout, resource depletion, data anomaly, etc.). The occurrence frequency (number of abnormalities per unit time) and the influence degree (size of influence on the overall performance of the system) of each type of abnormality are calculated. These statistical results are organized into an abnormal feature vector, each dimension of the vector corresponding to a feature index of a type of abnormality, so as to extract the features of the abnormal patterns.

[0067] The abnormal feature vector is the basis for constructing the task performance evaluation index. By analyzing the distribution characteristics of the feature vector, the performance bottleneck points and resource competition hotspots in the system are identified. Performance bottleneck points refer to those links that frequently appear abnormal and have a large impact range, while resource competition hotspots reflect the competition situation of multiple tasks for the same resource. Quantitative analysis is performed on these problem points to generate a performance evaluation report containing detailed data support. Based on the results of the performance evaluation report, the task scheduling rules are dynamically optimized and adjusted. The optimization process first re-evaluates the priority of the task, taking into account factors such as the urgency of the task, abnormal occurrence, and system load status. At the same time, the resource allocation weight is adjusted, and more resource buffer is reserved for tasks that frequently occur abnormally to avoid performance degradation caused by resource competition. These adjustment suggestions are integrated into a scheduling optimization suggestion.

[0068] The scheduling optimization suggestion is input into the language model for deep analysis. The language model learns the rules and experiences of abnormal handling through the correlation analysis of the optimization suggestion and historical scheduling data. The correlation analysis process includes pattern matching (finding similar historical cases), trend analysis (predicting possible future abnormal situations), and strategy generation (developing targeted optimization solutions). Based on the analysis results, the real-time scheduling scheme is dynamically updated, enabling the scheduling strategy to adapt to changes in the system running state.

[0069] Through a series of monitoring analysis and optimization adjustment, a dynamic and adaptive task scheduling system is established. From abnormal detection to optimization deployment, a complete closed loop is formed, ensuring that the meteorological forecasting and warning task can be executed stably and efficiently. The system can continuously optimize the scheduling strategy based on actual running data, improving the prevention and handling capacity of abnormal situations. The entire implementation process fully utilizes the advantages of the EI message mechanism and the language model, ensuring the real-time nature of abnormal monitoring and realizing the intelligentization of scheduling optimization.

[0070] The above is an introduction to the method embodiment. The following describes the disclosed scheme through a device embodiment.

[0071] Figure 2 A block diagram of a forecasting and warning device 200 based on workflow engine technology according to an embodiment of the present disclosure is shown. As shown in Figure 2 The device 200 includes:

[0072] The verification module 210 is configured to upload and verify the timeliness of the meteorological forecasting and warning algorithm through the algorithm registration interface, authorize and manage different users via the permission identifier, thereby constructing a meteorological forecasting and warning algorithm library and generating an algorithm feature index.

[0073] The monitoring module 220 is configured to continuously monitor the weather data source through a real-time message queue according to the algorithm feature index, generate a DI trigger signal when new weather data is acquired, and generate a real-time task description according to the DI trigger signal;

[0074] The analysis module 230 is configured to perform time series analysis on historical weather data and real-time monitoring data through a language model using the real-time task description, quickly allocate task urgency and computing resources via an adaptive optimization algorithm, and obtain a real-time scheduling scheme;

[0075] The arrangement module 240 is configured to perform parallel arrangement of weather forecast and warning task nodes using a workflow engine according to the real-time scheduling scheme, and convert trigger dependencies and data flow conversion between the task nodes into a workflow execution strategy;

[0076] The delivery module 250 is configured to perform precondition verification and data delivery on a plurality of weather forecast and warning tasks based on the workflow execution strategy, generate a priority task sequence, and output a task running state.

[0077] The feedback module 260 is configured to record and analyze task delays and abnormalities in real time according to the task running state through an EI message, and feed back the analysis results to the language model for dynamically updating the real-time scheduling scheme.

[0078] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described modules can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0079] The foregoing specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A forecasting and early warning method based on workflow engine technology, characterized in that, include: The algorithm registration interface is used to upload the code and verify the timeliness of the weather forecast and early warning algorithm. Different users are authorized and managed through permission identifiers, thereby building a weather forecast and early warning algorithm library and generating an algorithm feature index. Based on the algorithm feature index, the meteorological data source is continuously monitored through a real-time message queue. When new meteorological data is acquired, a DI trigger signal is generated, and a real-time task description is generated based on the DI trigger signal. Using the real-time task description, a time-series analysis of historical meteorological data and real-time monitoring data is performed through a language model. An adaptive optimization algorithm is then used to quickly allocate task urgency and computing resources to obtain a real-time scheduling scheme. Based on the real-time scheduling scheme, the workflow engine is used to orchestrate the meteorological forecast and early warning task nodes in parallel, and the triggering dependencies and data flow between task nodes are converted into workflow execution strategies. Based on the workflow execution strategy, precondition verification and data transmission are performed for multiple weather forecast and early warning tasks, a priority task sequence is generated, and the task running status is output. Based on the task running status, task delays and anomalies are recorded and analyzed in real time via EI messages, and the analysis results are fed back to the language model for dynamic updating of the real-time scheduling scheme.

2. The forecasting and early warning method based on workflow engine technology according to claim 1, characterized in that, The process involves uploading and verifying the timeliness of weather forecasting and early warning algorithms through an algorithm registration interface, authorizing different users via permission identifiers, thereby constructing a weather forecasting and early warning algorithm library and generating an algorithm feature index, including: The program code of the weather forecast and early warning algorithm is syntax-checked using predefined format specifications to obtain code compliance verification results; The code compliance verification results are compared and analyzed with historical runtime efficiency data to generate an algorithm timeliness evaluation report; Using the aforementioned algorithm timeliness evaluation report, scores are calculated from three dimensions: algorithm execution efficiency, resource consumption, and data processing capability, to obtain algorithm score data; Based on the algorithm scoring data, an access permission matrix is ​​established to map user identity information to algorithm operation permissions and generate permission identifiers; A unique identifier is generated for each algorithm based on the permission identifier, and the algorithm's version information, calling interface, input and output parameters are recorded to form an algorithm feature description; The algorithm feature descriptions are integrated into the meteorological forecast and early warning algorithm library, an algorithm index table is established, and the algorithm feature index is generated.

3. The forecasting and early warning method based on workflow engine technology according to claim 1, characterized in that, The step involves continuously monitoring meteorological data sources via a real-time message queue based on the algorithm's feature index, generating a DI trigger signal when new meteorological data is acquired, and generating a real-time task description based on the DI trigger signal, including: The data structure description of the meteorological data source is obtained based on the algorithm feature index, and the data structure description is converted into message queue listening parameters. A Kafka message topic is established using the message queue listening parameters, and the data update timestamps of the meteorological data source are periodically scanned. When a new timestamp appears in the meteorological data source, obtain the integrity verification result of the new meteorological data, and generate a data arrival notification for the data that passes the verification. The data arrival notification is compared with a preset trigger threshold. When the data change exceeds the threshold, the DI trigger signal is generated. Based on the DI trigger signal, key attribute information of meteorological data is extracted, including data type, collection time, and geographical location information, to form a task parameter set; The task parameter set is associated and mapped with the algorithm feature index to generate the real-time task description.

4. The forecasting and early warning method based on workflow engine technology according to claim 1, characterized in that, The process involves using the real-time task description to perform time-series analysis on historical meteorological data and real-time monitoring data through a language model, and then rapidly allocating task urgency and computing resources via an adaptive optimization algorithm to obtain a real-time scheduling scheme, including: The real-time task description is correlated with historical meteorological data along the time dimension to filter out a set of data features under similar historical scenarios. By matching the data feature set with real-time monitoring data, time-series change features are extracted, and a time-series data matrix is ​​generated. Based on the time-series data matrix, the data change rate and trend indicators are calculated to quantitatively assess the degree of data anomaly and form a quantitative indicator of task urgency. The task weight coefficient is constructed using the aforementioned task urgency metric, and then weighted by the computing resource utilization rate to obtain resource allocation parameters. Based on the resource allocation parameters, the load on computing nodes is dynamically balanced using an adaptive optimization algorithm, and resource scheduling instructions are output. The real-time scheduling scheme is generated based on the combination relationship between the resource scheduling instructions and the task urgency quantification indicators.

5. The forecasting and early warning method based on workflow engine technology according to claim 1, characterized in that, Based on the real-time scheduling scheme, the workflow engine is used to orchestrate the meteorological forecasting and early warning task nodes in parallel, transforming the trigger dependencies and data flows between task nodes into workflow execution strategies, including: The urgency of tasks in the real-time scheduling scheme is converted into node priority identifiers, and a dependency graph of weather forecast and early warning task nodes is established. Identify combinations of independent task nodes from the dependency graph, calculate the data transfer cost between nodes, and generate parallel execution groups. The input / output interfaces of the task nodes in the parallel execution group are matched to establish data flow paths and form a node connection relationship table. Based on the node connection relationship table, analyze the triggering conditions between task nodes, construct the node state transition matrix, and output the triggering rule set; The task nodes are sorted in time sequence using the triggering rule set, and the node execution order is integrated with the data flow path to generate a workflow structure description. The workflow structure description and the trigger rule set are combined and mapped to the workflow execution strategy.

6. The forecasting and early warning method based on workflow engine technology according to claim 1, characterized in that, Based on the workflow execution strategy, the process verifies preconditions and transmits data for multiple weather forecast and early warning tasks, generates a priority task sequence, and outputs the task running status, including: Extract the preconditions of the task nodes from the workflow execution strategy, parse the numerical range and constraint rules of each condition, and generate condition judgment criteria; The validity of the input data for the current weather forecast and early warning task is verified using the aforementioned condition judgment criteria, and the data verification results are output. Based on the data verification results, a set of tasks that meet the preconditions is selected, the data dependency strength between tasks is calculated, and a task correlation matrix is ​​formed. The task correlation matrix is ​​weighted and calculated with the task node priority to construct the task execution priority order, thus obtaining the priority task sequence; The tasks in the priority task sequence are marked with resource allocation, the data processing progress and resource usage are monitored, and the task execution parameters are recorded. Based on the task execution parameters, the task completion status is statistically analyzed, processing delay and resource utilization data are summarized, and the task running status is generated.

7. The forecasting and early warning method based on workflow engine technology according to claim 1, characterized in that, The step of recording and analyzing task delays and anomalies in real time via EI messages based on the task's running status, and feeding the analysis results back to the language model for dynamically updating the real-time scheduling scheme, includes: Extract task execution time, resource utilization, and data processing volume parameters from the task running status to construct a performance index matrix; The performance index matrix is ​​compared and calculated with standard operating parameters to identify timeout tasks and abnormal tasks, and to generate EI message records. The EI message records are classified and statistically analyzed according to task type and anomaly type. The frequency and impact of each type of anomaly are calculated to form an anomaly feature vector. The abnormal feature vectors are used to construct task performance evaluation indicators, quantitatively analyze system bottlenecks and resource competition, and output a performance evaluation report. Based on the performance evaluation report, the task scheduling rules are dynamically adjusted, task priorities and resource allocation weights are recalculated, and scheduling optimization suggestions are generated. The scheduling optimization suggestions are fed back to the language model, and correlation analysis is performed with historical scheduling data to dynamically update the real-time scheduling scheme.

8. A forecasting and early warning device based on workflow engine technology, used to implement the forecasting and early warning method based on workflow engine technology as described in any one of claims 1-7, characterized in that, The forecasting and early warning device based on workflow engine technology includes: The verification module is used to upload the code and verify the timeliness of weather forecast and early warning algorithms through the algorithm registration interface, and to manage the authorization of different users through permission identifiers, thereby building a weather forecast and early warning algorithm library and generating an algorithm feature index. The monitoring module is used to continuously monitor the meteorological data source through a real-time message queue based on the algorithm feature index, generate a DI trigger signal when new meteorological data is acquired, and generate a real-time task description based on the DI trigger signal. The analysis module is used to perform time-series analysis on historical meteorological data and real-time monitoring data using the real-time task description through a language model, and to quickly allocate task urgency and computing resources through an adaptive optimization algorithm to obtain a real-time scheduling scheme. The orchestration module is used to orchestrate the meteorological forecast and early warning task nodes in parallel using the workflow engine according to the real-time scheduling scheme, and to convert the triggering dependencies and data flow between task nodes into workflow execution strategies. The transmission module is used to perform precondition verification and data transmission for multiple weather forecast and early warning tasks based on the workflow execution strategy, generate a priority task sequence, and output the task running status. The feedback module is used to record and analyze task delays and anomalies in real time through EI messages based on the task running status, and feed the analysis results back to the language model for dynamic updating of the real-time scheduling scheme.

Images