Distributed scheduled task monitoring analysis method and system based on large model

Abstract

The invention relates to the technical field of task monitoring analysis, and discloses a distributed scheduled task monitoring analysis method and system based on a large model, and the method comprises the steps: collecting application operation data and task execution logs of scheduled tasks in real time through a plurality of program probes deployed in a distributed system; inputting the application operation data and the task execution log into a large model for analysis processing, generating a system resource optimization suggestion by the large model based on the application operation data, and generating a task exception correction suggestion based on the task execution log; and outputting an analysis result containing the system resource optimization suggestion and / or the task exception correction suggestion. According to the method, the large model is deeply embedded into the monitoring analysis process of the distributed scheduled task, so that the limitations of the prior art in the aspects of scale expansibility, analysis depth and intelligent decision making are effectively overcome.

Description

Technical Field

[0001] This invention relates to the field of task monitoring and analysis technology, and more specifically, to a distributed planned task monitoring and analysis method and system based on a large model. Background Technology

[0002] As enterprises deepen their digital transformation, distributed system architectures are widely used in various business scenarios, especially in increasingly complex systems that rely on scheduled or planned tasks (such as data synchronization, report generation, batch processing, etc.). In a clustered deployment model, a large system often contains multiple scheduling servers and hundreds or thousands of planned tasks, which may be distributed across different physical or virtual nodes and executed concurrently.

[0003] Traditional operation and maintenance monitoring methods rely heavily on manual inspections or simple log viewing tools, making it difficult to achieve comprehensive and real-time control over large-scale distributed task systems. For example, when a task fails, operations personnel need to manually log in to the corresponding server to search logs, analyze code, and troubleshoot resource bottlenecks, which is inefficient and prone to missing critical information. Furthermore, in the face of dynamically changing system loads and task behaviors, the lack of intelligent means to predict and intervene in potential risks makes it difficult to guarantee system stability and availability.

[0004] In recent years, Large Language Models (LLMs) have demonstrated great potential in the field of Intelligent Operations and Maintenance (AIOps) due to their powerful semantic understanding, contextual reasoning, and knowledge integration capabilities. However, existing technologies have not yet effectively integrated LLMs with the end-to-end monitoring system of distributed scheduled tasks, especially lacking an end-to-end solution that can automatically collect task execution data and logs and generate intelligent diagnostics and optimization suggestions based on LLMs. Therefore, this paper proposes a method and system for monitoring and analyzing distributed scheduled tasks based on LLMs. Summary of the Invention

[0005] In view of this, the present invention proposes a distributed planning task monitoring and analysis method and system based on a large model. By deeply embedding the large model into the monitoring and analysis process of distributed planning tasks, it effectively overcomes the limitations of existing technologies in terms of scalability, analysis depth and intelligent decision-making, and has outstanding substantive features and significant progress.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a distributed planned task monitoring and analysis method based on a large model, comprising: S1. Collect application runtime data and task execution logs of planned tasks in real time through multiple program probes deployed in the distributed system; S2. Input the application running data and task execution log into the big model for analysis and processing. The big model generates system resource optimization suggestions based on the application running data and task anomaly correction suggestions based on the task execution log. S3. Output the analysis results, which include the system resource optimization suggestions and / or the task anomaly correction suggestions.

[0007] Preferably, before step S2, a data preprocessing step is further included, specifically: The collected application runtime data and task execution logs are cleaned and format-converted, and the processed data is stored in the target database.

[0008] Preferably, the application runtime data includes at least one of the server's CPU utilization, memory utilization, and disk I / O; the task execution log includes at least one of the task identifier, start time, end time, execution status, and task code.

[0009] Preferably, in step S2, the large model determines whether there is a resource bottleneck in the system based on the application running data, and generates system resource optimization suggestions based on the determination result; And / or, the large model analyzes the error information in the task execution log and the corresponding task code to generate the task anomaly correction suggestions.

[0010] Preferably, after step S3, a result notification step is further included, specifically: The analysis results are sent to maintenance personnel via a preset communication method.

[0011] Preferably, in the result notification step, the preset communication method includes email or SMS.

[0012] Preferably, it also includes a monitoring and display step, specifically: Based on the real-time collected application operation data and task execution logs, a visual monitoring interface is generated and updated. The visual monitoring interface is used to display at least one of the following: system resource status, task execution status, and abnormal alarm information.

[0013] Preferably, in step S1, the program probe collects data at a fixed frequency or in an event-triggered manner.

[0014] Secondly, the present invention provides a distributed planned task monitoring and analysis system based on a large model, comprising: The data collection module is used to collect application runtime data and task execution logs of planned tasks in real time through multiple program probes deployed in the distributed system. The data analysis module is used to input the application running data and task execution logs into the big model for analysis and processing. The big model generates system resource optimization suggestions based on the application running data and task anomaly correction suggestions based on the task execution logs. The results generation module is used to output analysis results that include the system resource optimization suggestions and / or the task anomaly correction suggestions.

[0015] Preferably, it further includes a data preprocessing module, specifically: The collected application runtime data and task execution logs are cleaned and format-converted, and the processed data is stored in the target database.

[0016] This application discloses a distributed task monitoring and analysis method based on a large model. This method automatically and continuously collects task execution logs and system operation indicators through multiple program probes deployed in the distributed system, solving the problems of incomplete coverage and poor timeliness of traditional manual inspections. By using a large model to jointly analyze structured resource data and unstructured logs / code, it can accurately identify resource bottlenecks (such as CPU overload, memory leaks) or task logic defects (such as null pointers, timeout configuration errors), significantly shortening fault location time. The large model not only determines "whether it is abnormal," but also outputs specific optimization strategies (such as adjusting thread pool size, expanding nodes) or code-level correction suggestions (such as exception capture, retry mechanisms) based on its built-in knowledge base and reasoning capabilities, enabling operations and maintenance to shift from "passive response" to "proactive governance." This method provides high-quality data input and decision-making basis for subsequent alarm notifications, visualization, and automated repair, comprehensively improving the observability, self-healing, and operational efficiency of the distributed task system. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating a distributed planned task monitoring and analysis method based on a large model, provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a distributed planned task monitoring and analysis system based on a large model, provided in an embodiment of the present invention. Detailed Implementation

[0018] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0019] like Figure 1 As shown in some embodiments of this application, this embodiment provides a distributed planned task monitoring and analysis method based on a large model. Specifically, the method includes the following steps: Step S1: Collect application runtime data and task execution logs of planned tasks in real time by deploying multiple program probes in the distributed system.

[0020] As mentioned above, the core of this step lies in building a lightweight, low-intrusion distributed data acquisition mechanism. A program probe is a monitoring agent component embedded or mounted in various scheduling nodes (such as application servers, container instances, or virtual machines). It does not change the original business logic of the planned task, but only passively or actively captures two types of key information during task execution: application runtime data, which reflects the host system's resource usage and is used to assess the health of the task execution environment; and task execution logs, which record the entire behavioral trajectory of a single planned task from triggering, execution to completion, including state changes, exception stacks, and execution time.

[0021] These probes work collaboratively in a distributed manner to ensure that the execution context of a task can be fully captured regardless of which physical or logical node it is scheduled to execute. The data is then aggregated to the backend analysis platform through a unified data channel, providing a high-quality and timely input source for subsequent large-scale model analysis.

[0022] For example, in an enterprise-level data synchronization platform, hundreds of ETL (Extract-Transform-Load) scheduled tasks are triggered daily at 2:00 AM, running on 10 different application servers. Each server has a program probe deployed. When an ETL task starts, the probe automatically records the following information: Application runtime data: Current server CPU utilization (e.g., 78%), memory usage (e.g., 65%), and disk I / O throughput; Task execution log: The task ID is "SYNC_SALES_20251115", the start time is 02:00:03, the end time is 02:15:47, the execution status is "failed", and the error log "database connection timed out" is attached.

[0023] Meanwhile, probes on other servers are also synchronously collecting task execution data and logs on their respective nodes. All data is cached locally and then transmitted to the central storage system according to a preset strategy (such as batch reporting every 5 seconds) to ensure data integrity and timeliness and avoid information loss due to network jitter.

[0024] It should be noted that, in specific implementation scenarios, based on the above solutions, diversified probe deployment schemes can be adopted. Specifically, the program probe can run as an independent process or be integrated into a scheduling framework (such as Quartz, XXL-JOB, or Airflow) as an embedded library (e.g., Java Agent, Python Hook) to achieve seamless monitoring of task lifecycle events. A flexible data collection triggering mechanism can be adopted, meaning that in addition to continuous polling, the probe can also work in an event-driven manner, such as monitoring key events like "start execution," "execution completed," and "exception thrown," triggering data collection only when the event occurs, reducing system overhead. A dynamic content adaptation scheme can be adopted, dynamically adjusting the collection granularity according to task type or priority. For example, more detailed call chain information can be collected for core business tasks, while only basic status and resource indicators can be collected for low-priority tasks. All of the above optional schemes fall within the scope of protection of this application.

[0025] Step S2: Input the application runtime data and task execution logs into the large model for analysis and processing. The large model generates system resource optimization suggestions based on the application runtime data and task anomaly correction suggestions based on the task execution logs.

[0026] As mentioned above, the core of this step lies in leveraging the semantic understanding, contextual reasoning, and knowledge fusion capabilities of large-scale models to perform deep intelligent analysis on multi-source heterogeneous monitoring data from distributed systems. Specifically: Application runtime data (such as CPU, memory, disk I / O, etc.) are structured or semi-structured metrics used to reflect the resource load status of the system as a whole or local nodes. The large model analyzes the time series variation trend of these data and their correlation with the concurrency of other tasks to determine whether there are resource bottlenecks, unreasonable configurations or potential overload risks, and generates system resource optimization suggestions accordingly, such as suggesting to expand nodes, adjust task scheduling strategies or optimize JVM parameters. Task execution logs typically contain unstructured text information (such as error stacks, execution path descriptions, status codes, etc.), and sometimes also include corresponding task logic fragments (such as task name, calling method, configuration parameters). The large model, through understanding the semantics of the logs and combining them with its built-in operation and maintenance knowledge base (such as common exception patterns, best practices, and framework usage specifications), identifies the root cause of task failures and generates actionable task exception correction suggestions, such as "increase the database connection pool size", "add null value validation for file read operations", or "adjust the timeout threshold".

[0027] The entire analysis process emphasizes the combination of data-driven and knowledge-guided approaches. It not only relies on statistical patterns but also leverages large models to gain a deep understanding of the technological context, thus achieving a closed loop from "phenomenon description" to "root cause identification" and then to "solution recommendation".

[0028] For example, suppose an e-commerce platform's order reconciliation task frequently fails during nighttime execution. The system collects the following information through step S1: Application runtime data shows that the server executing the task consistently used over 95% of the CPU, had memory usage close to its limit, and experienced a significant increase in disk I / O wait time during the task's execution. The task execution log shows that the task threw an "OutOfMemoryError" while processing the 100,000th record, with stack traces pointing to the data batch loading module.

[0029] In step S2, the above data is fed into the large model for joint analysis. The large model first identifies a strong correlation between high memory consumption and OutOfMemoryError (OOM) exceptions. Then, based on its understanding of Java application memory management mechanisms, it determines that the root cause is loading too much data at once without pagination. Subsequently, the large model outputs two types of suggestions: System resource optimization suggestion: "The current node's memory resources are insufficient to support the peak load of this task. It is recommended to allocate a dedicated high-memory instance for this task or limit its concurrency to reduce resource contention." Recommendations for correcting task anomalies: "The task code should adopt a streaming or pagination mechanism to avoid loading all data into memory; it is also recommended to set reasonable memory monitoring and alarm thresholds in the task configuration."

[0030] This recommendation proposes both expansion solutions at the infrastructure level and code-level improvement directions at the application logic level, demonstrating the ability of large models to conduct cross-level collaborative analysis.

[0031] It should be noted that, in specific implementation scenarios, in addition to the above solutions, a multimodal data fusion analysis solution can be adopted. This means that besides basic resource indicators and logs, the large model can also combine auxiliary information such as task scheduling history, dependent service status, and network latency for comprehensive judgment, improving the accuracy of the analysis. A dynamic prompting engineering solution can be adopted, which automatically constructs targeted analysis prompt templates based on task type (such as data synchronization, report generation, and interface calls), guiding the large model to focus on relevant knowledge areas and improving the relevance and professionalism of the suggestions. A confidence assessment and suggestion grading solution can be adopted, where the large model can attach a confidence score or risk level (such as "highly reliable" or "requires manual review") when outputting suggestions, facilitating operations personnel to prioritize high-certainty issues. An incremental learning and feedback loop solution can be adopted, where the system can use the results of operations personnel's adoption or rejection of suggestions as feedback signals for subsequent fine-tuning or optimization, allowing the large model's analytical capabilities to continuously evolve with actual operations experience. All of the above optional solutions fall within the scope of protection of this application.

[0032] Step S3: Output the analysis results, which include the system resource optimization suggestions and / or the task anomaly correction suggestions.

[0033] As mentioned above, the core of this step lies in transforming the intelligent analysis results generated by the large model in step S2 into an output format that can be effectively utilized by operations and maintenance personnel or automated systems. This "analysis result" is not raw data or intermediate reasoning, but rather structured, semantically clear, and operationally oriented decision support information, and its content includes at least one of the following two categories: System resource optimization suggestions: suggestions for improvement on resource configuration, scheduling strategies, and hardware deployment for the distributed system as a whole or for specific nodes; Task anomaly correction suggestions: proposed fixes or optimizations for the execution logic, parameter configuration, dependencies, or error handling mechanisms of specific planned tasks.

[0034] The output process emphasizes the usability, traceability, and integrability of the results. Analysis results are typically packaged in standardized formats (such as JSON, XML, or structured text), which are both human-readable and comprehensible, and can be directly parsed and invoked by subsequent alarm systems, ticketing platforms, or automated operation and maintenance tools, thereby achieving a seamless connection from "intelligent analysis" to "action execution."

[0035] For example, continuing with the nighttime reconciliation task of an e-commerce platform: In step S2, the large model has identified a memory overflow issue and generated two suggestions. In step S3, the system integrates these into a structured analysis report, as follows: Task ID: ORDER_RECONCILE_DAILY; Anomaly Type: Insufficient memory resources caused task interruption; System Resource Optimization Suggestion: The current execution node's memory peak reaches 98%, it is recommended to migrate the task to a high-memory instance or limit its concurrent execution with other high-load tasks; Task Anomaly Correction Suggestion: The task did not use a pagination mechanism when batch loading order data, it is recommended to modify the data reading logic, process no more than 5000 records each time, and add a memory usage monitoring threshold alarm; Confidence: High (based on matching historical records of similar problem resolution); Correlation Indicators: Average CPU utilization 92%, peak memory 15.8GB / 16GB, abnormal increase in task execution time of 300%.

[0036] The analysis results were then pushed to the maintenance work order system and simultaneously displayed on the "Intelligent Diagnosis" panel of the monitoring screen for on-duty engineers to quickly review and handle.

[0037] It should be noted that, in specific implementation scenarios, based on the above solutions, diversified output formats can be adopted. Analysis results can be presented as structured reports, natural language summaries, visual charts (such as resource bottleneck heatmaps with suggested annotations), or embedded card-style prompts to adapt to the display needs of different terminals (Web console, mobile devices, command-line tools). A tiered output mechanism can be adopted, prioritizing analysis results based on the severity of the problem or the urgency of the suggestions (e.g., "Immediate Action," "Observation and Tracking," "Long-Term Optimization") to assist in operational decision-making. A multi-role targeted output solution can be adopted, customizing the granularity of output content for different user roles (such as system administrators, application developers, architects). For example, highlighting code-level correction suggestions to developers and emphasizing resource scheduling and alarm configuration to operations personnel. An output solution that integrates with external systems can be adopted, automatically writing analysis results into ITSM ticketing systems, DevOps pipelines, ChatOps robots, or automated repair engines to trigger subsequent processing flows and achieve a closed loop of "analysis-notification-action." All of the above optional solutions fall within the scope of protection of this application.

[0038] In some embodiments of this application, to improve the accuracy and reliability of the analysis results, a data preprocessing step is included before step S2, specifically: The collected application runtime data and task execution logs are cleaned and format-converted, and the processed data is stored in the target database.

[0039] As described above, before step S2, the system performs data preprocessing on the raw data collected from the program probe. This preprocessing step mainly includes two aspects: First, it cleans the application running data and task execution logs, removing invalid, duplicate, or obviously erroneous data items (such as missing timestamps, abnormal indicator values, empty log content, etc.), and reasonably completes or marks incomplete records; second, it converts the cleaned data into a format according to a unified structure and semantic standard, for example, standardizing log text from different sources into structured records with consistent fields, or normalizing various resource indicators to the same unit and time granularity. The cleaned and format-converted data is written to the target database, which is used to support the stable and efficient calling of subsequent large models, ensuring the consistency, integrity, and timeliness of input data, thereby improving the accuracy and reliability of the analysis results.

[0040] In some embodiments of this application, in order to reconstruct the task execution process, locate the cause of the anomaly, and support the generation of analysis and optimization suggestions at the task logic level, the application running data includes at least one of the server's CPU utilization, memory utilization, and disk I / O; the task execution log includes at least one of the task identifier, start time, end time, execution status, and task code.

[0041] As mentioned above, application runtime data refers to monitoring metrics that reflect the resource usage of each server node in a distributed system, specifically including but not limited to at least one of the following: server CPU utilization, memory utilization, and disk I / O throughput or latency. These metrics are used to assess the load level and stability of the task execution environment, providing a basis for determining whether resource bottlenecks exist.

[0042] A task execution log is a data set that records key behavioral information of a single planned task throughout its lifecycle. Specifically, it includes at least one of the following: task identifier (used to uniquely identify the task), start time, end time, execution status (such as success, failure, timeout, interruption, etc.), and task code or logical description associated with the task. This log information is used to reconstruct the task execution process, locate the causes of anomalies, and support the generation of analysis and optimization suggestions at the task logic level.

[0043] In some embodiments of this application, to provide more comprehensive and accurate diagnostic and optimization support, in step S2, the large model determines whether the system has resource bottlenecks based on the application's runtime data, and generates system resource optimization suggestions based on the determination results; And / or, the large model analyzes the error information in the task execution log and the corresponding task code to generate the task anomaly correction suggestions.

[0044] As described above, in step S2, the large model first analyzes the input application runtime data to determine whether there are resource bottlenecks in the system. This determination is based on a comprehensive evaluation of indicators such as CPU utilization, memory utilization, and disk I / O. For example, when a server experiences persistently high CPU usage, near memory exhaustion, or prolonged disk I / O blocking during task execution, the large model can identify resource overload or insufficient configuration and generate corresponding system resource optimization suggestions, such as suggesting expanding node capacity, adjusting task scheduling strategies, and optimizing resource configuration parameters.

[0045] Optionally, the large model can perform correlation analysis between error information in task execution logs (such as exception stack traces, failure status codes, timeout messages, etc.) and their corresponding task code or task logic descriptions to understand the context and possible causes of the exceptions. Based on this, the large model, combined with its built-in operational knowledge and experience in handling common problems, generates targeted suggestions for correcting task exceptions, such as modifying task parameters, adding exception handling mechanisms, optimizing data processing logic, or adjusting the way dependent services are called. These two types of analysis can be performed independently or in tandem to provide more comprehensive and accurate diagnostic and optimization support.

[0046] In some embodiments of this application, to improve fault response efficiency and system stability, a result notification step is included after step S3, specifically: The analysis results are sent to maintenance personnel via a preset communication method.

[0047] As described above, after step S3, the system executes the result notification step, which proactively pushes the generated analysis results to relevant operations and maintenance personnel via a pre-configured communication method. This analysis result includes system resource optimization suggestions and / or task anomaly correction suggestions, aiming to ensure that operations and maintenance personnel are promptly informed of potential problems and solutions. The preset communication method can be configured according to the actual operations and maintenance process, typically including email, SMS, instant messaging messages, or in-system notifications on the operations and maintenance platform. The notification content may include key elements such as a problem summary, suggestion details, related task information, and severity level, ensuring that operations and maintenance personnel can quickly understand the problem and take appropriate measures, thereby improving fault response efficiency and system stability.

[0048] In some embodiments of this application, to ensure that maintenance personnel can receive critical information and respond promptly in different scenarios, the preset communication method in the result notification step includes email or SMS.

[0049] As mentioned above, once the system generates the analysis results, it will automatically send emails containing system resource optimization suggestions and / or task anomaly correction suggestions to the designated email address of the operations and maintenance personnel, or SMS messages to their registered mobile phone numbers, based on pre-configured contact information and notification policies. Emails are suitable for delivering more detailed and complex analysis reports, while SMS messages are used to send brief notifications of high-priority or urgent alarms, ensuring that operations and maintenance personnel can receive key information and respond promptly in different scenarios.

[0050] In some embodiments of this application, to improve inspection efficiency and maintenance response speed, a monitoring and display step is also included, specifically: Based on the real-time collected application operation data and task execution logs, a visual monitoring interface is generated and updated. The visual monitoring interface is used to display at least one of the following: system resource status, task execution status, and abnormal alarm information.

[0051] As described above, this monitoring and display step refers to the system dynamically generating and continuously updating a visual monitoring interface based on real-time collected application runtime data and task execution logs. This interface is used to centrally display the operational status of the distributed scheduled task system, specifically including at least one of the following: system resource status (such as CPU, memory, and disk I / O usage of each server), task execution status (such as the total number of tasks, currently running tasks, and the number and distribution of successful / failed tasks), and abnormal alarm information (such as a list of failed tasks, resource bottleneck prompts, and tasks that have not been triggered for a long time). Through this interface, operations and maintenance personnel can intuitively grasp the overall system health and task execution status, quickly locate abnormal nodes or problematic tasks, and improve inspection efficiency and operational response speed.

[0052] In some embodiments of this application, to ensure the integrity of monitoring data and the timeliness of response, in step S1, the program probe collects data at a fixed frequency or in an event-triggered manner.

[0053] As described above, in step S1, the program probe employs two data collection mechanisms: one is periodic collection at a fixed frequency, such as reading the server's CPU utilization, memory usage, and the status of currently running tasks every 5 seconds or 1 minute; the other is an event-triggered approach, capturing relevant data in real time when planned tasks undergo critical actions (such as task start, completion, failure, or system resources exceeding a threshold). These two methods can be used individually or in combination to ensure data timeliness while considering system performance overhead, ensuring the integrity and timely response of monitoring data.

[0054] Compared with existing technologies, this application discloses a distributed task monitoring and analysis method based on a large model. This method automatically and continuously collects task execution logs and system operation indicators through multiple program probes deployed in the distributed system, solving the problems of incomplete coverage and poor timeliness of traditional manual inspections. By using a large model to jointly analyze structured resource data and unstructured logs / code, it can accurately identify resource bottlenecks (such as CPU overload, memory leaks) or task logic defects (such as null pointers, timeout configuration errors), significantly shortening fault location time. The large model not only judges "whether it is abnormal", but also outputs specific optimization strategies (such as adjusting thread pool size, expanding nodes) or code-level correction suggestions (such as catching exceptions, retry mechanisms) based on its built-in knowledge base and reasoning capabilities, enabling operation and maintenance to shift from "passive response" to "proactive governance". This method provides high-quality data input and decision-making basis for subsequent alarm notifications, visualization, and automated repair, and improves the overall observability, self-healing, and operation and maintenance efficiency of the distributed task system.

[0055] Based on the same inventive concept as the methods described above, embodiments of this application also propose a distributed planned task monitoring and analysis system based on a large model, such as... Figure 2 The diagram shown is a structural schematic of a distributed planned task monitoring and analysis system based on a large model. The system includes: The data collection module is used to collect application runtime data and task execution logs of planned tasks in real time through multiple program probes deployed in the distributed system. The data analysis module is used to input the application running data and task execution logs into the big model for analysis and processing. The big model generates system resource optimization suggestions based on the application running data and task anomaly correction suggestions based on the task execution logs. The results generation module is used to output analysis results that include the system resource optimization suggestions and / or the task anomaly correction suggestions.

[0056] Preferably, it further includes a data preprocessing module, specifically: The collected application runtime data and task execution logs are cleaned and format-converted, and the processed data is stored in the target database.

[0057] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0058] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0059] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0060] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A distributed planned task monitoring and analysis method based on a large model, characterized in that, include: S1. Collect application runtime data and task execution logs of planned tasks in real time through multiple program probes deployed in the distributed system; S2. Input the application running data and task execution log into the big model for analysis and processing. The big model generates system resource optimization suggestions based on the application running data and task anomaly correction suggestions based on the task execution log. S3. Output the analysis results, which include the system resource optimization suggestions and / or the task anomaly correction suggestions.

2. The method as described in claim 1, characterized in that, Before step S2, a data preprocessing step is also included, specifically: The collected application runtime data and task execution logs are cleaned and format-converted, and the processed data is stored in the target database.

3. The method as described in claim 1, characterized in that, The application runtime data includes at least one of the server's CPU utilization, memory utilization, and disk I / O; the task execution log includes at least one of the task identifier, start time, end time, execution status, and task code.

4. The method as described in claim 1, characterized in that, In step S2, the large model determines whether there is a resource bottleneck in the system based on the application's running data, and generates system resource optimization suggestions based on the determination results. And / or, the large model analyzes the error information in the task execution log and the corresponding task code to generate the task anomaly correction suggestions.

5. The method as described in claim 1, characterized in that, Following step S3, a result notification step is also included, specifically: The analysis results are sent to maintenance personnel via a preset communication method.

6. The method as described in claim 5, characterized in that, In the result notification step, the preset communication method includes email or SMS.

7. The method as described in claim 1, characterized in that, It also includes the monitoring and display steps, specifically: Based on the real-time collected application operation data and task execution logs, a visual monitoring interface is generated and updated. The visual monitoring interface is used to display at least one of the following: system resource status, task execution status, and abnormal alarm information.

8. The method as described in claim 1, characterized in that, In step S1, the program probe collects data at a fixed frequency or in an event-triggered manner.

9. A distributed planned task monitoring and analysis system based on a large model, characterized in that, include: The data collection module is used to collect application runtime data and task execution logs of planned tasks in real time through multiple program probes deployed in the distributed system. The data analysis module is used to input the application running data and task execution logs into the big model for analysis and processing. The big model generates system resource optimization suggestions based on the application running data and task anomaly correction suggestions based on the task execution logs. The results generation module is used to output analysis results that include the system resource optimization suggestions and / or the task anomaly correction suggestions.

10. The system as described in claim 9, characterized in that, It also includes a data preprocessing module, specifically: The collected application runtime data and task execution logs are cleaned and format-converted, and the processed data is stored in the target database.

Images