System interface intelligent operation and maintenance method and system based on ETL tool and large model

Abstract

The invention discloses a system interface intelligent operation and maintenance method and system based on an ETL tool and a large model, belongs to the technical field of computer system operation and maintenance and artificial intelligence, and aims to solve the technical problems that in the prior art, the system interface monitoring efficiency is low, fault processing is not timely, notification is not in place, and an alarm log is not visual. According to the technical scheme, system initialization and parameterization configuration are carried out, wherein an automatic monitoring process is set up through ETL, and a timed task automatic monitoring interface state is generated; data collection and large model analysis: collecting interface data and logs through ETL, and analyzing the logs by using the large model to generate structured fault tags; fault reasoning and early warning based on a large model: automatically associating a cross-interface fault chain based on an interface-dependent large model reasoning capability, and generating a fault analysis report; generating and executing a dynamic repair strategy: dynamically adjusting an ETL retry strategy or dynamically executing a repair script according to the fault label; and natural language alarm and interaction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer system operation and maintenance and artificial intelligence, in particular to a system interface intelligent operation and maintenance method and system based on an ETL tool and a large model. BACKGROUND

[0002] AIOps (Artificial Intelligence for IT Operations) is an IT operation solution based on artificial intelligence and machine learning technology, combined with big data analysis, algorithm model and other technical means, aiming to realize real-time monitoring, abnormal early warning and automatic and intelligent management of the whole life cycle of IT operation. The traditional operation has the following defects:

[0003] ①Inefficient processing of unstructured data: interface logs (such as Connection reset by peer) rely on manual analysis and cannot quickly locate deep faults;

[0004] ②Difficulty in locating cross-interface faults: in a multi-system dependent scenario (such as A→B→C), traditional rules cannot automatically associate slow queries of interface B with timeout faults of interface C;

[0005] ③Non-intuitive alarm information: traditional SMS only sends "interface fault", and maintenance personnel need to log in to the system to query details again, causing response delay. SUMMARY

[0006] The technical task of the present application is to provide a system interface intelligent operation and maintenance method and system based on an ETL tool and a large model, to solve the problems of low system interface monitoring efficiency, untimely fault handling, inadequate notification and non-intuitive alarm log in the prior art.

[0007] The technical task of the present application is achieved in the following manner: a system interface intelligent operation and maintenance method based on an ETL tool and a large model, which is as follows:

[0008] System initialization and parameterized configuration: use ETL to build an automatic monitoring process to generate a timing task to automatically monitor the interface state;

[0009] Data collection and large model analysis: collect interface data and logs through ETL, and use a large model to analyze the logs to generate structured fault labels;

[0010] Fault reasoning and early warning based on a large model: automatically associate cross-interface fault chains based on the reasoning ability of the interface dependent large model to generate a fault analysis report;

[0011] Dynamic repair strategy generation and execution: dynamically adjust the ETL retry strategy or dynamically execute the repair script according to the fault label;

[0012] Natural language alarm and interaction: convert fault information into natural language alarm and support interactive query details.

[0013] As preferred, the system initialization and parameterization configuration are as follows:

[0014] Build a system interface monitoring and maintenance process using ETL tools;

[0015] Configure interface basic attributes; wherein, the interface basic attributes include URL address, communication protocol and request message body structure;

[0016] Define the expected response data format; wherein, the expected response data format includes status code, data type, field meaning and return value range;

[0017] Define the inspection cycle and generate a timing task to automatically trigger interface state checking.

[0018] As preferred, the data collection and large model analysis are as follows:

[0019] Obtain the collected system interface response data; wherein, the interface response data includes interface response data, interface logs and error information; wherein, the interface response data includes status code and response time;

[0020] Use a large model to perform semantic analysis on the collected logs and error information, extract key information, and convert the key information into structured fault labels, thereby realizing log semantic analysis automation and effectively improving fault diagnosis efficiency; wherein, the key information includes error type, affected module, possible cause and timestamp.

[0021] As preferred, the fault reasoning and early warning based on the large model are as follows:

[0022] Based on historical call data, build a dependency graph of system interfaces to show the association between "interface-module-server";

[0023] When the large model analyzes the fault label of any interface, it automatically associates the cross-interface fault chain through the dependency graph and the reasoning ability of the large model itself, and quickly traces the fault.

[0024] As preferred, the dynamic repair strategy generation and execution are as follows:

[0025] According to the fault analysis results of the large model, dynamically adjust the ETL retry mechanism, as follows:

[0026] If the fault is determined to be a temporary problem caused by temporary network fluctuations, adjust the retry interval and number appropriately;

[0027] If it is judged that the service process is dead, a remote restart operation is triggered directly to realize automatic restart repair of the system interface and handle the fault in time.

[0028] As preferred, the natural language alarm and interaction are as follows:

[0029] When a fault occurs or a warning is triggered, the fault information is converted into an alarm content in a natural language form that is easy to understand through a large model; wherein the alarm content includes reason analysis, executed operation and subsequent suggestion;

[0030] The maintenance personnel are sent a short message to realize natural language interaction enhancement.

[0031] At the same time, the maintenance personnel can interactively query through replying to the short message; wherein the content of the interactive query includes querying fault details and historical similar cases.

[0032] More preferably, the fault chain reasoning supports automatic association of multi-layer interface dependency relationships.

[0033] An intelligent operation and maintenance system for system interfaces based on an ETL tool and a large model, which is used to realize the intelligent operation and maintenance method for system interfaces based on an ETL tool and a large model as described above; the system comprises:

[0034] A parameterized configuration module for utilizing an ETL to build an automatic monitoring process and generating a timing task to automatically monitor the interface state;

[0035] A data acquisition and large model analysis module for acquiring interface data and logs through an ETL and generating structured fault labels by analyzing the logs using a large model;

[0036] A fault reasoning and warning module for automatically associating cross-interface fault chains based on the interface dependency large model reasoning capability to generate a fault analysis report; wherein the fault chain reasoning supports automatic association of multi-layer interface dependency relationships;

[0037] A dynamic repair strategy generation and execution module for dynamically adjusting an ETL retry strategy or dynamically executing a repair script according to the fault labels;

[0038] A natural language alarm and interaction module for converting fault information into a natural language alarm and supporting interactive query details.

[0039] An electronic device comprising a memory and at least one processor;

[0040] Wherein the memory has stored thereon a computer program.

[0041] The at least one processor executes the computer program stored in the memory, so that the at least one processor performs the system interface intelligent operation and maintenance method based on the ETL tool and the large model as described above.

[0042] A computer readable storage medium, the computer readable storage medium has a computer program stored therein, the computer program can be executed by a processor to implement the system interface intelligent operation and maintenance method based on the ETL tool and the large model as described above.

[0043] The system interface intelligent operation and maintenance method and system based on the ETL tool and the large model of the application have the following advantages:

[0044] (I) The application collects interface data and logs through ETL, uses a large model to analyze log semantics and reason to locate faults, generates repair strategies, and real-time alarms, breaking through the processing bottleneck of traditional operation and maintenance on unstructured data, realizing rapid positioning, intelligent repair and intuitive notification of faults;

[0045] (II) The application provides a complete, intelligent and automated process, effectively solving the problems of low system interface monitoring efficiency, delayed fault handling, inadequate notification, and non-intuitive alarm logs in the prior art, greatly improving the stability, reliability and maintenance efficiency of system interfaces;

[0046] (III) The application performs log semantic analysis: using a large model to understand interface logs and error information to improve fault diagnosis efficiency;

[0047] (IV) The application realizes intelligent correlation of cross-interface fault chains: through the inference of interface dependency relationship by a large model, fault rapid tracing is realized;

[0048] (V) The application enhances natural language interaction: converting alarm information into understandable language and supporting interactive queries;

[0049] (VI) The application significantly improves the efficiency and stability of system interface operation and maintenance by integrating ETL tools and large model technology, and the specific effects are as follows:

[0050] ① The application enhances system stability: ETL automated inspection combined with large model fault prediction, real-time capture of potential interface anomalies (such as response delay, log anomaly), rapid repair through dynamic retry and remote restart mechanism, avoiding business interruption caused by sudden failure, improving core business continuity;

[0051] ② The application improves fault handling efficiency: large model analyzes log semantics and infers cross-interface fault chains, shortens positioning time, automatically executes repair scripts, and reduces manual intervention cost;

[0052] ③Alert notification is instant and accurate: the large model converts fault information into natural language alerts (such as "interface B timeout: database slow query"), which are pushed to maintenance personnel through SMS, including repair suggestions and historical case links. BRIEF DESCRIPTION OF DRAWINGS

[0053] The application will be further described below in conjunction with the accompanying drawings.

[0054] The accompanying drawings illustrate the present application. Figure 1 The flowchart of the system interface intelligent operation and maintenance method based on ETL tools and large models. DETAILED DESCRIPTION

[0055] The system interface intelligent operation and maintenance method and system based on ETL tools and large models of the present application will be described in detail below with reference to the accompanying drawings and specific examples.

[0056] Example 1:

[0057] As shown in the accompanying drawings, the present embodiment provides a system interface intelligent operation and maintenance method based on ETL tools and large models, which is specifically as follows: Figure 1

[0058] S1, system initialization and parameterized configuration: use ETL to build an automatic monitoring process to generate a timed task to automatically monitor the interface state;

[0059] S2, data collection and large model analysis: collect interface data and logs through ETL, and use a large model to analyze logs to generate structured fault labels;

[0060] S3, fault reasoning and early warning based on large models: automatically associate cross-interface fault chains based on the reasoning ability of the interface dependency large model to generate a fault analysis report;

[0061] S4, dynamic repair strategy generation and execution: dynamically adjust the ETL retry strategy or dynamically execute the repair script according to the fault label;

[0062] S5, natural language alert and interaction: convert fault information into natural language alerts and support interactive query details.

[0063] The system initialization and parameterized configuration in step S1 of the present embodiment are specifically as follows:

[0064] S101, use ETL tools to build system interface monitoring and maintenance processes;

[0065] S102, configure interface basic attributes; wherein the interface basic attributes include URL address, communication protocol and request message body structure;

[0066] ​S103, define the expected response data format; wherein, the expected response data format includes status code, data type, field meaning and return value range;

[0067] S104, define the inspection cycle, and generate a timing task to automatically trigger interface state checking.

[0068] The data collection and large model analysis in step S2 of the embodiment are specifically as follows:

[0069] S201, obtaining interface response data of the collection system; wherein, the interface response data includes interface response data, interface logs and error information; wherein, the interface response data includes status code and response time;

[0070] S202, using a large model to perform semantic analysis on the collected logs and error information, extracting key information therefrom, and converting the key information into structured fault labels, thereby realizing log semantic analysis automation and effectively improving fault diagnosis efficiency; wherein, the key information includes error type, affected module, possible cause and timestamp.

[0071] The fault reasoning and early warning based on the large model in step S3 of the embodiment are specifically as follows:

[0072] S301, constructing a dependency graph of the system interface based on historical call data, and displaying the association relationship between "interface-module-server";

[0073] S302, after the large model analyzes the fault label of any interface, automatically associating the cross-interface fault chain and quickly tracing the fault through the dependency graph and the reasoning ability of the large model.

[0074] The dynamic repair strategy generation and execution in step S4 of the embodiment are specifically as follows:

[0075] According to the fault analysis result of the large model, the ETL retry mechanism is dynamically adjusted, as follows:

[0076] If the fault is determined to be a temporary problem of temporary network fluctuation, the retry interval and number are appropriately adjusted;

[0077] If it is determined to be a more serious problem of service process freezing, a remote restart operation is directly triggered to realize automatic restart repair of the system interface and timely handle the fault.

[0078] The natural language alarm and interaction in step S5 of the embodiment are specifically as follows:

[0079] S501, when a fault occurs or an early warning is triggered, converting the fault information into alarm content in an understandable natural language form through the large model; wherein, the alarm content includes cause analysis, executed operation and subsequent suggestion;

[0080] S502, send to the maintenance personnel by short message, realize natural language interaction enhancement;

[0081] S503, at the same time, support the maintenance personnel to interactively query through replying short message and the like; wherein, the content of the interactive query includes querying fault details and historical similar cases.

[0082] The fault chain reasoning in step S3 of the embodiment supports automatic association of multi-layer interface dependency.

[0083] Embodiment 2:

[0084] The embodiment provides an ETL tool and large model-based system interface intelligent operation and maintenance system, which is used to implement the ETL tool and large model-based system interface intelligent operation and maintenance method in embodiment 1; the system comprises:

[0085] A parameterized configuration module is configured to use ETL to build an automatic monitoring process and generate a timing task to automatically monitor an interface state.

[0086] A data acquisition and large model analysis module is configured to acquire interface data and logs through ETL and use a large model to analyze the logs to generate structured fault labels.

[0087] A fault reasoning and early warning module is configured to automatically associate cross-interface fault chains based on the interface dependency large model reasoning capability to generate a fault analysis report; wherein, the fault chain reasoning supports automatic association of multi-layer interface dependency relationships.

[0088] A dynamic repair strategy generation and execution module is configured to dynamically adjust an ETL retry strategy or dynamically execute a repair script according to the fault labels.

[0089] A natural language alarm and interaction module is configured to convert fault information into a natural language alarm and support interactive query details.

[0090] Embodiment 3:

[0091] The embodiment also provides an electronic device comprising a memory and a processor.

[0092] The memory stores computer execution instructions.

[0093] The processor executes the computer execution instructions stored in the memory, so that the processor executes the ETL tool and large model-based system interface intelligent operation and maintenance method in any embodiment of the application.

[0094] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, etc. The processor can be a microprocessor or the processor can be any conventional processor.

[0095] The memory can be used to store computer programs and / or modules, and the processor can realize various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store operating systems, application programs required by at least one function, etc.; and the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can also include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0096] Embodiment 4:

[0097] The embodiment also provides a computer-readable storage medium, in which a plurality of instructions are stored, the instructions are loaded by a processor to make the processor execute the system interface intelligent operation and maintenance method based on the ETL tool and the large model in any embodiment of the application. Specifically, a system or device equipped with a storage medium can be provided, and the storage medium stores software program codes for realizing the functions of any one of the above embodiments, and the computer (or CPU or MPU) of the system or device reads and executes the program codes stored in the storage medium.

[0098] In this case, the program codes read from the storage medium can realize the functions of any one of the above embodiments, and therefore the program codes and the storage medium storing the program codes constitute a part of the application.

[0099] The storage medium for providing the program codes includes a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as a CD-ROM, a CD-R, a CD-RW, a DVD-ROM, a DVD-RYM, a DVD-RW, a DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program codes can be downloaded from a server computer via a communication network.

[0100] Furthermore, it should be understood that, not only the program codes read by the computer can be executed to complete the actual operation, but also the operating system and the like operating on the computer can be instructed based on the program codes to complete part or all of the actual operation, thereby realizing the function of any one of the above embodiments.

[0101] Furthermore, it should be understood that, not only the program codes read by the computer can be executed to complete the actual operation, but also the operating system and the like operating on the computer can be instructed based on the program codes to complete part or all of the actual operation, thereby realizing the function of any one of the above embodiments.

[0102] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the above embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A system interface intelligent operation and maintenance method based on ETL tools and large models, characterized in that, The method is as follows: System initialization and parameterized configuration: Utilize ETL to build an automated monitoring process and generate scheduled tasks to automatically monitor interface status; Data Acquisition and Large Model Analysis: Collect interface data and logs through ETL, and use the large model to analyze the logs to generate structured fault labels; Fault reasoning and early warning based on large models: Automatically associate cross-interface fault chains based on the reasoning capabilities of interface-dependent large models and generate fault analysis reports; Dynamic repair strategy generation and execution: dynamically adjust the ETL retry strategy or dynamically execute repair scripts based on the fault labels; Natural Language Alarms and Interaction: Converts fault information into natural language alarms and supports interactive querying of details.

2. The intelligent operation and maintenance method for system interfaces based on ETL tools and large models according to claim 1, characterized in that, The system initialization and parameterized configuration are as follows: Utilize ETL tools to build system interface monitoring and maintenance processes; Configure the basic properties of the interface; these properties include the URL address, communication protocol, and request message body structure. Define the expected response data format; the expected response data format includes status code, data type, field meaning, and return value range; Define the inspection cycle and generate a scheduled task to automatically trigger interface status checks.

3. The intelligent operation and maintenance method for system interfaces based on ETL tools and large models according to claim 1, characterized in that, The data collection and large model analysis are detailed below: Acquire the response data from the data acquisition system interface; the interface response data includes the interface response data, interface logs, and error information; the interface response data includes the status code and response time. A large model is used to perform semantic analysis on the collected logs and error information, extract key information, and transform the key information into structured fault labels, thereby automating log semantic parsing and effectively improving fault diagnosis efficiency. The key information includes error type, affected modules, possible causes, and timestamp.

4. The intelligent operation and maintenance method for system interfaces based on ETL tools and large models according to claim 1, characterized in that, The fault reasoning and early warning based on large models are as follows: A dependency graph of system interfaces is constructed based on historical call data to show the relationship between "interface-module-server"; Once the large model parses the fault label of any interface, it automatically associates fault chains across interfaces through the dependency graph and the large model's own reasoning capabilities, enabling rapid fault tracing.

5. The intelligent operation and maintenance method for system interfaces based on ETL tools and large models according to claim 1, characterized in that, The dynamic repair strategy generation and execution are as follows: Based on the fault analysis results of the large model, the ETL retry mechanism is dynamically adjusted as follows: If the fault is determined to be a temporary problem caused by temporary network fluctuations, adjust the retry interval and number of attempts accordingly. If the problem is determined to be a serious issue of service process freezing, a remote restart operation will be triggered directly to automate the restart and repair of the system interface and handle the fault in a timely manner.

6. The intelligent operation and maintenance method for system interfaces based on ETL tools and large models according to claim 1, characterized in that, The specific details of natural language alerts and interactions are as follows: When a fault occurs or an alert is triggered, the fault information is transformed into easily understandable natural language alarm content through a large model; the alarm content includes cause analysis, actions already performed, and follow-up suggestions; Sending messages to maintenance personnel enhances natural language interaction. Meanwhile, maintenance personnel can perform interactive queries by replying to text messages; the interactive queries include querying fault details and historical similar cases.

7. The intelligent operation and maintenance method for system interfaces based on ETL tools and large models according to any one of claims 1 to 6, characterized in that, Fault chain reasoning supports automatic association of multi-level interface dependencies.

8. An intelligent operation and maintenance system based on ETL tools and large models for system interfaces, characterized in that, This system is used to implement the intelligent operation and maintenance method for system interfaces based on ETL tools and large models as described in any one of claims 1 to 7; the system includes: The parameterized configuration module is used to build an automated monitoring process using ETL and generate scheduled tasks to automatically monitor the interface status. The data acquisition and large model parsing module is used to collect interface data and logs through ETL, and use the large model to parse the logs to generate structured fault labels; The fault reasoning and early warning module is used to automatically associate cross-interface fault chains based on the large model reasoning capability of interface dependencies and generate fault analysis reports; among them, fault chain reasoning supports automatic association of multi-level interface dependencies. The dynamic repair strategy generation and execution module is used to dynamically adjust the ETL retry strategy or dynamically execute repair scripts based on the fault labels. The natural language alarm and interaction module is used to convert fault information into natural language alarms and supports interactive query of details.

9. An electronic device, characterized in that, include: Memory and at least one processor; The memory contains computer programs; The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the intelligent operation and maintenance method based on ETL tools and large models as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by a processor to implement the intelligent operation and maintenance method for system interfaces based on ETL tools and large models as described in any one of claims 1 to 7.

Images