System maintenance method and device, equipment and storage medium

Abstract

The invention discloses a system maintenance method and device, equipment and a storage medium, and relates to the technical field of system operation and maintenance, and the method comprises the steps: obtaining historical operation data in a content generation system based on artificial intelligence, predicting an index value of the content generation system based on the historical operation data through employing a target time sequence prediction model, so as to determine an abnormal index, performing causal reasoning on the abnormal index based on a preset abnormal causal graph corresponding to the content generation system to obtain a corresponding abnormal root cause; and based on the real-time operation data, the abnormal root cause and the abnormal index of the content generation system, adjusting a preset abnormal cause and effect graph to construct a corresponding repair strategy, and maintaining the content generation system. The abnormal indexes are determined, causal reasoning is carried out based on the causal relationship graph to screen out abnormal root causes, a sensing blind area existing during anomaly detection is avoided, system repair is carried out in combination with real-time data and the causal relationship graph, time sequence relevance anomaly can be captured, and the dynamic relevance analysis capability is improved.

Description

Technical Field

[0001] The present application relates to the field of system operation and maintenance technology, and in particular to a system maintenance method, device, equipment and storage medium. Background Art

[0002] With breakthroughs in generative artificial intelligence (AIGC) technology (exemplified by GPT-4, Stable Diffusion, and Sora), its application in multimodal content generation has exploded, driving exponential growth in computing power. These systems, which generally utilize deep neural network architectures, are characterized by massive parameter sizes (billions to trillions), dynamically heterogeneous computational flows, and short model iteration cycles. This can lead to blind spots in anomaly detection during operation and maintenance. Furthermore, due to limitations in static rule configuration, they are unable to capture time-series correlation anomalies and lack the ability to dynamically analyze the correlations of multi-dimensional indicators. Therefore, improving the maintenance effectiveness of AI-generated content remains an unresolved challenge in this field. Summary of the Invention

[0003] The present application provides a system maintenance method, apparatus, device and storage medium to at least solve the problems of perception blind spots in anomaly detection in related technologies and poor correlation analysis capabilities due to the inability to capture time series correlation anomalies.

[0004] This application provides a system maintenance method, including:

[0005] Obtain historical operating data corresponding to a content generation model in an artificial intelligence-based content generation system, and use a target time series prediction model to predict a prediction indicator value of the content generation system based on the historical operating data;

[0006] Determine an abnormal indicator of the content generation system based on the predicted indicator value, and perform causal reasoning on the abnormal indicator based on a preset abnormal causal graph corresponding to the content generation system to obtain an abnormal root cause corresponding to the abnormal indicator;

[0007] Collecting real-time operating data of the content generation system, and adjusting the preset anomaly causal graph based on the real-time operating data, anomaly root causes, and anomaly indicators to obtain a target anomaly causal graph;

[0008] Based on the target anomaly causal graph, a corresponding repair strategy for the anomaly indicator is constructed, and the content generation system is maintained based on the repair strategy.

[0009] The present application also provides a system maintenance device, comprising:

[0010] An indicator prediction module is used to obtain historical operating data corresponding to the content generation model in the artificial intelligence-based content generation system, and use the target time series prediction model to predict the predicted indicator value of the content generation system based on the historical operating data;

[0011] A root cause determination module is used to determine an abnormal indicator of the content generation system based on the predicted indicator value, and perform causal reasoning on the abnormal indicator based on a preset abnormal causal graph corresponding to the content generation system to obtain an abnormal root cause corresponding to the abnormal indicator;

[0012] A causal graph adjustment module is used to collect real-time operation data of the content generation system and adjust the preset abnormality causal graph based on the real-time operation data, abnormality root causes and abnormality indicators to obtain a target abnormality causal graph;

[0013] The system repair module is used to build a repair strategy corresponding to the abnormal indicators based on the target abnormality causal graph, and maintain the content generation system based on the repair strategy.

[0014] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned system maintenance methods when executing the computer program.

[0015] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned system maintenance methods are implemented.

[0016] Through this application, the system's operating data can be obtained to determine the monitored abnormal indicators, and based on the pre-built causal relationship map, causal reasoning can be performed to screen out the most likely root cause of the abnormality. Real-time data during operation can be collected again and the causal relationship map can be adjusted in combination with the abnormal root cause so that the system can automatically repair the problem according to the causal relationship map. Therefore, the perception blind spots that exist during anomaly detection can be avoided, and the system can be repaired by combining real-time data and causal relationship maps, which can capture time-series correlation anomalies, improve dynamic correlation analysis capabilities, and effectively improve the availability and service quality assurance capabilities of the content generation system based on artificial intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A flow chart of a system maintenance method provided in an embodiment of the present application;

[0019] Figure 2 A flowchart of AIGC abnormal root cause analysis based on a dynamic causal graph is provided in an embodiment of the present application;

[0020] Figure 3 A flowchart of a specific system maintenance method provided in an embodiment of the present application;

[0021] Figure 4 A schematic diagram of root cause analysis of image blur provided in an embodiment of the present application;

[0022] Figure 5 A schematic diagram of analyzing the root causes of garbled text generation provided in an embodiment of the present application;

[0023] Figure 6 A schematic diagram of the structure of a system maintenance device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0025] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0026] Generative artificial intelligence generally adopts a deep neural network architecture, which has significant characteristics such as large parameter scale, dynamic heterogeneity of computing flow, and short model iteration cycle. As a result, it is easy to have perception blind spots at the anomaly detection level during its operation and maintenance. Moreover, due to the limitation of static rule configuration, it is unable to capture time series correlation anomalies and lacks the dynamic correlation analysis capability of multi-dimensional indicators. However, this application can obtain the system's abnormal indicators and perform causal reasoning based on a pre-built causal relationship map to screen out the root cause of the anomaly. It can avoid the perception blind spots in anomaly detection, and combine real-time data and causal relationship maps for system repair, which can capture time series correlation anomalies and improve dynamic correlation analysis capabilities.

[0027] It should be pointed out that the system in this application mainly refers to an artificial intelligence-based content generation system. Accordingly, the system maintenance method in this application is mainly used to perform runtime monitoring and maintenance of the content generation system and the content generation model (content generation model) therein.

[0028] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0029] The embodiment of the present application provides a system maintenance method. The method is described in detail below in conjunction with the execution process of the system maintenance method. The system maintenance method includes:

[0030] Step S11: Acquire historical operation data corresponding to the content generation model in the artificial intelligence-based content generation system, and use the target time series prediction model to predict the prediction index value of the content generation system based on the historical operation data.

[0031] In this embodiment, data collection and feature modeling are first required. This involves obtaining historical operational data corresponding to the content generation model in the AI-based content generation system, and using the target time series prediction model to predict the predicted indicator values ​​of the content generation system based on this historical operational data. These content generation models include, but are not limited to, AIGC-based AI-generated content models such as the text-graph model, which primarily use algorithms to automatically generate text, images, videos, and other content. Examples include GPT-4 (for text generation), Stable Diffusion (for image generation), and Sora (for video generation).

[0032] After obtaining the historical operation data corresponding to the content generation model in the artificial intelligence-based content generation system, it is also necessary to determine the change information corresponding to each historical operation indicator in the historical operation data; wherein the above-mentioned change information includes the parameter change trend, periodic change information and specific date change information of the historical operation indicator; and determine the historical indicator value corresponding to the historical operation indicator based on the change information, and then optimize the preset time series prediction model based on the historical indicator value to obtain the target time series prediction model, so as to use the target time series prediction model to make predictions based on the historical operation data.

[0033] It should be pointed out that if Figure 2As shown, data collection of the current system is first required. In this embodiment, a log collection module can be embedded in the generation model inference service of the artificial intelligence-based content generation system to collect model input, runtime indicators, model output, system environment information, etc. Among them, model input information mainly includes user request parameters (such as prompt content, generation parameter configuration, etc.); runtime indicators mainly include GPU (Graphics Processing Unit) video memory usage, latency, temperature, video memory fragmentation rate, CUDA (Compute Unified Device Architecture) kernel function call exception information, etc.; model output information includes multimodal features of generated content (text coherence score, image PSNR / SSIM (Peak Signal-to-Noise Ratio / Structural Similarity Index Measure), video frame rate stability, etc.); system environment information includes driver version, network fluctuation, etc.

[0034] Correspondingly, when using the target time series prediction model to predict the prediction index value of the content generation system based on historical operation data, several target time points can be determined, and the target time series prediction model can be used to predict the predicted operation data corresponding to each target time point based on the historical operation data, and then the prediction index value corresponding to each target time point can be determined based on the predicted operation data.

[0035] Specifically, this embodiment can start dynamic threshold anomaly detection, use the time series prediction model Prophet to predict the normal range of each indicator, and detect deviations exceeding Indicators (for example, a sudden 200% increase in video memory usage) trigger an alert. First, the collected abnormal indicators are decomposed into parameter change trends, periodic change information, and specific date change information according to the time series. These are trend items, seasonal items, and holiday items. The model structure is as follows:

[0036] ;

[0037] Among them, y(t) is the index value at time t, g(t) represents the trend term, which describes the long-term trend of the time series, s(t) is the seasonal term, which describes the periodic changes of the time series within a fixed period, and h(t) is the holiday term, which is used to capture the impact of special dates on the series. is the error term, which follows a normal distribution with a mean of 0. When performing anomaly monitoring, you can specifically collect CPU (Central Processing Unit) usage, GPU usage, GPU memory utilization, network bandwidth utilization, network latency, disk I / O read / write rate, disk space utilization, task processing time, service request success rate, etc. over the past period of time. The data format is {( ),( ),…,( )}, use the collected data to fit the Prophet model, and adjust the model parameters through the optimization algorithm so that the model can best match the trend, season and holiday characteristics of the historical data. Then, given m time points in the future, the model predicts the index value based on the fitted parameters, and predicts the index value at the next m time points to obtain the predicted sequence { }.

[0038] Step S12: determining an abnormality indicator of the content generation system according to the predicted indicator value, and performing causal reasoning on the abnormality indicator based on a preset abnormality causal graph corresponding to the content generation system to obtain an abnormality root cause corresponding to the abnormality indicator.

[0039] In this embodiment, the abnormal index of the content generation system can be determined based on the predicted index value, and causal reasoning can be performed on the abnormal index based on the preset abnormal causal graph corresponding to the content generation system to obtain the abnormal root cause corresponding to the abnormal index. Specifically, the normal index value threshold corresponding to each operating index of the content generation system can be determined first, and the difference between the predicted index value and the normal index value threshold can be determined. Then, the abnormal value in the predicted index value can be determined based on the difference and the preset difference threshold, and the abnormal index corresponding to the abnormal value can be determined. Based on the prediction sequence obtained in the above steps, this embodiment needs to calculate the standard deviation of the predicted value after obtaining the predicted value. , according to the pre-set Principle, the normal range of indicators is [ ],in When the indicator value deviates from this range, it is considered an abnormal indicator and an alarm is triggered.

[0040] Then, candidate root causes corresponding to the abnormal indicators are determined based on a preset abnormal causal graph corresponding to the content generation system, and the candidate root causes are adjusted. Causal reasoning is performed on the abnormal indicators based on the adjusted candidate root causes to obtain the abnormal root causes corresponding to the abnormal indicators in the candidate root causes. Furthermore, in the process of performing causal reasoning on the abnormal indicators based on the adjusted candidate root causes to obtain the abnormal root causes, causal reasoning can be performed on the abnormal indicators based on the adjusted candidate root causes to obtain abnormal indicator inference values ​​corresponding to the adjusted candidate root causes. The magnitude of the change between the abnormal indicator inference value and the corresponding abnormal value is then determined, and a target inference value that meets preset conditions is determined based on the magnitude of the change, so that the candidate root cause corresponding to the target inference value is used as the abnormal root cause.

[0041] like Figure 2 As shown, in this embodiment, causal intervention of abnormal indicators is required. Specifically, Do-Calculus can be used to simulate intervention for abnormal indicators and calculate the indicator changes after intervention. , j = 1, 2, ..., k, through The operation sets it to a specific state, and then calculates the candidate root cause node based on the structure of the causal graph and the known causal relationship, combined with historical data and statistical information. After intervention, abnormal indicators When making specific calculations, we can use the conditional probability relationship in the causal graph and assume that there is a causal relationship. , and the conditional probability distribution is known , in intervention Under the expected value of abnormal indicators

[0042] ;

[0043] In this way, by comparing the expected changes in abnormal indicators before and after intervention, the impact of each candidate root cause on the abnormal indicator is evaluated to screen the candidate root cause set. Based on the simulated change in the abnormal indicator after intervention, a screening threshold is set. Then, those candidate root causes that cause the abnormal indicator change to exceed the threshold are retained to form the most likely candidate root cause set, providing a basis for subsequent precise root cause location. For example, if "degradation in generated content quality" is detected, interventions are applied to possible cause nodes (such as "video memory fragmentation rate" and "input prompt complexity") in the causal graph, using virtual intervention simulations. Furthermore, if image generation is blurry, the possible cause is a high video memory fragmentation rate. Does the generation quality improve when the video memory fragmentation rate is reduced to a normal level? Does the generation quality improve when the input prompt complexity is reduced?

[0044] That is, in this embodiment, a log collection module can be embedded in the generation model inference service to collect model input, runtime indicators, model output, system environment information, etc., and through domain knowledge and statistical causal discovery algorithms, an initial causal graph is established to define the potential causal relationship between fault problems and indicator data, and between indicator data and root causes. Then, dynamic threshold anomaly detection is started, and the time series prediction model is used to predict the normal range of each indicator. The indicators trigger early warnings, and virtual interventions are performed on abnormal indicators. Based on the simulated changes in indicators after intervention, the most likely root cause candidate set is screened out to determine the root cause of the abnormality.

[0045] Step S13: collect real-time operation data of the content generation system, and adjust the preset abnormality causal graph based on the real-time operation data, abnormality root causes and abnormality indicators to obtain a target abnormality causal graph.

[0046] In this embodiment, real-time operational data from the content generation system can be collected. Based on this real-time operational data, anomaly root causes, and anomaly indicators, a preset anomaly causal graph can be adjusted to obtain a target anomaly causal graph. It should be noted that this preset anomaly causal graph is an initial causal graph established through domain knowledge and expert experience, and can define the potential causal relationship between the fault issue and its cause. For example, a video memory leak is often accompanied by an increase in video memory fragmentation, and excessive video memory usage can lead to inference delays.

[0047] Step S14: constructing a repair strategy corresponding to the abnormality indicator based on the target abnormality causal graph, and maintaining the content generation system based on the repair strategy.

[0048] In this embodiment, a repair strategy corresponding to the abnormality indicator can be constructed based on the causal relationship in the target abnormality causal graph, and the repair strategy corresponding to the abnormality indicator can be executed to maintain the content generation system.

[0049] Through the above technical solution, this embodiment proposes an AIGC anomaly root cause analysis system and method based on a dynamic causal graph. It can generate input, output and runtime indicators in the model reasoning process through multi-dimensional log point collection and construct a dynamic causal feature graph based on manual processing experience to describe the potential causal relationship between indicators, and locate the root cause of the anomaly based on virtual intervention and causal effect calculation. In this way, through causal reasoning technology, a causal relationship between anomalies and multi-dimensional indicators can be established, and combined with virtual intervention verification, causality and correlation interference can be effectively distinguished, solving the problems of high misjudgment rate and poor interpretability caused by traditional operation and maintenance systems relying on rule bases. Compared with traditional operation and maintenance systems relying on manual rules or statistical correlation analysis, the accuracy is higher.

[0050] Based on the previous embodiment, it can be seen that this application can obtain the abnormal indicators of the system, and perform causal reasoning based on the pre-built causal relationship map to screen out the root cause of the abnormality, and combine real-time data and the causal relationship map to repair the system. Next, this embodiment will explain the process of system repair in detail. Figure 3 As shown, the embodiment of the present application provides a specific system maintenance method, including:

[0051] Step S21: collect real-time operation data of the content generation system, and adjust the preset abnormality causal graph based on the real-time operation data, abnormality root causes and abnormality indicators to obtain a target abnormality causal graph.

[0052] In this embodiment, after collecting real-time operational data from the content generation system, each real-time indicator variable in the real-time operational data can be determined, and corresponding multivariate time series data can be constructed based on the real-time indicator variables. Independence tests can then be performed on each real-time indicator variable in the multivariate time series data to determine the interaction relationships between the real-time indicator variables. Based on these interaction relationships, a corresponding causal graph framework can be constructed to adjust a preset abnormal causal graph based on the causal graph framework, abnormal root causes, and abnormal indicators. Furthermore, during the adjustment of the preset abnormal causal graph, if the version of the content generation model is updated, the target operational data corresponding to the updated content generation model is determined, and the real-time operational data is updated based on the target operational data, thereby updating the preset abnormal causal graph based on the updated real-time operational data.

[0053] Specifically, in this embodiment, real-time data streams during service operation can be collected first, including computing resource indicators (CPU usage, GPU usage, memory occupancy, etc.), network resource indicators (network bandwidth utilization, network latency, etc.), storage resource indicators (disk I / O read / write rate, disk space utilization, etc.) and service performance indicators (task processing time, service request success rate, etc.), and combined with the candidate root causes in the previous embodiment, an incremental causal learning algorithm FCI (Fast Causal Inference, an algorithm for causal discovery) is used to construct a causal relationship graph. First, the collected data is organized into multivariate time series data. ,in The algorithm then performs a conditional independence test on pairs of variables in the dataset. By calculating the mutual information between the variables, it determines whether variables X and Y are independent given a set of other variables Z. If the mutual information I(X; Y|Z) = 0, then X and Y are unrelated. Based on the results of the conditional independence test, an undirected graph skeleton is constructed between the variables. For example, if X and Y are not independent under any conditions, the nodes X and Y are connected in the graph. Using the causal Markov conditions and the causal fidelity assumption, a series of rules (such as V-structure identification and collapse rules) determine the direction of the edges to establish a preliminary causal graph structure. As new real-time data continues to flow in, the algorithm repeats these steps to incrementally update the causal graph structure. Furthermore, when model version updates cause changes in data distribution, the algorithm can re-test the conditional independence and restructure the graph based on the new data, preventing the invalidation of historical causal relationships.

[0054] Step S22: constructing a repair strategy corresponding to the abnormality indicator based on the target abnormality causal graph, and maintaining the content generation system based on the repair strategy.

[0055] In this embodiment, the confidence level between each anomaly root cause and anomaly indicator is first determined based on the target anomaly causal graph. A corresponding anomaly root cause list is then constructed based on the confidence level, and a corresponding repair strategy is then constructed based on the anomaly root cause list. When constructing a corresponding repair strategy based on the anomaly root cause list, an anomaly root cause path corresponding to the anomaly indicator is generated based on the anomaly root cause list. A preset Bayesian network probabilistic inference model is then used to generate a corresponding logical chain based on the anomaly root cause path. The confidence level is then added to the logical chain to construct a corresponding repair strategy based on the logical chain.

[0056] In other words, this embodiment can calculate the root cause probability based on the strength of the causal effect. For candidate root cause nodes, the average causal effect (ACE) between them and the abnormal node is calculated. This is combined with expert experience (for example, the ACE for a video memory leak must exceed a threshold of 0.8) and a Bayesian network to output a ranked list of root causes. The strength of the causal effect is quantified by the average causal effect (ACE), calculated using the formula:

[0057] ACE=E[Y∣do(X=1)]−E[Y∣do(X=0)];

[0058] Where X is the candidate root cause node and Y is the abnormal node.

[0059] Accordingly, the content generation system is maintained based on the repair strategy. Specifically, the corresponding maintenance operations in the preset rule library can be determined based on the repair strategy, and the maintenance operations are executed to maintain the content generation system; wherein the preset rule library stores the mapping relationship between the root cause of the abnormality, the abnormality indicator and the maintenance operation.

[0060] That is to say, if Figure 4 、 Figure 5 As shown, in this embodiment, a Bayesian network probability inference model can be used to convert the root cause path into a logical chain with confidence weights, and the causal confidence of each link is marked. The system can execute an automated repair strategy based on a predefined root cause-action mapping rule library.

[0061] Step S23: Generate a current traceability report of the content generation system.

[0062] In this embodiment, after the content generation system is maintained based on the repair strategy, the current traceability report of the content generation system can be generated based on the logical chain and maintenance operations. That is, this embodiment supports the generation of an explainable traceability report, such as Figure 4 As shown in the figure, if the root cause of the image blur is "video memory leak", the repair system program is triggered to perform restart and video memory defragmentation operations; Figure 4 As shown, if the root cause of garbled text generation is an "input prompt attack," the user's IP address (Internet Protocol Address) is intercepted and adversarial training fine-tuning is initiated. If the root cause of the application's malfunction is a "driver version issue," the system automatically rolls back to a stable version and notifies operations personnel. It should be understood that fine-tuning refers to conducting small-scale training on a pre-trained model for specific task objectives (downstream tasks) and task data (downstream data), achieving minor adjustments to the pre-trained model parameters, ultimately resulting in a model adapted for the specific task and data.

[0063] In this embodiment, after the content generation system is maintained based on a repair strategy, if the maintenance is successful, the corresponding repair strategy is used as a positive sample; if the maintenance fails, the corresponding repair strategy is used as a negative sample. An enhanced dataset is then constructed based on the positive and negative samples, and the target anomaly causal graph is adjusted based on the enhanced dataset. Specifically, this embodiment can establish a three-level feedback verification process: the first level is automatic verification, which cross-validates the system's inferred root cause (for example, by calling the knowledge graph verification module to check the consistency of the "video memory leak" conclusion with the GPU memory dump file); the second level is expert review, which designs a multi-level review workflow (for example, operations engineer → senior engineer → architect) to support the review and correction of the root cause; and the third level is data enhancement, which develops an automated annotation toolchain to generate an enhanced dataset for the corrected cases. This is then used to train the labeled enhanced dataset, with accurately labeled data serving as positive samples and incorrectly labeled data serving as negative samples. This continuously improves the fault causal relationship chain graph, iteratively updates the analysis model, continuously optimizes the system, and improves the accuracy of root cause analysis.

[0064] That is, when dynamically updating the causal graph and locating the root cause, this embodiment can use an incremental causal learning algorithm, combined with real-time data streams to update the causal graph structure, solve the problem of historical causal failure caused by model version updates, calculate the root cause probability based on the causal effect strength, calculate the average causal effect of candidate root causes and anomalies, and combine expert experience and Bayesian networks to output a ranked list of root causes. Then, through causal chain visualization technology, the root cause path is converted into a logical chain that can be understood by the staff, and the causal confidence of each link is annotated to generate an explainable traceability report. After that, the automated repair strategy can be executed according to the predefined root cause-action mapping rule library, and it can automatically roll back to the stable version and notify the operation and maintenance personnel. Combined with the three-level feedback verification process, a feedback loop and model iteration are formed, supporting root cause cross-validation, manual review and correction, and annotating the analyzed root cause results. The labeled enhanced data set is trained to continuously improve the fault causal relationship link map.

[0065] Through the above technical solution, this embodiment uses real-time monitoring of anomalies and virtual intervention, combined with post-intervention indicator changes to screen out the most likely root cause candidate datasets. It then collects real-time data streams from the service runtime and, combined with the candidate root cause data, uses an incremental causal learning algorithm to construct a causal relationship graph. The system then automatically remediates issues based on the causal chain. Finally, operations personnel regularly mark and correct system processing issues and continuously optimize the analysis model to improve root cause location accuracy. This approach achieves intelligent operation and maintenance in AIGC scenarios by integrating dynamic causal discovery, virtual intervention, and automated remediation technologies. Causal chain visualization technology transforms internal anomalies in black box models into understandable logical chains, supporting manual review and audit tracking. This enhances the ability to map root cause analysis results into a visual causal chain, helping to improve interpretability and model accuracy. Furthermore, the incremental causal discovery algorithm captures dynamic changes such as model version upgrades and plugin extensions in real time, adaptively updating the causal graph and improving root cause location capabilities. By combining visual traceability with automated remediation strategies to achieve automatic problem remediation and employing manual review, the system can effectively reduce the mean time to repair (MTTR) and the average diagnostic error rate (AER) for mixed faults in the AIGC system.

[0066] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0067] like Figure 6 As shown, an embodiment of the present application further provides a system maintenance device, comprising:

[0068] An indicator prediction module 11 is used to obtain historical operating data corresponding to a content generation model in an artificial intelligence-based content generation system, and use a target time series prediction model to predict a prediction indicator value of the content generation system based on the historical operating data;

[0069] A root cause determination module 12 is configured to determine an abnormality indicator of the content generation system based on the predicted indicator value, and perform causal reasoning on the abnormality indicator based on a preset abnormality causal graph corresponding to the content generation system to obtain an abnormality root cause corresponding to the abnormality indicator;

[0070] The causal graph adjustment module 13 is used to collect real-time operation data of the content generation system and adjust the preset abnormality causal graph based on the real-time operation data, abnormality root causes and abnormality indicators to obtain a target abnormality causal graph;

[0071] The system repair module 14 is used to construct a repair strategy corresponding to the abnormality indicator based on the target abnormality causal graph, and maintain the content generation system based on the repair strategy.

[0072] For the description of the features in the embodiment corresponding to the system maintenance device, please refer to the relevant description of the embodiment corresponding to the system maintenance method, and no further details will be given here.

[0073] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above system maintenance method embodiments.

[0074] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above-mentioned system maintenance method embodiments when running.

[0075] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0076] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above system maintenance method embodiments are implemented.

[0077] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned system maintenance method embodiments are implemented.

[0078] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0079] The above is a detailed introduction to the system maintenance method, device, equipment and storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A system maintenance method, characterized in that: include: Acquire historical operating data corresponding to a content generation model in an artificial intelligence-based content generation system, and use a target time series prediction model to predict a prediction index value of the content generation system based on the historical operating data; Determining an abnormality indicator of the content generation system according to the predicted indicator value, and performing causal reasoning on the abnormality indicator based on a preset abnormality causal graph corresponding to the content generation system to obtain an abnormality root cause corresponding to the abnormality indicator; collecting real-time operating data of the content generation system, and adjusting the preset abnormality causal graph based on the real-time operating data, the abnormality root cause, and the abnormality indicator to obtain a target abnormality causal graph; A repair strategy corresponding to the abnormal indicator is constructed based on the target abnormality causal graph, and the content generation system is maintained based on the repair strategy.

2. The system maintenance method according to claim 1, characterized in that: After obtaining the historical operation data corresponding to the content generation model in the artificial intelligence-based content generation system, the method further includes: Determining change information corresponding to each historical operating indicator in the historical operating data; the change information includes parameter change trends, periodic change information, and specific date change information of the historical operating indicators; Determine the historical indicator value corresponding to the historical operation indicator according to the change information; The preset time series prediction model is optimized according to the historical indicator value to obtain the target time series prediction model, so as to use the target time series prediction model to perform prediction based on the historical operation data.

3. The system maintenance method according to claim 2, characterized in that: The method of using a target time series prediction model to predict a prediction index value of the content generation system based on the historical operation data includes: Determine a plurality of target time points, and respectively use the target time series prediction model to predict the predicted operating data corresponding to each of the target time points based on the historical operating data; The prediction index value corresponding to each target time point is determined according to the predicted operation data.

4. The system maintenance method according to claim 3, characterized in that: Determining the abnormality index of the content generation system according to the prediction index value includes: Determining a normal indicator value threshold corresponding to each operating indicator of the content generation system, and determining a difference between the predicted indicator value and the normal indicator value threshold; An abnormal value in the prediction index value is determined according to the difference and a preset difference threshold, and the abnormal index corresponding to the abnormal value is determined.

5. The system maintenance method according to claim 4, characterized in that: The performing causal reasoning on the abnormal indicator based on the preset abnormal causal graph corresponding to the content generation system to obtain the abnormal root cause corresponding to the abnormal indicator includes: Determine a candidate root cause corresponding to the abnormal indicator according to the preset abnormal cause-effect graph corresponding to the content generation system; The candidate root cause is adjusted, and causal reasoning is performed on the abnormal indicator based on the adjusted candidate root cause to obtain the abnormal root cause corresponding to the abnormal indicator in the candidate root cause.

6. The system maintenance method according to claim 5, characterized in that: The performing causal reasoning on the abnormal indicator based on the adjusted candidate root cause to obtain the abnormal root cause corresponding to the abnormal indicator in the candidate root cause includes: Performing causal reasoning on the abnormal indicators based on the adjusted candidate root causes to obtain abnormal indicator reasoning values ​​corresponding to the adjusted candidate root causes; Determining a variation range between the abnormality indicator inference value and the corresponding abnormal value, and determining a target inference value that meets a preset condition based on the variation range; The candidate root cause corresponding to the target inference value is used as the abnormal root cause.

7. The system maintenance method according to claim 1, characterized in that: After collecting the real-time operation data of the content generation system, the method further includes: Determining each real-time indicator variable in the real-time operation data, and constructing corresponding multivariate time series data according to the real-time indicator variables; Performing an independence test on each of the real-time indicator variables in the multivariate time series data to obtain an action relationship between each of the real-time indicator variables; A corresponding cause-and-effect graph framework is constructed according to the action relationship, so as to adjust the preset abnormal cause-and-effect graph according to the cause-and-effect graph framework, the abnormal root cause, and the abnormal indicator.

8. The system maintenance method according to claim 7, characterized in that: The process of adjusting the preset abnormal causal graph further includes: If the version of the content generation model is updated, determining the target operation data corresponding to the updated content generation model; The real-time operation data is updated based on the target operation data, so as to update the preset abnormality cause-effect graph based on the updated real-time operation data.

9. The system maintenance method according to any one of claims 1 to 8, characterized in that: The constructing of a repair strategy corresponding to the abnormal indicator based on the target abnormal causal graph includes: Determining the confidence between each of the abnormal root causes and the abnormal indicators based on the target abnormal causal graph; A corresponding abnormality root cause list is constructed according to the confidence level, and a corresponding repair strategy is constructed according to the abnormality root cause list.

10. The system maintenance method according to claim 9, characterized in that: The constructing the corresponding repair strategy according to the abnormal root cause list includes: Generate an abnormal root cause path corresponding to the abnormal indicator according to the abnormal root cause list, and generate a corresponding logic chain according to the abnormal root cause path using a preset Bayesian network probability inference model; The confidence level is added to the logic chain to construct the corresponding repair strategy based on the logic chain.

11. The system maintenance method according to claim 10, characterized in that: Maintaining the content generation system based on the repair strategy includes: Determining a corresponding maintenance operation in a preset rule base based on the repair strategy, and executing the maintenance operation to maintain the content generation system; the preset rule base stores a mapping relationship between the abnormality root cause, the abnormality indicator, and the maintenance operation; Accordingly, after maintaining the content generation system based on the repair strategy, the method further includes: A current traceability report of the content generation system is generated according to the logical chain and the maintenance operation.

12. The system maintenance method according to claim 1, characterized in that: After maintaining the content generation system based on the repair strategy, the method further includes: If the content generation system is successfully maintained, the corresponding repair strategy is used as a positive sample; If the content generation system fails in maintenance, the corresponding repair strategy is used as a negative sample; A corresponding enhanced data set is constructed according to the positive samples and the negative samples, and the target anomaly causal graph is adjusted based on the enhanced data set.

13. A system maintenance device, characterized in that: include: An indicator prediction module is used to obtain historical operation data corresponding to a content generation model in an artificial intelligence-based content generation system, and use a target time series prediction model to predict a prediction indicator value of the content generation system based on the historical operation data; a root cause determination module, configured to determine an abnormality indicator of the content generation system based on the predicted indicator value, and perform causal reasoning on the abnormality indicator based on a preset abnormality causal graph corresponding to the content generation system to obtain an abnormality root cause corresponding to the abnormality indicator; a causal graph adjustment module, configured to collect real-time operating data of the content generation system and adjust the preset abnormality causal graph based on the real-time operating data, the abnormality root cause, and the abnormality indicator to obtain a target abnormality causal graph; A system repair module is used to construct a repair strategy corresponding to the abnormal indicator based on the target abnormality causal graph, and maintain the content generation system based on the repair strategy.

14. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the system maintenance method according to any one of claims 1 to 12 when executing the computer program.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the system maintenance method according to any one of claims 1 to 12 when executed by a processor.

Images