Multi-modal fusion operation and maintenance method and system, electronic equipment and readable medium

Abstract

The invention provides a multi-modal fusion operation and maintenance method and system, electronic equipment and a readable medium, and relates to the technical field of system operation and maintenance. According to the scheme, operation and maintenance small models corresponding to at least two modalities are called through an operation and maintenance large model according to acquired intention categories, so that the operation and maintenance small models perform analysis based on operation and maintenance data corresponding to the modalities, and initial analysis results output by the operation and maintenance small models are acquired; and on the initial analysis results of the at least two modals, performing cross-modal association reasoning by using an association mode corresponding to the intention category to obtain a target analysis result. According to the scheme, under the cooperation of the large and small models, the large operation and maintenance model performs multi-modal small operation and maintenance model scheduling and cross-modal reasoning analysis based on intention categories, so that the accuracy of reasoning analysis is improved, the relevance can be widely and deeply mined, and false alarms are reduced; the method can adapt to variable and complex operation and maintenance requirements, accurately arrange, analyze and infer a process, perform targeted cross-modal correlation inference, and has good generalization ability.

Description

Technical Field

[0001] This disclosure relates to the field of system operation and maintenance technology, specifically to a multimodal fusion operation and maintenance method, a multimodal fusion operation and maintenance system, electronic devices, and computer-readable storage media. Background Technology

[0002] In operational scenarios, data sources for corresponding modalities are typically analyzed independently using rules and models. This localized analysis approach leads to scattered results, a lack of context, and difficulty in accurately locating root causes in complex and dynamic business environments. Furthermore, it results in a certain false positive rate, insufficient generalization ability, and difficulty in handling changing and unknown business failures.

[0003] It should be noted that the information in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this disclosure is to provide a multimodal fusion operation and maintenance method, a multimodal fusion operation and maintenance system, an electronic device, and a computer-readable storage medium. This solution can improve the accuracy of operation and maintenance inference, reduce the false alarm rate, adapt to complex and dynamically changing business environments, and has good generalization ability.

[0005] According to a first aspect of this disclosure, a multimodal fusion operation and maintenance method is provided. The method may include: obtaining at least one intent category corresponding to an operation and maintenance request; using a large operation and maintenance model, calling at least two small operation and maintenance models corresponding to the intent category to obtain initial analysis results; the initial analysis results are output by the small operation and maintenance models based on the operation and maintenance data analysis of the corresponding modality; using the large operation and maintenance model, performing cross-modal association reasoning based on the initial analysis results using the association pattern corresponding to the intent category to obtain target analysis results.

[0006] In one exemplary embodiment, the target analysis result is obtained by performing cross-modal association reasoning based on the initial analysis result using the operation and maintenance big model and the association pattern corresponding to the intent category. This includes: constructing an association graph based on the initial analysis result using the operation and maintenance big model and the association pattern corresponding to the intent category; and performing cross-modal reasoning on the association graph using the operation and maintenance big model to obtain the target analysis result corresponding to the intent category.

[0007] In one exemplary embodiment, cross-modal reasoning is performed on the association graph using a large-scale operation and maintenance model to obtain target analysis results corresponding to intent categories. This includes: performing cross-modal reasoning on the association graph using the large-scale operation and maintenance model, prioritizing the initial analysis results based on weight factors; the weight factors include at least one of historical experience, time overlap, relevance strength, and contextual information; and constructing target analysis results corresponding to intent categories based on the priority ranking, combined with the association graph and the initial analysis results using the large-scale operation and maintenance model.

[0008] In an exemplary embodiment, before obtaining the target analysis result by performing cross-modal association reasoning based on the initial analysis result using the operation and maintenance big model and the association pattern corresponding to the intent category, the method further includes: data augmentation of the initial analysis result; data augmentation includes at least one of context correction and domain knowledge supplementation.

[0009] In an exemplary embodiment, after obtaining the target analysis result by performing cross-modal association reasoning based on the initial analysis result using the operation and maintenance big model and the association pattern corresponding to the intent category, the method further includes: enhancing the reasoning of the target analysis result; the reasoning enhancement includes at least one of evidence chain verification, implicit association analysis, and cross-modal case annotation.

[0010] In one exemplary embodiment, after obtaining the target analysis result by performing cross-modal association reasoning based on the initial analysis result using the large operation and maintenance model and the association pattern corresponding to the intent category, the method further includes: iteratively updating the small operation and maintenance model based on the operation and maintenance feedback information of the target analysis result using the large operation and maintenance model; and / or iteratively updating the association pattern corresponding to the intent category based on the operation and maintenance feedback information of the target analysis result.

[0011] In one exemplary embodiment, the intent category includes fault diagnosis, and the target analysis result includes the root cause of the fault; and / or, the intent category includes performance optimization, and the target analysis result includes performance optimization suggestions; and / or, the intent category includes resource management, and the target analysis result includes resource allocation suggestions; and / or, the intent category includes fault recovery, and the target analysis result includes fault repair solutions.

[0012] In one exemplary embodiment, the intent category includes fault diagnosis, and the target analysis results include fault root causes and hierarchical decision recommendations.

[0013] According to a second aspect of this disclosure, a multimodal fusion operation and maintenance system is provided. The system includes an intent recognition module and an operation and maintenance reasoning module. The intent recognition module is used to obtain at least one intent category corresponding to an operation and maintenance request. The operation and maintenance reasoning module is used to call at least two operation and maintenance sub-models corresponding to the intent category through a large operation and maintenance model to obtain initial analysis results. The initial analysis results are output by the operation and maintenance sub-models based on the operation and maintenance data analysis of the corresponding modality. The operation and maintenance reasoning module is also used to perform cross-modal association reasoning based on the initial analysis results through the large operation and maintenance model, using the association mode corresponding to the intent category, to obtain target analysis results.

[0014] In an exemplary embodiment, the operation and maintenance reasoning module is further configured to construct an association graph based on the initial analysis results using the operation and maintenance big model and the association patterns corresponding to the intent categories; and to perform cross-modal reasoning on the association graph using the operation and maintenance big model to obtain the target analysis results corresponding to the intent categories.

[0015] In an exemplary embodiment, the operation and maintenance reasoning module is further configured to perform cross-modal reasoning on the association graph through the operation and maintenance big model, and prioritize the initial analysis results based on weight factors; the weight factors include at least one of historical experience, time overlap, relevance strength, and contextual information; and construct the target analysis results corresponding to the intent category by combining the association graph and the initial analysis results through the operation and maintenance big model based on the priority ranking.

[0016] In one exemplary embodiment, the system further includes a data enhancement module for enhancing the initial analysis results; the data enhancement includes at least one of context correction and domain knowledge supplementation.

[0017] In one exemplary embodiment, the system further includes a reasoning enhancement module for enhancing the reasoning of the target analysis results; the reasoning enhancement includes at least one of evidence chain verification, implicit association analysis, and cross-modal case annotation.

[0018] In one exemplary embodiment, the system further includes a continuous learning module, which is used to iteratively update the small operation and maintenance model based on the operation and maintenance feedback information of the target analysis results through the large operation and maintenance model; the continuous learning module is also used to iteratively update the association pattern corresponding to the intent category based on the operation and maintenance feedback information of the target analysis results.

[0019] In one exemplary embodiment, the intent category includes fault diagnosis, and the target analysis result includes the root cause of the fault; and / or, the intent category includes performance optimization, and the target analysis result includes performance optimization suggestions; and / or, the intent category includes resource management, and the target analysis result includes resource allocation suggestions; and / or, the intent category includes fault recovery, and the target analysis result includes fault repair solutions.

[0020] In one exemplary embodiment, the intent category includes fault diagnosis, and the target analysis results include fault root causes and hierarchical decision recommendations.

[0021] According to a third aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to implement the above-described method by executing the executable instructions.

[0022] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the method described above.

[0023] This disclosure provides a multimodal fusion operation and maintenance method, a multimodal fusion operation and maintenance system, an electronic device, and a computer-readable storage medium. The solution uses a large operation and maintenance model to call at least two corresponding small operation and maintenance models based on the acquired intent category. These small models then analyze the operation and maintenance data corresponding to that modality, obtaining initial analysis results. Based on these initial analysis results from at least two modalities, cross-modal correlation reasoning is performed using the association patterns corresponding to the intent category to obtain the target analysis result. This solution, through the synergy of the large and small models, utilizes the large operation and maintenance model to schedule multimodal small operation and maintenance models and perform cross-modal reasoning analysis based on intent categories. This improves the accuracy of reasoning analysis in operation and maintenance operations, enabling extensive and in-depth mining of data correlations and reducing false alarm rates. The large operation and maintenance model can adapt to changing and complex operation and maintenance needs, accurately orchestrating downstream analysis and reasoning processes, and performing cross-modal correlation reasoning using targeted association patterns, demonstrating good generalization ability.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0026] Figure 1 One of the flowcharts of a multimodal fusion operation and maintenance method provided according to an exemplary embodiment of the present disclosure is shown.

[0027] Figure 2 The second flowchart illustrates the steps of a multimodal fusion operation and maintenance method provided according to an exemplary embodiment of the present disclosure.

[0028] Figure 3 A flowchart illustrating an implementation example of an operation and maintenance method for multimodal fusion provided according to an exemplary embodiment of this disclosure is shown.

[0029] Figure 4 A structural block diagram of a multimodal fusion operation and maintenance system provided according to an exemplary embodiment of the present disclosure is shown.

[0030] Figure 5 A schematic diagram of the composition of an electronic device to which the exemplary embodiments of the present disclosure may be applied is shown. Detailed Implementation

[0031] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0032] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0033] Figure 1 This illustrates one of the steps of an operation and maintenance method for multimodal fusion provided according to an exemplary embodiment of this disclosure, such as... Figure 1 As shown, the method may include the following steps 101 to 103.

[0034] Step 101: Obtain at least one intent category corresponding to the operation and maintenance request.

[0035] In this embodiment of the disclosure, the operation and maintenance request includes descriptive information about the operation and maintenance needs during the operation of the business system. Intent identification of the operation and maintenance request can include a holistic understanding of the operation and maintenance needs, partial breakdown, and intent classification, thereby obtaining at least one intent category. Guided by the intent category, the large-scale operation and maintenance model can specifically orchestrate and execute the analysis and reasoning process, and output matching decision suggestions based on the intent category.

[0036] In this embodiment of the disclosure, maintenance requests can be input in different forms such as text, voice, image, and video; they can include descriptions of the operating status of the business system or questions based on the operating status of the business system; they can include user input from the business system or alarms automatically triggered by the business system, and this embodiment of the disclosure does not impose specific limitations on them.

[0037] In this embodiment of the disclosure, one or more intent categories can be determined through intent recognition based on the operation and maintenance request. When more than one intent category is determined, subsequent inference logic can be orchestrated and executed separately for each intent category. Intent categories can be orchestrated sequentially or in parallel based on logical relationships to improve operation and maintenance efficiency. The orchestration and execution of subsequent inference logic may include the selection and preprocessing of analysis data, the selection of analysis methods, etc. During the training phase, sample selection and labeling, downstream task design, model performance verification and evaluation, and selection of iteration methods may also be performed for intent categories.

[0038] Step 102: Using the large operation and maintenance model, call at least two corresponding small operation and maintenance models based on the intent category to obtain the initial analysis results; the initial analysis results are output by the small operation and maintenance models based on the operation and maintenance data analysis of the corresponding modes.

[0039] In this embodiment, the large-scale operation and maintenance model can be a large language model (LLM) used for task orchestration, analysis, reasoning, and decision-making in operation and maintenance operations. The large-scale operation and maintenance model can collaborate deeply with smaller, vertically oriented operation and maintenance models to address the problems of single-modality, localized, and inefficient / effective correlation in current system operation and maintenance solutions. It enables the fusion of different modalities and joint automated reasoning, thereby improving the timeliness of operation and maintenance. Furthermore, based on the dynamic adaptation of the large-scale operation and maintenance model, it can be continuously updated to enhance its generalization ability to complex and ever-changing business environments and requirements. On this basis, the aforementioned intent category recognition can be implemented by the large-scale operation and maintenance model or by other intent recognition models.

[0040] In this embodiment, the intent category points to a specific operational requirement, and analysis can be performed on different modalities for that requirement. Operational data can be data generated by the business system during operation and associated with its operational status, such as metrics, logs, call chains, etc. A modality can represent the source, format, and descriptive content of the operational data. An operational mini-model can be a vertical mini-model, that is, a mini-model that performs a specific analysis task on the operational data of a corresponding modality and outputs corresponding initial analysis results; for example, an operational mini-model can analyze one or more metrics, one or more logs, or one or more call chains.

[0041] Therefore, each modality can correspond to one or more small operation and maintenance (O&M) models. For the O&M requirements pointed to by the intent category, the large O&M model can analyze at least two corresponding modalities based on internalized O&M knowledge, and then call the corresponding small O&M model for each modality. This small O&M model then analyzes the O&M data based on the corresponding modality to obtain the initial analysis results output by the small O&M model. Specifically, when each modality corresponds to more than one small O&M model, the large O&M model can call one or more small O&M models for each modality according to the intent category.

[0042] Step 103: Using the large-scale operation and maintenance model, based on the association patterns corresponding to intent categories, perform cross-modal association reasoning based on the initial analysis results to obtain the target analysis results.

[0043] In this embodiment, the association pattern can include the association between operation and maintenance data, or the association between initial analysis results; the association pattern can describe correlational association or causal association. Each intent category can correspond to its own association pattern, and the association patterns between different intent categories can be the same, different, or partially the same. Association patterns can be mined from historical operation and maintenance cases, or discovered and updated based on business feedback during operation and maintenance. On this basis, the operation and maintenance big model can perform cross-modal association reasoning on the initial analysis results based on the association patterns corresponding to intent categories, such as identifying and matching association patterns on the initial analysis results, and obtaining the target analysis result by integrating the initial analysis results, association patterns, and other reasoning results. Among them, the association patterns and their corresponding intent categories can be internalized into the knowledge of the operation and maintenance big model through pre-training, or discovered by the operation and maintenance big model through on-site reasoning, or the storage and management of association patterns by intent category can be established for the operation and maintenance big model to call.

[0044] In this embodiment of the disclosure, the target analysis result can describe possible logical relationships such as the initial analysis result, the association pattern, and the intent category. For example, the specific requirement pointed to by the intent category may include "analyzing the reason for the timeout of downstream service A"; the initial analysis result may include "the CPU utilization of upstream service a suddenly increased" output by the operation and maintenance system 1, and "upstream service a corresponds to downstream service A" output by the operation and maintenance system 2; the association pattern may include "the processing capacity of the upstream service decreased, causing the downstream service to time out". Thus, the operation and maintenance big model performs cross-modal association reasoning based on the initial analysis result, and can obtain the target analysis result "the CPU utilization of upstream service a suddenly increased, causing its processing capacity to decrease, thereby triggering the timeout error of downstream service A".

[0045] In this embodiment, the large-scale operation and maintenance model can execute the above steps under the guidance of prompt text. The prompt text can be automatically assembled based on a preset template. The preset template can be designed according to specific operation and maintenance needs and iteratively adjusted during training and operation. The preset template of the prompt text may include role instructions to instruct the large-scale operation and maintenance model to act as an operation and maintenance worker to perform reasoning; task instructions to instruct reasoning tasks that match the intent category; and data instructions to instruct the modal, small operation and maintenance model, etc., corresponding to the intent category, or to instruct the large-scale operation and maintenance model to analyze the modal, small operation and maintenance model, etc., corresponding to the intent category.

[0046] The examples of target analysis results above are for illustrative purposes only. When the intent categories are different, the target analysis results obtained by performing matching inference tasks can include different data content. When the operation and maintenance request includes more than one intent category, the target analysis results corresponding to different intent categories can be merged based on logical relationships to obtain a multi-dimensional and comprehensive response result for the operation and maintenance request.

[0047] In an optional method embodiment of this disclosure, the intent category includes fault diagnosis, and the target analysis result includes the root cause of the fault.

[0048] In this embodiment of the disclosure, the intent category for business faults during the operation of the business system can include fault diagnosis. For example, for a fault alarm triggered by the business system, an exemplary example could be triggering an alarm message such as "Application response timeout, request failure rate increased," which indicates that the intent category includes fault diagnosis. Similarly, for a user-inputted description or question about a business system fault, an exemplary example could be receiving user input such as "My application response time has suddenly increased, and the request failure rate has increased" or "My application response time has suddenly increased, and the request failure rate has increased. What is the cause?", which indicates that the intent category includes fault diagnosis. Based on this, the large-scale operation and maintenance model can call the corresponding small-scale operation and maintenance model to analyze the multimodal operation and maintenance data related to business faults and perform inference tasks such as fault root cause analysis and verification, thereby obtaining target analysis results including fault root causes. For example, multimodal operation and maintenance data can include operational metrics, log text, call chains, etc., and operation and maintenance mini-models can include time series metric models, log models, call chain models, etc. Among them, the time series metric model can be an LSTM (Long Short Term Memory) model or a Prophet model to detect metric spikes or metric periodic anomalies, the log model can be based on BERT (Bidirectional Encoder Representation from Transformers) to cluster and analyze error patterns in log text, and the call chain model can be a graph neural network model to identify abnormal propagation paths in the graph of the call chain.

[0049] The root cause analysis can provide a descriptive text of the root cause or be presented in the form of a root cause analysis report, including data content in the form of text, tables, and visualizations. It records the data flow of the entire inference chain, including multimodal operation and maintenance data, preliminary analysis results, correlation patterns, and root causes, improving the readability and reliability of operation and maintenance analysis. It can also further verify the root cause to provide the confidence level of the root cause, and output information such as the root cause, preliminary analysis results, and confidence level in a structured manner to improve interpretability.

[0050] For example, presenting the root cause of a failure in the form of a root cause analysis report can be as follows: Root cause of the failure: Database master-slave switch; Multimodal evidence chain: 1. Time-series metrics show that the main database's QPS (queries per second) has dropped to zero; 2. The log text contains the message 'Failover triggered'. 3. The call chain shows a surge in query latency from the slave database.

[0051] And / or, the intent category includes performance optimization, and the target analysis results include performance optimization recommendations.

[0052] In this embodiment of the disclosure, the intent category for the operational status of the business system during operation can include performance optimization. Performance optimization can be an operational requirement for improving the performance of a specific service, component, or the overall business system during operation. For example, for a user input "How to improve database query performance?", the intent category can be determined to include performance optimization. Based on this, the large-scale operation and maintenance model can call the corresponding small-scale operation and maintenance model to analyze the multimodal operation and maintenance data related to specific services, components, or the business system, and perform inference tasks such as performance status and optimization suggestions, thereby obtaining target analysis results including performance optimization suggestions. For example, multimodal operation and maintenance data can include database query logs, database time-series indicators, etc., and the small-scale operation and maintenance model can be a log model, a time-series indicator model, etc.

[0053] Performance optimization recommendations may include a description of at least one optimization method for performance, and may also include a description of performance status, such as providing quantitative metrics for database query SQL. These performance optimization recommendations can be implemented manually or automatically.

[0054] And / or, the intent category includes resource management, and the goal analysis results include resource allocation recommendations.

[0055] In this embodiment of the disclosure, the intent category for the operational status of the business system during operation can include resource management. This resource management can target computing resources, storage resources, etc., and can include aspects such as resource scheduling, task scheduling, and load balancing. For example, for occupied but unallocated system resources, a user inputting "How to optimize the resource allocation of a Kubernetes cluster?" can be determined as an intent category of resource management. Based on this, the large-scale operation and maintenance model can call the small-scale operation and maintenance model to analyze multimodal data related to resources and resource allocation objects, and perform inference tasks such as resource allocation and balancing, thereby obtaining target analysis results including resource allocation suggestions. For example, multimodal operation and maintenance data can include Kubernetes event templates, service dependency graphs, etc., the small-scale operation and maintenance model can include graph neural networks, and the initial analysis results can include resource occupancy status.

[0056] Resource allocation optimization suggestions may include at least one reallocation scheme for both occupied and unallocated resources. The resource reallocation scheme can be executed manually or automatically.

[0057] And / or, the intent category includes fault recovery, and the target analysis results include fault repair solutions.

[0058] In this embodiment of the disclosure, when a failure occurs during the operation of the business system, the intent category can include fault recovery. Fault recovery can refer to an operational requirement that, based on determining the type and root cause of the business failure, provide at least one recovery solution. For example, a user inputting "How to quickly recover from a Pod crash caused by OOM (Out Of Memory)?" includes a detailed description of the business failure and the keyword "recovery" indicating the requirement, thus determining the intent category as fault recovery. Based on this, the large-scale operational model can call the small-scale operational model to analyze relevant operational data according to the specific description of the business failure, and perform reasoning tasks for generating and executing recovery solutions, thereby obtaining target analysis results including fault repair solutions. For example, multimodal operational data can include log error clustering labels, indicator spike patterns, etc., and the small-scale operational model can include log models, time-series indicator models, etc.

[0059] The fault repair plan can include at least one repair method for business faults. The fault repair plan can be implemented manually or automatically.

[0060] It should be noted that the above intent categories are merely examples. When identifying intents for maintenance requests, one or more intent categories can be obtained. For instance, when more than one intent category is obtained, for a user input such as "My application's response time has suddenly increased, and the request failure rate has risen. What are the reasons? How can I recover?", the intent category can be determined to include fault diagnosis and fault recovery. In practical applications, maintenance can also be performed through multi-turn dialogues. For example, after outputting the root cause of the fault based on the user input "My application's response time has suddenly increased, and the request failure rate has risen, what are the reasons?", a dialogue process for fault recovery can be prompted, or the fault recovery orchestration and execution can be automatically initiated. This disclosure does not impose specific limitations on this approach.

[0061] In an optional method embodiment of this disclosure, the intent category includes fault diagnosis, and the target analysis results include fault root causes and hierarchical decision recommendations.

[0062] In this embodiment of the disclosure, when the intent category includes fault diagnosis, the relevant description of fault diagnosis can be referred to, and will not be repeated here to avoid repetition. Based on this, the target analysis results may also include hierarchical decision suggestions. Hierarchical decision suggestions can be repair suggestions that classify and label one or more dimensions such as the complexity of the fault repair solution, the execution method, and the recovery effect. Complexity may include complex solutions, ordinary solutions, etc.; execution methods may include manual execution solutions, automated execution solutions, etc.; recovery effects may include global recovery, local recovery, etc. Hierarchical decision suggestions can more diversely adapt to operational needs. For example, for the root cause of multi-service outages, hierarchical decision suggestions can provide manual execution solutions; for single Pod OOM, hierarchical decision suggestions can provide automated execution solutions. Based on this, using the operational method provided in this embodiment of the disclosure, the accuracy of root cause localization in fault diagnosis can be improved by more than 40%, and the diagnosis time can be shortened from hours to minutes, significantly improving the timeliness of fault diagnosis.

[0063] Therefore, determining the intent classification is the starting point of the operation and maintenance process, guiding subsequent data selection, inference steps, and decision-making, and also affecting model training and iterative optimization. For example, the intent classification allows for the selection of data preprocessing methods, including multimodal operation and maintenance data selection and preprocessing; different intent classifications require the retrieval of different operation and maintenance mini-models, which can then undergo specialized pre-training for different intent classifications; intent classification directly corresponds to subsequent inference tasks, so the operation and maintenance large model can adjust the weights of attention mechanisms and the selection of sample associations based on intent classification, such as adjusting the attention weights of log-metric combinations with overlapping time periods in fault diagnosis; the inference results of the operation and maintenance large model can be labeled according to intent classification, such as labeling manual and automated execution scenarios for fault diagnosis and fault recovery, and the feasibility of performance optimization suggestions for performance optimization; test cases for model evaluation can also be selected based on intent categories, such as using historical fault cases to verify the accuracy of root cause localization in fault diagnosis, and evaluating the accuracy and practicality of performance optimization suggestions in performance optimization. Based on this, accurate intent classification allows for more efficient processing of operation and maintenance requests.

[0064] For example, regarding fault recovery, historical fault cases can be used to verify the accuracy of fault root cause localization through collaborative analysis of the large and small operation and maintenance models. An accuracy rate greater than or equal to 85% can be set to meet the evaluation expectations. Regarding performance optimization, the accuracy and practicality of performance optimization suggestions can be evaluated. An accuracy rate greater than or equal to 80% can be set to meet the evaluation expectations. For fault recovery, the mean time to repair (MTTR) of traditional single-modal analysis solutions can be compared. Model evaluation can be conducted offline or online.

[0065] For example, the selection of multimodal operation and maintenance data can include collecting metrics such as CPU and memory, error log text, and call chain data. Preprocessing can include outlier cleaning and time alignment. Based on this, corresponding operation and maintenance mini-models can be called to extract key features and obtain initial analysis results. For example, a time-series metric model can analyze metric spike patterns based on metric data, a log model can analyze log error clustering labels based on error logs, and a call chain model can analyze call topology delay nodes based on the call chain graph.

[0066] In an optional embodiment of the method disclosed herein, prior to step 101, the method may further include step A.

[0067] Step A: Perform data augmentation on the initial analysis results; data augmentation includes at least one of context correction and domain knowledge supplementation.

[0068] In this disclosure, based on the initial analysis results output by the small operation and maintenance model, further data augmentation can be performed. For example, data augmentation may include one or more of context correction and domain knowledge supplementation, and other data augmentation methods may also be extended; this disclosure does not impose specific limitations on these methods.

[0069] Context correction allows the large-scale operations and maintenance model to correct potential misclassifications in initial analysis results through in-context learning (ICL). For example, if a log model classifies "network timeout" as "disk I / O anomaly," it can automatically evaluate and correct this by combining contextual information such as timestamps and service names from cross-service call chain data. It can also cross-validate the accuracy of log text classification by combining call chain data and operational metrics. If a log entry incorrectly classified by the log model does not show anomalies in the call chain or exhibits no related spikes in operational metrics, the large-scale operations and maintenance model can then re-evaluate and correct it. Context correction better ensures data quality, effectively improves the accuracy of subsequent inference tasks, and enhances the reliability of the target analysis results.

[0070] Domain knowledge supplementation can involve introducing external knowledge from related fields to supplement and expand the initial analysis results, thereby increasing the breadth of the initial analysis results and supplementing implicit features not covered by the small-scale operation and maintenance model. For example, Kubernetes event templates, such as Pod OOM patterns, can be added to the initial analysis results; causal rules of service dependency graphs can be added, such as service B timeout causing queue backlog for service C. Domain indicator supplementation can be implemented based on the large-scale operation and maintenance model, or it can be implemented through query and supplementation rules and algorithms of other related knowledge. This disclosure does not impose specific limitations on this.

[0071] In an optional embodiment of the method disclosed herein, step B may be included after step 103.

[0072] Step B: Enhance the reasoning of the target analysis results; the reasoning enhancement includes at least one of the following: evidence chain verification, implicit association analysis, and cross-modal case annotation.

[0073] In this embodiment of the disclosure, based on the target analysis results obtained through cross-modal reasoning from the large-scale operation and maintenance model, reasoning enhancement can be performed on the target analysis results to improve their accuracy, reliability, and interpretability. For example, reasoning enhancement may include one or more of evidence chain verification, implicit association analysis, and cross-modal case annotation, and other reasoning enhancement methods may also be extended; this embodiment of the disclosure does not impose specific limitations on these methods.

[0074] Evidence chain verification assesses and verifies the logical plausibility between the reasoning basis and the target analysis result to determine the reliability and dependability of the target analysis result. The reasoning basis can include operational data, initial analysis results, and correlation patterns. For example, based on the causal reasoning capabilities of the large-scale operational model, target analysis results that causally contradict operational data, initial analysis results, or correlation patterns can be eliminated. Initial analysis results may undergo data augmentation, potentially leading to causal contradictions between the target analysis result and supplemented domain knowledge, or deviations from historical operational cases. The results of evidence chain verification can be provided by assigning credibility scores to the target analysis results, thereby eliminating target analysis results with credibility scores lower than expected. For example, in fault diagnosis, if the target analysis result includes a conflict between the root cause "single-node latency" and the operational data "global service unavailability," its credibility score can be lowered, thus eliminating the root cause based on the low credibility score.

[0075] Implicit correlation analysis can be achieved by generating dynamic implicit correlation features based on the reasoning basis of the large-scale operation and maintenance model. Based on cross-modal data, the large-scale operation and maintenance model mines potential correlation features that are difficult to observe directly in explicit data. It can further match and identify implicit correlation patterns and dependencies based on operation and maintenance data, initial analysis results, and correlation patterns. For example, for the target analysis results in fault diagnosis, by analyzing log text, operational metrics, and call chain data, a potential correlation between Pod restarts and sudden drops in CPU utilization can be discovered, such as finding the potential correlation feature "the probability of CPU utilization dropping within 5 minutes after a Pod restart reaches 82%". The initial analysis results can be data augmented, allowing the large-scale operation and maintenance model to further mine data by combining domain knowledge. For example, Kubernetes event templates may contain common patterns and impacts of Pod OOM events.

[0076] Building upon the mining of implicit correlation features, the rationality of these features can be evaluated and verified. This can include significance statistics to determine the frequency and significance of implicit correlation features in historical operational cases; causal correlation assessment to determine the strength of the causal relationship between implicit correlation features and the target analysis results; and further verification by incorporating domain knowledge after data augmentation of the initial analysis results. This involves analyzing the matching relationship between potential correlation features and rules and patterns in the domain knowledge. For example, if an implicit correlation feature does not match a known pattern in a Kubernetes event template, the corresponding target analysis result can be excluded or its weight reduced. The results of implicit correlation analysis can be provided by assigning credibility scores to the target analysis results, thereby eliminating target analysis results with credibility scores lower than expected.

[0077] For example, with the intent category being fault diagnosis, after obtaining the target analysis results, the large-scale operation and maintenance model further generates the implicit correlation feature "causal probability between Pod restart and CPU spikes." By introducing neighborhood knowledge "Kubernetes event templates," it is discovered that Pod restarts are usually related to insufficient memory events, and insufficient memory may lead to process termination, thereby reducing CPU utilization. This verifies the rationality of the implicit correlation feature and improves the credibility score of the corresponding root cause "Pod restarted due to insufficient memory." Based on this, and with a high credibility score, it is determined that the root cause of the fault has a certain degree of credibility, and can provide tiered decision-making suggestions for increasing memory resources or optimizing memory usage.

[0078] In an optional embodiment of the method disclosed herein, the target analysis results may be subjected to evidence chain verification, implicit association analysis, or a combination of evidence chain verification and implicit association analysis. When comprehensively performing evidence chain verification and implicit association analysis, the results can be fused, such as by addition, multiplication, averaging, weighted summation, weighted averaging, etc., to obtain a comprehensive evaluation result of the target analysis results, and thereby determine whether to remove or retain the target analysis results.

[0079] For example, a credibility assessment standard is set: when the credibility score is greater than or equal to 80%, the target analysis results can be retained; when the credibility score is less than 80% but greater than 70%, the target analysis results need further verification; and when the credibility score is less than or equal to 70%, the target analysis results should be discarded or temporarily shelved. In this case, regarding the target analysis results for fault diagnosis, the credibility score for the evidence chain verification of the root cause "database master-slave switch" is 85%, with a weight of 0.6; the credibility score for implicit association mining is 78%, with a weight of 0.4. The weighted summation of the overall credibility score is 82.2%. Since 82.2% is greater than 80%, the aforementioned root cause of the fault can be considered to have high credibility, and therefore tends to be retained.

[0080] In an optional embodiment of the method disclosed herein, before evidence chain verification and implicit association analysis, data augmentation can be performed in step A above, such as context correction, to ensure the quality of input data and reduce the impact of misjudgment on subsequent reasoning.

[0081] Cross-modal case annotation can be used to label inferences and target analysis results that have correlations, causal relationships, and cross-modal relationships, forming a multimodal evidence chain and thus creating interpretable historical operation and maintenance cases. For example, log error X and metric spike Y can be labeled with the root cause of database master-slave switch failure as a multimodal evidence chain to construct historical failure cases.

[0082] In an optional embodiment of the method disclosed herein, after step 103, the method may further include steps C1 and C2, steps C1 and C3, or steps C1, C2 and C3.

[0083] Step C1: Obtain the operation and maintenance feedback information of the target analysis results.

[0084] In this embodiment, the target analysis results can be applied to the operation and maintenance of business systems to guide the execution of specific performance maintenance and optimization, fault identification and repair, resource reallocation, and other solutions. Based on this, operation and maintenance feedback information after the application of the target analysis results can be obtained. This feedback information can include the feasibility, accuracy, and real-time effects of the target analysis results, such as whether it is executable, inexecutable, misjudged, correctly judged, meets expectations, falls short of expectations, or has no effect.

[0085] Step C2: Iteratively update the small operation and maintenance model based on the operation and maintenance feedback information from the target analysis results using the large operation and maintenance model.

[0086] In this embodiment of the disclosure, regarding the operation and maintenance feedback information, the operation and maintenance large model can dynamically update the operation and maintenance small model based on few-shot learning. For example, operation and maintenance cases marked as misjudged or incorrectly judged in the operation and maintenance feedback information can be injected into the corresponding operation and maintenance small model through few-shot learning to update the parameters, thereby improving the model accuracy.

[0087] And / or, step C3, iteratively update the association pattern corresponding to the intent category based on the operation and maintenance feedback information of the target analysis results.

[0088] In this embodiment of the disclosure, the association patterns corresponding to intent categories can also be updated based on the operation and maintenance feedback information entropy, such as adjusting or deleting existing association patterns or adding new association patterns. This update of association patterns can be triggered and executed by the large-scale operation and maintenance model, or it can trigger the execution of a pre-configured rule learning mechanism. Association patterns can be used to construct a multimodal association rule base for management and updates, or they can be transformed into knowledge of the large-scale operation and maintenance model through retraining for scheduling reference during inference. Among these, new association patterns can be discovered using machine learning models, such as frequent itemset mining using the Apriori algorithm and FP-growth algorithm, or clustering analysis using the K-means algorithm and DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm; they can also be discovered using deep learning models, such as analyzing associations using LSTM models and Transformer models, or constructing service dependency graphs and using GNN (Graph Neural Network) to analyze influence paths; or they can be discovered using pre-defined rules, such as using sliding windows to analyze the correlation of time-series feature changes, and using Pearson correlation coefficient and Spearman rank correlation coefficient features to calculate correlations.

[0089] In an optional embodiment of this disclosure, the updated association pattern can also be verified, and the machine learning model, deep learning model, large model, etc., can be retrained and updated based on the updated association pattern. Verification of the updated association pattern can include hypothesis testing, domain knowledge verification, etc. Hypothesis testing may use t-tests or chi-square tests to determine the statistical significance between features, and Granger causality tests, Bayesian networks, etc., to perform causal inference to verify the causal correlation between features. Domain knowledge verification may include historical operation and maintenance case matching, and assessment of the rationality and feasibility based on domain knowledge.

[0090] For example, after the target analysis results guide the business system to expand cloud disks, iterative updates of the associated patterns can be performed. This includes: first, data preprocessing and feature extraction; collecting descriptive information about cloud disk expansion events, such as the timestamp of the expansion, the capacity before expansion, the capacity after expansion, and whether the expansion was successful or failed; collecting log IO errors, such as the error event type, error event timestamp, the service where the error event occurred, and the node where the error event occurred; and collecting latency data, which can include service latency metrics and node latency metrics, such as average latency metrics and peak latency metrics. Based on this, data cleaning and alignment can be performed, such as removing noise and outliers from multimodal data, and aligning the timestamps of multimodal data to a unified benchmark to ensure data temporal consistency. After preprocessing, feature extraction can be performed. For example, time-series features before and after the expansion of cloud disk expansion events can be extracted, including the utilization rate and IOPS (IO requests per second) before and after the expansion. For log IO errors, log features such as error type and error frequency can be extracted. For delayed data, delay features such as the mean and variance of delay can be extracted.

[0091] Building upon this foundation, various methods can be employed to discover correlation patterns, including time-series analysis based on predefined rules, machine learning model analysis, and deep learning model analysis. For example, time-series analysis based on predefined rules can use sliding window techniques to analyze changes in log I / O errors and latency within a time window before and after a cloud disk expansion event; or, it can calculate the correlation coefficient between the cloud disk expansion event and log I / O errors and latency data. Machine learning model analysis can use clustering algorithms to cluster cloud disk expansion events and log I / O errors to discover potential correlation patterns; or, it can use frequent itemset mining algorithms to discover correlation rules between cloud disk expansion events and log I / O errors. Deep learning model analysis can use sequence models to analyze the temporal relationship between cloud disk expansion events and log I / O errors and latency data to discover correlation rules; or, it can construct a service dependency graph and use graph neural networks to analyze the impact path of cloud disk expansion events on service latency.

[0092] Furthermore, new correlation patterns can be validated, such as by using t-tests or chi-square tests to verify the statistical significance between cloud disk expansion events and log IO errors and latency data; by using Granger causality tests, Bayesian networks, and other causal inference methods to verify whether cloud disk expansion events are the cause of increased log IO errors and latency; new correlation patterns can also be evaluated based on domain knowledge or submitted to domain experts for manual review to confirm their rationality and feasibility; and historical operation and maintenance cases can be checked to confirm whether there are records of the same or similar correlation patterns.

[0093] Validated association patterns can be added to an association rule base for management. The weights of association rules can be adjusted within the rule base based on their importance and confidence. After an association pattern is updated, machine learning and deep learning models can be retrained to improve their accuracy and generalization ability by dynamically updating model parameters to adapt to the updated association pattern.

[0094] For association rules, continuous monitoring and optimization are also possible, such as real-time data stream processing: using stream processing frameworks such as Apache Kafka and Apache Flink to monitor cloud disk expansion events, log IO errors, and latency data in real time, and continuously perform dynamic updates to association patterns, triggering alarms and automated processing in a timely manner. Based on this, false alarm feedback and user feedback are collected and analyzed to update the operation and maintenance mini-model, or to update the discovery and verification process of association patterns, referring to the aforementioned step C2.

[0095] In this embodiment of the disclosure, the execution flow of the above-mentioned operation and maintenance method can be adjusted according to actual computing conditions, application requirements, etc. For example, based on the obtained target analysis results, the verification between the target analysis results and existing reasoning basis can be performed first, and then the verification between the target analysis results and domain knowledge and implicit correlation features can be performed. The iterative update of the operation and maintenance small model based on operational feedback information and the update of the correlation pattern can be performed in parallel. On this basis, the key technologies used in the execution steps can be further explained. When performing cross-modal reasoning and verifying the target analysis results, degree-modal data cross-validation can be performed to ensure data accuracy and consistency. When verifying the target analysis results, domain knowledge can be introduced for correction and verification. When iteratively updating based on operational feedback information, continuous learning is emphasized to improve the adaptability to complex and changing requirements.

[0096] Figure 2 This illustrates a second flowchart of the multimodal fusion operation and maintenance method provided according to an exemplary embodiment of the present disclosure, such as... Figure 2 As shown, the method may include the following steps 201 to 204.

[0097] Step 201: Obtain at least one intent category corresponding to the operation and maintenance request.

[0098] In this embodiment, step 201 can be referred to in the relevant description of step 101 above, and will not be repeated here to avoid repetition. The intent category can be referred to in the aforementioned... Figure 1 To avoid repetition, the relevant descriptions of the intended categories will not be repeated here.

[0099] Step 202: Using the large operation and maintenance model, call at least two corresponding small operation and maintenance models according to the intent category to obtain the initial analysis results; the initial analysis results are output by the small operation and maintenance models based on the operation and maintenance data analysis of the corresponding modes.

[0100] In this embodiment of the disclosure, step 202 can be referred to the relevant description of step 102 above. To avoid repetition, it will not be repeated here.

[0101] Step 203: Using the large-scale operation and maintenance model, construct an association graph based on the initial analysis results, according to the association patterns corresponding to intent categories.

[0102] In this embodiment of the disclosure, based on the initial analysis results obtained from the multimodal model, a correlation graph can be constructed using the large-scale operation and maintenance model to represent the correlation patterns corresponding to intent categories. The large-scale operation and maintenance model integrates the acquired initial analysis results, such as the anomaly detection results "CPU utilization surge" and "memory utilization anomaly" output by the time-series indicator model, the error clustering results "Timeout keyword" output by the log model, and the topology and time-consuming analysis such as "service call dependencies and latency" output by the call chain model. The correlation patterns corresponding to intent categories can be patterns discovered from historical operation and maintenance cases, discovered and identified through correlation analysis based on operation and maintenance data, initial analysis results, or other data, or patterns discovered and identified through correlation analysis of the current operation and maintenance data, initial analysis results, or other data. The correlation graph can display the correlation relationships between different modalities of data, such as the correlation between the CPU utilization of service D and log errors, and the call latency in the call chain.

[0103] Step 204: Using the large-scale operation and maintenance model, perform cross-modal reasoning on the association graph to obtain the target analysis results corresponding to the intent category.

[0104] In this embodiment, the multimodal association graph integrates multimodal data to provide a comprehensive inference and analysis perspective across multiple modalities, effectively assisting in identifying analysis results that might be overlooked in locally dispersed single-modal scenarios. For example, the latency issue of service E may be caused by a combination of multimodal factors, including sudden increases in CPU utilization, log error messages, and call latency. Based on this, the large-scale operation and maintenance model, through the association graph, can perform cross-modal inference for a more comprehensive analysis, improving the accuracy of the target analysis results. Furthermore, the multimodal association graph also possesses visualization characteristics, making the cross-modal inference process and its basis more transparent and intuitive, with better interpretability, and providing more credible target analysis results.

[0105] In this embodiment, the obtained target analysis results can be further verified, such as through simulation testing or log backtracking. Based on the verification results, the confidence level of the target analysis results can be adjusted, and the target analysis results can be updated based on the confidence level. For example, target analysis results with lower-than-expected confidence levels can be removed, or target analysis results with lower-than-expected confidence levels can be shelved for further verification analysis, thereby improving the accuracy and timeliness of the target analysis results.

[0106] In this embodiment of the disclosure, data augmentation can also be performed on the initial analysis results, as described above. Figure 1 To avoid repetition, the relevant descriptions of data augmentation will not be repeated here; inference augmentation can also be performed on the target analysis results, as mentioned above. Figure 1 To avoid repetition, the relevant descriptions of enhanced reasoning will not be repeated here.

[0107] In this embodiment, the target analysis results can also be verified based on a multimodal association graph, thereby updating the target analysis results. This can include evaluating the credibility of the evidence chain based on the multimodal association graph, assessing the credibility of different reasoning bases based on the association relationships represented by the association graph. For example, if the initial analysis results of multiple modalities all point to the same target analysis result, the credibility of the target analysis result is increased. Furthermore, the weight priority of the initial analysis results can be dynamically adjusted according to the credibility of the evidence chain, thereby updating their impact on the target analysis result. It can also include performing implicit association analysis based on the multimodal association graph, discovering potential implicit association features through association graph analysis. Based on this, the influence of implicit association features on the target analysis result can be enhanced by dynamically adjusting the weights. For example, if implicit association features repeatedly appear in multiple operation and maintenance cases, their weight priority is dynamically adjusted to update their impact on the target analysis result.

[0108] In an optional embodiment of the method disclosed herein, step 204 may include steps D1 to D2.

[0109] Step D1: Using the large-scale operation and maintenance model, perform cross-modal reasoning on the correlation graph and prioritize the initial analysis results based on weight factors; the weight factors include at least one of historical experience, time overlap, correlation strength, and contextual information.

[0110] In this embodiment of the disclosure, the large-scale operation and maintenance model, based on the obtained correlation graph, can perform cross-modal reasoning and analysis based on weight factors to dynamically prioritize the dispersed initial analysis results. The weight factors can represent the relevance and importance of the initial analysis results based on the multimodal correlation graph. Weight factors can include one or more of the following: historical experience, temporal overlap, correlation strength, and contextual information. Those skilled in the art can also extend the scope of weight factors according to actual needs. Historical experience can be used to prioritize initial analysis results based on the frequency of correlation between historical operational cases and specific target analysis results. For example, if error types, abnormal metrics, and specific root causes are frequently correlated, their priority can be increased. Temporal overlap can be used to prioritize initial analysis results based on the degree of overlap in timestamps between multimodal initial analysis results. For example, if log errors and sudden increases in CPU usage highly overlap, it may indicate a correlation between the two modal initial analysis results, thus adjusting their priority. Correlation strength can be used to prioritize results based on correlation, causal correlation, etc. For example, high latency in a call chain node and timeout errors in downstream services may have a causal correlation, thus adjusting their priority. Contextual information can be used to prioritize initial analysis results that may point to operational needs based on the operational status and environment of the business system. Generally, the more modalities an initial analysis result is related to, the higher its weight can be. Based on priority adjustments, the initial analysis results can be sorted to form priority-ranked initial analysis results, allowing for more efficient execution of subsequent reasoning. Therefore, the scattered clues represented by the initial analysis results can be prioritized using multiple weighted factors, clearly demonstrating their importance and relevance.

[0111] Step D2: Using the large-scale operation and maintenance model, based on priority ranking, construct the target analysis results corresponding to the intent category by combining the association graph and the initial analysis results.

[0112] In this embodiment, the large-scale operation and maintenance model can construct target analysis results corresponding to intent categories based on priority ranking, using association patterns represented by association graphs, and using initial analysis results as evidence for analysis. Priority ranking is implemented by dynamically adjusting weights, which helps the large-scale operation and maintenance model focus on initial analysis results with high importance and strong relevance, improving the accuracy and efficiency of operation and maintenance analysis, and reducing misjudgments and omissions. Furthermore, since priorities can be dynamically adjusted based on real-time collected operation and maintenance data, initial analysis results, and historical operation and maintenance cases, it can adapt to complex and ever-changing operation and maintenance needs, quickly adjusting strategies in the face of new business failures, improving operation and maintenance flexibility and robustness. On this basis, priority ranking can also reduce the impact of low-weight initial analysis results on target analysis results, and can also exclude initial analysis results with low relevance or logical contradictions, effectively improving the accuracy of constructing target analysis results and providing effective support for solving operation and maintenance needs during business system operation.

[0113] In this embodiment, the multimodal association graph and dynamically updated priority ranking can synergistically improve analytical capabilities and decision-making quality in the reasoning of target analysis results. The association graph provides a comprehensive data perspective and visualization support, while the joint priority ranking represents the importance and relevance of the initial analysis results, optimizing the operation and maintenance analysis process and results, and enabling more accurate and efficient intelligent reasoning analysis.

[0114] Figure 3 A flowchart illustrating an implementation example of an operation and maintenance method for multimodal fusion provided according to an exemplary embodiment of this disclosure is shown, such as... Figure 3 As shown, taking fault diagnosis as an example and fault root cause as an example, the method may include the following steps 301 to 307.

[0115] Step 301: Based on the user's input question text, perform intent recognition and determine the intent category, including fault diagnosis.

[0116] Step 302: Based on fault diagnosis, the operation and maintenance big data model calls the log model to detect error logs in the log text, the call time series indicator model calls the CPU indicator to detect a sudden increase in CPU utilization, and the call chain model calls the service call latency in the call chain data, to obtain the initial analysis results including error logs, sudden increases in CPU utilization, and service call latency.

[0117] Step 303: Generate a correlation graph based on the initial analysis results using the large-scale operation and maintenance model, and adjust the priority of the initial analysis results based on the correlation graph and weight factors. Sort the initial analysis results according to the adjusted priority, and perform cross-modal reasoning analysis by combining the correlation graph and the priority-sorted initial analysis results to obtain the root cause of the failure, "database master-slave switch".

[0118] Step 304: Verify the credibility of the root cause of the failure, "database master-slave switch," using the large-scale operation and maintenance model, the correlation graph, and the initial analysis results. If the verification passes, proceed to step 305; otherwise, proceed to step 303.

[0119] Step 305: Introduce the domain knowledge "Kubernetes event template" to verify the correlation between log errors and sudden increases in CPU utilization. Perform contextual correction on the log model to generate the implicit correlation feature "Master-slave database switch caused sudden increase in CPU utilization," and verify the root cause of the failure based on the implicit correlation feature. If the verification passes, proceed to step 306; otherwise, proceed to step 303.

[0120] Step 306: Provide the root cause of the problem.

[0121] Step 307: Obtain the operation and maintenance feedback information of the root cause of the fault, and update and adjust the operation and maintenance process based on the operation and maintenance feedback information.

[0122] It should be noted that the verification processes in steps 304 and 305, as well as the update and adjustment process in step 307, can be referred to in the aforementioned sections. Figure 1 or Figure 2 To avoid repetition, the relevant descriptions will not be repeated here.

[0123] This disclosure provides a multimodal fusion-based operation and maintenance (O&M) method. A large O&M model invokes at least two corresponding small O&M models based on the acquired intent category. These small O&M models analyze the O&M data corresponding to that modality, obtaining initial analysis results. Then, based on these initial analysis results from at least two modalities, cross-modal correlation reasoning is performed using the association patterns corresponding to the intent category to obtain the target analysis result. This scheme, through the synergy of the large and small models, utilizes the large O&M model to schedule multimodal small O&M models and perform cross-modal reasoning analysis based on intent categories. This improves the accuracy of reasoning analysis in O&M operations, enabling extensive and in-depth mining of data correlations and reducing false alarm rates. The large O&M model can adapt to diverse and complex O&M needs, accurately orchestrating downstream analysis and reasoning processes, and performing cross-modal correlation reasoning using targeted association patterns, demonstrating good generalization capabilities.

[0124] Figure 4 A structural block diagram of a multimodal fusion operation and maintenance system 400 provided according to an exemplary embodiment of the present disclosure is shown, such as... Figure 4As shown, the system may include an intent recognition module 401 and an operation and maintenance reasoning module 402. The intent recognition module 401 is used to obtain at least one intent category corresponding to the operation and maintenance request. The operation and maintenance reasoning module 402 is used to call at least two operation and maintenance sub-models corresponding to the intent category through the operation and maintenance large model to obtain initial analysis results. The initial analysis results are output by the operation and maintenance sub-model based on the operation and maintenance data analysis of the corresponding mode. The operation and maintenance reasoning module 402 is also used to perform cross-modal association reasoning based on the initial analysis results through the operation and maintenance large model, according to the association mode corresponding to the intent category, to obtain the target analysis result.

[0125] In an optional embodiment, the operation and maintenance reasoning module 402 is further configured to construct an association graph based on the initial analysis results using the operation and maintenance big model and the association patterns corresponding to the intent categories; and to perform cross-modal reasoning on the association graph using the operation and maintenance big model to obtain the target analysis results corresponding to the intent categories.

[0126] In an optional embodiment, the operation and maintenance reasoning module 402 is further configured to perform cross-modal reasoning on the association graph through the operation and maintenance big model, and prioritize the initial analysis results based on weight factors; the weight factors include at least one of historical experience, time overlap, relevance strength, and contextual information; and construct the target analysis results corresponding to the intent category by combining the association graph and the initial analysis results based on the priority ranking through the operation and maintenance big model.

[0127] In an optional embodiment, the system further includes a data augmentation module for augmenting the initial analysis results; the data augmentation includes at least one of context correction and domain knowledge supplementation.

[0128] In an optional embodiment, the system further includes a reasoning enhancement module for enhancing the reasoning of the target analysis results; the reasoning enhancement includes at least one of evidence chain verification, implicit association analysis, and cross-modal case annotation.

[0129] In an optional embodiment, the system further includes a continuous learning module, which is used to iteratively update the small operation and maintenance model based on the operation and maintenance feedback information of the target analysis results through the large operation and maintenance model; the continuous learning module is also used to iteratively update the association pattern corresponding to the intent category based on the operation and maintenance feedback information of the target analysis results.

[0130] In an optional embodiment, the intent category includes fault diagnosis, and the target analysis result includes the root cause of the fault; and / or, the intent category includes performance optimization, and the target analysis result includes performance optimization suggestions; and / or, the intent category includes resource management, and the target analysis result includes resource allocation suggestions; and / or, the intent category includes fault recovery, and the target analysis result includes fault repair solutions.

[0131] In an optional embodiment, the intent category includes fault diagnosis, and the target analysis results include root cause analysis and hierarchical decision recommendations.

[0132] This disclosure provides a multimodal fusion operation and maintenance system. A large operation and maintenance model invokes at least two corresponding small operation and maintenance models based on the acquired intent category. These small models analyze the operation and maintenance data corresponding to that modality, obtaining initial analysis results. Then, based on these initial analysis results from at least two modalities, cross-modal correlation reasoning is performed using the association patterns corresponding to the intent category to obtain the target analysis result. This scheme, through the synergy of the large and small models, utilizes the large operation and maintenance model to schedule multimodal small operation and maintenance models and perform cross-modal reasoning analysis based on intent categories. This improves the accuracy of reasoning analysis in operation and maintenance operations, enabling extensive and in-depth mining of data correlations and reducing false alarm rates. The large operation and maintenance model can adapt to changing and complex operation and maintenance needs, accurately orchestrating downstream analysis and reasoning processes, and performing cross-modal correlation reasoning using targeted association patterns, demonstrating good generalization ability.

[0133] It should be noted that the specific details of each module in the above-mentioned multimodal fusion operation and maintenance system have been described in detail in the implementation method section. For any undisclosed details, please refer to the implementation method section. That is, the explanation and beneficial effects of the multimodal fusion operation and maintenance method in the above-mentioned embodiments are also applicable to the multimodal fusion operation and maintenance system of this disclosure, and will not be elaborated further here.

[0134] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0135] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0136] The following reference Figure 5 To describe an electronic device 500 according to such an embodiment of the present disclosure. Figure 5 The electronic device 500 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0137] like Figure 5 As shown, the electronic device 500 is presented in the form of a general-purpose computing device. The components of the electronic device 500 may include, but are not limited to: at least one processing unit 510, at least one storage unit 520, a bus 530 connecting different system components (including storage unit 520 and processing unit 510), and a display unit 540.

[0138] The storage unit stores program code that can be executed by the processing unit 510, causing the processing unit 510 to perform the steps described in the "Exemplary Methods" section above, according to various exemplary embodiments of this disclosure.

[0139] Storage unit 520 may include readable media in the form of volatile storage units, such as random access memory (RAM) 521 and / or cache memory 522, and may further include read-only memory (ROM) 523.

[0140] Storage unit 520 may also include a program / utility 524 having a set (at least one) program module 525, such program module 525 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0141] Bus 530 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0142] Electronic device 500 can also communicate with one or more external devices 600 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 500, and / or with any device that enables electronic device 500 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 550. Furthermore, electronic device 500 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 560. As shown, network adapter 560 communicates with other modules of electronic device 500 via bus 530. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0143] Furthermore, exemplary embodiments of this disclosure also provide a computer-readable storage medium storing a program product capable of implementing the methods described above. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product including program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0144] It should be noted that the computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0145] In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0146] Furthermore, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0147] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A multimodal fusion operation and maintenance method, characterized in that, The method includes: Obtain at least one intent category corresponding to the operation and maintenance request; By using the large-scale operation and maintenance model, at least two smaller operation and maintenance models corresponding to the intent category are invoked to obtain initial analysis results; the initial analysis results are output by the smaller operation and maintenance models based on the operation and maintenance data analysis corresponding to the modality. Using the aforementioned large-scale operation and maintenance model, and based on the initial analysis results, cross-modal association reasoning is performed according to the association patterns corresponding to the intent categories to obtain the target analysis results.

2. The method according to claim 1, characterized in that, The step of obtaining target analysis results by performing cross-modal association reasoning based on the initial analysis results using the operation and maintenance big model and the association pattern corresponding to the intent category includes: Using the aforementioned large-scale operation and maintenance model, an association graph is constructed based on the initial analysis results, according to the association patterns corresponding to the intent categories. Using the aforementioned large-scale operation and maintenance model, cross-modal reasoning is performed on the association graph to obtain the target analysis results corresponding to the intent category.

3. The method according to claim 2, characterized in that, The step of performing cross-modal reasoning on the association graph using the large-scale operation and maintenance model to obtain the target analysis results corresponding to the intent category includes: Using the aforementioned large-scale operation and maintenance model, cross-modal reasoning is performed on the correlation graph, and the initial analysis results are prioritized based on weight factors; the weight factors include at least one of historical experience, time overlap, correlation strength, and contextual information. Based on the priority ranking, the operation and maintenance model, combined with the association graph and the initial analysis results, constructs the target analysis results corresponding to the intent category.

4. The method according to any one of claims 1 to 3, characterized in that, Before obtaining the target analysis result by performing cross-modal association reasoning based on the initial analysis result using the operation and maintenance big model and the association pattern corresponding to the intent category, the method further includes: The initial analysis results are augmented with data; the data augmentation includes at least one of context correction and domain knowledge supplementation.

5. The method according to any one of claims 1 to 3, characterized in that, After obtaining the target analysis result by performing cross-modal association reasoning based on the initial analysis results using the operation and maintenance big model and the association pattern corresponding to the intent category, the method further includes: The analysis results of the target are subjected to reasoning enhancement; the reasoning enhancement includes at least one of evidence chain verification and implicit association analysis.

6. The method according to any one of claims 1 to 3, characterized in that, After obtaining the target analysis result by performing cross-modal association reasoning based on the initial analysis results using the operation and maintenance big model and the association pattern corresponding to the intent category, the method further includes: The large-scale operation and maintenance model is used to iteratively update the small-scale operation and maintenance model based on the operation and maintenance feedback information from the target analysis results. And / or, based on the operation and maintenance feedback information of the target analysis results, the association pattern corresponding to the intent category is iteratively updated.

7. The method according to any one of claims 1 to 3, characterized in that, The intent category includes fault diagnosis, and the target analysis results include root causes of faults; and / or, The intent category includes performance optimization, and the target analysis results include performance optimization suggestions. And / or, The intent category includes resource management, and the objective analysis results include resource allocation recommendations. And / or, The intent category includes fault recovery, and the target analysis results include fault repair solutions.

8. The method according to claim 7, characterized in that, The intent categories include fault diagnosis, and the target analysis results include root causes of faults and hierarchical decision recommendations.

9. A multimodal fusion operation and maintenance system, characterized in that, The system includes an intent recognition module and an operation and maintenance reasoning module; The intent recognition module is used to obtain at least one intent category corresponding to the operation and maintenance request; The operation and maintenance reasoning module is used to call at least two operation and maintenance sub-models corresponding to the intent category through the large operation and maintenance model to obtain initial analysis results; the initial analysis results are output by the operation and maintenance sub-model based on the operation and maintenance data analysis of the corresponding mode; The operation and maintenance reasoning module is also used to perform cross-modal association reasoning based on the initial analysis results using the operation and maintenance big model and the association pattern corresponding to the intent category, to obtain the target analysis results.

10. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 8 by executing the executable instructions.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

Images