System fault diagnosis method and device based on multiple modes, equipment and medium

Abstract

The invention relates to the technical field of data processing, and discloses a multi-modal-based system fault diagnosis method, device, equipment and medium, which can be applied to the financial and medical fields, and the method comprises the following steps: processing obtained historical fault event data to obtain a historical vector association set, and performing index construction to obtain a multi-modal fault knowledge base object; performing isomorphic preprocessing on the real-time multi-modal data based on an isomorphic rule to obtain a query representation set; performing staged similarity retrieval to obtain a target historical event subset; and inputting the corresponding data into the model for processing to obtain current fault diagnosis data. In the invention, aiming at the problems that the fault diagnosis of the existing intelligent operation and maintenance system excessively depends on a single data mode and the model adaptability is poor, a multi-mode fault knowledge base object can be constructed, the staged similarity retrieval can be carried out, and finally a diagnosis scheme is output by utilizing a large model, so that the multi-mode data can be fused and the model adaptability can be improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a system fault diagnosis method, apparatus, equipment and medium based on multimodality. Background Technology

[0002] In critical industry scenarios with strong regulation and high continuity, such as finance and healthcare (e.g., core financial transaction clearing systems, medical HIS / EMR and PACS systems), existing intelligent operation and maintenance (AIOps) systems mostly rely on a single data modality (logs only or performance metrics only), making it difficult to comprehensively utilize the complementary information from logs, performance metrics, and natural language fault reports. When the system architecture or data distribution changes, traditional supervised learning-based models are prone to degradation and have high maintenance costs. At the same time, the system's understanding and reasoning capabilities are insufficient when faced with colloquial and ambiguous fault reports, making it difficult to automatically generate structured and executable repair plans, thereby lengthening the average fault repair time and increasing reliance on manual intervention. Summary of the Invention

[0003] This invention provides a system fault diagnosis method, apparatus, equipment, and medium based on multimodal operation and maintenance, in order to solve the technical problem that the fault diagnosis of existing intelligent operation and maintenance systems relies too much on a single data modality and has poor model adaptability.

[0004] Firstly, a multimodal system fault diagnosis method is provided, including: Historical fault event data is acquired, and modal preprocessing is performed on the historical fault event data to obtain a historical vector association set; The historical vector association set is indexed and constructed to obtain a multimodal fault knowledge base object; Based on the preset isomorphic rules in the multimodal fault knowledge base object, the real-time multimodal data of the current fault to be diagnosed isomorphically preprocessed to obtain a query representation set. Based on the query representation set and the multimodal fault knowledge base object, a phased similarity retrieval is performed to obtain a subset of target historical events; The target subset of historical events and the query representation set are input together into a preset evaluation large language model for context matching and reasoning to obtain the best matching historical event; The best matching historical events and the query representation set are input together into a preset generative large language model to generate a solution and obtain the current fault diagnosis data.

[0005] Secondly, a multimodal system fault diagnosis device is provided, comprising: The data acquisition module is used to acquire historical fault event data and perform modal preprocessing on the historical fault event data to obtain a historical vector association set; The index building module is used to build an index on the historical vector association set to obtain a multimodal fault knowledge base object; The data processing module is used to perform isomorphic preprocessing on the real-time multimodal data of the current fault to be diagnosed based on the preset isomorphic rules in the multimodal fault knowledge base object, so as to obtain a query representation set. The similarity retrieval module is used to perform phased similarity retrieval based on the query representation set and the multimodal fault knowledge base object to obtain a subset of target historical events; The event reasoning module is used to input the target subset of historical events and the query representation set into a preset evaluation large language model for context matching reasoning to obtain the best matching historical event; The data output module is used to input the best matching historical events and the query representation set into a preset large language model to generate a solution and obtain the current fault diagnosis data.

[0006] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described multimodal system fault diagnosis method.

[0007] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described multimodal system fault diagnosis method.

[0008] In the above-mentioned scheme implemented by a multimodal system fault diagnosis method, device, equipment, and medium, historical fault event data can be acquired, and modal preprocessing can be performed on the historical fault event data to obtain a historical vector association set; the historical vector association set can be indexed and constructed to obtain a multimodal fault knowledge base object; based on the preset isomorphism rules in the multimodal fault knowledge base object, the real-time multimodal data of the current fault to be diagnosed can be preprocessed to obtain a query representation set; based on the query representation set and the multimodal fault knowledge base object, a phased similarity retrieval can be performed to obtain a target historical event subset; the target historical event subset and the query representation set can be jointly input into a preset evaluation large language model for context matching reasoning to obtain the best matching historical event; the best matching historical event and the query representation set can be jointly input into a preset generative large language model for scheme generation to obtain the current fault diagnosis data. In this invention, to address the problem that fault diagnosis in existing intelligent operation and maintenance systems relies excessively on a single data modality and has poor model adaptability, a multimodal fault knowledge base object can be constructed, and phased similarity retrieval can be performed. Finally, a large model is used to output a diagnostic solution. In this way, multimodal data can be integrated and the model adaptability can be improved. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart illustrating a system fault diagnosis method based on multimodality according to an embodiment of the present invention; Figure 2 yes Figure 1 A detailed implementation flow diagram of step S10 Figure 1 ; Figure 3 yes Figure 1 A detailed implementation flow diagram of step S20 Figure 2 ; Figure 4 yes Figure 1 A detailed implementation flow diagram of step S30 Figure 3 ; Figure 5 yes Figure 1 A detailed implementation flow diagram of step S40 Figure 4 ; Figure 6 yes Figure 1 A detailed implementation flow diagram of step S50 Figure 5 ; Figure 7 yes Figure 1 A detailed implementation flow diagram of step S60 Figure 6 ; Figure 8 This is a schematic diagram of a multimodal system fault diagnosis device according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 10 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] Please see Figure 1 As shown, Figure 1 A flowchart illustrating a multimodal system fault diagnosis method provided in this embodiment of the invention includes the following steps: S10: Obtain historical fault event data and perform modal preprocessing on the historical fault event data to obtain a historical vector association set.

[0013] First, historical fault event data is acquired, and then modal preprocessing is performed on the historical fault event data to obtain a historical vector association set. This process is equally applicable to scenarios such as "abnormal transaction peak latency" in bank payment gateways and "slow prescription issuance response" in hospital outpatient and emergency systems, to ensure unified representation across systems and modalities.

[0014] Combination Figure 2 As shown, step S10 specifically includes the following steps: S101: Read the resolved and closed fault work orders to obtain the historical work order set.

[0015] All resolved and closed fault tickets can be retrieved from the fault tracking platform to form a historical ticket collection. To ensure industry adaptability, this step can cover tickets from the financial sector (such as payment gateways, clearing batch processing, and transaction matching) and the medical sector (such as HIS / EMR opening and billing, PACS image archiving and distribution), ensuring that subsequent database construction has a representative data foundation across business domains. Each ticket records its final status after the processing loop is closed, so that a stable set of "positive examples" (i.e., events with confirmed root causes and solutions) can be extracted later.

[0016] S102: Aggregate the historical work order set based on the work order identifier to obtain the corresponding full lifecycle data, thus obtaining historical fault event data.

[0017] Using the work order identifier (unique fault ID) as the primary key, multi-source data generated throughout the entire lifecycle of the work order is aggregated to obtain historical fault event data. The data includes at least three core modalities: (1) system log data, which is semi-structured text logs collected by the relevant microservice containers during the fault occurrence period; (2) system performance index data, which is multi-dimensional time series collected by the monitoring system (such as core indicators such as CPU utilization, memory usage, network / disk I / O, and average service latency); (3) fault report data, including user fault reports, SRE analysis records, and the text of the finally determined root causes and solutions. The aggregation of the above three modalities ensures a deep and comprehensive picture of the entire "phenomenon-system-business" link in scenarios such as peak windows of financial clearing and high concurrency of medical outpatient and emergency departments.

[0018] S103: The historical fault event data is processed by timestamp parsing, noise reduction and field standardization on the log channel to obtain the first multimodal processing intermediate object.

[0019] Prioritize the retention or standardization of log timestamps and key fields (such as service name, instance ID, thread / request ID, etc.), and combine them with a large language model fine-tuned by instructions to extract key lines from the original logs: only retain statements with high diagnostic value such as "warning", "error", "fatal", "exception", and lines that explicitly record user input, special system events or performance metric readings. At the same time, convert the retained content into a unified text format to obtain the first multimodal processing intermediate object, so as to eliminate the log style differences generated by different microservices and components and improve the robustness of subsequent vectorization and similarity calculation.

[0020] S104: Vectorize the historical fault event data on the log channel and align it according to the time index to obtain the second multimodal processing intermediate object.

[0021] The standardized log text is vectorized and aligned by time index to obtain the second multimodal processing intermediate object. Specifically, historical fault event data is input into the text embedding model to generate high-dimensional log embedding vectors. To ensure comparability, the log embeddings are aligned with the time indices corresponding to the fault occurrence intervals (e.g., "T−Δ, T+Δ" windows) to support subsequent approximate nearest neighbor retrieval and historical event backtracking based on time consistency. This process is applicable to real-time business window scenarios such as "sudden increase in timeout rate" in financial payment channels and "archiving retry anomalies" in medical PACS.

[0022] S105: The historical fault event data is normalized and the gradient, mean and standard deviation are calculated on the indicator channel to obtain the third multimodal processing intermediate object.

[0023] To eliminate absolute value biases due to different hardware and environments, the time series of each indicator can be first normalized to the [0,1] interval using Min-Max. Then, the first gradient (characterizing the rate of change) and the mean and standard deviation (characterizing fluctuation and stability) are calculated. The three types of statistical features corresponding to the seven core indicators are then concatenated to form a fixed-length 21-dimensional performance indicator vector, resulting in the third multimodal processing intermediate object. This allows for efficient and stable similarity calculation without relying on the sampling duration.

[0024] S106: Encapsulate the first multimodal processing intermediate object, the second multimodal processing intermediate object, and the third multimodal processing intermediate object to obtain a historical vector association set.

[0025] The three types of intermediate objects mentioned above are encapsulated to obtain a historical vector association set. During the encapsulation process, using the unique fault ID as the primary key, the "log vector" in the second multimodal processing intermediate object and the "performance indicator vector" in the third multimodal processing intermediate object are bound at the primary key level, and a one-to-one mapping relationship is established with the corresponding "fault report data." This ensures that subsequent vector retrieval, text backtracking, root cause analysis, and semantic alignment of solutions can be completed at the same historical event granularity. This encapsulation result provides a structured and associative underlying data unit for subsequent index construction and online retrieval.

[0026] S20: Index and construct the historical vector association set to obtain a multimodal fault knowledge base object.

[0027] A multimodal fault knowledge base object that can be retrieved and invoked online is constructed based on the historical vector association set.

[0028] Combination Figure 3 As shown, step S20 specifically includes the following steps: S201: Perform field normalization on the historical vector association set to obtain a normalized entry set.

[0029] First, the historical vector association set is normalized to form a set of normalized entries. Field normalization includes at least: a unified and unique fault ID (as the primary key), timestamp format and time zone, service name, instance ID, tenant identifier, modal tags (log vector, performance metric vector, fault report text) and their minimum required metadata; structural flattening and key name standardization of text and JSON-type fields; ensuring that the time indexes of logs and metrics are aligned so that the same isomorphic rules are used for comparison with the subsequent real-time query side. Normalization provides a data consistency prerequisite for stable similarity retrieval and semantic backtracking across industries and systems.

[0030] S202: Write the log vector and performance index vector in the normalized entry set into the corresponding vector database to generate an approximate nearest neighbor index, and obtain indexed vector storage.

[0031] Log vectors and performance metric vectors from the normalized entry set are written into a vector database, and an efficient index supporting Approximate Nearest Neighbor (ANN) retrieval (such as an index structure based on cosine similarity or Euclidean distance) is constructed to obtain indexed vector storage. Log vectors are used to represent code execution and exception patterns, while performance metric vectors (a fixed-length representation obtained by concatenating the normalized gradients, means, and standard deviations of multiple metrics) are used to represent macroscopic performance status. The database sharding and index sharding design allows for coarse screening of log similarity and fine screening based on metric similarity during the online phase.

[0032] S203: Write the fault report text in the standardized entry set into object storage or document database to obtain a document storage set.

[0033] The fault report text (including user fault descriptions, SRE process records, root causes, and solutions) from the standardized item set is written to object storage or a document database to obtain a document storage set. This set preserves the original semantic material, allowing for retrospective analysis of business context and handling basis during subsequent evaluation, reasoning, and solution generation stages, thus meeting the audit traceability needs in financial compliance and medical scenarios.

[0034] S204: Using the unique fault ID in the normalized entry set as the primary key, establish a correspondence between the vector primary key in the indexed vector storage and the document primary key in the document storage set, and generate a mapping table.

[0035] Using a unique fault ID as the primary key, a one-to-one correspondence is established between the vector primary keys (vector IDs of logs and metrics) in the indexed vector storage and the document primary keys in the document storage collection in the mapping table. This relationship enables precise binding across modalities, allowing for rapid backtracking to the corresponding text reports and historical solutions after candidate results are obtained through vector retrieval during online diagnostics, thus supporting semantic-level comparison and contextual assembly during the generation phase.

[0036] S205: Merge and encapsulate the indexed vector storage, document storage set, and mapping relationship table to obtain a multimodal fault knowledge base object.

[0037] The indexed vector storage, document storage set, and mapping relationship table are merged and encapsulated to obtain a multimodal fault knowledge base object that can be called as a service. This multimodal fault knowledge base object exposes a standardized retrieval interface, which supports both staged similarity retrieval using "log vector to be queried / performance indicator vector" as the key and cross-modal backtracking by primary key. At the same time, since its internal structure adopts the same modality and field specifications as the real-time side, it can ensure efficient matching and semantic adjudication of subsequently received real-time system logs, performance indicators, and fault reports within the homogeneous space.

[0038] S30: Based on the preset isomorphic rules in the multimodal fault knowledge base object, perform isomorphic preprocessing on the real-time multimodal data of the current fault to be diagnosed to obtain a query representation set.

[0039] Based on the pre-defined isomorphism rules in the multimodal fault knowledge base object, isomorphic preprocessing can be performed on the real-time multimodal data of the current fault to be diagnosed, thus obtaining a query representation set. This isomorphism processing ensures that data such as the peak window of financial clearing batch processing and the high-concurrency period of hospital image storage are comparable and aligned in the representation space.

[0040] Combination Figure 4 As shown, step S30 specifically includes the following steps: S301: When an abnormality is detected in a fault work order, a diagnostic trigger object is generated, and event trigger data is obtained by integration.

[0041] When the alarm system detects an anomaly or the SRE team creates a new fault ticket, the system automatically or manually triggers the diagnostic process to generate a diagnostic trigger object. This trigger object aggregates basic event elements, including a unique event ID, trigger time window, alarm source and severity level, a list of potentially affected services, and environmental tags (production / pre-production / tenant information, etc.), forming event trigger data. During peak periods of bank clearing batch processing or high-concurrency periods of hospital HIS medical orders, the trigger object can accurately define the fault observation window and service boundaries, providing an anchor point for subsequent data collection and alignment.

[0042] S302: Use the event-triggered data acquisition to collect the real-time multimodal raw data corresponding to the current fault, and obtain the multimodal raw data packet.

[0043] Based on event-triggered data collected using the same channel settings as historical sample collection phases, real-time multimodal raw data of the current fault is collected, yielding multimodal raw data packets. These data packets include at least three core modalities: first, semi-structured system logs from relevant microservice containers; second, multivariate time-series performance metrics collected by the monitoring system (such as CPU utilization, memory usage, network / disk I / O, average service latency, etc.); and third, natural language fault reports in work orders (including the user's initial fault report and SRE on-site records). This data collection strategy, consistent with historical rules, ensures complete consistency in data type and semantic meaning between the online and offline sides in scenarios such as financial transaction / clearing and medical PACS archiving / distribution.

[0044] S303: The multimodal raw data packet is vectorized in the log channel according to the isomorphic rules consistent with the multimodal fault knowledge base object to obtain the log vector to be queried.

[0045] The multimodal raw data packets are processed and represented in the same way as the database creation through the log channel: first, timestamp parsing and field standardization (service name, instance ID, request ID, etc.) are performed, and then the large language model with fine-tuned instructions is called to extract key lines from the raw logs and unify the text format; then the obtained clean log text is sent into the text embedding model to generate a high-dimensional log vector, and aligned according to the fault observation window and time index to obtain the log vector to be queried.

[0046] S304: Based on the isomorphic rule, the system performance indicators in the multimodal raw data packet are vectorized using the log vector to be queried to obtain the performance indicator vector to be queried.

[0047] The performance metrics are vectorized according to the isomorphic rules consistent with the database creation phase to obtain the performance metric vector to be queried.

[0048] S305: The fault report data in the multimodal raw data packet is normalized using the performance index vector to be queried, and the fault report data is obtained.

[0049] By utilizing the time window and service context defined by the performance metric vector to be queried, the fault report text in the multimodal raw data packets is normalized to obtain fault report data that can be used for downstream evaluation and generation. Normalization includes: extracting key information such as phenomenon description, scope of impact, attempted operations, and constraints based on field templates; standardizing time and unit formats; and resolving colloquial expressions and ambiguous words. The context anchoring in this step relies on the aforementioned temporal alignment of logs and metrics to ensure that the report text and objective observations are within the same semantic framework.

[0050] S306: Encapsulate the fault report data, the log vector to be queried, and the performance indicator vector to be queried to obtain a query representation set.

[0051] The fault report data, the log vector to be queried, and the performance indicator vector to be queried are encapsulated to form a query representation set. This query representation set is a set of cross-modal online retrieval units with consistent primary keys. Its type and processing method are consistent with the historical database construction stage (steps S101 and S102), facilitating direct entry into the subsequent phased similarity retrieval and semantic reasoning process. For example, in scenarios such as a sudden increase in timeout rate in financial payment channels or an abnormal archive failure rate in medical PACS, the query representation set can ensure that the online side is highly isomorphic to the historical knowledge base in terms of vector space and textual context, laying the foundation for Top-K candidate convergence and best matching decision.

[0052] S40: Based on the query representation set and the multimodal fault knowledge base object, perform phased similarity retrieval to obtain a subset of target historical events.

[0053] It can perform phased similarity retrieval (such as two-stage retrieval) based on query representation sets and multimodal fault knowledge base objects to obtain a subset of target historical events, thereby achieving fast and robust candidate convergence.

[0054] Combination Figure 5 As shown, step S40 specifically includes the following steps: S401: Parse the log vector to be queried and the performance index vector to be queried in the query representation set to obtain the retrieval input object.

[0055] First, the query representation set is parsed to extract the log vector and performance metric vector to be queried. These two vectors, along with their time indexes, service instance identifiers, and other necessary metadata, are then uniformly encapsulated into a retrieval input object. This encapsulation ensures that the modal fields and time windows on the query side remain consistent with the historical database construction phase.

[0056] S402: Based on the multimodal fault knowledge base object, perform an approximate nearest neighbor search on the retrieval input object and calculate the similarity with the historical log vector. Select a first preset number of historical events in descending order of similarity to obtain a candidate historical event set.

[0057] A coarse-grained approximate nearest neighbor (ANN) search is performed along the log dimension: using the log vector to be queried as the query key, the similarity (e.g., cosine similarity) with all historical log vectors is calculated on the log vector index of the multimodal fault knowledge base object, and a first preset number of Top-K historical events are selected according to the similarity from high to low to obtain a candidate historical event set (e.g., K=300). This stage utilizes the direct representation of code execution exceptions and stack patterns in logs to quickly narrow down the comparison range in massive samples, adapting to the rapid localization needs such as "sudden increase in payment gateway timeout rate" or "PACS data entry retry exception".

[0058] S403: Based on the candidate historical event set, extract the corresponding performance index vector set from the multimodal fault knowledge base object, merge the retrieval input object and the performance index vector set, and calculate the similarity one by one to obtain a candidate list sorted by similarity.

[0059] Fine-grained performance similarity screening is performed on the candidate historical event set: The performance indicator vector set corresponding to each candidate event is extracted from the knowledge base, and its similarity to the query performance indicator vector in the retrieval input object is calculated one by one (using Euclidean distance or cosine similarity), resulting in a candidate list sorted by similarity. This stage utilizes performance vectors to characterize the system's macroscopic performance (such as CPU / memory, network / disk I / O, average service latency, etc.), and performs secondary confirmation on situations where log-level appearances are similar but have different business impacts, thus closely approximating the actual operating state during high-concurrency periods such as financial clearing batch windows or medical emergency room visits.

[0060] S404: Based on the candidate list, select a second preset number of historical events in descending order of similarity to obtain a subset of target historical events.

[0061] Based on the similarity order of the candidate list, a second preset number of Top-M historical events (e.g., M=30) are selected, and a target historical event subset is output. This target historical event subset is highly similar to the current event in both log pattern and performance characteristics, and can be directly used as a high-quality final set for subsequent contextual semantic adjudication and solution generation, significantly reducing the risk of mismatches and improving the efficiency of subsequent reasoning and generation.

[0062] S50: Input the target historical event subset and the query representation set together into a preset evaluation large language model for context matching reasoning to obtain the best matching historical event.

[0063] The target subset of historical events and the query representation set are input together into a pre-defined evaluation large language model for context matching inference, and the one with the highest matching degree is selected as the best matching historical event.

[0064] Combination Figure 6 As shown, step S50 specifically includes the following steps: S501: Obtain a text set from the fault report texts corresponding to the query representation set and the target historical event subset.

[0065] The corresponding fault report texts are extracted from the query representation set and the target historical event subset to obtain the text set to be evaluated. The former represents the natural language description of the current fault at the scene, while the latter consists of text reports of several historical events retained after dual filtering by logs and indicators. As semantic materials of the same source dimension, both provide a contextual basis for the final semantic decision.

[0066] S502: The text set is labeled and concatenated according to preset labeling rules to obtain the evaluation prompt object.

[0067] The text set is labeled and concatenated line by line according to preset labeling rules: the current fault text is labeled as "[Current Fault]", and each historical text is labeled as "[Candidate Historical Fault 1]...[Candidate Historical Fault N]", and organized into structured paragraphs according to a unified field order (phenomenon description → scope of impact → attempted operations → constraints / environmental information) to obtain the evaluation prompt object. This object ensures that texts from different sources have a consistent discourse structure and semantic focus before output, facilitating the evaluation model to perform one-by-one comparison and inductive reasoning.

[0068] S503: Combine the evaluation prompt object with the preset evaluation instruction set to generate an evaluation input object.

[0069] The evaluation prompts are merged with a pre-defined set of evaluation instructions to generate the evaluation input. The evaluation instruction set assigns the model the role of an "SRE expert" and employs a heuristic, step-by-step reasoning prompting strategy (thinking chain prompts). The model is required to first extract the core phenomena and possible root causes of the current failure, then perform a comparative analysis of similarities and differences among candidate texts, and finally output matching conclusions with clearly defined formats to ensure the interpretability and verifiability of the evaluation process and conclusions.

[0070] S504: Input the evaluation input object into a preset evaluation language model for context matching and reasoning to obtain a candidate matching judgment list.

[0071] The evaluation input is fed into a pre-defined large-scale evaluation language model (such as Evaluation LLM) for context-based matching and reasoning, producing a candidate matching decision list containing the matching degree (or equivalent decision signal) of each candidate text. This model, through a fine-grained understanding of the report's semantics, comprehensively examines semantic cues such as the problem phenomenon, the scope of impact, and user perception, and performs logical comparisons among multiple candidates to form a ranking result, providing an interpretable quantitative (or semi-quantitative) basis for the final selection.

[0072] S505: Extract the candidate identifier with the highest matching degree from the candidate matching determination list to obtain the target candidate identifier.

[0073] The candidate identifier with the highest matching degree is extracted from the candidate matching judgment list and used as the target candidate identifier. When there are ties, a local review can be triggered within the same evaluation session (e.g., reviewing key segments on the log / metric side, or supplementing a narrower business context in the prompt) to ensure that semantic consistency and business interpretability are not lower than the preset threshold.

[0074] S506: Map the target candidate identifier to the target historical event subset to obtain the best matching historical event.

[0075] Based on the target candidate identifier, object mapping is performed within the subset of target historical events to obtain the best matching historical event. Then, the complete object (including log / metric vectors and original report documents) in the knowledge base is traced back using the unique fault ID, so that it can be used as a key reference context for the next stage of input generation. The mapping and tracing rely on a cross-modal mapping table in the knowledge base, established with the unique fault ID as the primary key, to achieve accurate association and rapid location of vectors and document objects.

[0076] S60: Input the best matching historical events and the query representation set into a preset generative large language model to generate a solution and obtain the current fault diagnosis data.

[0077] The system constructs and generates prompts by combining complete data from best-matched historical events (including historical repair steps) with current real-time multimodal data. This information is then input into a large language model, requesting the generation of structured, executable diagnostic and treatment plans based on the on-site context. The system can be adapted according to the service name, deployment information, and parameter constraints of the target system (e.g., the clearing core and HIS medical order subsystem), and the output provides executable text for maintenance ledgers and change systems, i.e., the current fault diagnosis data.

[0078] Combination Figure 7 As shown, step S60 specifically includes the following steps: S601: Extract the corresponding real-time multimodal data from the best matching historical events and the query representation set to obtain the original set of input.

[0079] The necessary real-time multimodal data for generation is extracted from the best-matching historical events and query representation sets to construct the original set of generation inputs. The original set of generation inputs includes at least: complete data of historical events (including text of historically validated solutions) and the query log vector, query performance index vector, and normalized fault report text of the current event, to ensure that historical experience can be reused and the on-site context can be accurately aligned during generation.

[0080] S602: The original set of generated inputs is labeled and concatenated according to preset labels and field specifications to construct a generated prompt object.

[0081] Based on preset labels and field specifications, the original set of inputs is labeled and concatenated to obtain the generated prompt object. Specifically, the historical part is labeled as "[Resolved Fault Details]" and "[Historical Solutions]", and the current part is labeled as "[Current Fault Details]". The prompts are organized into a chapter structure that can be directly consumed by the model according to a unified field order (phenomenon - scope of impact - environment, constraints - logs, key indicators) to ensure consistency and comparability across industries and systems.

[0082] S603: Merge the generated prompt object with the preset generated instruction set to obtain a generated input object.

[0083] The generated prompt object is combined with the preset set of generation instructions to obtain the generated input object. The generation instruction set explicitly assigns the model the role of "SRE assistant" and adopts a step-by-step reasoning prompt strategy. It requires the model to first extract reusable common steps from historical solutions, and then perform contextual adaptation based on the service name, deployment parameters, and network and storage constraints of the current fault. Finally, it outputs executable text containing the operation sequence and key verification points, along with a brief explanation of the root cause mechanism to ensure interpretability and verifiability.

[0084] S604: Input the generated input object into a preset large language model to generate text and obtain candidate solution text.

[0085] The generated input object is fed into a pre-defined generative large language model (such as Generator LLM) to generate text, resulting in candidate solution text. This candidate text typically covers: tiered handling steps (such as observation—bypass—rate limiting / rollback—root cause repair—review and archiving), execution preconditions (window / permission / isolation measures), validation items (metric thresholds, key log signals, business regression use cases), and rollback strategies, to adapt to the risk and timeliness requirements of financial clearing time windows or critical medical business periods.

[0086] S605: Map the candidate solution text to the structured context in the query representation set to obtain a structured solution object.

[0087] The candidate solution text is mapped at the field level to the structured context in the query representation set to obtain a structured solution object. The mapping includes: binding the specific service name, instance identifier, change parameters, and monitoring items in the solution to the unified primary key and time window of the current event; extracting the thresholds, inspection items, and verification scripts in the steps as executable entries; and recording root cause explanations and risk points in the audit field to meet the requirements for record keeping and compliance in financial and medical scenarios (such as change logs / operation audits). This structured process allows text-based solutions to directly drive operation and maintenance orchestration or work order systems for execution.

[0088] S606: Encapsulate the structured scheme object to obtain the current fault diagnosis data.

[0089] The structured solution object is encapsulated to obtain the current fault diagnosis data, which is then submitted to the SRE team for review and execution closure. In addition to the main solution, the encapsulated content also includes rollback strategies, verification checklists, and links or references to key points of the review, in order to support subsequent knowledge accumulation and continuous optimization.

[0090] As can be seen, compared with existing AIOps solutions, the beneficial effects of this invention are reflected in the following aspects: By unifying the representation of system logs, performance indicators, and natural language fault reports into a three-modal approach, and adopting a phased similarity retrieval mechanism of "log coarse screening - indicator fine selection," the retrieval accuracy of similar historical cases can be significantly improved, continuously providing high-quality context for root cause analysis, and improving overall diagnostic accuracy; Based on the collaborative link of evaluating a large language model and generating a large language model, end-to-end automation is achieved from receiving fault information to outputting an executable handling plan, significantly shortening the mean time to repair (MTTR) and reducing S The RE burden is reduced, resulting in faster response times. Employing a Retrieval Enhanced Generation (RAG) architecture, knowledge updates can be completed by incrementally adding new cases to the knowledge base, eliminating the need to retrain the large language model. This makes the system more adaptable to evolving underlying environments and reduces maintenance costs. During the matching and decision-making stages and solution generation, the system can require the output of decision-making reasoning basis, facilitating manual review and intervention, and enhancing interpretability and verifiability. In terms of business value, it is particularly suitable for critical industry scenarios such as finance and insurance, more efficiently ensuring the continuity and stability of core businesses, meeting stringent SLA and compliance requirements, and reducing business risks and losses caused by failures.

[0091] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0092] In one embodiment, a multimodal system fault diagnosis device is provided, which corresponds one-to-one with the multimodal system fault diagnosis method described in the above embodiments. For example... Figure 8 As shown, the multimodal system fault diagnosis device includes: a data acquisition module 100, an index construction module 200, a data processing module 300, a similarity retrieval module 400, an event reasoning module 500, and a data output module 600.

[0093] Detailed descriptions of each functional module are as follows: The data acquisition module 100 is used to acquire historical fault event data and perform modal preprocessing on the historical fault event data to obtain a historical vector association set. The index building module 200 is used to build an index on the historical vector association set to obtain a multimodal fault knowledge base object; The data processing module 300 is used to perform isomorphic preprocessing on the real-time multimodal data of the current fault to be diagnosed based on the preset isomorphic rules in the multimodal fault knowledge base object, so as to obtain a query representation set. The similarity retrieval module 400 is used to perform phased similarity retrieval based on the query representation set and the multimodal fault knowledge base object to obtain a subset of target historical events; The event reasoning module 500 is used to input the target subset of historical events and the query representation set into a preset evaluation large language model for context matching reasoning to obtain the best matching historical event; The data output module 600 is used to input the best matching historical events and the query representation set into a preset large language model to generate a solution and obtain the current fault diagnosis data.

[0094] In one embodiment, the data acquisition module 100 is specifically used for: Read the resolved and closed fault work orders to obtain a set of historical work orders; The historical work order set is aggregated based on the work order identifier to obtain the corresponding full lifecycle data, thus obtaining historical fault event data; The historical fault event data is processed by timestamp parsing, noise reduction and field standardization on the log channel to obtain the first multimodal processing intermediate object; The historical fault event data is vectorized on the log channel and aligned by time index to obtain the second multimodal processing intermediate object; The historical fault event data is normalized and its gradient, mean and standard deviation are calculated on the indicator channel to obtain the third multimodal processing intermediate object. The first multimodal processing intermediate object, the second multimodal processing intermediate object, and the third multimodal processing intermediate object are encapsulated to obtain a historical vector association set.

[0095] In one embodiment, the index building module 200 is specifically used for: The historical vector association set is normalized to obtain a normalized entry set; Log vectors and performance metric vectors from the normalized entry set are written into the corresponding vector databases to generate an approximate nearest neighbor index, resulting in indexed vector storage. Write the fault report text from the standardized entry set into object storage or a document database to obtain a document storage set; Using the unique fault ID in the set of normalized entries as the primary key, a mapping relationship is established between the vector primary key in the indexed vector storage and the document primary key in the document storage set, generating a mapping relationship table; The indexed vector storage, document storage set, and mapping relationship table are merged and encapsulated to obtain a multimodal fault knowledge base object.

[0096] In one embodiment, the data processing module 300 is specifically used for: When an anomaly is detected in a fault work order, a diagnostic trigger object is generated, and event trigger data is obtained by integrating the data. The event triggers the acquisition of real-time multimodal raw data corresponding to the current fault, resulting in a multimodal raw data packet. The original multimodal data packets are vectorized in the log channel according to the same isomorphic rules as the multimodal fault knowledge base object to obtain the log vector to be queried. Based on the isomorphic rule, the system performance indicators in the multimodal raw data packet are vectorized using the log vector to be queried, to obtain the performance indicator vector to be queried. The fault report data in the multimodal raw data packet is normalized using the performance metric vector to be queried, and the fault report data is obtained. The fault report data, the log vector to be queried, and the performance metric vector to be queried are encapsulated to obtain a query representation set.

[0097] In one embodiment, the similarity retrieval module 400 is specifically used for: The retrieval input object is obtained by parsing the log vector and performance metric vector to be queried in the query representation set. Based on the multimodal fault knowledge base object, an approximate nearest neighbor search is performed on the retrieval input object and the similarity with the historical log vector is calculated. A first preset number of historical events are selected according to the similarity from high to low to obtain a candidate historical event set. Based on the candidate historical event set, extract the corresponding performance index vector set from the multimodal fault knowledge base object, merge the retrieval input object and the performance index vector set, and calculate the similarity one by one to obtain a candidate list sorted by similarity; Based on the candidate list, a second preset number of historical events are selected according to their similarity from high to low to obtain a subset of target historical events.

[0098] In one embodiment, the event reasoning module 500 is specifically used for: A text set is obtained from the fault report texts corresponding to the query representation set and the target historical event subset; The text set is labeled and concatenated according to preset labeling rules to obtain the evaluation prompt object; The evaluation prompt object is combined with the preset set of evaluation instructions to generate an evaluation input object; The evaluation input object is input into a preset evaluation language model for context matching and reasoning to obtain a candidate matching judgment list; Extract the candidate identifier with the highest matching degree from the candidate matching determination list to obtain the target candidate identifier; The target candidate identifiers are mapped to objects in the target historical event subset to obtain the best matching historical event.

[0099] In one embodiment, the data output module 600 is specifically used for: Extract the corresponding real-time multimodal data from the best matching historical events and the query representation set to obtain the original set of generation input; The original set of generated inputs is labeled and concatenated according to preset labels and field specifications to construct a generated prompt object; The generated prompt object is merged with a preset set of generated instructions to obtain a generated input object; The generated input object is input into a preset large language model to generate text, and candidate solution text is obtained. The candidate solution text is mapped to the structured context in the query representation set to obtain a structured solution object; The structured scheme object is encapsulated to obtain the current fault diagnosis data.

[0100] For specific limitations regarding the multimodal system fault diagnosis device, please refer to the limitations of the multimodal system fault diagnosis method above, which will not be repeated here. Each module in the aforementioned multimodal system fault diagnosis device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0101] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a multimodal system fault diagnosis method on the server side.

[0102] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a multimodal system fault diagnosis method.

[0103] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed, can perform the steps provided in the above embodiments.

[0104] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0105] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0106] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0107] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A system fault diagnosis method based on multimodal analysis, characterized in that, include: Historical fault event data is acquired, and modal preprocessing is performed on the historical fault event data to obtain a historical vector association set; The historical vector association set is indexed and constructed to obtain a multimodal fault knowledge base object; Based on the preset isomorphic rules in the multimodal fault knowledge base object, the real-time multimodal data of the current fault to be diagnosed isomorphically preprocessed to obtain a query representation set. Based on the query representation set and the multimodal fault knowledge base object, a phased similarity retrieval is performed to obtain a subset of target historical events; The target subset of historical events and the query representation set are input together into a preset evaluation large language model for context matching and reasoning to obtain the best matching historical event; The best matching historical events and the query representation set are input together into a preset generative large language model to generate a solution and obtain the current fault diagnosis data.

2. The system fault diagnosis method based on multimodal analysis according to claim 1, characterized in that, The process of acquiring historical fault event data and performing modal preprocessing on the historical fault event data to obtain a historical vector association set includes: Read the resolved and closed fault work orders to obtain a set of historical work orders; The historical work order set is aggregated based on the work order identifier to obtain the corresponding full lifecycle data, thus obtaining historical fault event data; The historical fault event data is processed by timestamp parsing, noise reduction and field standardization on the log channel to obtain the first multimodal processing intermediate object; The historical fault event data is vectorized on the log channel and aligned by time index to obtain the second multimodal processing intermediate object; The historical fault event data is normalized and its gradient, mean and standard deviation are calculated on the indicator channel to obtain the third multimodal processing intermediate object. The first multimodal processing intermediate object, the second multimodal processing intermediate object, and the third multimodal processing intermediate object are encapsulated to obtain a historical vector association set.

3. The system fault diagnosis method based on multimodal analysis according to claim 1, characterized in that, The process of indexing and constructing the historical vector association set to obtain a multimodal fault knowledge base object includes: The historical vector association set is normalized to obtain a normalized entry set; Log vectors and performance metric vectors from the normalized entry set are written into the corresponding vector databases to generate an approximate nearest neighbor index, resulting in indexed vector storage. Write the fault report text from the standardized entry set into object storage or a document database to obtain a document storage set; Using the unique fault ID in the normalized entry set as the primary key, a mapping relationship is established between the vector primary key in the indexed vector storage and the document primary key in the document storage set, generating a mapping relationship table; The indexed vector storage, document storage set, and mapping relationship table are merged and encapsulated to obtain a multimodal fault knowledge base object.

4. The system fault diagnosis method based on multimodal analysis according to claim 1, characterized in that, Based on the preset isomorphic rules in the multimodal fault knowledge base object, the real-time multimodal data of the current fault to be diagnosed isomorphically preprocessed to obtain a query representation set, including: When an anomaly is detected in a fault work order, a diagnostic trigger object is generated, and event trigger data is obtained by integrating the data. The event triggers the acquisition of real-time multimodal raw data corresponding to the current fault, resulting in a multimodal raw data packet. The original multimodal data packets are vectorized in the log channel according to the same isomorphic rules as the multimodal fault knowledge base object to obtain the log vector to be queried. Based on the isomorphic rule, the system performance indicators in the multimodal raw data packet are vectorized using the log vector to be queried, to obtain the performance indicator vector to be queried. The fault report data in the multimodal raw data packet is normalized using the performance metric vector to be queried, and the fault report data is obtained. The fault report data, the log vector to be queried, and the performance metric vector to be queried are encapsulated to obtain a query representation set.

5. The system fault diagnosis method based on multimodal analysis according to claim 1, characterized in that, The step of performing phased similarity retrieval based on the query representation set and the multimodal fault knowledge base object yields a subset of target historical events, including: The retrieval input object is obtained by parsing the log vector and performance metric vector to be queried in the query representation set. Based on the multimodal fault knowledge base object, an approximate nearest neighbor search is performed on the retrieval input object and the similarity with the historical log vector is calculated. A first preset number of historical events are selected according to the similarity from high to low to obtain a candidate historical event set. Based on the candidate historical event set, extract the corresponding performance index vector set from the multimodal fault knowledge base object, merge the retrieval input object and the performance index vector set, and calculate the similarity one by one to obtain a candidate list sorted by similarity; Based on the candidate list, a second preset number of historical events are selected according to their similarity from high to low to obtain a subset of target historical events.

6. The system fault diagnosis method based on multimodal analysis according to claim 1, characterized in that, The step of inputting the target subset of historical events and the query representation set into a preset evaluation large language model for context matching inference to obtain the best matching historical event includes: A text set is obtained from the fault report texts corresponding to the query representation set and the target historical event subset; The text set is labeled and concatenated according to preset labeling rules to obtain the evaluation prompt object; The evaluation prompt object is combined with the preset set of evaluation instructions to generate an evaluation input object; The evaluation input object is input into a preset evaluation language model for context matching and reasoning to obtain a candidate matching judgment list; Extract the candidate identifier with the highest matching degree from the candidate matching determination list to obtain the target candidate identifier; The target candidate identifiers are mapped to objects in the target historical event subset to obtain the best matching historical event.

7. The system fault diagnosis method based on multimodal analysis according to claim 1, characterized in that, The step of inputting the best-matching historical events and the query representation set into a preset generative large language model to generate a solution and obtain the current fault diagnosis data includes: Extract the corresponding real-time multimodal data from the best matching historical events and the query representation set to obtain the original set of generation input; The original set of generated inputs is labeled and concatenated according to preset labels and field specifications to construct a generated prompt object; The generated prompt object is merged with a preset set of generated instructions to obtain a generated input object; The generated input object is input into a preset large language model to generate text, and candidate solution text is obtained. The candidate solution text is mapped to the structured context in the query representation set to obtain a structured solution object; The structured scheme object is encapsulated to obtain the current fault diagnosis data.

8. A system fault diagnosis device based on multimodal analysis, characterized in that, include: The data acquisition module is used to acquire historical fault event data and perform modal preprocessing on the historical fault event data to obtain a historical vector association set; The index building module is used to build an index on the historical vector association set to obtain a multimodal fault knowledge base object; The data processing module is used to perform isomorphic preprocessing on the real-time multimodal data of the current fault to be diagnosed based on the preset isomorphic rules in the multimodal fault knowledge base object, so as to obtain a query representation set. The similarity retrieval module is used to perform phased similarity retrieval based on the query representation set and the multimodal fault knowledge base object to obtain a subset of target historical events; The event reasoning module is used to input the target subset of historical events and the query representation set into a preset evaluation large language model for context matching reasoning to obtain the best matching historical event; The data output module is used to input the best matching historical events and the query representation set into a preset large language model to generate a solution and obtain the current fault diagnosis data.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the multimodal system fault diagnosis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the multimodal system fault diagnosis method as described in any one of claims 1 to 7.

Images