A collaborative evolution method and system for service knowledge completion and anomaly detection based on LLM

Abstract

The application discloses a kind of service knowledge completion and abnormality detection's collaborative evolution system based on LLM, belong to artificial intelligence and abnormality detection technical field.System includes unified knowledge hub, knowledge completion subsystem and abnormality detection subsystem;Unified knowledge hub is used to receive service data generated in real time by service running and store;Abnormality detection subsystem is used to receive service data generated in real time by service running, and according to unified knowledge hub to service data carries out mixed abnormality detection, if it is judged as no exception, then enter next time, if it is judged as simple exception, then output abnormality detection result, if it is judged as complex exception, then generate collaborative prompt instruction and input knowledge completion subsystem, further according to the completion information obtains abnormality detection result;Knowledge completion subsystem is based on collaborative prompt instruction and current time unified knowledge hub, uses LLM to generate completion information.The application can significantly improve the discovery ability to complex hidden exception and knowledge automation level.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and anomaly detection technology, specifically relating to a collaborative evolution method and system for service knowledge completion and anomaly detection based on LLM. Background Technology

[0002] As enterprises deepen their digital transformation, massive amounts of service data have become one of their most important assets. However, traditional knowledge management methods heavily rely on manual organization and keyword retrieval, resulting in low utilization of these valuable data resources and the formation of information silos. Meanwhile, complex and hidden anomalies are constantly emerging in IT operations and business operations. For example, in a microservice architecture, slow queries in the underlying database can trigger cascading timeouts in upstream service chains. Traditional monitoring based on fixed thresholds can only capture single-node metric violations but cannot combine service call topology and historical operational knowledge for root cause localization. Another example is that business systems may experience hidden logical anomalies due to configuration drift or changes in third-party interface policies (such as financial losses caused by conflicting promotional rules).

[0003] Meanwhile, traditional rule-based or simple statistical models often fail because they cannot capture contextual relationships, resulting in high false positive rates and severe false negatives. Following the emergence of Large Language Model (LLM) technology, several mainstream solutions have been developed in the industry: The first approach is a static knowledge base based on Retrieval Augmentation Generation (RAG). This model combines LLM with an external knowledge base, and the standard workflow includes data preprocessing, chunking, embedding, storage, and similarity-based retrieval. RAG guides the model to generate more accurate answers through external knowledge, mitigating the problems of knowledge obsolescence and illusion in LLM.

[0004] The second approach is building specialized models for specific tasks. To improve the performance of LLMs in vertical domains, technical approaches such as hint engineering, fine-tuning, and building domain-specific LLMs have been developed. For example, in the IT operations field, models like LogBERT and LogLLM are specifically optimized for log sequences; in the cybersecurity field, Cyber ​​Language Models are used to generate synthetic attack logs to augment training data.

[0005] The third type is a hybrid analysis system that integrates multimodal and heterogeneous data. To gain more comprehensive insights, researchers are trying to integrate different types of data sources into a unified framework. For example, GraphRAG introduces knowledge graphs to enable retrieval to discover deep relationships between entities; RAAD-LLM transforms the statistical characteristics of time series signals into natural language descriptions to achieve cross-modal understanding; and the LEMAD system integrates multiple analysis methods such as log semantic parsing, time series prediction, and context variable monitoring.

[0006] The fourth type is the human-machine collaborative interactive system, represented by SymbioticRAG. This framework incorporates human experts into the AI ​​system's decision-making loop, allowing users to modify, rate, or supplement the search results and LLM-generated answers through an interactive interface. This feedback is used to optimize current responses and improve the system in the long term.

[0007] However, existing technical solutions have the following inherent drawbacks: (1) Staticity and unidirectionality: The performance of RAG-based static knowledge bases is highly dependent on the quality and real-time performance of external knowledge bases. The knowledge base needs to be manually updated regularly and cannot dynamically adapt to business changes. The retrieval process adopts a fixed Top-K strategy, which is difficult to dynamically adjust according to the complex intent of users and is prone to ignoring key information.

[0008] (2) Poor versatility and high cost: Although dedicated models perform well in specific tasks, they require a lot of manpower and resources to collect data, label samples and retrain the model every time a new service area or new type of anomaly is encountered. The cycle is long and the cost is high, making it difficult to respond quickly to changing business needs.

[0009] (3) Rigid architecture: Most multimodal fusion systems are designed as one-off systems with fixed data flows and model combinations. When new data sources are introduced or the analysis dimensions are changed, the entire system architecture often needs to be reconstructed. Efficient and lossless fusion of multimodal information is also a technical challenge that has not yet been fully solved.

[0010] (4) Efficiency bottleneck: Although the human-machine collaborative system improves the quality and credibility of the results, each query requires manual review and intervention, which greatly reduces the value of automation. Especially in the scenario of processing massive service data, the cost of labor becomes the main factor restricting large-scale application.

[0011] (5) Functional separation: Most existing technologies regard "knowledge completion" and "anomaly detection" as two isolated tasks, failing to achieve deep coupling and co-evolution between the two, and thus failing to form a self-reinforcing intelligent system.

[0012] Therefore, there is an urgent need to propose a new way of managing service data. Summary of the Invention

[0013] To address the technical problems of insufficient utilization of existing unstructured product service knowledge and the difficulty in detecting some complex and hidden anomalies, this invention designs a dynamic and adaptive service knowledge management system. This system can adapt to constantly changing business needs and data formats with a general and scalable architecture. It automatically extracts, organizes, and completes service knowledge from massive amounts of unstructured data. Based on the completed knowledge base, it effectively detects complex and hidden anomalies. While maintaining high accuracy, it ensures the stable, secure, and reliable operation of the system in the production environment, so that the knowledge completion process and the anomaly detection process promote each other and evolve collaboratively.

[0014] Firstly, this invention proposes a collaborative evolutionary system for service knowledge completion and anomaly detection based on LLM. The specific technical solution is as follows: The collaborative evolution system includes a unified knowledge center, a knowledge completion subsystem, and an anomaly detection subsystem; The unified knowledge hub is used to receive and store service data generated during service runtime in real time. The anomaly detection subsystem is used to receive service data generated during service operation in real time, and perform hybrid anomaly detection on the service data at the current moment based on the unified knowledge center. If no anomaly is found, the system proceeds to the next moment. If a simple anomaly is found, the anomaly detection result is directly output. If a complex anomaly is found, a collaborative prompt instruction is generated and input into the knowledge completion subsystem. The anomaly detection result is then obtained based on the completion information generated by the knowledge completion subsystem. The knowledge completion subsystem is used to generate completion information using LLM based on collaborative prompts and a unified knowledge hub; the completion information is then transmitted back to the anomaly detection subsystem.

[0015] Furthermore, the unified knowledge hub comprises an unstructured knowledge layer, a semi-structured knowledge layer, and a structured knowledge layer. The unstructured knowledge layer is used to store raw service data; the semi-structured knowledge layer is used to convert service data into a fixed format and store it; and the structured knowledge layer is used to extract services as nodes, logical relationships between services as edges, and attributes corresponding to services as node features from the service data at the current time, based on the knowledge graph at the previous time step, to obtain the knowledge graph at the current time step. Furthermore, the specific process of the hybrid anomaly detection is as follows: The anomaly detection subsystem is based on a unified knowledge hub and calculates the anomaly probability of each service in the service data. If the anomaly probability is lower than or equal to the threshold, it is considered that there is no anomaly. If the anomaly probability is greater than the threshold, it is further judged whether the anomaly conforms to the preset rules. If it conforms, the current anomaly is considered to be a simple anomaly. If it does not conform to any rule, the anomaly is considered to be a complex anomaly.

[0016] Furthermore, the anomaly probability of each service in the service data is obtained through a pre-trained prediction model; the pre-training process is as follows: The training knowledge graph is input into the prediction model to obtain the predicted anomaly probability between service pairs in the training knowledge graph. The cross-entropy loss is then calculated with the corresponding true anomaly probability, and the prediction model parameters are updated until convergence, resulting in a pre-trained prediction model.

[0017] Furthermore, the process of generating the collaborative prompt instruction is as follows: Based on the service corresponding to the complex anomaly, a search is performed in the unified knowledge hub to retrieve relevant data for the service from the unstructured knowledge layer, the semi-structured knowledge layer, and the structured knowledge layer, respectively. Then, a second LLM is used to generate collaborative prompt instructions.

[0018] Furthermore, the process of retrieving data related to the target service specifically involves: In the unstructured knowledge layer, semantic similarity calculation is used to obtain text data related to the target service; In the semi-structured knowledge layer, structured data related to the target service is obtained through exact matching or range queries; In the structured knowledge layer, a subgraph centered on the target service is extracted as contextual information related to the target service.

[0019] Furthermore, the process of generating the supplementary information is as follows: S1, the knowledge completion subsystem performs semantic parsing on the collaborative prompting instructions to obtain the task description, and further retrieves the contextual corpus in the unified knowledge hub; S2, after concatenating the contextual corpus, preliminary hypotheses and the preset structured output template, inputs them into the LLM to obtain several structured conclusions to be verified and the corresponding conditional probability scores and sequence cumulative probability scores; S3: Calculate the similarity score between each conclusion to be verified and the retrieval content of the unstructured knowledge layer and the retrieval content of the semi-structured knowledge layer in the corresponding context corpus; and perform logical consistency verification between each conclusion to be verified and the retrieval content of the structured knowledge layer in the corresponding context corpus and output the verification score. S4. Rank all the conclusions to be verified by score, and then calculate the RRF score of each conclusion based on the ranking of all scores, which is the confidence level. S5. Based on the conclusion to be verified and the confidence level, complete information is obtained.

[0020] Furthermore, the process by which the anomaly detection subsystem obtains the anomaly detection result based on the completed information is as follows: The anomaly detection subsystem performs a consistency comparison based on the completed information and the service operating environment. If the comparison result is not abnormal, it stops. If the comparison result is abnormal but the completed information is incorrect, the knowledge completion subsystem is re-triggered to generate completed information. If the comparison result is abnormal and the completed information is correct, an alarm is triggered and the anomaly detection result is output.

[0021] Furthermore, the collaborative evolution system also includes positive feedback, in which the knowledge completion subsystem transmits the correct completion information to the unified knowledge center for storage.

[0022] Secondly, this invention proposes a collaborative evolution method for service knowledge completion and anomaly detection based on LLM, implemented using the aforementioned system.

[0023] The beneficial effects of this invention are: (1) This invention provides a novel collaborative evolution service knowledge management framework based on bidirectional feedback loop, which integrates knowledge completion and anomaly detection. It drives anomaly discovery through knowledge flow, and then supplements and improves knowledge through anomaly discovery, forming a self-reinforcing learning loop.

[0024] (2) The framework constructed by this invention is a general top-level design. There is no need to retrain the model for each new service area or anomaly type. The system continues to evolve with the inflow of data and business development, reducing long-term maintenance costs.

[0025] (3) The anomaly detection of the present invention integrates rich context from a unified knowledge center, which greatly improves the ability to discover and judge complex and hidden anomalies.

[0026] (4) Through bidirectional feedback loop, the system continuously learns from newly discovered anomalies and transforms them into new knowledge, thereby continuously enhancing its own capabilities over time. It is a continuously evolving intelligent system.

[0027] (5) This invention introduces an AI control platform to ensure the system is stable, secure, and controllable in the production environment. It also improves version management, monitoring, and rollback mechanisms to address concerns after actual deployment. Attached Figure Description

[0028] Figure 1 This is an overall framework diagram of the system of the present invention.

[0029] Figure 2 This is a flowchart illustrating the method of the present invention.

[0030] Figure 3 This is a schematic diagram of the bidirectional feedback loop in this invention. Detailed Implementation

[0031] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0032] First, for ease of explanation, some terms in the specific embodiments of the present invention will be explained: Service Anomaly: An anomaly occurs when the actual output behavior of a service (including response content, timing, status, and side effects) deviates unacceptably from the expected business rules or Service Level Agreement (SLA). For example, an e-commerce platform's order service calls an inventory service, which in turn depends on a MySQL database. One day, the operations team performs an index change operation on the database, but the new index is not hit, causing the inventory query to become a full table scan. When the upstream order service calls the inventory service's interface, the response time is slowed from 20 milliseconds to nearly 3 seconds. Front-end users experience prolonged unresponsiveness and repeated retries after clicking to place an order on the app, causing gateway traffic to double and cascading timeouts to occur throughout the call chain.

[0033] Large Language Model (LLM): refers to a deep neural network language model with a huge number of parameters (usually billions to hundreds of billions) that is pre-trained on massive text corpora through self-supervised learning. It has powerful natural language understanding and generation capabilities, but has inherent limitations such as knowledge lag and illusion.

[0034] Retrieval-Augmented Generation (RAG): A technical paradigm that combines external knowledge retrieval with LLM generation, which can significantly improve the accuracy of factual answers and alleviate the problem of outdated LLM knowledge. It is the basic architecture of current mainstream knowledge question answering systems.

[0035] Knowledge graph: A semantic network that organizes knowledge in a graph structure, consisting of nodes (entities) and edges (relationships), with its core data model being triples.

[0036] Vector databases: Database systems specifically designed for the efficient storage and retrieval of high-dimensional vectors. Their core capability is to find several results from millions to billions of vectors that are semantically most similar to the query vector.

[0037] Embedding refers to the technique of mapping discrete symbols (such as words, sentences, and documents) into continuous, dense, low-dimensional real vectors. High-quality embeddings can preserve the geometric relationships of the original semantics.

[0038] Structured generation: refers to the technique of guiding LLM to output text content that conforms to a predefined syntactic structure.

[0039] Hint engineering: refers to the technical practice of designing and optimizing input hints to guide LLMs to perform tasks in the intended manner.

[0040] Concept drift refers to the phenomenon in dynamic systems where the data distribution or the relationship between the target variable and features changes non-stationarily over time. In anomaly detection, this manifests as the failure of the original normal pattern and the inapplicability of historical rules, leading to model performance degradation.

[0041] Unsupervised anomaly detection: refers to identifying samples that deviate from normal patterns by modeling the inherent distribution of data under unlabeled conditions.

[0042] MLOps: An engineering practice system that covers the entire lifecycle of machine learning, including model development, training, validation, deployment, monitoring, retraining and decommissioning.

[0043] AIOps: refers to the application of artificial intelligence to the IT operations and maintenance field to achieve intelligent operations and maintenance goals such as fault prediction, root cause analysis, and automated repair.

[0044] Knowledge completion refers to the process of filling in missing facts based on an existing knowledge base through automatic reasoning or information extraction.

[0045] Log parsing: refers to the technical process of converting semi-structured raw log text into structured fields.

[0046] Based on the above background, such as Figure 1 As shown, this invention designs a comprehensive system that integrates a unified knowledge hub, a collaborative engine, a two-way feedback loop, and an AI control platform.

[0047] (1) Unifying the knowledge center The unified knowledge hub receives real-time input service data and stores it in three independent knowledge layers: unstructured knowledge layer, semi-structured knowledge layer, and structured knowledge layer. Unstructured Knowledge Layer: This layer directly stores raw service data, including logs, PDF manuals, and conversation logs. It is managed using traditional RAG technology, which utilizes text embedding models to convert text into high-dimensional vectors and store them in an efficient vector database to support fast retrieval based on semantic similarity.

[0048] Structured Knowledge Layer: Exists in the form of a knowledge graph, specifically used to store entities, attributes, and the relationships between them. Knowledge graphs can explicitly model connections in the real world, supporting complex multi-hop reasoning and causal analysis, which is crucial for understanding dependencies in service systems and locating root causes. LLM plays a bridging role here, leveraging its powerful information extraction capabilities to automatically extract triples containing product services, logical relationships between product services, and corresponding attributes from the unstructured text stored in the unstructured knowledge layer. This gradually builds and enriches the knowledge graph, where edges represent logical relationships between product services, and nodes represent product services and their attributes.

[0049] Semi-structured knowledge layer: LLM uses structured generation techniques to extract the input service data into structured data with a fixed format (such as JSON format) and store it, thereby ensuring the structured and consistent output.

[0050] The design of a unified knowledge hub is the fundamental difference between this invention and existing technologies. It abandons the practice of fixing the form of knowledge storage, and instead adopts a flexible and scalable hybrid data structure that seamlessly integrates knowledge at three levels.

[0051] (2) Collaborative Engine The collaborative engine comprises two core subsystems: a knowledge completion subsystem and an anomaly detection subsystem, and ensures that they work together.

[0052] 1) Knowledge Completion Subsystem The goal of the knowledge completion subsystem is to proactively discover and fill knowledge gaps from massive amounts of unstructured data. First, the subsystem scans the unified knowledge hub to identify records with incomplete structures or missing information. Then, utilizing LLM suggestion engineering, the subsystem calls the LLM to analyze the service data stored in the unified knowledge hub, extracting the missing key information.

[0053] Meanwhile, to ensure the structured and consistent output, the knowledge completion subsystem adopts a structured generation technique, allowing users to define the desired output structure, thereby guiding the LLM to generate JSON objects that conform to a specific format from the extracted key information.

[0054] After information extraction is completed, the knowledge completion subsystem further verifies and cleans the information, and then writes the completed knowledge back to the unified knowledge hub, especially updating the relevant nodes and edges in the knowledge graph of the structured knowledge layer, thereby achieving continuous knowledge growth.

[0055] Specifically, this process employs a collaborative processing mechanism of "hint parsing - cross-layer retrieval - targeted reasoning - multi-dimensional confidence quantification - structured output," with detailed steps as follows: Step 1: Collaborative prompt parsing and intent routing.

[0056] The knowledge completion subsystem receives collaborative prompts from the anomaly detection subsystem. These prompts encapsulate the target service entity identifier, fields / relationships to be completed or verified (e.g., "What downstream service is causing the slow response?"), anomaly contextual feature fragments (e.g., "No error logs"), and preliminary detection hypotheses (e.g., "A cascading timeout may have occurred; the root cause lies in the downstream service"). The subsystem utilizes a semantic parsing module to extract key entities, attribute constraints, and inference objectives from the prompts, converting them into internally executable task descriptions, thus clarifying completion boundaries and verification focus.

[0057] Step 2: Unify cross-layer retrieval and context fusion of the knowledge hub.

[0058] Based on the extracted entities and constraints, the subsystem initiates targeted retrieval within a unified knowledge hub: First, it retrieves the topological neighbors, historical attribute states, and business dependencies of the target entity from the structured knowledge layer; second, it retrieves standardized historical snapshots of similar services from the semi-structured knowledge layer; finally, it uses vector similarity matching to recall relevant original data from the unstructured knowledge layer. The system aligns the multi-source retrieval results by time series to construct a high-information-density contextual corpus.

[0059] Step 3: Enhanced hint construction and LLM-oriented reasoning.

[0060] The fused contextual corpus, anomaly detection hypotheses, and predefined structured output templates are dynamically concatenated to generate enhanced reasoning hints. These hints explicitly constrain the LLM's reasoning path, requiring it to perform fact completion or logical verification under given business logic and data constraints. Based on the enhanced reasoning hints, the subsystem invokes the LLM and forces the activation of structured generation technology. The model then performs chained reasoning based on contextual semantic associations and domain prior knowledge, outputting candidate completion values ​​or verification conclusions for the target field.

[0061] Step 4: Calculate the multi-dimensional confidence metric.

[0062] To ensure the objectivity and interpretability of the confidence level, the system adopts a multi-dimensional feature fusion calculation strategy: (1) Probabilistic confidence level: extract the Softmax conditional probability and sequence cumulative probability of the key decision token in the LLM output sequence; (2) Retrieval support level: calculate the semantic vector cosine similarity between the generated result and the retrieved unstructured / semi-structured context corpus; (3) Logical consistency: verify the logical consistency between the generated result and the retrieval result obtained in the structured knowledge layer and output the verification score.

[0063] The RRF score is calculated after each score is ranked, and the contribution details of each score are retained.

[0064] Step 5: Structured verification and result encapsulation and return.

[0065] A JSON parser is used to perform strong syntax and format validation on the LLM output to ensure that the output strictly conforms to the predefined structure.

[0066] After the verification is successful, the knowledge completion subsystem will encapsulate a standardized data package containing the completion result, overall confidence level, confidence level decomposition basis, and key supporting evidence chain and return it to the anomaly detection subsystem.

[0067] After receiving the data packet, the anomaly detection subsystem uses it as the core criterion to perform a cross-comparison of the completion result with the facts, thereby completing the final classification of complex and hidden anomalies, generating correction directions, and dynamically adjusting alarm strategies.

[0068] 2) Anomaly Detection Subsystem The anomaly detection subsystem focuses on discovering deviant behaviors from real-time data streams. Unlike traditional methods, its detection process is context-rich, receiving not only the raw service data but also auxiliary information from a unified knowledge hub in real time.

[0069] The detailed processing steps are as follows: Based on contextual information in the knowledge graph and raw data in the unified knowledge hub, the anomaly detection subsystem employs a hybrid anomaly detection strategy for the received service data, namely: First, the anomaly detection subsystem uses a classic unsupervised learning algorithm to detect the real-time received service data based on a unified knowledge center, and obtains the anomaly probability of each service in the service data. If the anomaly probability is lower than or equal to a threshold, it is considered that there is no anomaly; if the anomaly probability is greater than the threshold, it is further judged whether the anomaly conforms to the preset rules. If it does, the current anomaly is considered a simple anomaly; if it does not conform to any rule, the anomaly is considered a complex anomaly.

[0070] For known, simple anomalies with a clear pattern (i.e., information with semantic and logical errors, such as a service response time that must be greater than 0, but the response time of a certain service in the system is -50ms), the anomaly detection subsystem directly outputs the corresponding anomaly correction direction and confidence score.

[0071] For complex and subtle anomalies (i.e., information that is semantically logically correct but contains incorrect content, such as "the user completed the order payment ten minutes ago" which is semantically logically correct but contains incorrect content, and the user did not actually complete the payment), the system switches to an LLM-based semantic reasoning mode. It constructs a collaborative prompt instruction by combining the original data related to the service entity in the unified knowledge hub, the relevant contextual information in the knowledge graph, and the text data related to the service entity in the unstructured layer. This instruction is then handed over to the knowledge completion subsystem for comprehensive analysis.

[0072] Specifically, for the unstructured knowledge layer, the anomaly detection subsystem retrieves the corresponding text data based on semantic similarity within the unstructured knowledge layer according to the service entity, thereby meeting the context requirements of LLM-based semantic reasoning; for the semi-structured knowledge layer, the anomaly detection subsystem locates the corresponding node in the knowledge graph based on the service entity and extracts the target subgraph centered on that service as context information; for the structured knowledge layer, the anomaly detection subsystem obtains the retrieval results through exact search or range query (such as SQL statements).

[0073] After the knowledge completion subsystem generates the completion result based on the collaborative prompts, the anomaly detection subsystem compares the completion result with the actual service configuration for consistency. If the comparison shows no anomaly, the output stops. If the comparison shows an anomaly but the completion result is incorrect, the knowledge completion subsystem is triggered to regenerate the completion result. If the comparison shows an anomaly and the completion information is correct, an alarm is triggered and a repair plan is output, including the correction direction of the anomaly information, confidence score, possible cause speculation, and related evidence chain, which greatly facilitates subsequent root cause analysis.

[0074] The unsupervised learning algorithm uses a trained prediction model (such as a neural network). The training process is as follows: the unsupervised learning algorithm predicts the anomaly probability of the relationship between service entities based on the input product / service entities and the actual logical relationship between entities. It then calculates the cross-entropy loss based on the anomaly probability corresponding to the logical relationship between entities and the predicted anomaly probability, and updates the parameters in the algorithm until convergence.

[0075] (3) Two-way feedback loop A two-way feedback loop constitutes the core mechanism of this invention, overcoming the limitations of unidirectional or lack of interaction between knowledge modules and detection tasks in traditional AI systems. The overall process of this loop is as follows: Figure 3 As shown, this includes two complementary directions of information flow: Positive Feedback Path (From Anomaly to Knowledge): This feedback path primarily targets complex and subtle anomalies with low confidence levels in the unsupervised learning algorithm of the anomaly detection subsystem. It is triggered when the anomaly detection subsystem identifies novel anomaly patterns not recorded in the knowledge center. These anomaly events are marked as high-priority pending tasks, automatically triggering the knowledge discovery process. At this point, the collaborative engine guides the knowledge completion subsystem, using LLM to extract key contextual information from relevant raw data and identify potential failure modes. Subsequently, the knowledge completion subsystem structures the newly identified patterns and stores the completion results detected by the anomaly subsystem as new knowledge in the unified knowledge center. After manual verification and adoption, this knowledge enhances the system's ability to provide early warnings and remedial suggestions for similar future situations, thereby transforming occasional failures into an improvement in the system's overall understanding.

[0076] Reverse Feedback Path (From Knowledge to Anomalies): When new or updated knowledge entries are added to the unified knowledge hub, the anomaly detection subsystem will sense this in real time and retrain the unsupervised learning algorithm accordingly to adapt to the evolution of the environment and business rules. Subsequently, when a suspected anomaly log is detected, the system can query the knowledge graph to confirm that it is a high-priority pending item marked in the new version, thereby reducing the alarm level. Similarly, if an old anomaly pattern disappears due to changes in business processes, the system can also correspondingly reduce its confidence level, effectively alleviating alarm fatigue. This mechanism enables the anomaly detection model to continuously adapt to environmental changes, maintain high detection accuracy and a low false alarm rate, and effectively address the concept drift problem.

[0077] (4) AI control platform To ensure the stable, secure, and efficient operation of this dynamic and complex system with a feedback mechanism in a production environment, this invention introduces a top-level AI control platform that integrates best practices from MLOps and AIOps, providing system-level governance and assurance capabilities. Its key functions include: Version control: Strict version control is implemented for all system components, covering LLM models, prompt templates, knowledge base indexes and system software versions, to ensure the reproducibility and traceability of the system, and to support accurate recording and retrospection of any changes.

[0078] Comprehensive monitoring and observability: Fine-grained logging of every request during system operation, with key metrics including hint hash value, model and index version, token consumption, response latency, operating costs and output quality, and transparent insight into system status through a visual dashboard.

[0079] Security and compliance assurance: Integrating multi-layered security mechanisms, including input purging to prevent prompt injection attacks, personally identifiable information filtering to protect data privacy, and policy-based code execution, ensuring that all operations comply with enterprise security and compliance standards.

[0080] Rapid rollback mechanism: When a new version causes performance degradation or serious failure, the system can immediately start the rollback process to quickly restore to the previous stable version, minimizing the impact on business continuity.

[0081] Based on the above system architecture, this invention, combined with specific application backgrounds and embodiments, elaborates in detail on the collaborative evolution method for service knowledge completion and anomaly detection based on LLM, taking the e-commerce promotional activity rule management scenario as an example, such as... Figure 2 As shown, the specific process is as follows: Step 1: Access and layered construction of multi-source service knowledge.

[0082] The system collects service data, activity configuration tables, and promotional rule documents generated during e-commerce promotional activities, and automatically stores them in layers by a unified knowledge hub: the original documents are stored in the unstructured layer, the rule entities and logical relationships are extracted into triples by LLM to update the knowledge graph, and the key configuration items are generated in a structured manner and converted into standard JSON and stored in the semi-structured layer.

[0083] Step 2: Initial check of knowledge consistency and context fusion.

[0084] The anomaly detection subsystem monitors service data and the unified knowledge hub in real time during e-commerce promotional activities for anomalies. When it identifies a potential conflict between the discount rules and coupon stacking logic in the knowledge graph, but the unsupervised rule engine cannot reach a conclusion due to a lack of complete business context, the system determines this to be a complex anomaly. It then performs a cross-layer search of relevant activity configurations and constructs collaborative prompts.

[0085] Step 3: Semantic reasoning and knowledge-oriented completion.

[0086] The knowledge completion subsystem invokes LLM to perform rule logic deduction through corresponding collaborative prompts. LLM identifies knowledge inconsistencies such as "the document describes it as allowing overlay, but the configuration table actually contains mutually exclusive rules," and outputs structured completion results, including conflict location, correct rule suggestions, and confidence scores.

[0087] Step 4: Cross-validation and bidirectional feedback evolution.

[0088] The anomaly detection subsystem compares the completion result with the actual configuration of the online service during runtime. If the comparison shows no anomaly, it stops and returns to step two. If the comparison shows an anomaly but the completion result is incorrect, it returns to step three. If the comparison shows an anomaly and the completion information is correct, it triggers an alarm and generates a recommended repair plan, and then initiates bidirectional feedback: the forward path writes the verified correct rule triplet back to the knowledge center to complete the domain knowledge; the reverse path uses the new knowledge to update the detection model and improve the sensitivity of identifying similar conflicts.

[0089] The entire execution process of the method proposed in this invention is ensured by an AI control platform to guarantee version traceability and execution security, thereby completing a closed loop of collaborative evolution at the service knowledge level.

[0090] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A collaborative evolutionary system for service knowledge completion and anomaly detection based on LLM, characterized in that, The collaborative evolution system includes a unified knowledge center, a knowledge completion subsystem, and an anomaly detection subsystem; The unified knowledge hub is used to receive and store service data generated during service runtime in real time. The anomaly detection subsystem is used to receive service data generated during service operation in real time, and perform hybrid anomaly detection on the service data at the current moment based on the unified knowledge center. If no anomaly is found, the system proceeds to the next moment. If a simple anomaly is found, the anomaly detection result is directly output. If a complex anomaly is found, a collaborative prompt instruction is generated and input into the knowledge completion subsystem. The anomaly detection result is then obtained based on the completion information generated by the knowledge completion subsystem. The knowledge completion subsystem is used to generate completion information using LLM based on collaborative prompts and a unified knowledge hub; the completion information is then transmitted back to the anomaly detection subsystem.

2. The collaborative evolution system for service knowledge completion and anomaly detection based on LLM as described in claim 1, characterized in that, The unified knowledge hub comprises an unstructured knowledge layer, a semi-structured knowledge layer, and a structured knowledge layer. The unstructured knowledge layer is used to store raw service data; the semi-structured knowledge layer is used to convert service data into a fixed format and store it. The structured knowledge layer is used to extract services as nodes, logical relationships between services as edges, and attributes corresponding to services as node features from the service data at the current time, based on the knowledge graph at the previous time step.

3. The collaborative evolution system for service knowledge completion and anomaly detection based on LLM as described in claim 1, characterized in that, The specific process of the hybrid anomaly detection is as follows: The anomaly detection subsystem is based on a unified knowledge hub and calculates the anomaly probability of each service in the service data. If the anomaly probability is lower than or equal to the threshold, it is considered that there is no anomaly. If the anomaly probability is greater than the threshold, it is further judged whether the anomaly conforms to the preset rules. If it conforms, the current anomaly is considered to be a simple anomaly. If it does not conform to any rule, the anomaly is considered to be a complex anomaly.

4. The collaborative evolution system for service knowledge completion and anomaly detection based on LLM as described in claim 3, characterized in that, The anomaly probability of each service in the service data is obtained through a pre-trained prediction model; the pre-training process is as follows: The training knowledge graph is input into the prediction model to obtain the predicted anomaly probabilities among all service pairs in the training knowledge graph. The cross-entropy loss is then calculated with the corresponding true anomaly probabilities, and the prediction model parameters are updated until convergence, resulting in a pre-trained prediction model.

5. The collaborative evolution system for service knowledge completion and anomaly detection based on LLM as described in claim 1, characterized in that, The specific process for generating the collaborative prompt instruction is as follows: Based on the service corresponding to the complex anomaly, a search is performed in the unified knowledge hub to retrieve relevant data for the service from the unstructured knowledge layer, the semi-structured knowledge layer, and the structured knowledge layer, respectively. Then, a second LLM is used to generate collaborative prompt instructions.

6. The collaborative evolution system for service knowledge completion and anomaly detection based on LLM as described in claim 5, characterized in that, The process of retrieving data related to the target service is as follows: In the unstructured knowledge layer, semantic similarity calculation is used to obtain text data related to the target service; In the semi-structured knowledge layer, structured data related to the target service is obtained through exact matching or range queries; In the structured knowledge layer, a subgraph centered on the target service is extracted as contextual information related to the target service.

7. The collaborative evolution system for service knowledge completion and anomaly detection based on LLM as described in claim 1, characterized in that, The process of generating the complete information is as follows: S1, the knowledge completion subsystem performs semantic parsing on the collaborative prompting instructions to obtain the task description, and further retrieves the contextual corpus in the unified knowledge hub; S2, after concatenating the contextual corpus, preliminary hypotheses and the preset structured output template, inputs them into the LLM to obtain several structured conclusions to be verified and the corresponding conditional probability scores and sequence cumulative probability scores; S3, calculate the similarity score between each conclusion to be verified and the retrieval content of the unstructured knowledge layer and the retrieval content of the semi-structured knowledge layer in the corresponding context corpus; Each conclusion to be verified is logically consistent with the retrieved content of the structured knowledge layer in the corresponding context corpus, and a verification score is output. S4. Rank all the conclusions to be verified by score, and then calculate the RRF score of each conclusion based on the ranking of all scores, which is the confidence level. S5. Based on the conclusion to be verified and the confidence level, complete information is obtained.

8. The collaborative evolution system for service knowledge completion and anomaly detection based on LLM as described in claim 1, characterized in that, The anomaly detection subsystem obtains the anomaly detection result based on the completed information as follows: The anomaly detection subsystem performs a consistency comparison based on the completed information and the service operating environment. If the comparison result is not abnormal, it stops. If the comparison result is abnormal but the completed information is incorrect, the knowledge completion subsystem is re-triggered to generate completed information. If the comparison result is abnormal and the completed information is correct, an alarm is triggered and the anomaly detection result is output.

9. The collaborative evolution system for service knowledge completion and anomaly detection based on LLM as described in claim 1, characterized in that, The collaborative evolution system also includes positive feedback, in which the knowledge completion subsystem transmits the correct completion information to the unified knowledge center for storage.

10. A collaborative evolution method for service knowledge completion and anomaly detection based on LLM, implemented using the collaborative evolution system described in claim 1, characterized in that, include: S11, the service data generated during service runtime is input into the unified knowledge hub and anomaly detection subsystem in real time, and the unified knowledge hub stores the service data; The anomaly detection subsystem performs hybrid anomaly detection on the service data at the current moment based on the unified knowledge center. If no anomaly is found, it proceeds to the next moment. If a simple anomaly is found, it outputs the anomaly correction direction, confidence level, cause speculation and evidence chain. If a complex anomaly is found, it generates a collaborative prompt instruction and inputs it into the knowledge completion subsystem. S12, the knowledge completion subsystem uses LLM to generate completion information based on the received collaborative prompting instructions and unified knowledge center, and transmits it back to the anomaly detection subsystem; S13, the anomaly detection subsystem cross-compares the received completion information with the service runtime environment. If the comparison result is not anomaly, it proceeds to the next moment. If the comparison result is anomaly but the completion information is incorrect, it returns to S2 to generate new completion information. If the comparison result is anomaly and the completion information is correct, it outputs the completion information and the collaborative prompt instruction together as the anomaly detection result. S14, the knowledge completion subsystem transmits the correct completion information to the unified knowledge hub for storage.

Images