Root cause positioning method and device, storage medium, electronic equipment and program product

Abstract

The present disclosure relates to a root cause positioning method and device, a storage medium, an electronic device and a program product, and relates to the technical field of large model retrieval enhancement generation operation and maintenance monitoring. The root cause positioning method can reversely identify implicit faults according to multi-order retrieval data, and quickly and accurately perform root cause positioning. The root cause positioning method comprises the following steps: in response to a query request of a user, determining multi-order retrieval data corresponding to the query request, wherein the multi-order retrieval data is obtained according to first-order retrieval data based on a vector coarse sorting result and second-order retrieval data based on a vector rearrangement result; determining a multi-dimensional difference value according to the multi-order retrieval data, and performing dynamic anomaly determination according to the multi-dimensional difference value to obtain a determination result; if the determination result represents that the multi-order retrieval data is abnormal, determining a fault root cause according to the multi-order retrieval data and performing abnormal alarm.

Description

Technical Field

[0001] This disclosure relates to the field of large model retrieval enhancement generation operation and maintenance monitoring technology, specifically, to a root cause localization method, device, storage medium, electronic device and program product. Background Technology

[0002] Among related technologies, RAG (Retrieval-Augmented Generation) operation and maintenance technology lacks in-depth analysis of retrieval quality, resulting in one-sided monitoring; moreover, RAG operation and maintenance technology cannot achieve differentiated analysis of the two-level retrieval results, fault judgment relies entirely on manual methods, and the troubleshooting results are affected by human subjectivity, resulting in low accuracy; in addition, RAG operation and maintenance technology cannot quantify the multidimensional differences between the two-level retrieval and lacks differentiated diagnostic approaches. Summary of the Invention

[0003] To address the shortcomings of related technologies, this disclosure provides a root cause localization method, apparatus, storage medium, electronic device, and program product.

[0004] To achieve the above objectives, in a first aspect, this disclosure provides a root cause localization method, comprising: In response to a user's query request, determine the multi-level retrieval data corresponding to the query request. The multi-level retrieval data is obtained based on the first-level retrieval data obtained from the vector coarse ranking result retrieval and the second-level retrieval data obtained from the vector rearrangement result retrieval. Based on the multi-level retrieval data, multidimensional difference values ​​are determined, and dynamic anomaly determination is performed based on the multidimensional difference values ​​to obtain the determination result; If the determination result indicates that the multi-level retrieval data is abnormal, then the root cause of the fault is determined based on the multi-level retrieval data and an abnormality alarm is issued.

[0005] Secondly, this disclosure provides a root cause localization device, comprising: The response module is configured to respond to a user's query request and determine the multi-level retrieval data corresponding to the query request. The multi-level retrieval data is obtained based on the first-level retrieval data obtained from the vector coarse ranking result retrieval and the second-level retrieval data obtained from the vector rearrangement result retrieval. The execution module is configured to determine multidimensional difference values ​​based on the multi-level retrieval data, and to perform dynamic anomaly determination based on the multidimensional difference values ​​to obtain a determination result; The root cause localization module is configured to determine the root cause of the fault and issue an anomaly alarm based on the multi-level retrieval data if the determination result indicates that the multi-level retrieval data is abnormal.

[0006] Thirdly, this disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method described in the first aspect.

[0007] Fourthly, this disclosure provides an electronic device, comprising: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the method described in the first aspect.

[0008] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0009] The above technical solution responds to user query requests and determines the multi-level retrieval data corresponding to the query request. This multi-level retrieval data is obtained from first-level retrieval data based on vector coarse ranking results and second-level retrieval data based on vector rearrangement results. It can capture data differences at different precision levels, avoiding biases or omissions caused by a single retrieval stage. Multi-dimensional difference values ​​are determined based on the multi-level retrieval data, and dynamic anomaly judgment is performed based on these values ​​to obtain the judgment result. Introducing multiple difference values ​​for anomaly analysis, compared to traditional threshold or single-index alarms, the judgment result based on multiple feature dimensions is more accurate. When the judgment result indicates anomaly in the multi-level retrieval data, the root cause of the fault is determined based on the multi-level retrieval data, and an anomaly alarm is issued, shortening the manual troubleshooting time. The entire process uses first-level and second-level retrieval data to reverse-identify hidden faults and can adaptively adjust the judgment representation according to the query request, avoiding the failure of static rules in real-world, variable scenarios and improving the accuracy and reliability of root cause localization results.

[0010] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0011] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a root cause localization method according to an exemplary embodiment of the present disclosure.

[0012] Figure 2 This is another flowchart illustrating a root cause localization method according to an exemplary embodiment of the present disclosure.

[0013] Figure 3 This is a block diagram of a root cause localization device according to an exemplary embodiment of the present disclosure.

[0014] Figure 4 This is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0015] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0016] It should be understood that with the rapid implementation of generative artificial intelligence technology, RAG operation and maintenance technology has been widely applied in business scenarios such as enterprise knowledge bases, intelligent question answering, and government affairs consultation. The standard RAG architecture includes three links: vector coarse sorting, vector reordering, and large model generation. These links are strongly coupled and errors are propagated at each level.

[0017] As mentioned in the background section, RAG performance monitoring solutions in related technologies are mainly divided into three categories: the first category includes basic operation and maintenance indicator monitoring solutions; the second category includes independent quality monitoring solutions for single modules; and the third category includes offline manual evaluation solutions. However, the first category of solutions only collects operation and maintenance level indicators such as interface response time, request success rate, and concurrency, without involving the analysis of search content quality; and it can only detect explicit faults such as system crashes and interface errors, but cannot identify implicit silent degradations such as embedding drift and reordering disorder, resulting in a one-sided monitoring dimension. The second category of solutions performs independent detection on a single module among the embedding model, vector index, and reordering model, such as monitoring vector distribution offset and model accuracy decay, but there is a problem of module fragmentation, making it impossible to distinguish whether the fault occurs in the coarse-sorting stage or the reordering stage, and the fault coupling cannot be decoupled, resulting in ambiguous localization. The third category of solutions periodically evaluates the RAG retrieval accuracy offline by manually annotating the test set, but it relies on a large amount of manual annotation costs, has poor real-time performance, cannot achieve real-time online anomaly alerts, and cannot trace the specific cause of the fault.

[0018] In view of this, the present disclosure provides a root cause localization method, apparatus, storage medium, electronic device and program product, which can identify latent faults in reverse based on multi-level retrieval data and perform root cause localization quickly and accurately.

[0019] Figure 1 This is a flowchart illustrating a root cause localization method according to an exemplary embodiment of this disclosure, such as... Figure 1 As shown, the root cause localization method may include the following steps: In step S11, in response to the user's query request, the multi-level retrieval data corresponding to the query request is determined. The multi-level retrieval data is obtained based on the first-level retrieval data obtained from the vector coarse ranking result and the second-level retrieval data obtained from the vector rearrangement result.

[0020] In step S12, multidimensional difference values ​​are determined based on multi-level retrieval data, and dynamic anomaly determination is performed based on the multidimensional difference values ​​to obtain the determination result.

[0021] In step S13, if the determination result indicates that the multi-level retrieval data is abnormal, the root cause of the fault is determined based on the multi-level retrieval data and an abnormality alarm is issued.

[0022] The number of documents in the first-order retrieval data can be from Top 40 to Top 60, and the number of documents in the second-order retrieval data can be from Top 5 to Top 15. This disclosure does not limit this number.

[0023] The above technical solution responds to user query requests and determines the multi-level retrieval data corresponding to the query request. This multi-level retrieval data is obtained from first-level retrieval data based on vector coarse ranking results and second-level retrieval data based on vector rearrangement results. It can capture data differences at different precision levels, avoiding biases or omissions caused by a single retrieval stage. Multi-dimensional difference values ​​are determined based on the multi-level retrieval data, and dynamic anomaly judgment is performed based on these values ​​to obtain a judgment result. Introducing multiple difference values ​​for anomaly analysis, compared to traditional threshold or single-index alarms, provides a more accurate judgment result by comprehensively judging anomalies from multiple feature dimensions. When the judgment result indicates anomalies in the multi-level retrieval data, the root cause of the fault is determined based on the multi-level retrieval data, and an anomaly alarm is issued, shortening manual troubleshooting time. The entire process uses first-level and second-level retrieval data to reverse-identify hidden faults and can adaptively adjust the judgment representation according to the query request, avoiding the failure of static rules in real-world, variable scenarios, and improving the accuracy and reliability of root cause localization results. To facilitate a better understanding of the root cause localization method provided in this disclosure by those skilled in the art, the root cause localization method is described in detail below.

[0024] In a feasible embodiment, in step S11, responding to the user's query request and determining the multi-level retrieval data corresponding to the query request may include: In response to the user's query request, the system retrieves first-order search data from the database based on the vector coarse ranking results, and retrieves second-order search data based on the vector rearrangement results. Intersection alignment and filtering are performed on the first-order and second-order retrieval data to obtain multi-order retrieval data.

[0025] It's worth noting that, in response to user queries, a vector retrieval model is used to retrieve Top-K documents from the database based on the initial vector ranking results. Each Top-K document includes its document ID, vector similarity score, and ranking position. Then, a reranking model is used to further score the Top-K documents, and based on these scores, Top-N documents are determined from the Top-K documents in descending order. These Top-N documents include their final ranking scores. The intersection of the Top-K and Top-N documents is then aligned, and invalid documents are removed to obtain multi-level retrieval data.

[0026] The vector retrieval models include, but are not limited to, existing retrieval models such as general document vector models, multimodal vector models, and vertical domain vector models. The Rerank models include, but are not limited to, existing reranking models such as general text reranking models and multimodal reranking models. This disclosure does not limit the types of models mentioned.

[0027] It should be understood that coarse vector ranking results in high recall but low precision, while rearranged vector ranking results in high precision but may miss some results. Taking the intersection of first-order and second-order retrieval data preserves the breadth of coarse ranking while using the precision of rearrangement for filtering, thus improving the accuracy and reliability of multi-order retrieval data.

[0028] In one feasible embodiment, the multidimensional difference value includes at least one of the following: ranking position difference value, relevance score difference value, and semantic distribution difference value; In step S12, determining the multidimensional difference value based on the multi-level retrieval data may include: Substituting the multi-level retrieval data into the following formula for calculating the sorting position difference, we obtain the sorting position difference value: , D pos The rank represents the difference in ranking position, where n represents the total number of documents in the multi-level retrieval data. raw (i) represents the ranking position of document i in the first-order retrieval data, rank rerank (i) Characterizes the sorting position of document i in the second-order retrieval data; and / or, Substituting the multi-level retrieval data into the following relevance score difference calculation formula, we obtain the relevance score difference value: , D score Characterizing the difference in correlation scores, s raw (i) represents the normalized relevance score of document i in the first-order retrieval data, s rerank (i) Characterize the normalized relevance score of document i in the second-order retrieval data; and / or, Substituting the multi-level retrieval data into the following semantic distribution difference calculation formula, we obtain the semantic distribution difference value: D sem =1-cos(mean vec(raw) ,mean vec(rerank) ), D sem The mean value represents the semantic distribution difference. vec(raw) The vector mean representing all documents in the first-order retrieval data. vec(rerank) The vector mean of all documents in the second-order retrieval data.

[0029] It is worth noting that the difference value D in the sorting position pos Used to measure the degree of disorder in the arrangement relative to the coarse arrangement, and can be based on D pos The reordering was deemed disordered; the correlation score difference value D was also noted. score This is used to detect whether a re-ranking model is lowering or raising the scores of certain documents, and can be based on D. score Identify whether the re-ranking model scores are abnormal; semantic distribution difference value D sem Used to detect overall semantic shift in multi-level retrieval data, based on D sem Identify whether the embedded model has drifted and whether the knowledge base has been polluted.

[0030] In one feasible embodiment, the multidimensional difference value includes at least one of the following: ranking position difference value, relevance score difference value, and semantic distribution difference value; In step S12, dynamic anomaly determination is performed based on multidimensional difference values ​​to obtain the determination result, which may include: The difference values ​​of sorting position, relevance score, and semantic distribution are matched with their corresponding dynamic threshold ranges to obtain the matching results. If the matching result indicates a difference value that exceeds its corresponding dynamic threshold range, the multi-level retrieval data collected this time is determined to be abnormal; otherwise, the multi-level retrieval data collected this time is marked as normal, and the dynamic threshold ranges are updated.

[0031] It should be understood that the consistency of different retrieval stages can be monitored through the ranking position difference value, avoiding drastic fluctuations in results caused by feature perturbation or model switching; the relevance score difference value can identify documents with excessive score offset, which can be used for model distillation or alignment training, making it easier to find overfitting or scoring inconsistency problems; the semantic distribution difference value can detect the drift of speech features between different scenarios, users or devices, providing a quantitative basis for dynamic feature normalization and domain adaptive retrieval, and reducing the domain mismatch loss of speech retrieval tasks.

[0032] It is worth noting that in real business scenarios, there are natural fluctuations between coarse ranking and re-ranking under different query semantics and different business periods. If a static fixed threshold is used, it is very easy to cause invalid alarms or missed hidden faults.

[0033] Therefore, a dynamic threshold range is introduced. Specifically, the dynamic threshold ranges corresponding to the differences in ranking position, relevance score, and semantic distribution can be preset as follows: Historical normal business samples are collected through a time-sliding window, and the mean and standard deviation of the three indicators—ranking position difference, relevance score difference, and semantic distribution difference—are calculated for each historical normal business sample. The corresponding dynamic threshold ranges are then constructed based on the mean and standard deviation of each difference indicator. The sampling length of the time-sliding window can be 150 to 250 normal business samples; this disclosure does not limit this.

[0034] In addition, to construct a dynamic benchmark range that fits the current business scenario, each dynamic threshold range is automatically updated as business data iterates. Specifically, the dynamic threshold range corresponding to each difference indicator is iteratively updated based on continuously collected historical normal business samples. The dynamic threshold range dynamically drifts with the distribution of normal data, adapting to the natural fluctuations of the retrieval system under different time periods and different query request distributions, avoiding frequent false alarms caused by overly strict static thresholds or missed alarms caused by overly lenient static thresholds.

[0035] The above technical solution simultaneously matches the difference value with the dynamic threshold range from three dimensions: ranking position, relevance score, and speech distribution. Compared with single-dimensional judgment, it can more comprehensively capture abnormal fluctuations in multi-level retrieval data and effectively reduce the false negative rate. After each judgment, the dynamic threshold range is updated, allowing it to adjust synchronously with the natural drift of data distribution. This solves the problem of increased false negative rate when fixed thresholds change data characteristics, improving the system's robustness in long-term operation. Furthermore, the calculation of the difference value and the matching between the difference value and the dynamic threshold range are both lightweight operations, not relying on complex model inference, thus meeting real-time requirements.

[0036] In a feasible embodiment, step S13, determining the root cause of the fault based on multi-level retrieval data and issuing an anomaly alarm, may include: The root cause of the failure is determined based on the matching results and the preset mapping rule library between the difference value and the root cause of the anomaly. A diagnostic report is generated based on the root cause of the fault, and an anomaly alarm is issued.

[0037] In this embodiment, the difference values ​​of the three dimensions are directly associated with the specific root cause of the fault through a mapping rule base. Operation and maintenance personnel can directly locate the source of the problem based on the diagnostic report, shortening the troubleshooting time. The difference values ​​of the three dimensions of ranking position, relevance score, and voice distribution participate in the mapping, which is equivalent to replacing single-point triggering with multi-dimensional fingerprint matching. This can effectively distinguish different root causes such as ranking anomalies caused by relevance decline and retrieval anomalies caused by voice feature shift, avoiding confusion between different faults. In addition, the diagnostic report is generated by the matching results and mapping rules. Each alarm is accompanied by a traceable source of difference value and root cause derivation link, avoiding alarm fatigue caused by false alarms and improving the operation and maintenance team's willingness to respond to system alarms. Moreover, the content of the diagnostic report is generated by the rule base, with a fixed format and complete information. It can be directly connected to the work order system or notification channel to realize full-link automation from anomaly judgment to alarm push, reducing the repetitive work of manually writing diagnostic reports.

[0038] In one feasible embodiment, the preset mapping rule base between difference values ​​and root causes of anomalies may include: If the difference in sorting position exceeds its corresponding dynamic threshold range, the corresponding abnormal root cause is the weight drift of the reordering model or the abnormality of the reordering logic. Only when the semantic distribution difference value exceeds its corresponding dynamic threshold range, the corresponding abnormal root cause is the embedding model drift or knowledge base document pollution. Only when the difference in relevance score exceeds its corresponding dynamic threshold range, the corresponding abnormal root cause is the failure of vector index and the deterioration of coarse-rank recall quality. The differences in sorting position, relevance score, and semantic distribution all exceeded their corresponding dynamic threshold ranges. The root causes of these anomalies were batch updates of business data and abnormal system environments.

[0039] It is worth noting that the mapping rule base between the difference values ​​and the root causes of anomalies, as well as the corresponding construction principle, can be found in the table below: Table 1

[0040] The root cause localization method provided in this disclosure will now be described using a complete implementation method, such as... Figure 3 As shown, the root cause localization method may include the following steps: In step S111, in response to the user's query request, the database is searched based on the vector coarse ranking result to obtain first-order search data, and then searched based on the vector rearrangement result to obtain second-order search data. In step S112, the first-order retrieval data and the second-order retrieval data are subjected to intersection alignment and filtering to obtain multi-order retrieval data.

[0041] In step S121, at least one of the following is determined based on the multi-level retrieval data: the ranking position difference value, the relevance score difference value, and the semantic distribution difference value. In step S122, the difference values ​​of sorting position, relevance score, and semantic distribution are matched with their corresponding dynamic threshold ranges to obtain the matching results. In step S123, if the matching result indicates a difference value that exceeds its corresponding dynamic threshold range, the multi-level retrieval data collected this time is determined to be abnormal; otherwise, the multi-level retrieval data collected this time is marked as normal, and each dynamic threshold range is updated.

[0042] In step S131, if the determination result indicates that the multi-level retrieval data is abnormal, the root cause of the fault is determined according to the matching result and the preset mapping rule library between the difference value and the root cause of the abnormality. In step S132, a diagnostic report is generated based on the root cause of the fault, and an abnormal alarm is issued.

[0043] For example, taking a company's internal knowledge base RAG question-and-answer system as an application scenario, this RAG question-and-answer system can be used for employee business consultation and information retrieval. The underlying layer uses the BGE embedding model (BAAI General Embedding) and a vector database for coarse ranking, and the Cross-BERT (Cross-Modal BERT) model for re-ranking. The specific implementation process is as follows: I. System Parameter Configuration: The number of documents in the vector coarse ranking is set to Top 50, and the number of documents in the vector rearrangement is set to Top 10; the sampling length of the sliding window is 200 normal business query data, and the confidence interval is set to 95% to generate dynamic benchmark thresholds for three types of difference indicators; a mapping rule library between difference values ​​and abnormal root causes is preset.

[0044] II. Construction of Normal State Benchmark: Collect 200 routine business query data without anomalies, calculate the difference value of sort position, the difference value of relevance score, and the difference value of semantic distribution for each data item, and calculate the mean and fluctuation range of each difference value to generate the corresponding dynamic threshold range. The normal range is: the dynamic threshold range corresponding to the difference value of sort position is [0, 8], the dynamic threshold range corresponding to the difference value of relevance score is [0, 0.15], and the dynamic threshold range corresponding to the difference value of semantic distribution is [0, 0.08].

[0045] III. Real-time data acquisition and difference calculation: At a certain moment, a user initiates a business information query request. The system simultaneously collects 50 coarsely sorted documents (first-order retrieval data) and 10 rearranged documents (second-order retrieval data) for this query, aligns the document IDs, removes invalid and redundant documents, and obtains multi-order retrieval data. The multi-order retrieval data is then substituted into relevant calculation formulas to calculate in real time: the difference in sorting position is 15.6, the difference in relevance score is 0.12, and the difference in semantic distribution is 0.05.

[0046] IV. Anomaly Detection: The ranking position difference value of 15.6, the relevance score difference value of 0.12, and the semantic distribution difference value of 0.05 were matched with their corresponding dynamic threshold ranges. It was determined that the ranking position difference value exceeded its corresponding dynamic threshold range, while the relevance score difference value and the semantic distribution difference value were within the normal range. Therefore, it was determined that the current RAG system has an abnormal performance degradation.

[0047] V. Automatic Root Cause Analysis: The mapping rule library is retrieved to match the abnormal difference feature that "only the difference value of the sorting position exceeds its corresponding dynamic threshold range", and the root cause of the fault is accurately located as the weight drift of the Rerank model and the offset of the re-sorting scoring logic.

[0048] VI. Alarm Output and Fault Repair: The system generates visual diagnostic reports, pushes operation and maintenance alarms, and marks abnormal indicators, fault locations, and fault types; operation and maintenance personnel do not need to conduct full-chain troubleshooting, but can fine-tune and calibrate the Rerank model and update the model weights.

[0049] VII. Repair and Verification: After the model optimization was completed, the same type of query data was collected again, and the retest ranking difference was reduced to 4.2. The regression dynamic threshold range was reached, the system resumed stable operation, and the sliding window updated the baseline data to complete the closed-loop diagnostic optimization.

[0050] The aforementioned technical solution can accurately identify latent faults in Rerank models, reducing troubleshooting time from several hours in traditional manual investigation to less than 5 minutes. This demonstrates the technology's lightweight nature, strong practicality, and compatibility with various commercial RAG systems. Unlike traditional single-stage monitoring, it utilizes the multi-dimensional differences between coarse and rearrangement results to identify latent faults, filling a technological gap in the industry. The entire process involves no complex large-scale model training or complex graph algorithms, relying solely on three types of quantitative indicators and a mapping rule base, resulting in low computational costs. It is compatible with various enterprise RAG architectures, enabling lightweight implementation and easy deployment. Furthermore, it achieves fully automated anomaly identification and root cause localization without manual intervention, significantly reducing business losses caused by the silent degradation of RAG systems.

[0051] Based on the same inventive concept, this disclosure also provides a root cause localization device, such as... Figure 3 As shown, the root cause localization device may include: The response module 301 is configured to respond to a user's query request and determine the multi-level retrieval data corresponding to the query request. The multi-level retrieval data is obtained based on the first-level retrieval data obtained from the vector coarse ranking result retrieval and the second-level retrieval data obtained from the vector rearrangement result retrieval. The execution module 302 is configured to determine a multidimensional difference value based on the multi-level retrieval data, and to perform dynamic anomaly determination based on the multidimensional difference value to obtain a determination result; The root cause localization module 303 is configured to determine the root cause of the fault and issue an anomaly alarm based on the multi-level retrieval data if the determination result indicates that the multi-level retrieval data is abnormal.

[0052] In one feasible embodiment, the multidimensional difference value includes at least one of the following: ranking position difference value, relevance score difference value, and semantic distribution difference value; The execution module 302 is configured to match the sorting position difference value, the relevance score difference value, and the semantic distribution difference value with their corresponding dynamic threshold ranges to obtain matching results; If the matching result indicates a difference value that exceeds the corresponding dynamic threshold range, the multi-level retrieval data collected this time is determined to be abnormal; otherwise, the multi-level retrieval data collected this time is marked as normal, and the dynamic threshold ranges are updated.

[0053] In one feasible embodiment, the root cause localization module 303 is configured to determine the root cause of the fault based on the matching results and a preset mapping rule base between the difference value and the abnormal root cause; A diagnostic report is generated based on the root cause of the fault, and an anomaly alarm is issued.

[0054] In one feasible embodiment, the preset mapping rule base between difference values ​​and root causes of anomalies includes: If the difference in the sorting position exceeds the corresponding dynamic threshold range, the corresponding abnormal root cause is the weight drift of the reordering model or the abnormality of the reordering logic. Only when the semantic distribution difference value exceeds its corresponding dynamic threshold range, the corresponding abnormal root cause is embedding model drift or knowledge base document pollution; If the difference in the relevance score exceeds the corresponding dynamic threshold range, the corresponding abnormal root cause is that the vector index is invalid and the coarse-ranked recall quality is degraded. The difference values ​​of the sorting position, the relevance score, and the semantic distribution all exceed their corresponding dynamic threshold ranges, and the root causes of the anomalies are batch updates of business data and abnormal system environments.

[0055] In one feasible embodiment, the multidimensional difference value includes at least one of the following: ranking position difference value, relevance score difference value, and semantic distribution difference value; Execution module 302 is configured to substitute the multi-level retrieval data into the following sorting position difference calculation formula to obtain the sorting position difference value: , D pos The rank represents the difference in ranking position, where n represents the total number of documents in the multi-level retrieval data. raw (i) represents the ranking position of document i in the first-order retrieval data, rank rerank (i) Characterizes the sorting position of document i in the second-order retrieval data; and / or, Substituting the multi-level retrieval data into the following relevance score difference calculation formula, the relevance score difference value is obtained: , D score Characterizing the difference in correlation scores, s raw (i) represents the normalized relevance score of document i in the first-order retrieval data, s rerank (i) Characterize the normalized relevance score of document i in the second-order retrieval data; and / or, Substituting the multi-level retrieval data into the following semantic distribution difference calculation formula, the semantic distribution difference value is obtained: D sem =1-cos(mean vec(raw) ,mean vec(rerank) ), D sem The mean value represents the semantic distribution difference. vec(raw) The vector mean representing all documents in the first-order retrieval data. vec(rerank) The vector mean of all documents in the second-order retrieval data.

[0056] In one feasible embodiment, the response module 301 is configured to respond to a user's query request, retrieve the first-order retrieval data from the database based on the vector coarse ranking result, and retrieve the second-order retrieval data based on the vector rearrangement result; The first-order retrieval data and the second-order retrieval data are subjected to intersection alignment and filtering to obtain multi-order retrieval data.

[0057] Regarding the root cause localization device in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0058] Based on the same inventive concept, this disclosure also provides an electronic device, comprising: A memory on which computer programs are stored; A processor is configured to execute the computer program in the memory to implement the root cause localization method described above.

[0059] Figure 4 This is a block diagram illustrating an electronic device 400 according to an exemplary embodiment. Figure 4 As shown, the electronic device 400 may include a processor 401 and a memory 402. The electronic device 400 may also include one or more of a multimedia component 403, an input / output (I / O) interface 404, and a communication component 405.

[0060] The processor 401 controls the overall operation of the electronic device 400 to complete all or part of the steps in the root cause localization method described above. The memory 402 stores various types of data to support the operation of the electronic device 400. This data may include, for example, instructions for any application or method operating on the electronic device 400, and application-related data, such as first-order retrieval data, second-order retrieval data, multi-order retrieval data, and multidimensional difference values. The memory 402 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 403 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 402 or transmitted via communication component 405. The audio component also includes at least one speaker for outputting audio signals. I / O interface 404 provides an interface between processor 401 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 405 is used for wired or wireless communication between the electronic device 400 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of these. Therefore, the corresponding communication component 405 may include a Wi-Fi module, a Bluetooth module, or an NFC module.

[0061] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the root cause localization method described above.

[0062] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the root cause localization method described above. For example, the computer-readable storage medium may be the memory 402 including the program instructions described above, which may be executed by the processor 401 of the electronic device 400 to complete the root cause localization method described above.

[0063] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a processor, which, when executed by the processor, implements the steps of the root cause localization method described above.

[0064] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0065] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0066] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A root cause localization method, characterized in that, include: In response to a user's query request, determine the multi-level retrieval data corresponding to the query request. The multi-level retrieval data is obtained based on the first-level retrieval data obtained from the vector coarse ranking result retrieval and the second-level retrieval data obtained from the vector rearrangement result retrieval. Based on the multi-level retrieval data, multidimensional difference values ​​are determined, and dynamic anomaly determination is performed based on the multidimensional difference values ​​to obtain the determination result; If the determination result indicates that the multi-level retrieval data is abnormal, then the root cause of the fault is determined based on the multi-level retrieval data and an abnormality alarm is issued. The multidimensional difference value includes at least one of the following: ranking position difference value, relevance score difference value, and semantic distribution difference value; determining the multidimensional difference value based on the multi-level retrieval data includes: Substituting the multi-level retrieval data into the following sorting position difference calculation formula, the sorting position difference value is obtained: ， D pos The rank represents the difference in ranking position, where n represents the total number of documents in the multi-level retrieval data. raw (i) represents the ranking position of document i in the first-order retrieval data, rank rerank (i) Characterizes the ranking position of document i in the second-order retrieval data; and / or, Substituting the multi-level retrieval data into the following relevance score difference calculation formula, the relevance score difference value is obtained: ， D score Characterizing the difference in correlation scores, s raw (i) The normalized relevance score of document i in the first-order retrieval data, s rerank (i) Characterize the normalized relevance score of document i in the second-order retrieval data; and / or, Substituting the multi-level retrieval data into the following semantic distribution difference calculation formula, the semantic distribution difference value is obtained: D sem =1-cos(mean vec(raw) ,mean vec(rerank) ), D sem The mean value represents the semantic distribution difference. vec(raw) The vector mean representing all documents in the first-order retrieval data. vec(rerank) The vector mean representing all documents in the second-order retrieval data; The step of responding to a user's query request and determining the multi-level retrieval data corresponding to the query request includes: In response to the user's query request, the system retrieves the first-order retrieval data from the database based on the vector coarse ranking result, and retrieves the second-order retrieval data based on the vector rearrangement result. The first-order retrieval data and the second-order retrieval data are subjected to intersection alignment and filtering to obtain multi-order retrieval data.

2. The root cause localization method according to claim 1, characterized in that, The multidimensional difference value includes at least one of the following: ranking position difference value, relevance score difference value, and semantic distribution difference value; The dynamic anomaly determination based on the multidimensional difference values, to obtain the determination result, includes: The ranking position difference value, the relevance score difference value, and the semantic distribution difference value are matched with their corresponding dynamic threshold ranges to obtain the matching results; If the matching result indicates a difference value that exceeds the corresponding dynamic threshold range, the multi-level retrieval data collected this time is determined to be abnormal; otherwise, the multi-level retrieval data collected this time is marked as normal, and the dynamic threshold ranges are updated.

3. The root cause localization method according to claim 2, characterized in that, The step of determining the root cause of the fault and issuing an anomaly alarm based on the multi-level retrieval data includes: The root cause of the failure is determined based on the matching results and the preset mapping rule library between the difference value and the root cause of the anomaly. A diagnostic report is generated based on the root cause of the fault, and an anomaly alarm is issued.

4. The root cause localization method according to claim 3, characterized in that, The preset mapping rule base between difference values ​​and root causes of anomalies includes: If the difference in the sorting position exceeds the corresponding dynamic threshold range, the corresponding abnormal root cause is the weight drift of the reordering model or the abnormality of the reordering logic. Only when the semantic distribution difference value exceeds its corresponding dynamic threshold range, the corresponding abnormal root cause is embedding model drift or knowledge base document pollution; If the difference in the relevance score exceeds the corresponding dynamic threshold range, the corresponding abnormal root cause is that the vector index is invalid and the coarse-ranked recall quality is degraded. The difference values ​​of the sorting position, the relevance score, and the semantic distribution all exceed their corresponding dynamic threshold ranges, and the root causes of the anomalies are batch updates of business data and abnormal system environments.

5. A root cause localization device, characterized in that, include: The response module is configured to respond to a user's query request and determine the multi-level retrieval data corresponding to the query request. The multi-level retrieval data is obtained based on the first-level retrieval data obtained from the vector coarse ranking result retrieval and the second-level retrieval data obtained from the vector rearrangement result retrieval. The execution module is configured to determine multidimensional difference values ​​based on the multi-level retrieval data, and to perform dynamic anomaly determination based on the multidimensional difference values ​​to obtain a determination result; The root cause localization module is configured to determine the root cause of the fault and issue an anomaly alarm based on the multi-level retrieval data if the determination result indicates that the multi-level retrieval data is abnormal. The execution module is further configured to, when the multidimensional difference value includes at least one of the ranking position difference value, the relevance score difference value, and the semantic distribution difference value, substitute the multi-level retrieval data into the following ranking position difference calculation formula to obtain the ranking position difference value: ， D pos The rank represents the difference in ranking position, where n represents the total number of documents in the multi-level retrieval data. raw (i) represents the ranking position of document i in the first-order retrieval data, rank rerank (i) Characterize the ranking position of document i in the second-order retrieval data; and / or, substitute the multi-order retrieval data into the following relevance score difference calculation formula to obtain the relevance score difference value: ， D score Characterizing the difference in correlation scores, s raw (i) The normalized relevance score of document i in the first-order retrieval data, s rerank (i) Characterize the normalized relevance score of document i in the second-order retrieval data; and / or, substitute the multi-order retrieval data into the following semantic distribution difference calculation formula to obtain the semantic distribution difference value: D sem =1-cos(mean vec(raw) ,mean vec(rerank) ), D sem The mean value represents the semantic distribution difference. vec(raw) The vector mean representing all documents in the first-order retrieval data. vec(rerank) The vector mean representing all documents in the second-order retrieval data; The response module is also configured to respond to user query requests, retrieve the first-order retrieval data from the database based on the vector coarse ranking result, and retrieve the second-order retrieval data based on the vector rearrangement result; perform intersection alignment and filtering processing on the first-order retrieval data and the second-order retrieval data to obtain multi-order retrieval data.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-4.

7. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the method of any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-4.

Images