Failure analysis method and related device, electronic device and storage medium

Abstract

The application discloses a failure analysis method and related device, electronic equipment and storage medium, wherein the failure analysis method comprises: detecting based on the description document of the target agent to obtain the design elements of the target agent; analyzing based on the failure mode knowledge base and each design element to determine the first element suspected of existing failure risk and the first risk degree of the first element in each design element; selecting the first element as the second element based on the first risk degree of each first element; generating the test session and the adjudication rule of the second element based on the attack strategy knowledge base and the second element; determining the test reply of the second element based on the adjudication rule of the second element to obtain the second risk degree of the second element. The above scheme can identify the failure risk of the agent in advance before the agent goes online, and quantize the failure risk.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a failure analysis method and related apparatus, electronic devices and storage media. Background Technology

[0002] With the rapid development of artificial intelligence technology, intelligent agents are being increasingly widely used in e-commerce, finance, government affairs and other fields to provide 24 / 7 automated services.

[0003] However, due to numerous complex factors, the reliability, security, and accuracy of intelligent agents face significant challenges. Currently, the common methods for monitoring agent failures include manual sampling and post-event analysis of dialogue data after the agent has gone live, or real-time monitoring based on rule engines. However, these methods are both reactive, failing to prevent failures in advance, and also lack the ability to quantify failure risks. Therefore, identifying and quantifying failure risks before the agent goes live has become a pressing issue. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a failure analysis method and related devices, electronic equipment, and storage media that can identify the failure risks of an agent before it goes online and quantify those risks.

[0005] To address the aforementioned technical problems, the first aspect of this application provides a failure analysis method, comprising: detecting design elements of the target intelligent agent based on its description document; analyzing the design elements based on a failure mode knowledge base and each design element to identify a first element suspected of having failure risk and a first risk level of the first element; selecting the first element as a second element based on the first risk level of each first element; generating a test session and adjudication rules for the second element based on an attack strategy knowledge base and the second element; wherein the test session is used to test whether the second element of the target intelligent agent truly has a failure risk before the target intelligent agent goes online; judging the test response of the second element based on the adjudication rules of the second element to obtain a second risk level of the second element; wherein the test response of the second element is obtained by the target intelligent agent responding to the test session of the second element.

[0006] To address the aforementioned technical problems, a second aspect of this application provides a failure analysis apparatus, comprising: an element detection module, an element analysis module, an element selection module, a test generation module, and a test determination module. The element detection module is used to detect elements based on the description document of the target intelligent agent to obtain the design elements of the target intelligent agent. The element analysis module is used to analyze the design elements based on a failure mode knowledge base and each design element, identifying a first element suspected of having a failure risk and its first risk level among the design elements. The element selection module is used to select the first element as a second element based on the first risk level of each first element. The test generation module is used to generate a test session and adjudication rules for the second element based on an attack strategy knowledge base and the second element. The test session is used to test whether the second element of the target intelligent agent truly has a failure risk before the target intelligent agent goes online. The test determination module is used to determine the test response of the second element based on the adjudication rules of the second element to obtain the second risk level of the second element. The test response of the second element is obtained by the target intelligent agent responding to the test session of the second element.

[0007] To address the aforementioned technical problems, a third aspect of this application provides an electronic device comprising at least a memory and a processor coupled to each other, wherein the memory stores at least program instructions, and the processor executes the program instructions to implement the failure analysis method described in the first aspect.

[0008] To address the aforementioned technical problems, a fourth aspect of this application provides a computer-readable storage medium storing program instructions executable by a processor, the program instructions being used to implement the failure analysis method of the first aspect described above.

[0009] The above scheme detects the target agent's design elements based on its description document. It then analyzes these elements using a failure mode knowledge base, identifying the first element suspected of having a failure risk and its first risk level. Based on the first risk level of each first element, it selects it as the second element. Using the attack strategy knowledge base and the second element, it generates a test session and adjudication rules for the second element. The test session is used to test whether the second element of the target agent truly has a failure risk before the target agent goes live. Finally, based on the adjudication rules for the second element, it judges the test response to obtain the second risk level of the second element. The test response for the second element is obtained by the target agent's response to the test session for the second element. On one hand, because the first element potentially at risk of failure is pre-analyzed based on its description document before the target agent goes live, and real-world dialogue is simulated during the testing phase to test whether the first element truly poses a failure risk, the failure risk of the agent can be identified in advance before its deployment. On the other hand, because the first risk level of the first element potentially at risk of failure is determined in the design phase based on the failure mode knowledge base before the target agent goes live, and further assessed in the testing phase based on the test responses to the test session to obtain the second risk level during the testing phase, the failure risk can be quantified. Therefore, the failure risk of the agent can be identified and quantified in advance before its deployment. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating an embodiment of the failure analysis method of this application; Figure 2 This is a schematic diagram of a process of an embodiment of the failure analysis method of this application; Figure 3 This is a schematic diagram of the framework of an embodiment of the failure analysis device of this application; Figure 4 This is a schematic diagram of the framework of an embodiment of the electronic device of this application; Figure 5 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0011] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0012] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0013] In this paper, the terms "system" and "network" are often used interchangeably. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the slash " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this paper indicates two or more objects.

[0014] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the failure analysis method of this application. It should be noted that the process operations of the failure analysis method in this embodiment can be executed by an electronic device with computing capabilities or by related equipment containing such an electronic device. The specific type and structure of the electronic device or related equipment containing such an electronic device are not limited herein. Specifically, the embodiments of this disclosure may include the following steps: Step S11: Detect based on the description document of the target intelligent agent to obtain the design elements of the target intelligent agent.

[0015] In one implementation scenario, the target intelligent agent can be a business intelligent agent applied to fields such as e-commerce, finance, and government affairs. The specific settings can be configured according to the actual application scenario. Here, there is no limitation on the specific business scenario in which the target intelligent agent is applied.

[0016] In an implementation scenario, the description document of the target intelligent agent may include, but is not limited to, the target intelligent agent's requirements document, product document, etc., with no limitation on the specific type of requirements document. Furthermore, the description document of the target intelligent agent may specifically describe the target intelligent agent's functional points, interaction flows, business rules, etc., with no limitation on the specific content of the description document. Additionally, design elements may include, but are not limited to, at least one of the following: functional points, interaction flows of functional points, and business rules of functional points, with no limitation on the specific types of design elements.

[0017] In one implementation scenario, as a possible approach, to detect descriptive documents to obtain the design elements of the target agent, a document detection model can be pre-trained. This model can then be used to detect the descriptive documents of the target agent and obtain its design elements. It should be noted that, to maximize the detection accuracy of the document detection model, several sample requirement documents can be pre-collected, and these sample requirement documents may contain sample design elements. The document detection model can then be used to detect these sample requirement documents, obtaining predicted design elements. The difference between the sample design elements and the predicted design elements is then measured using a cross-entropy loss function, yielding the training loss of the document detection model. Based on this training loss, the network parameters of the document detection model can be adjusted. Of course, the above example is merely one possible way to train a document detection model in practical applications; other possible training methods are not limited here, nor will they be listed in detail.

[0018] In another implementation scenario, as a different possible approach, distinct from the aforementioned implementation, to detect the design elements of the target agent from the description document, the analysis agent can also detect the design elements in the description document. Specifically, the document big model within the analysis agent can detect the design elements from the description document. For example, based on the target agent's requirement document, big model instructions can be constructed. These instructions can be used to instruct the document big model to simulate the analyst's chain of thought, scanning the requirement document segment by segment, identifying intent, and decomposing elements to obtain the design elements. Then, the design elements of the target agent can be obtained by acquiring the output of the document big model in response to the big model instructions. It should be noted that the document big model can include, but is not limited to, open-source big models such as Llama, or it can be obtained by fine-tuning the parameters of an open-source big model based on a specific corpus (e.g., a corpus containing the aforementioned sample requirement documents), or it can be a custom big model. The specific source of the document big model is not limited here. Furthermore, the analytical agent can be called an FMEA (Failure Modes and Effects Analysis) agent, and the specific name of the analytical agent is not limited here.

[0019] In a specific implementation scenario, while inspecting the requirements document to obtain design elements, it is also possible to pinpoint the exact location and original reference of each design element within the requirements document.

[0020] In a specific implementation scenario, while inspecting the requirements document to obtain design elements, each design element can be assigned a unique identifier to distinguish different design elements.

[0021] In a specific implementation scenario, for ease of understanding, the following example illustrates the process of testing requirement documents using the target intelligent agent applied in the e-commerce field. When testing a requirement document named "V2.5 Requirement Document.docx" containing the clause "...the system needs to support users simultaneously using platform coupons, merchant coupons, and new user bonuses. The calculation order is: first subtract the bonus, then calculate the merchant coupon discount on the remaining amount, and finally use the platform coupon for a full reduction...", the document's large-scale model simulating the analyst's thought process can be illustrated as follows: 1. Scanning and Intent Recognition: Original quote: "...The system needs to support users using platform coupons, merchant coupons, and new user bonuses simultaneously. Calculation order..." The procedure is as follows: first subtract the red envelope amount, then calculate the merchant coupon discount on the remaining amount, and finally use the platform coupon for a full reduction... Reflection: This passage describes a complete, multi-step business logic. The core intent is to define multiple discounts. The final price calculation rules when both exist simultaneously. This is a complex feature that is very prone to errors.

[0022] 2. Element Decomposition and Classification: Reflection: I can break this description down into the following different levels of "design elements": A top-level feature: The system has the ability to combine and use multiple coupons.

[0023] A core business rule: there is a strict order in which these coupons can be used.

[0024] Two auxiliary business rules: the specific calculation methods for merchant coupons and platform coupons (one is...) Discounts include two types: spending-based reductions.

[0025] 3. ID assignment and anchoring: Think about it: Now assign them unique IDs and associate them precisely with the original text.

[0026] FE-012 (Feature): Named "Supports multiple coupon combinations". Related original text: "The system needs to support..." Users can simultaneously use platform coupons, merchant coupons, and new user bonuses. BR-021 (Business Rule): Named "Coupon Calculation Order Rule". This is the most crucial logic. Related original text: "The calculation order is: first subtract the red envelope, then calculate the merchant coupon discount on the remaining amount, and finally use the platform coupon." "Spend more and get a discount..." BR-022 (Business Rule): Named "Merchant Coupon Discount Rule". Related original text: "...calculate merchant..." Coupon discount..." BR-023 (Business Rule): Named "Platform Coupon Discount Rules". Related original text: "...Using the platform..." Coupons are available for discounts... 4. Establish connections: Reflection: These elements are not isolated. Rules BR-021, BR-022, and BR-023 are all designed to achieve the functional points. It exists because of FE-012. I need to record this hierarchical relationship.

[0027] Of course, the above example is merely one possible illustration of the thought process of a large document model in practical applications. Other possible scenarios are not limited here, nor will they be listed in detail. Furthermore, the large document model can output structured design elements. Using the aforementioned example, the output of the large document model may include the following: { "documentName": "V2.5 Requirements Document.docx", "elements": [ { "id": "FE-012", "type": "Feature", "name": "Supports multiple coupon combinations", "description": "The system allows users to combine platform coupons, merchant coupons, and new user bonuses in a single transaction." Bag.", "source": { "location": "Page 12, Section 3.4.1", "quote": "The system needs to support users using platform coupons, merchant coupons, and new user bonuses simultaneously." }, "relatedElements": ["BR-021", "BR-022", "BR-023"] / / Explicitly indicates that this function is provided by What rules make up }, { "id": "BR-021", "type": "Business Rule", "name": "Coupon Calculation Order Rules", "description": "When using multiple coupons simultaneously, the following order must be followed: 'Red Packet -> Merchant Coupon -> Platform Coupon'" The order of calculation. "source": { "location": "Page 12, Section 3.4.1", "quote": "The calculation order is: first subtract the red envelope amount, then calculate the merchant coupon discount on the remaining amount, and finally use..." Platform coupons offer discounts for purchases over a certain amount... }, "relatedElements": ["FE-012"] / / Indicates which function this rule serves. } / / ... Other elements, such as BR-022, BR-023 ] } It should be noted that in the above example, the field "id" represents the unique identifier of the design element, the field "type" represents the type of the design element (e.g., function point, interaction flow, business rule, etc.), the field "name" represents the name of the design element, the field "description" represents the description of the design element, the field "source" represents the source of the design element, where the field "location" represents the exact location of the design element, and the field "quote" represents the original text reference of the design element. Of course, the above example is only one possible example of the output content of the document model, and other possible situations are not limited here, nor will they be listed one by one.

[0028] Step S12: Based on the failure mode knowledge base and each design element, analyze and identify the first element that is suspected of having failure risk and the first risk level of the first element.

[0029] In an implementation scenario, a failure mode knowledge base can define triggering rules, risk attributes, and other content for various failure modes. It's important to note that the attribution of failure risks differs for each failure mode. For example, the failure mode "calculation error under complex conditions" could be attributed as "when business rules involve multiple variables, multiple states, or complex formulas, the program is prone to producing calculation results that do not match expectations." Of course, the above example is merely one possible illustration of failure modes in practical applications; other possible failure modes are not limited here, nor will they be listed individually. Furthermore, the triggering rules for a failure mode can include at least one triggering feature. Taking the aforementioned failure mode "calculation error under complex conditions" as an example, its triggering rules could include, but are not limited to, the following triggering features: "involving multiple discounts / coupons stacked," "prices dynamically changing based on user level / region / time," "requiring depreciation or handling fees to be calculated during refunds / returns," "complex calculation formulas for points / levels," "regular expression matching," and "multi-level nested IF-ELSE logic descriptions," etc., which are not limited here. Furthermore, the risk attributes of failure modes can include, but are not limited to, severity, occurrence, and detectability. It should be noted that severity characterizes the impact of the failure risk corresponding to the failure mode on the normal operation of the target agent (e.g., the higher the value, the stronger the severity); occurrence characterizes the frequency of the failure risk corresponding to the failure mode in historical interactions (e.g., the higher the value, the higher the frequency); and detectability characterizes the ease with which the failure risk corresponding to the failure mode is detected before the target agent goes online (e.g., the higher the value, the harder it is to detect). To simplify the above risk attributes, severity can be abbreviated as S, occurrence as O, and detectability as D. For ease of understanding, the failure mode knowledge base will be illustrated below using the aforementioned failure mode "computational error under complex conditions" as an example: { 1. "failureModeId": "FM-001", 2. "category": "Business logic risk", 3. "failureModeName": "Calculation error under complex conditions", 4. "description": "When business rules involve multiple variables, multiple states, or complex formulas, the program can..." The calculated results may not match the expected results. 5. "triggeringFeatures": [ 6. / / These are the signals that LLM looks for when matching [structured design elements]. 7. "Involves multiple discounts / coupons combined", 8. "Prices change dynamically based on user level / region / time". 9. "Depreciation or handling fees need to be calculated when refunding / returning goods". 10. "The formula for calculating points / grades is complex." 11. "Regular expression matching", 12. "Description of multi-level nested IF-ELSE logic" 13.], 14. "potentialImpacts": [ 15. / / Used to evaluate the S value 16. "Caused errors in user payment amounts, leading to customer complaints." 17. "Causing company asset losses (overpayment refunds, underpayment charges)" 18.], 19. "baselineSeverity": { 20. "default": 6, 21. "conditions": [ 22.{ "ifContext": ["Payment", "Transaction", "Settlement"], "value": 10} 23.] 24.}, 25."baselineDetection": { "default": 5}, 26."historicalOccurrence": { "rate": "0.05%"}, 27. "recommended Actions": { 28. "prevention": ["During the design process, complex computational logic should be modularized, and detailed unit code should be written."] test"], 29. "detection": ["QA is required to design specific test cases for boundary conditions and abnormal inputs"] 30.} 31.} It should be noted that in the above example, the numbers before each line of code, such as 1, 2, ..., 31, represent line numbers and have no practical meaning. The fields "failureModeId" represent the unique identifier of the failure mode, "category" represents the category to which the failure mode belongs, "failureModeName" represents the name of the failure mode, "description" represents the description of the failure mode, "triggeringFeatures" represents the triggering rules of the failure mode, "potentialImpacts" represents the potential impact of the failure mode, "baselineSeverity" represents the severity of the failure mode, "baselineDetection" represents the detectability of the failure mode, "historicalOccurrence" represents the historical occurrence rate of the failure mode, and "recommendedActions" represents the recommended actions for the failure mode, where "prevention" represents the blocking measures and "detection" represents the detection measures. Of course, the above example is only one possible example of a failure mode knowledge base in practical applications; other possible scenarios are not limited here, nor will they be listed in detail.

[0030] In one implementation scenario, to analyze various design elements based on a failure mode knowledge base (FMD) to identify the first element with suspected failure risk and its first risk level, semantic matching can be performed between the design elements and the triggering rules of various failure modes in the FMD knowledge base. This yields a matching result indicating whether the design element matches the triggering rule. As mentioned earlier, the FMD knowledge base also defines risk attributes for various failure modes. Responding to the matching result indicating that the design element matches the triggering rule, the design element is selected as the first element with suspected failure risk. The failure mode to which the triggering rule matches the design element is selected as the target mode. Analysis is then performed based on the risk attribute of the target mode to obtain the first risk level of the first element. This approach first performs semantic matching between the failure mode triggering rule and the design element, allowing for the selection of design elements that match the triggering rule from a semantic perspective as the first element with suspected failure risk. Then, based on the risk attribute of the failure mode to which the triggering rule matches, the first risk level of the first element is calculated. This improves the accuracy and quantification of whether a design element has suspected failure risk during the design phase.

[0031] In a specific implementation scenario, semantic extraction can be performed based on design elements (or design elements combined with their descriptive information) to obtain the first semantic feature of the design elements. Then, semantic extraction can be performed based on the triggering rules of the failure modes to obtain the second semantic feature of the triggering rules. A similarity measurement can then be performed between the first and second semantic features to obtain the semantic similarity between them. If the semantic similarity meets the screening conditions regarding a similarity threshold (e.g., semantic similarity is higher than the similarity threshold, semantic similarity is not lower than the similarity threshold), the design element is determined to match the triggering rule, and thus selected as the first element. It should be noted that the aforementioned semantic extraction can be obtained using language models such as BERT (Bidirectional Encoder Representations from Transformers). The specific method of semantic extraction is not limited here.

[0032] In a specific implementation scenario, if the matching result indicates that the design element did not match the triggering rule, it can be considered that the design element does not have a failure risk. At least during the design phase, the design element cannot be identified as having a failure risk. In this case, there is no need to select the design element as the first element.

[0033] In a specific implementation scenario, for ease of understanding, taking the design element "coupon calculation order rule" from the aforementioned example as an example, we can semantically match the design element "coupon calculation order rule" and its description information "when multiple coupons are used simultaneously, the calculation order must follow 'red envelope -> merchant coupon -> platform coupon'" with each trigger feature in the trigger rule of the aforementioned failure mode "calculation error under complex conditions". Since the trigger feature "involving multiple discounts / coupons stacked" is matched, it can be determined that the design element "coupon calculation order rule" is suspected of having a failure risk and can be selected as the first element. Of course, the above example is only one possible example of semantic matching in practical application, and other possible situations are not limited here, nor will they be listed one by one.

[0034] In a specific implementation scenario, as mentioned earlier, risk attributes can include severity, historical occurrence rate, and detectability. As a possible implementation method, to calculate the first risk level of the first element, the severity, historical occurrence rate, and detectability of the failure mode (i.e., the aforementioned target mode) to which the triggering rule of the first element belongs can be directly fused to obtain the first risk level of the failure risk represented by the target mode that the first element is suspected of having. For example, the severity, historical occurrence rate, and detectability of the failure mode (i.e., the aforementioned target mode) to which the triggering rule of the first element belongs can be directly multiplied to obtain the first risk level RPN of the failure risk represented by the target mode that the first element is suspected of having: RPN=S*O*D In the above formula, S represents severity, O represents historical occurrence rate, and D represents detectability. Alternatively, as another possible implementation, to calculate the first risk level of the first element, one can first infer based on the context of the first element to obtain the business impact on the target agent when the failure risk represented by the target pattern is assumed to exist (or, the business impact can be directly extracted from the failure mode knowledge base, such as the potential impact in the previous example, i.e., the field "potentialImpacts"). Then, the severity of the target pattern is updated based on the business impact to obtain the new severity of the target pattern. For example, when the impact represented by the business impact is large, the severity can be increased by a larger margin; conversely, when the impact represented by the business impact is small, the severity can be increased by a smaller margin. It should be noted that, specifically, a large language model can be used, leveraging its general understanding capabilities, to infer based on the context of the first element to obtain the business impact on the target agent when the failure risk represented by the target pattern is assumed to exist. Based on this, the severity, historical occurrence rate, and detectability of the target pattern can be fused (e.g., multiplied, see the previous implementation method for details) to obtain the first risk level of the first element suspected of having the failure risk represented by the target pattern. Of course, the above examples are only a few possible examples of calculating the first risk level in practical applications. Other possible calculation methods are not limited here, nor will they be listed one by one. The above method deduces the business impact on the target agent when the first element is assumed to have the failure risk represented by the target pattern, based on the context of the first element. Then, the severity of the target pattern is updated based on the business impact to obtain the new severity of the target pattern. Finally, the first risk level of the first element suspected of having the failure risk represented by the target pattern is obtained by fusing the new severity, historical occurrence rate, and detectability of the target pattern. This method can adaptively adjust the severity of the first element when the failure risk represented by the target pattern is assumed according to the original citation of the first element in the design document, which helps to improve the adaptability of the first risk level.

[0035] In one implementation scenario, during the aforementioned process of detecting design elements of the target agent based on the description document, the descriptive text of the design elements in the description document (as cited in the original text above) can also be detected simultaneously. This allows for analysis based on the descriptive text of the first element in the description document, yielding an inference chain indicating a potential failure risk for the first element. Then, based on the first element, its initial risk level, and the inference chain, a failure analysis report for the target agent during the design phase can be obtained. It should be noted that the inference chain indicating a potential failure risk for the first element contains several sequentially connected nodes, with progressive reasoning between nodes, which can prove that the first element is likely to have a failure risk. Furthermore, the inference chain can be obtained from the aforementioned large document model analysis or generated by the aforementioned document detection model; the method of obtaining the inference chain is not limited here. The above method analyzes the descriptive text of the first element in the description document to obtain a reasoning chain indicating that the first element may have a failure risk. Then, based on the first element, its first risk level, and the reasoning chain, a failure analysis report for the target agent in the design phase is obtained. This enables two-layer interpretable reasoning and attribution, explaining not only "why a design element might cause a failure risk" (risk attribution) but also "why this design element was identified as having a suspected failure risk" (design attribution), helping to establish an impeccable trust chain. Furthermore, the failure analysis report in the design phase can also include recommended measures for the first element. The specific content of the failure analysis report in the design phase is not limited here, nor will it be listed in detail. For ease of understanding, an example of a possible failure analysis report in the design phase in a practical application is provided below. Please refer to Table 1, which is a schematic table of an embodiment of a failure analysis report in the design phase.

[0036] Table 1. Schematic diagram of an embodiment of the failure analysis report in the design phase.

[0037] It should be noted that, in Table 1, each row other than the header represents a design element suspected of having a failure risk, including its associated information such as the functional module (i.e., function point), potential failure mode (i.e., the failure mode to which the triggering rule belongs), potential impact, S (severity), O (historical occurrence rate), D (detectability), RPN (first risk level), causation and reasoning (e.g., reasoning chain), and recommended measures. Of course, the above example is merely one possible illustration of a failure analysis report during the design phase in practical applications; other possible scenarios are not limited here, nor will they be listed individually.

[0038] Step S13: Based on the first risk level of each first element, select the first element as the second element.

[0039] It should be noted that the second element can be selected from the various first elements after they are ranked according to the first level of risk.

[0040] In one implementation scenario, as a possible approach, each first element can be sorted according to its first risk level (e.g., from high to low, or from low to high), and each first element can be selected as a second element in turn.

[0041] In another implementation scenario, as another possible implementation method, different from the aforementioned implementation method, the elements can be sorted according to their first risk level. For example, when sorting in descending order, the first elements located before the preset position (e.g., the first 5 positions) can be selected as the second elements. When sorting in ascending order, the first elements located after the preset position (e.g., the last 5 positions) can be selected as the second elements.

[0042] Step S14: Based on the attack strategy knowledge base and the second element, generate the test session and referee rules for the second element.

[0043] In this embodiment of the disclosure, the test session can be specifically used to test whether the second element of the target agent truly has a risk of failure before the target agent goes online.

[0044] In one implementation scenario, the attack strategy knowledge base can contain attack strategies associated with various failure modes, i.e., attack strategies related to failure modes, suitable for attack testing of failure modes. For example, an attack strategy can include rules, templates, etc., for generating attack samples; the specific content of the attack strategy is not limited here. For ease of understanding, taking the aforementioned failure mode "computation error under complex conditions" as an example, its attack strategy knowledge base may include, but is not limited to, the following: { "attackScenarioId": "AS-003", "linked_d_fmea_risk_ids": ["FM-001"], / / Linked to the failure mode ID predicted by D-FMEA "objective": "Verify the risk of 'computational errors under complex conditions'", "attackStrategy": "Constructs a purchase scenario that includes multiple, sequentially dependent coupon combinations." It also requires the agent to calculate the final price. "promptTemplate": [ "Hello, I would like to buy an item priced at {price} yuan. I have a new customer coupon worth {value1} yuan." The package includes a merchant coupon with a discount of 10%, and a platform coupon for 2% off purchases over a certain threshold. Please... How much will I ultimately need to pay? "If I use the platform coupon first, then the merchant coupon, and finally the cashback, what will the price be? Are you sure your algorithm is correct?" Is that right? ], "judgeCriteria": { / / Judgment criteria for use by judges (LLM) "expectedCorrectAnswer": "({price} - {value1}) * (1 - {discount} / 100) - {value2}", "failureCondition": "The agent's calculated final price does not match the expected correct answer.", "failureModeToLabel": "FM-001" } } It should be noted that in the above example, the field "attackScenarioId" represents the unique identifier of the attack scenario, the field "linked_d_fmea_risk_ids" represents the unique identifier of the associated failure mode during the design phase, the field "objective" represents the attack target, the field "attackStrategy" represents the attack strategy, the field "promptTemplate" represents the prompt template used to implement the attack strategy (i.e., using a generative model to generate attack examples), the field "judgeCriteria" represents the judging rules, where the field "expectedCorrectAnswer" is the expected correct response, the field "failureCondition" is the failure condition, and the field "failureModeToLabel" is the unique identifier of the failure mode. Of course, the above example is only one possible attack strategy in a real-world attack strategy knowledge base; other possible scenarios are not limited here, nor will they be listed in detail.

[0045] In one implementation scenario, to generate test sessions and adjudication rules for the second element, a query can be performed in the attack strategy knowledge base based on the second element to obtain attack strategies that match the second element. These adjudication strategies can then be parsed to obtain session generation instructions and adjudication rules for the second element. The session generation instructions instruct the generative large model to generate sessions according to these instructions. Several sessions generated by the generative large model in response to the session generation instructions for the second element can then be obtained and used as test sessions for the second element. This method, by querying the attack strategy knowledge base for the second element to obtain suitable attack strategies, and then parsing them to obtain session generation instructions and adjudication rules, allows the generative large model to generate test sessions in response to these instructions. This approach enables the testing phase to simulate real-world environments for testing failure risks by configuring the attack strategy knowledge base as needed, thus reducing iteration costs and improving maintainability during the testing phase.

[0046] In a specific implementation scenario, the generative large model in the testing phase can be regarded as the "red team large model" of the FMEA agent in the testing phase. It is used to simulate "tricky," non-standard, or even malicious users, and it is not for routine functional testing, but to systematically generate session data aimed at probing the boundaries, security vulnerabilities, and logical flaws of the target agent, so as to realize automated and large-scale "red team vs. blue team" exercises. In addition, the attack testing of the "red team large model" is not aimless. Its core task is to verify and reproduce the high-risk items predicted by the FMEA agent in the design phase (i.e., the second element mentioned above), and transform them into test scenarios for the FMEA agent in the testing phase, forming a closed loop of "prediction-verification".

[0047] In a specific implementation scenario, for ease of understanding, let's take the second element mentioned above, "computational errors under complex conditions," as an example. Based on the aforementioned attack strategy knowledge base, we can find a suitable attack strategy: "Construct a purchase scenario containing multiple coupon combinations with sequential dependencies, and require the agent to calculate the final price." Then, based on its prompt template, "Hello, I want to buy a product priced at {price} yuan. I have a {value1} yuan newcomer bonus, a {discount}% merchant coupon, and a {threshold}-{value2} platform coupon. How much do I ultimately need to pay?", we can generate several test sessions through a generative large model. Each test session can be input into the target agent to obtain the target agent's response data, which can then serve as the test response for the test session. Of course, the above example is merely one possible illustration in practical applications. We do not limit the design elements that need to be tested for actual failure risks during the testing phase, nor do we provide further examples. It should be noted that the prompt word template can provide a standardized "attack paradigm", which means that hundreds or thousands of the same type of attack tests can be easily reproduced through generative large models, the only difference being the parameters (e.g., the product price, the specific face value of the "new user bonus" value1, the discount of the merchant coupon, and the platform coupon with a discount of value2 when the threshold is reached).

[0048] Step S15: Determine the test response of the second element based on the adjudication rules of the second element to obtain the second risk level of the second element.

[0049] In this embodiment of the disclosure, as described above, the test response of the second element is obtained by the target agent responding to the test session of the second element. Specifically, each test response of the second element can be detected separately based on the referee rules to obtain the numerical value of the actual failure risk of the second element. Then, based on the ratio of the number of detections to the total number of test sessions of the second element, the first occurrence rate of the actual failure risk of the second element during the testing phase can be obtained. Subsequently, the second risk level of the second element can be obtained by fusing the severity, the first occurrence rate, and the detectability of the failure mode to which the failure risk of the second element belongs. It should be noted that, as mentioned above, the failure mode knowledge base can define risk attributes for various failure modes, and the risk attributes at least include severity and detectability. For details, please refer to the aforementioned descriptions, which will not be repeated here. The above method, by statistically analyzing the first occurrence rate of the actual failure risk during the testing phase and fusing it with the severity and detectability of the failure mode to which the failure risk of the second element belongs, can obtain the second risk level of the second element, enabling the measurement of the second risk level of the second element as accurately as possible during the testing phase.

[0050] In one implementation scenario, as mentioned earlier, failure analysis of the target agent can be implemented by an analysis agent (i.e., the aforementioned FMEA agent). This analysis agent can include a generative large model for generating test sessions and a referee large model for judging test responses. Based on this, large model instructions can be constructed using the referee's standard rules and the test sessions and responses of the second element. These instructions instruct the referee large model to judge the test responses of the test sessions according to the referee's rules to determine whether a real failure risk has occurred. The output of the referee large model in response to the large model instructions can then be obtained to determine whether a real failure risk has occurred in the test responses. Furthermore, the number of times a real failure risk exists in the second element can be statistically determined. For ease of understanding, let's take the second element, "computational error under complex conditions," as an example. According to the aforementioned attack strategy knowledge base, its "judgeCriteria" includes the expected correct answer and the failure condition. Based on this, the large-scale judge model can detect whether the test answer is the expected correct answer. If it is, it can be considered that the test answer has not encountered a failure risk (i.e., the failure risk corresponding to the failure mode "computational error under complex conditions"). Otherwise, it can be considered that the test answer has encountered a failure risk (i.e., the failure risk corresponding to the failure mode "computational error under complex conditions"). Of course, the above example is only one possible example of detecting the test answer of the second element in practical applications. Other possible situations are not limited here, nor will they be listed one by one.

[0051] In one implementation scenario, for any second element, after detecting each test response individually to obtain the number of times the second element actually has a failure risk, the first occurrence rate of the second element's actual failure risk during the testing phase can be obtained based on the ratio of the first response to the total number of test sessions for the second element. For ease of understanding, let's take the aforementioned second element "computation error under complex conditions" as an example. Assume it generates a total of 10 test sessions, and the target agent can respond to each of these 10 test sessions, obtaining 10 test responses. Based on the referee rules, each test response can be detected individually, resulting in a value of 8 times the second element "computation error under complex conditions" actually has a failure risk during the testing phase. Therefore, the first occurrence rate of the second element "computation error under complex conditions" actually having a failure risk during the testing phase is 8 / 10 = 0.8. Of course, the above example is merely one possible example for calculating the first occurrence rate in practical applications; other possible scenarios are not limited here, nor will they be listed individually.

[0052] In one implementation scenario, similar to the calculation method of the first risk level mentioned above, after calculating the first occurrence rate, the second risk level of the second element can be obtained by multiplying the severity and detectability of the failure mode to which the failure risk of the second element belongs with the first occurrence rate. For details, please refer to the calculation formula and related description of the first risk level RPN mentioned above, which will not be repeated here.

[0053] In one implementation scenario, as mentioned earlier, the failure mode knowledge base can define recommended measures for various failure modes. After judging the test responses of the second element based on the adjudication rules of the second element and obtaining the second risk level of the second element, it can also construct evidence data of the failure mode to which the failure risk of the second element belongs based on the test responses and test sessions where the failure risk of the second element actually exists. Then, based on the evidence data of the failure mode to which the failure risk of the second element belongs and the recommended measures, a failure analysis report of the target agent in the testing phase can be obtained. For details, please refer to the failure analysis report in the design phase mentioned above, which will not be repeated here. It should be noted that the analysis agent (i.e., the FMEA agent) can transform its theoretical predictions in the design phase (i.e., the first element) into actual, quantifiable risk assessments of the target agent before deployment in the testing phase. That is, what it delivers is no longer a warning of "potential error", but irrefutable evidence of "already error", which helps to provide the development team of the target agent with the most direct remediation targets and provides the management with the most powerful basis for deployment decisions.

[0054] In one implementation scenario, after judging the test response of the second element based on the adjudication rules of the second element to obtain the second risk level of the second element, the target agent can either optimize accordingly before going online or go online directly. In either case, the target agent's post-going session data can be filtered to obtain real dialogues, including the first session of real users and the second session of the target agent's response to the first session. Based on this, each real dialogue can be analyzed separately to obtain analysis results, including whether a failure mode is triggered. Based on real dialogues that trigger the same failure mode, an optimization task for the target agent is created, instructing the target agent to be optimized. In response to the target agent's re-launch after optimization, the session data after the target agent re-launches is analyzed to obtain the rate of change of the target agent's triggered failure modes. Based on the rate of change, it is determined whether to close the optimization task. For example, if the rate of change indicates a significant decrease in the occurrence rate of the failure mode, it proves that the optimization is effective, and the optimization task can be closed. The above method, after the target agent goes online, filters the conversation data to obtain real dialogues, and analyzes them to determine whether the real dialogues trigger failure modes. Then, combined with real dialogues that trigger the same failure mode, an optimization task is created for the target agent. After optimization and re-going online, the analysis is performed again to obtain the rate of change of the target agent's failure mode triggering, so as to determine whether to shut down the optimization task. This can form a closed-loop process of "discovery-analysis-repair-verification".

[0055] In a specific implementation scenario, before filtering the session data, the session data can be preprocessed, such as including but not limited to: data cleaning (e.g., removing irrelevant system logs), desensitizing user privacy information, formatting, etc., which can ultimately form unified structured session data (e.g., JSON format, including session identifier, timestamp, first session, second session).

[0056] In a specific implementation scenario, various filtering strategies can be combined when screening conversation data, such as rules / keywords, sentiment analysis, and random sampling. For example, when using rules / keywords for filtering, negative keywords such as "can't understand," "transfer to human agent," "complaint," "lie," and "wrong" can be matched, and those that match negative keywords can be filtered as real conversations. As another example, when using sentiment analysis for filtering, sentiment analysis can be performed on each conversation (e.g., using a sentiment analysis model) to filter out real conversations where the user's emotions are negative, such as "anger" or "disappointment." Yet another example is using random sampling for filtering, where random sampling can be performed on conversations where no problems are found (e.g., no negative keywords are matched, no negative emotions are present) (e.g., random sampling at a rate of 1%) to obtain real conversations. Of course, the above examples are just a few possible filtering strategies for conversation data; other possible methods are not limited here, nor will they be listed in detail.

[0057] In a specific implementation scenario, after filtering out real dialogues, it is possible to analyze whether the real dialogues trigger failure modes. For example, the analysis agent (i.e., the aforementioned FMEA agent) can also include a large dialogue analysis model (acting as a seasoned "AI dialogue quality analyst") used to analyze whether the real dialogues trigger failure modes. Specifically, based on real dialogues and various failure modes in the failure mode knowledge base, large model instructions can be constructed. These instructions instruct the large dialogue analysis model to analyze whether the real dialogues trigger failure modes and, if so, which failure modes are triggered. The analysis results of the real dialogues can then be obtained based on the output of the large model instructions, and these results may also include the type of failure mode triggered. For example, various failure modes may include, but are not limited to: failure to understand intent (i.e., inability to recognize user intent), factual errors / illusions (i.e., providing incorrect or outdated information), logical inconsistencies (i.e., contradictory answers in multi-turn dialogues), attitude / etiquette issues (i.e., impolite or harsh answers), security and compliance (i.e., disclosure of sensitive information), refusal to answer (i.e., refusing service for reasonable questions), etc. Furthermore, the output of the large dialogue analysis model can be a structured JSON object, which may include: { "conversation_id": "conv_12345", "is_failure": true, "failure_mode": "Factual_Error_Outdated_Info", "severity_suggestion": 7, / / 1-10 scale, for expert reference "failure_description": "The robot provided the old 7-day return policy, while the current policy has been updated." It is 14 days. "key_evidence": "User: 'Didn't you change to a 14-day no-reason return policy last month?' Bot: Our return policy is within 7 days... } It should be noted that in the above example, the field "conversation_id" represents the unique identifier of the real conversation, the field "is_failure" indicates whether the failure mode is triggered, the field "failure_mode" indicates the failure mode that is triggered, the field "severity_suggestion" indicates the severity of the suggestion, the field "failure_description" indicates the specific reason for triggering the failure mode, and the field "key_evidence" indicates the evidence data that triggers the failure mode (such as the real conversation itself).

[0058] In a specific implementation scenario, as one possible approach, after obtaining the analysis results of each real dialogue, an optimization task for the corresponding failure mode can be constructed based on the real dialogues that trigger the same failure mode, thus instructing the target agent to be optimized. As another possible approach, before creating the optimization task, statistics can be performed based on the analysis results of each real dialogue to obtain the second occurrence rate of each failure mode during the deployment phase (the specific calculation method can be found in the aforementioned calculation formula and related description of the first occurrence rate RPN). Then, based on the real dialogues that trigger the same failure mode and the third risk level of the corresponding failure mode, an optimization task for the target agent can be created, thus instructing the target agent to be optimized. In this way, failure evidence can be further provided when instructing the target agent to be optimized.

[0059] In a specific implementation scenario, during the deployment phase, the second occurrence rate of triggered failure modes can be continuously monitored and sorted chronologically to generate a trend chart, providing a clear understanding of the trend of failure modes triggered by the target agent. Furthermore, when the third risk level of a failure mode exceeds a set threshold (or shows a rapid upward trend), an optimization task can be generated. For example, a bug or task with detailed information (such as failure description, evidence data, and third risk level) can be created in a project management tool (e.g., Jira) and assigned to the appropriate development or operations team for optimization. Of course, if new failure modes are discovered during deployment, they can be added to the failure mode knowledge base, and experts can define initial S and D values.

[0060] In a specific implementation scenario, for ease of understanding, let's take an example after the system goes live: The target agent recently updated its return policy, extending it from 7 days no-reason return to 14 days, but the old information in its knowledge base has not been completely cleaned up. The following dialogue might occur at this point: User: "Hello, I'd like to inquire about the return policy for these headphones I just bought?" Target agent: "Hello! Our policy supports 7-day no-reason returns." User: "That doesn't make sense, does it? My friend bought it last week, and customer service said it was valid for 14 days. How come your information is inconsistent?" Customer service AI: "I'm sorry, according to my information, the current policy is still 7-day no-reason return. If..." You need further assistance; we can connect you to a live agent. Based on this, during the online phase, because the above dialogue contained keywords such as "incorrect," "inconsistent," and "transfer to human operator," and also detected negative emotions, it could be sent for further analysis. The large-scale dialogue analysis model analyzed this dialogue and obtained the following output results: { "conversation_id": "conv_xyz789", "is_failure": true, "failure_mode": "Factual_Error_Outdated_Info", "severity_suggestion": 8, "failure_description": "The robot provides an outdated 7-day return policy, compared to the current 14-day policy." Inconsistent policies can confuse users and potentially lead to complaints. "key_evidence": "User: 'That's not right, is it? My friend bought it last week, and customer service said it was 14 days...'" } It should be noted that the specific meanings of the relevant fields in the above examples can be found in the aforementioned descriptions, and will not be repeated here. After aggregation and statistics, it was found that in the past 24 hours, among 2000 inquiries about the return policy, 100 of them had the same error. Therefore, the occurrence rate can be calculated as 100 / 20000=0.5%. According to the preset mapping table, this ratio corresponds to O=6 in the FMEA scale (i.e., the score corresponding to the degree of occurrence rate). The S and D values ​​of this failure mode are found to be 8 and 3 respectively from the failure mode knowledge base. Therefore, the third risk level RPN can be calculated as 8*6*3=144. Since it has exceeded the set threshold of 100, a high-priority bug can be automatically created in Jira. The title can be "[FMEA Alert] Return Policy Information Error, RPN=144". The content can include all the above analysis data and dialogue data, so that the product and development teams can quickly locate the omission in the knowledge base update after receiving the notification and fix and optimize it. After actual testing, in the days following the fix's deployment, the occurrence rate of the failure mode "Factual_Error_Outdated_Info" rapidly decreased to 0, thus verifying the effectiveness of the fix and optimization.

[0061] In one implementation scenario, after the target agent goes online, based on the session data of the target agent after going online, the second occurrence rate of each failure mode in the online stage can be calculated. After calculating the second occurrence rate of the failure mode, the historical occurrence rate of the failure mode in the failure mode knowledge base can be updated based on the second occurrence rate of the failure mode.

[0062] In an implementation scenario, to facilitate understanding of the entire lifecycle process of failure analysis for a target intelligent agent, please refer to [reference needed]. Figure 2 , Figure 2 This is a schematic diagram of a process according to an embodiment of the failure analysis method of this application. Figure 2 As shown, the entire lifecycle can include the design phase, testing phase, and deployment phase. In the design phase, analysis can be performed based on the target agent's requirements document to identify the first risk factor and the first risk level (RPN) of the first risk factor, using a failure mode knowledge base. High-risk items are then selected as the second risk factor for task management and closed-loop verification. Specifically, in the testing phase, for the second risk factor, generative large-scale models can be used to simulate real-world test sessions to perform attack tests on the target agent. This allows for the analysis of the second risk level of the second risk factor, generating a failure analysis report for the testing phase. In the deployment phase, failure analysis can be performed using session data from a real-world environment. It should be noted that failure analysis can be performed by an analytical agent (FMEA agent) in all phases—design, testing, and deployment—as detailed in the preceding descriptions, and will not be repeated here.

[0063] The above scheme detects the target agent's design elements based on its description document. It then analyzes these elements using a failure mode knowledge base, identifying the first element suspected of having a failure risk and its first risk level. Based on the first risk level of each first element, it selects it as the second element. Using the attack strategy knowledge base and the second element, it generates a test session and adjudication rules for the second element. The test session is used to test whether the second element of the target agent truly has a failure risk before the target agent goes live. Finally, based on the adjudication rules for the second element, it judges the test response to obtain the second risk level of the second element. The test response for the second element is obtained by the target agent's response to the test session for the second element. On one hand, because the first element potentially at risk of failure is pre-analyzed based on its description document before the target agent goes live, and real-world dialogue is simulated during the testing phase to test whether the first element truly poses a failure risk, the failure risk of the agent can be identified in advance before its deployment. On the other hand, because the first risk level of the first element potentially at risk of failure is determined in the design phase based on the failure mode knowledge base before the target agent goes live, and further assessed in the testing phase based on the test responses to the test session to obtain the second risk level during the testing phase, the failure risk can be quantified. Therefore, the failure risk of the agent can be identified and quantified in advance before its deployment.

[0064] Please see Figure 3 , Figure 3 This is a schematic diagram of the framework of an embodiment of the failure analysis device of this application. The failure analysis device 30 includes: an element detection module 31, an element analysis module 32, an element selection module 33, a test generation module 34, and a test judgment module 35. The element detection module 31 is used to detect based on the description document of the target intelligent agent to obtain the design elements of the target intelligent agent; the element analysis module 32 is used to analyze based on the failure mode knowledge base and each design element to determine the first element suspected of having failure risk and the first risk level of the first element among the design elements; the element selection module 33 is used to select the first element as the second element based on the first risk level of each first element; the test generation module 34 is used to generate a test session and a referee rule for the second element based on the attack strategy knowledge base and the second element; wherein, the test session is used to test whether the second element of the target intelligent agent actually has a failure risk before the target intelligent agent goes online; the test judgment module 35 is used to judge the test response of the second element based on the referee rule of the second element to obtain the second risk level of the second element; wherein, the test response of the second element is obtained by the target intelligent agent responding to the test session of the second element.

[0065] In the above scheme, the failure analysis device 30 detects the target agent's description document to obtain the target agent's design elements. Based on the failure mode knowledge base and each design element, it analyzes them to identify the first element suspected of having a failure risk and its first risk level. Then, based on the first risk level of each first element, it selects the first element as the second element. Based on the attack strategy knowledge base and the second element, it generates a test session and adjudication rules for the second element. The test session is used to test whether the second element of the target agent truly has a failure risk before the target agent goes online. Finally, based on the adjudication rules for the second element, it judges the test response of the second element to obtain the second risk of the second element. The degree of failure risk is determined by analyzing the target agent's response to the test session of the second element. Firstly, because the first element with potential failure risk is pre-analyzed based on its description document before the target agent goes live, and real dialogue is simulated during the testing phase to test whether the first element truly has a failure risk, the failure risk of the agent can be identified in advance before the agent goes live. Secondly, because the first risk level of the first element with potential failure risk is determined in the design phase based on the failure mode knowledge base before the target agent goes live, and further determined in the testing phase based on the test response to the test session to obtain the second risk level during the testing phase, the failure risk can be quantified. Therefore, the failure risk of the agent can be identified and quantified in advance before the agent goes live.

[0066] In some disclosed embodiments, the element analysis module 32 includes a semantic matching submodule, which performs semantic matching between the design element and the triggering rules of various failure modes in the failure mode knowledge base, respectively, to obtain a matching result indicating whether the design element hits the triggering rule; wherein, the failure mode knowledge base also defines risk attributes of various failure modes; the element analysis module 32 includes a first analysis submodule, which, in response to the matching result indicating that the design element hits the triggering rule, selects the design element as a first element with suspected failure risk, selects the failure mode to which the triggering rule hit by the design element belongs as the target mode, and performs analysis based on the risk attributes of the target mode to obtain the first risk level of the first element.

[0067] In some publicly disclosed embodiments, the risk attributes include severity, historical occurrence rate, and detectability. The element analysis module 32 includes an impact deduction submodule, which is used to perform deduction based on the context of the first element to obtain the business impact on the target agent when the first element is assumed to have the failure risk represented by the target pattern. The element analysis module 32 includes a degree update submodule, which is used to update the severity of the target pattern based on the business impact to obtain a new severity of the target pattern. The first analysis submodule is specifically used to fuse the new severity of the target pattern, historical occurrence rate, and detectability to obtain a first risk level of the first element that is suspected to have the failure risk represented by the target pattern.

[0068] In some disclosed embodiments, the description document also detects the description text of the design elements in the description document. The failure analysis device 30 includes a text analysis module for analyzing the description text of the first element in the description document to obtain a reasoning chain that the first element is suspected of having a failure risk. The failure analysis device 30 includes a first generation module for obtaining a failure analysis report of the target intelligent agent in the design stage based on the first element, the first risk level of the first element, and the reasoning chain.

[0069] In some disclosed embodiments, the test generation module 34 includes a policy query submodule, used to query the attack policy knowledge base based on the second element to obtain the attack policy in the attack policy knowledge base that matches the second element; the test generation module 34 includes a policy parsing submodule, used to parse based on the attack policy that matches the second element to obtain the session generation instruction and referee rules of the second element; wherein, the session generation instruction is used to instruct the generative large model to generate a session according to the requirements of the session generation instruction; the test generation module 34 includes a session generation submodule, used to obtain several sessions generated by the generative large model in response to the session generation instruction of the second element, which are respectively used as test sessions of the second element.

[0070] In some disclosed embodiments, the test determination module 35 includes a response detection submodule, used to detect each test response of the second element according to the adjudication rules to obtain the number of times the second element actually has a failure risk; the test determination module 35 includes a frequency calculation submodule, used to obtain the first occurrence rate of the second element's actual failure risk during the testing phase based on the ratio of the number of times to the total number of test sessions of the second element; the test determination module 35 includes a risk calculation submodule, used to fuse the severity, first occurrence rate, and detectability of the failure mode to which the failure risk of the second element belongs to obtain the second risk level of the second element; wherein, the failure mode knowledge base defines risk attributes of various failure modes, and the risk attributes include at least severity and detectability.

[0071] In some disclosed embodiments, the failure mode knowledge base defines recommended measures for various failure modes. The failure analysis device 30 includes an evidence construction module for constructing evidence data of the failure mode to which the failure risk of the second element belongs, based on test responses and test sessions where the failure risk of the second element actually exists. The failure analysis device 30 also includes a second generation module for obtaining a failure analysis report of the target agent during the testing phase based on the evidence data of the failure mode to which the failure risk of the second element belongs and the recommended measures.

[0072] In some disclosed embodiments, the failure analysis device 30 includes a data filtering module for filtering based on the target agent's session data after it goes online to obtain real dialogues; wherein, the real dialogues include a first session of a real user and a second session in which the target agent responds to the first session; the failure analysis device 30 includes a dialogue analysis module for analyzing each real dialogue separately to obtain analysis results for each real dialogue; wherein, the analysis results include whether a failure mode is triggered; the failure analysis device 30 includes a task creation module for creating an optimization task for the target agent based on real dialogues that trigger the same failure mode; wherein, the optimization task is used to instruct the target agent to be optimized; the failure analysis device 30 includes a shutdown confirmation module for responding to the target agent's re-entry after optimization, analyzing the session data after the target agent re-enters online to obtain the rate of change of the target agent's triggered failure modes, and determining whether to shut down the optimization task based on the rate of change.

[0073] In some disclosed embodiments, the failure analysis device 30 includes a result statistics module for performing statistics based on the analysis results of each real dialogue to obtain a second occurrence rate of each failure mode in the online phase; the failure analysis device 30 includes a risk calculation module for fusing the severity, second occurrence rate and detectability of the failure mode to obtain a third risk level of the failure mode; the task creation module is specifically used to create an optimization task for the target agent based on the real dialogue that triggers the same failure mode and the third risk level of the corresponding failure mode.

[0074] In some disclosed embodiments, the design elements include at least one of the following: functional points, interaction flows of functional points, and business rules of functional points; and / or, the second element is selected from each of the first elements after being sorted according to the first risk level; and / or, the failure analysis of the target intelligent agent is implemented by the analysis intelligent agent, which includes a generative large model and a referee large model, the test session is generated by the generative large model, and the judgment of the test response is implemented by the referee large model with reference to the referee rules; and / or, the failure mode knowledge base defines risk attributes of various failure modes, the risk attributes include at least the historical occurrence rate of the failure mode, the historical occurrence rate is used to calculate the first risk level, and after the target intelligent agent goes online, the second occurrence rate of each failure mode in the online stage is statistically obtained based on the session data of the target intelligent agent after going online, and after the second occurrence rate of the failure mode is statistically obtained, the historical occurrence rate of the failure mode in the failure mode knowledge base is updated based on the second occurrence rate of the failure mode.

[0075] Please see Figure 4 , Figure 4 This is a schematic diagram of a framework of an embodiment of the electronic device of this application. The electronic device 40 includes at least a memory 41 and a processor 42 coupled to each other. The memory 41 stores at least program instructions, and the processor 42 is used to execute the program instructions to implement the steps in any of the failure analysis method embodiments described above. For details, please refer to the foregoing disclosed embodiments, which will not be repeated here.

[0076] Specifically, processor 42 controls itself and memory 41 to implement the steps in any of the failure analysis method embodiments described above. Processor 42 can also be referred to as a CPU (Central Processing Unit). Processor 42 may be an integrated circuit chip with signal processing capabilities. Processor 42 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 42 can be implemented using integrated circuit chips.

[0077] In the above scheme, the electronic device 40 performs detection based on the description document of the target intelligent agent to obtain the design elements of the target intelligent agent. It then analyzes each design element based on a failure mode knowledge base, identifying the first element suspected of having a failure risk and its first risk level. Based on the first risk level of each first element, it selects the first element as the second element. Then, based on the attack strategy knowledge base and the second element, it generates a test session and adjudication rules for the second element. The test session is used to test whether the second element of the target intelligent agent truly has a failure risk before the target intelligent agent goes online. Finally, based on the adjudication rules for the second element, it judges the test response of the second element to obtain the second risk level of the second element. Furthermore, the test response of the second element is obtained by the target agent's response to the test session of the second element. On the one hand, because the first element with potential failure risk is pre-analyzed based on its description document before the target agent goes live, and real dialogue is further simulated during the testing phase to test whether the first element truly has a failure risk, the failure risk of the agent can be identified in advance before the agent goes live. On the other hand, because the first risk level of the first element with potential failure risk is determined in the design phase based on the failure mode knowledge base before the target agent goes live, and further judged in the testing phase based on the test response of the test session to obtain the second risk level in the testing phase, the failure risk can be quantified. Therefore, the failure risk of the agent can be identified and quantified in advance before the agent goes live.

[0078] Please see Figure 5 , Figure 5 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 50 stores program instructions 51 that can be executed by a processor. The program instructions 51 are used to implement the steps in any of the failure analysis method embodiments described above.

[0079] In the above scheme, the computer-readable storage medium 50 performs detection based on the description document of the target intelligent agent to obtain the design elements of the target intelligent agent. Based on the failure mode knowledge base and each design element, it analyzes and identifies the first element suspected of having a failure risk and its first risk level. Then, based on the first risk level of each first element, it selects the first element as the second element. Based on the attack strategy knowledge base and the second element, it generates a test session and adjudication rules for the second element. The test session is used to test whether the second element of the target intelligent agent truly has a failure risk before the target intelligent agent goes online. Finally, based on the adjudication rules for the second element, it judges the test response of the second element to obtain the second element's second... The risk level is determined by the test response of the target agent to the test session of the second element. Firstly, because the first element with potential failure risk is pre-analyzed based on its description document before the target agent goes live, and real dialogue is simulated during the testing phase to test whether the first element truly has a failure risk, the failure risk of the agent can be identified in advance before the agent goes live. Secondly, because the first risk level of the first element with potential failure risk is determined in the design phase based on the failure mode knowledge base before the target agent goes live, and further determined in the testing phase based on the test response to the test session to obtain the second risk level in the testing phase, the failure risk can be quantified. Therefore, the failure risk of the agent can be identified and quantified in advance before the agent goes live.

[0080] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0081] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0083] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0084] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0085] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. A failure analysis method, characterized in that, include: The design elements of the target intelligent agent are obtained by detecting the description document of the target intelligent agent. Based on the failure mode knowledge base and the various design elements, an analysis is performed to identify the first element that is suspected of having a failure risk and the first risk level of the first element among the various design elements. Based on the first risk level of each of the first elements, the first element is selected as the second element; Based on the attack strategy knowledge base and the second element, a test session and referee rules for the second element are generated; wherein, the test session is used to test whether the second element of the target agent truly has a risk of failure before the target agent goes online; The test response to the second element is judged based on the adjudication rules of the second element to obtain the second risk level of the second element; wherein, the test response to the second element is obtained by the target agent responding to the test session of the second element.

2. The method according to claim 1, characterized in that, The analysis based on the failure mode knowledge base and each of the design elements, identifying a first element suspected of having failure risk and a first risk level of the first element among the design elements, includes: Based on the triggering rules of various failure modes in the failure mode knowledge base, semantic matching is performed with the design elements to obtain matching results that characterize whether the design elements match the triggering rules; wherein, the failure mode knowledge base also defines risk attributes of various failure modes. In response to the matching result indicating that the design element hits the triggering rule, the design element is selected as the first element suspected of having a failure risk, and the failure mode to which the triggering rule hit by the design element belongs is selected as the target mode. Based on the risk attributes of the target mode, the first risk level of the first element is obtained.

3. The method according to claim 2, characterized in that, The risk attributes include severity, historical incidence, and detectability. Before analyzing the risk attributes based on the target pattern to obtain the first risk level of the first element, the method further includes: Based on the context of the first element, the business impact on the target intelligent agent is obtained when the failure risk represented by the target pattern is assumed to exist in the first element. The severity of the target pattern is updated based on the business impact to obtain a new severity of the target pattern; The analysis of risk attributes based on the target pattern to obtain the first risk level of the first element includes: Based on the fusion of the new severity, historical occurrence rate and detectability of the target pattern, a first risk level is obtained for the first element that is suspected of having the failure risk represented by the target pattern.

4. The method according to claim 1, characterized in that, The description document also detects the descriptive text of the design element in the description document, and the method further includes: Based on the analysis of the description text of the first element in the description document, a reasoning chain is obtained that the first element is suspected of having a failure risk. Based on the first element and the first risk level and reasoning chain of the first element, a failure analysis report of the target intelligent agent in the design phase is obtained.

5. The method according to claim 1, characterized in that, The process of generating test sessions and referee rules for the second element based on the attack strategy knowledge base and the second element includes: Based on the second element, a query is performed in the attack strategy knowledge base to obtain an attack strategy in the attack strategy knowledge base that is compatible with the second element. Based on the attack strategy adapted to the second element, the session generation instructions and referee rules of the second element are obtained through analysis; wherein, the session generation instructions are used to instruct the generative large model to generate sessions according to the requirements of the session generation instructions; Several sessions generated by the generative large model in response to the session generation instruction of the second element are obtained and used as test sessions of the second element.

6. The method according to claim 1, characterized in that, The adjudication rules based on the second element determine the test response to the second element, thereby obtaining the second risk level of the second element, including: Based on the adjudication rules of the second element, each test response is tested to obtain the number of times the second element actually has a failure risk; Based on the ratio of the number of times to the total number of test sessions for the second element, the first occurrence rate of the actual failure risk of the second element during the testing phase is obtained; The second risk level of the second element is obtained by fusing the severity, first occurrence rate, and detectability of the failure mode to which the failure risk of the second element belongs; wherein, the failure mode knowledge base defines risk attributes of various failure modes, and the risk attributes include at least the severity and the detectability.

7. The method according to claim 1, characterized in that, The failure mode knowledge base defines recommended measures for various failure modes. After determining the test response of the second element based on the adjudication rules of the second element to obtain the second risk level of the second element, the method further includes: Based on test responses and test sessions where the second element has a real risk of failure, construct evidence data of the failure mode to which the failure risk of the second element belongs; Based on the evidence data and recommended measures regarding the failure modes to which the failure risk exists in the second element, a failure analysis report of the target intelligent agent during the testing phase is obtained.

8. The method according to claim 1, characterized in that, After determining the test response to the second element based on the adjudication rules for the second element and obtaining the second risk level of the second element, the method further includes: The real dialogues are obtained by filtering the conversation data of the target intelligent agent after it goes online; wherein, the real dialogues include the first conversation of the real user and the second conversation of the target intelligent agent's reply to the first conversation; Each of the aforementioned real dialogues is analyzed separately to obtain analysis results for each of the real dialogues; wherein, the analysis results include whether a failure mode is triggered. Based on real dialogues that trigger the same failure mode, an optimization task is created for the target agent; wherein the optimization task is used to instruct the target agent to be optimized. In response to the target agent's re-launch after optimization, the session data after the target agent's re-launch is analyzed to obtain the rate of change of the target agent triggering the failure mode, and based on the rate of change, it is determined whether to close the optimization task.

9. The method according to claim 8, characterized in that, Before creating the optimization task for the target agent based on real dialogues that trigger the same failure mode, the method further includes: Based on the analysis results of each of the real dialogues, the second occurrence rate of each failure mode in the online phase is obtained. A third risk level of the failure mode is obtained by fusing the severity, second occurrence rate, and detectability of the failure mode. The optimization task for creating the target agent based on real dialogues that trigger the same failure mode includes: Based on real dialogues that trigger the same failure mode and the third risk level corresponding to the failure mode, an optimization task is created for the target agent.

10. The method according to any one of claims 1 to 9, characterized in that, The design elements include at least one of the following: functional points, the interaction flow of the functional points, and the business rules of the functional points; And / or, the second element is selected from each of the first elements after they have been sorted according to the first risk level; And / or, the failure analysis of the target agent is implemented by an analysis agent, which includes a generative large model and a referee large model. The test session is generated by the generative large model, and the judgment of the test response is implemented by the referee large model with reference to the referee rules. And / or, the failure mode knowledge base defines risk attributes for various failure modes, the risk attributes including at least the historical occurrence rate of the failure mode, the historical occurrence rate being used to calculate the first risk level, and after the target agent goes online, based on the session data of the target agent after going online, a second occurrence rate of each failure mode in the online phase is statistically obtained, and after obtaining the second occurrence rate of the failure mode, the historical occurrence rate of the failure mode in the failure mode knowledge base is updated based on the second occurrence rate of the failure mode.

11. A failure analysis device, characterized in that, include: The element detection module is used to detect the design elements of the target intelligent agent based on the description document of the target intelligent agent. The element analysis module is used to analyze the design elements based on the failure mode knowledge base and each of the design elements, and to identify the first element that is suspected of having a failure risk and the first risk level of the first element among the design elements. The element selection module is used to select the first element as the second element based on the first risk level of each first element. The test generation module is used to generate a test session and referee rules for the second element based on the attack strategy knowledge base and the second element; wherein, the test session is used to test whether the second element of the target intelligent agent actually has a risk of failure before the target intelligent agent goes online; The test judgment module is used to judge the test response of the second element based on the adjudication rules of the second element, and obtain the second risk level of the second element; wherein, the test response of the second element is obtained by the target intelligent agent responding to the test session of the second element.

12. An electronic device, characterized in that, It includes at least a memory and a processor coupled to each other, wherein the memory stores at least program instructions, and the processor is used to execute the program instructions to implement the failure analysis method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The system stores program instructions that can be executed by a processor, the program instructions being used to implement the failure analysis method according to any one of claims 1 to 10.

Images