A failure test case automation processing method

Abstract

The application provides a failure test case automatic processing method, which is applied to automatic test maintenance in EDA software development, and comprises the following steps: classifying collected failure test case lists through set calculation and incremental analysis; structuring an external rule library and dynamically loading; based on the external rule library, using a conditional matching engine to perform multi-modal rule matching on the failure test cases; based on an atomized repair action execution engine, repairing the matched use cases; performing aggregate analysis on the failure test cases, and outputting classification statistical reports and running logs. The application abstracts the identification logic and repair execution logic of the failure mode into independent configurable objects by constructing a decoupled rule engine, and realizes accurate triggering through an incremental analysis mechanism, realizes automatic processing of the failure test cases, greatly shortens the overall test iteration cycle, ensures that the failure test cases finally submitted to the developers are real software defects, and greatly improves the work efficiency.

Description

Technical Field

[0001] This application relates to the field of EDA automated testing technology, and in particular to a method for automated handling of failed test cases. Background Technology

[0002] In the development of large-scale electronic design automation (EDA) software, to ensure the correctness of software functions and version stability, it is necessary to establish and maintain a large automated test suite (test case library). These test suites contain tens of thousands of test cases used to verify various functions of the software, such as schematic editing, layout generation, accuracy of specific simulators (such as harmonic balance and electromagnetic simulation), and post-processing of results. With the continuous iteration and updates of EDA software, due to differences in the underlying operating system, changes in environment configuration, or outdated reference data, a large number of "invalid failure" test cases (i.e., failures not caused by software defects) frequently occur.

[0003] Existing technical solutions mainly rely on the following two methods for failure handling: 1) Manual troubleshooting and repair: Test engineers manually check logs, compare output files, and reproduce steps. This method is extremely inefficient and highly dependent on personal experience, making it difficult to handle a large number of test cases.

[0004] 2) Hard-coded fixed scripts: Some teams write fixed Shell or Python scripts for batch processing of known common errors. However, this existing approach has significant drawbacks: lack of versatility and flexibility: fixed scripts typically hard-code the "checking logic" and "repair actions" together, making them unable to be dynamically expanded, difficult to support complex combinations of conditions, and unable to dynamically load new processing rules without restarting the service; inability to accumulate experience: processing logic is scattered across various temporary scripts, failing to form a unified, configurable "rule base," making it difficult for the team to accumulate and reuse testing and maintenance experience.

[0005] Furthermore, existing technologies often struggle to effectively distinguish between "real failures caused by software defects" and "invalid failures caused by environment / configuration," leading to developers being overwhelmed by noise. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this application aims to provide an automated method for handling failed test cases, applicable to EDA software testing. By constructing a decoupled rule engine, the failure mode identification logic (Condition) and repair execution logic (Action) are abstracted into independent configurable objects, and precise triggering is achieved through an incremental analysis mechanism, thereby realizing the automated handling of failed test cases.

[0007] To achieve the above objectives, the automated failure test case processing method provided in this application is applied to automated test maintenance in EDA software development, including the following steps: classifying the collected failure test case list through set computation and incremental analysis; structuring and dynamically loading an external rule base; performing multimodal rule matching on the failure test cases using a conditional matching engine based on the external rule base; repairing successfully matched test cases based on an atomic repair action execution engine; and performing aggregate analysis on the failure test cases to output a classification statistical report and runtime logs. Further, the step of classifying the collected failure test case list through set computation and incremental analysis also includes: collecting the failure test case list from multiple dimensions such as time, operating platform, and underlying dependencies by accessing emails and specified file directories; and using a set difference algorithm to perform set computation and incremental analysis on the failure test case list to obtain a complete failure test case list, a recently added failure test case list, and a difference list of failure test cases caused by different platforms or underlying dependencies. Furthermore, the structured external rule base also includes: a structured external YAMLS rule base, which includes: a core rule set comprising multiple independent rules; a secure script directory for storing verified and trusted script files; and a working copy directory for storing temporary files and backup data during rule execution. Furthermore, each of the independent rules includes: an identifier, a unique identifier for the rule, used for quick location and reference in the rule base; an enable switch for controlling whether the independent rule is effective; a check condition group for the rule triggering logic, including the condition type and the target object of the condition; and a repair action group for the operation to be performed after the condition is met, including the action type and the target object of the action. Furthermore, the step of dynamically loading the structured external rule base also includes: customizing the external rule base according to the following: cleaning up absolute paths in failed test cases; updating the software license name in the comparison file; extracting failed test cases that use images as comparison files; supporting custom third-party scripts as rules; and replacing the text difference comparison tool for specified files in failed test cases with tolerance difference values. Furthermore, the step of performing multimodal rule matching on failed test cases using a conditional matching engine based on the external rule base further includes: traversing the entire list of failed test cases, checking the failed test cases using atomic conditional detection units within the conditional matching engine, and if each conditional detection unit meets the conditions of a certain failed test case, then the failed test case is considered a matching object. Further, the atomic conditional detection units construct atomic conditional detection operators through logical combination; the atomic conditional detection operators include: a file matching operator for matching files whose filenames contain specified action characteristic characters; a file content feature matching operator for configuring specified content of specified files; and a sandboxed script probe operator for constructing a sandboxed script execution environment.Furthermore, the step of repairing successfully matched use cases based on the atomic repair action execution engine also includes: reading the action type in the YAML rule, determining the repair action to be executed, and creating an isolated temporary directory for all action executions; for target content filtering / query actions, performing the following steps to find and filter specific content in the target file: locating the target file specified in the YAML rule and matching the target_member file; searching for content in the file that meets the rule conditions and matching the target content; matching command target parameters; or, for third-party script execution actions, performing the following steps to securely call external scripts and perform complex automated operations: finding third-party scripts: locating the script file specified in the YAML rule from the predefined trusted script directory; constructing an execution command based on script parameters: reading the script parameters defined in the YAML rule and concatenating them into a complete execution command; executing the third-party script: executing the script in an isolated environment and capturing the script's output, return code, and logs; or, for target content modification / replacement actions, performing the following steps to securely modify the target file content: matching the target_member file: locating the target file to be modified; replacing the target content file by file: modifying the file content in a temporary copy according to the replacement rules defined in the YAML rule.

[0008] To achieve the above objectives, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the automated test case processing method described above.

[0009] To achieve the above objectives, this application also provides a computer-readable storage medium storing a computer program that is loaded and executed by a processor to implement the automated test case processing method described above.

[0010] Compared with existing technologies, the automated failure test case handling method of this application has the following significant advantages: 1) Significantly Improves Test Operation and Maintenance Efficiency: This invention significantly reduces the cost of manual intervention, compressing the time spent investigating and fixing similar failed test cases from hours of manual work to minutes of system work, greatly shortening the overall test iteration cycle. For the current scale of hundreds of failed test cases, through the automated analysis and intelligent allocation mechanism of this application, the number of test cases to be analyzed per person is drastically reduced from screening all cases one by one to the ten level (approximately 10-50 cases). It eliminates the inefficient manual review and distribution process of the traditional model, reducing the time cost for analysts in the fault attribution stage by approximately 90%, achieving an order-of-magnitude leap in test efficiency. 2) Further Automates the Test Process: Transforms individual, implicit debugging experience into explicit, executable rules for the team. Based on automated testing, it achieves automated analysis and processing as much as possible. 3) Precisely Focuses on Real Defects: By automating the handling of known issues, it ensures that the failed test cases ultimately submitted to developers are highly likely to be real software defects, significantly improving work efficiency. 4) Reduces Test Case Library Maintenance Costs: Automated handling of test case "failures" due to differences in platform, environment, and configuration significantly reduces long-term maintenance costs. Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing this application. Attached Figure Description

[0011] The accompanying drawings are provided to further illustrate the present application and form part of the specification. Together with the embodiments of the present application, they serve to explain the present application but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of the automated test case handling method according to an embodiment of this application; Figure 2 This is a diagram of the YAMLS rule base structure according to an embodiment of this application; Figure 3 This is a flowchart illustrating the action engine processing of an embodiment of this application. Figure 4 This is a list of all failed test cases in the embodiments of this application; Figure 5 This is a list of recently added failed test cases in the embodiments of this application; Figure 6 This is a differentiated list of failure test cases caused by different platforms or underlying dependencies in the embodiments of this application. Figure 7 This is a schematic diagram of absolute path rules used for filtering test cases in an embodiment of this application; Figure 8 This is a schematic diagram of the ARM license repair rules in an embodiment of this application; Figure 9 This is a schematic diagram of the file matching operator in an embodiment of this application; Figure 10 This is a schematic diagram of the sandboxed script probe operator in an embodiment of this application; Figure 11 This is a schematic diagram illustrating the replacement of structured content in an embodiment of this application; Figure 12 This is a schematic diagram of a preliminary classification report for failed test cases that cannot be processed automatically, according to an embodiment of this application. Figure 13 This is a schematic diagram illustrating the allocation results of recently added test cases in an embodiment of this application; Figure 14 This is a schematic diagram illustrating the classification of processing methods for failed test cases in embodiments of this application; Figure 15 This is a log illustration of the automated test case processing method according to an embodiment of this application; Figure 16 This is a schematic diagram showing the comparison results of test failures in an embodiment of this application; Figure 17 This is a schematic diagram of an electronic device structure according to an embodiment of this application.

[0012] The above figures are only used to illustrate the technical solution of this application and are not intended to limit the scope of protection of this application. Detailed Implementation

[0013] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application.

[0014] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0015] The term "comprising" and its variations as used in this application are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0016] It should be noted that the terms "first" and "second" may be used in this application only to distinguish different devices, components or parts, and are not used to define the order of functions performed by these devices, components or parts or their interdependence.

[0017] It should be noted that the terms "one" and "more" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless explicitly stated otherwise in the context, they should be understood as "one or more". "More" should be understood as two or more.

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0019] Example 1 The technical solution of this application will be described in detail below with reference to specific embodiments. Figure 1 This is a flowchart of the automated failure test case processing method according to an embodiment of this application. The following will be combined with... Figure 1 The method for automating the handling of failed test cases according to the embodiments of this application is described in further detail.

[0020] First, in step 101, a list of failed test cases is collected from multiple dimensions, and the list of failed test cases is classified through set computation and incremental analysis.

[0021] In this embodiment of the application, a list of failed test cases is collected from multiple dimensions such as time, operating platform, and underlying dependencies by accessing emails and specified file directories.

[0022] In this embodiment of the application, in order to solve the problem of low efficiency of full analysis, a set difference algorithm is adopted to perform set calculation and incremental analysis on the obtained multi-dimensional failure test case list. The latest test report is compared with the historical benchmark report and the test reports of cross-platform and cross-dependency versions, and the difference set of failure test cases is automatically extracted. The algorithm outputs a complete list of failure test cases, a list of recently added failure test cases, and a list of failure test case differences caused by different platforms or underlying dependencies.

[0023] The failure test case filtering mechanism based on incremental set operation in this application can mathematically eliminate long-standing known failure interference, ensuring that the subsequent rule engine only focuses on the latest anomalies, and significantly reducing the computational load.

[0024] In step 102, the external rule base is structured and dynamically loaded, and the check condition group and repair action group are parsed.

[0025] In this embodiment, the traditional hard-coding method is abandoned, and a structured rule description language based on YAML format is adopted. Each rule object in the external rule base consists of a unique identifier (ID), an enable switch, a condition set, and an action set, which can achieve physical separation of business logic and program code. Users do not need to modify the core engine code, but only need to edit the configuration file to define new failure handling strategies, supporting hot reloading and version control.

[0026] Figure 2 This is a diagram of the YAMLS rule base structure according to an embodiment of this application, as follows: Figure 2 As shown in the embodiment of this application, the YAMLS rule base, based on a YAML rule engine, includes the structure of individual rules and the overall organization of the rule base, wherein... The YAMLS rule base is a collection of rules, along with associated security and operational directory configurations: The core set of rules is YAML rules, which contains multiple independent rules: rule1, rule2, rule3, etc. Each rule follows the YAML structure on the left, achieving decoupling and extensibility of rules.

[0027] Key directory configuration The safe script directory (safe_script_dir) stores verified and trusted script files to prevent the execution of malicious code.

[0028] The working copy directory (working_copy_dir) is used to store temporary files and backup data when the rules are running, ensuring the safety of the original data.

[0029] YAML rules, each rule consists of the following components: ID Index: A unique identifier for rules, used for quick location and reference within the rule base; Enabled switch: This is the master switch that controls whether a rule is effective, making it easy to start and stop rules in batches; The condition set is the judgment logic triggered by the rule, including the condition type (such as equal to, greater than, regular expression matching, etc.) and the target object of the condition (such as a configuration item or log field). Action Set: This set defines the operations to be performed when certain conditions are met. It includes the action type (actiontype, such as modifying configuration, repairing scripts, alarm notifications, etc.) and the target object of the action (action target_member, such as the file to be modified or the interface to be called).

[0030] In this embodiment, the enable switch is used to set the current rule to be enabled or disabled. The condition and action consist of a large number of reusable key-value pairs, which together constitute the specific parameters for the inspection and repair operations. Furthermore, different types of conditions and actions can be randomly combined to form new rules.

[0031] In this embodiment, the action engine loads the YAMLS rule base and parses out the ID, Enabled, condition, and action.

[0032] In step 103, the Condition Engine performs multimodal rule matching on the failed test cases based on an external rule base.

[0033] In this application embodiment, three atomic condition detection operators are designed to address the diversity of EDA test case failures and support logical combinations: (1) File matching operator: Introducing the gold_mapping mapping mechanism to match the comparison file (i.e., the gold file and its corresponding result file) in the test case, solving the matching problem caused by dynamic file names; supporting case-sensitive control and multi-keyword Boolean matching; (2) File content feature matching operator: Based on the regular expression capture of the test case name or the test case file content, quickly locate the failure of a specific module; (3) Sandboxed script probe operator: By executing a third-party script in an isolated secure directory and capturing the return code, arbitrarily complex custom logic is transformed into standardized Boolean judgment conditions.

[0034] In step 104, the atomic repair action execution engine is started to repair the successfully matched test cases.

[0035] In this embodiment of the application, in order to ensure the security of the repair process, a "temporary directory isolation execution" mechanism is designed: (1) Data isolation: All successfully matched test cases are first copied to an independent temporary workspace, and all repair operations are performed only on the copy. If the execution fails or the verification fails, the original data is not affected in any way, ensuring the consistency of the test assets. (2) Diverse actions: Streaming text filtering: Dynamically generate sed / awk command sequences, accurately inject them into the post-processing script, and automatically remove environmental noise such as absolute paths. Structured content replacement: Based on regular tree multi-set mapping replacement, it supports batch correction of parameter errors in the result comparison file. External process call: Call the authorized script to execute complex repair tasks in a restricted environment, and prevent the process from hanging through timeout control.

[0036] Figure 3 This is a flowchart of the action engine processing in an embodiment of this application, such as... Figure 3 As shown, the core execution logic of the action engine starts from "rule matching successful" and is divided into three independent execution branches according to different action.types. Each branch eventually generates an analysis report, forming a complete action execution loop.

[0037] Receive test cases where the rule matches successfully: The rule engine's condition matcher has confirmed that the current event / data meets the condition of the YAML rule and enters the action execution phase; Parse action.type: Read the type of action in the YAML rule to determine the remediation action to be performed; Create a temporary processing directory: Create an isolated temporary directory for all actions, corresponding to working_copy_dir, to avoid direct manipulation of the original files and ensure data security; The action engine in this application performs the following steps for target content filtering / query actions (filter_target_line) to find and filter specific content in the target file: Match target_member file: Locate the target file (such as configuration file, log file) specified in the YAML rule; Match target content: Search for content in a file that matches the specified rules (such as specific strings, regular expressions, or field values). Match command target parameters: If the rule defines additional filtering parameters (such as line number, range, quantity limit), then these parameters are applied to further filter the results; Summarize the results and generate an analysis report.

[0038] The action engine in this application performs the following steps for executing third-party script-based actions (execute_script) to securely invoke external scripts and perform complex automated operations: Locate third-party scripts: Locate the script files (such as Shell or Python scripts) specified in the YAML rules from the predefined trusted script directory (safe_script_dir); Construct execution commands based on script parameters: Read the script parameters defined in the YAML rules and concatenate them into complete execution commands (such as passing in the matched file path and target value); Executing third-party scripts: Scripts are executed in an isolated environment, and the engine is responsible for capturing the script's output, return codes, and logs; Summarize the execution results and generate an analysis report.

[0039] The action engine in this application performs the following steps for target content modification / replacement actions (replace_target_member) to securely modify the target file content: Match the target_member file: Locate the target file that needs to be modified; Replace target content file by file: Modify file content in a temporary copy according to the replacement rules defined in the YAML rules (such as regular expression replacement, field value modification, template overriding) (without directly manipulating the original file); Summarize the modification results and generate an analysis report.

[0040] The core execution logic of the Action engine in this application, and its key technical features include: 1. Atomized repair ensures data consistency: All modifications are made in a separate temporary directory; even if a fix fails, the original test case package remains unchanged, ensuring system robustness.

[0041] 2. Safety execution mechanisms for three types of actions: 1) replace_target_member (content replacement): Precise positioning: Accurately matches files related to target_member; Batch replacement: Supports multiple sets of old→new mappings.

[0042] 2) filter_target_line (target line filtering) Intelligent association: Automatically associates files related to target_member; Dynamically generate cleanup commands: Generate sed deletion commands based on filter_contains; Safety append: Ensures commands are written to the end of the script without disrupting the original logic.

[0043] 3) execute_script (external script call) Third-party scripts: Locate the specified script in the secure directory and execute it in a secure environment; Parameter settings: Specify the script parameters to replace, such as tar_path.

[0044] In step 105, the failed test cases are aggregated and analyzed, and a classification statistical report and operation log are output.

[0045] In this embodiment of the application, based on the rule matching results, failed test cases are automatically classified into three categories: "automatically repaired", "real defects requiring manual intervention", and "unknown mode".

[0046] For failed test cases that cannot be automatically repaired, this application further classifies them according to recent additions and long-term legacy cases: newly added failed test cases are automatically associated with the most recent list of failed test cases, reducing intermediate environment losses and improving the efficiency of test case analysis and feedback. Finally, a visual report containing difference statistics, rule hit rate and analysts is generated. This feedback mechanism not only guides the maintenance of current failed test cases, but also provides data support for the continuous iteration of the rule base. The specific implementation process is as follows: (1) Result collection and incremental calculation: obtain test reports, calculate cross-environment common, single-environment unique and newly added failure sets; (2) Rule parsing and instantiation: load YAML rule base, instantiate Condition and Action objects; (3) Matching and execution: traverse newly added test cases, and perform multimodal matching through the Condition engine; for successfully matched test cases, call the Action engine in the temporary directory to perform repair; (4) Archiving and reporting: archive according to the execution results and output analysis reports.

[0047] Example 2 In this embodiment, an actual list of failed test cases is used to illustrate the automated failure test case processing flow of this application: Step 1: Failure test case screening mechanism based on incremental set operation: The set difference algorithm is used to analyze the failure test case list from multiple dimensions such as time, running platform and underlying dependency, extract the complete failure test case list, the recently added failure test case list, and the difference list of failure test cases caused by different platforms or underlying dependencies, and perform preliminary classification of failure test cases.

[0048] Figure 4 This is a list of all failed test cases in the embodiments of this application, such as Figure 4As shown, this list of failed tests is for March 20, 2026, and it contains a total of 454 failed tests.

[0049] Figure 5 This is a list of recently added failed test cases in the embodiments of this application, such as Figure 5 As shown, in this list of recently added failed test cases, 18 failed test cases were added on March 20, 2026.

[0050] Figure 6 This is a differentiated list of failure test cases caused by different platforms or underlying dependencies in the embodiments of this application, such as... Figure 6 As shown, there are 207 cross-regional data discrepancies as of March 20, 2026, where the Beijing environment contains entries but the Shanghai environment has entries that are missing or inconsistent. Additionally, the script stores a list of failed test cases common to all environments.

[0051] The combination of the three types of failure test case lists forms a complete closed loop for test result analysis: first look at the overall change volume, then look at cross-regional consistency, and finally locate the newly added failure issues.

[0052] Step 2: Structured External Rule Base and Dynamic Loading In this embodiment, an external rule base, rules.yaml, is customized according to actual needs. This rule base is independent of the automation script, can be customized according to actual needs, and is dynamically loaded during script runtime. Current customized rules include: absolute_path_cleanup: Cleans up absolute paths in failed test cases; arm_license_fix: Updates the software's license name in the comparison file; png_gold_fix: Extracts failed test cases that use images as comparison files; arm_double_case: Extracts special type failure test cases by test case name; custom_validation: Supports custom third-party scripts as rules; txtdiff_tolerancediff_replace: Replaces the specified file txtdiff (text difference comparison tool) in the failed test cases with tolerancediff (tolerance difference).

[0053] The current system uses a YAML-based configuration-driven architecture to load the rule base. Each independent rule in the base is defined as a structured entity containing four core attributes: a unique identifier (ID), an enable switch (Enable), a condition set (Condition Set), and an action set (Action Set).

[0054] In this embodiment, the check condition group and repair action group not only support modular reuse across rules, but are also composed of several atomic condition and action units. Through this combinatorial design pattern, users can flexibly arrange basic units to quickly construct complex composite check logic and customized repair strategies.

[0055] The following section will use the absolute_path_cleanup rule as an example to explain the definition specifications of the rule in detail: Figure 7 This is a schematic diagram of the absolute path in the cleanup failure test case of this application embodiment, such as... Figure 7 As shown in the embodiment of this application, the rule for cleaning up absolute paths in failed test cases can be applied to many other scenarios. By simply modifying a few conditions such as gold_mapping_member_pattern (matches all files with the .gold extension) and contains (the condition matches successfully when the .gold file contains these absolute path strings), the rule can be directly applied to other scenarios.

[0056] 1. The rule ID is absolute_path_cleanup, and each rule has a unique ID for identification; 2. Fill in the rule description information in the description field. This rule adds a command to delete absolute paths in the post-processing file of the test case. 3. The enabled switch is used to determine whether the rule needs to be enabled; 4. Condition is a group of check conditions for a rule: 1) Type indicates the type of check condition. The same type of check condition group can be used in different rules. The rule type `file_content_contains` indicates that the specified file in the test case needs to be checked for the existence of the specified content. 2) `Gold_mapping_member_pattern` and `member_pattern` are two conditional units of the same type. `member_pattern` directly matches the specified file. `gold_mapping_member_pattern` matches the `gold` file, which needs to be mapped to the result file generated after the test case is run. 3) After obtaining the files matched by member_pattern, it is necessary to find the content corresponding to contains; 4) During the inspection process, determine whether to ignore case when searching for strings based on case_sensitive.

[0057] 5. Action is a group of rule-based repair actions, and its structure is similar to that of condition: 1) When Type is filter_target_line, it means that a command to filter the specified content needs to be written into the specified file in the test case; 2) Match the specified file to be written by the target_member command; 3) Match the specified rows to be filtered using filter_contains; 4) `Gold_mapping_args_pattern` and `args_pattern` are action units of the same type, used to match the target files of filtering commands. The difference between them is the same as that between `Gold_mapping_member_pattern` and `member_pattern` above.

[0058] The following section will elaborate on the definition and specification of the arm_license_fix rule: Figure 8 This is a schematic diagram illustrating the rules for software license names in the updated prior art documents of this application, as shown below. Figure 8The image shows the `arm_license_fix` rule, whose core function is to dynamically modify the license of failed test cases running on the aarch64 platform. This rule also boasts high platform independence and scalability: by simply adjusting the `contains` field in the matching conditions and the `script` in the repair action for the target platform, this logic can be ported to other hardware platforms. The functions of `id`, `description`, and `enable` in `arm_license_fix` are consistent with those in the `absolute_path_cleanup` rule. The main differences lie in the `condition` and `action`. 1. `Condition` directly reuses `file_content_contains`. Only the internal conditions need to be changed. 2. `Action`: 1) `Type` is `execute_script`. This indicates that the specified test case is processed by running a third-party script. 2) `script` is the script name, supporting both Python and shell scripts. 3) `args` and `timeout` are the parameters passed to the script execution and the timeout setting. 4) `working_dir` is the working directory where the script runs. 5) `log_output` indicates that the log content of the third-party script is recorded in the log file. As can be seen from the above, external rule bases can be flexibly customized simply by combining different types of conditions and actions.

[0059] In this embodiment, the processing method of the failed test case matched by the arm_license_fix rule will be explained by the actual processing flow of a failed test case matched by the arm_license_fix rule: (1) The content of gold_mapping_member_pattern in this rule is "*.gold". In this test case, it will first match sleGuiLic.txt.gold, and then it needs to be mapped to the result file sleGuiLic.txt generated after the test case is run; (2) The content of contains is "_KP", which means that the content needs to be searched in the matched specified file. If the content exists in any specified file, the check will pass. In this case, it is necessary to check whether the string "_KP" exists in sleGuiLic.txt, such as Figure 16 The results shown are a comparison of test failures. (3) The action type is execute_script. This indicates that the specified test case is processed by running a third-party script; (4) The script in the script is addArmKP.sh. The function of this script is to add a piece of code to the post-processing script postOperate.sh in the test case. The function of this code is to check whether the current system is aarch64 after the test case is run. If it is, add the "_KP" suffix to all licenses in the gold file.

[0060] Step 3: Use the condition matching engine to match failed test cases. In this embodiment, after loading the rule base, the entire list of failed test cases is traversed, and the condition units within the condition engine are used to check the test cases. If each unit meets the condition, the test case is considered a matching object. The condition matching engine is composed of atomic condition detection units, which can be logically combined to construct new condition detection operators. The following will further describe the three atomic condition detection operators that have been designed so far: 1. File matching operator: such as Figure 9 As shown, filename_contains consists of three units: type, contains, and case_sensitive, with contains being the core unit. Contains supports list format.

[0061] In this embodiment of the application, the file matching operator is used to match files whose filenames contain the specified characteristic characters.

[0062] 2. File content feature matching operators: such as Figure 7 As stated in the condition `file_content_contains`, the file content feature matching operator requires configuration of specified content of a specified file, which is logically combined by two types of condition units: `member_pattern` and `contains`. First, `member_pattern` matches the feature file to be detected, and then `contains` specifies the feature characters to be detected.

[0063] 3. Sandboxed script probe operators: such as Figure 10 As shown, the `execute_script` function constructs a sandboxed script execution environment. The execution logic of its internal conditional units is highly similar to the action units in Rule 2, both adhering to a unified script-driven architecture. Specifically, this unit specifies the third-party executable program to be invoked through the `script` field, dynamically constructs a command-line input list using the `args` parameter, and implements execution time limit control through a `timeout` mechanism.

[0064] Step 4: The Atomic Repair Action Engine repairs failed test cases, ensuring data consistency.

[0065] In this embodiment, after the test case is matched with the rules, the corresponding repair action engine is started through the matched rule engine. When the repair action execution engine operates on a failed test case, it first copies the failed test case to a temporary workspace, and subsequent operations are performed on the copy in the temporary workspace. The path to the temporary workspace is also defined in rules.yaml: working_copy_dir: "auto_fixed_cases".

[0066] In this embodiment of the application, a variety of action operators are also implemented in the action execution engine: 1. Streaming text filtering: such as Figure 7 The repair operation `filter_target_line` is shown in the diagram. In this repair action, the file to which commands need to be inserted is obtained, the content and target file are filtered, a sed / awk command sequence is dynamically generated, and injected into the post-processing script; 2. Structured content replacement: such as... Figure 11 As shown, replace_target_member replaces objects under replacements in batches in the target file specified by target_member.

[0067] 3. External process calls: such as Figure 8 As shown in the `execute_script` action, the task is executed by calling an external script, and a timeout is used to prevent the process from hanging. Step 5: Multi-dimensional result classification and feedback loop.

[0068] In this embodiment, after automated processing of failed test cases is completed, the execution data is aggregated and analyzed from multiple dimensions, outputting a categorized statistical report and detailed operation logs. 1. For test cases that cannot be directly processed by automation strategies, the system automatically generates a categorized statistical report based on its functional module. This report aims to provide accurate task allocation criteria for subsequent manual intervention. The specific report structure and content are as follows: Figure 12 As shown.

[0069] 2. In addition to the basic classification based on functional modules mentioned above, this embodiment further implements an intelligent routing mechanism based on responsibility attribution. Specifically, for recently added test cases, the system automatically associates them with the most recent updater of the failed test case through version control metadata and triggers targeted emails to achieve precise task assignment. The final allocation results are output in a hierarchical directory structure, such as... Figure 13As shown. 1) Statistical summary layer: Generates author_file_counts.txt to quantify the load distribution of each analyst, and archives legacy test cases that have not been matched with a responsible person through unclassified_files.txt; 2) Time dimension subdivision layer: Establishes multi-level subdirectories based on the last update time of failed test cases (covering five time windows: "within one week", "within one month", "within three months", "within one year" and "more than one year"). In these subdirectories, the system aggregates specific test case files by responsible person, thereby realizing a closed-loop management of the entire chain from macro load statistics to micro task tracing. 3. For test cases that have been successfully processed automatically, in order to build a feedback loop for continuous optimization, the system performs fine-grained classification based on the repair rules they have hit. The classification results are as follows. Figure 14 As shown.

[0070] 4. Finally, the system automatically summarizes the execution metadata of all stages and outputs a comprehensive operation log. This log, indexed by time series, details the processing path of each test case, rule hit details, exception stack information, and final allocation results, such as... Figure 15 As shown.

[0071] Example 3 In embodiments of this application, an electronic device is also provided. Figure 17 This is a schematic diagram of the electronic device structure according to an embodiment of this application, such as... Figure 17 As shown, the electronic device of this application includes a processor 1701 and a memory 1702, wherein, The memory 1702 stores a computer program, which, when read and executed by the processor 1701, performs the steps described above in the embodiment of the automated failure test case processing method.

[0072] Example 4 In the embodiments of this application, a computer-readable storage medium is also provided, which stores a computer program, wherein the computer program is configured to execute the steps in the embodiments of the automated failure test processing method described above when it runs.

[0073] In this embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0074] It will be understood by those skilled in the art that the above descriptions are merely preferred embodiments of this application and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An automated method for handling failed test cases, applied to automated testing and maintenance in EDA software development, comprising the following steps: The collected list of failed test cases is categorized using set computation and incremental analysis; Structure the external rule base and load it dynamically; Based on the external rule base, a condition matching engine is used to perform multimodal rule matching on failed test cases; Repairing successfully matched test cases is performed using an atomic repair action execution engine; Perform aggregate analysis on failed test cases and output a classification statistical report and operation log.

2. The automated test case processing method according to claim 1, characterized in that, The step of classifying the collected list of failed test cases through set computation and incremental analysis further includes: By accessing emails and specified file directories, a list of failed test cases is collected from multiple dimensions, including time, operating platform, and underlying dependencies. The set difference algorithm is used to perform set calculation and incremental analysis on the list of failed test cases to obtain the complete list of failed test cases, the list of recently added failed test cases, and the list of differences in failed test cases caused by different platforms or underlying dependencies.

3. The automated test case processing method according to claim 1, characterized in that, The structured external rule base also includes: A structured external YAMLS rule base, comprising: A core set of rules comprising multiple independent rules; A secure script directory for storing verified and trusted script files; This directory is used to store temporary files and working copies of backup data during rule execution.

4. The automated test case processing method according to claim 3, characterized in that, Each of the independent rules includes: An identifier is a unique identifier for a rule, used for quick location and reference within the rule base. An enable switch is used to control whether the independent rule is effective; The condition group is used for the judgment logic of rule triggering, including the condition type and the target object of the condition. Repair action groups are used to perform operations when conditions are met, including the action type and the target object of the action.

5. The automated test case processing method according to claim 1, characterized in that, The step of structuring and dynamically loading the external rule base also includes: The external rule base is structured according to the following customized rules: Clean up absolute paths in failed test cases; Update the software license name in the comparison file; Extract failed test cases that use images as comparison files; It supports custom third-party scripts as rules; Replace the text difference comparison tool for the specified file in the failed test cases with tolerance difference.

6. The automated test case processing method according to claim 1, characterized in that, The step of performing multimodal rule matching on failed test cases using a condition matching engine based on the external rule base further includes: traversing the entire list of failed test cases, checking the failed test cases using atomic condition detection units within the condition matching engine, and if each condition detection unit meets the condition of a certain failed test case, then the failed test case is regarded as a matching object.

7. The automated test case processing method according to claim 6, characterized in that, The atomized conditional detection unit constructs an atomized conditional detection operator through logical combination; The atomized conditional detection operator includes: a file matching operator, used to match files whose filenames contain action-specified feature characters; File content feature matching operator, used to configure specified content of specified files; Sandboxed script probe operator, used to construct a sandboxed script execution environment.

8. The automated test case processing method according to claim 6, characterized in that, The step of repairing successfully matched use cases based on the atomic repair action execution engine also includes: Read the action type in the YAML rule to determine the remediation action to be performed. Create isolated temporary directories for all actions performed; For content filtering / querying actions, perform the following steps to find and filter specific content in the target file: Locate the target file specified in the YAML rule and match the target_member file; Search the file for content that matches the rules and conditions, and match the target content. Match the target parameter of the command; or, To execute third-party scripts, follow these steps to securely call external scripts and perform complex automated operations: Find third-party scripts: Locate the script file specified in the YAML rule from the predefined trusted script directory; Construct execution commands based on script parameters: Read the script parameters defined in the YAML rules and concatenate them into complete execution commands; Execute third-party scripts: Execute the script in an isolated environment, capturing the script's output, return codes, and logs; or, For actions involving modification / replacement of target content, perform the following steps to securely modify the target file content: Match the target_member file: Locate the target file that needs to be modified; Replace target content file by file: Modify file content in a temporary copy according to the replacement rules defined in the YAML rules.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor is configured to execute a computer program stored in the memory to implement the automated test case processing method according to any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that, The computer program is loaded and executed by a processor to implement the automated test case processing method according to any one of claims 1-8.

Images