An intelligent test case generation method

Abstract

The application discloses an intelligent test case generation method, which comprises the following steps: obtaining test requirements input by a user; receiving a requirement analysis trigger operation sent by the user; after receiving the requirement analysis trigger operation, showing a selectable configuration interface to the user; generating a model calling prompt word according to the selection result of the user in the selectable configuration interface, and showing an abstract of the model calling prompt word; and calling a large language model to generate a candidate test case based on the model calling prompt word. The application can automatically check and update the candidate test case generated by the large language model, improve the standardization, completeness and reliability of the test case, and further preferentially present more important test cases through a sorting mechanism to improve the rationality of test resource allocation and test execution arrangement.

Description

Technical Field

[0001] This invention relates to the field of large language model application technology, and in particular to an intelligent test case generation method. Background Technology

[0002] With the development of artificial intelligence technology, text understanding and content generation capabilities based on large language models are increasingly being applied to the field of software testing. Testers can input requirements documents, interface specifications, business rules, or user stories into the large language model, which will automatically understand the object under test and generate corresponding test cases. This reduces the workload of manually writing test cases and improves test design efficiency. Compared to the traditional method of writing test cases based on human experience, the test case generation method based on large language models can output a large number of candidate test scenarios, test steps, test data, and expected results in a short time. Therefore, it has high application value in automated testing, interface testing, regression testing, and complex system testing.

[0003] In the aviation field, software systems typically involve a wide range of complex information, including flight phases, operational modes, sensor signals, interface data, fault states, redundant channels, safety constraints, and state transitions. Aviation software test cases need to cover not only general functional scenarios but also scenarios involving abnormal inputs, boundary conditions, mode switching, fault injection, primary / backup channel switching, alarm outputs, and safety-related expected results. Therefore, introducing large language models to assist in test case generation in the aviation field can help testers quickly extract test points from complex requirements and generate preliminary test case sets, thereby improving the efficiency of aviation software test design.

[0004] However, existing test case generation methods based on large language models typically rely on users directly writing prompts or the system using fixed prompt templates to call the large language model. For testers in the aviation field, requiring them to manually write complete prompts not only necessitates understanding test requirements, domain constraints, and the interaction methods of the large language model, but also requires repeated adjustments to the prompts to obtain test cases that meet expectations, resulting in high operational costs. On the other hand, if the system directly displays long prompt templates to users, they need to read, understand, and modify a large amount of prompt content, which can easily increase the user's burden; if the system automatically generates complex prompts without user confirmation, it may lead to a discrepancy between the generated goals and the user's true intentions.

[0005] Furthermore, test cases generated by large language models are typically candidate results, which may contain issues such as missing fields, non-standard formatting, insufficient coverage, and unclear expected results. In the aviation field, more domain-specific problems may arise, such as generating non-existent interface fields, flight phases, operating modes, or state transition relationships; incorrect use of units or thresholds; omission of redundant channels or fault injection scenarios; or generating expected results inconsistent with requirement constraints. If the system lacks an automatic verification mechanism for the generated results, testers need to manually check each candidate test case, which is not only inefficient but also prone to overlooking hidden logical defects or domain inconsistencies, thus affecting the reliability and traceability of the test cases.

[0006] Furthermore, after a large language model generates a large number of candidate test cases at once, existing methods often struggle to automatically determine the execution priority of different test cases. In the aviation field, different test cases may exhibit significant differences in terms of safety impact, operational phase, interface state coupling, and contribution to coverage gaps. Without a rule-based sorting mechanism, testers must rely on manual experience to filter and sort a large number of test cases, which can easily lead to high-risk scenarios, critical operational phase scenarios, or important coverage gap scenarios not being prioritized, thus reducing the efficiency of test resource allocation.

[0007] Therefore, existing technologies still need an intelligent test case generation method that can reduce the burden of writing prompt words for users, dynamically generate model-call prompt words based on user selections, automatically verify candidate test cases generated by large language models, and sort test cases based on preset rules. Summary of the Invention

[0008] This invention provides an intelligent test case generation method. Its main technical means include: acquiring user-inputted test requirements; upon receiving a user-triggered requirement analysis operation, displaying an optional configuration interface to the user, including requirement parsing items, generation target items, constraint coverage items, and supplementary instruction items; generating model call prompts based on the user's selections in the optional configuration interface, and displaying a summary of the model call prompts; subsequently, calling a large language model to generate candidate test cases based on the model call prompts. Further, this invention uses a preset set of validators to validate the candidate test cases and updates them based on the validation results; the update may include automatic correction, secondary generation based on supplementary prompts, or user confirmation processing. For candidate test cases that pass validation and updated candidate test cases, the system can also calculate ranking scores based on security influencing factors, operational phase factors, interface state coupling factors, and coverage gap contribution factors, and rank them based on the ranking scores. With the above solution, users can express their test case generation intent by clicking on options without having to directly write or maintain complete prompts, reducing the difficulty of writing prompts. At the same time, the system can automatically verify and update candidate test cases generated by the large language model, improving the standardization, completeness, and reliability of test cases. Furthermore, the system prioritizes more important test cases through a sorting mechanism, improving the rationality of test resource allocation and test execution arrangements.

[0009] This invention provides a method for generating intelligent test cases, characterized in that the method includes:

[0010] Test requirements for obtaining user input;

[0011] Receive user-triggered requests for analysis.

[0012] After receiving the requirement analysis trigger operation, the user is shown an optional configuration interface, which includes requirement parsing items, generation target items, constraint coverage items, and supplementary instruction items.

[0013] Generate model call suggestions based on the user's selections in the optional configuration interface, and display a summary of the model call suggestions;

[0014] The large language model is invoked to generate candidate test cases based on model invocation prompts.

[0015] In a preferred embodiment, the method further includes:

[0016] The candidate test cases are validated using a pre-defined set of validators;

[0017] Update the candidate test cases based on the verification results;

[0018] The candidate test cases that have passed the verification and the updated candidate test cases are sorted according to preset rules.

[0019] In a preferred embodiment, sorting the candidate test cases that have passed verification and the updated candidate test cases based on preset rules includes:

[0020] The ranking scores of candidate test cases that pass the verification and the updated candidate test cases are calculated based on security impact factors, operational phase factors, interface state coupling factors, and coverage gap contribution factors.

[0021] Based on the ranking score, the candidate test cases that have passed the verification and the updated candidate test cases are ranked.

[0022] In a preferred embodiment, updating candidate test cases based on the verification results includes:

[0023] Based on the verification results, candidate test cases are modified, regenerated, or subject to user confirmation.

[0024] Specifically, when the verification result indicates that the candidate test case has missing format or missing fields, the candidate test case is automatically corrected.

[0025] When the verification result indicates that there is a coverage gap in the candidate test case, supplementary prompt words are generated based on the coverage gap item and the large language model is called to regenerate the candidate test case.

[0026] When the verification result indicates that there is content that cannot be determined in the candidate test cases, the candidate test cases and corresponding issue items are displayed to the user for confirmation.

[0027] In a preferred embodiment, the preset validator set includes at least one of the following: requirement identifier validator, interface field validator, status validator, stage validator, state transition validator, unit dimension validator, threshold boundary validator, expected result validator, integrity validator, and traceability relationship validator.

[0028] This invention provides an intelligent test case generation system, characterized in that the system includes:

[0029] processor;

[0030] Memory, coupled to the processor, stores processor-executable instructions that, when executed by the processor, cause the system to perform the following operations:

[0031] Test requirements for obtaining user input;

[0032] Receive user-triggered requests for analysis.

[0033] After receiving the requirement analysis trigger operation, the user is shown an optional configuration interface, which includes requirement parsing items, generation target items, constraint coverage items, and supplementary instruction items.

[0034] Generate model call suggestions based on the user's selections in the optional configuration interface, and display a summary of the model call suggestions;

[0035] The large language model is invoked to generate candidate test cases based on model invocation prompts.

[0036] In a preferred embodiment, the instructions also cause the system to perform the following operations when executed by the processor:

[0037] The candidate test cases are validated using a pre-defined set of validators;

[0038] Update the candidate test cases based on the verification results;

[0039] The candidate test cases that have passed the verification and the updated candidate test cases are sorted according to preset rules.

[0040] In a preferred embodiment, sorting the candidate test cases that have passed verification and the updated candidate test cases based on preset rules includes:

[0041] The ranking scores of candidate test cases that pass the verification and the updated candidate test cases are calculated based on security impact factors, operational phase factors, interface state coupling factors, and coverage gap contribution factors.

[0042] Based on the ranking score, the candidate test cases that have passed the verification and the updated candidate test cases are ranked.

[0043] In a preferred embodiment, updating candidate test cases based on the verification results includes:

[0044] Based on the verification results, candidate test cases are modified, regenerated, or subject to user confirmation.

[0045] Specifically, when the verification result indicates that the candidate test case has missing format or missing fields, the candidate test case is automatically corrected.

[0046] When the verification result indicates that there is a coverage gap in the candidate test case, supplementary prompt words are generated based on the coverage gap item and the large language model is called to regenerate the candidate test case.

[0047] When the verification result indicates that there is content that cannot be determined in the candidate test cases, the candidate test cases and corresponding issue items are displayed to the user for confirmation.

[0048] In a preferred embodiment, the preset validator set includes at least one of the following: requirement identifier validator, interface field validator, status validator, stage validator, state transition validator, unit dimension validator, threshold boundary validator, expected result validator, integrity validator, and traceability relationship validator.

[0049] Compared with existing technologies, the present invention has the following advantages: Through the above solution, users do not need to directly write or maintain complete prompt words, but can express the intention of test case generation by clicking, which reduces the difficulty of writing prompt words; at the same time, the system can automatically verify and update the candidate test cases generated by the large language model, improving the standardization, completeness and reliability of test cases; furthermore, the ranking mechanism prioritizes the presentation of more important test cases, improving the rationality of test resource allocation and test execution arrangement. Attached Figure Description

[0050] Figure 1 This is a system logic block diagram of one embodiment of the present invention.

[0051] Figure 2 This is a flowchart of a method according to an embodiment of the present invention.

[0052] Figure 3 This is a schematic diagram of the system interface of an embodiment of the present invention.

[0053] Figure 4 This is a schematic diagram of the system interface of another embodiment of the present invention.

[0054] Figure 5 This is a schematic diagram of the system interface of another embodiment of the present invention. Detailed Implementation

[0055] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but it should be understood that the scope of protection of the present invention is not limited to the specific embodiments.

[0056] Figure 1 A logical block diagram of an intelligent test case generation system provided in an embodiment of the present invention is shown. Figure 1 As shown, the intelligent test case generation system may include a requirement acquisition module 105, a trigger receiving module 110, a configuration interface display module 115, a prompt word generation module 120, a summary display module 125, a test case generation module 130, a verification module 135, an update processing module 140, and a sorting module 145.

[0057] The requirement acquisition module 105 is used to acquire test requirements input by the user. The trigger receiving module 110 is used to receive requirement analysis trigger operations sent by the user. The configuration interface display module 115 is used to display an optional configuration interface to the user after receiving the requirement analysis trigger operation, wherein the optional configuration interface includes requirement parsing items, generation target items, constraint coverage items, and supplementary instruction items. The prompt word generation module 120 is used to generate model call prompt words based on the user's selection results in the optional configuration interface. The summary display module 125 is used to display a summary of the model call prompt words. The test case generation module 130 is used to call a large language model to generate candidate test cases based on the model call prompt words. The validation module 135 is used to validate the candidate test cases using a preset set of validators. The update processing module 140 is used to update the candidate test cases based on the validation results, wherein the update may include correction, secondary generation, or user confirmation processing. The sorting module 145 is used to sort the candidate test cases that have passed the validation and the updated candidate test cases based on preset rules.

[0058] In some embodiments, the verification module 135 may include at least one of a requirement identifier verifier, an interface field verifier, a state verifier, a stage verifier, a state transition verifier, a unit dimension verifier, a threshold boundary verifier, an expected result verifier, an integrity verifier, and a traceability relationship verifier. The ranking module 145 may calculate ranking scores for candidate test cases based on security influencing factors, operational stage factors, interface state coupling factors, and coverage gap contribution factors, and rank the candidate test cases based on the ranking scores.

[0059] It should be noted that the above modules can be implemented as software modules, hardware modules, firmware modules, or any combination of software, hardware, and firmware. For example, the above modules can be implemented by a processor executing computer program instructions stored in memory, or by application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), controllers, microprocessors, or other programmable logic devices. The above module division is only used to illustrate the functional logic of the present invention and does not constitute a limitation on the actual physical structure. In actual implementation, multiple modules can be integrated into one module, and a module can be divided into multiple sub-modules. Data interaction between modules can be achieved through buses, interfaces, networks, shared memory areas, or message queues, etc.

[0060] This invention provides a method for generating intelligent test cases, characterized in that the method includes:

[0061] Step 1: Obtain the test requirements input by the user; In a specific embodiment, the intelligent test case generation method of the present invention is illustrated using the test requirements of the flight crew alarm function as an example. This test requirement originates from the relevant requirements for flight crew alarms in transport aircraft. In this embodiment, for ease of system processing, this test requirement is instantiated in the project requirement library as requirement identifier REQ-FCA-001, the content of which is: When the aircraft monitoring system detects an abnormal system state that requires the flight crew to immediately perceive and respond to, the flight crew alarm function should generate a warning-level alarm; this alarm should enable the flight crew to identify the corresponding abnormal state and determine the appropriate handling action; when multiple alarms exist simultaneously, the warning-level alarm should have a higher display priority than the attention-level alarm and the advisory-level alarm; when the conditions for triggering the alarm no longer exist, the alarm should be removed. This test requirement will be referred to as the target test requirement below.

[0062] Step 2: Receive the requirement analysis trigger action sent by the user; a screenshot of the user-triggered requirement analysis interface can be found here. Figure 3 .like Figure 3 As shown, users can enter their test requirements for "flight crew alarm function" in the input box corresponding to the test requirements. If users want the system to provide further requirements analysis, they can click the "Execute Requirements Analysis" button with their mouse.

[0063] Step 3: After receiving the requirement analysis trigger operation, display the optional configuration interface to the user. The optional configuration interface includes requirement parsing items, generation target items, constraint coverage items, and supplementary instruction items. The requirement parsing items are used to display the structured content identified by the system from the target test requirements. In this embodiment, the requirement parsing items include: requirement identifier REQ-FCA-001; the test object is the flight crew alarm function; the functional points include warning level alarm generation, alarm priority handling, alarm information display, and alarm removal; input parameters include alarm condition identifier, alarm category, alarm condition validity flag, flight phase, alarm priority, and alarm confirmation command; output results include visual alarm indication, audible alarm indication, alarm message, alarm status, and alarm removal status; preconditions include the flight crew alarm function being in working condition, the alarm definition table being loaded, and the input interface communication being normal; status conditions include no alarm status, alarm unconfirmed status, alarm confirmed status, and alarm removal status; constraint rules include warning level alarms taking precedence over attention level alarms and advisory level alarms, removing the corresponding alarm when the alarm condition disappears, and the generated alarm fields must originate from the interface definition or alarm definition table. An example of displaying the requirement parsing items to the user in the optional configuration interface can be found in [link to configuration]. Figure 4 Due to the limitations of the display interface area, in this invention, the requirement analysis items, generation target items, constraint coverage items, and supplementary instruction items can be sequentially displayed to the user on the monitor. Figure 4In the middle, under the function points, users can select the function points they want to include by clicking with the mouse. For example, if a user selects two functions, alarm priority handling and alarm information display, the large language model will then generate test cases for "alarm priority handling" and "alarm information display".

[0064] The generated target items allow users to select the type of test cases to be generated. In this embodiment, functional testing refers to verifying whether the alarm generation function specified in the target test requirements is implemented; boundary testing refers to verifying the processing results when the alarm category, alarm priority, or alarm condition validity flag is at a critical change; anomaly testing refers to verifying whether the system produces the expected processing results under conditions such as missing input fields, illegal alarm categories, and abnormal alarm condition jumps; state transition testing refers to verifying whether the transition between the no-alarm state, the alarm unacknowledged state, the alarm acknowledged state, and the alarm removed state is correct; interface testing refers to verifying whether the field values, field transmission, and output results in the alarm input interface and alarm output interface conform to the definition; fault injection testing refers to injecting alarm conditions, interface anomalies, or alarm state anomalies through simulation to verify the response of the alarm function; regression testing refers to re-executing the test cases related to REQ-FCA-001 after the alarm function is modified; and automated script generation refers to converting test steps into script drafts that can be executed by test benches, simulation environments, or automated testing frameworks. The above test types are used as engineering classifications of system generation targets in this embodiment and are not limited to the aviation field. The UI design for generating target items can be referenced. Figure 4 The present invention will not be described in detail here.

[0065] Constraint coverage items are used to restrict the conditions that must be followed when generating test cases for a large language model. In this embodiment, interface field constraints mean that the generated input and output fields must belong to fields that already exist in the interface definition or alarm definition table, such as AlertConditionID, AlertCategory, ConditionActive, FlightPhase, VisualCue, AuralCue, AlertMessage, and AlertState. State constraints mean that the generated test cases can only use defined states such as no alarm, unacknowledged alarm, acknowledged alarm, and removed alarm. Stage constraints mean that the flight stage used by the test case should belong to the project's flight stage set, such as ground, takeoff, climb, cruise, descent, approach, and landing. Unit dimension constraints mean that when time thresholds, priority values, or count values ​​are involved, the units or dimensions defined by the project should be used, such as milliseconds, seconds, or dimensionless enumeration values. Threshold boundary constraints mean that when there is a time threshold, priority threshold, or trigger count threshold in the target test requirements or project data, the system should generate test data equal to, below, and above the threshold. Redundancy channel constraints mean that when the test object has primary and backup display channels, primary and backup alarm calculation channels, or redundant input sources, it should cover scenarios such as primary channel validity, backup channel validity, primary channel failure, and inconsistency between primary and backup channels. Expected result constraints mean that the expected result of the test case must be deducible from the target test requirements, interface definitions, alarm definition tables, or state transition tables. Traceability constraints mean that test cases should be associated with corresponding requirement identifiers, coverage items, and verification criteria. The UI design for constraint coverage items can be referenced. Figure 4 The present invention will not be described in detail here.

[0066] The supplementary instruction field is used to receive additional requirements from user input. In this embodiment, the user can input "Prioritize generating state transition use cases for warning-level alarms, and cover scenarios where multiple alarms exist simultaneously".

[0067] Step 4: Generate model call prompts based on the user's selections in the optional configuration interface and display a summary of the model call prompts. The system generates model call prompts based on the user's selections in the optional configuration interface and displays a summary of the model call prompts to the user. This summary may include the following: This test will generate flight crew alarm function test cases based on REQ-FCA-001; the generation objectives include functional testing, state transition testing, interface testing, fault injection testing, and regression testing; the generation process is limited to using identified interface fields, state sets, and flight phase sets; output fields include test case number, associated requirements, test objectives, preconditions, input data, execution steps, expected results, and coverage items; after generation, the system will verify the requirement identifier, interface fields, states, flight phases, state transitions, unit dimensions, threshold boundaries, expected results, completeness, and traceability relationships. The system generates complete model call prompts in the background, which may include more detailed prohibitions and format constraints, such as prohibiting the generation of alarm types not appearing in the alarm definition table, prohibiting the use of undefined interface fields, and prohibiting placing consultative alarms before warning alarms, etc. The full model call prompt is not displayed to the user by default; it is only shown when the user triggers the action to view the full prompt. A screenshot of the interface displaying the summary of the model call prompt can be found here. Figure 5 ,exist Figure 5 In the system, the "Model Invocation Prompt Summary" section only displays a summary of the model invocation prompts to the user. If the user clicks the "Edit Prompt" option, the system can display the complete prompts to the user, and the user can customize the prompts in this interface.

[0068] Step 5: Use the large language model to generate candidate test cases based on the model call prompts.

[0069] In a preferred embodiment, the method further includes:

[0070] The candidate test cases are validated using a pre-defined set of validators;

[0071] Update the candidate test cases based on the verification results;

[0072] The candidate test cases that have passed the verification and the updated candidate test cases are sorted according to preset rules.

[0073] In a preferred embodiment, sorting the candidate test cases that have passed verification and the updated candidate test cases based on preset rules includes:

[0074] The ranking scores of candidate test cases that pass the verification and the updated candidate test cases are calculated based on security impact factors, operational phase factors, interface state coupling factors, and coverage gap contribution factors.

[0075] Based on the ranking score, the candidate test cases that have passed the verification and the updated candidate test cases are ranked.

[0076] In a preferred embodiment, updating candidate test cases based on the verification results includes:

[0077] Based on the verification results, candidate test cases are modified, regenerated, or subject to user confirmation.

[0078] Specifically, when the verification result indicates that the candidate test case has missing format or missing fields, the candidate test case is automatically corrected.

[0079] When the verification result indicates that there is a coverage gap in the candidate test case, supplementary prompt words are generated based on the coverage gap item and the large language model is called to regenerate the candidate test case.

[0080] When the verification result indicates that there is content that cannot be determined in the candidate test cases, the candidate test cases and corresponding issue items are displayed to the user for confirmation.

[0081] The first candidate test case is TC-FCA-001, with the test objective of verifying the generation of warning-level alarms. Its main contents are as follows: Preconditions include the flight crew's alarm function being operational, the existence of ENG_FIRE_L in the alarm definition table, and the alarm category being Warning; input data includes AlertConditionID=ENG_FIRE_L, AlertCategory=Warning, ConditionActive=true, and FlightPhase=Takeoff; execution steps involve injecting the above input data into the alarm input interface and collecting alarm output; the expected result is the generation of both visual and audible alarms, with the alarm status being unacknowledged. After being checked by the integrity checker, this candidate test case was determined to have missing fields because its expected result did not include the AlertMessage field, making it impossible to verify whether the flight crew could identify abnormal states. Accordingly, the system automatically corrected the candidate test case, adding "AlertMessage displays the alarm message corresponding to ENG_FIRE_L" to the expected result, and retaining both versions before and after the correction.

[0082] The second candidate test case is TC-FCA-002, whose objective is to verify the alarm priority when multiple alarms exist simultaneously. Its main content is: simultaneously injecting a Warning category alarm and a Caution category alarm, and checking whether the Warning category alarm is displayed first. This candidate test case itself can be verified through requirement identifier validation, interface field validation, and status validation. However, the system found in the coverage matrix that the coverage item "remove the corresponding alarm when the alarm condition no longer exists" in the target test requirement has not been covered by any test cases. Therefore, the validation result indicates a coverage gap. The system generates a supplementary prompt based on this coverage gap and re-invokes the large language model to generate a supplementary test case TC-FCA-002A. This supplementary test case is used to verify that when ENG_FIRE_L's ConditionActive changes from true to false, the corresponding Warning alarm should be removed. The system saves the original candidate test case, supplementary prompt, supplementary test case, and coverage gap record.

[0083] The third candidate test case is TC-FCA-003, whose test objective is to verify the attention acquisition prompts for warning-level alarms. Its main content is: input a Warning category alarm, with expected results including VisualCue=true, AuralCue=true, and TactileCue=true. After checking with the interface field validator, the system found that the interface definition in this embodiment includes VisualCue and AuralCue, but does not include the TactileCue field. Since flight crew alarm requirements allow attention acquisition prompts to be formed through combinations of different senses such as sight, hearing, or touch, and whether the interface in this project supports tactile prompts needs confirmation from project personnel, the system does not directly delete this field. Instead, it displays this candidate test case and issue item to the user for confirmation. The user can choose to delete the TactileCue field, mark it as a newly added interface field for the project, retain the original test case, or mark the test case as pending further confirmation. If the user chooses to delete the TactileCue field, the system generates an updated TC-FCA-003 and retains both the original and updated versions.

[0084] After candidate test cases pass verification or are updated, the system sorts them based on preset rules. In this embodiment, the sorting score is determined by security impact factors, operational phase factors, interface state coupling factors, and coverage gap contribution factors, with each factor ranging from 0 to 100 points.

[0085] The security impact factor indicates the alarm category or security importance of the requirement involved in the test case. In this embodiment, if the test case involves a Warning alarm, the security impact factor is scored as 100 points; if it involves a Caution alarm, it is scored as 70 points; if it involves an Advisory alarm, it is scored as 40 points; and if it does not involve any security-related alarms, it is scored as 10 points. If a test case involves multiple alarm categories, the highest value is taken.

[0086] The operational phase factor indicates the importance of the flight phases covered by the test cases to operational safety and system verification. In this embodiment, takeoff, approach, and landing phases are scored out of 100, climb and descent phases out of 80, cruise phase out of 60, and ground phase out of 40. If a test case involves multiple flight phases, the highest score is used.

[0087] Interface state coupling factors represent the number of interface fields, the number of states, and the complexity of alarm interactions involved in the test case. In this embodiment, if a test case involves no more than 2 interface fields and only 1 state, it scores 20 points; if it involves 3 to 5 interface fields or 2 states, it scores 50 points; if it involves 6 to 8 interface fields or 3 states, it scores 80 points; and if it involves more than 8 interface fields, more than 3 states, or multiple alarm concurrent interactions, it scores 100 points.

[0088] The coverage gap contribution factor represents the supplementary value of a test case to currently uncovered items. In this embodiment, if a test case does not cover the new coverage item, it receives 0 points; if the newly added coverage item accounts for no more than 30% of the test case's coverage items, it receives 40 points; if it exceeds 30% but does not exceed 70%, it receives 70 points; and if it exceeds 70%, it receives 100 points. Coverage items may include alarm generation, alarm priority, alarm removal, state transition, interface fields, flight phases, abnormal inputs, and tracing relationships, etc.

[0089] In this embodiment, the ranking score is calculated using a weighted method, with security influencing factors accounting for 35%, operational phase factors accounting for 20%, interface state coupling factors accounting for 20%, and coverage gap contribution factors accounting for 25%. This weighting is only an example and can be adjusted according to the testing strategy in actual projects.

[0090] For example, the updated TC-FCA-001 is a test case that passed verification. Its content verifies the triggering of a Warning alarm by ENG_FIRE_L during the takeoff phase, and verifies VisualCue, AuralCue, AlertMessage, and AlertState. This test case involves Warning alarms, therefore its safety impact factor is 100; it involves the takeoff phase, therefore its operational phase factor is 100; it involves eight fields including AlertConditionID, AlertCategory, ConditionActive, FlightPhase, VisualCue, AuralCue, AlertMessage, and AlertState, and involves both no-alarm and unacknowledged-alarm states, therefore its interface state coupling factor is 80; it covers newly added coverage items such as alarm generation, attention acquisition prompts, and alarm message display, therefore its coverage gap contribution factor is 100. The system calculates its ranking score to be 96.

[0091] The updated TC-FCA-002A is a verified test case. Its content verifies that when the warning alarm trigger condition changes from present to absent during the cruise phase, the system removes the corresponding alarm. This test case involves warning alarms, therefore its security impact factor is 100; it involves the cruise phase, therefore its operational phase factor is 60; it involves fields such as ConditionActive, AlertState, and AlertRemoved, and involves the transition from an unacknowledged alarm state or an acknowledged alarm state to an alarm removal state, therefore its interface state coupling factor is 60; it supplements previously uncovered alarm removal coverage items, therefore its coverage gap contribution factor is 70. The system calculates its ranking score to be 76.5 points.

[0092] TC-FCA-003, confirmed by the user, is a valid test case. It verifies that during the approach phase, when both Warning and Caution alarms are present, Warning alarms are displayed first, and VisualCue and AuralCue are used for attention acquisition prompts. This test case involves Warning alarms, therefore its safety impact factor is 100; it involves the approach phase, therefore its operational phase factor is 100; it involves multiple alarm inputs, multiple alarm categories, display priorities, visual cues, and auditory cues, and involves multiple alarm states, therefore its interface state coupling factor is 100; it covers two coverage items: multiple alarm concurrency and priority handling, with some coverage items being newly added, therefore its coverage gap contribution factor is 70. The system calculates its ranking score to be 92.5 points.

[0093] Based on the above ranking scores, the system ranks the three test cases that have passed verification or updates. The ranking results are as follows: TC-FCA-001 is ranked first with a score of 96; TC-FCA-003 is ranked second with a score of 92.5; and TC-FCA-002A is ranked third with a score of 76.5. The system can display a list of test cases to the user according to this ranking result, allowing the user to prioritize the review and execution of test cases with higher security impact, more critical operational phases, higher interface state coupling, or greater contribution to coverage gaps.

[0094] In a preferred embodiment, the preset validator set includes at least one of the following: a requirement identifier validator, an interface field validator, a state validator, a phase validator, a state transition validator, a unit dimension validator, a threshold boundary validator, an expected result validator, an integrity validator, and a traceability relationship validator. In this embodiment, the preset validator set may include the following validators: The requirement identifier validator checks whether the requirement identifier in the test case exists in the requirement library; for example, if a test case references REQ-FCA-999, but this identifier does not exist in the requirement library, the requirement identifier validation will fail. The interface field validator checks whether the input and output fields in the test case exist in the interface definition or data dictionary; for example, if a TactileCue field is generated but this field does not exist in the interface definition, the interface field validation will fail directly. The state validator checks whether the state in the test case belongs to the state set; for example, if an "automatic recovery normal alarm state" is generated, but this state does not exist in the state set, the state validation will fail. The phase validator checks whether the flight phase in the test case belongs to the project's flight phase set; for example, if a "supersonic cruise phase" is generated, but this phase is not defined in the project, the phase validation will fail. The state transition validator checks whether the preceding state, triggering condition, and target state conform to the state transition table; for example, directly transitioning from the alarm unacknowledged state to the alarm removal state while the alarm condition is still valid will fail the state transition validation. The unit unit validator checks whether the numerical value and unit match; for example, writing the alarm removal delay as "100 meters" or writing the millisecond threshold as the second threshold will fail the unit unit validator. The threshold boundary validator checks whether boundary test data is generated around the project threshold; for example, if the project specifies an alarm output delay threshold of 500 milliseconds, but the test cases do not cover inputs or observations near 500 milliseconds, the threshold boundary validator will fail. The expected result validator checks whether the expected result can be derived from the requirements, interface definition, or alarm definition table; for example, if the AlertCategory is Warning but the expected result is written as Advisory with priority, the expected result validator will fail. The integrity validator checks whether test cases contain necessary fields such as test case number, associated requirements, test objectives, preconditions, input data, execution steps, expected results, and coverage items; for example, if expected results or input data are missing, the integrity check will fail. The traceability validator checks whether a relationship is established between the test case and the requirements, coverage items, and validation criteria; for example, if a test case is not associated with REQ-FCA-001 or does not specify which aspect of alarm generation, priority handling, or alarm removal it covers, the traceability validator will fail.

[0095] This invention provides an intelligent test case generation system, characterized in that the system includes:

[0096] processor;

[0097] Memory, coupled to the processor, stores processor-executable instructions that, when executed by the processor, cause the system to perform the following operations:

[0098] Test requirements for obtaining user input;

[0099] Receive user-triggered requests for analysis.

[0100] After receiving the requirement analysis trigger operation, the user is shown an optional configuration interface, which includes requirement parsing items, generation target items, constraint coverage items, and supplementary instruction items.

[0101] Generate model call suggestions based on the user's selections in the optional configuration interface, and display a summary of the model call suggestions;

[0102] The large language model is invoked to generate candidate test cases based on model invocation prompts.

[0103] In a preferred embodiment, the instructions also cause the system to perform the following operations when executed by the processor:

[0104] The candidate test cases are validated using a pre-defined set of validators;

[0105] Update the candidate test cases based on the verification results;

[0106] The candidate test cases that have passed the verification and the updated candidate test cases are sorted according to preset rules.

[0107] In a preferred embodiment, sorting the candidate test cases that have passed verification and the updated candidate test cases based on preset rules includes:

[0108] The ranking scores of candidate test cases that pass the verification and the updated candidate test cases are calculated based on security impact factors, operational phase factors, interface state coupling factors, and coverage gap contribution factors.

[0109] Based on the ranking score, the candidate test cases that have passed the verification and the updated candidate test cases are ranked.

[0110] In a preferred embodiment, updating candidate test cases based on the verification results includes:

[0111] Based on the verification results, candidate test cases are modified, regenerated, or subject to user confirmation.

[0112] Specifically, when the verification result indicates that the candidate test case has missing format or missing fields, the candidate test case is automatically corrected.

[0113] When the verification result indicates that there is a coverage gap in the candidate test case, supplementary prompt words are generated based on the coverage gap item and the large language model is called to regenerate the candidate test case.

[0114] When the verification result indicates that there is content that cannot be determined in the candidate test cases, the candidate test cases and corresponding issue items are displayed to the user for confirmation.

[0115] In a preferred embodiment, the preset validator set includes at least one of the following: requirement identifier validator, interface field validator, status validator, stage validator, state transition validator, unit dimension validator, threshold boundary validator, expected result validator, integrity validator, and traceability relationship validator.

[0116] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Those skilled in the art can make various modifications, equivalent substitutions, or variations to the above embodiments without departing from the concept of the present invention. For example, the foregoing embodiments illustrate the intelligent test case generation method using testing requirements in the aviation field as an example, but the present invention is not limited to the aviation field and can also be applied to technical scenarios with high requirements for test case quality, traceability, or verification, such as rail transportation, automotive electronics, industrial control, medical equipment, financial systems, and communication systems.

[0117] Furthermore, the requirement parsing items, generation target items, constraint coverage items, supplementary instruction items, validator sets, and sorting factors described in the foregoing embodiments are merely illustrative examples. In practical applications, they can be added, deleted, or adjusted according to specific business systems, test specifications, interface documents, requirement management methods, or test management platforms. The large language model can be a text generation model deployed on a local server, private cloud, or public cloud, or it can be a generation system formed by multiple models or multiple intelligent agents working together. The model call prompt words can be generated using template filling, rule concatenation, retrieval enhancement, or a combination thereof.

[0118] It should be understood that the steps described in this specification are not necessarily executed in the order shown, unless the context explicitly requires a specific execution order. Multiple steps may be executed together, and a single step may be broken down into multiple sub-steps. The aforementioned modules may be implemented by software, hardware, firmware, or any combination thereof, or by a processor executing program instructions stored in a computer-readable storage medium.

[0119] Any modifications, equivalent substitutions, improvements, or combinations made within the spirit and principles of this invention shall be included within the scope of protection of this invention. The scope of protection of this invention shall be determined by the scope defined in the claims.

Claims

1. A method for generating intelligent test cases, characterized in that, The method includes: Test requirements for obtaining user input; Receive user-triggered requests for analysis. After receiving the requirement analysis trigger operation, an optional configuration interface is displayed to the user, which includes requirement parsing items, generation target items, constraint coverage items, and supplementary instruction items. Generate model invocation prompts based on the user's selections in the optional configuration interface, and display a summary of the model invocation prompts; The large language model is invoked to generate candidate test cases based on the prompt words invoked by the model.

2. The method according to claim 1, characterized in that, The method further includes: The candidate test cases are validated using a preset set of validators; The candidate test cases are updated based on the verification results; The candidate test cases that have passed the verification and the updated candidate test cases are sorted according to preset rules.

3. The method according to claim 2, characterized in that, The candidate test cases that have passed the verification and the updated candidate test cases are sorted according to preset rules, including: The ranking scores of the candidate test cases that pass the verification and the updated candidate test cases are calculated based on security impact factors, operational phase factors, interface state coupling factors, and coverage gap contribution factors. Based on the ranking score, the candidate test cases that have passed the verification and the updated candidate test cases are ranked.

4. The method according to claim 2, characterized in that, The update of the candidate test cases based on the verification results includes: Based on the verification results, the candidate test cases are modified, regenerated, or subject to user confirmation. Specifically, when the verification result indicates that the candidate test case has missing format or missing fields, the candidate test case is automatically corrected. When the verification result indicates that the candidate test case has a coverage gap, supplementary prompt words are generated based on the coverage gap item and the large language model is called to regenerate the candidate test case. When the verification result indicates that there is content that cannot be determined in the candidate test case, the candidate test case and the corresponding issue item are displayed to the user for confirmation.

5. The method according to claim 2, characterized in that, The preset set of validators includes at least one of the following: requirement identifier validator, interface field validator, status validator, stage validator, state transition validator, unit dimension validator, threshold boundary validator, expected result validator, integrity validator, and traceability relationship validator.

6. An intelligent test case generation system, characterized in that, The system includes: processor; A memory coupled to the processor, the memory storing processor-executable instructions that, when executed by the processor, cause the system to perform the following operations: Test requirements for obtaining user input; Receive user-triggered requests for analysis. After receiving the requirement analysis trigger operation, an optional configuration interface is displayed to the user, which includes requirement parsing items, generation target items, constraint coverage items, and supplementary instruction items. Generate model invocation prompts based on the user's selections in the optional configuration interface, and display a summary of the model invocation prompts; The large language model is invoked to generate candidate test cases based on the prompt words invoked by the model.

7. The system according to claim 6, characterized in that, When executed by the processor, the instructions also cause the system to perform the following operations: The candidate test cases are validated using a preset set of validators; The candidate test cases are updated based on the verification results; The candidate test cases that have passed the verification and the updated candidate test cases are sorted according to preset rules.

8. The system according to claim 7, characterized in that, The candidate test cases that have passed the verification and the updated candidate test cases are sorted according to preset rules, including: The ranking scores of the candidate test cases that pass the verification and the updated candidate test cases are calculated based on security impact factors, operational phase factors, interface state coupling factors, and coverage gap contribution factors. Based on the ranking score, the candidate test cases that have passed the verification and the updated candidate test cases are ranked.

9. The system according to claim 7, characterized in that, The update of the candidate test cases based on the verification results includes: Based on the verification results, the candidate test cases are modified, regenerated, or subject to user confirmation. Specifically, when the verification result indicates that the candidate test case has missing format or missing fields, the candidate test case is automatically corrected. When the verification result indicates that the candidate test case has a coverage gap, supplementary prompt words are generated based on the coverage gap item and the large language model is called to regenerate the candidate test case. When the verification result indicates that there is content that cannot be determined in the candidate test case, the candidate test case and the corresponding issue item are displayed to the user for confirmation.

10. The system according to claim 7, characterized in that, The preset set of validators includes at least one of the following: requirement identifier validator, interface field validator, status validator, stage validator, state transition validator, unit dimension validator, threshold boundary validator, expected result validator, integrity validator, and traceability relationship validator.

Images