Software testing method and device, electronic equipment and storage medium

Abstract

The invention provides a software testing method and device, electronic equipment and a storage medium, and the method comprises the steps: generating a structured test case and procedure set according to data input by a user according to a preset test case and procedure framework; generating a to-be-filled test record table according to a preset test record framework and the test case and regulation set; receiving a structured test result report generated by an external automatic test tool according to the test case and the procedure set; and extracting an execution result from the test result report, and filling the execution result into the test record table to obtain a test result. Standardization of test cases, regulations and record table information and machine analysis are achieved through a structured framework, and a data foundation is laid for automatic processing. And the external test tool report is automatically analyzed, and the result is matched into the test record table, so that the manual filling and reporting are remarkably reduced, the input error is reduced, and the test traceability and efficiency are improved.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and in particular to a software testing method and apparatus, electronic device, and storage medium. Background Technology

[0002] In software development processes across high-safety-requirement fields such as aviation, aerospace, and industrial control, software testing is a core component ensuring product safety and reliability. International airworthiness standard DO-178C and its supplementary standard DO-330 impose stringent requirements on airborne software testing processes, emphasizing that testing activities must be standardized, traceable, and adequately verified. This means that every software requirement must have corresponding test cases, and a complete two-way traceability chain must be established between requirements, test cases, and test procedures to form an easily auditable evidence system.

[0003] Currently, the industry commonly uses a combination of documentation and test management tools for software testing. Testers typically use word processing software or spreadsheets to write test cases, procedures, and records, and then use professional automated testing tools to execute tests and generate results reports. However, existing methods rely heavily on manual operation and experience-based judgment, and test case templates lack standardized structure, leading to inefficient understanding, reuse, and review of test cases.

[0004] The main drawbacks of existing technologies are as follows: First, the lack of uniformity in test cases, procedures, and record templates makes it difficult for machines to automatically parse test information, hindering automated processing. Second, the tracking relationships between test requirements, test cases, procedures, and results rely on manual establishment and maintenance, which is inefficient and prone to errors, making it difficult to meet the comprehensive traceability required by the DO-178C standard. Finally, manually filling in test records is not only time-consuming and inefficient in analysis, but also prone to human error, posing a significant challenge to compliance audits. Summary of the Invention

[0005] In view of the above-mentioned deficiencies of the prior art, the present invention provides a software testing method and apparatus, electronic device, and storage medium to solve the technical problems of incomplete traceability chain and low testing efficiency in the testing process.

[0006] To achieve the above and other related objectives, this invention provides a software testing method, comprising: generating a structured set of test cases and procedures based on data input by a user according to a preset test case and procedure framework, wherein the test case and procedure framework includes a set of test case and test procedure codes with a unified format and fields; generating a test record table to be filled based on a preset test record framework and the test case and procedure set, wherein the test record table includes configuration identifiers, expected results, judgment criteria fields, and status fields for filling in actual test results from the test case and procedure set; receiving a structured test result report generated by an external automated testing tool based on the test case and procedure set; extracting execution results from the test result report and filling the execution results into the test record table to obtain test results.

[0007] In one embodiment of the present invention, the test cases and the procedure framework are integrated code files, and the test record framework is a table file.

[0008] In one embodiment of the present invention, after the step of generating a structured test case and procedure set, the method further includes: generating a traceability matrix based on the test case and procedure set, and performing test design verification based on the traceability matrix; and generating the test record table after the test design verification is qualified.

[0009] In one embodiment of the present invention, generating a tracing matrix based on the test cases and procedure set includes: parsing the test cases and procedure set, and automatically generating a first tracing matrix and a second tracing matrix, wherein the first tracing matrix is ​​used to characterize the association between test requirements and test cases, and the second tracing matrix is ​​used to characterize the association between test cases and test procedures.

[0010] In one embodiment of the present invention, test design verification based on the tracking matrix includes: verifying the coverage missing items and abnormal correlations in the first tracking matrix and the second tracking matrix based on a predefined rule set.

[0011] In one embodiment of the present invention, after generating the test record table, the method further includes: automatically checking whether the test case identifier set in the test record table is completely consistent with the test case identifier set in the test case and procedure set; if yes, then the subsequent step of receiving the first test result is executed; if no, then an error message is output and the process is terminated.

[0012] In one embodiment of the present invention, the execution result is filled into the test record table to obtain the test result, including: automatically matching the execution result according to the number to generate a comparison analysis table to assist in filling in the test record table, so as to facilitate subsequent manual verification and supplementation to form a complete test result.

[0013] To achieve the above and other related objectives, the present invention also provides a software testing apparatus, comprising: a first generation unit, configured to generate a structured set of test cases and procedures based on data input by a user according to a preset test case and procedure framework, wherein the test case and procedure framework includes a set of test case and test procedure codes with a unified format and fields; a second generation unit, configured to generate a test record table to be filled based on a preset test record framework and the test case and procedure set, wherein the test record table includes configuration identifiers, expected results, judgment criteria fields, and status fields for filling in actual test results from the test case and procedure set; a receiving unit, configured to receive the test case and procedure set and a structured test result report generated by an external automated testing tool based on the test case and procedure set, and to automatically check the filled test record table according to predefined verification rules; and a third generation unit, configured to extract execution results from the test result report and fill the execution results into the test record table to obtain test results.

[0014] To achieve the above and other related objectives, the present invention also provides an electronic device, including a processor, a memory, and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the method provided in any of the above embodiments.

[0015] To achieve the above and other related objectives, the present invention also provides a computer-readable storage medium having a computer program stored thereon, the computer program being used to cause a computer to perform the method provided in any of the above embodiments.

[0016] The beneficial effects of this invention are as follows: This invention proposes a software testing method, apparatus, electronic device, and storage medium. The method, through a pre-defined structured framework, standardizes and machine-parses test cases, procedures, and record table information, laying a data foundation for subsequent automated processing. By receiving and automatically parsing structured reports from external automated testing tools, it supports the automatic filling of automated results into structured test record templates, reducing manual operations and improving test data entry efficiency. Compared to traditional methods relying on manual filling of test records, this invention significantly reduces the risk of human input errors and improves the traceability and efficiency of testing activities. Furthermore, this invention supports manual verification and supplementation of automatically filled content, incorporating document checking functions, making it particularly suitable for high-reliability applications with stringent requirements for test documentation. Overall, this invention improves the standardization and automation level of the testing process, possessing good engineering feasibility and widespread application value. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The accompanying drawings are incorporated in and constitute a part of this specification, illustrating embodiments consistent with this application, and are used together with the description to explain the principles of this application. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0018] Figure 1 A flowchart of a software testing method provided in an embodiment of the present invention; Figure 2 A schematic diagram of a software testing apparatus provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0019] Explanation of reference numerals in the attached drawings: 201, first generation unit; 202, second generation unit; 203, receiving unit; 204, third generation unit; 301, processor; 302, memory. Detailed Implementation

[0020] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. It should be noted that, unless otherwise specified, the following embodiments and features can be combined with each other. In addition to the specific methods, equipment, and materials used in the embodiments, based on the knowledge of the prior art and the description of the present invention by those skilled in the art, any prior art methods, equipment, and materials similar to or equivalent to the methods, equipment, and materials in the embodiments of the present invention can be used to implement the present invention.

[0021] It should be understood that the terminology used in the embodiments of this invention is for describing specific implementations and not for limiting the scope of protection of this invention. Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art.

[0022] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In some embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0023] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functions, and operations that may be implemented in the methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0024] Please see Figure 1 , Figure 1 A software testing management and comparison method provided in an embodiment of the present invention includes steps S101 to S104.

[0025] Step S101: Based on the data input by the user according to the preset test case and procedure framework, a structured set of test cases and procedures is generated. The test case and procedure framework includes a set of test case and test procedure code with a unified format and fields. Generating the test case and procedure set through a preset structured template ensures the standardization and consistency of test design, avoiding problems such as inconsistent formatting and missing content caused by manual writing. At the same time, the structured data format provides a foundation for subsequent automated processing (such as automatic generation of traceability matrices and result comparison), significantly improving the efficiency and traceability of the testing process.

[0026] In this step, the test case and procedure framework can be defined, for example, as comments containing fields such as "test case number, test objective, test input, expected output, and judgment criteria," and written in the code document; the test procedure can be defined, for example, as comments and code containing "test procedure number, test suite registration framework, input data code, and stub function code." Users write structured test cases and procedure sets based on the template (e.g., / *Test case start TC_001:{Objective: Verify login function, Input: Username / Password, Expected output: Login successful} Test case end* / , / *Test procedure start TP_001:{Steps: 1. Open login page 2. Enter username and password 3. Click login} Test procedure end* / ).

[0027] Step S102: Based on the preset test record framework and test case and procedure set, generate a test record table to be filled in. The test record table includes configuration identifiers, expected results, decision criteria fields from the test case and procedure set, and a status field for filling in the actual test results. In this step, a formatted and fully-fielded test record table is automatically generated. This table inherits from the template structure; for example, information such as the test case number, expected results, and decision criteria can be pre-filled, while the fields for actual execution results, pass / fail status, and remarks are left blank to be filled in after testing. This format facilitates recording by testers during execution and also facilitates unified collection and analysis later.

[0028] For example, the generated test record table may contain the following fields: "Test Case Number: TC_001", "Test Procedure Number: TPH-001, TPF-002, TPS-001", "Expected Output: Return FALSE", "Judgment Criterion: Return value is consistent with expected output", "Actual Execution Result: ______", "Pass / Fail: ______", "Remarks: ______". The "Actual Execution Result" field is currently empty and will be filled in later.

[0029] In one specific embodiment of the present invention, the test cases and the procedure framework are integrated into a single code file, such as C, C++, or other formats; the test record framework is a table file, such as XLS, XLSX, or other formats, as long as it supports field standardization and information extraction.

[0030] In a specific embodiment of the present invention, after step S101, the method further includes: (1) generating a traceability matrix based on the test cases and procedure set, and performing test design verification based on the traceability matrix; S202, after the test design verification is qualified, generating a test record table.

[0031] In this embodiment, the generated traceability matrix and structured test case content assist test reviewers in verifying test designs. On the one hand, reviewers can use the traceability matrix generated by the automation tool in the previous stage to determine whether the test design is sufficient and check whether the test procedures correspond one-to-one with the test objectives, improving review efficiency and accuracy. On the other hand, the tool supports automatically highlighting missing items or many-to-one abnormal relationships in test cases, helping to discover problems and enhancing quality control in the design phase.

[0032] In a specific embodiment of the present invention, a traceability matrix is ​​generated based on the test cases and procedure sets. This includes parsing the test cases and procedure sets and automatically generating a first traceability matrix and a second traceability matrix. The first traceability matrix represents the relationship between test requirements and test cases, and the second traceability matrix represents the relationship between test cases and test procedures. The automatic generation of the first traceability matrix (requirement-test case) and the second traceability matrix (test case-procedure) can intuitively display the test coverage, ensuring that each requirement has a corresponding test case and each test case has a corresponding test procedure. This greatly improves the auditability of test design, facilitates the discovery of coverage gaps or abnormal relationships, and meets the stringent traceability requirements of high security standards.

[0033] Table 1: Example of the first tracing matrix (requirement-use case).

[0034]

[0035] Table 2: Example of the second tracing matrix (use case - procedure).

[0036]

[0037] In a specific embodiment of the present invention, test design verification based on the tracking matrix includes: verifying the coverage missing items and abnormal correlations in the first tracking matrix and the second tracking matrix based on a predefined rule set.

[0038] For example, the predefined rule set includes: "Each requirement is covered by at least one test case." During verification, if requirement REQ_003 is found to have no corresponding test case in the matrix, the row will be automatically highlighted and a "Coverage Missing" message will be displayed. As another example, if multiple test cases (TC_004, TC_005) are detected to correspond to the same test procedure (TP_004) and have the same test objective, a "Suspected Duplicate Test" message will be displayed.

[0039] In a specific embodiment of the present invention, after generating the test record table, the method further includes: automatically checking whether the set of test case identifiers in the test record table is completely consistent with the set of test case identifiers in the test case and procedure set; if so, the subsequent step of receiving the first test result is executed; if not, an error message is output and the process is terminated. Automatically checking the consistency of test case identifiers effectively avoids omissions or misfilling of test cases due to human error, ensuring the integrity of test execution. This step, as an automated quality checkpoint in the process, interrupts abnormal processes in a timely manner before test execution, preventing invalid or incomplete test records from entering subsequent stages and improving the reliability of the overall testing process.

[0040] Step S103: Receive the structured test result report generated by the external automated testing tool based on the test cases and procedure set. Receiving the structured test result report (e.g., XML format) generated by the external automated testing tool (e.g., C++Test) allows the acquisition of machine-determined test results, providing an objective and parsable data source for subsequent test record form completion. For example, the system parses the XML report... <testcase id="TC_001"> <result> PASS< / result> < / testcase> The automated test result for test case TC_001 was extracted as "pass".

[0041] Step S104: Extract the execution results from the test result report and fill the execution results into the test record table to obtain the test results. Extract the execution results corresponding to the test case identifier from the test result report, and automatically fill the execution results into the corresponding "actual execution result" and / or "pass / fail" status fields in the test record table based on the identifier to generate a preliminary test record.

[0042] In a specific embodiment of the present invention, the execution results are filled into a test record table to obtain test results. This includes: automatically matching the execution results according to their numbers to generate a comparison analysis table to assist in filling out the test record table, facilitating subsequent manual verification and supplementation to form a complete test result. The execution results are automatically matched to the corresponding fields in the test record table according to the test case numbers to generate a comparison analysis table as an intermediate result. This table simultaneously presents the preset expected results and the actual results of the automated test, facilitating manual verification by testers. Testers can use this table to confirm results, add notes, or correct anomalies, ultimately forming a complete and accurate formal test record.

[0043] In a specific embodiment of the present invention, after matching and filling in the comparison analysis table, the method further includes: automatically checking for errors in the comparison analysis table or the completed test record table or the input test cases and procedures according to predefined verification rules. Dimension 1: Automatic error checking in the comparison analysis table or the completed test record table further ensures the completeness and standardization of test records; for example, the system checks whether the "Pass / Fail" field in the record table is empty. If it is empty, it prompts "Missing Item" and highlights it, recording it in the output error log. Dimension 2: Automatic error checking of the received test cases and procedures, for example, checking whether the test case number or procedure number format conforms to the standard according to the selected setting rules. If it does not conform, it prompts "Incorrect Number" and records it in the output error log.

[0044] It should be noted that the steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they contain the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0045] Please see Figure 2 , Figure 2 A software testing apparatus according to an embodiment of the present invention includes a first generation unit 201, a second generation unit 202, a receiving unit 203, and a third generation unit 204. The first generation unit 201 generates a structured set of test cases and procedures based on data input by a user according to a preset test case and procedure framework. The test case and procedure framework includes a set of test case and test procedure codes with a unified format and fields. The second generation unit 202 generates a test record table to be filled based on a preset test record framework and the test case and procedure set. The test record table includes configuration identifiers, expected results, judgment criteria fields, and status fields for filling in actual test results from the test case and procedure set. The receiving unit 203 receives a structured test result report generated by an external automated testing tool based on the test cases and procedure set. The third generation unit 204 extracts execution results from the test result report and fills the execution results into the test record table to obtain the test results.

[0046] It should be noted that the software testing apparatus of this embodiment corresponds to the software testing method described above, and the functional modules in the software testing apparatus may correspond to the corresponding steps in the software testing method. The software testing apparatus of this embodiment can be implemented in conjunction with the software testing method; that is, where there is no conflict, the relevant technical details mentioned in the software testing method of the above embodiments can also be applied to the software testing apparatus of this embodiment.

[0047] Please see Figure 3 , Figure 3 An electronic device provided in one embodiment of the present invention includes a processor 301, a memory 302, and a communication bus; the communication bus is used to connect the processor 301 and the memory 302; the processor 301 is used to execute a computer program stored in the memory 302 to implement the above-mentioned software testing method.

[0048] The aforementioned electronic device is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0049] The aforementioned electronic devices can be any electronic product capable of human-computer interaction with users, such as personal computers, tablets, smartphones, personal digital assistants (PDAs), game consoles, interactive network television (IPTV), smart wearable devices, etc.

[0050] The aforementioned electronic devices may also include network devices and / or user devices. Among them, network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0051] The networks in which the aforementioned electronic devices are located include, but are not limited to, the Internet, wide area networks, metropolitan area networks, local area networks, and virtual private networks (VPNs).

[0052] The processors mentioned above can be, for example, general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components; the memory mentioned above may include random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device.

[0053] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, the computer program being used to cause a computer to execute the above-described software testing method.

[0054] In summary, this invention achieves full standardization and automation of test design, execution, recording, and comparison by constructing a structured template system and automated processing flow. This method significantly improves the traceability of the testing process, the accuracy and efficiency of result comparison, and effectively supports test adequacy verification and compliance review in high-security fields.

[0055] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A software testing method characterized by, include: Based on the data input by the user according to the preset test case and procedure framework, a structured set of test cases and procedures is generated, wherein the test case and procedure framework includes a set of test case and test procedure code with a unified format and fields; Based on the preset test record framework and the test case and procedure set, a test record table to be filled is generated, wherein the test record table includes configuration identifiers, expected results, judgment criteria fields and status fields for filling in actual test results from the test case and procedure set; Receive structured test result reports generated by external automated testing tools based on the test cases and procedure sets; The execution results are extracted from the test result report and filled into the test record table to obtain the test results.

2. The software testing method of claim 1, wherein, The test cases and procedure framework are integrated into a single code file, and the test record framework is a table file.

3. The software testing method of claim 1, wherein, The steps following the generation of structured test cases and procedure sets also include: Based on the test cases and procedure set, a traceability matrix is ​​generated, and test design verification is performed based on the traceability matrix; The test record table is generated after the test design has been verified as qualified.

4. The software testing method of claim 3, wherein, Based on the test cases and procedure set, a traceability matrix is ​​generated, including: The test cases and procedure sets are parsed to automatically generate a first trace matrix and a second trace matrix. The first trace matrix is ​​used to represent the relationship between test requirements and test cases, and the second trace matrix is ​​used to represent the relationship between test cases and test procedures.

5. The software testing method according to claim 3, characterized in that, Test design verification is performed based on the aforementioned tracking matrix, including: Based on a predefined set of rules, the missing coverage items and abnormal relationships in the first and second tracking matrices are checked.

6. The software testing method according to claim 1, characterized in that, After generating the test record table, the following is also included: Automatically check whether the set of test case identifiers in the test record table is completely consistent with the set of test case identifiers in the test case and procedure set: If so, proceed with the next step of receiving the first test result. If not, output an error message and abort the process.

7. The software testing method according to claim 1, characterized in that, The execution results are entered into the test record table to obtain the test results, including: The execution results are automatically matched according to the number to generate a comparison and analysis table for filling in the auxiliary test record form, which facilitates subsequent manual verification and supplementation to form a complete test result.

8. A software testing apparatus, characterized in that, include: The first generation unit is used to generate a structured set of test cases and procedures based on the data input by the user according to the preset test case and procedure framework. The test case and procedure framework includes a set of test case and test procedure codes with a unified format and fields. The second generation unit is used to generate a test record table to be filled in according to the preset test record framework and the test case and procedure set, wherein the test record table includes configuration identifiers, expected results, judgment criteria fields and status fields for filling in actual test results from the test case and procedure set. The receiving unit is used to receive the test cases and procedure sets, as well as the structured test result reports generated by external automated testing tools based on the test cases and procedure sets, and to automatically check the completed test record forms according to predefined verification rules. The third generation unit is used to extract the execution results from the test result report and fill the execution results into the test record table to obtain the test results.

9. An electronic device, characterized in that, It includes a processor, a memory, and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program thereon, the computer program being used to cause the computer to perform the method as described in any one of claims 1 to 7.

Images