An automatic testing method for WAT testing software

Abstract

The present invention provides an automatic testing method for WAT test software, including the following steps during the testing phase: step S1. setting a test execution file and running it, automatically sending a number of test cases to the WAT test software for testing, and automatically generating a log file to record the execution results of all or part of the test cases in the test execution file; step S2. judging whether the test results of the test cases meet the preset standards according to the log file, and judging whether the quality of the WAT test software meets the standards. By setting the test execution file and running it, the automated testing for the WAT test software is realized, and different test scenarios can be covered by setting the test case set in the test execution file in a targeted manner, so that the software has stronger applicability in different application scenarios, effectively avoids the generation of human errors, and can greatly improve the test efficiency, optimize the R&D cycle of the test software to the greatest extent, and thus comprehensively improve the work efficiency.

Description

Technical Field

[0001] The invention belongs to the technical field of automated testing, and in particular relates to a method for automatically testing testing software used for WAT (wafer acceptance testing). Background Art

[0002] Using software for testing is an important method to improve product quality and user experience. Before the release of the test software for WAT (wafer acceptance test), it is necessary to verify the availability and ease of use of new functions. Before each version is released, in addition to verifying the new functions, all previous functions need to be tested, such as regression testing, performance testing, stress testing, etc., to ensure that the released software version is reliable and stable. Traditionally, the pre-release testing of the WAT test software system is done manually. During these testing processes, a large number of repetitive operations are required on the software, which consumes a lot of manpower and time costs, and inevitably generates some human errors.

[0003] In order to reduce the workload and error probability of manual testing and ensure the correctness of basic functions, the development of automated testing technology is particularly necessary for the application of WAT testing software. Existing automated testing technology is mainly aimed at WEB-side testing, and the automated testing of CS architecture software is not yet mature, especially in the testing of multi-terminal communication software systems such as WAT testing software systems, the effective application of existing automated testing is still very lacking.

[0004] In addition, in the process of developing WAT test software, in order to achieve a better user experience and customized needs between different customers, the software functions need to be continuously updated and the code size will continue to expand, which will directly lead to a significant increase in compilation time and error probability. During the manual compilation process, compilation errors caused by not selecting the right macro or not clearing the cache will occur, seriously affecting the quality of software release. Currently, manual code compilation is more common, and automatic compilation has not yet added an error correction mechanism, resulting in the inability to guarantee the correctness of code macro selection during the compilation process. At present, automatic testing does not provide a feasible solution for self-checking during the compilation stage.

[0005] Therefore, there is a great need to study an automatic testing method for WAT testing software, which can be used for effective automatic testing of WAT testing software, so as to further promote the development of WAT testing software and the widespread application of automatic testing technology. Summary of the invention

[0006] The present invention is to solve all or part of the problems of the above-mentioned prior art. The present invention provides an automatic testing method for WAT testing software, which is suitable for performing effective automatic testing on WAT testing software.

[0007] The present invention provides an automatic testing method for WAT testing software, comprising the following steps during the testing phase: step S1. setting a test execution file and running it, automatically sending a number of test cases to the WAT testing software for testing, and automatically generating a log file to record the execution results of all or part of the test cases in the test execution file; step S2. judging whether the test results of the test cases meet preset standards according to the generated log file, and if so, judging that the quality of the WAT testing software meets the standards; if not, judging that the quality of the WAT testing software does not meet the standards; wherein the test execution file includes the test cases and instructions for controlling the WAT testing software to execute the test cases; and the test execution file is used to cover different testing requirements for the WAT testing software.

[0008] In step S1, the test execution file is generated in the following manner: obtaining a test case set including the test cases to cover different test requirements for the WAT test software; using the test case set, adding several instructions and automatically generating the test execution file. Running the test execution file can automatically send the test cases in the test case set to the WAT test software for testing.

[0009] The method for generating the test execution file also includes sorting the test cases in the test case set before generating the test execution file; running the test execution file can automatically send the several test cases to the WAT test software in sequence for testing according to the sorting result.

[0010] The test execution file is provided with a progress confirmation instruction; the progress confirmation instruction can be issued after a preset period of time or every preset time period after the current test case starts testing, and is used to confirm whether the WAT test software has completed the test of the current test case, and then determine the time to start the next test case.

[0011] In step S1, the test case includes the input required for simulating the WAT test software to perform an electrical test, including: test case name, test items, wafer information, grain information, information on whether the wafer number and slot number are consistent, limit parameter information and limit format information.

[0012] In the step S2, if the determination conclusion is that the quality of the WAT test software meets the standard, the determination conclusion is fed back to the user.

[0013] In step S2, if the conclusion is that the quality of the WAT test software does not meet the standards, a test report is generated and provided to the user. In general, the user includes a technician who tests the WAT test software. The feedback and the way of providing the test report to the user include notifying the user by phone, message and / or email.

[0014] The automatic testing method for WAT test software also includes a self-checking step performed in the compilation stage, including: step 1. obtaining the source code and preset type macro of the WAT test software; step 2. using the source code and the type macro to automatically compile and generate a cache temporary file; the cache temporary file includes macro information and configuration information used in the automatic compilation process; checking the macro information in the cache temporary file, when it is checked that the macro information is incorrect, stopping the current automatic compilation and after clearing the cache temporary file, reusing the source code and the type macro to automatically compile and checking the macro information of the newly generated cache temporary file; step 3. completing the current automatic compilation and generating the required version of the WAT test software; wherein different type macros are used to compile different versions of the WAT test software to suit different test environments; the type macro includes selection information of several macros, and different macros correspond to different functions or settings of the WAT test software. During the compilation phase, problems such as incorrect macro selection often lead to the software code of the compiled WAT test software being disordered and some functions not working. If these problems can be discovered before the testing phase before the software is released, it will avoid having to recompile and then retest all functions, which will greatly reduce the workload of testers and help optimize generation efficiency and ensure software delivery time.

[0015] In step 3, after the current automatic compilation is completed, it is also determined whether the required version of the WAT test software is successfully generated. If it is successfully generated, the automatic compilation process is completed, otherwise an error is reported. The error reporting method includes notifying the user by phone, text message and / or email.

[0016] The automatic testing method for WAT testing software of the present invention obtains test results including error information, conclusion on whether the software quality meets the standards and / or test report in the compilation stage and the testing stage; the way of feeding back the test results to the user includes notifying the user by phone, message and / or email.

[0017] Compared with the prior art, the main beneficial effects of the present invention are:

[0018] An automatic testing method for WAT test software of the present invention realizes automatic testing for WAT test software by setting a test execution file and running it, covers all possible test processes, and observes the execution of the WAT test software under different test conditions, thereby improving test efficiency. According to the test requirements of different versions of WAT test software, different test scenarios can be covered by setting the test case set in the test execution file in a targeted manner, so that the method of the present invention is more applicable. The present invention can also provide a self-checking function for the compilation stage of WAT test software, can effectively ensure the correctness of the compilation process, optimize the development cycle of qualified test software to the greatest extent, and thus comprehensively improve work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The figure is a schematic diagram of the communication between the WAT test software and the EAP and Prober ends.

[0020] Figure 2 Schematic diagram of an automatic testing method for WAT testing software according to Embodiment 1 of the present invention.

[0021] Figure 3 The figure is a schematic diagram of the execution flow of the test execution file according to the first embodiment of the present invention.

[0022] Figure 4 A schematic diagram of a test case set according to Embodiment 1 of the present invention.

[0023] Figure 5 Schematic diagram of the self-checking process at the compile stage of the second embodiment of the present invention. DETAILED DESCRIPTION

[0024] The technical solutions in the specific embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] In the following embodiments, operations of the embodiments are described in specific sequences. The description of these sequences is for better understanding of the details in the embodiments and a comprehensive understanding of the present invention, but is not intended to limit the scope of the present invention.

[0026] Embodiment 1

[0027] The automatic test phase is when the tester simulates the generated WAT test software. Generally, the EAP end needs to manually send multiple GEM commands to the WAT test software. At the same time, the Prober end also needs to send multiple GEM commands to the WAT test software. The communication between the WAT test software and the EAP end and the Prober end can be referred to Figure 1 In this embodiment, in order to improve the testing efficiency of testers, automated testing is implemented, such as Figure 2 As shown, the automatic testing method for WAT test software in this embodiment includes the following specific process steps in the automatic testing stage: step S1. Set the test execution file and run it, automatically send several test cases to the WAT test software for testing, and automatically generate a log file to record the execution results of each test case in the test execution file; step S2. According to the generated log file, determine whether the test results of the test case meet the preset standard. If the test results of all test cases meet the preset standard, the quality of the WAT test software is determined to be up to standard; if there is a test case whose test result does not meet the preset standard, the quality of the WAT test software is determined to be not up to standard; wherein, the test execution file includes several test cases, and instructions for controlling the WAT test software to execute several test cases; the test execution file is used to cover different test requirements for the WAT test software. In some implementation situations, the log file is automatically generated in step S1 to record the execution results of some test cases in the test execution file. In step S2, the judgment is based on the generated log file to judge the partial test results. If the partial test results meet the preset standard, the quality of the WAT test software is also determined to be up to standard; if the partial test results do not meet the preset standard, the quality of the WAT test software is determined to be not up to standard. Some test results may be test results that are more critical to achieving the test requirements.

[0028] The method for generating the test execution file in step S1 of the example includes: obtaining a test case set containing several test cases to cover different test requirements for the WAT test software; using the test case set, adding several instructions and automatically generating a test execution file. Running the test execution file can automatically send the test cases in the test case set to the WAT test software for testing. The test cases in the test case set of the example are independent of each other and have no dependencies. They include as many testable processes as possible to cover the actual test requirements.

[0029] The example test execution file generation method also includes sorting the test cases in the test case set before generating the test execution file; running the test execution file can automatically send a number of test cases to the WAT test software in sequence for testing according to the sorting result.

[0030] In this embodiment, a progress confirmation instruction is provided in the test execution file; after the current test case starts testing, the progress confirmation instruction can be issued after a preset period of time or every preset time period to confirm whether the WAT test software has completed the testing of the current test case, and then determine the time to start testing the next test case.

[0031] In step S2 of this embodiment, if the judgment conclusion is that the quality of the WAT test software meets the standard, the judgment conclusion is fed back to the user. If the judgment conclusion is that the quality of the WAT test software does not meet the standard, a test report is also generated and provided to the user.

[0032] In this embodiment, the test process of running the test execution file can refer to Figure 3 . The Prober end is simulated to send the Carrier Waiting for host command to the WAT test software, and the EAP end is simulated to send the Proceed With carrier, Proceed with Carrier verify ok, Create Process Job, and Create Control Job commands to the WAT test software. Then the Prober end is simulated to send the Create Job command to the WAT test software to create a Process Job, and then the Prober end is simulated to send the Start Job command to the WAT test software to start the Process Job execution test, that is, to start testing the test cases in the test case set; by simulating the EAP end to send commands to the WAT test software to inquire about the status of the Process Job, the status of the current test case is identified to determine whether to test the next test case, until all test cases in the test case set are completed. After the test is completed, the Prober end is simulated to send the Foup Release and CarrierRemove commands to the WAT test software, and determine whether the test of all test case sets is completed. If it is completed, the test is ended.

[0033] like Figure 4As shown, the test case includes the input required for simulating the WAT test software to perform an electrical test, including: 1) RecipeName, which is used to uniquely identify a test case; 2) Test Items; 3) Wafer information, which represents the number of selected wafers; 4) Die information, which represents the number of selected dies; 5) Information on whether waferID and slot ID are consistent; 6) Limit parameter information, which includes all output items in a test item TestItem; 7) Lim format, that is, limit file format, and different customers use different file formats. The test case set includes different test cases, including different test conditions, and then executes different test situations. In this embodiment, the following can be effectively tested: 1) normal test process and test data correctness; 2) when the amount of test data is large, the performance of the software such as memory occupancy and CPU occupancy; 3) software execution under different test conditions for different customers; 4) multi-user concurrent execution; 5) software compatibility.

[0034] Embodiment 2

[0035] like Figure 5 As shown, the automatic testing method for WAT test software in Example 2 also includes a self-check step performed during the compilation phase. The compilation phase generally involves first downloading the software code from SVN, and then automatically calling the Cmake software to select different type macros to compile the corresponding different versions of the software. The current routine process during compilation is: open Cmake, select the corresponding type macro to generate the project file on the VS (Visual Studio) compiler, and then open the VS compiler to compile the required version of the WAT test software; during this compilation process, the Cmake macro is often not added or the project in VS is not cleared, resulting in the lack of some functions of the compiled software or disordered code, which seriously affects the quality of the software version release. In the automatic testing method for WAT test software exemplified in this embodiment, the self-check performed during the compilation phase before the test phase before release, such as Figure 5As shown, the steps include: step 1. Obtain the source code and preset type macro of WAT test software; step 2. Use the source code and type macro to automatically compile and generate a cache temporary file; the cache temporary file includes macro information and configuration information used in the automatic compilation process; check the macro information in the cache temporary file, and when the macro information is found to be incorrect, stop the current automatic compilation and after clearing the cache temporary file, reuse the source code and type macro to automatically compile and check the macro information of the newly generated cache temporary file; step 3. End the current automatic compilation and obtain the generated required version of WAT test software; wherein different type macros are used to compile different versions of WAT test software to suit different test environments; the type macro includes selection information of several macros, and different macros correspond to different functions or settings of WAT test software. In this embodiment, the cache temporary file refers to the cmakecache.txt generated after executing the Cmake script.

[0036] In step 3 of the example, after the current automatic compilation is completed, it is also determined whether the required version of the WAT test software is successfully generated. If it is successfully generated, the automatic compilation process is completed. If it is not successfully generated, an error is reported.

[0037] The test results that can be obtained by the automatic testing method for WAT test software in the compilation stage and the automatic testing stage include but are not limited to error information, conclusions on whether the software quality meets the standards, and test reports. The test results are fed back to the user in a manner including notifying the user by phone, message, and email. The user is allowed to obtain the results of the automatic test in real time by a variety of communication methods, which is conducive to timely making corresponding adjustments during the entire test process, as well as recording after the test and guiding the optimization of subsequent design and process according to the results of the automatic test.

[0038] In order to facilitate the description, some common English nouns or letters used in the present invention are only used for exemplary reference rather than restrictive interpretation or specific usage, and the protection scope of the present invention should not be limited by their possible Chinese translations or specific letters.

[0039] The present invention is described in detail above. The structure and working principle of the present invention are described in detail using specific examples herein. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. It should be pointed out that for ordinary technicians in this technical field, the present invention can also be improved and modified without departing from the principle of the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. An automatic testing method for WAT testing software, characterized in that: The testing phase includes: Step S1. Set up a test execution file and run it, automatically send several test cases to the WAT test software for testing, and automatically generate a log file to record the execution results of all or part of the test cases in the test execution file; Step S2. The test case includes limit parameter information. According to the generated log file, it is determined whether the test result of the test case meets the preset standard. If it meets the preset standard, it is determined that the quality of the WAT test software meets the standard; If the preset standard is not met, the quality of the WAT test software is determined to be substandard; The test execution file includes the test cases and instructions for controlling the WAT test software to execute the test cases; the test execution file is used to cover different test requirements for the WAT test software; The test process for running a test executive file includes: The simulated Prober end sends the Create Job command to the WAT test software to create a Process Job, and the simulated Prober end sends the Start Job command to the WAT test software to start the Process Job execution test, that is, to start testing the test cases in the test case set; by simulating the EAP end to send instructions to the WAT test software to inquire about the status of the Process Job, the status of the current test case is identified to determine whether to test the next test case, until the testing of all test cases in the test case set is completed.

2. The automatic testing method for WAT testing software according to claim 1, characterized in that: The method for generating the test execution file in step S1 includes: Obtaining a test case set including the plurality of test cases to cover different test requirements for the WAT test software; The test execution file is automatically generated by using the test case set and adding several instructions.

3. The automatic testing method for WAT testing software according to claim 2, characterized in that: The method for generating the test execution file further includes sorting the test cases in the test case set before generating the test execution file; Running the test execution file can automatically send the plurality of test cases to the WAT test software in sequence for testing according to the sorting result.

4. The automatic testing method for WAT testing software according to claim 1, characterized in that: The test execution file is provided with a progress confirmation instruction; the progress confirmation instruction can be issued after a preset period of time or every preset time period after the current test case starts testing, and is used to confirm whether the WAT test software has completed the test of the current test case, and then determine the time to start the next test case.

5. The automatic testing method for WAT testing software according to claim 1, characterized in that: In step S1, the test case includes the input required for simulating the WAT test software to perform an electrical test, including: test case name, test items, wafer information, grain information, information on whether the wafer number and slot number are consistent, limit parameter information and limit format information.

6. The automatic testing method for WAT testing software according to claim 1, characterized in that: In the step S2, if the determination conclusion is that the quality of the WAT test software meets the standard, the determination conclusion is fed back to the user.

7. The automatic testing method for WAT testing software according to claim 1, characterized in that: In the step S2, if the conclusion is that the quality of the WAT test software does not meet the standards, a test report is generated and provided to the user.

8. The automatic testing method for WAT testing software according to any one of claims 1 to 7, characterized in that: It also includes self-checking steps performed during the compilation phase, including: Step 1. Obtain the source code of the WAT test software and the preset type macro; Step 2. Automatically compile using the source code and the type macro, and generate a cache temporary file; the cache temporary file includes macro information and configuration information used in the automatic compilation process; check the macro information in the cache temporary file, and when it is found that the macro information is incorrect, stop the current automatic compilation and after clearing the cache temporary file, re-use the source code and the type macro to automatically compile and check the macro information of the newly generated cache temporary file; Step 3. Complete the current automatic compilation and generate the required version of WAT test software; Among them, different type macros are used to compile different versions of WAT test software to suit different test environments; the type macro includes selection information of several macros, and different macros correspond to different functions or settings of the WAT test software.

9. The automatic testing method for WAT testing software according to claim 8, characterized in that: In the step 3, after the current automatic compilation is completed, it is also determined whether the required version of the WAT test software is successfully generated. If it is successfully generated, the automatic compilation process is completed, otherwise an error is reported.

10. The automatic testing method for WAT testing software according to claim 9, characterized in that: The test results obtained in the compilation stage and the testing stage include error information, conclusions on whether the software quality meets the standards, or test reports; the test results are fed back to the user in a manner including notifying the user via phone, text message, and / or email.

Images