Demand conclusion automatic collection and judgment method and device and storage medium

Abstract

The invention relates to a demand conclusion automatic collection and judgment method and device and a storage medium, and the method comprises the steps: obtaining case results of a system-level test and a subsystem test after the test work is finished; performing classification statistics on demand test evidences and distribution modes of system-level and subsystem-level tests based on use case results; and for the demand test evidence subjected to classification statistics, judging a demand conclusion according to a set rule, and updating a conclusion of a demand corresponding to the test report according to the demand conclusion. Compared with the prior art, all evidences of system-level and subsystem-level demand tests can be effectively collected, classified and summarized, a demand conclusion is flexibly judged through an algorithm, a regression judgment mode is combined, and judgment logic design is more comprehensive.

Description

Technical Field

[0001] The present invention relates to the technical field of train operation system testing, and in particular to a method, device and storage medium for automatically collecting and determining demand conclusions. Background Art

[0002] As a new generation product for urban rail transit, the Train Autonomous Control System (TACS) has extremely high reliability and safety. During the delivery of this product, system-level testing is one of the important activities for product function reliability verification, which plays a vital role in ensuring product quality.

[0003] In the TACS system-level testing process, after the integration test and confirmation test are completed, the system-level test report needs to be written. In the system-level test report writing work, a very important part is to draw a conclusion on whether the design requirements have been passed. This requires first collecting the conclusions of the test cases, and summarizing the conclusions of all the test cases corresponding to the requirements one by one, so as to draw a conclusion on whether the requirements have finally been passed.

[0004] At present, in the process of report writing, there are a large number of system-level requirements, generally more than 2,000, and each requirement often corresponds to dozens of use cases. If manual retrieval and conclusion are adopted, at the current level, at least tens of thousands of searches are required, which is very time-consuming. At the same time, such a large workload is more prone to omissions and misjudgments, and the efficiency is also very low. In addition, part of the system-level requirements will also need to be allocated to subsystem-level testing. The test conclusions of these requirements need to be retrieved and judged from multiple test reports, and the workload is usually around hundreds of items.

[0005] In view of the above, how to design and implement a method for automatically collecting and determining demand conclusions, thereby improving work efficiency, becomes a technical problem that needs to be solved. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a method, device and storage medium for automatically collecting and determining demand conclusions.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] As a first aspect of the present invention, a method for automatically collecting and determining demand conclusions is provided, the steps comprising:

[0009] After the testing work is completed, obtain the use case results of system-level testing and subsystem testing;

[0010] Based on the use case results, the requirement test evidence and distribution methods of system-level and subsystem-level tests are classified and counted;

[0011] For the requirement test evidence that has been classified and counted, the requirement conclusion is determined according to the set rules, and the conclusion of the test report corresponding to the requirement is updated based on the requirement conclusion.

[0012] As a preferred technical solution, the use case results of the system-level test are obtained as follows:

[0013] The test conclusion of the system-level test is updated to the test results of this round in a prescribed format. In the test form of this round, the test case number is used as an index to update the corresponding test evidence and the test results of the case.

[0014] As a preferred technical solution, the use case results of the subsystem level test are obtained as follows:

[0015] The test conclusion of the subsystem level test is indexed by the test case number, and the corresponding test evidence and the test result of the case are updated in the subsystem test report.

[0016] As a preferred technical solution, when classifying and counting the system-level test, the evidence is first judged whether it is the first round or regression: if it is the first round of testing, all use case results corresponding to the requirements are covered; if it is a regression test, the test evidence will only update the use case results of this round.

[0017] As a preferred technical solution, for the first round of testing, the requirements in the use case requirement coverage matrix are retrieved based on all use case numbers in the historical test form, and the retrieved requirements are updated in the prescribed format with the results taken from the use cases.

[0018] As a preferred technical solution, for the regression test, the test evidence will only update the use case results of this round, using all the use case numbers in the test form of this round to retrieve the requirements in the use case requirement coverage matrix. For all retrieved requirements, the use case results corresponding to the requirements will be updated in the specified format.

[0019] As a preferred technical solution, the format for updating the results of the first round of tests and regression tests is: use case number-use case result-use case remarks.

[0020] As a preferred technical solution, the test evidence of the subsystem-level test is classified as follows: the requirement numbers linked to the system level in the Requirement Text in multiple subsystem reports are compared one by one with the requirement numbers assigned to Equipment in the system-level requirements. If there is a match, the test evidence of each subsystem report is summarized in the report.

[0021] As a preferred technical solution, the requirement conclusions include: failed KO, unmeasurable NT, passed OK, the product does not have NA, and uncovered.

[0022] As a preferred technical solution, the requirement conclusion determined according to the set rules is specifically:

[0023] After completing the classification and statistics of the test evidence of the requirements, determine whether the allocation method is that the product does not have NA. If the requirement conclusion is that the product does not have NA, then the requirement conclusion is directly determined as the product does not have NA;

[0024] If the allocation method is not product does not have NA, the requirement test evidence results are further classified in the order of fail KO greater than untestable NT greater than pass OK greater than product does not have NA.

[0025] As a preferred technical solution, the requirements test evidence results are further classified and determined as follows:

[0026] If the evidence result fails to pass the KO, the requirement conclusion is determined to fail the KO;

[0027] There is no KO in the evidence results, but there is untestable NT, and the required conclusion is determined to be untestable NT;

[0028] There is no KO in the evidence results, no untestable NT, but there is OK, and the requirement conclusion is judged as OK;

[0029] There is no KO in the evidence results, no NT in the untestable results, no OK in the test results, only NA in the product, and the demand conclusion is that the product does not have NA.

[0030] There is no failure KO, no unmeasurable NT, no pass OK, no product not possess NA in the evidence results, and the evidence result is empty, and the demand conclusion is judged to be uncovered.

[0031] As a preferred technical solution, when further determining the classification of the requirement test evidence results:

[0032] If there is no KO, NT, OK, or NA in the evidence result, and the evidence result is not empty, check whether the previous version requirement conclusion is uncovered. If the previous version requirement conclusion is not uncovered, the requirement conclusion is determined to be NA.

[0033] There is no failure KO, no unmeasurable NT, no pass OK, no product not possessing NA in the evidence results, and the evidence results are not empty, and the previous version of the requirement conclusion is uncovered, the requirement conclusion remains the result of the previous version and will not be updated.

[0034] As a preferred technical solution, the method does not update or automatically updates to a new result based on the determination requirement conclusion.

[0035] As a second aspect of the present invention, there is provided an apparatus for automatically collecting and determining demand conclusions, comprising a memory, a processor, and a program stored in the memory, wherein the apparatus is characterized in that when the processor executes the program, the method for automatically collecting and determining demand conclusions as described above is implemented.

[0036] As a third aspect of the present invention, there is provided a storage medium having a program stored thereon, wherein when the program is executed, the method for automatically collecting and determining demand conclusions as described above is implemented.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1) The method for automatically collecting and determining demand conclusions proposed in the present invention can automatically collect and classify test demand evidence, and can determine conclusions based on an algorithm, which is faster and more accurate than the original manual identification and listing of evidence, thereby effectively improving the efficiency of test report writing;

[0039] 2) Aiming at the needs of large systems, the present invention has a more comprehensive judgment logic design. The advantages of this judgment method are: the judgment of NT and NA is added; the OK result can be judged quickly and correctly, and the KO result has a regression judgment logic, which is equivalent to further judgment based on the previous KO judgment, and the accuracy is greatly increased.

[0040] 3) The present invention uses the Test Record of this round of testing as the test evidence input instead of traversing all the test results. On the one hand, it improves the efficiency. On the other hand, it only changes the conclusions of the changed parts of this round, and does not affect the existing correct results. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flow chart of the method for automatically collecting and determining demand conclusions of the present invention;

[0042] Figure 2 It is a schematic diagram of executing the system-level requirement conclusion determination of the present invention;

[0043] Figure 3 It is a schematic diagram of the execution of subsystem level requirement conclusion determination of the present invention;

[0044] Figure 4 This is a flow chart for determining the demand conclusion of the present invention. DETAILED DESCRIPTION

[0045] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0046] Example 1

[0047] The present invention provides a method for automatically collecting and determining demand conclusions, such as Figure 1 The specific steps of the method are as follows:

[0048] Step S1: After the test work is completed, the materials for demand evidence collection and result judgment must be prepared first, that is, the use case results of system-level test and subsystem test. The system-level test updates the test conclusion to the current round of test results in the prescribed format. In the current round of test form (Test Record), the corresponding test evidence and test results of the use case are updated with the test case number as the index; the subsystem test conclusion is also updated in the subsystem test report.

[0049] Step S2: Classify and count the test evidence for the requirements test based on the test results prepared in step S1. First, it is necessary to distinguish whether it is system-level evidence or subsystem-level evidence. The classification methods for the two are different. Figure 2 As shown, the system-level test evidence is classified as follows: If it is a regression test, first use all the use case numbers in the current round of test form Test Record to retrieve the requirements in the use case requirement coverage matrix (Traceability). All retrieved requirements need to update the conclusions. The conclusions of these requirements are taken from the results of the corresponding use cases. According to the prescribed format (use case number-use case result-use case notes), the use case results corresponding to the requirements are updated; if it is the first round of testing, all the use case numbers in the historical test form (All TestCase) are used to retrieve the requirements in the use case requirement coverage matrix (Traceability), and the retrieved requirements are updated with the results taken from the use cases according to the prescribed format (use case number-use case result-use case notes). And if it is the first round of testing, the use case results corresponding to the requirements will be fully covered; if it is a regression test, the test evidence will only update the use case results of this round. As Figure 3As shown, the test evidence of the subsystem is classified as follows: Here, taking the subsystem (Equipment test) as an example, there are 3 test reports, namely CC subsystem test report (Carbone Controller), WRC subsystem test report (Wayside Resource Controller), and WTC subsystem test report (Wayside Train Controller). The requirement numbers linked to the system level in the Requirement Text in the three subsystem reports should be compared with the requirement numbers assigned to Equipment in the system-level requirements. If there is a match, the test evidence of the three reports will be summarized in the report.

[0050] Step S3: After the system-level and subsystem-level requirement test evidence is classified and counted, the requirement conclusion needs to be determined. The requirement conclusions include KO (failed), NT (untestable), OK (passed), NA (not available in this product), and uncovered. The determination rules need to be designed with algorithms so that the final conclusion of the requirement can be determined from the classified and summarized massive requirement test evidence.

[0051] Step S4, run the tool, automatically read the requirement test evidence, determine the requirement conclusion according to the rules set by the algorithm, and automatically update it to the conclusion column of the corresponding requirement in the test report.

[0052] like Figure 4 As shown, the specific steps of step S3 are as follows:

[0053] Step S31: After completing the test evidence and allocation method classification statistics of the requirements, first determine whether the "allocation method" is NA. If the "allocation method" is NA, the requirement conclusion can be directly determined as NA;

[0054] Step S32: If the "allocation method" is not NA, further judgment is required, and the requirement test evidence results need to be classified and judged. Judge according to KO>NT>OK>NA, that is:

[0055] 1. The evidence result shows KO, and the demand conclusion is judged as KO;

[0056] 2. There is no KO in the evidence results, but there is NT, and the required conclusion is determined to be NT;

[0057] 3. There is no KO or NT in the evidence results, but there is OK, and the required conclusion is judged as OK;

[0058] 4. There is no KO, NT, or OK in the evidence results, only NA, and the required conclusion is determined to be NA;

[0059] 5. There is no KO, NT, OK, or NA in the evidence results, and the evidence results are empty, and the requirement conclusion is determined to be uncovered;

[0060] 6. If there is no KO, NT, OK, or NA in the evidence result, and the evidence result is not empty, you need to check whether the previous version of the requirement conclusion is uncovered. If the previous version of the requirement conclusion is not uncovered, the requirement conclusion is determined to be NA;

[0061] 7. There is no KO, NT, OK, or NA in the evidence results, and the evidence results are not empty, and the previous version of the requirement conclusion is uncovered. The requirement conclusion needs to maintain the result of the previous version and will not be updated;

[0062] Step S33: determine the demand conclusion according to the above algorithm, and the final conclusion is not updated according to the algorithm or is automatically updated to a new result.

[0063] Example 2

[0064] As one of the specific implementation examples of the present invention, the method for automatically collecting and determining demand conclusions in this embodiment is as follows:

[0065] Step 1: Collect test evidence for the requirements. On the Test Record page, update several system-level use case results, such as Qiji-SyRTD-1100NA; update several requirement results in the WRC subsystem report, such as [TACS-CDS-13529]FailQiji-SyRTD-1043Fail:1.1.0:TACS100000506atvcm01296270, etc.

[0066] Step 2: After completing the test evidence collection of the requirement, it is necessary to determine whether it is a system-level requirement. If so, the system-level requirement classification and statistical method needs to be used; if not, the subsystem-level requirement classification and statistical method needs to be used;

[0067] refer to Figure 2At the system level, it is necessary to make a first-round or regression judgment on the evidence. In the first round, all the evidence will be updated, and in the regression, only the specified content will be updated. At the subsystem level, since the conclusions of the corresponding report need to be imported into the system-level report, this step requires selecting the correct file format and selecting the system report file. Classify and count the test evidence and allocation methods of the requirements. Taking the system level as an example, there are 3 use cases associated with the requirement Qiji-SYSDES-11713, namely Qiji-SyRTD-1100, Qiji-SyRTD-1106, and Qiji-SyRTD-1113. The test results of all the associated use cases in the test record are summarized one by one, and Qiji-SyRTD-1100NA, Qiji-SyRTD-1106NA, and Qiji-SyRTD-1113NA are obtained. Reference Figure 3 ,Subsystem-level requirement evidence is also collected in the same way.

[0068] Step 3: Design the requirement conclusion judgment rules so that the classified and statistically tested evidence can be used to make conclusion judgments;

[0069] Step 4: Automatically determine the requirements based on the rules set by the algorithm, and automatically update the requirements conclusion into the report. The conclusion is that Qiji-SYSDES-11713 is NA and Qiji-SYSDES-20968 is KO;

[0070] After the evidence is collected, the requirement conclusion judgment begins. The requirement conclusions include KO (failed), NT (untestable), OK (passed), NA (not available in this product), and uncovered. When the allocation method is directly NA (that is, the requirement is not available in this product), the requirement conclusion can be directly judged as NA; when the requirement allocation method is not NA, it will enter the requirement test evidence result classification judgment branch. Here is the example.

[0071] As shown in the above steps, the system-level test evidence is that all three use cases are NA. Figure 4 In the process judgment, whether the "allocation method" is NA: No; go to the next step, whether the evidence result is KO, No; go to the next step, whether the evidence result is NT, No; go to the next step, whether the evidence result is OK, No; go to the next step, whether the evidence result is NA, Yes; the requirement conclusion is judged to be NA; there are a lot of test evidences at the subsystem level, follow the results in the evidence according to the flow chart, whether the "allocation method" is NA: No; go to the next step, whether the evidence result is KO, Yes; the requirement conclusion is judged to be KO;

[0072] Based on the demand results determined in the previous step, update the demand conclusion NA / KO in the corresponding demand row in the report.

[0073] At the system level, it is necessary to make a first-round or regression judgment on the evidence. In the first round, all evidence will be updated, while in the regression, only the specified content will be updated. At the subsystem level, since the conclusions of the corresponding report need to be imported into the system-level report, this step requires selecting the correct file format and selecting the system report file.

[0074] Example 2

[0075] As a second aspect of the present invention, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method of automatically collecting and determining the above-mentioned demand conclusions. In addition to the above-mentioned processors, memories, and interfaces, any device with data processing capabilities in which the device in the embodiment is located may also include other hardware, usually according to the actual functions of the device with data processing capabilities, which will not be described in detail.

[0076] The processor may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of processors include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor performs the various methods and processes described above, such as methods of image processing and / or methods of model training. For example, in some embodiments, the methods of image processing and / or methods of model training may be implemented as computer software programs, which are tangibly contained in machine-readable media, such as storage units. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the processor, one or more steps of the methods of image processing and / or methods of model training described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform methods of image processing and / or methods of model training by any other appropriate means (e.g., by means of firmware).

[0077] Example 3

[0078] As a third aspect of the present invention, the present application also provides a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by the processor, the method for automatically collecting and determining the above-mentioned demand conclusions is realized. The machine-readable medium can be a tangible medium, which can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0079] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for automatically collecting and determining demand conclusions, characterized in that the steps include: After the testing work is completed, obtain the use case results of system-level testing and subsystem testing; Based on the use case results, the requirement test evidence and distribution methods of system-level and subsystem-level tests are classified and counted; For the requirement test evidence that has been classified and counted, the requirement conclusion is determined according to the set rules, and the conclusion of the test report corresponding to the requirement is updated based on the requirement conclusion.

2. The method for automatically collecting and determining demand conclusions according to claim 1, characterized in that: The specific results of the use cases for obtaining the system-level test are as follows: The test conclusion of the system-level test is updated to the test results of this round in a prescribed format. In the test form of this round, the test case number is used as an index to update the corresponding test evidence and the test results of the case.

3. The method for automatically collecting and determining demand conclusions according to claim 1 is characterized in that: The specific results of the use cases for obtaining the subsystem level test are as follows: The test conclusion of the subsystem level test is indexed by the test case number, and the corresponding test evidence and the test result of the case are updated in the subsystem test report.

4. The method for automatically collecting and determining demand conclusions according to claim 1 is characterized in that: When classifying and counting the system-level tests, the evidence is first judged as to whether it is a first-round test or a regression test: if it is a first-round test, all the use case results corresponding to the requirements are covered; If it is a regression test, the test evidence will only update the use case results of this round.

5. The method for automatically collecting and determining demand conclusions according to claim 4 is characterized in that: For the first round of testing, the requirements in the use case requirement coverage matrix are retrieved based on all use case numbers in the historical test form, and the retrieved requirements are updated in accordance with the prescribed format to the results taken from the use cases.

6. A method for automatically collecting and determining demand conclusions according to claim 4, characterized in that: For the regression test, the test evidence will only update the use case results of this round, using all the use case numbers in the test form of this round to retrieve the requirements in the use case requirement coverage matrix. For all retrieved requirements, the use case results corresponding to the requirements will be updated in the specified format.

7. A method for automatically collecting and determining demand conclusions according to claim 5 or 6, characterized in that: The format for updating the results of the first round of testing and regression testing cases is: case number-case result-case remarks.

8. The method for automatically collecting and determining demand conclusions according to claim 1, characterized in that: The test evidence of the subsystem-level test is classified as follows: the requirement numbers linked to the system level in the Requirement Text in multiple subsystem reports are compared one by one with the requirement numbers assigned to Equipment in the system-level requirements. If there is a match, the test evidence of each subsystem report is summarized in the report.

9. The method for automatically collecting and determining demand conclusions according to claim 1, characterized in that: The requirement conclusions include: failed KO, unmeasurable NT, passed OK, the product does not have NA, and uncovered.

10. A method for automatically collecting and determining demand conclusions according to claim 9, characterized in that: The conclusion of determining the demand according to the set rules is specifically: After completing the classification and statistics of the test evidence of the requirements, determine whether the allocation method is that the product does not have NA. If the requirement conclusion is that the product does not have NA, then the requirement conclusion is directly determined as the product does not have NA; If the allocation method is not product does not have NA, the requirement test evidence results are further classified in the order of fail KO greater than untestable NT greater than pass OK greater than product does not have NA.

11. A method for automatically collecting and determining demand conclusions according to claim 10, characterized in that: The requirements test evidence results are further classified and determined as follows: If the evidence result fails to pass the KO, the requirement conclusion is determined to fail the KO; There is no KO in the evidence results, but there is untestable NT, and the required conclusion is determined to be untestable NT; There is no KO in the evidence results, no untestable NT, but there is OK, and the requirement conclusion is judged as OK; There is no KO in the evidence results, no NT in the untestable results, no OK in the test results, only NA in the product, and the demand conclusion is that the product does not have NA. There is no failure KO, no unmeasurable NT, no pass OK, no product not possess NA in the evidence results, and the evidence result is empty, and the demand conclusion is judged to be uncovered.

12. A method for automatically collecting and determining demand conclusions according to claim 11, characterized in that: When further determining the classification of the evidence results of the requirement test: If there is no failure KO, no unmeasurable NT, no pass OK, no product not having NA in the evidence result, and the evidence result is not empty, then check whether the previous version requirement conclusion is uncovered. If the previous version requirement conclusion is not uncovered, the requirement conclusion is determined to be product not having NA.

13. The method for automatically collecting and determining demand conclusions according to claim 11, characterized in that: When further determining the classification of the evidence results of the requirement test: There is no failure KO, no unmeasurable NT, no pass OK, no product not possessing NA in the evidence results, and the evidence results are not empty, and the previous version of the requirement conclusion is uncovered, the requirement conclusion remains the result of the previous version and will not be updated.

14. A method for automatically collecting and determining demand conclusions according to any one of claims 9 to 13, characterized in that: The method does not update or automatically updates to a new result based on the determination requirement conclusion.

15. A device for automatically collecting and determining demand conclusions, comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, it implements the method for automatically collecting and determining demand conclusions as described in any one of claims 1-14.

16. A storage medium having a program stored thereon, characterized in that: When the program is executed, the method for automatically collecting and determining demand conclusions as described in any one of claims 1-14 is implemented.

Images