Use case testing method, system, device and storage medium

Abstract

The present invention discloses a use case testing method, system, device and storage medium, the method comprising: a use case generation thread sequentially generates multiple use cases according to a preset order; at least one use case testing thread sequentially implements the multiple use cases to complete the test of the object under test; wherein the use case generation thread and the at least one use case testing thread run in parallel in a time series. The present invention realizes the improvement of automated testing efficiency by executing the test steps of different use cases in parallel in a pipeline manner.

Description

Technical Field

[0001] The present invention relates to the field of testing, and in particular to a use case testing method, system, device and storage medium. Background Art

[0002] With the development of science and technology, it is becoming more and more important to perform performance testing on hardware and / or software products, which puts higher requirements on test efficiency. For example, in 5G wireless communication, the current 5G NR (new radio) physical layer test process is generally divided into the following three steps: Step 1, use a vector generation tool to generate a test vector for a test case according to the test requirements; Step 2, inject the test vector into the object under test through a simulation tool; Step 3, after the object under test outputs the result, use the result analysis tool to analyze the result and obtain the test result.

[0003] In the prior art, in order to improve the test efficiency, some automation means are usually used for the above test process, for example, the above test process is repeatedly executed one vector at a time. However, due to the particularity of the 5G NR physical layer test, a large number of mathematical operations will be performed during the execution of each step, so each step takes longer time than product testing in other fields (for example, the Internet field). At the same time, 5G NR testing usually tests the performance of embedded hardware products with limited resources. How to improve the efficiency of physical layer testing under limited hardware resources has become a problem that needs to be solved urgently. Summary of the invention

[0004] In view of the above defects of the prior art, the present invention provides a use case testing method, system, device and storage medium, which execute the test steps of multiple use cases in parallel in a pipeline manner to improve the efficiency of automated testing.

[0005] To achieve the above-mentioned purpose, an embodiment of the present invention provides a use case testing method, which includes: a use case generation thread generates multiple use cases in sequence according to a preset order; at least one use case testing thread implements the multiple use cases in sequence to complete the test of the object under test; wherein the use case generation thread and the at least one use case testing thread run in parallel in a time series.

[0006] An embodiment of the present invention also provides a use case testing system, which includes: a use case generation unit, used to generate multiple use cases in sequence according to a preset order; at least one use case testing unit, used to implement the multiple use cases in sequence to complete the test of the object under test; wherein the use case generation unit and the at least one use case testing unit run in parallel on a time series.

[0007] An embodiment of the present invention further provides a computer device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the steps in the method described in any of the above embodiments are implemented.

[0008] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the method described in any of the above embodiments are implemented.

[0009] The advantage of the present invention is that by executing the test steps of multiple use cases in parallel in a pipeline manner, the total test time of the multiple use cases is shortened, thereby improving the test efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The technical solutions and other beneficial effects of the present invention will be made apparent by describing in detail the specific embodiments of the present invention in conjunction with the accompanying drawings.

[0011] Figure 1 A schematic flow chart of a use case testing method provided by an embodiment of the present invention is shown.

[0012] Figure 2 Shows the use of Figure 1 The use case test method shown executes the timing diagram of the use case test.

[0013] Figure 3 The structure block diagram of the use case testing system provided by the embodiment of the present invention is shown.

[0014] Figure 4 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0016] The terms "first", "second", "third", etc. (if any) in the specification and claims of the present invention and the drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the objects described in this way can be interchanged where appropriate. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware circuits or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0017] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] Figure 1 A flow chart of a use case test method provided in an embodiment of the present invention is shown. In this embodiment, the use case test method can be applied to a 5G NR physical layer test scenario. In other embodiments, the use case test method can also be applied to other scenarios for performance testing of hardware and / or software products, and this application does not limit this.

[0019] Specifically, the steps of the use case testing method include:

[0020] Step S10, the use case generation thread generates multiple use cases in sequence according to a preset order;

[0021] Step S20: at least one use case test thread sequentially implements the multiple use cases to complete the test of the object under test, wherein the use case generation thread and the at least one use case test thread run in parallel in a time series.

[0022] Specifically, in step S10, each of the use cases defines a test vector, wherein the test vector includes a set of data for testing the performance of the object under test. Exemplarily, in a 5G NR physical layer test scenario, the test vector includes both configuration parameters of the object under test and reference output data for comparison with the output value of the object under test.

[0023] In step S20, the at least one use case test thread includes a first test thread and a second test thread, wherein the first test thread and the second test thread run in parallel in a time series.

[0024] Specifically, the first test thread receives the multiple use cases and sends the multiple test vectors defined by the multiple use cases to the object under test in sequence according to a preset time sequence and a preset format to trigger the object under test to perform actions based on the test vectors and output actual operating results; the second test thread receives the actual operating results output by the object under test and analyzes the actual operating results in sequence according to the chronological order to obtain the test results.

[0025] Specifically, the first test thread simulates the upper node of the object under test to convert the test vector into a signal with a preset format that can be recognized by the object under test, and then the upper node sends the signal to the object under test, where the signal includes instructions instructing the object under test to perform an action.

[0026] In this embodiment, when the first test thread is running, the use case generation thread generates multiple use cases in sequence according to a preset order. When the second test thread is running, the use case generation thread generates multiple use cases in sequence according to a preset order, and the first test thread sends multiple test vectors defined by the multiple use cases to the object under test.

[0027] Figure 2 Shows the use of Figure 1 The use case test method shown in the figure performs a timing diagram of the use case test. Specifically, in this embodiment, taking the 5G NR physical layer test process as an example, the test process includes step one, step two and step three, wherein step one is run on thread A, step two is run on thread B, and step three is run on thread C. Among them, step one includes: using a vector generation tool to generate a test vector for a certain use case according to the test requirements; step two includes: injecting (i.e., sending) the test vector into the object under test through a simulation tool, and outputting the actual operation result; step three includes: after the object under test outputs the actual operation result, using the result analysis tool to analyze the actual operation result, and then obtain the test result.

[0028] Exemplarily, taking the PRACH channel test as an example, in step one, the test vector of a certain use case is generated using MATLAB software running on a WINDOWS system, and the test vector may include the following data: system configuration of the object under test (for example, system configuration of the physical layer software (L1 software) running on the Linux system on the DU (distributed unit of the base station) side), slot configuration, uplink in-phase and quadrature (IQ) data, and reference results (Reference). In step two, the upper-layer simulation tool running on the Linux system on the DU side is used to send the test vector to the object under test (L1 software) to trigger the object under test to perform an action based on the test vector and output the actual operation result; wherein the upper-layer simulation tool converts the test vector into a signaling with a preset format that can be recognized by the object under test, and then the upper-layer simulation tool sends the signaling to the object under test, and the signaling includes instructions to instruct the object under test to perform an action. In step three, after the L1 software completes processing according to the test vector issued, the actual operation result of the L1 software is sent to the MATLAB software running on the WINDOWS system for analysis to obtain the final test result; wherein, the output result of the L1 software may specifically include the following contents: a preamble sequence identification number (Preamble ID) and a timing advance (TA); the result analysis tool analyzes the actual operation result, and the analysis process may include: comparing the actual operation result of the L1 software with the Preamble ID in the reference result (Reference) to determine whether they match (bit match), and comparing the actual operation result of the L1 software with the TA in the reference result (Reference) to determine whether they match.

[0029] Assuming that step 1 of the above single use case needs to run for 5 minutes, step 2 needs to run for 6 minutes, and step 3 needs to run for 3 minutes, in the automation environment of the prior art, the three steps will be executed in sequence. For example, first run steps 1 to 3 of the first use case, and after all three steps of the first use case are executed, run steps 1 to 3 of the second use case, and so on. Then running the test process of N use cases will take time: T = N*(5+6+3) minutes. However, if the pipelined parallel use case testing method provided by the present invention is adopted, the efficiency can be further improved on the above basis. The specific steps are as follows:

[0030] First, the above three steps are transformed so that they run independently on threads A, B, and C respectively, without interfering with each other, and the three threads are aligned in timing; and files (such as files FA, FB, FC) are used as the only interface for data exchange between the steps. Specifically, the output of step one is set to file FA, and file FA is used as the input of step two; similarly, the output of step two is set to file FB, and is used as the input of step three. The output of step three is file FC, which corresponds to the test result.

[0031] Among them, the output FA after each execution of step one is uniformly numbered and recorded as FA1, FA2, ..., FAn. The output FB after each execution of step two is uniformly numbered and recorded as FB1, FB2, ..., FBn. The output FC after each execution of step three is uniformly numbered and recorded as FC1, FC2, ..., FCn. At the same time, step one, step two, and step three are independently executed on three different threads A, B, and C, where each thread includes a set of PROCx, where x = 1, 2, ..., n, which represents the execution process of specific steps of N use cases on the corresponding thread. Specifically, as Figure 2 As shown:

[0032] Thread A (i.e., the test case generation thread) executes the processing of step 1. Thread A has no standby time, so after thread A is started, it executes step 1 to generate the test vector of the first test case, that is, it outputs FA1 after processing; then it starts to generate the test vector of the second test case, that is, it outputs FA2 after processing; thread A repeats the actions of step 1 until the test vector of the Nth test case is generated, and it outputs FAn after processing.

[0033] Thread B (i.e., the first test thread) executes the processing of step 2. Thread B has a standby time (IDLE), and it needs to wait for thread A's step 1 to output FA1 of the first test case before executing step 2, and output FB1 after processing; after processing FB1, it starts to read FA2 of the second test case. In this embodiment, the processing time of step 1 is shorter than the processing time of step 2, so thread B does not need to wait after executing step 2 and outputting FB1, that is, it can read FA2 and process it, and output FB2 after processing. In other embodiments, the processing time of step 1 can also be longer than the processing time of step 2. In this case, thread B needs to experience a standby time before reading the output of step 1, so as to wait for thread A's step 1 to output FA1 before executing step 2. Thread B repeats the above reading and processing operations until it starts to read FAn of the Nth test case, and outputs FBn after processing.

[0034] Thread C (i.e., the second test thread) performs the processing of step three. After thread C waits for thread B to execute step two and output the result FB1, it starts processing after reading FB1, and outputs the test result FC1 after processing is completed; then it starts reading the output result FB2 of the object under test, and outputs the test result FC2 after processing is completed. Thread C repeats the reading and processing operations until it starts reading the output result FBn of the object under test, and outputs the test result FCn after processing is completed. In this embodiment, the processing time of step two is longer than that of step three, so thread C needs to experience a standby time before reading the output of step two, so as to wait for thread B to output FB1 in step two before executing the processing of step three. In other embodiments, the processing time of step two can also be shorter than that of step three. In this case, thread C does not need to wait after executing step three and outputting FC1, that is, it can read FB2 and perform the processing of step three, and output the test result FC2 after processing is completed.

[0035] Therefore, in this embodiment, if the processing time of step 2 is longer than that of step 1 and step 3, the time (in minutes) consumed to complete the execution of all three steps of all N use cases is:

[0036] T=5+6*N+3

[0037] Therefore, compared with the existing test process of executing multiple use cases in sequence, the test process using the embodiment of the present invention will save time as follows:

[0038] Δt=N*(5+6+3)–(5+6*N+3)=5*(N-1)+3*(N-1),

[0039] It can be seen from the above formula that as the number of use cases increases, the time saved by the use case testing method of the embodiment of the present invention will be longer and the efficiency will be improved more.

[0040] In summary, for an automated test with S steps, the total time T consumed to execute the test using the use case test method of the embodiment of the present invention can be calculated by the following formula:

[0041] T=N*Tmax+T1+T2+…+Ts-1

[0042] Among them, Tmax represents the time consumed by the longest step in a single run, N represents the total number of use cases, and T1~Ts-1 represents the time consumed by the other steps in a single run.

[0043] Figure 3The structural block diagram of the use case test system 100 provided in an embodiment of the present invention is shown. Specifically, the system 100 includes: a use case generation unit 10, which is used to sequentially generate multiple use cases in a preset order; at least one use case test unit 20, which is used to sequentially implement the multiple use cases to complete the test of the object under test; wherein the use case generation unit 10 and the at least one use case test unit 20 run in parallel in a time series.

[0044] In this embodiment, each of the use cases defines a test vector, and the test vector includes multiple test parameters.

[0045] In this embodiment, the at least one use case testing unit 20 includes a first testing element 21 and a second testing element 22 , wherein the first testing element 21 and the second testing element 22 run in parallel in a time sequence.

[0046] In this embodiment, the first test element 21 receives the multiple use cases and sends the multiple test vectors defined by the multiple use cases to the object under test in sequence according to a preset time sequence and a preset format to trigger the object under test to perform actions based on the test vectors and output actual operating results; the second test element 22 receives the actual operating results output by the object under test and analyzes the actual operating results in sequence according to the chronological order to obtain the test results.

[0047] In this embodiment, the first test element 21 is also used to simulate the upper node of the object under test to convert the test vector into a signaling with a preset format that can be recognized by the object under test, and then the upper node sends the signaling to the object under test, and the signaling includes instructions instructing the object under test to perform an action.

[0048] In this embodiment, the use case is a use case for physical layer testing of wireless communications.

[0049] In this embodiment, other specific definitions and implementation methods of the various units and elements of the system 100 are the same as or similar to the use case testing method described above, and are not described in detail here.

[0050] Figure 4The schematic diagram of the structure of the computer device 200 provided by the embodiment of the present invention is shown. The computer device includes a processor 310, a memory 320, a network interface 330, a display screen 340 and an input device 350 connected through a system bus 300. Among them, the processor 310 of the computer device 200 is used to provide computing and control capabilities. The memory 320 of the computer device 200 includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface 330 of the computer device 200 is used to communicate with an external computer device through a network connection. When the computer program is executed by the processor 310, a use case testing method is implemented. The display screen 340 of the computer device 200 can be a liquid crystal display screen or an electronic ink display screen, and the input device 350 of the computer device 200 can be a touch layer covered on the display screen 340, or a key, trackball or touchpad set on the shell of the computer device 200, or an external keyboard, touchpad or mouse, etc.

[0051] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0052] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program that can be run on the processor is stored on the memory, and the processor implements the steps of the above-mentioned use case testing method when executing the computer program. The specific definition and implementation of the above-mentioned steps can be referred to the embodiment of the above-mentioned use case testing method, which will not be repeated here.

[0053] In another embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the above-mentioned use case testing method are implemented. The specific definition and implementation of the above-mentioned steps can be referred to the embodiment of the above-mentioned use case testing method, and will not be repeated here.

[0054] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0055] The above is a detailed introduction to a use case testing method, system, device and storage medium provided in an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the technical scheme and core idea of ​​the present invention. Ordinary technicians in this field should understand that they can still modify the technical schemes recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents; and these modifications or replacements do not make the essence of the corresponding technical scheme deviate from the scope of the technical scheme of the embodiments of the present invention.

Claims

1. A use case testing method, characterized in that: The method comprises: The use case generation thread generates multiple use cases in a preset order; At least one use case test thread sequentially implements the multiple use cases to complete the test of the object under test; Wherein, the use case generation thread and the at least one use case test thread run in parallel in a time series; The at least one use case test thread includes a first test thread and a second test thread, wherein the first test thread and the second test thread run in parallel in a time series; the first test thread receives the multiple use cases and sends the multiple test vectors defined by the multiple use cases to the object under test in sequence according to a preset time sequence and a preset format to trigger the object under test to perform actions based on the test vectors and output actual operation results; the second test thread receives the actual operation results output by the object under test and analyzes the actual operation results in sequence according to the chronological order to obtain test results.

2. The use case testing method according to claim 1, characterized in that: Each of the use cases defines a test vector, and the test vector includes multiple test parameters.

3. The use case testing method according to claim 1, characterized in that: The first test thread simulates the upper node of the object under test to convert the test vector into a signaling with a preset format that can be recognized by the object under test, and then the upper node sends the signaling to the object under test, where the signaling includes instructions instructing the object under test to perform an action.

4. The use case testing method according to claim 1, characterized in that: When the first test thread is running, the use case generation thread is running simultaneously to sequentially generate the multiple use cases in a preset order.

5. The use case testing method according to claim 1, characterized in that: When the second test thread is running, the use case generation thread is running simultaneously to generate the multiple use cases in sequence according to a preset order, and the first test thread is also running simultaneously to send the multiple test vectors defined by the multiple use cases to the object under test in sequence.

6. The use case testing method according to claim 1, characterized in that: The use case is a use case for physical layer testing of wireless communications.

7. A use case testing system, characterized in that: The system comprises: A use case generation unit, used to sequentially generate multiple use cases in a preset order; At least one use case testing unit, used to sequentially implement the multiple use cases to complete the test of the object under test; wherein the use case generation unit and the at least one use case testing unit are run in parallel in a time series; The at least one use case test unit includes a first test element and a second test element, wherein the first test element and the second test element run in parallel in a time series; the first test element receives the multiple use cases and sends the multiple test vectors defined by the multiple use cases to the object under test in sequence according to a preset time sequence and a preset format to trigger the object under test to perform actions based on the test vectors and output actual operation results; the second test element receives the actual operation results output by the object under test and analyzes the actual operation results in sequence according to the chronological order to obtain test results.

8. The use case testing system according to claim 7, characterized in that: Each of the use cases defines a test vector, and the test vector includes multiple test parameters.

9. The use case testing system according to claim 7, characterized in that: The first test element is also used to simulate the upper node of the object under test to convert the test vector into a signal with a preset format that can be recognized by the object under test, and then the upper node sends the signal to the object under test, and the signal includes instructions instructing the object under test to perform an action.

10. The use case testing system according to claim 7, characterized in that: The use case is a use case for physical layer testing of wireless communications.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 6 are implemented.

Images