Signal-oriented test strategy verification method and system

Abstract

The invention relates to the technical field of automatic testing, and discloses a signal-oriented test strategy verification method and system.The signal-oriented test strategy verification method comprises the steps that firstly, test items and corresponding mapping thereof are loaded according to test requirements, and the test items further comprise a test strategy set and a corresponding test station model thereof; then, test signals involved in the test strategy set are extracted, signal correctness verification, grammar correctness verification and mapping relation missing verification are conducted on the test signals in sequence, and finally test strategy rationality verification is completed. According to the method, the rationality of the existing test strategy can be checked, and the correctness and safety of the test strategy are improved, so that the problems of test efficiency, test safety and the like caused by the test strategy can be avoided as far as possible.

Description

Technical Field

[0001] The present invention relates to the field of automated testing technology, and more particularly to a signal-oriented testing strategy verification method and system. Background Art

[0002] As the basis for automatic testing, the quality of the test strategy directly determines whether the test task can be executed normally. Therefore, the test strategy should be verified before execution to discover problems such as unreasonable design, writing errors, and test signal coverage that does not meet requirements in advance. This can effectively avoid problems such as delays in test progress and test safety of the tested product caused by the strategy during the test execution process. Among the existing test strategy-related technologies, research mainly focuses on the automatic generation of test cases and the improvement of generation efficiency, and lacks consideration of test case verification. The few studies conducted from the perspective of test case verification only implement the verification of the description specifications of the strategy development script language, without considering the correlation between the test case and the object under test. Summary of the invention

[0003] The present invention proposes a signal-oriented test strategy verification method and system. Based on the present invention, the deficiencies of the current test strategy and the deviations from the actual signal requirements to be tested can be discovered in advance, thereby improving the correctness and safety of the test strategy during the test process, thereby avoiding problems such as test efficiency and test safety caused by the test strategy as much as possible.

[0004] In order to achieve the above-mentioned object of the invention, the technical solution of the present invention is as follows: The present invention discloses a signal-oriented test strategy verification method, comprising the following steps: Step S1. Based on the test requirements, load the test items and their corresponding mappings, wherein the test items include a test strategy set and its corresponding test station model; Step S2: Extract the test signals involved in the test strategy set and complete the test strategy rationality check, including signal correctness check, syntax correctness check and mapping relationship missing check.

[0005] Step S3. Extract the description file in the test strategy set and calculate the coverage rate TRCover based on the preset effective test requirement Set And preset effective test signal coverage TRSCover Set , complete the test strategy coverage verification.

[0006] Preferably, in step S2, signal correctness check, syntax correctness check and mapping relationship missing check are performed in sequence, and finally the test strategy rationality check is completed.

[0007] Preferably, in step S2, the signal correctness verification includes: based on the extracted test signal, comparing it with the signals in the bound UUT model, and determining whether the signals involved in the current strategy exist in the bound UUT model. If all exist, the verification passes; otherwise, it fails.

[0008] Preferably, in step S2, the syntax correctness verification includes: extracting the syntax keywords involved in the test strategy set, and retrieving and matching the keywords in the syntax toolbox. If the match is consistent, the verification passes; otherwise, it fails.

[0009] Preferably, when performing the syntax correctness verification, if the format of the extracted syntax keywords is consistent with the format in the syntax toolbox, the verification passes; otherwise, it fails.

[0010] Preferably, in step S2, the mapping relationship missing verification includes: automatically extracting the key information related to mapping in the strategy set through the send / receive signal keywords, and automatically extracting the internal ICA, ITA, and instrument information of the test station model, and comparing them with the resource mapping relationship description corresponding to the currently loaded test project. If the description is consistent, the verification passes; otherwise, it fails.

[0011] Preferably, the test strategy coverage verification specifically includes the following steps: Step X1. Select the test project to be verified for coverage, then automatically extract the test strategy set description file contained therein, and further automatically extract the bound UUT model, the corresponding test requirement information set, and the test signal set related to the UUT under test in the strategy set; Step X2. For the test requirement set and the test signal set related to the UUT under test included in the current test strategy set, perform a deduplication operation to delete duplicate information respectively, and then count the number of requirements TR TS and the number of signals TRS TS ; Step X3. Based on the test requirements, calculate the actual effective test requirement coverage rate TRCover and / or the actual effective test signal coverage rate TRSCover; Step X4. Compare the actual effective test requirement coverage rate TRCover with the preset effective test requirement coverage rate TRCover Set , and the actual effective test signal coverage rate TRSCover with the preset effective test signal coverage rate TRSCover Set , and judge the test integrity of the current test strategy for the entire UUT object to be tested; Step X5. When the actual effective test requirement coverage rate TRCover ≥ the preset effective test requirement coverage rate TRCover Set, or the actual effective test signal coverage rate TRSCover ≥ the preset effective test signal coverage rate TRSCover Set , indicating that the current test strategy meets the preset coverage requirements; otherwise, it does not meet the requirements.

[0012] Based on the same inventive concept, on the other hand, the present invention also discloses a test strategy verification system for signals, which is used to implement the above-mentioned test strategy verification method for signals, including: An initial parameter loading module, used to load the test item loading and its corresponding mapping; A rationality verification module, used to extract the test signals involved in the test strategy set and complete the rationality verification of the test strategy; A coverage verification module, used to extract the description file in the test strategy set and, based on the preset effective test requirement coverage rate TRCover Set and the preset effective test signal coverage rate TRSCover Set , complete the coverage verification of the test strategy.

[0013] Preferably, the initialization parameter module includes: A test strategy set loading unit, used to automatically load the UUT model description file and the test requirement file of the object under test; A test station model loading unit, used to automatically execute the loading of ICA, ITA, and instrument models.

[0014] Preferably, the rationality verification module includes: A signal correctness verification unit, used to automatically extract the product-related test signals involved in the test case set by keyword recognition and compare them with the signals in the bound UUT model to determine whether the test signals are correct; A syntax correctness verification unit, used to determine whether the description corresponding to the syntax keyword in the currently extracted test strategy set conforms to the description specification defined by the syntax toolbox to determine whether the test strategy syntax is correct; A mapping relationship missing verification unit, which automatically extracts the key information related to mapping in the strategy set by sending / receiving signal keywords, and automatically extracts the ICA, ITA, and instrument information inside the test station model, and compares them with the resource mapping relationship description corresponding to the currently loaded test item to determine whether the mapping relationship of the test strategy is missing.

[0015] Preferably, the coverage verification module includes: A parameter selection unit, used to select the test item to be verified for coverage, as well as the preset effective test requirement coverage rate TRCoverSet and the test effective test signal coverage rate TRSCoverSet; A data extraction unit is used to automatically extract the test strategy set description file contained according to the selected items to be tested, so as to automatically extract the bound UUT model, the corresponding test requirement information set, and the test signal set related to the UUT under test in the strategy set; A coverage calculation unit is used to calculate the actual effective test requirement coverage rate TRCover or the actual effective test signal coverage rate; An integrity judgment unit is used to judge the test integrity of the current test strategy for the entire UUT object to be tested by comparing the actual effective test requirement coverage rate TRCoverSet with the preset test requirement coverage rate TRCover, or comparing the actual effective test signal coverage rate TRSCoverSet with the preset test signal coverage rate TRSCover.

[0016] On the other hand, the present invention also discloses a computer device, including a memory, a processor, and a computer program stored on the memory and executable in the processor. When the processor executes the computer program, the above-mentioned signal-oriented test strategy verification method is implemented.

[0017] On another aspect, the present invention also discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed in a computer processor, the above-mentioned signal-oriented test strategy verification method is implemented.

[0018] Advantages of the present invention: The present invention realizes the detection of the test signal coverage rate, and at the same time can effectively improve the probability of early exposure of problems in the test process caused by test strategies in signal-oriented ATS, and improve the test efficiency and test safety of the airborne product to be tested. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The foregoing and following specific descriptions of the present invention become clearer when read in conjunction with the following drawings, in which: Figure 1 is a flowchart of the method of the present invention; Figure 2 is a schematic diagram of the system of the present invention; Figure 3 is a flowchart of the rationality verification of the test strategy of the present invention; Figure 4 is a flowchart of the coverage verification of the test strategy of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] To enable those skilled in the art to better understand the technical solutions in the present invention, the following will further illustrate the technical solutions for achieving the objectives of the present invention through specific embodiments. It should be noted that the technical solutions claimed in the present invention include but are not limited to the following embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

[0021] An embodiment of the present invention first proposes a test strategy verification method for signals. Figure 1 This is the general flowchart of the method of the present invention. Refer to the attached Figure 1 The test strategy verification method mainly includes the following steps: Step S1. Initial parameter loading The initial parameter loading is based on the actual test requirements. The system completes the loading of test items and their corresponding mappings. The test item loading includes the loading of the test strategy set and the corresponding test station model, jointly providing a data basis for subsequent verification.

[0022] Step S2. Rationality verification of the test strategy The rationality verification of the test strategy mainly completes the judgment of signal correctness, grammar correctness, and the judgment of mapping relationship deficiency, discovers problems existing in the test strategy in advance, and ensures the correctness and security of the test strategy during the test process, thereby ensuring the smooth execution of the test task and improving the test efficiency and test security.

[0023] Step S3. Coverage verification of the test strategy The coverage verification of the test strategy realizes the deviation calculation between the measured signal and the actual signal to be measured requirements, providing an index basis for determining whether the test task is complete. This step can be implemented according to the actual needs of the user as required.

[0024] The following will elaborate on and explain the above three verification steps in detail as follows.

[0025] First, regarding Step S1. Initial parameter loading, the following content is as follows: 1) Test item loading (a) Loading of the test strategy set: Since the test strategy is bound to the UUT model of the device under test and the corresponding test requirements, when the system executes the loading of the test strategy set, it not only completes the loading of all test strategies under the UUT product of the device under test, but also automatically loads the UUT model description file and the test requirement file of the device under test. The UUT model description file contains all ICD (Interface Control Document) signals of the UUT model; (b)Test station model loading: Since the test station model contains ICA model, ITA model, instrument model, etc., the system will automatically execute the loading of ICA, ITA and instrument models while loading the test station model.

[0026] 2) Loading of the corresponding mapping relationship file The system selects the resource mapping relationship file corresponding to the test project from the mapping management library to complete the loading of the mapping relationship.

[0027] Step S2. Rationality verification of the test strategy Figure 3 This is the flowchart for the rationality verification of the test strategy of the present invention. Referring to the attached Figure 3 shown in the specification, the specific verification process is as follows: 1) Signal correctness verification: Automatically extract the test signals related to the product involved in the test strategy set through keyword recognition, and compare them with the signals in the bound UUT model to determine whether the signals involved in the current strategy exist in the bound UUT model. If all exist, it means that the signal correctness verification is passed; otherwise, the verification fails. When the verification fails, the current strategy can be quickly modified through the located error position to meet the signal requirements in the bound UUT model. Signal correctness verification can effectively ensure the correctness of the test strategy. Especially in the scenario of reusable test cases, it can effectively accelerate the modification efficiency of the test strategy, improve the development efficiency of the test strategy, and thus improve the test efficiency.

[0028] 2) On the basis of passing the signal correctness verification, further complete the verification of the syntax correctness of the test strategy, details are as follows: First, extract the syntax keywords involved in the strategy set and retrieve and match them in the defined syntax toolbox. If the match is unsuccessful, it means that the rationality verification of the current strategy fails, and the strategy needs to be quickly modified according to the error position; otherwise, further syntax verification of the strategy is performed; Furthermore, the main principles of syntax verification are as follows: Step a. Since the test strategy usually exists in XML form, first judge whether the current strategy file meets the XML format in the syntax correctness verification. If it meets, proceed to the next step; otherwise, modify the file format; Step b. The description or corresponding format corresponding to the syntax keywords in the test strategy should follow the description specifications defined in the syntax toolbox. If the two match, it means that the syntax correctness verification is passed; otherwise, the verification fails. Taking "Wait" in the syntax toolbox as an example, its description specification should include the syntax keyword "Wait duration=" and the specific time value, and its value must be enclosed in quotes, etc.; or taking "Send data" as an example, its description specification is as follows: <senddata alias="" comment="" source=" " type=" " value="" / >, where the source describes the detailed information of the signal to be sent, which can be a single signal description or a cascaded description of multiple signals. The description of a single signal should include the protocol name, ICD data block name, ICD domain name, and ICD signal name, and be separated by an English semicolon; different signals are distinguished by "|", etc.

[0029] 3) After passing the above verification, automatically extract the key information related to mapping in the send / receive signal keyword automatic extraction policy set (such as: protocol name, signal type, etc.), as well as automatically extract the ICA, ITA, and instrument information inside the test station model, and compare it with the resource mapping relationship description corresponding to the currently loaded test project, to achieve a preliminary judgment on the missing mapping relationship corresponding to this policy set. If there is a situation where the descriptions are inconsistent, it means that the rationality verification fails, and the policy needs to be quickly adjusted according to the error. If they are consistent, it means that the rationality verification of the current policy set passes. While ensuring the correctness and safety of the test policy, it effectively improves the matching rate of this test policy set and the current test task, etc.

[0030] Step S3. Test policy coverage verification Figure 4 This is the flowchart of the test policy rationality verification of the present invention. Refer to the attached Figure 4 description in the specification. The specific verification process is as follows: 1) First, select the test project to be verified for coverage, and then automatically extract the test policy set description file contained in the selected project to be tested, and preset the effective test requirement coverage rate TRCover Set and the effective test signal coverage rate TRSCover Set ; 2) Based on the extracted test policy set description file, automatically extract the bound UUT model, the corresponding test requirement information set, and the test signal set related to the UUT under test in the policy set; 3) For the bound UUT model, count the number of test requirement information TR UUT and the number of test signals TRS UUT in this UUT model. For the test requirement set included in the current test policy set, delete the same test requirement information through deduplication, and then count its requirement number TR TS . For the test signal set related to the UUT under test in the policy set, delete the same test signal information through deduplication, and then count its signal number TRS TS ; 4) Calculate the actual effective test requirement coverage rate TRCover and the actual effective test signal coverage rate TRSCover. The calculation formulas are as follows: TRCover = TR TS / TRUUT *100%; TRSCover = TRS TS / TRS UUT *100%; 5) Compare the actual effective test requirement coverage rate TRCover and the preset effective test requirement coverage rate TRCoverSet, or the actual effective test signal coverage rate TRSCover and the preset effective test signal coverage rate TRSCoverSet, and determine the test integrity of the entire UUT under test for the current test strategy; The comparison principle is: when the actual effective test requirement coverage rate TRCover ≥ the preset effective test requirement coverage rate TRCoverSet, or the actual effective test signal coverage rate TRSCover ≥ the preset effective test signal coverage rate TRSCoverSet, it means that the current test strategy meets the preset coverage requirements; otherwise, it does not meet the requirements and fails the verification.

[0031] Based on the same inventive concept, the embodiments of the present invention also disclose a signal-oriented test strategy verification system. Since the principle of solving problems by this system is similar to that of the test strategy verification method, the implementation of this system can refer to the implementation of the method, and the repeated parts will not be described again. Hereinafter, the term "unit" or "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and contemplated. Figure 2 It is a schematic structural diagram of the test strategy verification system provided by the embodiments of the present invention, as Figure 2 shown, the system may include: An initial parameter loading module, configured to load the test project loading and its corresponding mapping; A rationality verification module, configured to extract the test signals involved in the test strategy set and complete the rationality verification of the test strategy; A coverage verification module, configured to extract the description file in the test strategy set and complete the coverage verification of the test strategy based on the preset effective test requirement coverage rate TRCoverSet and the preset effective test signal coverage rate TRSCoverSet.

[0032] Further, the initialization parameter module further includes: A test strategy set loading unit, configured to automatically load the UUT model description file of the object under test and the test requirement file; A test station model loading unit, configured to automatically execute the loading of ICA, ITA, and the instrument model.

[0033] Further, the rationality verification module further includes: A signal correctness verification unit, which is used to automatically extract the test signals related to the product involved in the test case set through keyword recognition, and compare them with the signals in the bound UUT model to determine whether the test signals are correct; A syntax correctness verification unit, which is used to determine whether the description corresponding to the syntax keywords in the currently extracted test strategy set conforms to the description specifications defined in the syntax toolbox, and determine whether the test strategy syntax is correct; A mapping relationship missing verification unit, which automatically extracts the key information related to mapping in the strategy set through sending / receiving signal keywords, and automatically extracts the internal ICA, ITA and instrument information of the test station model, and compares them with the resource mapping relationship description corresponding to the currently loaded test project to determine whether the mapping relationship of the test strategy is missing.

[0034] Furthermore, the coverage verification module further includes: A parameter selection unit, which is used to select the test project to be verified for coverage, and preset the effective test requirement coverage rate TRCoverSet and the effective test signal coverage rate TRSCoverSet of the test; A data extraction unit, which is used to automatically extract the test strategy set description file contained according to the selected item to be tested, so as to automatically extract the bound UUT model, the corresponding test requirement information set, and the test signal set related to the UUT under test in the strategy set; A coverage rate calculation unit, which is used to calculate the actual effective test requirement coverage rate TRCover or the actual effective test signal coverage rate; An integrity judgment unit, which is used to judge the test integrity of the current test strategy for the entire UUT object to be tested by comparing the actual effective test requirement coverage rate TRCoverSet with the preset test requirement coverage rate TRCover, or comparing the actual effective test signal coverage rate TRSCoverSet with the preset test signal coverage rate TRSCover.

[0035] It should be noted that the systems, devices, models or units described in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. For the convenience of description, in this specification, when describing the above systems, various units are described separately according to functions. Of course, when implementing the present invention, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0036] In addition, in this specification, adjectives such as first and second can only be used to distinguish one element or action, and do not necessarily imply any actual such relationship or order.

[0037] Furthermore, on another aspect of this embodiment, a computer device is further provided. The computer device includes a processor, an input device, an output device, and a memory, and the processor, the input device, the output device, and the memory are interconnected. Among them, the memory is used to store a computer program, and the computer program includes program instructions. The processor is configured to call the program instructions to execute the steps in the above embodiment.

[0038] Furthermore, on yet another aspect of this embodiment, a computer-readable storage medium is further provided, characterized in that: the computer-readable storage medium stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the steps in the above embodiment.

[0039] In this embodiment, the processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., or a combination of the above types of chips.

[0040] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and units, such as the corresponding program units in the above method embodiment of the present invention. The processor runs the non-transitory software programs, instructions, and modules stored in the memory, thereby executing various functional applications and work data processing of the processor, that is, implementing the method in the above method embodiment.

[0041] The memory may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created by the processor, etc. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0042] The one or more units are stored in the memory and, when executed by the processor, perform the method in the above embodiments.

[0043] Those skilled in the art will understand that the embodiments of the present invention may be provided as a method, a system, or a computer program product. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects.

[0044] As mentioned above, the above are only the preferred embodiments of the present invention and do not impose any form of limitation on the present invention. Any simple modification or equivalent change made to the above embodiments based on the technical essence of the present invention falls within the protection scope of the present invention.

Claims

1. A signal-oriented test strategy verification method, characterized in that: The following steps are involved: Step S1. Based on the test requirements, load the test items and their corresponding mappings, wherein the test items include a test strategy set and its corresponding test station model; Step S2: Extract the test signals involved in the test strategy set and complete the test strategy rationality check, including signal correctness check, syntax correctness check and mapping relationship missing check.

2. A signal-oriented test strategy verification method according to claim 1, characterized in that: The method further comprises: step S3. extracting the description file in the test strategy set, and based on the preset effective test requirement coverage TRCover Set And preset effective test signal coverage TRSCover Set , complete the test strategy coverage verification.

3. A signal-oriented test strategy verification method according to claim 1, characterized in that: In step S2, the signal correctness check, syntax correctness check and mapping relationship missing check are performed in sequence, and finally the test strategy rationality check is completed.

4. A signal-oriented test strategy verification method according to claim 1, characterized in that: In step S2, the signal correctness check includes: based on the extracted test signal, comparing it with the signal in the bound UUT model, judging whether the signal involved in the current strategy exists in the bound UUT model, if all exist, then the check passes, otherwise it fails.

5. A signal-oriented test strategy verification method according to claim 1, characterized in that: In step S2, the grammatical correctness check includes: extracting grammatical keywords involved in the test strategy set, searching and matching the keywords in the grammar toolbox, and if the match is consistent, the check passes, otherwise it fails.

6. A signal-oriented test strategy verification method according to claim 5, characterized in that: When checking the grammatical correctness, if the format of the extracted grammatical keywords is consistent with the format in the grammar toolbox, the check passes, otherwise it fails.

7. A signal-oriented test strategy verification method according to claim 1, characterized in that: In step S2, the mapping relationship missing check includes: automatically extracting key information related to mapping in the strategy set by sending / receiving signal keywords, and automatically extracting ICA, ITA and instrument information inside the test station model, and comparing it with the resource mapping relationship description corresponding to the currently loaded test item. If the description is consistent, the check passes, otherwise it fails.

8. A signal-oriented test strategy verification method according to claim 2, characterized in that: The test strategy coverage verification specifically includes the following steps: Step X1. Select the test item to be covered, then automatically extract the test strategy set description file, and further automatically extract the bound UUT model, the corresponding test requirement information set, and the test signal set related to the UUT under test in the strategy set; Step X2. Perform deduplication operations on the test requirement set contained in the current test strategy set and the test signal set related to the UUT under test to remove duplicate information, and then count the number of requirements TR respectively. TS Sum signal number TRS TS ; Step X3. Calculate the actual effective test requirement coverage TRCover and the actual effective test signal coverage TRSCover respectively; Step X4. Compare the actual effective test requirement coverage TRCover and the preset effective test requirement coverage TRCoverSet, or the actual effective test signal coverage TRSCover and the preset effective test signal coverage TRSCoverSet, to determine the test integrity of the current test strategy for the entire UUT object to be tested; Step X5. When the actual effective test requirement coverage TRCover≥the preset effective test requirement coverage TRCoverSet, or the actual effective test signal coverage TRSCover≥the preset effective test signal coverage TRSCoverSet, it indicates that the current test strategy meets the preset coverage requirements.

9. A signal-oriented test strategy verification system, the system being used to implement the signal-oriented test strategy verification method according to any one of claims 2 to 8, characterized in that: include: Initial parameter loading module, used to load test items and their corresponding mappings; A rationality check module is used to extract the test signals involved in the test strategy set and complete the rationality check of the test strategy; The coverage verification module is used to extract the description files in the test strategy set and calculate the coverage rate TRCover based on the preset effective test requirements. Set And preset effective test signal coverage TRSCover Set , complete the test strategy coverage verification.

10. A computer device, characterized in that: The system comprises a memory, a processor and a computer program stored in the memory and executable in the processor. When the processor executes the computer program, the signal-oriented test strategy verification method is implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed in a computer processor, the signal-oriented test strategy verification method is implemented.

Images