Communication module robustness test method, system, device, medium and program product

Abstract

The invention discloses a communication module robustness test method, system and device, a medium and a program product, and the method comprises the steps: inputting a test demand file of a communication module into a pre-trained test demand recognition model, so as to enable the test demand recognition model to output a test demand specification of the communication module; secondly, inputting the test requirement specification into a pre-trained test case design model, so that the test case design model outputs a test case; generating a corresponding test script according to the test case; and finally, carrying out robustness test on the communication module according to the test script. According to the method, the automation of test requirement identification and test case design is realized by applying the large language model, so that the efficiency of the robustness test of the communication module is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to a communication module robustness testing method, system, equipment, medium and program product. Background Art

[0002] Communication module, also known as wireless cellular communication module, refers to a wireless module that integrates baseband chip, memory, power amplifier and other components on a circuit board and has standard interface functions. It integrates components including wireless communication protocol, radio frequency (RF) circuit, processor, memory and power management, so that the device can exchange data with other devices wirelessly, connect to the network or perform positioning, sensor data transmission and other operations. In order to ensure that the communication module can maintain efficient, reliable and secure communication performance under various possible usage scenarios and conditions, it is usually necessary to perform robustness testing (Robustness Testing) on ​​the communication module, also known as fault tolerance testing (FaultTolerance Testing), which is used to test whether the communication module can automatically recover or ignore the fault and continue to operate when a fault occurs.

[0003] Traditional communication module robustness testing is mainly done by test engineers who manually identify test requirements based on product technical specifications, function definitions, application manuals, and other materials, then design a requirement specification, and then design use cases based on the requirement specification. The design method mainly involves manual verification and parameter traversal of the function according to its positive and negative responses, and then automating the use cases to form a test script, and finally running the test script to verify the robustness of the module instructions. However, during the test process, the input resource information is messy, and manual identification of test requirements and design of requirement specifications often have the risk of low recognition efficiency and omissions; and there are differences in the design results of different testers, and the test script development efficiency is not high, resulting in low efficiency in the robustness testing of communication modules. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a communication module robustness testing method, system, equipment, medium and program product, which automatically identify test requirements through a large language model and generate test cases that meet the test requirements, thereby improving the efficiency of communication module robustness testing.

[0005] To solve the above technical problems, an embodiment of the present invention provides a communication module robustness testing method, comprising:

[0006] Inputting the test requirement file of the communication module into a pre-trained test requirement identification model so that the test requirement identification model outputs a test requirement specification;

[0007] Inputting the test requirement specification into a pre-trained test case design model so that the test case design model outputs a test case;

[0008] Generate a corresponding test script according to the test case;

[0009] Perform a robustness test on the communication module according to the test script.

[0010] As an improvement of the above solution, the robustness test of the communication module according to the test script includes:

[0011] Performing a black box fuzz test on the communication module according to the test script to obtain a black box fuzz test result;

[0012] Obtaining an initial seed for white box testing according to the black box fuzzy testing result;

[0013] The communication module is white-box tested according to the initial seed.

[0014] As an improvement of the above solution, after obtaining the black box fuzzy test result, the method further includes:

[0015] Determining the state of the communication module according to the black box fuzzy test result;

[0016] If the state of the communication module is an old state or a non-frozen state, repeating the black box fuzzy test;

[0017] If the state of the communication module is a new state or a dead state, the state information and log of the communication module are recorded in a pre-established function log library to obtain a black box fuzz test function log library.

[0018] As an improvement of the above solution, the step of obtaining an initial seed for a white box test according to the black box fuzzy test result includes:

[0019] The state information and log records of the communication module are extracted from the black box fuzz testing function log library as the initial seed for the white box test.

[0020] As an improvement of the above solution, performing white box testing on the communication module according to the initial seed includes:

[0021] generating a white-box test case based on the initial seed;

[0022] Using the white-box test case as an input of a communication module to run the white-box test case;

[0023] Re-insert and compile the code of the communication module to monitor the code coverage during the running of the white box test case in real time;

[0024] The seed pool of the white box test is updated according to the code coverage situation, and new white box test cases are generated until the code coverage situation meets the requirements.

[0025] As an improvement of the above solution, the test requirement identification model is obtained by training a large language model, and the test requirement identification model is trained by the following steps:

[0026] Prompts and multiple rounds of dialogue are used to collect model data to train the test requirement recognition model;

[0027] The model parameters of the trained test requirement identification model are fine-tuned to obtain a trained test requirement identification model.

[0028] As an improvement of the above solution, the test case design model is obtained by training a large language model, and the test case design model is trained through the following steps:

[0029] The knowledge data of fuzz testing tools and automated test script tools are collected by using prompts and multi-round dialogues to train the test case design model;

[0030] Fine-tune the model parameters of the trained test case design model to obtain a trained test case design model.

[0031] As an improvement of the above solution, the model parameters include a sampling degree adjustment parameter, a first sampling range adjustment parameter and a second sampling range adjustment parameter.

[0032] As an improvement of the above solution, the test requirement specification includes the product project test function points, test methods, functional states and switching methods, and data variation requirements of the communication module.

[0033] As an improvement of the above solution, the test case includes test preconditions, test steps, expected results, test instructions and test configuration items;

[0034] The test scripts include functional test scripts and performance test scripts.

[0035] To solve the above technical problems, an embodiment of the present invention further provides a communication module robustness testing system, including:

[0036] A requirement identification module, used for inputting the test requirement file of the communication module into a pre-trained test requirement identification model, so that the test requirement identification model outputs a test requirement specification;

[0037] A test case generation module, used for inputting the test requirement specification into a pre-trained test case design model so that the test case design model outputs a test case;

[0038] A test script generation module, used to generate a corresponding test script according to the test case;

[0039] A robustness testing module is used to perform a robustness test on the communication module according to the test script.

[0040] To solve the above technical problems, an embodiment of the present invention further provides a communication module robustness testing device, comprising: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the communication module robustness testing method as described in the above embodiment when executing the computer program.

[0041] To solve the above technical problems, an embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the communication module robustness testing method as described in the above embodiment.

[0042] To solve the above technical problems, an embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the communication module robustness testing method as described in the above embodiment.

[0043] Compared with the prior art, the embodiments of the present invention provide a communication module robustness test method, system, device, medium and program product, firstly inputting the test requirement file of the communication module into a pre-trained test requirement identification model so that the test requirement identification model outputs the test requirement specification of the communication module; secondly inputting the test requirement specification into a pre-trained test case design model so that the test case design model outputs the test case, and then generating a corresponding test script according to the test case; finally, performing a robustness test on the communication module according to the test script. The embodiments of the present invention do not require manual identification of test requirements and manual design of requirement specifications, and use a large language model to realize the automation of test requirement identification and test case design, thereby improving the recognition efficiency and reducing the risk of identification omissions, thereby improving the efficiency of communication module robustness testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solution of the present invention, the drawings used in the implementation mode will be briefly introduced below. Obviously, the drawings described below are only some implementation modes of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 It is a flow chart of a communication module robustness testing method provided by an embodiment of the present invention;

[0046] Figure 2 It is a schematic diagram of state switching of a test step in a communication module robustness test method provided by an embodiment of the present invention;

[0047] Figure 3 It is a flow chart of obtaining an initial seed for a white box test in a communication module robustness test method provided by an embodiment of the present invention;

[0048] Figure 4 It is another flow chart of a communication module robustness testing method provided by an embodiment of the present invention;

[0049] Figure 5 It is a structural block diagram of a communication module robustness test system provided by an embodiment of the present invention;

[0050] Figure 6 It is a structural block diagram of a communication module robustness testing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] See also Figure 1 , Figure 1 1 is a flow chart of a communication module robustness testing method provided by an embodiment of the present invention, the communication module robustness testing method comprising:

[0053] S1. The test requirement file of the communication module is input into a pre-trained test requirement identification model so that the test requirement identification model outputs a test requirement specification;

[0054] S2, inputting the test requirement specification into a pre-trained test case design model so that the test case design model outputs a test case;

[0055] S3, generating a corresponding test script according to the test case;

[0056] S4. Perform a robustness test on the communication module according to the test script.

[0057] Exemplarily, the communication module robustness testing method first inputs the test requirement file of the communication module to be tested into a pre-trained test requirement identification model, and outputs the test requirement specification of the communication module through the test requirement identification model, and then inputs the test requirement specification into a pre-trained test case design model to output the required test cases, and then generates corresponding test scripts through an automated test script tool based on the test cases, and finally performs a robustness test on the communication module according to the generated test script, so as to improve the efficiency and accuracy of the robustness testing of the communication model.

[0058] It should be noted that, in addition to using the test requirement file of the communication module as the input of the test requirement identification model, the generation of the test requirement specification in the embodiment of the present invention (ie, the test requirement identification of the communication module) also relies on a pre-configured test knowledge base.

[0059] Specifically, the pre-configured test knowledge base includes existing product technical parameter files, existing product project files, existing product project application manuals, a test case library, and a corresponding test script library.

[0060] Specifically, the test requirement file of the communication module includes at least one of a newly developed product technical file of the communication module, a newly developed product project file, a newly developed product project application manual and a newly developed special requirement file.

[0061] Among them, the newly developed product technical parameter files, product project files and product project application manuals of the communication module can be formed or modified with reference to the existing product technical parameter files, product project files and product project application manuals in the test knowledge base.

[0062] For example, the newly developed product technical parameter file can divide the functional points of the communication module into levels, including information such as the hardware size of the product and the functionality of software and hardware, such as first-level functional points, second-level functional points, third-level functional points and parameter types. The first-level functional points are generally the main functions or core parameter characteristics of the product; the second-level functional points are further refinements of the first-level functional points, usually describing the specific parameter indicators under the first-level functional points; the third-level functional points are further refinements of the second-level functional points, describing more specific parameter data; the parameter type refers to the specific category or attribute of the product technical parameters, which is used to annotate product information or software and hardware information. The format of the newly developed product technical parameter file can be referred to Table 1:

[0063] Table 1 Newly developed product technical parameter file format

[0064]

[0065] It is worth noting that the data of each level of function points shown in Table 1 are only for example and can be changed according to actual conditions in actual application.

[0066] Exemplarily, the newly developed product project file may include the association relationship between product projects, may also include all the contents of the product technical parameter file, and may also record other sub-model change points of the product in a table in text. Such as including communication standard, product main model, product sub-model and product difference. Among them, the communication standard refers to the network communication standard or protocol supported by the product, such as NB-IOT (Narrow Band Internet of Things, a narrowband Internet of Things technology), Cat.1 (full name LTE UE-Category 1, a standard for user terminal categories under 4G communication LTE network); the product main model is the main identifier of the product, generally the model or code of the product, usually used to distinguish products of different series or categories; the product sub-model is based on the product main model, and further distinguishes the specific product sub-version or variant; the product difference refers to the specific differences between different models or versions in the same product series, which is used to understand the unique functions or specifications of each model. The format of the newly developed product project file can refer to Table 2:

[0067] Table 2 Newly developed product project file format

[0068]

[0069] It is worth noting that the data on the products shown in Table 2 are only examples and can be changed according to actual conditions in actual applications.

[0070] Exemplarily, the newly developed special requirements file mainly includes additional customized requirements in addition to the pre-configured test knowledge base, such as the product sub-model of the tested communication module, the special requirements function points that summarize the special requirements of the test, and the function point description that describes the special requirements function points in detail. An example of the format of the newly developed special requirements file can be found in Table 3:

[0071] Table 3 Newly developed special requirements file format

[0072] Product sub-model Special requirements function points Function point description NB-P1-M3 Special function point 1 Function point description 1 ... ... ...

[0073] It is worth noting that the data of product sub-models shown in Table 3 are only for example and can be changed according to actual conditions in actual application.

[0074] Furthermore, the above test requirement files are provided as input to the test requirement identification model for data cleaning, preprocessing and other operations, and then the test requirement specification is output.

[0075] Specifically, the test requirement specification includes the product project test function points, test methods, functional states and switching methods, and data variation requirements of the communication module to be tested.

[0076] More specifically, the product project test function points may include primary function points, secondary function points, tertiary function points and parameter information.

[0077] For example, the format of the test requirement specification can refer to Table 4:

[0078] Table 4 Test requirement specification template

[0079]

[0080]

[0081] It should be noted that the test requirement specification generally needs to describe the functional points and parameter information of all levels of all projects (which can be given according to the above-mentioned product technical parameter file), and give the test method and functional status of each level of functional points according to the test knowledge base, and then give the state switching method of each functional state according to the product application manual, and then identify the instruction parameters in the state switching method (such as AT+xxx, AT represents the "Attention" instruction, and the following xx is the specific command or parameter), and list the data variation requirements. Among them, the data variation requirements can be to transform (mutate) the data type or data range of the communication module, such as integer to character type, or mutate the data range 1-10 to 1-7. The main purpose is to verify whether the abnormal input data has an impact on the function / performance of the communication module, that is, to perform robustness test verification of the communication module. In addition, the functional point data of the project shown in Table 4 is only used as an example. In actual application, it can be changed according to actual conditions.

[0082] As an improvement of the above solution, after outputting the test requirement specification, the method further includes:

[0083] Review the test requirement specification;

[0084] When the test requirement specification does not meet the requirements, the test requirement identification model is optimized.

[0085] Exemplarily, after the test requirement specification is output, the test engineer reviews the test requirement specification to ensure that the software and hardware test functional points are fully covered, and reviews the status and switching methods, and parameter contents. When there are major problems with the output test requirement specification, it is necessary to re-optimize the data set to adjust the test requirement identification model to ensure that the automatically generated test requirement specification meets the test requirements of the delivered test cases.

[0086] Specifically, the generated test cases include test preconditions, test steps, expected results, test instructions, and test configuration items.

[0087] It should be noted that the generation of test cases through the test case design model is also based on the above-mentioned pre-configured test knowledge base, mainly the test cases of the test case library in the test knowledge base.

[0088] For example, the test case format in the test case library can refer to Table 5:

[0089] Table 5 Test case format

[0090]

[0091] It should be noted that the test cases in the test case library may include test types, three-level function points, test preconditions, test steps, expected results, test instructions and test configuration items, among which the test type is used to define the test objectives and methods, such as baseband testing and black box fuzz testing; the three-level function points can be obtained from the product technical parameter file; the test preconditions are the objects or conditions required to execute the test; the test steps are automatically designed by the test case design model according to the state switching method in the test requirement specification corresponding to the state switching sequence, and the state switching instructions and configuration item parameters are listed in combination with the contents of the product application manual; the expected result is the correct result that the system should produce after executing the test steps; the test instruction is a command format commonly used for communication equipment testing and configuration, such as AT+xx; the test configuration items are the configuration parameters required for test execution.

[0092] For example, see Figure 2 , Figure 2 1 is a schematic diagram of state switching of a test step of a communication module robustness test method provided by an embodiment of the present invention. Figure 2As shown, when the source state is state 1 and the target state to be switched is state 2, test step 1 is designed according to the instruction on switching from state 1 to state 2 in the test requirement specification. Specifically, test step 1 can be sending a module instruction to make the module enter state 2 from state 1; conversely, when the source state is state 2 and the target state to be switched is state 1, test step 2 is designed according to the instruction on switching from state 2 to state 1 in the test requirement specification. Specifically, test step 2 can be sending a module instruction to make the module enter state 1 from state 2. By analogy, the corresponding test steps 1 to 6 are automatically designed through the switching mode between states 1 to 3, and the mutual conversion between states 1 to 3 can be realized by executing steps 1 to 6 in the subsequent test process.

[0093] As an improvement of the above solution, after outputting the test case, the method further includes:

[0094] Reviewing the test cases;

[0095] When the test case cannot generate a test script file that meets the requirements of the test tool, the test case design model is optimized.

[0096] Furthermore, an automated test script tool may be used to generate corresponding test scripts according to the test cases output above.

[0097] Specifically, the test script includes a functional test script and a performance test script.

[0098] Preferably, the functional test script can be generated based on the Robot test tool (i.e., Robot Framework, a framework completely based on keyword test drive), such as including test steps, test parameters, and test scripts. Exemplarily, the functional test script file format can refer to Table 6:

[0099] Table 6 Functional test script file format

[0100]

[0101] Preferably, for the performance test script, it can be generated based on the Peachfuzzer fuzz testing tool. It should be noted that Peachfuzzer, commonly referred to as Peach Fuzzer or simply Peach, is an intelligent fuzzing tool that is mainly used in the field of software security testing. It discovers and identifies potential security vulnerabilities, stability issues, and functional defects by automatically sending a large amount of specially designed, unexpected input data to the target system or application. Exemplarily, a performance test script file may include test steps, source state (of the communication module), target state, Peachfuzzer parameters, and corresponding test scripts. The performance test script file format can be referred to Table 7:

[0102] Table 7 Performance test script file format

[0103]

[0104] During the specific implementation, after the test script is generated, the test engineer can also debug and verify the script content, optimize the script content, and then perform test verification by setting up a test environment and communication module.

[0105] In one implementation, the performing a robustness test on the communication module according to the test script includes:

[0106] Performing a black box fuzz test on the communication module according to the test script to obtain a black box fuzz test result;

[0107] Obtaining an initial seed for white box testing according to the black box fuzzy testing result;

[0108] The communication module is white-box tested according to the initial seed.

[0109] In specific implementation, the black box fuzz test includes using the Robot test tool to perform test verification according to the above-mentioned functional test script; and also includes using the Peachfuzzer fuzz test tool to perform test verification according to the above-mentioned performance test script.

[0110] Specifically, after obtaining the black box fuzzy test result, the method further includes:

[0111] Determining the state of the communication module according to the black box fuzzy test result;

[0112] If the state of the communication module is an old state or a non-frozen state, repeating the black box fuzzy test;

[0113] If the state of the communication module is a new state or a dead state, the state information and log of the communication module are recorded in a pre-established function log library to obtain a black box fuzz test function log library.

[0114] Further, obtaining an initial seed for a white box test according to the black box fuzzy test result includes:

[0115] The state information and log records of the communication module are extracted from the black box fuzz testing function log library as the initial seed for the white box test.

[0116] It should be noted that the initial seed for the white box test is specifically the result of the test using the Peachfuzzer fuzz testing tool.

[0117] In one implementation, AFL-Fuzz (American Fuzzy Lop, a coverage-guided fuzz testing tool) may be used for white box testing.

[0118] It is worth noting that although white-box testing can realize fuzz testing of the C code of the communication module, due to the randomness of its initialization seed, it is impossible to quickly and efficiently discover code defects. Therefore, the embodiment of the present invention uses the black-box fuzzy testing results as the initial seed source, and screens the initial seeds through state exploration conditions, which can improve the quality of the initial seeds, thereby improving the efficiency and quality of white-box fuzzy testing.

[0119] For example, see Figure 3 , Figure 3 FIG. 1 is a flowchart of obtaining an initial seed for a white box test in a communication module robustness test method provided by an embodiment of the present invention. Figure 3 As shown, first, based on the black box fuzz result (black box fuzz test result), the interaction log of the communication module to be tested is continuously collected and saved, and the state of the communication module is judged. If it is an old state, the black box fuzz test is repeated. If it is a new state, it is recorded in a pre-established function log library; further, the crash of the communication module state is judged. If it is not crashed, the black box fuzz test is continued. If it is a crash state or other abnormal state, the abnormal state information and log records are saved in a pre-established function log library. Exemplarily, the function log library finally saved includes function description, state description, sent instruction information, received instruction information, and crash state information. Its format can be referred to Table 8:

[0120] Table 8 Black box fuzz testing function log library template

[0121] Function state send take over Is the system frozen? Feature 1 State 1 AT+xxx1 Response1 none Feature 1 State 2 AT+xxx2 Response2 yes ... ... ... ... ...

[0122] Furthermore, when white-box testing of the code is required, the logs of the new state and the crash state in the functional log library can be called and used as the initial seed library for white-box fuzz (white-box testing), and seed files can be continuously generated and optimized for white-box testing.

[0123] As an improvement of the above solution, performing white box testing on the communication module according to the initial seed includes:

[0124] generating a white-box test case based on the initial seed;

[0125] Using the white-box test case as an input of a communication module to run the white-box test case;

[0126] Re-insert and compile the code of the communication module to monitor the code coverage during the running of the white box test case in real time;

[0127] The seed pool of the white box test is updated according to the code coverage situation, and new white box test cases are generated until the code coverage situation meets the requirements.

[0128] It should be noted that the seed pool is a data set for generating white-box test cases, and the initial seed is the initial data set for generating white-box test cases. Taking the white-box test process of the communication module using AFL-Fuzz (fuzz testing tool) as an example, first AFL-Fuzz needs some initial "seed" inputs, which can be randomly generated or provided by the user. In the embodiment of the present invention, they are obtained from the black-box test results. Then the initial seeds will be used as the starting point by AFL-Fuzz to generate test cases; then AFL-Fuzz will pass the generated test cases to the communication module and observe its running results; at the same time, re-insert the compilation, which will embed the monitoring code of AFL-Fuzz into the communication module. When the communication module runs the test case, AFL-Fuzz can track which code is executed and which branches are traversed in real time, and then analyze the code coverage to find out which code paths have not been executed. It will update the seed pool based on this information and generate new test cases, trying to cover the unexecuted code paths. This process will be repeated until the expected code coverage is reached.

[0129] It is worth noting that during the white-box code testing process, AFL-Fuzz (fuzz testing tool) can be used to re-insert and compile the C code of the communication module to generate firmware that can monitor the code status. AFL-Fuzz monitors the code coverage, state coverage and other indicators of the test in real time. The seed status output by the black-box fuzz test can also be used to correspond to the code coverage to locate problems more quickly, thereby improving test efficiency.

[0130] In a preferred embodiment, the test requirement identification model is obtained by training a large language model, and the test requirement identification model is trained by the following steps:

[0131] Prompts and multiple rounds of dialogue are used to collect model data to train the test requirement recognition model;

[0132] The model parameters of the trained test requirement identification model are fine-tuned to obtain a trained test requirement identification model.

[0133] In a preferred embodiment, the test case design model is obtained by training a large language model, and the test case design model is trained by the following steps:

[0134] The knowledge data of fuzz testing tools and automated test script tools are collected by using prompts and multi-round dialogues to train the test case design model;

[0135] Fine-tune the model parameters of the trained test case design model to obtain a trained test case design model.

[0136] It should be noted that the training data of the above test requirement identification model and the test case design model are collected through prompts and multi-round dialogues. The difference is that the training data of the test case design model needs to use the knowledge data of fuzzy testing tools and automated test script tools. Among them, in multi-round dialogues, the role of prompts needs to be clarified.

[0137] Exemplarily, the training data collection format of the test requirement identification model can refer to Table 9, wherein the header information of the data collection table includes prompt, output, and history. Specifically, the prompt data is used as the input of the large language model, and the output data is used as the output of the large language model to train a test requirement identification model that meets the requirements. It should be noted that the history data here is the historical record of the output data, such as the history of the second model training is the output data of the first model training, which is used to allow the model to analyze and compare the current training data and historical data, thereby optimizing the training process.

[0138] Table 9 Format of the training data collection table for the test demand identification model

[0139]

[0140]

[0141] Exemplarily, the training data collection table format of the test case design model can refer to Table 10, where the header information includes prompt, output, and history. Similarly, the prompt data is used as the input of the large language model, and the output data is used as the input of the large language model to train a test case design model that meets the requirements.

[0142] Table 10 Training data collection table format for the test case design model

[0143]

[0144] In a preferred embodiment, the model parameters include a sampling degree adjustment parameter, a first sampling range adjustment parameter and a second sampling range adjustment parameter.

[0145] Specifically, the sampling degree adjustment parameter is Temperature, the first sampling range adjustment parameter is Top P, and the second sampling range adjustment parameter is Top K.

[0146] Among them, Temperature is used to adjust the degree of random sampling from the generative model. The larger the value, the greater the randomness. The same question may produce different answers. The larger the sampling degree, the greater the randomness. Top P is used to dynamically set the size of the tokens candidate list, and the top tokens whose sum of possibilities does not exceed the specified value are included in the candidate list. For example, if p = 0.9, then only one token is selected from the minimum token set with a cumulative probability of 0.9, and other tokens with a cumulative probability less than 0.9 are not considered. Top K is used to dynamically set the size of the tokens candidate list. After sorting by possibility, the top tokens with a specified number of values ​​are included in the candidate list. For example, if k = 2, then only one token is selected from the first two sorted tokens in the token set, and other tokens are not considered. Generally speaking, the larger the k, the higher the generated diversity, but the lower the generated quality; the smaller the k, the higher the generated quality, but the lower the generated diversity.

[0147] It is worth noting that the model trained based on the prompts + data collected from multiple rounds of dialogue can be adapted to more accurate model output by adjusting the model parameters. In specific implementation, after completing the model training, the fine-tuned and compressed model can be published to the model warehouse for model deployment, so that the model can be used to test the output of related files in the future.

[0148] See also Figure 4 , Figure 4 FIG. 1 is another flow chart of a communication module robustness testing method provided by an embodiment of the present invention. Figure 4 As shown, by inputting the newly developed product technical parameter file, the newly developed product project file, the newly developed product application manual and the test requirement file of the newly developed special demand waiting test communication module into the test requirement identification model for test requirement identification, the test requirement specification is output, wherein the output test requirement specification includes test function points, state machine diagram, data structure and its variation requirements, etc. Further, the output test requirement specification will be manually reviewed. If the review fails, the test requirement identification model will continue to be optimized to output a test requirement specification that meets the requirements; if the review passes, it will be used as the input of the test case design model to generate corresponding functional test cases and performance test cases, and generate corresponding functional test scripts and performance test scripts according to the corresponding test cases. Further, the generated test script will be manually reviewed. If the review passes, the communication module will be subjected to black box fuzz testing according to the test script. Specifically, the communication module is functionally tested by the robot test tool according to the functional test script; the communication module is performance tested by the fuzz test tool Peachfuzzer according to the performance test script. After the black-box fuzz test is completed, the communication module is tested with a white-box code using the AFL-Fuzz test tool. Specifically, the test log of the fuzz test tool Peachfuzzer in the black-box fuzz test is used as the seed library of the AFL-Fuzz test tool to obtain the initial seed of the white-box test to perform a white-box test on the source code of the communication module. At the same time, the code can also be re-inserted and compiled using AFL-Fuzz to automatically monitor the code coverage or the status of the communication module.

[0149] See also Figure 5 , Figure 5 : is a structural block diagram of a communication module robustness test system 200 provided in an embodiment of the present invention, and the communication module robustness test system 200 includes:

[0150] The requirement identification module 21 is used to input the test requirement file of the communication module into a pre-trained test requirement identification model so that the test requirement identification model outputs a test requirement specification;

[0151] A test case generation module 22, used for inputting the test requirement specification into a pre-trained test case design model so that the test case design model outputs a test case;

[0152] A test script generation module 23, used to generate a corresponding test script according to the test case;

[0153] The robustness testing module 24 is used to perform a robustness test on the communication module according to the test script.

[0154] In one embodiment, the robustness testing module 24 includes:

[0155] A black box fuzzy testing unit, used for performing a black box fuzzy test on the communication module according to the test script to obtain a black box fuzzy testing result;

[0156] A white box test initial seed acquisition unit, used to acquire an initial seed for the white box test according to the black box fuzzy test result;

[0157] The white box testing unit is used to perform a white box test on the communication module according to the initial seed to obtain a robustness test result of the communication module.

[0158] Specifically, the robustness testing module 24 further includes a black box fuzzy testing function logging unit, which is used to:

[0159] Determining the state of the communication module according to the black box fuzzy test result;

[0160] If the state of the communication module is an old state or a non-frozen state, repeating the black box fuzzy test;

[0161] If the state of the communication module is a new state or a dead state, the state information and log of the communication module are recorded in a pre-established function log library to obtain a black box fuzz test function log library.

[0162] Specifically, the white box test initial seed acquisition unit is specifically used for:

[0163] The state information and log records of the communication module are extracted from the black box fuzz testing function log library as the initial seed for the white box test.

[0164] Preferably, the white box testing unit is also used for:

[0165] Performing white box testing on the communication module according to the initial seed includes:

[0166] generating a white-box test case based on the initial seed;

[0167] Using the white box test case as an input of a communication module to run the white box test case;

[0168] Re-insert and compile the code of the communication module to monitor the code coverage during the running of the white box test case in real time;

[0169] The seed pool of the white-box test is updated according to the code coverage situation, and new white-box test cases are generated until the code coverage situation meets the requirements.

[0170] In one embodiment, the communication module robustness testing system also includes a test requirement specification review module, which is used to review the test requirement specification after outputting the test requirement specification; when the test requirement specification does not meet the requirements, the test requirement identification model is optimized.

[0171] In one embodiment, the communication module robustness testing system also includes a test case review module, which is used to review the test case after outputting the test case; when the test case cannot generate a test script file that meets the requirements of the test tool, the test case design model is optimized.

[0172] It should be noted that a communication module robustness testing system provided in an embodiment of the present invention is used to execute all process steps of a communication module robustness testing method in the above embodiment. The working principles and beneficial effects of the two correspond one to one, and thus will not be elaborated on.

[0173] See also Figure 6 , Figure 6 1 is a structural block diagram of a communication module robustness test device 300 provided in an embodiment of the present invention, wherein the test device 300 includes a processor 31, a memory 32, and a computer program stored in the memory 32 and executable on the processor 31. When the processor 31 executes the computer program, the steps in the above-mentioned embodiments of the communication module robustness test method are implemented, such as steps S1 to S4.

[0174] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 32 and executed by the processor 31 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the test device 300.

[0175] The test device 300 may include, but is not limited to, a processor 31 and a memory 32. Those skilled in the art will appreciate that the schematic diagram is merely an example of the test device 300 and does not limit the test device 300. The test device 300 may include more or fewer components than shown in the figure, or may combine certain components, or may include different components. For example, the test device 300 may also include input and output devices, network access devices, buses, etc.

[0176] The processor 31 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 31 is the control center of the test device 300, and various parts of the entire test device 300 are connected using various interfaces and lines.

[0177] The memory 32 can be used to store the computer program and / or module. The processor 31 realizes various functions of the test device 300 by running or executing the computer program and / or module stored in the memory 32 and calling the data stored in the memory 32. The memory 32 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 32 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0178] Wherein, if the module / unit integrated in the test device 300 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor 31. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0179] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A communication module robustness testing method, characterized in that: include: Inputting the test requirement file of the communication module into a pre-trained test requirement identification model so that the test requirement identification model outputs a test requirement specification; Inputting the test requirement specification into a pre-trained test case design model so that the test case design model outputs a test case; Generate a corresponding test script according to the test case; Perform a robustness test on the communication module according to the test script.

2. The communication module robustness testing method according to claim 1, characterized in that: The robustness test of the communication module according to the test script includes: Performing a black box fuzz test on the communication module according to the test script to obtain a black box fuzz test result; Obtaining an initial seed for white box testing according to the black box fuzzy testing result; The communication module is white-box tested according to the initial seed.

3. The communication module robustness testing method according to claim 2, characterized in that: After obtaining the black box fuzzy test result, the method further includes: Determining the state of the communication module according to the black box fuzzy test result; If the state of the communication module is an old state or a non-frozen state, repeating the black box fuzzy test; If the state of the communication module is a new state or a dead state, the state information and log of the communication module are recorded in a pre-established function log library to obtain a black box fuzz test function log library.

4. The communication module robustness testing method according to claim 3, characterized in that: The obtaining of an initial seed for a white box test according to the black box fuzzy test result comprises: The state information and log records of the communication module are extracted from the black box fuzz testing function log library as the initial seed for the white box test.

5. The communication module robustness testing method according to claim 2, characterized in that: The performing white box testing on the communication module according to the initial seed includes: generating a white-box test case based on the initial seed; Using the white-box test case as an input of a communication module to run the white-box test case; Re-insert and compile the code of the communication module to monitor the code coverage during the running of the white box test case in real time; The seed pool of the white box test is updated according to the code coverage situation, and new white box test cases are generated until the code coverage situation meets the requirements.

6. The communication module robustness testing method according to claim 1, characterized in that: The test requirement identification model is obtained by training a large language model, and the test requirement identification model is trained by the following steps: Prompts and multiple rounds of dialogue are used to collect model data to train the test requirement recognition model; The model parameters of the trained test requirement identification model are fine-tuned to obtain a trained test requirement identification model.

7. The communication module robustness testing method according to claim 1, characterized in that: The test case design model is obtained by training a large language model, and the test case design model is trained by the following steps: The knowledge data of fuzz testing tools and automated test script tools are collected by using prompts and multi-round dialogues to train the test case design model; Fine-tune the model parameters of the trained test case design model to obtain a trained test case design model.

8. The communication module robustness testing method according to any one of claims 6 or 7, characterized in that: The model parameters include a sampling degree adjustment parameter, a first sampling range adjustment parameter and a second sampling range adjustment parameter.

9. The communication module robustness testing method according to claim 1, characterized in that: The test requirement specification includes the product project test function points, test methods, functional states and switching methods, and data variation requirements of the communication module.

10. The communication module robustness testing method according to claim 1, characterized in that: The test case includes test preconditions, test steps, expected results, test instructions and test configuration items; The test scripts include functional test scripts and performance test scripts.

11. A communication module robustness testing system, characterized in that: include: A requirement identification module, used for inputting the test requirement file of the communication module into a pre-trained test requirement identification model, so that the test requirement identification model outputs a test requirement specification; A test case generation module, used for inputting the test requirement specification into a pre-trained test case design model so that the test case design model outputs a test case; A test script generation module, used to generate a corresponding test script according to the test case; A robustness testing module is used to perform a robustness test on the communication module according to the test script.

12. A communication module robustness test device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the communication module robustness testing method as described in any one of claims 1 to 10 when executing the computer program.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the communication module robustness testing method as described in any one of claims 1 to 10.

14. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the communication module robustness testing method as described in any one of claims 1 to 10 is implemented.

Images