A firmware testing method, an electronic device, a storage medium, and a program product

Abstract

The present application discloses a firmware testing method, an electronic device, a storage medium, and a program product, which relate to the field of computer technology. By locating the function modules to be tested with changes in the firmware to be tested, the function testing of the firmware to be tested is carried out in units of function modules, and the automatic screening and distribution of target test cases can be realized according to the change types and attribute characteristics of the function modules to be tested, improving the testing efficiency of the firmware to be tested.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a firmware testing method, an electronic device, a storage medium, and a program product. Background Art

[0002] With the continuous increase in the business requirements of products such as storage devices and servers by users, the firmware of the products is also constantly iterated and updated. Therefore, the development work of firmware code has become an important part of product development. Before the newly developed firmware is put into product use, a series of functional tests need to be carried out on the firmware to ensure its security.

[0003] In the related art, usually, the method of manually issuing test cases is adopted to perform functional tests on the firmware code to be tested. However, due to the continuous update and replacement of the firmware, the functions supported by the firmware are becoming more and more complex, and the number of test cases that need to be issued is also increasing. The current test case issuing method will reduce the firmware testing efficiency. Summary of the Invention

[0004] This application provides a firmware testing method, an electronic device, a storage medium, and a program product to at least solve the problem of reducing the firmware testing efficiency in the related art.

[0005] This application provides a firmware testing method, including:

[0006] Obtaining the latest development code of the firmware to be tested;

[0007] Determining at least one to-be-tested functional module that has been modified according to the latest development code; wherein, the firmware to be tested includes multiple functional modules;

[0008] For any to-be-tested functional module, determining a target test case set of the to-be-tested functional module according to the modification type and attribute characteristics of the to-be-tested functional module;

[0009] Based on the target test case set, performing multiple functional tests on the to-be-tested functional module to obtain the functional test result of the to-be-tested functional module.

[0010] This application also provides a firmware testing device, including:

[0011] An obtaining module, configured to obtain the latest development code of the firmware to be tested;

[0012] A first determination module, configured to determine at least one to-be-tested functional module that has been modified according to the latest development code; wherein, the firmware to be tested includes multiple functional modules;

[0013] A second determination module, configured to determine a target test case set of the to-be-tested functional module according to the modification type and attribute characteristics of the to-be-tested functional module for any to-be-tested functional module;

[0014] A test module, configured to perform various function tests on a function module to be tested based on a target test case set, and obtain a function test result of the function module to be tested.

[0015] This application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of any of the above firmware test methods when executing the computer program.

[0016] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program implements the steps of any of the above firmware test methods when executed by a processor.

[0017] This application also provides a computer program product including a computer program, and the computer program implements the steps of any of the above firmware test methods when executed by a processor.

[0018] Through this application, the function module to be tested with changes is located in the firmware to be tested, so that the function test of the firmware to be tested is carried out in units of function modules, and the automatic screening and distribution of target test cases can be realized according to the change type and attribute characteristics of the function module to be tested, improving the test efficiency of the firmware to be tested. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] To more clearly illustrate the embodiments of this application, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 It is a schematic structural diagram of the firmware test system based on the embodiments of this application;

[0021] Figure 2 It is a schematic flowchart of the firmware test method provided by the embodiments of this application;

[0022] Figure 3 It is a schematic overall flowchart of the firmware test method provided by the embodiments of this application;

[0023] Figure 4 It is a schematic structural diagram of the firmware test device provided by the embodiments of this application;

[0024] Figure 5 It is a schematic structural diagram of the electronic device provided by the embodiments of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] In the following, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0026] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variation thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0027] In the related art, there are many disadvantages in manually issuing test cases, such as low efficiency, error-prone, lack of consistency, difficult to track and manage, high communication cost, lack of automation support, and difficult to adapt to rapid changes. These disadvantages limit the efficiency, accuracy and consistency of testing.

[0028] Embodiments of the present application provide a firmware testing method, an electronic device, a storage medium and a program product to solve the above technical problems. The method includes: obtaining the latest development code of the firmware to be tested; determining at least one function module to be tested with changes according to the latest development code, where the firmware to be tested includes multiple function modules; for any function module to be tested, determining a target test case set for the function module to be tested according to the change type and attribute characteristics of the function module to be tested; and based on the target test case set, performing various function tests on the function module to be tested to obtain the function test result of the function module to be tested. The method provided by the above solution locates the function modules to be tested with changes in the firmware to be tested, enables the function testing of the firmware to be tested to be carried out in units of function modules, and can automatically screen and issue target test cases according to the change type and attribute characteristics of the function modules to be tested, improving the testing efficiency of the firmware to be tested while ensuring the accuracy and consistency of function testing.

[0029] In order to enable those skilled in the art of the present technology to better understand the solution of the present application, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0030] Combined with the specific application environment architecture or specific hardware architecture on which the execution of the firmware testing method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0031] First, the structure of the firmware testing system based on the present application will be described:

[0032] The firmware testing method, electronic device, storage medium, and program product provided by the embodiments of the present application are applicable to functional testing of firmware development codes of products such as servers and storage devices. As Figure 1 shown, it is a schematic structural diagram of the firmware testing system based on the embodiments of the present application, mainly including a firmware to be tested, a data acquisition device, and a firmware testing device. Among them, the data acquisition device is used to collect the latest development code provided by developers to improve the firmware to be tested, and provide the obtained latest development code to the firmware testing device. The firmware testing device performs targeted functional model testing on the firmware to be tested according to the obtained latest development code.

[0033] The embodiments of the present application provide a firmware testing method for functional testing of firmware development codes of products such as servers and storage devices. The execution subject of the embodiments of the present application is an electronic device, such as a server, a desktop computer, a laptop computer, a tablet computer, and other electronic devices that can be used for development code testing.

[0034] As Figure 2 shown, it is a schematic flowchart of the firmware testing method provided by the embodiments of the present application. The method includes:

[0035] Step 201, obtain the latest development code of the firmware to be tested.

[0036] Specifically, in the continuous integration and continuous delivery (CI / CD) process of the firmware, after developers complete the development of any functional module, they submit the development code to the code management tool Gerrit. Through the linkage relationship established between Gerrit and Jenkins, when there is an update in the Gerrit code repository, Jenkins automatically pulls the code of this change to obtain the latest development code of the firmware to be tested. Among them, Jenkins is an open-source continuous integration and continuous delivery deployment tool.

[0037] Step 202, determine at least one functional module to be tested that has been changed according to the latest development code.

[0038] Among them, the firmware to be tested includes multiple functional modules.

[0039] Specifically, by analyzing the obtained latest development code, the identification information such as the function name in the modified code can be determined, and then according to the identification information in the modified code, which functions in the firmware to be tested have been changed in the latest development code can be determined.

[0040] Step 203, for any functional module to be tested, determine the target test case set of the functional module to be tested according to the change type and attribute characteristics of the functional module to be tested.

[0041] It should be noted that the types of changes at least include three types: function addition, function modification, and function optimization. The attribute features are used to characterize which attributes are included in the function module to be tested. The attributes included in the function module to be tested are the functions supported by the function module to be tested. The attributes include Object Access Control System (abbreviated as: OACS), Access Control List (abbreviated as: ACL), and Network Performance and Security System (abbreviated as: NPSS), etc.

[0042] Specifically, according to the change type and attribute features of the function module to be tested, multiple target test cases that meet the function test requirements of the function module to be tested can be screened from the preset test case library to obtain the target test case set of the function module to be tested.

[0043] Step 204, based on the target test case set, perform various function tests on the function module to be tested to obtain the function test result of the function module to be tested.

[0044] Specifically, by running each target test case in the target test case set, various function tests on the function module to be tested can be realized, and the test case test result corresponding to each target test case can be obtained. Finally, by summarizing the test case test results of each target test case, the function test result of the function module to be tested can be obtained.

[0045] Based on the above embodiments, as an implementable manner, in one embodiment, according to the latest development code, at least one function module to be tested with changes is determined, including:

[0046] Step 2021, extract keywords from the latest development code to obtain the keyword extraction result of the latest development code;

[0047] Step 2022, according to the keyword extraction result of the latest development code, determine at least one function module to be tested with changes.

[0048] Among them, the keywords include attribute names, function names, etc.

[0049] Specifically, based on the Spark NLP natural language processing model, keywords such as attribute names and function names can be extracted from the latest development code to obtain the keyword extraction result of the latest development code. Then, according to the keyword extraction result, it can be determined which function module's development code the latest development code belongs to, and then this modified function module can be used as the function module to be tested with changes.

[0050] Specifically, in one embodiment, the main functional module with direct changes can be determined according to the keyword extraction result of the latest development code; according to the association relationship between each functional module in the firmware under test, the slave functional module associated with the main functional module can be determined; the main functional module and the slave functional module are used as the functional modules under test.

[0051] Specifically, the extracted keywords can be matched with the characteristic keywords of each predefined functional module. If the characteristic keywords of a certain functional module overlap with the keywords in the extraction result, it is determined that the functional module has direct changes and is used as the main functional module. Since in the firmware under test, each functional module usually does not exist in isolation, and there are various association relationships between them, such as call relationships and data dependency relationships, etc. Therefore, after determining the main functional module, it is necessary to further find out the slave functional module associated with it.

[0052] Specifically, during the firmware development process, an association relationship model between each functional module can be established in advance. The association relationship model is used to record information such as the call path and data flow direction between each functional module. After determining the main functional module, the input and output of the main functional module, the other modules it calls, and the situation of being called by other modules can be analyzed by querying the association relationship model to determine the associated slave functional module. By determining the slave functional module and including it in the test scope, potential problems caused by the changes in the slave functional module affected by the main functional module can be avoided from being missed.

[0053] Specifically, in one embodiment, the attributes with direct changes can be determined according to the keyword extraction result of the latest development code; in the case where any attribute of any functional module has direct changes, the functional module is used as the main functional module with direct changes.

[0054] Specifically, each functional module is provided with a specific set of attributes. The set of attributes jointly defines the behavior and characteristics of the functional module. When any attribute in the functional module has direct changes, it indicates that the behavior or characteristics of the functional module may have changed. Therefore, the functional module is used as the main functional module with direct changes.

[0055] Based on the above embodiments, as an implementable method, in one embodiment, for any functional module under test, according to the change type and attribute characteristics of the functional module under test, the target test case set of the functional module under test is determined, including:

[0056] Step 2031, for any functional module under test, determine the target test range according to the change type of the functional module under test;

[0057] Step 2032: According to the attribute characteristics of the function module to be tested, screen the test case sets corresponding to each attribute in the preset test case library;

[0058] Step 2033: For any attribute, according to the target test scope, screen at least one target test case from the test case set of this attribute;

[0059] Step 2034: Determine the target test case set of the function module to be tested according to at least one target test case corresponding to each attribute.

[0060] Among them, the change types at least include three types: function addition, function modification, and function optimization.

[0061] It should be noted that different change types correspond to different test scopes. For function addition, it is necessary to test whether the basic functions of the new function are normal, the compatibility with the existing system, etc.; for function modification, it is necessary to test the functional correctness of the modified part and whether it affects other related functions; for function optimization, the key is to test whether the optimized performance indicators meet the optimization expectations.

[0062] Specifically, after confirming the target test scope, according to this target test scope, screen at least one target test case of each attribute that matches the target test scope in the preset test case library, so as to obtain the target test case set of the function module to be tested. By determining the target test scope according to the change type and screening test cases based on the attribute characteristics, the content that needs to be tested can be accurately located, avoiding testing irrelevant content, and improving the pertinence, efficiency, and comprehensiveness of the test while improving the test efficiency.

[0063] Specifically, in an embodiment, the attribute characteristics of each function module of the firmware to be tested can be obtained; according to the attribute characteristics of each function module, multiple test cases are created, and the corresponding relationships among each function module, attribute, and test case are established; according to the corresponding relationships among each function module, attribute, and test case, a preset test case library is created.

[0064] Specifically, in order to clearly express which attribute of which function module each test case is for testing, the present application embodiment pre-establishes the corresponding relationships among the function module, attribute, and test case. Specifically, a database or a table can be used to record this corresponding relationship. For example, a table is created, with one column recording the function module name, one column recording the attribute name, and one column recording the corresponding test case number. In practical applications, all test cases corresponding to a certain attribute of a certain function module can be quickly located by querying the table.

[0065] Specifically, in one embodiment, the historical test records of the firmware to be tested can be obtained; according to the historical test records, the change type of the function module to be tested is determined.

[0066] Specifically, if in the historical test records, the previous test did not cover this function module, that is, no functional test has been performed on the function module to be tested or a certain attribute in the function module to be tested, then it is determined that the change type of the function module to be tested is function addition. If in the historical test records, the last functional test result of the function module to be tested is failed, then it is determined that the modification type this time is function modification. If in the historical test records, the last functional test result of the function module to be tested is passed, then it is determined that the modification type this time is function optimization.

[0067] Based on the above embodiments, as an implementable manner, in one embodiment, based on the target test case set, multiple functional tests are performed on the function module to be tested, and the functional test results of the function module to be tested are obtained, including:

[0068] Step 2041, deploy the firmware to be tested to multiple test devices;

[0069] Step 2042, send the target test case set to multiple test devices, so as to perform multiple functional tests on the function module to be tested based on the test devices running each target test case in the target test case set, and obtain the test case test results corresponding to each target test case;

[0070] Step 2042, determine the functional test results of the function module to be tested according to the test case test results corresponding to each target test case.

[0071] Specifically, by deploying the firmware to be tested on multiple test devices, a more extensive actual usage scenario can be simulated, so as to discover possible problems of the firmware to be tested in different environments. And by using multiple test devices to execute each target test case in the target test case set in parallel, the running time of the test cases is reduced, and the firmware function test efficiency is further improved.

[0072] Specifically, in one embodiment, the multiple test devices can be classified according to the configuration information of each test device to obtain multiple test device pools; for any target test case in the target test case set, according to the test type of the target test case, determine the target test device pool of the target test case; when any target test device in the target test device pool is in an idle state, send the target test case to the target test device to obtain the test case test result corresponding to the target test case based on the target test device running the target test case.

[0073] Among them, the test types of test cases are at least divided into two types: compatibility test and performance test. Compatibility test mainly focuses on the running situation of the firmware under test in different environments to ensure that the software can work properly on devices with various configurations; performance test focuses on evaluating the performance indicators of the firmware under test, such as response time, throughput, and resource utilization, etc.

[0074] It should be noted that the configuration information of the test device includes hardware configuration information and software configuration information. The hardware configuration information includes CPU model, memory size, hard disk capacity, and graphics card performance, etc. The software configuration information includes the type and version of the operating system, etc. Different configurations will affect the running performance of the test device for test cases.

[0075] Specifically, for compatibility test cases, they can be run on multiple test device pools with different configurations to cover more software usage scenarios. For example, to test the compatibility of an application under different combinations of operating systems and browsers, the test cases can be assigned to device pools installed with different operating systems and browsers. While performance test cases usually need to be run on a device pool with a specific configuration to ensure the accuracy and comparability of test results. For example, to test the performance upper limit of software in a high - configuration environment, a device pool with a higher configuration can be selected. During the process of issuing target test cases, the status of each device in the target test device pool can be continuously monitored to determine whether it is in an idle state. An idle state means that the device is not currently executing other test tasks and can receive new test cases. Therefore, when any target test device in the target test device pool is in an idle state, the corresponding target test case can be issued to this target test device.

[0076] Among them, during the process of issuing test cases, the issuing order of test cases can be determined according to the priorities of each attribute in the functional module under test. For example, test cases related to core computing services are issued first. Among them, the priorities of each attribute in the functional module can be determined according to the user's usage frequency of the service and the importance of each attribute in the functional module.

[0077] Specifically, in one embodiment, for any target test case, according to the test result corresponding to the target test case, determine the function coverage rate and code coverage rate of the target test case for the function module to be tested; when both the function coverage rate and code coverage rate of the target test case for the function module to be tested reach the preset coverage threshold, use the test result of the target test case as the test result to be fused; when either the function coverage rate or code coverage rate of the target test case for the function module to be tested does not reach the preset coverage threshold, use the test result of the target test case as an abnormal test result; when the test result of any target test case is an abnormal test result, use the target test case as a test case to be optimized, and add a new target test case to the target test case set to replace the test case to be optimized until the obtained test result is the test result to be fused; perform fusion processing on multiple test results to be fused to obtain the functional test result of the function module to be tested.

[0078] It should be noted that the function coverage rate refers to the proportion of the functions executed in the function module to be tested to the total functions when running a certain target test case. For example, if there are 10 functions in the function module to be tested and 6 functions are executed when running the target test case, the function coverage rate is 60%. The code coverage rate refers to the proportion of the code lines executed in the function module to be tested to the total code lines when running the target test case. For example, if there are 100 lines of code in total in the function module to be tested and 70 lines of code are executed after the test case is executed, the code coverage rate is 70%. By analyzing the test result corresponding to the target test case, this execution information can be collected, and thus the function coverage rate and code coverage rate can be calculated. In the embodiments of the present application, by paying attention to the function coverage rate and code coverage rate, it is ensured that the test case can cover as many parts of the function module to be tested as possible, avoiding test blind spots, so as to improve the accuracy of the test result.

[0079] Specifically, when the function coverage and code coverage of the target test case for the functional module to be tested reach the preset coverage threshold, it means that the test case tests the functional module more comprehensively and can cover enough functions and lines of code, so its case test results are used as the test results to be merged. For example, the preset function coverage threshold is 90%, and the code coverage threshold is 90%. If the function coverage of a test case is 95% and the code coverage is 95%, its results can be used as the test results to be merged. If the function coverage or code coverage of the target test case does not reach the preset coverage threshold, it means that the test case does not fully cover the functions or codes of the functional module, and there may be some functions that have not been tested, and its case test results are used as abnormal test results. When the case test result of the target test case is an abnormal test result, the test case is marked as a test case to be optimized. This shows that the current test case is insufficient and needs to be improved later.

[0080] Specifically, in order to improve the comprehensiveness of the test, new target test cases are added to the target test case set to replace the test cases to be optimized. The new test case design will pay more attention to covering functions and codes that have not been fully tested before, and the test process will be repeated until the test case test results obtained are the test results to be integrated. Finally, multiple test results to be integrated are integrated, and the results of each test case are comprehensively considered to finally obtain the functional test results of the functional module to be tested. Fusion processing can be carried out in a variety of ways, such as counting the number of passed test cases, analyzing the problems found in different test cases, etc., to comprehensively evaluate whether the functions of the functional modules are normal.

[0081] Specifically, in one embodiment, for the test cases to be optimized, artificial intelligence and machine learning technologies can be used to analyze the reasons for the low coverage, conduct in-depth analysis of the code structure and logic of the functional modules to be tested, and use code analysis tools to mine uncovered code areas and functions. Then, based on the uncovered code areas and functions, new target test cases are automatically and dynamically generated to improve the coverage of the test cases.

[0082] Based on the above embodiment, as an implementable manner, in one embodiment, the method further includes:

[0083] Step 301, generating a corresponding functional test report according to the functional test result of the functional module to be tested;

[0084] Step 302: Perform risk assessment on the functional module to be tested according to the functional test report to obtain a risk assessment result; wherein the risk assessment result at least includes code complexity, historical defect rate and functional importance;

[0085] Step 303: Determine the test case delivery strategy for the functional module to be tested based on the risk assessment result.

[0086] Specifically, each target test case will generate log records during the execution process. The log records include the execution status of each test step in the target test case. The functional test report of the functional module to be tested can be generated by summarizing the test results and log records of each target test case. Among them, the functional test results of the functional module to be tested include the test results and log records of each target test case.

[0087] Specifically, factors such as the code structure, logical nesting depth, and function call relationship of the functional module can be analyzed through the functional test report to evaluate the complexity of the code. Complex code means higher maintenance costs and potential error risks. For the statistics of the historical defect rate, the number and type of defects that occurred in the functional module during previous tests can be referred to calculate the historical defect rate. If a functional module frequently has problems during previous tests, it indicates that its stability is poor and the risk is high. For the evaluation of functional importance, its functional importance can be determined according to the role of the functional module in the entire system and the degree of impact on the user experience. Specifically, its functional importance can be determined according to the usage frequency of different functional modules by users. Finally, a comprehensive risk assessment result is obtained by integrating factors such as code complexity, historical defect rate, and functional importance. For example, a functional module with high code complexity, high historical defect rate, and high functional importance has a high-risk assessment result; while a functional module with low code complexity, low historical defect rate, and low functional importance has a low-risk assessment result.

[0088] Specifically, for functional modules with a high-risk assessment result, more comprehensive and strict test cases can be issued regularly to ensure the stability of the module. For low-risk functional modules such as simple auxiliary functional modules, relatively fewer test cases can be issued to improve the utilization rate of test cases.

[0089] Specifically, in one embodiment, the target number of target test cases corresponding to each attribute in the functional module to be tested can be determined according to the test case distribution strategy of the functional module to be tested.

[0090] It should be noted that in any functional module of the firmware to be tested, any one attribute corresponds to multiple test cases (the set of attribute test cases). In step 2033 above, when screening target test cases, the test cases in each set of attribute test cases can be sorted by priority according to the target test range of the functional module to be tested, and then according to the previously determined target number of target test cases corresponding to each attribute in the functional module to be tested, the target number of test cases with the highest priority is extracted from the set of attribute test cases corresponding to each attribute as the target test case.

[0091] Among them, when the test resources (test equipment) are sufficient, the target quantity of the target test cases corresponding to each attribute in the function module to be tested can be appropriately increased; when the test resources are scarce, the target quantity of the target test cases corresponding to each attribute in the function module to be tested can be appropriately decreased.

[0092] Exemplarily, as Figure 3 shown, it is a schematic diagram of the overall process of the firmware testing method provided by an embodiment of the present application. First, firmware code developers such as R & D engineers develop function modules locally according to R & D requirements. For example, the Identify function module. After the local development is completed, the code of this function (the latest developed code) is submitted to the code management tool Gerrit and added to the code repository of Gerrit. Through the linkage relationship established between Gerrit and Jenkins, the update of the code repository of Gerrit automatically triggers Jenkins to implement the test function for this code. Jenkins pulls the changed code this time to the local and compiles a new firmware using this code, that is, the firmware to be tested is obtained. Then, through the firmware binary extractor, the function module of the firmware to be tested is divided and the attributes are analyzed, and then by creating a mapping database, a test case database (preset test case library) is obtained. After determining the function module to be tested with changes this time, automatic allocation of test cases is performed based on the preset test case library, and then by executing the test cases, a function test report of the function module to be tested is obtained. Among them, when Gerrit determines that the function test result of the function module to be tested is passed, the latest developed code is coupled to the development code data packet of the development firmware; if the function test result is not passed, a prompt message is generated for the developer to prompt that the latest developed code uploaded by him fails the function test, and the developer needs to modify the code again to ensure the security of the firmware development.

[0093] The firmware testing method provided by the embodiments of this application includes obtaining the latest development code of the firmware to be tested; determining at least one function module to be tested with changes based on the latest development code, where the firmware to be tested includes multiple function modules; for any function module to be tested, determining the target test case set of the function module to be tested according to the change type and attribute characteristics of the function module to be tested; and performing multiple function tests on the function module to be tested based on the target test case set to obtain the function test result of the function module to be tested. The method provided by the above solution locates the function modules to be tested with changes in the firmware to be tested, enables the function testing of the firmware to be tested to be carried out in units of function modules, and can automatically screen and distribute the target test cases according to the change type and attribute characteristics of the function modules to be tested, improving the testing efficiency of the firmware to be tested, while ensuring the accuracy and consistency of the function testing. Moreover, by automatically analyzing code changes through the Spark NLP natural language processing model, automatically establishing a mapping table of functions according to the code changes, querying the corresponding test case set through the mapping table, automatically generating a test plan, and directly calling device testing, the entire process does not require manual participation, forming a complete CI / CD process, and can accurately test the changed points, adapting to the changing test requirements, improving the testing efficiency while ensuring the effectiveness and practicality of the test plan.

[0094] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0095] The embodiments of this application also provide a firmware testing device for executing the firmware testing method provided by the above embodiments.

[0096] As Figure 4 shown, it is a schematic structural diagram of the firmware testing device provided by the embodiments of this application. The firmware testing device 40 includes: an acquisition module 401, a first determination module 402, a second determination module 403, and a testing module 404.

[0097] Among them, the acquisition module is used to obtain the latest development code of the firmware to be tested; the first determination module is used to determine at least one function module to be tested with changes based on the latest development code, where the firmware to be tested includes multiple function modules; the second determination module is used to, for any function module to be tested, determine the target test case set of the function module to be tested according to the change type and attribute characteristics of the function module to be tested; the testing module is used to perform multiple function tests on the function module to be tested based on the target test case set to obtain the function test result of the function module to be tested.

[0098] For the description of the features in the corresponding embodiments of the firmware testing apparatus, reference may be made to the relevant descriptions in the corresponding embodiments of the firmware testing method, which will not be elaborated here one by one.

[0099] An embodiment of the present application also provides an electronic device, such as Figure 5 As shown, it is a schematic structural diagram of the electronic device provided by the embodiment of the present application, including a processor 10 and a memory 20. A computer program is stored in the memory 20, and the processor 10 is configured to run the computer program to execute the steps in any of the above-mentioned firmware testing method embodiments.

[0100] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. Wherein, the computer program is configured to execute the steps in any of the above-mentioned firmware testing method embodiments when running.

[0101] In an exemplary embodiment, the above-mentioned computer-readable storage medium may include, but is not limited to: USB flash drive, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disk, magnetic disk or optical disc and other various media that can store computer programs.

[0102] An embodiment of the present application also provides a computer program product. The above-mentioned computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-mentioned firmware testing method embodiments.

[0103] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-mentioned firmware testing method embodiments.

[0104] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0105] The above has introduced in detail a firmware testing method, an electronic device, a storage medium, and a program product provided by the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and modifications can still be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A firmware testing method, characterized in that, Including: Obtain the latest development code of the firmware to be tested; Determine at least one function module to be tested with changes according to the latest development code; wherein, the firmware to be tested includes multiple function modules; For any one of the function modules to be tested, determine the target test case set of the function module to be tested according to the change type and attribute characteristics of the function module to be tested; Based on the target test case set, perform multiple function tests on the function module to be tested to obtain the function test result of the function module to be tested; For any one of the function modules to be tested, determining the target test case set of the function module to be tested according to the change type and attribute characteristics of the function module to be tested includes: For any one of the function modules to be tested, determine the target test range according to the change type of the function module to be tested; According to the attribute characteristics of the function module to be tested, screen the test case sets corresponding to each attribute in the preset test case library; For any one of the attributes, screen at least one target test case in the test case set of the attribute according to the target test range; Determine the target test case set of the function module to be tested according to at least one target test case corresponding to each attribute; Wherein, the change type at least includes three types: function addition, function modification, and function optimization; The performing multiple function tests on the function module to be tested based on the target test case set to obtain the function test result of the function module to be tested includes: Deploy the firmware to be tested to multiple test devices; Send the target test case set to the multiple test devices, so as to run each target test case in the target test case set based on the test devices to perform multiple function tests on the function module to be tested, and obtain the use case test result corresponding to each target test case; Determine the function test result of the function module to be tested according to the use case test result corresponding to each target test case.

2. The firmware testing method according to claim 1, wherein The determining at least one function module to be tested with changes according to the latest development code includes: Extract keywords from the latest development code to obtain the keyword extraction result of the latest development code; Determine at least one function module to be tested with changes according to the keyword extraction result of the latest development code.

3. The firmware testing method according to claim 2, wherein The determining at least one function module to be tested with changes according to the keyword extraction result of the latest development code includes: Determine the main function module with direct changes according to the keyword extraction result of the latest development code; Determine the slave function module associated with the main function module according to the association relationship between the function modules in the firmware to be tested; Take the main function module and the slave function module as the function modules to be tested.

4. The firmware testing method according to claim 3, wherein The determining the main function module with direct changes according to the keyword extraction result of the latest development code includes: Determine the directly changed attribute according to the keyword extraction result of the latest development code; In the case where any attribute of any function module has a direct change, take this function module as the main function module with direct changes.

5. The firmware testing method according to claim 1, wherein The method further includes: Obtain the attribute characteristics of each functional module of the firmware to be tested; According to the attribute characteristics of each functional module, create multiple test cases, and establish the corresponding relationship among each functional module, attribute, and the test cases; Create the preset test case library according to the corresponding relationship among each functional module, attribute, and the test cases; 6. The firmware testing method according to claim 1, characterized in that The method further includes: Obtain the historical test records of the firmware to be tested; Determine the change type of the functional module to be tested according to the historical test records; 7. The firmware testing method according to claim 1, wherein The step of sending the target test case set to the multiple test devices includes: Classify the multiple test devices according to the configuration information of each test device to obtain multiple test device pools; For any target test case in the target test case set, determine the target test device pool of the target test case according to the test type of the target test case; When any target test device in the target test device pool is in an idle state, send the target test case to the target test device to run the target test case based on the target test device to obtain the test result corresponding to the target test case; Among them, the test types of test cases are at least divided into two types: compatibility test and performance test; 8. The firmware testing method according to claim 1, wherein The step of determining the functional test result of the functional module to be tested according to the test results corresponding to each target test case includes: For any target test case, determine the function coverage rate and code coverage rate of the target test case for the functional module to be tested according to the test result corresponding to the target test case; When the function coverage rate and code coverage rate of the target test case for the functional module to be tested both reach the preset coverage rate threshold, use the test result of the target test case as the test result to be fused; When the function coverage rate or code coverage rate of the target test case for the functional module to be tested does not reach the preset coverage rate threshold, use the test result of the target test case as an abnormal test result; When the test result of any target test case is an abnormal test result, use the target test case as a test case to be optimized, and add a new target test case to the target test case set to replace the test case to be optimized until the obtained test result is the test result to be fused; Perform a fusion process on multiple test results to be fused to obtain the functional test result of the functional module to be tested; 9. The firmware testing method according to claim 1, characterized in that The method further includes: Generate a corresponding functional test report according to the functional test result of the functional module to be tested; Perform a risk assessment on the functional module to be tested according to the functional test report to obtain a risk assessment result; where the risk assessment result at least includes code complexity, historical defect rate, and functional importance; Determine the test case sending strategy of the functional module to be tested according to the risk assessment result; 10. The firmware testing method according to claim 9, wherein The method further includes: Determine the target quantity of the target test case corresponding to each attribute in the functional module to be tested according to the test case sending strategy of the functional module to be tested; 11. An electronic device, characterized in that, Includes: A memory for storing a computer program; A processor for implementing the steps of the firmware testing method according to any one of claims 1 to 10 when executing the computer program.

12. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein the computer program implements the steps of the firmware testing method according to any one of claims 1 to 10 when executed by a processor.

13. A computer program product, comprising a computer program, characterized in that, The computer program implements the steps of the firmware testing method according to any one of claims 1 to 10 when executed by a processor.

Images