Firmware testing method, electronic equipment, storage medium and program product

Abstract

The invention discloses a firmware testing method, electronic equipment, a storage medium and a program product, and relates to the technical field of computers, functional testing of to-be-tested firmware is carried out with a functional module as a unit by positioning a modified to-be-tested functional module in the to-be-tested firmware, and according to the modification type and the attribute feature of the to-be-tested functional module, the to-be-tested firmware can be tested according to the modification type and the attribute feature of the to-be-tested functional module. Automatic screening and issuing of the target test case are realized, and the test efficiency of the to-be-tested firmware is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a firmware testing method, electronic equipment, storage medium and program product. Background Art

[0002] As users' business demands for products such as storage devices and servers continue to increase, the product firmware is also constantly iterating and updating. Therefore, the development of firmware code has become an important task in product development. Before the newly developed firmware is put into use in the product, a series of functional tests are required to ensure its security.

[0003] In the related art, the method of manually sending test cases is usually adopted to perform functional tests on the firmware code to be tested. However, as the firmware is constantly updated, the functions supported by the firmware become more and more complex, and more and more test cases need to be sent. The current method of sending test cases will reduce the efficiency of firmware testing. Summary of the invention

[0004] The present application provides a firmware testing method, electronic device, storage medium and program product to at least solve the problem of reducing the firmware testing efficiency in the related art.

[0005] The present application provides a firmware testing method, comprising: Get the latest development code of the firmware to be tested; According to the latest development code, determining at least one functional module to be tested that has been modified; wherein the firmware to be tested includes multiple functional modules; For any function module to be tested, determine the target test case set of the function module to be tested according to the modification type and attribute characteristics of the function module to be tested; Based on the target test case set, multiple functional tests are performed on the functional module to be tested to obtain the functional test results of the functional module to be tested.

[0006] The present application also provides a firmware testing device, comprising: The acquisition module is used to obtain the latest development code of the firmware to be tested; A first determination module is used to determine at least one functional module to be tested that has been modified according to the latest development code; wherein the firmware to be tested includes multiple functional modules; The second determination module is used to determine, for any function module to be tested, a target test case set of the function module to be tested according to the modification type and attribute characteristics of the function module to be tested; The test module is used to perform multiple functional tests on the functional module to be tested based on the target test case set, and obtain the functional test results of the functional module to be tested.

[0007] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned firmware testing methods when executing the computer program.

[0008] The present application also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned firmware testing methods are implemented.

[0009] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned firmware testing methods when executed by a processor.

[0010] Through the present application, the modified functional modules to be tested are located in the firmware to be tested, so that the functional test of the firmware to be tested is performed in units of functional modules, and the target test cases can be automatically screened and issued according to the modification types and attribute characteristics of the functional modules to be tested, thereby improving the test efficiency of the firmware to be tested. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 This is a schematic diagram of the structure of the firmware testing system based on the embodiments of the present application; Figure 2 A flowchart of a firmware testing method provided in an embodiment of the present application; Figure 3 A schematic diagram of the overall process of the firmware testing method provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of a firmware testing device provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0015] In the related art, manual delivery of test cases has many disadvantages, such as low efficiency, prone to errors, lack of consistency, difficulty in tracking and management, high communication costs, lack of automation support, and difficulty in adapting to rapid changes. These disadvantages limit the efficiency, accuracy and consistency of testing.

[0016] In order to solve the above technical problems, the embodiments of the present application provide a firmware testing method, an electronic device, a storage medium and a program product, the method comprising: obtaining the latest development code of the firmware to be tested; determining at least one functional module to be tested that has been modified based on the latest development code; wherein the firmware to be tested includes multiple functional modules; for any functional module to be tested, determining a target test case set for the functional module to be tested based on the modification type and attribute characteristics of the functional module to be tested; based on the target test case set, performing multiple functional tests on the functional module to be tested to obtain functional test results of the functional module to be tested. The method provided by the above scheme locates the modified functional module to be tested in the firmware to be tested, so that the functional test of the firmware to be tested is performed in units of functional modules, and can realize automatic screening and distribution of target test cases based on the modification type and attribute characteristics of the functional module to be tested, thereby improving the testing efficiency of the firmware to be tested and ensuring the accuracy and consistency of the functional test.

[0017] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0018] In conjunction with the specific application environment architecture or the specific hardware architecture on which the execution of the firmware testing method depends, the specific application environment architecture or the specific hardware architecture is described herein.

[0019] First, the structure of the firmware test system on which this application is based is described: The firmware testing method, electronic device, storage medium and program product provided in the embodiments of the present application are suitable for functional testing of firmware development codes of products such as servers and storage devices. Figure 1As shown, it is a schematic diagram of the structure of the firmware test system based on the embodiment of the present application, which mainly includes the firmware to be tested, a data acquisition device and a firmware test device. Among them, the data acquisition device is used to collect the latest development code provided by the developer to improve the firmware to be tested, and provide the obtained latest development code to the firmware test device, and the firmware test device performs targeted functional model testing on the firmware to be tested according to the obtained latest development code.

[0020] The embodiment of the present application provides a firmware testing method for functional testing of firmware development code of products such as servers and storage devices. The execution subject of the embodiment of the present application is an electronic device, such as a server, a desktop computer, a laptop computer, a tablet computer, and other electronic devices that can be used for development code testing.

[0021] like Figure 2 FIG. 1 is a flow chart of a firmware testing method provided in an embodiment of the present application, and the method includes: Step 201, obtaining the latest development code of the firmware to be tested.

[0022] Specifically, in the continuous integration and continuous delivery (CI / CD) process of the firmware, after completing the development of any functional module, the developer submits the development code to the code management tool Gerrit. Through the linkage relationship established between Gerrit and Jenkins, when the Gerrit code repository is updated, Jenkins automatically pulls the modified code to obtain the latest development code of the firmware to be tested. Among them, Jenkins is an open source continuous integration and continuous delivery deployment tool.

[0023] Step 202: Determine at least one modified function module to be tested according to the latest development code.

[0024] The firmware to be tested includes multiple functional modules.

[0025] Specifically, the latest development code obtained can be analyzed to determine identification information such as function names in the modified code, and then based on the identification information in the modified code, determine which functions in the firmware to be tested have been changed in the latest development code.

[0026] Step 203 : for any function module to be tested, determine a target test case set of the function module to be tested according to the modification type and attribute characteristics of the function module to be tested.

[0027] It should be noted that the types of changes include at least three types: function addition, function modification and function optimization. The attribute characteristics are used to characterize which attributes are included in the functional module to be tested. The attributes included in the functional module to be tested are the functions supported by the functional module to be tested. The attributes include Object Access Control System (OACS), Access Control List (ACL) and Network Performance and Security System (NPSS).

[0028] Specifically, according to the modification type and attribute characteristics of the functional module to be tested, multiple target test cases that meet the functional test requirements of the functional module to be tested can be screened in the preset test case library to obtain a target test case set for the functional module to be tested.

[0029] Step 204 , based on the target test case set, multiple functional tests are performed on the functional module to be tested to obtain functional test results of the functional module to be tested.

[0030] Specifically, by running each target test case in the target test case set, multiple functional tests can be implemented on the functional module to be tested, and the case test results corresponding to each target test case can be obtained. Finally, by summarizing the case test results of each target test case, the functional test results of the functional module to be tested can be obtained.

[0031] Based on the above embodiment, as an implementable manner, in one embodiment, determining at least one functional module to be tested that has been modified according to the latest development code includes: Step 221, extracting keywords from the latest development code to obtain keyword extraction results of the latest development code; Step 2022: Determine at least one modified function module to be tested based on the keyword extraction result of the latest development code.

[0032] Among them, keywords include attribute names and function names, etc.

[0033] Specifically, based on the Spark NLP natural language processing model, keywords such as attribute names and function names can be extracted from the latest development code to obtain the keyword extraction results of the latest development code. Then, based on the keyword extraction results, it is determined which functional module the latest development code belongs to, and then the modified functional module is used as the modified functional module to be tested.

[0034] Specifically, in one embodiment, the main function module that has been directly modified can be determined based on the keyword extraction results of the latest development code; the slave function module that has an association relationship with the main function module can be determined based on the association relationship between the various function modules in the firmware to be tested; and the main function module and the slave function module are used as the function modules to be tested.

[0035] Specifically, the extracted keywords can be matched with the pre-defined characteristic keywords of each functional module. If the characteristic keywords of a functional module overlap with the keywords in the extraction result, it is determined that the functional module has been directly changed and it is used as the main functional module. Since each functional module in the firmware to be tested usually does not exist in isolation, there are various associations between them, such as call relationships and data dependency relationships. Therefore, after determining the main functional module, it is necessary to further find the slave functional modules associated with it.

[0036] Specifically, during the firmware development process, an association model between various functional modules can be established in advance, and the association model is used to record the call path, data flow and other information between various functional modules. After the main functional module is determined, the input and output of the main functional module, other modules called, and the situation of being called by other modules can be analyzed by querying the association model to determine the associated slave functional modules. By determining the slave functional modules and including them in the test scope, potential problems caused by the slave functional modules being affected by the changes in the main functional modules can be avoided from being missed.

[0037] Specifically, in one embodiment, the attributes that are directly modified can be determined based on the keyword extraction results of the latest development code; when any attribute of any functional module is directly modified, the functional module is regarded as the main functional module that is directly modified.

[0038] Specifically, each functional module has a specific set of attributes, which together define the behavior and characteristics of the functional module. When any attribute in a functional module is directly changed, it is indicated that the behavior or characteristics of the functional module may have changed, so the functional module is regarded as the main functional module that has undergone direct changes.

[0039] Based on the above embodiments, as an implementable manner, in one embodiment, for any functional module to be tested, according to the modification type and attribute characteristics of the functional module to be tested, a target test case set of the functional module to be tested is determined, including: Step 2031, for any functional module to be tested, determine the target test range according to the modification type of the functional module to be tested; Step 2032, according to the attribute characteristics of the functional module to be tested, filter the test case set corresponding to each attribute in the preset test case library; Step 2033, for any attribute, according to the target test scope, filter at least one target test case in the test case set of the attribute; Step 2034: Determine a target test case set for the functional module to be tested according to at least one target test case corresponding to each attribute.

[0040] Among them, the types of changes include at least three types: function addition, function modification and function optimization.

[0041] It should be noted that different types of changes correspond to different test scopes. For function addition, it is necessary to test whether the basic functions of the new functions are normal and whether they are compatible with the existing system; for function modification, it is necessary to test the functional correctness of the modified part and whether it affects other related functions; for function optimization, the focus is on testing whether the performance indicators after optimization meet the optimization expectations.

[0042] Specifically, after confirming the target test scope, at least one target test case for each attribute matching the target test scope can be screened in the preset test case library according to the target test scope, thereby obtaining a target test case set for the functional module to be tested. By determining the target test scope according to the modification type and screening the test cases according to the attribute characteristics, the content to be tested can be accurately located to avoid testing irrelevant content, thereby improving the targetedness of the test and the test efficiency and comprehensiveness.

[0043] Specifically, in one embodiment, the attribute characteristics of each functional module of the firmware to be tested can be obtained; based on the attribute characteristics of each functional module, multiple test cases are created, and a correspondence between each functional module, attribute and test case is established; based on the correspondence between each functional module, attribute and test case, a preset test case library is created.

[0044] Specifically, in order to clearly express which attribute of which functional module is tested for each test case, the embodiment of the present application pre-establishes a correspondence between the functional modules, attributes and test cases. Specifically, this correspondence can be recorded in the form of a database or table. For example, a table is created in which one column records the functional module name, one column records the attribute name, and one column records the corresponding test case number. In actual applications, all test cases corresponding to a certain attribute of a certain functional module can be quickly located by querying the table.

[0045] Specifically, in one embodiment, historical test records of the firmware to be tested may be obtained; and the modification type of the functional module to be tested may be determined according to the historical test records.

[0046] Specifically, if in the historical test records, the previous test did not involve the functional module, that is, the functional module to be tested or a certain attribute in the functional module to be tested has not been functionally tested before, then the change type of the functional module to be tested is determined to be function addition. If in the historical test records, the last functional test result of the functional module to be tested was failed, then the modification type of this time is determined to be function modification. If in the historical test records, the last functional test result of the functional module to be tested was passed, then the modification type of this time is determined to be function optimization.

[0047] Based on the above embodiment, as an implementable manner, in one embodiment, based on the target test case set, multiple functional tests are performed on the functional module to be tested to obtain the functional test results of the functional module to be tested, including: Step 2041, deploying the firmware to be tested to multiple test devices; Step 2042, sending the target test case set to multiple test devices, so as to run each target test case in the target test case set based on the test device, so as to perform multiple functional tests on the function module to be tested, and obtain the test case test result corresponding to each target test case; Step 2042: Determine the functional test result of the functional module to be tested according to the test case test results corresponding to each target test case.

[0048] Specifically, by deploying the firmware to be tested on multiple test devices, a wider range of actual usage scenarios can be simulated, thereby discovering possible problems of the firmware to be tested in different environments. In addition, by using multiple test devices to execute each target test case in the target test case set in parallel, the running time of the test case is reduced, further improving the efficiency of firmware function testing.

[0049] Specifically, in one embodiment, multiple test devices can be classified according to the configuration information of each test device to obtain multiple test device pools; for any target test case in the target test case set, the target test device pool of the target test case is determined according to the test type of the target test case; when any target test device in the target test device pool is in an idle state, the target test case is sent to the target test device to run the target test case based on the target test device to obtain the use case test result corresponding to the target test case.

[0050] Among them, the test types of test cases are divided into at least two types: compatibility testing and performance testing. Compatibility testing mainly focuses on the operation of the firmware under test in different environments to ensure that the software can work normally on devices with various configurations; performance testing focuses on evaluating the performance indicators of the firmware under test, such as response time, throughput, and resource utilization.

[0051] It should be noted that the configuration information of the test equipment includes hardware configuration information and software configuration information. The hardware configuration information includes CPU model, memory size, hard disk capacity and graphics card performance, etc. The software configuration information includes operating system type and version, etc. Different configurations will affect the performance of the test equipment in running test cases.

[0052] Specifically, for compatibility test cases, they can be run on multiple test device pools with different configurations to cover more software usage scenarios. For example, to test the compatibility of an application under different operating system and browser combinations, the test cases can be assigned to device pools with different operating systems and browsers installed. Performance test cases usually need to be run on device pools with specific configurations to ensure the accuracy and comparability of the test results. For example, to test the performance ceiling of the software in a high-configuration environment, you can choose a device pool with a higher configuration. In the process of issuing target test cases, the status of each device in the target test device pool can be continuously monitored to determine whether it is in an idle state. The idle state means that the device is not currently performing other test tasks and can receive new test cases. Therefore, when any target test device in the target test device pool is in an idle state, the corresponding target test case can be issued to the target test device.

[0053] In the process of issuing test cases, the order of issuing test cases can be determined according to the priority of each attribute in the functional module to be tested, for example, the test cases related to the core computing service are issued first. The priority of each attribute in the functional module can be determined according to the frequency of user use of the service and the importance of each attribute in the functional module.

[0054] Specifically, in one embodiment, for any target test case, the function coverage and code coverage of the target test case for the functional module to be tested are determined according to the use case test results corresponding to the target test case; when the function coverage and code coverage of the target test case for the functional module to be tested reach the preset coverage threshold, the use case test result of the target test case is used as the test result to be merged; when the function coverage or code coverage of the target test case for the functional module to be tested does not reach the preset coverage threshold, the use case test result of the target test case is used as an abnormal test result; when the use case test result of any target test case is an abnormal test result, the target test case is used as a test case to be optimized, and a new target test case is added to the target test case set to replace the test case to be optimized, until the obtained use case test result is a test result to be merged; multiple test results to be merged are merged to obtain the functional test result of the functional module to be tested.

[0055] It should be noted that function coverage refers to the ratio of the functions executed in the function module to be tested to the total functions when running a certain target test case. For example, the function module to be tested has 10 functions, and when running the target test case, 6 functions are executed, so the function coverage is 60%. Code coverage refers to the ratio of the lines of code executed in the function module to be tested to the total lines of code when running the target test case. For example, the function module to be tested has a total of 100 lines of code, and after the test case is executed, 70 lines of code are executed, and the code coverage is 70%. By analyzing the test case test results corresponding to the target test case, these execution information can be collected, thereby calculating the function coverage and code coverage. The embodiment of the present application ensures that the test case can cover the various parts of the function module to be tested as much as possible by paying attention to function coverage and code coverage, avoiding the occurrence of test blind spots, so as to improve the accuracy of the test results.

[0056] Specifically, when the function coverage and code coverage of the target test case for the functional module to be tested reach the preset coverage threshold, it means that the test case tests the functional module more comprehensively and can cover enough functions and lines of code, so its case test results are used as the test results to be merged. For example, the preset function coverage threshold is 90%, and the code coverage threshold is 90%. If the function coverage of a test case is 95% and the code coverage is 95%, its results can be used as the test results to be merged. If the function coverage or code coverage of the target test case does not reach the preset coverage threshold, it means that the test case does not fully cover the functions or codes of the functional module, and there may be some functions that have not been tested, and its case test results are used as abnormal test results. When the case test result of the target test case is an abnormal test result, the test case is marked as a test case to be optimized. This shows that the current test case is insufficient and needs to be improved later.

[0057] Specifically, in order to improve the comprehensiveness of the test, new target test cases are added to the target test case set to replace the test cases to be optimized. The new test case design will pay more attention to covering functions and codes that have not been fully tested before, and the test process will be repeated until the test case test results obtained are the test results to be integrated. Finally, multiple test results to be integrated are integrated, and the results of each test case are comprehensively considered to finally obtain the functional test results of the functional module to be tested. Fusion processing can be carried out in a variety of ways, such as counting the number of passed test cases, analyzing the problems found in different test cases, etc., to comprehensively evaluate whether the functions of the functional modules are normal.

[0058] Specifically, in one embodiment, for the test cases to be optimized, artificial intelligence and machine learning technologies can be used to analyze the reasons for the low coverage, conduct in-depth analysis of the code structure and logic of the functional modules to be tested, and use code analysis tools to mine uncovered code areas and functions. Then, based on the uncovered code areas and functions, new target test cases are automatically and dynamically generated to improve the coverage of the test cases.

[0059] Based on the above embodiment, as an implementable manner, in one embodiment, the method further includes: Step 301, generating a corresponding functional test report according to the functional test result of the functional module to be tested; Step 302: Perform risk assessment on the functional module to be tested according to the functional test report to obtain a risk assessment result; wherein the risk assessment result at least includes code complexity, historical defect rate and functional importance; Step 303: Determine the test case delivery strategy for the functional module to be tested based on the risk assessment result.

[0060] Specifically, each target test case will generate a log record during execution, and the log record includes the execution status of each test step in the target test case. By summarizing the case test results and log records of each target test case, a functional test report of the functional module to be tested can be generated. Among them, the functional test results of the functional module to be tested include the case test results and log records of each target test case.

[0061] Specifically, the code structure, logic nesting depth, function call relationship and other factors of the functional module can be analyzed through the functional test report to evaluate the complexity of the code. Complex code means higher maintenance costs and potential error risks. For the statistics of historical defect rates, the historical defect rates can be calculated by referring to the number and type of defects that occurred in the functional module in previous tests. If a functional module frequently encounters problems during the previous test, it means that its stability is poor and the risk is high. For the evaluation of functional importance, the functional importance of the functional module can be determined according to its role in the entire system and the degree of impact on the user experience. Specifically, the functional importance can be determined according to the frequency of user use of different functional modules. Finally, a comprehensive risk assessment result is obtained by combining factors such as code complexity, historical defect rate and functional importance. For example, a functional module with high code complexity, high historical defect rate and high functional importance has a high risk risk assessment result; while a functional module with low code complexity, low historical defect rate and low functional importance has a low risk risk assessment result.

[0062] Specifically, for function modules with high risk in risk assessment results, more comprehensive and stricter test cases can be issued regularly to ensure the stability of the module. For low-risk function modules such as simple auxiliary function modules, the number of test cases issued can be relatively small to improve the utilization rate of test cases.

[0063] Specifically, in one embodiment, the target number of target test cases corresponding to each attribute in the functional module to be tested may be determined according to the test case delivery strategy of the functional module to be tested.

[0064] It should be noted that, in any functional module of the firmware to be tested, any attribute corresponds to multiple test cases (attribute test case set). In the above step 2033, when screening the target test cases, the test cases in each attribute test case set can be prioritized according to the target test range of the functional module to be tested, and then according to the target number of target test cases corresponding to each attribute in the functional module to be tested previously determined, the target number of test cases with the highest priority in the attribute test case set corresponding to each attribute are extracted as target test cases.

[0065] Among them, when the test resources (test equipment) are sufficient, the target number of target test cases corresponding to each attribute in the functional module to be tested can be appropriately increased; when the test resources are scarce, the target number of target test cases corresponding to each attribute in the functional module to be tested can be appropriately reduced.

[0066] For example, Figure 3As shown, the overall process diagram of the firmware testing method provided by the embodiment of the present application is that first, the firmware code developers such as R&D engineers develop functional modules locally according to R&D requirements, such as the Identify functional module. After the local development is completed, the code of the function (the latest development code) is submitted to the code management tool Gerrit and added to the code repository of Gerrit. Through the linkage relationship established between Gerrit and Jenkins, the update of the code repository of Gerrit automatically triggers Jenkins to implement the test function for the code. Jenkins pulls the modified code to the local and uses the code to compile a new firmware to obtain the firmware to be tested. Then, through the firmware binary extractor, the functional modules of the firmware to be tested are divided and the attributes are analyzed, and then a test case database (preset test case library) is obtained by creating a mapping database. After determining the functional module to be tested that has been modified this time, the test case is automatically allocated based on the preset test case library, and then the functional test report of the functional module to be tested is obtained by executing the test case. Among them, when Gerrit determines that the functional test result of the functional module to be tested has passed, it couples the latest development code to the development code data package of the development firmware; if the functional test result is failed, a prompt message is generated to the developer to remind him that the latest development code he uploaded has not passed the functional test, and the developer needs to modify the code again to ensure the security of firmware development.

[0067] The firmware testing method provided in the embodiment of the present application obtains the latest development code of the firmware to be tested; determines at least one functional module to be tested that has been modified based on the latest development code; wherein the firmware to be tested includes multiple functional modules; for any functional module to be tested, determines the target test case set of the functional module to be tested based on the modification type and attribute characteristics of the functional module to be tested; based on the target test case set, performs multiple functional tests on the functional module to be tested to obtain the functional test results of the functional module to be tested. The method provided by the above scheme locates the modified functional module to be tested in the firmware to be tested, so that the functional test of the firmware to be tested is performed in units of functional modules, and can realize automatic screening and distribution of target test cases based on the modification type and attribute characteristics of the functional module to be tested, thereby improving the testing efficiency of the firmware to be tested and ensuring the accuracy and consistency of the functional test. In addition, the Spark NLP natural language processing model is used to automatically analyze code changes, automatically create a function mapping table based on code changes, query the corresponding test case set through the mapping table, automatically generate a test plan, and directly call the device test. The entire process does not require manual participation, forming a complete CI / CD process. It can also accurately test the change points and adapt to the ever-changing test requirements. While improving the test efficiency, it ensures the effectiveness and practicality of the test plan.

[0068] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0069] An embodiment of the present application also provides a firmware testing device for executing the firmware testing method provided in the above embodiment.

[0070] like Figure 4 FIG. 4 is a schematic diagram of the structure of a firmware testing device provided in an embodiment of the present application. The firmware testing device 40 comprises: an acquisition module 401 , a first determination module 402 , a second determination module 403 and a testing module 404 .

[0071] Among them, the acquisition module is used to obtain the latest development code of the firmware to be tested; the first determination module is used to determine at least one functional module to be tested that has been changed based on the latest development code; wherein the firmware to be tested includes multiple functional modules; the second determination module is used to determine the target test case set of the functional module to be tested according to the change type and attribute characteristics of the functional module to be tested for any functional module to be tested; the testing module is used to perform multiple functional tests on the functional module to be tested based on the target test case set to obtain the functional test results of the functional module to be tested.

[0072] The description of the features in the embodiment corresponding to the firmware testing device can refer to the relevant description of the embodiment corresponding to the firmware testing method, which will not be repeated here.

[0073] The embodiment of the present application also provides an electronic device, such as Figure 5 As shown, it is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, including a processor 10 and a memory 20, in which a computer program is stored, and the processor 10 is configured to run the computer program to execute the steps in any one of the above-mentioned firmware testing method embodiments.

[0074] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above firmware testing method embodiments when running.

[0075] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0076] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above firmware testing method embodiments are implemented.

[0077] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned firmware testing method embodiments are implemented.

[0078] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0079] The above is a detailed introduction to a firmware testing method, electronic device, storage medium and program product provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A firmware testing method, characterized in that: include: Get the latest development code of the firmware to be tested; According to the latest development code, determining at least one functional module to be tested that has been modified; wherein the firmware to be tested includes multiple functional modules; For any of the functional modules to be tested, determining a target test case set for the functional module to be tested according to the modification type and attribute characteristics of the functional module to be tested; Based on the target test case set, multiple functional tests are performed on the functional module to be tested to obtain functional test results of the functional module to be tested.

2. The firmware testing method according to claim 1, characterized in that: The step of determining at least one functional module to be tested that has been modified according to the latest development code includes: Perform keyword extraction on the latest developed code to obtain a keyword extraction result of the latest developed code; At least one modified function module to be tested is determined according to the keyword extraction result of the latest development code.

3. The firmware testing method according to claim 2, characterized in that: The step of determining at least one functional module to be tested that has been modified according to the keyword extraction result of the latest development code includes: Determine the main functional module that has been directly modified based on the keyword extraction result of the latest development code; Determine, according to the association relationship between the functional modules in the firmware to be tested, a slave functional module that has an association relationship with the master functional module; The master function module and the slave function module are used as the function modules to be tested.

4. The firmware testing method according to claim 3, characterized in that: The step of determining the main functional modules that are directly modified based on the keyword extraction result of the latest development code includes: Determine the attributes that are directly modified based on the keyword extraction results of the latest development code; In the case that any attribute of any of the functional modules is directly modified, the functional module is regarded as the main functional module that is directly modified.

5. The firmware testing method according to claim 1, characterized in that: The step of determining, for any of the functional modules to be tested, a target test case set for the functional module to be tested according to the modification type and attribute characteristics of the functional module to be tested, comprises: For any of the functional modules to be tested, determining a target test range according to the modification type of the functional module to be tested; According to the attribute characteristics of the functional module to be tested, the test case set corresponding to each attribute is selected in the preset test case library; For any of the attributes, according to the target test scope, filter at least one target test case in the test case set of the attribute; Determine a target test case set for the functional module to be tested according to at least one target test case corresponding to each of the attributes; Among them, the modification types include at least three types: function addition, function modification and function optimization.

6. The firmware testing method according to claim 5, characterized in that: The method further comprises: Obtaining attribute characteristics of each functional module of the firmware to be tested; According to the attribute characteristics of each of the functional modules, a variety of test cases are created, and a corresponding relationship between each of the functional modules, the attributes and the test cases is established; The preset test case library is created according to the correspondence between the functional modules, the attributes and the test cases.

7. The firmware testing method according to claim 5, characterized in that: The method further comprises: Obtaining historical test records of the firmware to be tested; According to the historical test records, the modification type of the functional module to be tested is determined.

8. The firmware testing method according to claim 1, characterized in that: The step of performing multiple functional tests on the functional module to be tested based on the target test case set to obtain functional test results of the functional module to be tested includes: Deploy the firmware to be tested to multiple test devices; The target test case set is sent to the multiple test devices, so as to run each target test case in the target test case set based on the test device, so as to perform multiple functional tests on the functional module to be tested, and obtain a test case test result corresponding to each target test case; The functional test results of the functional modules to be tested are determined according to the test case test results corresponding to the target test cases.

9. The firmware testing method according to claim 8, characterized in that: The sending the target test case set to the multiple test devices includes: Classifying the multiple test devices according to the configuration information of each test device to obtain multiple test device pools; For any target test case in the target test case set, determining a target test device pool for the target test case according to the test type of the target test case; When any target test device in the target test device pool is in an idle state, the target test case is sent to the target test device, so as to run the target test case based on the target test device to obtain a test result corresponding to the target test case; Among them, the test types of test cases are divided into at least two types: compatibility testing and performance testing.

10. The firmware testing method according to claim 8, characterized in that: Determining the functional test result of the functional module to be tested according to the test case test results corresponding to each of the target test cases includes: For any of the target test cases, determine the function coverage and code coverage of the target test case for the functional module to be tested according to the test case test result corresponding to the target test case; When both the function coverage and the code coverage of the target test case on the functional module to be tested reach a preset coverage threshold, the test result of the target test case is used as the test result to be merged; When the function coverage or code coverage of the target test case for the functional module to be tested does not reach a preset coverage threshold, the test result of the target test case is regarded as an abnormal test result; When the case test result of any of the target test cases is an abnormal test result, the target test case is used as a test case to be optimized, and a new target test case is added to the target test case set to replace the test case to be optimized, until the obtained case test result is a test result to be integrated; The plurality of test results to be merged are merged to obtain the functional test result of the functional module to be tested.

11. The firmware testing method according to claim 1, characterized in that: The method further comprises: Generate a corresponding functional test report according to the functional test results of the functional module to be tested; According to the functional test report, risk assessment is performed on the functional module to be tested to obtain a risk assessment result; wherein the risk assessment result at least includes code complexity, historical defect rate and functional importance; According to the risk assessment result, a test case delivery strategy for the functional module to be tested is determined.

12. The firmware testing method according to claim 11, characterized in that: The method further comprises: According to the test case sending strategy of the functional module to be tested, a target number of target test cases corresponding to each attribute in the functional module to be tested is determined.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the firmware testing method according to any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the firmware testing method according to any one of claims 1 to 12 when executed by a processor.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the firmware testing method according to any one of claims 1 to 12 are implemented.

Images