Firmware testing methods, apparatus, equipment, storage media and testing systems

Abstract

This application provides a firmware testing method, apparatus, device, storage medium, and testing system. The method is applied to a testing system, which includes a test management device and a DUT (Device Under Test), wherein the test management device and the DUT are connected by a communication link. The method includes: obtaining a test task corresponding to the system boot firmware through the test management device; wherein the test task includes at least one sub-task file, each sub-task file corresponding to a sub-firmware of the system boot firmware; and assigning each sub-task file of the test task to different sets of DUTs through the test management device to obtain test results. This allows for the splitting and testing of the system boot firmware. Furthermore, it enables parallel testing of two versions of the system boot firmware, reducing the overall testing cycle and improving overall testing efficiency.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 202210731205.1, entitled "Processing Method, Firmware Testing Method, Apparatus, Device, System, and Storage Medium", filed on June 24, 2022. TECHNICAL FIELD

[0002] The present application relates to the field of testing, in particular to a firmware testing method, apparatus, device, storage medium, and testing system. BACKGROUND

[0003] System startup firmware is the core firmware in a computer device, and the security and reliability of system firmware directly affect the security and availability of the computer device. Therefore, after a version of system startup firmware is developed, the system startup firmware needs to be tested to determine the availability of the version of system startup firmware. SUMMARY

[0004] Embodiments of the present application aim to provide a firmware testing method, apparatus, device, storage medium, and testing system to implement testing of system startup firmware.

[0005] To implement testing of system startup firmware, embodiments of the present application provide a firmware testing method, which is applied to a testing system. The testing system includes a computer device and a device under test (DUT) in communication with the computer device. The DUT is equipped with system startup firmware. The method includes:

[0006] The computer device obtains a test task corresponding to the system startup firmware. The test task includes at least one sub-task file, and each sub-task file corresponds to a sub-firmware of the system startup firmware. Each sub-firmware of the system startup firmware is used to implement a stage of the system startup process. The computer device assigns each sub-task file of the test task to a different set of DUTs to obtain a test result. Each set of DUTs is composed of at least one DUT.

[0007] In the implementation process, the computer device respectively allocates the sub-task files corresponding to the sub-firmwares of the system startup firmware to different DUT sets to be tested to obtain test results. In this way, the system startup firmware can be split and tested. In addition, in the development process, multiple versions of the system startup firmware may be continuously developed. In the implementation process, different DUT sets can be used to execute corresponding sub-task files. If a new version of the system startup firmware is updated during the execution of the test task and needs to be tested, the new version of the system startup firmware can be rearranged in the DUT set that has executed the sub-task files and tested, so that the parallel testing of the new and old versions of the system startup firmware can be implemented, the overall testing period can be reduced, and the overall testing efficiency can be improved.

[0008] Further, the respective allocation of the sub-task files of the test task to different DUT sets to be tested includes: according to the running sequence of the sub-firmwares of the system startup firmware, allocating the sub-task file corresponding to a sub-firmware running first to the DUT set to be tested corresponding to the sub-task file, and after receiving the execution completion signal returned by the DUT set to be tested, sending the sub-task file corresponding to the next sub-firmware to the DUT set to be tested corresponding to the sub-task file.

[0009] In the implementation process, the respective allocation of the sub-task files of the test task to different DUT sets to be tested includes: according to the running sequence of the sub-firmwares of the system startup firmware, allocating the sub-task file corresponding to a sub-firmware running first to the DUT set to be tested corresponding to the sub-task file, and after receiving the execution completion signal returned by the DUT set to be tested, sending the sub-task file corresponding to the next sub-firmware to the DUT set to be tested corresponding to the sub-task file.

[0010] Further, the system startup firmware includes: a trusted firmware running in a trusted execution environment and a normal firmware running in a normal execution environment; the trusted firmware runs before the normal firmware; the test task includes: a first sub-task file corresponding to the trusted firmware and a second sub-task file corresponding to the normal firmware; and according to the running sequence of the sub-firmwares of the system startup firmware, allocating the sub-task file corresponding to a sub-firmware running first to the DUT set to be tested corresponding to the sub-task file includes: allocating the first sub-task file to the DUT set to be tested corresponding to the first sub-task file for execution; and after receiving the execution completion signal of the first sub-task file, allocating the second sub-task file to the DUT set to be tested corresponding to the second sub-task file for execution.

[0011] In the implementation process, the trusted firmware running in the trusted execution environment is tested first, and then the normal firmware running in the normal execution environment is tested, so that the test result reliability can be ensured according to the system startup process.

[0012] Further, the trusted firmware includes BL1 firmware, BL2 firmware, BL31 firmware and BL32 firmware; the running order of each trusted firmware is BL1 firmware, BL2 firmware, BL31 firmware and BL32 firmware in sequence; the first sub-task file includes a first sub-task file corresponding to the BL1 firmware, a first sub-task file corresponding to the BL2 firmware, a first sub-task file corresponding to the BL31 firmware and a first sub-task file corresponding to the BL32 firmware.

[0013] Further, the normal firmware includes BL33 firmware and operating system (OS) firmware; the BL33 firmware runs before the OS firmware; the second sub-task file includes a second sub-task file corresponding to the BL33 firmware and a second sub-task file corresponding to the OS firmware.

[0014] Further, each DUT in the DUT set receiving the first sub-task file executes the first sub-task file in the trusted execution environment; each DUT in the DUT set receiving the second sub-task file executes the second sub-task file in the normal execution environment.

[0015] In the implementation process, the first sub-task file is executed in the trusted execution environment, and the second sub-task file is executed in the normal execution environment, so that the test safety of the sub-firmware with higher safety requirements can be improved.

[0016] Embodiments of the present application also provide a firmware test device applied to a computer device of a test system, the test system including the computer device and a DUT under test (DUT) in communication connection with the computer device; the DUT under test is equipped with a system startup firmware; the device includes an acquisition unit configured to acquire a test task corresponding to the system startup firmware; the test task includes at least one sub-task file, and each sub-task file corresponds to a sub-firmware of the system startup firmware; each sub-firmware of the system startup firmware is used to implement a stage of a system startup process; and an allocation unit configured to allocate each sub-task file of the test task to different DUT sets for execution, so as to obtain a test result through execution results of DUTs constituting each DUT set.

[0017] The embodiment of the present application further provides a test system, comprising: a computer device, and a DUT to be tested in communication connection with the computer device; the DUT to be tested is provided with a system startup firmware;

[0018] The computer device is used for obtaining a test task corresponding to the system startup firmware, and respectively assigning each subtask file of the test task to different DUT sets to be tested, so as to obtain a test result; wherein the test task comprises at least one subtask file, and each subtask file corresponds to a subfirmware of the system startup firmware; each subfirmware of the system startup firmware is used for implementing a stage of a system startup process; wherein each DUT set to be tested is composed of at least one DUT to be tested;

[0019] The DUT to be tested is used for executing the assigned subtask file, and feeding back an execution result of the subtask file to the computer device, so that the computer device determines the test result according to the execution result.

[0020] The embodiment of the present application further provides a computer device, comprising a communication module, a processor and a memory; the communication module is used for being in communication connection with a DUT to be tested; the processor is used for executing one or more programs stored in the memory, so as to implement the processing method of any one of the above or implement the firmware test method of any one of the above.

[0021] The embodiment of the present application further provides a computer readable storage medium, which stores one or more programs; the one or more programs can be executed by one or more processors, so as to implement the processing method of any one of the above or implement the firmware test method of any one of the above. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of the drawings.

[0023] Figure 1 A basic structure schematic diagram of a test system provided by the embodiment of the present application;

[0024] Figure 2 A flow schematic diagram of a processing method provided by the embodiment of the present application;

[0025] Figure 3 A structure schematic diagram of a specific test system provided by the embodiment of the present application;

[0026] Figure 4 A flowchart of a firmware testing method provided in an embodiment of the present application is shown in FIG. 1.

[0027] Figure 5 A modular diagram of a system startup firmware provided in an embodiment of the present application is shown in FIG. 2.

[0028] Figure 6 A more specific structural diagram of a testing system provided in an embodiment of the present application is shown in FIG. 3.

[0029] Figure 7 A structural diagram of a processing device provided in an embodiment of the present application is shown in FIG. 4.

[0030] Figure 8 A structural diagram of a firmware testing device provided in an embodiment of the present application is shown in FIG. 5.

[0031] Figure 9 A basic structural diagram of a computer device provided in an embodiment of the present application is shown in FIG. 6.

[0032] Figure 10 A structural diagram of a computer device in a software and hardware dual level provided in an embodiment of the present application is shown in FIG. 7. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. It should be noted that the embodiments, the implementation manners and features provided in the embodiments of the present application can be combined with each other without conflict.

[0034] Embodiment one:

[0035] In order to reduce the labor cost and time cost required for generating a test task and improve the testing efficiency, a testing system is provided in an embodiment of the present application. As shown in FIG. 1, the testing system includes a computer device. In addition, the testing system can also include a DUT to be tested which is in communication connection with the computer device. The computer device and the DUT to be tested can cooperate with each other to realize the testing of a chip or firmware. Figure 1

[0036] In the embodiments of the present application, a processing method which can be applied to a computer device in a testing system is also provided, as shown in FIG. 2, the method includes the following steps. Figure 2

[0037] S201: Obtain a target test task to be cropped.

[0038] ​​It should be understood that in the embodiments of the present application, one or more test tasks can be pre-configured in the computer device. The target test task is the currently selected test task that needs to be pruned for chip or firmware testing.

[0039] In the embodiments of the present application, one or more subtask files can be included in the target test task, and each subtask file can include attribute information of one or more test subitems. The subtask file is the smallest unit that is issued to the DUT to be tested for execution.

[0040] It should be noted that the so-called test subitem refers to each subitem that needs to be tested in the entire test process. The test of each test subitem is usually implemented through a configured test subitem case.

[0041] Therefore, in an optional implementation of the embodiments of the present application, each test subitem case pre-configured can be stored in one or more test case servers or databases. Then, the address of the device where the corresponding test subitem case of the required test subitem is located, and the unique identifier of the test subitem case corresponding to the test subitem (for example, it can be the file name of the test subitem case) are recorded in the subtask file as the attribute information of the test subitem. Therefore, when the DUT to be tested executes the subtask file, the required test subitem case can be downloaded and executed based on the address of the device where the test subitem case is located and the unique identifier of the test subitem case recorded in the subtask file, so as to implement the test of the corresponding test subitem.

[0042] It should be noted that in another optional implementation of the embodiments of the present application, the test subitem case of the test subitem can also be written into the subtask file as the attribute information of the test subitem. Therefore, when the DUT to be tested executes the subtask file, the test subitem case in the subtask file can be directly executed to implement the test of the test subitem.

[0043] S202: Obtain a target test subitem that does not meet a preset condition based on historical test data of each test subitem of the target test task.

[0044] It should be understood that in the embodiments of the present application, the computer device can receive and save all test process data, test result data and test completion time length generated when each DUT to be tested executes the subtask file. The test process data can include but is not limited to abnormal data generated in the test process, error data generated in the test process, etc. The test result data can include data representing the success of the test subitem test and data representing the failure of the test subitem test. The test completion time length refers to the total time length from the start of execution to the end of execution of the test subitem case of the test subitem executed by the DUT to be tested.

[0045] It should also be understood that, in the embodiments of this application, the preset conditions can be set by an engineer. For example, the preset conditions may include at least one of the following:

[0046] Condition 1: The number of errors occurring in the test item is less than the first threshold.

[0047] Condition 2: The number of times the test result of the test sub-item satisfies the preset result (i.e. the test is successful) is greater than the second threshold;

[0048] Condition 3: The test completion time of the tested sub-item is greater than the third threshold.

[0049] The specific values ​​of the first, second, and third thresholds can be determined by the engineer. For example, the first threshold can be set to a small constant value, such as 1; the second threshold can be set to a large constant value, such as 100; and the third threshold can be set to a large constant value, such as 10 days. These values ​​are merely illustrative examples and are not intended to limit the scope of this application.

[0050] In this way, by taking the test items that have fewer than the first threshold number of errors during testing, or the test items that meet the preset results more than the second threshold number of times, as the target test items, step S203 can effectively eliminate these test items that have no significant impact on the test effect, thereby retaining only the test items with better test effects, reducing test time and improving test efficiency.

[0051] Furthermore, in actual testing, if a certain test item takes a particularly long time during the testing process, it will seriously drag down the efficiency of the entire testing process. Therefore, in order to improve testing efficiency, test items whose test completion time exceeds the third threshold can be deleted as target test items, which can also reduce testing time and improve testing efficiency.

[0052] However, for the scheme where any test sub-item is determined as a target test sub-item as long as it meets condition 3, it should be noted that if the test sub-item is a necessary test item to complete the test task, it should not be deleted. Therefore, in one feasible implementation of this application embodiment, before deleting test sub-items whose test completion time exceeds the third threshold, the test sub-items whose test completion time exceeds the third threshold can be first fed back to the engineer for deletion confirmation. After the engineer confirms that deletion is permissible, the deletion can then be performed.

[0053] Furthermore, in another feasible implementation of this application, the scheme of determining any test sub-item as a target test sub-item as long as it meets condition 3 can also be achieved by searching the test sub-item test case library for a test sub-item test case that performs the same test function as the test sub-item but has a shorter test completion time. If no such test sub-item exists, it will not be determined as a target test sub-item. If such a test sub-item exists, its attribute information can be deleted, and the attribute information of the test sub-item corresponding to the test sub-item test case that performs the same test function as the test sub-item but has a shorter test completion time can be added to the target test task. This avoids deleting necessary test items.

[0054] It should be understood that, for conditions 1 and 2, the target test items determined based on conditions 1 and / or 2 are test items that have little impact on the test results, and therefore are not necessary test items. Therefore, they do not need to be sent to engineers for confirmation, or the search and replacement of test cases with substantially the same test function can be omitted.

[0055] Optionally, conditions 1 to 3 above can be used simultaneously, or only some of them can be used. For example, conditions 1 and 2 can be used simultaneously to determine the target test items, thereby eliminating test items that do not have a significant impact on the test results.

[0056] Optionally, when determining the target test items using both conditions 1 and 2, after identifying the test items that satisfy conditions 1 and 2, further evaluation can be performed on these test items to determine if they meet condition 3. Test items that meet condition 3 are then deleted as target test items. This controls the number of deleted test items and avoids a situation where too few test items remain after deleting target test items.

[0057] S203: Delete the attribute information of the target test sub-item in the target test task.

[0058] Optionally, considering that in practical applications, there may be execution dependencies between certain test sub-items (i.e., a test sub-item's execution depends on the execution result of a test sub-item executed earlier), in such cases, if the attribute information of the test sub-item tested earlier in the two test sub-items with execution dependencies is deleted, the attribute information of the test sub-item tested later can be fed back to the engineer. This allows the engineer to modify the test sub-item's test case or change the attribute information of the test sub-item tested later, ensuring that all test sub-items in the target test task are testable.

[0059] Optionally, in this embodiment of the application, after deleting the attribute information of the target test sub-item in the target test task, feedback can also be given to the engineer on the deleted target test sub-item so that the engineer can decide whether to accept the deletion, or so that the engineer can add attribute information of the new test sub-item to the computer device, so as to ensure the richness and comprehensiveness of the items to be tested in the target test task.

[0060] Optionally, in one feasible implementation of this application, after deleting the attribute information of the target test sub-items in the target test task, the sub-task files of the target test task can be reconstructed based on the test completion time of each test sub-item whose attribute information has not been deleted.

[0061] For example, test sub-items can be divided based on their completion time, minimizing the maximum total test time among the sets of sub-items. Then, a new subtask file is generated for each set of sub-items. This allows different subtask files to be assigned to different DUTs during subsequent testing, thus minimizing the time required for all subtask files to be executed.

[0062] In this embodiment, all partitioning methods can be exhausted first to determine the maximum total test duration corresponding to each test sub-item set divided by each partitioning method, and then the partitioning method with the minimum maximum total test duration can be selected. If multiple partitioning methods exist with the minimum maximum total test duration, then the partitioning method with the smallest fluctuation in the pairwise difference between the total test durations corresponding to each test sub-item set (i.e., the smallest mean squared error of the difference) can be selected to partition each test sub-item, thereby generating a new subtask file.

[0063] For example, assuming A, B, and C are three test sub-items, there are several ways to divide them: Method 1: Divide into three sets, with each set containing test sub-items A, B, and C respectively; Method 2: Divide into two sets, with each set containing test sub-items A, B, and C respectively; Method 3: Divide into two sets, with each set containing test sub-items A, B, and C respectively; Method 4: Divide into one set, with each set containing test sub-items A, B, and C. Assuming the test completion times for these three test sub-items A, B, and C are 1 hour, 2 hours, and 2 hours respectively, then Method 1 has a maximum total test time of 2 hours, which is shorter than the other methods. Therefore, Method 1 can be used for partitioning, and then three sub-task files can be generated based on test sub-items A, B, and C respectively. Assuming the test completion times for the three test sub-items A, B, and C are 1 hour, 2 hours, and 4 hours respectively, then the maximum total test time for both Method 1 and Method 2 is 4 hours, which is shorter than the other methods. However, the difference fluctuation in Method 1 is more drastic than that in Method 2. Therefore, Method 2 can be used for the division, and then a sub-task file can be generated based on test sub-items A and B, and C respectively, and a sub-task file can be generated based on test sub-item C.

[0064] It should also be noted that when partitioning, the number of sets created can be less than or equal to the number of DUTs under test. This facilitates the subsequent allocation of subtask files and avoids situations where some subtask files need to be allocated due to insufficient DUTs. For example, continuing with the previous example, if the number of DUTs under test is 2, then method 1 can be directly excluded.

[0065] It should be noted that if there are execution dependencies between the test sub-items (i.e., the execution of a certain test sub-item depends on the execution result of a test sub-item that was executed earlier), then the execution order of each sub-task file needs to be generated according to the execution dependencies between the test sub-items to ensure that each test sub-item can be executed correctly.

[0066] Of course, in this embodiment, it is also possible not to rebuild the sub-task files of the target test task, and this embodiment does not impose any restrictions on this.

[0067] Optionally, in one feasible implementation of this application, if there is no execution dependency between the test sub-items and the number of subtask files to be divided is greater than the number of DUTs to be tested, after deleting the attribute information of the target test sub-items in the target test task, the test sub-items can be sorted in descending order based on the error rate of each test sub-item whose attribute information has not been deleted; wherein, test sub-items with the same error rate are sorted in ascending order according to the test completion time.

[0068] At this point, the sorted test items can be divided to obtain the required number of subtask files. This allows for priority testing of test items with high error rates, and among test items with the same error rate, priority testing of those with shorter completion times. This ensures that the point at which execution failure occurs is brought forward as much as possible during the entire testing process, thereby reducing testing time and improving testing efficiency.

[0069] Optionally, when dividing the subtask files, the difference in the total test completion time of each subtask file can be minimized based on the test completion time of each test item.

[0070] For example, suppose there are five test items a, b, c, d, and e, with error rates of 30%, 33%, 31%, 26%, and 26% respectively, and test completion times of 5 hours, 2 hours, 6 hours, 8 hours, and 7 hours respectively. Then the following order can be obtained: b, c, a, e, d.

[0071] Assuming we need to divide the task into two sub-task files, we can divide them according to their order of arrangement. We can divide b, c, and a into one sub-task file, and e and d into another, minimizing the difference in total test completion time between the sub-task files. (In this division, the total test completion time for b, c, and a in one sub-task file is 13 hours, while the total test completion time for e and d is 15 hours, a difference of 2 hours. Other division methods include: dividing b into a separate sub-task file and c, a, e, and d into another; dividing b and c into one sub-task file and a, e, and d into another; and dividing b, c, a, and e into one sub-task file and d into another. These division methods result in a difference in total test completion time between the sub-task files greater than 2 hours.)

[0072] It should be understood that, in the embodiments of this application, the testing phase can be formally entered after deleting the attribute information of the target test sub-item or after reconstructing the sub-task files of the target test task.

[0073] When conducting tests, if the test object is a chip, the chip to be tested needs to be installed on the DUT; if the test object is firmware, a normal processor needs to be installed on the DUT, and then the firmware to be tested needs to be installed.

[0074] Before testing, the DUT needs to be started. In this embodiment, a remote power-on command can be sent to the DUT via a computer device, and then the subtasks corresponding to the DUT can be sent to the DUT for execution.

[0075] It should be noted that, in one feasible implementation of the embodiments of this application, see... Figure 3 As shown, the test system may also include a network switch, through which the computer device connects to each DUT. A microcontroller (e.g., an STM32 microcontroller) is deployed on the DUT, and the microcontroller is connected to the GPIO (General-purpose input / output) port of the DUT. Thus, the computer device can send a remote power-on command to the microcontroller of the DUT through the network switch, enabling the microcontroller to power on the DUT via the GPIO port after receiving the remote power-on command.

[0076] In the embodiments of this application, the computer device can assign each subtask file of the target test task to different sets of DUTs to be tested, so that each set of DUTs can execute it and return the test results.

[0077] It should be understood that in the embodiments of this application, each set of DUTs consists of at least one DUT. The set of DUTs can be randomly selected from all DUTs, or it can be preset by the engineer.

[0078] In this embodiment, when there are sub-task files with an execution order among the sub-task files of the target test task, these sub-task files can be sent to the corresponding DUT set according to their execution order. After receiving the execution completion signal from the DUT set, the next sub-task file of that sub-task file is then sent to the corresponding DUT set. Sub-task files without an execution order can be randomly sent to any available DUT set.

[0079] For example, if the sub-task files of the target test task have an execution order, then according to the execution order of the sub-task files, the first executed sub-task file will be sent to the DUT set corresponding to that sub-task file. After receiving the execution completion signal returned by the DUT set, the next sub-task file of that sub-task file will be sent to the DUT set corresponding to that next sub-task file, until all sub-task files have been sent.

[0080] In this approach, if the firmware under test is updated to a new version during the execution of the test task, and it is necessary to test the new version of the firmware under test, the new version of the firmware under test can be rearranged and tested in the set of DUTs that have completed the subtask files. This enables parallel testing of the old and new versions of the firmware under test, thereby reducing the overall test cycle and improving the overall test efficiency.

[0081] For example, if the sub-task files of the target test task do not have an execution order, the sub-task files can be randomly sent to each set of DUTs under test, so that each set of DUTs under test executes the sub-task files in parallel. This makes the total test completion time equal to the execution completion time of the longest sub-task file, thereby effectively reducing the test cycle and improving test efficiency.

[0082] It should be understood that in the above example method, if the number of DUTs available for allocation of subtask files is less than the number of subtask files that need to be allocated, then some subtask files will need to wait in line for allocation.

[0083] In this embodiment of the application, in order to save testing time and improve testing efficiency, the entire testing task can be deemed to have failed when any sub-task file of the testing task fails to be executed, thereby ending the test, so as to shorten the testing cycle and improve testing efficiency.

[0084] Therefore, based on this, in order to maximize testing efficiency when the number of available subtask files for allocation is less than the number of subtask files to be allocated, and considering that the subtask files of the target test task do not have an execution order, the subtask files of the target test task can be pre-sorted according to the following sorting rules:

[0085] The sub-task files are sorted in descending order based on their error rates. If multiple sub-task files have the same error rate, they are sorted in ascending order based on their test completion time.

[0086] After the sorting is complete, the subtask files can be distributed sequentially according to the sorting order. In this way, if the number of DUTs available for subtask file allocation is less than the number of subtask files to be allocated, the subtask files that are more prone to errors and have shorter test completion times will be allocated first. This will allow the test tasks to fail as early as possible during the test process, thereby enabling the test to be terminated as soon as possible and reducing the total test time.

[0087] For example, suppose there are five subtask files A, B, C, D, and E, with error rates of 40%, 45%, 41%, 36%, and 36% respectively, and test completion times of 15 hours, 12 hours, 6 hours, 8 hours, and 7 hours respectively. Then the following order can be obtained: B, C, A, E, D.

[0088] Assuming there are 3 idle DUTs (Device Under Test), subtasks B, C, and A will be randomly assigned to these 3 DUTs for execution. During execution, if any of B, C, or A fails, the entire test ends. If any of B, C, or A completes execution before a failure occurs (based on the assumed test completion time, subtask file C completes first), then subtask file E is assigned to the DUT that was originally executing subtask file C for execution. Assuming no subtasks fail during execution, based on the assumed test completion time, if subtask file B completes next, then subtask file D is assigned to the DUT that was originally executing subtask file B for execution.

[0089] It should be noted that, in the embodiments of this application, the error rate of the subtask file can be calculated based on the historical execution failure count and the historical total execution count of the subtask file (the error rate of the subtask file can be equal to the ratio of the historical execution failure count to the historical total execution count).

[0090] Furthermore, the error rate of a subtask file can also be calculated based on the error rates of each test item within that subtask file. For example, the average error rate of each test item in the subtask file can be taken as the error rate of the subtask file. The error rate of a test item can be calculated based on the historical number of failed executions of the test case for that test item and the historical total number of executions (the error rate of a test item can be equal to the ratio of the historical number of failed executions of the test case to the historical total number of executions).

[0091] It should also be noted that the subtask file allocation method described above can also be applied to test scenarios where the test object is a chip. The implementation method is the same as that for test scenarios where the test object is the firmware to be tested, so it will not be repeated here.

[0092] In this embodiment of the application, after receiving the subtask file, the DUT under test can first download and run the script of the test sub-item test case according to the address and unique identifier of the device where the test sub-item test case is located in the subtask file, and determine whether the test software required to run the script is in its own system. If not, it can download and install the required test software from the pre-designed firmware test software deployment server or software library.

[0093] It should be noted that, in the embodiments of this application, the computer device may be a server, a fixed terminal (such as a desktop computer, a console, etc.), a mobile terminal (such as a laptop computer, etc.), etc., but this is not a limitation.

[0094] Example 2:

[0095] It should be understood that system boot firmware is the core firmware of a computer device, and its security and reliability directly affect the security and usability of the computer device. Therefore, after developing a version of system boot firmware, it is necessary to test it to determine its usability.

[0096] To test the system boot firmware, this application provides a testing system. The structure of the testing system can be found in [reference needed]. Figure 1 As shown, the setup includes a computer device and a device under test (DUT). The computer device and the DUT are connected via a communication link, and the DUT contains the system boot firmware.

[0097] To enable testing of system boot firmware, this application embodiment also provides a firmware testing method applicable to computer devices using the provided testing system. (See [link to relevant documentation]). Figure 4 As shown, it includes:

[0098] S401: Obtain the test task corresponding to the system boot firmware.

[0099] It should be understood that, in this embodiment, the computer device may have one or more pre-configured test tasks. When testing the system boot firmware is required, the test task corresponding to the system boot firmware from these pre-configured test tasks is obtained. Of course, the test task corresponding to the system boot firmware can also be entered into the computer device by the engineer before testing. This embodiment does not limit the method of obtaining the test task corresponding to the system boot firmware.

[0100] In this embodiment, the test task may include one or more subtask files, each subtask file being the smallest unit sent to the DUT for execution. Each subtask file corresponds to a sub-firmware of the system startup firmware. Each sub-firmware of the system startup firmware is used to implement a stage of the system startup process.

[0101] S402: Assign the sub-task files of the test task to different sets of DUTs to obtain test results.

[0102] In this embodiment, each set of DUTs consists of at least one DUT. The set of DUTs can be randomly selected from all DUTs, or it can be preset by the engineer.

[0103] In this embodiment, the DUT (Device Under Test) executes the assigned sub-task files and reports the execution results to the computer device. The computer device can determine the overall test result based on the execution results returned by each DUT. For example, if any DUT returns an execution result indicating failure during the test, the computer device can directly determine that the entire test task has failed and can notify other DUTs to stop the test. If the computer device receives execution results indicating success from all DUTs, it can determine that the entire test task has passed.

[0104] In this embodiment of the application, in order to save testing time and improve testing efficiency, the entire testing task can be considered to have failed when any sub-task file of the testing task fails to be executed, thereby ending the test, so as to shorten the testing cycle and improve testing efficiency.

[0105] In one optional embodiment of this application, the subtask files corresponding to the first running sub-firmware can be assigned to the DUT set corresponding to that sub-task file according to the execution order of the sub-firmwares of the system startup firmware. After receiving the execution completion signal from the DUT set, the subtask file corresponding to the next sub-firmware can be sent to the DUT set corresponding to that sub-task file. In this way, the testing process of the system startup firmware can be guaranteed to be consistent with the execution process of the system startup firmware, thereby ensuring the test effect.

[0106] In this embodiment of the application, the hardware resources of the DUT can be divided into two parts: the secure world (trusted execution environment, i.e.) Figure 5 The Secure part of the code and the normal world (referring to the normal execution environment, i.e.) Figure 5 (Non-Secure section). The resource access permissions of a trusted execution environment and a normal execution environment are different. When the CPU (Central Processing Unit) of the DUT is running in a trusted execution environment, it can access all hardware resources, but when running in a normal execution environment, it can only access the resources of the normal execution environment.

[0107] In this embodiment, the system boot firmware may include trusted firmware running in a trusted execution environment and ordinary firmware running in a normal execution environment, with the trusted firmware running before the ordinary firmware. Correspondingly, the test task may include a first subtask file corresponding to the trusted firmware and a second subtask file corresponding to the ordinary firmware. Thus, during testing, each DUT (Device Under Test) first loads the system boot firmware, and then the computer device can first assign the first subtask file to the set of DUTs corresponding to the first subtask file for execution. After receiving the completion signal of the first subtask file, the computer device then assigns the second subtask file to the set of DUTs corresponding to the second subtask file for execution.

[0108] It should be noted that, in this embodiment, each DUT in the set of DUTs receiving the first subtask file can execute the first subtask file in a trusted execution environment; while each DUT in the set of DUTs receiving the second subtask file can execute the second subtask file in a normal execution environment. Thus, by executing the first subtask file in a trusted execution environment and the second subtask file in a normal execution environment, the testing security of sub-firmware with higher security requirements can be improved.

[0109] For example, such as Figure 5 As shown, the trusted firmware may include: BL1 firmware, BL2 firmware, BL31 firmware, and BL32 firmware. The ordinary firmware may include: BL33 firmware and OS (operating system) firmware; wherein, the execution order of the sub-firmwares of the system boot firmware is as follows: BL1 firmware, BL2 firmware, BL31 firmware, BL32 firmware, BL33 firmware, and OS firmware.

[0110] In this way, during testing, the computer device first sends the first subtask file corresponding to the BL1 firmware to the first set of DUTs under test for execution.

[0111] After receiving the test results of the first set of DUTs under test, if the test results of the first set of DUTs under test meet the preset results, the first subtask file corresponding to the BL2 firmware will be sent to the second set of DUTs under test for execution.

[0112] After receiving the test results of the second set of DUTs under test, if the test results of the second set of DUTs under test meet the preset results, the first subtask file corresponding to the BL31 firmware will be sent to the third set of DUTs under test for execution.

[0113] After receiving the test results of the third set of DUTs under test, if the test results of the third set of DUTs under test meet the preset results, the first subtask file corresponding to the BL32 firmware will be sent to the fourth set of DUTs under test for execution.

[0114] After receiving the test results of the fourth set of DUTs under test, if the test results of the fourth set of DUTs under test meet the preset results, the second subtask file corresponding to the BL33 firmware will be sent to the fifth set of DUTs under test for execution.

[0115] After receiving the test results of the fifth set of DUTs under test, if the test results of the fifth set of DUTs under test meet the preset results, the second subtask file corresponding to the OS firmware will be sent to the sixth set of DUTs under test for execution.

[0116] For example, such as Figure 5 As shown in the embodiments of this application, BL1 firmware can be Boot ROM (diskless boot ROM interface) firmware, BL2 firmware can be Platform initialization firmware, BL31 firmware can be EL3 15 runtime firmware, BL32 firmware can be OP-TEE, BL33 firmware can be U-Boot / UEFI firmware, and OS firmware can be Kernel (the kernel of the operating system OS) boot firmware.

[0117] It should be noted that the subtask file can contain attribute information for the test sub-items. A test sub-item refers to each sub-item that needs to be tested during the entire testing process, and the testing of each test sub-item is usually implemented through configured test case files.

[0118] If the test items corresponding to each sub-firmware are not related, the sub-task files corresponding to each sub-firmware can be randomly assigned to the set of idle DUTs to achieve parallel execution of the sub-task files corresponding to each sub-firmware, thereby improving test efficiency.

[0119] In one optional embodiment of this application, pre-configured test cases for each test sub-item can be stored in one or more test case servers or databases. Then, the address of the device where the corresponding test case for the required test sub-item is located, and the unique identifier of the test case for the corresponding test sub-item (e.g., the filename of the test case for the test sub-item) are recorded as attribute information of the test sub-item in the subtask file. Thus, when the DUT under test executes the subtask file, it can download the required test case for execution based on the address of the device where the test case for the test sub-item is located and the unique identifier of the test case for the test sub-item recorded in the subtask file, thereby realizing the testing of the corresponding test sub-item.

[0120] In another optional embodiment of this application, the test cases of the test sub-items can also be written into the subtask file as attribute information of the test sub-items, so that when the DUT under test executes the subtask file, it can directly execute the test cases of the test sub-items in the subtask file to realize the testing of the test sub-items.

[0121] It should be understood that the solution in this embodiment can also be implemented in combination with the solution in Embodiment 1. Specifically, the test task corresponding to the system startup firmware can be processed according to the processing method in Embodiment 1 to obtain the target test task. Of course, the solution in this embodiment can also be implemented independently of the solution in Embodiment 1, and this application does not limit this.

[0122] It should be noted that in this embodiment, each sub-firmware of the system startup firmware can correspond to one or more subtask files. When the target test task is obtained by processing according to the processing method in Embodiment 1, if it is necessary to reconstruct the subtask files, the subtask files corresponding to each sub-firmware can be reconstructed on a unit basis. The subtask file reconstruction process can be found in the relevant description in Embodiment 1, and will not be repeated here.

[0123] Optionally, if the sub-firmware of the system startup firmware needs to correspond to multiple sub-task files, and the number of sub-task files to be divided is greater than the number of DUTs in the DUT set corresponding to the sub-firmware, the test sub-items can be sorted in descending order based on the error rate of each test sub-item corresponding to the sub-firmware; among them, the test sub-items with the same error rate are sorted in ascending order according to the test completion time.

[0124] Similar to Example 1, the sorted test items can be divided to generate the required number of subtask files. This allows for priority testing of test items with high error rates, and among test items with the same error rate, priority testing of those with shorter completion times. This ensures that the time point of execution failure occurs as early as possible during the entire testing process, thereby reducing testing time and improving testing efficiency.

[0125] Similarly, when dividing the test, the difference in the total test completion time of each sub-task file can be minimized based on the test completion time of each sub-item.

[0126] Furthermore, when a sub-firmware corresponds to multiple sub-task files, if there is no execution dependency between these multiple sub-task files, when allocating the sub-task files corresponding to the sub-firmware, each sub-task file corresponding to the sub-firmware can be randomly assigned to each idle DUT in the set of DUTs to be tested corresponding to the sub-firmware for execution.

[0127] In this embodiment, to save testing time and improve testing efficiency, the entire test task can be considered a failure and the test terminated if any subtask file of the test task fails to execute. Furthermore, in actual testing, there may be situations where the number of idle DUTs in the set of DUTs under test is less than the number of subtask files to be allocated. Therefore, to maximize testing efficiency in this case, the subtask files corresponding to the sub-firmware can be pre-sorted according to the following sorting rules:

[0128] The sub-task files are sorted in descending order based on their error rates. If multiple sub-task files have the same error rate, they are sorted in ascending order based on their test completion time.

[0129] After the arrangement is complete, the subtask files can be distributed according to the arrangement order. In this way, if the number of idle DUTs in the set of DUTs corresponding to the sub-firmware is less than the number of subtask files to be allocated, the subtask files that are more prone to errors and have shorter test completion times will be allocated first. This will make the error time of the test task as early as possible during the test, so that the test can be stopped as soon as possible and the total test time can be reduced.

[0130] It should be noted that, in the embodiments of this application, the error rate of the subtask file can be calculated based on the historical execution failure count and the historical total execution count of the subtask file (the error rate of the subtask file can be equal to the ratio of the historical execution failure count to the historical total execution count).

[0131] Furthermore, the error rate of a subtask file can also be calculated based on the error rates of each test item within that subtask file. For example, the average error rate of each test item in the subtask file can be taken as the error rate of the subtask file. The error rate of a test item can be calculated based on the historical number of failed executions of the test case for that test item and the historical total number of executions (the error rate of a test item can be equal to the ratio of the historical number of failed executions of the test case to the historical total number of executions).

[0132] Similar to Embodiment 1, a startup operation is required before testing the DUT. In this embodiment, a remote power-on command can be sent to the DUT via a computer device, and then the subtasks corresponding to the DUT can be sent to the DUT for execution. The implementation method of remote power-on can be found in [link to relevant documentation]. Figure 3 The relevant descriptions of Example 1 will not be repeated here.

[0133] In this embodiment of the application, after the DUT under test powers on and receives the subtask file, it can request the system boot firmware to be tested from the preset system boot firmware storage database or storage server and update it, thereby performing the test process of the system boot firmware during the restart process.

[0134] Simultaneously, the DUT (Device Under Test) can download the scripts for the test sub-items based on the address and unique identifier of the device containing the test sub-items in the subtask file. It can then determine whether the required test software is present on its system. If not, it can download and install the necessary test software from a pre-designed firmware test software deployment server or software repository.

[0135] It should be noted that, in the embodiments of this application, the computer device may be a server, a fixed terminal (such as a desktop computer, a console, etc.), a mobile terminal (such as a laptop computer, etc.), etc., but this is not a limitation.

[0136] Example 3:

[0137] To facilitate understanding of the solutions provided in the embodiments of this application, the solutions in the embodiments of this application will be further explained below with reference to the figures.

[0138] Please see Figure 6 The test system shown includes a client, a firmware automated test server (i.e., the computer equipment mentioned above), a firmware test case and software deployment server, a network switch, a serial server, and a DUT. The DUT is equipped with an STM32 microcontroller and a serial port. The client is networked with the firmware automated test server. The firmware automated test server (i.e., the computer equipment mentioned above), the firmware test case and software deployment server, the network switch, and the serial server are also networked. The serial server and the DUT communicate via an RS232 serial port and the STM32 microcontroller.

[0139] During testing, you can follow these steps:

[0140] Step 1: When the firmware version of the firmware to be tested is updated, the firmware automated test server is triggered to obtain the target test task.

[0141] The process of obtaining the target test task includes: obtaining the test task of the previous version of the firmware to be tested.

[0142] The process of obtaining the target test task includes: obtaining the test task of the previous version of the firmware to be tested. Based on the test process data, test result data, and test completion time of each test sub-item in the test task of the previous version of the firmware to be tested, the attribute information of test sub-items in each test sub-task file that have fewer than a first threshold number of errors during testing and test sub-items whose test results meet the preset result more than a second threshold are deleted, thus obtaining the target test task. In addition, if there is no execution dependency between test sub-items, they can be sorted in descending order of error rate, and those with the same error rate can be sorted in ascending order of test completion time. The test order of the test sub-items is then reordered, and the sub-task files are re-divided and generated to obtain the target test task. The attribute information of the test sub-items includes the address of the device where the test sub-item test case is located and the unique identifier of the test sub-item test case corresponding to the test sub-item.

[0143] In this application, the target test task may include functional test subtask files, performance test subtask files, and stability test subtask files.

[0144] In this embodiment of the application, the content of the subtask file may include:

[0145]

[0146] Among them, test sub-item test cases refer to pre-written test scripts that can be run on the DUT.

[0147] It should be noted that, in this embodiment, the firmware to be tested can be the system boot firmware.

[0148] Step 2: The firmware automated test server assigns task IDs according to the submission or generation order of each subtask file of the target task and puts the subtask file into the task queue.

[0149] It should be noted that when the firmware to be tested is the system boot firmware, the firmware automated testing server puts the sub-task files corresponding to each sub-firmware into the task queue in the order of BL1 firmware, BL2 firmware, BL31 firmware, BL32 firmware, BL33 firmware and OS firmware.

[0150] Step 3: If the sub-task files have an execution order, that is, they must be executed in sequence, then proceed to Step 4; if the sub-task files do not have an execution order, that is, they do not need to be executed in sequence, then proceed to Step 5.

[0151] Step 4: The firmware automated test server determines the set of DUTs corresponding to each subtask file. It selects subtask files from the task queue in ascending order of ID, and then selects available DUTs from the set of DUTs corresponding to that subtask file to run the subtask file.

[0152] Once the test of a subtask file is completed on the DUT, the next subtask file is retrieved from the task queue and assigned to the corresponding DUT set. This process continues until all subtask files have been assigned. Skip to step 6.

[0153] Step 5: The firmware automated test server retrieves subtask files from the task queue in ascending order of ID, randomly selects available DUTs to run each subtask file, without waiting for previously assigned subtask files to complete, until all subtask files have been assigned.

[0154] Step 6: Wait for the test task to be triggered again.

[0155] The completion process for each subtask file is as follows:

[0156] Step 1: The firmware automated test server sends a remote power-on command to the DUT to connect to the DUT and sends the subtask file to the DUT.

[0157] The power-on / off command first passes through a serial port server, where it is converted into an RS232 serial signal. This RS232 serial signal is then converted into a TTL signal by an RS232-to-TTL (Transistor-to-Transistor Logic) converter. This TTL signal, through the TTL interface of the STM32 microcontroller, calls the pre-programmed power-on / off program within the STM32 microcontroller to send a power-on signal to the GPIO port of the DUT (Device Under Test), thus controlling the power-on of the DUT.

[0158] Step 2: The DUT downloads the firmware and performs a firmware update.

[0159] The DUT can download the firmware to be tested according to the download address of the firmware to be tested written in the subtask file, and perform the firmware update operation according to the firmware update command.

[0160] Step 3: The DUT (Device Under Test) downloads and runs the test sub-cases based on the firmware test cases, software deployment server address, and test sub-case filenames in the subtask file. It then determines whether the required test software, specified in the test sub-case, is present in its system. If not, it downloads and installs the test software from the firmware test cases and software deployment server.

[0161] It should be noted that, although Figure 5 In this paper, test sub-item test cases and test software are deployed on a single server. However, in actual applications, test sub-item test cases and test software can also be deployed on different servers or databases, and this application does not impose any restrictions on this.

[0162] Furthermore, the firmware automated test server and the firmware test case and software deployment server can be two different servers in terms of their physical implementation, or they can be the same server, and this application does not impose any restrictions on this.

[0163] Step 4: The DUT under test continues to execute the test sub-item test cases. If any test sub-item test case fails, the test ends, and a task failure message is sent to the firmware automated test server. If all test sub-item test cases corresponding to the sub-task file are successfully tested, the DUT under test sends a task completion message to the firmware automated test server.

[0164] During execution, all output information is uploaded to the firmware automated test server for storage.

[0165] It should be noted that when the firmware under test is the system boot firmware, the DUT will first perform a firmware update operation according to the firmware update command and restart the system, thereby executing the test sub-cases during the restart process.

[0166] Step 5: After receiving the task failure message, the firmware automated test server sends a task abort message to other DUTs under test, so that the other DUTs under test stop the sub-task files that are being executed, and cancels all sub-task files of the test task that have not yet been issued in the task queue.

[0167] Meanwhile, the firmware automated test server continues to allocate subtask files of other test tasks from the task queue to idle DUTs.

[0168] If the firmware automated test server receives a message from the DUT that the task has been successfully completed and the standard subtask file has been successfully executed, it will continue to assign subtask files from the task queue to the DUT.

[0169] At this point, the firmware automated test server can store relevant information during the test process in a file and display it on a webpage or sent to the client.

[0170] Example 4:

[0171] Based on the same inventive concept, this application also provides a processing device 700 and a firmware testing device 800 that can be applied to the aforementioned computer device. Please refer to... Figure 7 andFigure 8 As shown, Figure 7 It shows the use of Figure 2 The processing apparatus of the method shown Figure 8 It shows the use of Figure 4 The processing apparatus of the method shown. It should be understood that the specific functions of apparatus 700 and apparatus 800 can be found in the description above; to avoid repetition, detailed descriptions are appropriately omitted here. Apparatus 700 and apparatus 800 include at least one software functional module that can be stored in memory or embedded in the operating system of the aforementioned computer device in the form of software or firmware. Specifically:

[0172] See Figure 7 As shown, the device 700 is used in the computer equipment of the test system and includes:

[0173] The acquisition module 701 is used to acquire the target test task to be trimmed; the target test task includes at least one sub-task file, and each sub-task file includes attribute information of at least one test item;

[0174] The acquisition module 701 is also used to acquire target test sub-items that do not meet preset conditions based on the historical test data of each test sub-item of the target test task;

[0175] Processing module 702 is used to delete the attribute information of the target test sub-item.

[0176] In one feasible embodiment of this application, the historical test data includes the test process data, test result data, and test completion time of the test sub-item; wherein:

[0177] The target test item is used to characterize the test item that has fewer than a first threshold number of errors when being tested;

[0178] Alternatively, the target test item is used to characterize the test item whose test result satisfies the preset result more than the second threshold number of times;

[0179] Alternatively, the target test item can be used to characterize the test item whose test completion time is greater than a third threshold when it is being tested.

[0180] In this embodiment of the application, the attribute information of the test sub-item includes: the address of the device where the test sub-item test case is located, and the unique identifier of the test sub-item test case corresponding to the test sub-item.

[0181] In this embodiment of the application, the processing module 702 is further configured to, after deleting the attribute information of the target test sub-item, reconstruct each sub-task file of the target test task based on the test completion time of each test sub-item whose attribute information has not been deleted.

[0182] In this embodiment of the application, the processing module 702 is further configured to, after deleting the attribute information of the target test sub-item, sort the test sub-items in descending order based on the error rate of each test sub-item whose attribute information has not been deleted; wherein, the test sub-items with the same error rate are sorted in ascending order according to the test completion time.

[0183] In one feasible embodiment of this application, the test system further includes a device under test (DUT), and the computer device and the DUT establish a communication connection; the target test task is a test task for the firmware under test.

[0184] The processing module 702 is further configured to, after deleting the attribute information of the target test item, assign each sub-task file of the target test task to different sets of DUTs to be tested, so as to obtain the test results of each sub-task file; wherein each set of DUTs to be tested consists of at least one DUT to be tested.

[0185] In one optional example of the above feasible implementation, the processing module 702 is specifically used to, when there is an execution order among the sub-task files of the target test task, send the first executed sub-task file to the DUT set corresponding to the sub-task file according to the execution order of the sub-task files, and after receiving the execution completion signal returned by the DUT set, send the next sub-task file of the sub-task file to the DUT set corresponding to the next sub-task file.

[0186] In the above optional example, the DUT under test is equipped with system startup firmware; the firmware under test can be the system startup firmware; a sub-firmware of the system startup firmware is used to implement a stage of the system startup process;

[0187] Specifically, the processing module 702 can be used to allocate the subtask file corresponding to the first running sub-firmware to the set of DUTs corresponding to the subtask file according to the running order of each sub-firmware of the system startup firmware, and after receiving the execution completion signal returned by the set of DUTs, send the subtask file corresponding to the next sub-firmware to the set of DUTs corresponding to the subtask file.

[0188] In the above optional example, the system startup firmware includes: trusted firmware running in a trusted execution environment and ordinary firmware running in a normal execution environment; the trusted firmware runs before the ordinary firmware; the target test task includes: a first subtask file corresponding to the trusted firmware and a second subtask file corresponding to the ordinary firmware;

[0189] Specifically, the processing module 702 can be used to assign the first subtask file to the DUT set corresponding to the first subtask file for execution, and after receiving the execution completion signal of the first subtask file, assign the second subtask file to the DUT set corresponding to the second subtask file for execution.

[0190] In the above optional examples, the trusted firmware includes: BL1 firmware, BL2 firmware, BL31 firmware and BL32 firmware; the ordinary firmware includes: BL33 firmware and operating system OS firmware; wherein, the running order of each of the sub-firmwares of the system startup firmware is as follows: BL1 firmware, BL2 firmware, BL31 firmware, BL32 firmware, BL33 firmware, OS firmware.

[0191] The first subtask file includes: a first subtask file corresponding to the BL1 firmware, a first subtask file corresponding to the BL2 firmware, a first subtask file corresponding to the BL31 firmware, and a first subtask file corresponding to the BL32 firmware; the second subtask file includes: a second subtask file corresponding to the BL33 firmware, and a second subtask file corresponding to the OS firmware.

[0192] In the above optional example, each of the DUTs in the set of DUTs to be tested that receives the first subtask file executes the first subtask file in the trusted execution environment; each of the DUTs in the set of DUTs to be tested that receives the second subtask file executes the second subtask file in the normal execution environment.

[0193] In another optional example of the above feasible implementation, the processing module 702 is specifically used to randomly send each of the sub-task files to each idle set of DUTs when there is no execution order among the sub-task files of the test task.

[0194] See Figure 8 As shown, the device 800 is applied in the computer equipment of the aforementioned test system and includes:

[0195] The acquisition unit 801 is used to acquire the test task corresponding to the system startup firmware; wherein, the test task includes at least one subtask file, and each subtask file corresponds to a sub-firmware of the system startup firmware; each sub-firmware of the system startup firmware is used to implement a stage of the system startup process;

[0196] The allocation unit 802 is used to allocate each subtask file of the test task to different sets of DUTs to be tested for execution, so as to obtain the test results by the execution results of the DUTs constituting each set of DUTs to be tested.

[0197] In one feasible embodiment of this application, the allocation unit 802 is specifically used to allocate the subtask file corresponding to the first running sub-firmware to the DUT set corresponding to the subtask file according to the running order of each sub-firmware of the system startup firmware, and after receiving the execution completion signal returned by the DUT set, send the subtask file corresponding to the next sub-firmware to the DUT set corresponding to the subtask file.

[0198] In one optional example of the above feasible implementation, the system startup firmware includes: trusted firmware running in a trusted execution environment and ordinary firmware running in a normal execution environment; the trusted firmware runs before the ordinary firmware; the test task includes: a first subtask file corresponding to the trusted firmware and a second subtask file corresponding to the ordinary firmware;

[0199] The allocation unit 802 can be specifically used to allocate the first subtask file to the DUT set corresponding to the first subtask file for execution; and after receiving the execution completion signal of the first subtask file, allocate the second subtask file to the DUT set corresponding to the second subtask file for execution.

[0200] In the above optional example, the trusted firmware includes: BL1 firmware, BL2 firmware, BL31 firmware, and BL32 firmware; wherein, the execution order of each trusted firmware is: BL1 firmware, BL2 firmware, BL31 firmware, and BL32 firmware; the first subtask file includes: the first subtask file corresponding to the BL1 firmware, the first subtask file corresponding to the BL2 firmware, the first subtask file corresponding to the BL31 firmware, and the first subtask file corresponding to the BL32 firmware.

[0201] In the above optional example, the ordinary firmware includes: BL33 firmware and operating system OS firmware; the BL33 firmware runs before the OS firmware; the second subtask file includes: a second subtask file corresponding to the BL33 firmware and a second subtask file corresponding to the OS firmware.

[0202] In the above optional example, each of the DUTs in the set of DUTs to be tested that receives the first subtask file executes the first subtask file in the trusted execution environment; each of the DUTs in the set of DUTs to be tested that receives the second subtask file executes the second subtask file in the normal execution environment.

[0203] It should be understood that, for the sake of brevity, some of the content described in Embodiment 1 and Embodiment 2 will not be repeated in this embodiment.

[0204] Based on the same inventive concept, this application also provides a computer device, see [link to relevant documentation]. Figure 9 As shown, it includes a processor 901, a memory 902, and a communication module 903. Wherein:

[0205] The communication module 903 is used for communication connection with the DUT (Device Under Test).

[0206] The processor 901 is used to execute one or more programs stored in the memory 902 to implement the processing method or firmware testing method executed by the computer device in the above embodiments.

[0207] In this embodiment, the communication module 903 may be a module capable of remote data communication, such as GPRS (General Packet Radio Service), IP communication module, TCP communication module, etc.

[0208] Understandable. Figure 9 The structure shown is for illustrative purposes only; the computer device may also include more advanced features. Figure 9 The more or fewer components shown, or having the same Figure 9 The different configurations shown. For example, Figure 9 It may also have a communication bus to enable internal communication between the processor 901 and the memory 902.

[0209] It's understandable, such as Figure 10 As shown, the specific structure of the computer device described in the embodiments of this application can all be abstracted into... Figure 10 The structure shown. That is, in addition to the hardware level, such as... Figure 9 As shown, in addition to hardware structures such as processors, memory, and communication modules, there are also software-level components such as operating systems, BIOS (Basic Input / Output System), and application software.

[0210] The processor is the control center of the device, used to execute relevant programs to implement the solutions provided in the embodiments of this application. The memory can store the operating system and other application software. When implementing the solutions provided in the embodiments of this application through software or firmware, the code used to implement the technical solutions provided in the embodiments of this application is stored in the memory and executed by the processor. The memory can be integrated with the processor or integrated inside the processor, or it can be one or more memory units independent of the processor.

[0211] An operating system is system software that manages hardware and software resources; it is the kernel and foundation of a device. Operating systems handle fundamental tasks such as managing and configuring memory, prioritizing system resource allocation, controlling input and output devices, operating networks, and managing file systems. To facilitate user interaction, most operating systems provide a user interface for interaction with the system.

[0212] The role of the BIOS is to perform hardware initialization during the power-on boot phase and to provide runtime services for the operating system and programs. In addition to hardware initialization, the BIOS typically also has functions such as displaying processor temperature and adjusting temperature protection policies.

[0213] Application software, also known as application programs, is one of the main categories of computer software. It refers to software written for a specific user application purpose. For example, application software can be a program used to achieve purposes such as power control and temperature management.

[0214] In this embodiment of the application, in addition to computer equipment, the main components of the servers involved in the test system and the DUT under test are also the same as those in the previous embodiment. Figure 10 The consistency allows for the collaborative implementation of the solutions provided in the embodiments of this application.

[0215] This embodiment also provides a computer-readable storage medium, such as a floppy disk, optical disk, hard disk, flash memory, USB flash drive, SD (Secure Digital Memory Card), MMC (Multimedia Card), etc. This computer-readable storage medium stores one or more programs that implement the above steps. These one or more programs can be executed by one or more processors to implement the processing method or firmware testing method executed by the computer device in Embodiment 1 or Embodiment 2. Further details will not be elaborated here.

[0216] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The embodiments described above are merely illustrative.

[0217] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0218] In this article, "multiple" refers to two or more.

[0219] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method of testing firmware, characterized by, The application is applied to a test system, the test system comprises a computer device and a DUT to be tested which is in communication connection with the computer device; the DUT to be tested is provided with a system startup firmware; the method comprises: obtaining a test task corresponding to the system startup firmware through the computer device; wherein the test task comprises at least one subtask file, one subtask file corresponds to one subfirmware of the system startup firmware respectively; one subfirmware of the system startup firmware is used for realizing one stage of a system startup process respectively; the subtask file is a minimum unit of the test task which is issued to the DUT to be tested for execution; allocating each subtask file of the test task to different DUT sets to be tested through the computer device to obtain a test result; wherein each DUT set to be tested is composed of at least one DUT to be tested; the DUT set to be tested is multiple; wherein the test task corresponding to the system startup firmware is obtained by the following way: obtaining a target test task to be trimmed through the computer device; the target test task comprises at least one subtask file, each subtask file comprises attribute information of at least one test subitem; each subfirmware corresponds to at least one test subitem; obtaining a target test subitem which does not satisfy a preset condition based on historical test data of each test subitem of the target test task through the computer device; deleting the attribute information of the target test subitem through the computer device; dividing each test subitem according to a test completion time length of each test subitem, so that the maximum test total time length in the test total time lengths corresponding to each divided test subitem set is the smallest in all division manners, generating a new subtask file for each divided test subitem set; and in the case that there is an execution dependency relationship between each test subitem, generating an execution order between each subtask file according to the execution dependency relationship between the test subitems, obtaining the test task corresponding to the system startup firmware; or, in the case that there is no execution dependency relationship between each test subitem and the number of each required divided subtask file is greater than the number of the DUT to be tested, arranging each test subitem in descending order based on the error rate of each test subitem whose attribute information is not deleted, obtaining the test task corresponding to the system startup firmware; wherein each test subitem with the same error rate is arranged in ascending order according to the test completion time length.

2. The firmware test method of claim 1, wherein, the allocation of each subtask file of the test task to different DUT sets to be tested comprises: allocating the subtask file corresponding to a subfirmware which is run first to the DUT set to be tested corresponding to the subtask file according to the running order of each subfirmware of the system startup firmware, and sending the subtask file corresponding to the next subfirmware to the DUT set to be tested corresponding to the subtask file after receiving the execution completion signal returned by the DUT set to be tested.

3. The firmware test method of claim 2, wherein, The system startup firmware comprises: trusted firmware running in a trusted execution environment and normal firmware running in a normal execution environment; the trusted firmware runs prior to the normal firmware; The test task comprises: a first sub-task file corresponding to the trusted firmware, and a second sub-task file corresponding to the normal firmware; According to the running sequence of each sub-firmware of the system startup firmware, the sub-task file corresponding to a sub-firmware running first is allocated to a DUT set to be tested corresponding to the sub-task file, comprising: The first sub-task file is allocated to the DUT set to be tested corresponding to the first sub-task file for execution; After receiving an execution completion signal of the first sub-task file, the second sub-task file is allocated to the DUT set to be tested corresponding to the second sub-task file for execution.

4. The firmware test method of claim 3, wherein, The trusted firmware comprises: BL1 firmware, BL2 firmware, BL31 firmware and BL32 firmware; wherein the running sequence of each trusted firmware is in turn: BL1 firmware, BL2 firmware, BL31 firmware, BL32 firmware; the first sub-task file comprises: a first sub-task file corresponding to the BL1 firmware, a first sub-task file corresponding to the BL2 firmware, a first sub-task file corresponding to the BL31 firmware, and a first sub-task file corresponding to the BL32 firmware.

5. The firmware test method of claim 4, wherein, The normal firmware comprises: BL33 firmware and operating system (OS) firmware; the BL33 firmware runs prior to the OS firmware; The second sub-task file comprises: a second sub-task file corresponding to the BL33 firmware, and a second sub-task file corresponding to the OS firmware.

6. The firmware test method of any one of claims 3-5, wherein, Each DUT in the DUT set receiving the first sub-task file executes the first sub-task file in the trusted execution environment; Each DUT in the DUT set receiving the second sub-task file executes the second sub-task file in the normal execution environment.

7. A firmware testing apparatus, characterized by comprising: The device is applied to a computer device of a test system, the test system comprising the computer device and a DUT to be tested having a communication connection with the computer device; the DUT to be tested is equipped with system startup firmware; the device comprises: An acquisition unit is configured to acquire a test task corresponding to the system startup firmware; wherein the test task comprises at least one sub-task file, one sub-task file corresponding to one sub-firmware of the system startup firmware; one sub-firmware of the system startup firmware is used to implement one stage of a system startup process; the sub-task file is a minimum unit of a DUT to be tested for execution; An allocation unit is configured to allocate each sub-task file of the test task to different DUT sets to be tested for execution, so as to obtain a test result through execution results of DUTs constituting each DUT set to be tested; the DUT set to be tested is a plurality of DUT sets to be tested; The test task corresponding to the system startup firmware is obtained in the following manner: obtaining, by the computer device, a target test task to be pruned; the target test task comprises at least one subtask file, each of the subtask files comprises attribute information of at least one test subitem, and each of the subfirmwares corresponds to at least one test subitem; obtaining, by the computer device, a target test subitem that does not meet a preset condition based on historical test data of each test subitem of the target test task; deleting, by the computer device, the attribute information of the target test subitem; dividing, according to a test completion time length of each test subitem, each test subitem, so that a maximum test total time length in each divided test subitem set is minimum in all division manners, generating a new subtask file for each divided test subitem set, and generating an execution sequence between each subtask file according to an execution dependency relationship between the test subitems, to obtain a test task corresponding to the system startup firmware; or when there is no execution dependency relationship between each test subitem and the number of each required division subtask file is greater than the number of the DUT to be tested, arranging, in descending order, each test subitem based on a fault rate of each test subitem with un-deleted attribute information, to obtain the test task corresponding to the system startup firmware; wherein, each test subitem with the same fault rate is arranged in ascending order according to the test completion time length.

8. A test system, characterized by comprise: a computer device and a DUT to be tested in communication connection with the computer device; the DUT to be tested is provided with a system startup firmware; the computer device is configured to obtain a test task corresponding to the system startup firmware, and distribute each subtask file of the test task to different DUT sets to obtain a test result; wherein, the test task comprises at least one subtask file, one of the subtask files corresponds to one of the subfirmwares of the system startup firmware, one of the subfirmwares of the system startup firmware is configured to implement one stage of a system startup process; each of the DUT sets is composed of at least one DUT to be tested; and the subtask file is a minimum unit of execution of the DUT to be tested; the DUT to be tested is configured to execute the distributed subtask file and feed back an execution result of the subtask file to the computer device, so that the computer device determines the test result according to the execution result; and the DUT set is composed of a plurality of DUTs to be tested; wherein, the test task corresponding to the system startup firmware is obtained by: obtaining, by the computer device, a target test task to be pruned; the target test task comprises at least one subtask file, each of the subtask files comprises attribute information of at least one test subitem, and each of the subfirmwares corresponds to at least one test subitem; obtaining, by the computer device, a target test subitem that does not meet a preset condition based on historical test data of each test subitem of the target test task; delete attribute information of the target test sub-item by the computer device; according to a test completion time length of each test sub-item, divide each test sub-item, so that a maximum test total time length in test total time lengths corresponding to each divided test sub-item set is minimum in all division manners, generate a new sub-task file for each divided test sub-item set, and in a case where there is an execution dependency relationship between test sub-items, generate an execution sequence between the sub-task files according to the execution dependency relationship between the test sub-items, and obtain a test task corresponding to the system startup firmware; or in a case where there is no execution dependency relationship between test sub-items and a number of required divided sub-task files is greater than a number of DUTs to be tested, arrange each test sub-item in descending order based on a fault rate of each test sub-item whose attribute information is not deleted, and obtain a test task corresponding to the system startup firmware; and in a case where there is no execution dependency relationship between test sub-items and a number of required divided sub-task files is greater than a number of DUTs to be tested, arrange each test sub-item in descending order based on a fault rate of each test sub-item whose attribute information is not deleted, and obtain a test task corresponding to the system startup firmware; wherein each test sub-item with a same fault rate is arranged in ascending order according to a test completion time length.

9. A computer device, comprising: comprise: a communication module, a processor, and a memory; the communication module is configured to be in communication connection with a DUT to be tested; the processor is configured to execute one or more programs stored in the memory, so as to implement the method according to any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores one or more programs, which when executed by one or more processors, cause the processors to execute the method according to any one of claims 1-6.

Images