Embedded device dynamic test method and system

Abstract

The invention relates to a dynamic testing method and system for embedded equipment, belongs to the technical field of software testing, and solves the problems of mismatching of testing requirements and hardware resources and low concurrency efficiency in the prior art. The method comprises the steps of obtaining a CI / CD variable and a to-be-tested source code, and determining a test task type; determining a target test case set according to the test task type and the to-be-tested source code; according to the case type, dividing the target test case set into test case subsets, and determining a subset type; starting a corresponding container for the test case subset; generating a test task list according to all the test case subsets and the hardware resource state table by utilizing a hardware resource allocation service of the container; based on the test task list, the container starts a thread for the corresponding hardware equipment, the hardware equipment is controlled to execute a test, and a test result is output. According to the method and the device, the test cases and hardware resources with adaptive performance are automatically, efficiently and dynamically selected based on the source code change condition.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and in particular to a dynamic testing method and system for embedded devices. Background Technology

[0002] In the field of software testing, traditional testing methods typically employ static test equipment configuration and full test case execution, leading to wasted testing resources and low efficiency. With the widespread adoption of Continuous Integration / Continuous Deployment (CI / CD) processes, existing technologies propose dynamically adjusting test resource allocation schemes based on the physical connectivity and resource occupancy status of hardware devices. However, these technologies focus on hardware-level topology adaptation and resource scheduling, neglecting to consider code change characteristics. This results in a disconnect between equipment resource configuration and software testing requirements, potentially leading to situations where high-performance equipment is used for low-complexity testing or hardware-sensitive code changes are not matched with appropriate test equipment.

[0003] These limitations lead to significant resource waste and technical disconnect in embedded system testing. For example, in automated tests triggered by Git commits, even if only the underlying driver code is modified, a high-performance application processor device still needs to be scheduled to run the complete test suite, resulting in hardware resource mismatch and making it difficult to meet the needs for rapid feedback during the development process.

[0004] Therefore, there is an urgent need for a method in automated testing that can dynamically select test cases and configure hardware resources based on code modifications. Summary of the Invention

[0005] Based on the above analysis, the embodiments of the present invention aim to provide a dynamic testing method and system for embedded devices to solve the problem of mismatch between existing testing requirements and hardware resources.

[0006] On one hand, embodiments of the present invention provide a dynamic testing method for embedded devices, including: Obtain CI / CD variables and the source code to be tested, and determine the test task type; Based on the test task type and the source code to be tested, a target test case set is determined; wherein, the target test case set includes at least one test case; Based on the test case type, the target test case set is divided into test example sets, and the subset type of the test example sets is determined. Start the corresponding container for the test example set; wherein, the container includes a hardware resource allocation service; Using the hardware resource allocation service, a test task list is generated based on all the test example sets and the hardware resource status table; wherein, the test task list includes the test example sets and the device information of the corresponding hardware devices, and the hardware devices are embedded devices; Based on the test task list, the container starts a thread for the corresponding hardware device, controls the hardware device to execute tests, and outputs test results.

[0007] Furthermore, the test task types include full-scale test types and non-full-scale test types; Obtain CI / CD variables and the source code to be tested, and determine the test task type, including: When the CI / CD variable is a predefined variable, the test task type is determined to be the full test class; When the CI / CD variable is not a predefined variable, determine whether the source code to be tested involves core modifications; If core modifications are involved, then the test task type is determined to be the full test class; If no core modifications are involved, then the test task type is determined to be the non-full-scale test class.

[0008] Further, based on the test task type and the source code to be tested, a target test case set is determined, including: When the test task type is the full test class, a preset test case set is obtained, and the preset test case set is used as the target test case set; When the test task type is the non-full test class, determine the changed code type of the source code to be tested, and determine the target test case set based on the changed code type.

[0009] Further, the modified code type of the source code to be tested is determined, and based on the modified code type, a target test case set is determined, including: Obtain the change information corresponding to the source code to be tested, and determine the first changed code type based on the change information; Obtain the description information corresponding to the source code to be tested, and determine the second modified code type based on the description information; wherein, the description information includes a modification description and a modification file; Based on the first change code type and the second change code type, and using the change type-use case mapping table, determine the first set of test cases; Based on the first set of test cases, and considering the dependency relationships between test cases, a second set of test cases is determined. Based on the first test case set and the second test case set, construct the target test case set.

[0010] Further, based on the test case type, the target test case set is divided into a test example set, and the subset type of the test example set is determined, including: Obtain the test case type for each test case in the target test case set; Test cases with the same test case type are grouped into the same test case set, and the test case type is used as a subset type of the test case set.

[0011] Furthermore, the hardware resource status table includes an allocable resource table; wherein, the allocable resource table includes device information of the hardware devices in the allocable pool; Using the hardware resource allocation service, a test task list is generated based on all the test example sets and the hardware resource status table, including: Determine the estimated time required to perform the test using the test example set; Obtain the expected completion time corresponding to the source code to be tested; Based on the subset type, the corresponding device information is extracted from the allocable resource table. The device information includes device identifier, device status, and historical test records. Based on the estimated time, the expected completion time, and the device information, the hardware device used by the test example set is determined, and the test task list is constructed.

[0012] Furthermore, the hardware resource status table also includes a reserved resource table; wherein the reserved resource table includes device information of the hardware devices in the reserved pool; Based on the estimated time consumption, the expected completion time, and the device information, the hardware device used by the test example set is determined, and the test task list is constructed, which further includes: Determine whether the test task for the container is an important task; If it is an important task, determine whether the test task list meets the expected completion time requirement; If the conditions are not met, then based on the subset type, the corresponding device information is extracted from the reserved resource table, and the test task list is updated.

[0013] Furthermore, based on the test task list, the container starts a thread for the corresponding hardware device, controls the hardware device to execute tests, and outputs test results, including: The source code to be tested is compiled to generate the corresponding binary file; Determine whether the test task list has stopped generating; If generation has stopped, the container will start the thread for the corresponding hardware device according to the test task list and burn the binary file to the hardware device. Control the hardware device to execute tests according to the test task list; After the test is completed, the hardware device transmits the test results to the container through the thread.

[0014] Furthermore, the dynamic testing method for embedded devices also includes: When the hardware device fails to perform a test, identify the target test case that failed. Obtain the target source code corresponding to the target test case from the source code to be tested; Based on the source code call chain, obtain the associated source code corresponding to the target source code, and obtain the associated test cases corresponding to the associated source code; Based on the target test case and the associated test case, a list of associated test tasks is generated, the tests are executed, and the associated test results are output.

[0015] On the other hand, embodiments of the present invention provide an embedded device dynamic testing system, comprising: The task type determination module is used to obtain CI / CD variables and the source code to be tested, and determine the test task type. The test case determination module is used to determine a target test case set based on the test task type and the source code to be tested; wherein the target test case set includes at least one test case; The test case determination module is further configured to divide the target test case set into a test example set according to the test case type of the test case, and determine the subset type of the test example set; A container startup module is used to start a corresponding container for the test example set; wherein, the container includes a hardware resource allocation service; The task allocation module is used to generate a test task list based on all the test example sets and the hardware resource status table using the hardware resource allocation service; wherein, the test task list includes the test example sets and the device information of the corresponding hardware devices, and the hardware devices are embedded devices; The task execution module is used to start a thread for the corresponding hardware device based on the test task list, control the hardware device to perform tests, and output test results.

[0016] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects: 1. In automated testing, the test task type is determined based on CI / CD variables and the source code to be tested. The test case set is determined based on the test task type and the source code. Furthermore, a test task list is generated through a container, prompting the appropriate hardware device to execute the test. This establishes a full-chain, automated, and efficient configuration of source code-test case-device, improving the accurate matching and timely response of test requirements and test hardware resources.

[0017] 2. Determine the code change type based on the change information and description information. Then, dynamically select the target test case set based on the code change type and test case dependencies to ensure that the determined test cases are compatible with the source code changes, so as to fully cover the test requirements and avoid redundant test execution.

[0018] 3. Divide the test case set according to the test case type, and then select the corresponding hardware resources (i.e., embedded hardware devices) from the hardware resource status table to ensure that the performance of the hardware resources is adapted to the test cases, that is, to match the test requirements of source code changes, realize the dynamic allocation of hardware resources, and improve resource utilization.

[0019] 4. Based on the estimated time and expected completion time of testing with the test case set, determine the required number of devices, and based on the device status in the hardware resource status table, finally determine the hardware devices that can be used to execute the test case set, so as to realize the concurrent execution of the test case set on multiple hardware devices and improve testing efficiency.

[0020] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0021] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Figure 1 This is a flowchart illustrating a dynamic testing method for embedded devices according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the main modules of an embedded device dynamic testing system according to an embodiment of the present invention. Detailed Implementation

[0022] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0023] A specific embodiment of the present invention discloses a dynamic testing method for embedded devices, such as... Figure 1 As shown, it includes: Step S1: Obtain CI / CD variables and the source code to be tested, and determine the test task type.

[0024] Specifically, CI / CD variables are a set of parameters used to configure and optimize continuous integration / continuous delivery processes. They are mainly divided into two categories: predefined variables and user-defined variables, such as variables defined by maintenance or management personnel in Git code management tools. When automated tests are triggered by Git commits, the CI / CD variables of the test branch and the source code to be tested can be obtained to determine the test task type.

[0025] Furthermore, the test task categories include full test and non-full test. The process involves obtaining CI / CD variables and the source code to be tested, and determining the test task type, including steps S11-S12.

[0026] Step S11: When the CI / CD variable is a predefined variable, determine the test task type as the full test class.

[0027] Specifically, the test branch's CI / CD variables are checked to see if they are predefined variables. If so, the test task type is determined to be a full test, and a full test is executed. Here, predefined variables refer to one or more predefined CI / CD variables that match a full test. For example, a predefined variable named "enable" with a value of 1. When the obtained CI / CD variable is named "enable" and has a value of 1, the test task type is determined to be a full test. If the value is not 1, step S12 is executed.

[0028] Step S12: When the CI / CD variable is not the predefined variable, determine whether the source code to be tested involves core modifications.

[0029] If the CI / CD variable is not a predefined variable, the next step is to determine whether the source code under test involves core modifications. Specifically, based on the functionality involved, the source code under test can be divided into multiple source code blocks. Comment information is extracted from the source code, and for source code blocks with special markers, it is determined whether changes have occurred. If changes have occurred to source code blocks with special markers, the source code under test is considered to have involved core modifications. These special markers are added to the comment information during the compilation and commenting process of the source code under test (e.g., recorded as comments in each function header) to identify the corresponding code block as core code. For example, by comparing the code of the source code block with the code of the corresponding source code block in the current version, it can be determined whether the source code block has been changed.

[0030] Furthermore, by determining whether the source code to be tested involves core modifications, the type of test task is determined, specifically including: Step S121: If core modifications are involved, the test task type is determined to be the full-scale test class. Specifically, if the source code to be tested involves core modifications, considering that the core modifications will have indirect impacts on other functions, in order to ensure that all indirect impacts are fully tested, the test task type is determined to be the full-scale test class.

[0031] Step S122: If no core modifications are involved, the test task type is determined to be the non-full-scale test class. Specifically, if the source code to be tested does not involve core modifications, and the main focus is on testing the modified content in the source code, then the test task type is determined to be the non-full-scale test class.

[0032] Step S2: Determine the target test case set according to the test task type and the source code to be tested; wherein the target test case set includes at least one test case.

[0033] After determining the test task type, select the test cases required for this test based on the test task type and the source code to be tested, forming a target test case set, specifically including steps S21-S22.

[0034] Step S21: When the test task type is the full test class, obtain the preset test case set and use the preset test case set as the target test case set.

[0035] Specifically, when the test task type is a full-scale test class, a preset test case set is obtained as the target test case set. The preset test case set is a pre-defined set of test cases for full-scale testing, ensuring that the target test case set can be quickly selected when the test task type is a full-scale test class.

[0036] Step S22: When the test task type is the non-full test class, determine the changed code type of the source code to be tested, and determine the target test case set based on the changed code type.

[0037] When the test task is similar to a non-full-scale test class, it is necessary to further determine the type of changed code in the source code to be tested, and then determine the target test case set, which specifically includes: Step S221: Obtain the change information corresponding to the source code to be tested, and determine the first changed code type based on the change information.

[0038] Specifically, you can directly obtain the change information corresponding to the source code to be tested via Git push, or you can determine the change information of the source code to be tested by the code differences between the main branch and branches, or the code differences between two commits on the same branch, and determine the first change code type based on the change information. It should be noted that the first change code type may be one or more.

[0039] For example, the corresponding first change code type can be determined by the change information. For instance, if the code type of the login function is "identity authentication", then the first change code type can be determined to be "identity authentication".

[0040] When determining the first code change type based on code differences between the main branch and branches, the methods include: obtaining the main branch code from continuous integration (CI); obtaining the code difference file between the current branch and the main branch, for example, by using the `git diff` method; and using a script to analyze the difference file to determine the corresponding first code change type.

[0041] When determining the first code change type based on the code differences between two commits on the same branch, the methods include: obtaining the commit ID of the last commit of the current branch and the commit ID of the last commit before the push; obtaining the difference file between the last commit after the push and the last commit before the push, for example, by using the git diff method; and using a script to analyze the difference file to determine the corresponding first code change type.

[0042] Furthermore, a script is used to analyze the difference file to determine the corresponding first change code type. An example is shown below, where the difference file content is: --- a / lib / login.c +++ b / lib / login.c @@ -192,6 +199,6 @@ void func1(struct ul_fileeq *eq) #ifdef USE_FILEEQ_CRYPTOAPI deinit_crypto_api(eq); #endif -free(a); +wait_free (a); } The script analysis then identifies the login.c file to which the modified code belongs, and the function to which it belongs is func1. Based on the code type of the func1 function, the corresponding first modified code type is determined.

[0043] Step S222: Obtain the description information corresponding to the source code to be tested, and determine the second modified code type based on the description information; wherein, the description information includes a modification description and a modification file.

[0044] Specifically, the change description and change files are retrieved from the COMMIT statement corresponding to the source code under test to determine the second change code type. The change description describes the functionality of the changes made to the submitted source code, the specific functions modified, the reason for the modification, and the modules affected after the fix. For example, function A was modified, and function B was added. The change files describe the changes to the specific files involved in the submitted source code. For example, file C was deleted, and file D was modified. For instance, if the change description shows a modification to the login password verification function, and the description includes a corresponding change file description, then the change code type is determined to be a login fix type. It should be noted that the second change code type may be the same as or different from the first change code type, and there may be one or more such types.

[0045] Step S223: Based on the first change code type and the second change code type, determine the first test case set according to the change type-test case mapping table.

[0046] Specifically, based on the first change code type and the change type-test case mapping table, multiple corresponding test cases are determined from the test case resource library. Then, based on the second change code type and the change type-test case mapping table, multiple corresponding test cases are determined. After deduplication of the two sets of test cases, a first test case set is determined. Alternatively, the first and second change code types can be deduplicated and merged first, and then multiple test cases can be determined based on the change type-test case mapping table to form the first test case set. The change type-test case mapping table is a pre-established mapping table between change code types and test cases; one change code type may correspond to multiple test cases, and the test case resource library stores all test cases.

[0047] Step S224: Based on the first test case set and the test case dependencies, determine the second test case set.

[0048] Specifically, the dependency relationships of each test case in the first test case set are obtained. Based on these dependencies, the corresponding dependent test cases are retrieved from the test case resource library. The dependency relationships identify the dependencies between different test cases. For example, if performance test case A requires the execution of hardware function test case B, then test case B is a dependent test case of test case A. It is then determined whether the dependent test cases of each test case already exist in the first test case set. If not, the dependent test cases are incorporated into the second test case set. In other embodiments, this determination can be skipped initially, and all dependent test cases can be directly incorporated into the second test case set, with the determination performed during the construction of the target test case set.

[0049] Step S225: Construct a target test case set based on the first test case set and the second test case set.

[0050] Specifically, the target test case set is formed by merging and deduplicating the test cases in the first test case set and the test cases in the second test case set.

[0051] Based on the change information and description information, the code change type is determined. Then, the target test case set is dynamically selected based on the code change type and test case dependencies to ensure that the determined test cases are compatible with the source code changes. This ensures that subsequent tests fully cover the test requirements while avoiding redundant test execution.

[0052] Step S3: Based on the test case type, divide the target test case set into test example sets and determine the subset type of the test example sets.

[0053] First, obtain the test case type for each test case in the target test case set. In this embodiment, the test case types include driver type, security type, and system type. Each test case type includes multiple test case subtypes. For example, driver type includes UART class and I2C class, and security type includes RMAC class and SMBC class.

[0054] Secondly, test cases with the same test case type are divided into the same test case set, and the test case type is used as a subset type of the test case set.

[0055] Specifically, based on the test case type of each test case, test cases of the same type are grouped together, forming a test case set, and the corresponding test case type is determined as a subset type of this test case set. For example, if the first code change type is a driver type, then the target test case set contains driver type test cases. All driver type test cases in the target test case set are grouped into a test case set, and the subset type of this subset is the driver type.

[0056] Test cases are subsetted based on test case type to ensure that different types of test cases are executed in different containers. Test cases of the same type are synchronously allocated hardware resources for testing, which facilitates the matching and management of test cases and hardware resources, accurately matches test case performance requirements with hardware resource performance, and achieves efficient utilization of hardware resources.

[0057] Step S4: Start the corresponding container for the test example set; wherein the container includes a hardware resource allocation service.

[0058] Specifically, a container is started for each test case set. In this embodiment, the open-source containerization platform (Docker) is used to start a corresponding container for each test case set, and the hardware resource allocation and scheduling is performed by a hardware resource allocation service in the form of a daemon process in Docker.

[0059] Step S5: Using the hardware resource allocation service, generate a test task list based on all the test example sets and the hardware resource status table; wherein, the test task list includes the test example sets and the device information of the corresponding hardware devices, and the hardware devices are embedded devices.

[0060] Based on the test requirements of the test case set and the performance information and usage status of each hardware device in the hardware resource status table, the hardware resource allocation service allocates executable hardware devices to each test case set, thereby generating a test task list, including the test case set and the corresponding hardware device information. In this embodiment, embedded devices are used as hardware devices.

[0061] Furthermore, the hardware resource status table includes an allocable resource table, which includes device information of the hardware devices in the allocable pool; using the hardware resource allocation service, a test task list is generated based on all the test example sets and the hardware resource status table, including steps S51-S54.

[0062] Step S51: Determine the estimated time for testing with the test example set.

[0063] First, by analyzing the historical testing time of each test case in the test case set, the estimated test case time for testing with that test case is determined. For example, the average time of each test case in multiple historical tests can be used as the estimated test case time for that test case. Second, based on the estimated test case time for each test case, the estimated test time for testing with the test case set is determined. For example, the total estimated test case time for each test case can be used as the estimated test time for the test case set.

[0064] Step S52: Obtain the expected completion time corresponding to the source code to be tested.

[0065] Obtain the expected completion time corresponding to the source code to be tested, i.e., the expected completion time of this test. This can be set when starting the container, or the default single test duration can be used; there is no restriction here. Determine the expected time for each test case set based on the expected completion time. This can be done by using the expected completion time as the expected time for each test case set, i.e., testing all test case sets simultaneously. Alternatively, the expected completion time can be evenly divided according to the expected completion time and the number of test case sets, and the expected time can be allocated proportionally based on the estimated time. Each test case set can be tested sequentially; there is no restriction here. In this embodiment, only the expected completion time is used as the expected time for illustration.

[0066] Step S53: Based on the subset type, extract the corresponding device information from the allocable resource table. The device information includes device identifier, device status, and historical test records.

[0067] Based on the subset type, device information that supports the test case set is selected from the allocable resource table, including device identifier, device status, and historical test records. Specifically, based on the subset-device mapping table and the subset type, device information of the corresponding device type can be selected from the allocable resource table. For example, if the subset type is driver type, device information of hardware devices whose device type includes driver type can be selected from the allocable resource table. In this embodiment, the device type can be obtained by reading the system's built-in configuration information of the hardware device. An embedded device may include multiple device type tags, that is, it can support test cases of different test case types. For example, the device type can be determined based on the hardware device's capabilities. If it can support testing all driver code, the device type is driver type; if it can support the I2C interface, the device type is I2C type.

[0068] Step S54: Based on the estimated time consumption, the expected completion time, and the device information, determine the hardware device used by the test example set, and construct the test task list.

[0069] Specifically, this includes steps S541-S544.

[0070] Step S541: Based on the estimated time and expected completion time, calculate the number of devices required for the test case set. Specifically, the required number of devices N... need =G / T, where N need G represents the number of required devices, G represents the estimated time, and T represents the expected completion time.

[0071] Step S542: Determine the hardware device ranking based on historical test records. Specifically, based on the historical test records of each hardware device in the device information, the ranking of the hardware devices is determined by weighted summation according to the number of test executions and the test execution duration. This prioritizes the hardware devices with the fewest usages and durations, ensuring balanced load across all hardware devices, guaranteeing the stability of test verification speed, and forming a closed-loop cycle of "execution-recording-optimization".

[0072] Step S543: Determine the hardware devices corresponding to the test case set based on the device status. Specifically, based on the device status in the device information, determine the number of available devices (N0). When the required number of devices is not greater than the number of available devices, select the corresponding number of hardware devices according to the hardware device order and the required number of devices as the hardware devices used for the test case set, i.e., the test case execution hardware devices for the test case set. When the required number of devices is greater than the number of available devices, use all available hardware devices as the hardware devices used for the test case set, i.e., the test case execution hardware devices. It can be understood that one test case set may correspond to one test case execution hardware device, or it may correspond to multiple test case execution hardware devices, with multiple test case execution hardware devices performing concurrent testing.

[0073] Step S544: Based on the test case set and the test case execution hardware devices, establish a mapping relationship between test cases and test case execution hardware devices to form a test task list. Specifically, based on the test case set and the device information of the test case execution hardware devices, i.e., the device information of the hardware devices corresponding to the test case set, establish a mapping relationship between each test case and the test case execution hardware devices to form a test task list. It can be understood that one test case execution hardware device may correspond to one test case in the test case set, or it may correspond to multiple test cases. When the required number of devices is not greater than the number of available devices, the test cases in the test case set are allocated according to the principle of average distribution to establish a mapping relationship between test cases and test case execution hardware devices. When the required number of devices is greater than the number of available devices, the test case set is divided into allocable test cases and unallocated test cases according to the number of test case execution hardware devices. Allocable test cases are allocated according to the principle of average distribution to establish a mapping relationship with test case execution hardware devices, and the mapping of unallocated test cases is set to empty.

[0074] Furthermore, for cases with pending assignments, the device information in the allocable resource table is monitored. When the device status of the corresponding device type is detected as available, a mapping relationship between the pending assignments and the hardware device is established, and the test task list is updated.

[0075] Furthermore, the hardware resource status table also includes a reserved resource table, which contains device information of the hardware devices in the reserved pool. For example, 80% of all hardware devices are designated as allocable devices, forming an allocable pool. Based on the device information of the hardware devices in the allocable pool, a corresponding allocable resource table is constructed. The remaining 20% ​​are designated as reserved devices, forming a reserved pool, and a corresponding reserved resource table is constructed. Simultaneously, it is ensured that each pool contains hardware devices of different device types. Step S54 also includes steps S545-S547.

[0076] Step S545: Determine whether the test task of the container is an important task.

[0077] Specifically, when starting a container, the system determines whether a test task is critical based on the source of the source code corresponding to the container's test task. For example, if the test case corresponds to the source code from the mainline source code, the test task of the started container is considered critical.

[0078] Step S546: If it is an important task, determine whether the test task list meets the expected completion time requirement.

[0079] If it is an important task, then it is further determined whether the test task list meets the expected completion time requirement. Specifically, this can be determined by whether there are any test cases to be assigned. If there are no test cases to be assigned, the requirement is met; if there are test cases to be assigned, the requirement is not met.

[0080] Step S547: If the condition is not met, then based on the subset type, extract the corresponding device information from the reserved resource table and update the test task list.

[0081] If the conditions are not met, then based on the subset type, the corresponding device information is extracted from the reserved pool resource table, the mapping relationship between the test cases to be assigned and the corresponding hardware devices is determined, and the test task list is updated. The method principle is the same as steps S53 and S541-S544, and will not be repeated here.

[0082] To prevent the hardware devices corresponding to the reserved pool resource table from being idle for a long time, an idle time judgment is established for the hardware devices in the reserved pool. When the idle time exceeds the idle time threshold and there are test cases to be assigned, the idle hardware devices are moved to the allocatable pool to participate in the hardware device matching and mapping of all test tasks, thereby further realizing device load balancing and improving test efficiency.

[0083] The test case set is divided according to the test case type, and then the corresponding hardware resources (i.e. embedded hardware devices) are selected from the hardware resource status table. A full-chain, automated, and efficient configuration of source code-test case-device is established to ensure that the performance of hardware resources is adapted to the test cases, that is, to match the test requirements of source code changes, realize the dynamic allocation of hardware resources, and improve resource utilization.

[0084] Step S6: Based on the test task list, the container starts a thread for the corresponding hardware device, controls the hardware device to execute the test, and outputs the test results.

[0085] After the test task list is built, the test is started. The container starts a thread for the corresponding hardware device, and the hardware device executes the test and outputs the test results, specifically including steps S61-S65.

[0086] Step S61: Compile the source code to be tested to generate the corresponding binary file.

[0087] Specifically, the source code to be tested is compiled to generate a binary file that the hardware device can understand. The compilation tool can be selected according to the programming language of the source code to be tested, and existing tools such as GCC and Javac can be used. There are no restrictions here.

[0088] Step S62: Determine whether the test task list has stopped generating.

[0089] Specifically, the status of the mapping relationship between test cases and hardware devices in the hardware resource allocation service can be used to determine whether the test task list has stopped generating. For example, if all test cases in the test case set corresponding to the hardware resource allocation service have established mapping relationships with hardware devices, or if there are no hardware devices in the test case set that are in a "available" state and have not established mapping relationships, then it is determined that the test task list has stopped generating.

[0090] Step S63: If generation has stopped, the container starts the thread for the corresponding hardware device according to the test task list and burns the binary file to the hardware device.

[0091] If the test task list has stopped generating, meaning the test task list is complete, then the test will immediately start based on the automated testing framework. The container, according to the test task list, starts a thread for each corresponding hardware device (test case execution hardware device) to interact with the corresponding hardware device and burn the binary file corresponding to the source code to be tested onto the hardware device.

[0092] Step S64: Control the hardware device to perform tests according to the test task list.

[0093] The embedded device executes the test actions in the test cases corresponding to the burned binary files in the test task list.

[0094] Step S65: After the test is completed, the hardware device transmits the test results to the container through the thread.

[0095] After the test is completed, the hardware device returns the test results to the container via a thread, and the container integrates the test results from different hardware devices.

[0096] Furthermore, test failures may occur during the testing process. For failed tests, an automatic test case dependency optimization mechanism is activated to obtain and test related test cases, specifically including: Step S66: When the hardware device fails to execute a test, identify the target test case that caused the failure. Specifically, when the hardware device fails to execute a test, identify the corresponding target test case according to the mapping relationship.

[0097] Step S67: Obtain the target source code corresponding to the target test case from the source code to be tested. Specifically, according to the path determined by the test case, obtain the target source code corresponding to the target test case from the source code to be tested.

[0098] Step S68: Based on the source code call chain, obtain the associated source code corresponding to the target source code, and obtain the associated test cases corresponding to the associated source code. Specifically, based on the target source code, obtain the corresponding associated source code based on the source code call chain, wherein the source code call chain can be determined based on parameters or data interfaces in the target source code. Refer to steps S1-S2 to determine the associated test cases corresponding to the associated source code.

[0099] Step S69: Based on the target test case and the associated test case, generate a list of associated test tasks, execute the tests, and output the associated test results. Specifically, generating a list of associated test tasks based on the target test case and the associated test cases, executing the tests, and outputting the associated test results can be performed in accordance with steps S3-S6, and will not be repeated here.

[0100] Furthermore, based on the results of the associated tests, the test case dependencies are updated. Specifically, after the associated test task list is executed, if the associated test case causes the test to fail, the associated test case is added as a dependent test case of the target test case, and the test case dependencies are updated; if the associated test case does not cause the test to fail, the current test case dependencies are maintained.

[0101] Furthermore, based on the test results and related test results, the hardware resource status table can be updated, and the number of times the hardware device was executed, the execution time, and the execution result can be updated in the device information of the corresponding hardware device in the hardware resource status table.

[0102] This invention provides a dynamic testing system for embedded devices, such as... Figure 2 As shown, it includes: The task type determination module is used to obtain CI / CD variables and the source code to be tested, and determine the test task type. The test case determination module is used to determine a target test case set based on the test task type and the source code to be tested; wherein the target test case set includes at least test cases; The test case determination module is further configured to divide the target test case set into a test example set according to the test case type of the test case, and determine the subset type of the test example set; A container startup module is used to start a corresponding container for the test example set; wherein, the container includes a hardware resource allocation service; The task allocation module is used to generate a test task list based on all the test example sets and the hardware resource status table using the hardware resource allocation service; wherein, the test task list includes the test example sets and the device information of the corresponding hardware devices, and the hardware devices are embedded devices; The task execution module is used to start threads for the corresponding hardware devices based on the test task list, control the hardware devices to execute tests, and output test results. The above method and system embodiments are based on the same principles, and their related aspects can be referenced each other to achieve the same technical effects. For specific implementation processes, please refer to the foregoing embodiments, which will not be repeated here.

[0103] In summary, the embedded device dynamic testing method and system of this invention has at least one of the following beneficial effects: 1. In automated testing, the test task type is determined based on CI / CD variables and the source code to be tested. The test case set is determined based on the test task type and the source code. Furthermore, a test task list is generated through a container, prompting the appropriate hardware device to execute the test. This establishes a full-chain, automated, and efficient configuration of source code-test case-device, improving the accurate matching and timely response of test requirements and test hardware resources.

[0104] 2. Determine the code change type based on the change information and description information. Then, dynamically select the target test case set based on the code change type and test case dependencies to ensure that the determined test cases are compatible with the source code changes, so as to fully cover the test requirements and avoid redundant test execution.

[0105] 3. Divide the test case set according to the test case type, and then select the corresponding hardware resources (i.e., embedded hardware devices) from the hardware resource status table to ensure that the performance of the hardware resources is adapted to the test cases, that is, to match the test requirements of source code changes, realize the dynamic allocation of hardware resources, and improve resource utilization.

[0106] 4. Based on the estimated time and expected completion time of testing with the test case set, determine the required number of devices, and based on the device status in the hardware resource status table, finally determine the hardware devices that can be used to execute the test case set, so as to realize the concurrent execution of the test case set on multiple hardware devices and improve testing efficiency.

[0107] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0108] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A dynamic testing method for embedded devices, characterized in that, include: Obtain CI / CD variables and the source code to be tested, and determine the test task type; Based on the test task type and the source code to be tested, a target test case set is determined; wherein, the target test case set includes at least one test case; Based on the test case type, the target test case set is divided into test example sets, and the subset type of the test example sets is determined. Start the corresponding container for the test example set; wherein, the container includes a hardware resource allocation service; Using the hardware resource allocation service, a test task list is generated based on all the test example sets and the hardware resource status table; wherein, the test task list includes the test example sets and the device information of the corresponding hardware devices, and the hardware devices are embedded devices; Based on the test task list, the container starts a thread for the corresponding hardware device, controls the hardware device to execute tests, and outputs test results.

2. The method according to claim 1, characterized in that, The test task types include full-scale test type and non-full-scale test type; Obtain CI / CD variables and the source code to be tested, and determine the test task type, including: When the CI / CD variable is a predefined variable, the test task type is determined to be the full test class; When the CI / CD variable is not a predefined variable, determine whether the source code to be tested involves core modifications; If core modifications are involved, then the test task type is determined to be the full test class; If no core modifications are involved, then the test task type is determined to be the non-full-scale test class.

3. The method according to claim 2, characterized in that, Based on the test task type and the source code to be tested, a target test case set is determined, including: When the test task type is the full test class, a preset test case set is obtained, and the preset test case set is used as the target test case set; When the test task type is the non-full test class, determine the changed code type of the source code to be tested, and determine the target test case set based on the changed code type.

4. The method according to claim 3, characterized in that, Determine the modified code type of the source code to be tested, and based on the modified code type, determine the target test case set, including: Obtain the change information corresponding to the source code to be tested, and determine the first changed code type based on the change information; Obtain the description information corresponding to the source code to be tested, and determine the second modified code type based on the description information; wherein, the description information includes a modification description and a modification file; Based on the first change code type and the second change code type, and using the change type-use case mapping table, determine the first set of test cases; Based on the first set of test cases, and considering the dependency relationships between test cases, a second set of test cases is determined. Based on the first test case set and the second test case set, construct the target test case set.

5. The method according to claim 1, characterized in that, Based on the test case types, the target test case set is divided into test example sets, and the subset types of the test example sets are determined, including: Obtain the test case type for each test case in the target test case set; Test cases with the same test case type are grouped into the same test case set, and the test case type is used as a subset type of the test case set.

6. The method according to claim 1, characterized in that, The hardware resource status table includes an allocable resource table; wherein, the allocable resource table includes device information of the hardware devices in the allocable pool; Using the hardware resource allocation service, a test task list is generated based on all the test example sets and the hardware resource status table, including: Determine the estimated time required to perform the test using the test example set; Obtain the expected completion time corresponding to the source code to be tested; Based on the subset type, the corresponding device information is extracted from the allocable resource table. The device information includes device identifier, device status, and historical test records. Based on the estimated time, the expected completion time, and the device information, the hardware device used by the test example set is determined, and the test task list is constructed.

7. The method according to claim 6, characterized in that, The hardware resource status table also includes a reserved resource table; wherein, the reserved resource table includes device information of the hardware devices in the reserved pool; Based on the estimated time consumption, the expected completion time, and the device information, the hardware device used by the test example set is determined, and the test task list is constructed, which further includes: Determine whether the test task for the container is an important task; If it is an important task, determine whether the test task list meets the expected completion time requirement; If the conditions are not met, then based on the subset type, the corresponding device information is extracted from the reserved resource table, and the test task list is updated.

8. The method according to claim 1, characterized in that, Based on the test task list, the container starts a thread for the corresponding hardware device, controls the hardware device to execute tests, and outputs test results, including: The source code to be tested is compiled to generate the corresponding binary file; Determine whether the test task list has stopped generating; If generation has stopped, the container will start the thread for the corresponding hardware device according to the test task list and burn the binary file to the hardware device. Control the hardware device to execute tests according to the test task list; After the test is completed, the hardware device transmits the test results to the container through the thread.

9. The method according to claim 1, characterized in that, Also includes: When the hardware device fails to perform a test, identify the target test case that failed. Obtain the target source code corresponding to the target test case from the source code to be tested; Based on the source code call chain, obtain the associated source code corresponding to the target source code, and obtain the associated test cases corresponding to the associated source code; Based on the target test case and the associated test case, a list of associated test tasks is generated, the tests are executed, and the associated test results are output.

10. A dynamic testing system for embedded devices, characterized in that, include: The task type determination module is used to obtain CI / CD variables and the source code to be tested, and determine the test task type. The test case determination module is used to determine a target test case set based on the test task type and the source code to be tested; wherein the target test case set includes at least test cases; The test case determination module is further configured to divide the target test case set into test example sets according to the test case type of the test cases, and determine the subset type of the test example sets; A container startup module is used to start a corresponding container for the test example set; wherein, the container includes a hardware resource allocation service; The task allocation module is used to generate a test task list based on all the test example sets and the hardware resource status table using the hardware resource allocation service; wherein, the test task list includes the test example sets and the device information of the corresponding hardware devices, and the hardware devices are embedded devices; The task execution module is used to start a thread for the corresponding hardware device based on the test task list, control the hardware device to perform tests, and output test results.

Images