A test method, device, electronic device and storage medium

Abstract

The application discloses a test method and device, electronic equipment and storage medium, relates to the firmware updating technical field, analyzes the changed code module of the firmware to be tested, traces the associated influence code module and influence factor, objectively quantitatively determines the range of the change spread, and replaces the artificial subjective judgment; a test case set is generated according to the influence factor and a preset rule, the test is focused on the affected area, and the pertinence and accuracy of the test are improved; the test case set is executed in the preconfigured test environment, and the test environment deployment can be automated. The application reduces the manual intervention demand of the tester, does not need to spend a lot of energy to manually screen test cases, solves the technical problems of low accuracy and lack of automation caused by the dependence on manual selection of test cases, and achieves the technical effects of more efficient and more accurate test process.

Description

Technical Field

[0001] The present application relates to the technical field of firmware updating, and in particular to a testing method, device, electronic device, and storage medium. Background Art

[0002] With the continuous development of big data and storage technologies, the demand for real-time data is becoming higher. The application of SSDs (Solid State Drives) is becoming increasingly widespread, and SSD technology is also rapidly developing. As a result, the complexity and functionality of SSD firmware are increasing. To ensure the functionality, performance, reliability, and compatibility of SSDs, extensive testing is required during the firmware development process.

[0003] The testing methods in related technologies only involve the compilation of firmware programs and the management of firmware codes. The selection of test cases for the firmware to be tested during the testing process mainly relies on manual selection, and it is impossible to automatically and accurately select the corresponding test cases. The test accuracy is low. In addition, the deployment of the test environment and the analysis of the test results are not automated, which is not friendly to testers. Summary of the Invention

[0004] The present application provides a testing method, device, electronic device and storage medium to at least solve the problems of low accuracy and lack of automation caused by manual selection of test cases in related technologies.

[0005] The present application provides a testing method, comprising: obtaining firmware to be tested, the firmware to be tested including at least one modified code module; obtaining an impact code module corresponding to the modified code module, and an impact factor corresponding to at least one of the impact code modules; generating a set of test cases for the firmware to be tested based on the impact factors and preset rules; and executing the set of test cases in a preconfigured test environment to test the firmware to be tested.

[0006] The present application also provides a testing device, comprising: a first acquisition module, used to acquire firmware to be tested, the firmware to be tested including at least one modified code module; a second acquisition module, used to acquire an impact code module corresponding to the modified code module, and an impact factor corresponding to at least one of the impact code modules; a use case generation module, used to generate a set of test cases for the firmware to be tested based on the impact factors and preset rules; and a testing module, used to execute the set of test cases in a preconfigured test environment to test the firmware to be tested.

[0007] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any one of the above-mentioned testing methods when executing the computer program.

[0008] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned testing methods are implemented.

[0009] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned testing methods when executed by a processor.

[0010] Through this application, the modified code modules of the firmware to be tested are analyzed, the associated affected code modules and influencing factors are tracked, and the scope of the changes is determined objectively and quantitatively, replacing manual subjective judgment; a test case set is generated based on the influencing factors and preset rules to ensure that the test focuses on the affected area, improving the pertinence and accuracy of the test; and the test case set is executed in a preconfigured test environment, showing that the test environment deployment can be automated. This application reduces the need for manual intervention by testers and eliminates the need to spend a lot of effort on manually screening test cases. It solves the technical problems of low accuracy and lack of automation caused by relying on manual selection of test cases, achieving a more efficient and accurate testing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] Figure 1 A flow chart of a testing method provided in an embodiment of the present application;

[0013] Figure 2 A flowchart of a developer submitting firmware to be tested provided in an embodiment of the present application;

[0014] Figure 3 A flowchart for determining the final impact factor provided in an embodiment of the present application;

[0015] Figure 4 A schematic diagram of a testing device provided in an embodiment of the present application;

[0016] Figure 5 A schematic diagram of an electronic device provided in an embodiment of the present application;

[0017] Figure 6 A schematic diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0019] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0020] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0021] like Figure 1 , this application provides a testing method, comprising:

[0022] S11: Obtain a firmware to be tested, where the firmware to be tested includes at least one modified code module.

[0023] Specifically, the firmware to be tested is accurately identified and located to determine the starting point of the test scope. When developers update or modify SSD firmware, it usually involves changes to one or more code modules. These changes may be to fix vulnerabilities, add new features, or optimize performance.

[0024] This step can be accomplished by using the version control system's differential analysis function to automatically compare the code differences between the currently submitted version and the baseline version (such as the version that last passed testing), accurately identifying the set of modified code modules. Developers can also directly mark the modified code modules when uploading the firmware to be tested.

[0025] This code change tracking-based mechanism ensures that testing activities can focus on the parts that have actually changed, avoiding redundant testing of unchanged code, thereby significantly improving testing efficiency.

[0026] S12: Obtain an impact code module corresponding to the modified code module and an impact factor corresponding to at least one impact code module.

[0027] Specifically, a precise mapping relationship from the change point to the affected scope is constructed, and the ripple effect of the changed code module on the entire firmware system is identified by combining static code analysis with dynamic dependency modeling.

[0028] First, you can use code dependency analysis tools to scan the call chain, data flow, and interface references of the modified module, identify the directly affected downstream modules (such as called functions and dependent library files), and recursively analyze the indirect impact of these downstream modules on other modules to form a complete set of affected code modules. Alternatively, developers can upload the affected code modules of the modified code module (specifically, they can upload the affected code module list) and their corresponding impact factors (such as Figure 2 ), the developer submits the firmware to be tested, updates the list of code modules and influencing factors of the changed code module design of the firmware to be tested, and when uploading to the system, the development manager needs to check whether the firmware to be tested is complete and accurate, and whether the list of affected code modules meets the requirements. If it meets the requirements, the development manager needs to further check whether the submitted materials are complete. If not, the developer enters the step of submitting the firmware to be tested. If it is complete, it is determined that the firmware to be tested has been successfully uploaded to the system and enters the next process. If it is incomplete, the development manager re-enters the step of checking whether the firmware to be tested is complete and accurate, and whether the list of affected code modules meets the requirements.

[0029] At the same time, in order to quantify the extent to which different modules are affected, the impact factor of each code module can be calculated based on factors such as the type and complexity of the code change (such as the number of modified lines and the number of code paths involved) and the dependency strength between modules (such as call frequency and data coupling).

[0030] This multi-dimensional analysis mechanism based on code logic and runtime behavior ensures that the identification of the impact scope is both comprehensive and accurate, providing a reliable quantitative basis for the intelligent generation of subsequent test cases, and avoiding the problems of incomplete test coverage or redundant testing caused by insufficient manual experience in traditional methods.

[0031] S13: Generate a test case set for the firmware to be tested based on the influencing factors and preset rules.

[0032] Specifically, a test case generation mechanism based on data-driven and rule engine is constructed to achieve automatic and accurate screening and combination of test cases by mapping and matching influencing factors with preset test strategies.

[0033] First, establish a correspondence between the impact factor and the test coverage dimension. For example, set the coverage depth of the test case according to the impact factor (for example, high-impact factor modules need to cover full-path testing, and low-impact factor modules can focus on core functional testing). At the same time, combine the test types defined in the preset rules (such as functional testing, performance testing, compatibility testing), input boundary conditions, abnormal scenario triggering logic, etc., to generate corresponding candidate test cases for each influencing code module.

[0034] The rule engine prioritizes, de-duplicates, and merges candidate test cases, ultimately generating a test case set that covers the modified code module and its impact area. This mechanism, which translates quantified impact into specific test requirements, avoids the randomness and bias of manual case selection while dynamically adjusting test coverage based on the actual impact of the code change. This ensures that the generated test case set comprehensively covers potential risk points while precisely focusing on high-impact areas, fundamentally improving the relevance and effectiveness of testing.

[0035] In one embodiment, a list of firmware code modules and a list of test cases are maintained. The firmware code module divides the firmware code into multiple modules based on function, such as a command processing module, a data transmission module, an error handling module, a power management module, a garbage collection module, a wear leveling module, and a bad block management module. The test case module designs test cases based on firmware functions, such as command test cases, data transmission test cases, error injection test cases, power management test cases, performance test cases, and reliability test cases.

[0036] Map the modules above according to their functions. Map test cases to firmware code modules according to their functions. For example, command processing test cases correspond to command processing modules, data transmission test cases correspond to data transmission modules, error injection test cases correspond to error handling modules, and so on.

[0037] The following formula is used to explain it in detail:

[0038] Firmware code module list: FirmwareModules={FM1, FM2, ..., FMn}; each firmware code module FMi (representing the i-th firmware code module) contains the following properties: the name of the firmware code module (such as command processing, data transmission, etc.), the functional description of the firmware code module, and the code path of the firmware code module.

[0039] Test case module list: TestCases = {TC1, TC2, ..., TCm}; each test case module TCj (representing the j-th test case module) contains the following attributes: test case name, test case description, test case classification, such as basic test case A, abnormal test case B, extended test case C, etc.

[0040] The basic mapping table is as follows:

[0041] MappingTable={(FM1, [TC1; TC2, TC3]), (FM2, [TC4; TC5]), ...}; Each firmware code module FMi corresponds to a set of test cases [TCj; ...].

[0042] According to the influencing factors, a mapping relationship table is generated, as follows: MappingTable1={(FM1, [TC1, X1; TC2, X2; TC3, X3]), (FM2, [TC4, X4; TC5, X5]), ...}; where Xi represents the influencing factor of TCi, and each firmware code module FMi corresponds to a set of test cases [TCi, Xi; ...].

[0043] S14: Execute the test case set in a preconfigured test environment to test the firmware to be tested.

[0044] Specifically, a standardized and automated test execution system is built, enabling full-chain automated control of the test process through a preconfigured test environment. This step leverages the test environment configuration module's fixed and dynamic adjustment mechanism to directly reuse pre-configured hardware resources (such as test machines and SSD devices), software dependencies, and network parameters when the test environment is stable, eliminating repeated manual deployment. When the environment changes (such as hardware replacements or adjustments to incorrectly injected test machines), the module allows for rapid reconfiguration, ensuring that the environment status matches test requirements in real time. SSD firmware is then automatically refreshed and logged, providing consistent and traceable initial conditions for test execution. Next, test case collections are automatically loaded based on the preconfigured environment, supporting parallel execution to improve efficiency. Furthermore, features such as real-time test progress monitoring, device status visualization, one-click debug log export, and interrupt recovery support ensure transparency and control of the test process. Finally, test execution results are automatically transmitted back for statistical analysis, forming a complete automated test loop.

[0045] This mechanism, which deeply integrates environment configuration, firmware updates, use case execution, and result processing, completely changes the inefficient traditional model of manually deploying environments and triggering tests. It ensures that testing activities run efficiently and reliably in a standardized environment, significantly improving the automation level and human-friendliness of the testing process.

[0046] like Figure 3In an exemplary embodiment, obtaining an impact code module corresponding to a modified code module and an impact factor corresponding to at least one impact code module includes: obtaining an impact code module affected by the modified code module and an initial impact factor corresponding to at least one impact code module; performing defect prediction on the modified code module, adjusting the initial impact factor according to the prediction result to obtain a final impact factor; generating a test case set for the firmware to be tested according to the impact factor and preset rules, including: generating a test case set for the firmware to be tested according to the final impact factor and preset rules.

[0047] Specifically, the accuracy of the impact factor is improved through the defect prediction mechanism, and a more scientific impact range quantification model is constructed. First, through static code dependency analysis and dynamic call chain tracing, the code modules directly and indirectly affected by the modified code module are identified, and the initial impact factor is assigned based on parameters such as the call frequency between modules and the degree of data coupling. Subsequently, a defect prediction algorithm is introduced to conduct a risk assessment on each modified code module. This defect prediction algorithm trains a machine learning model based on historical firmware defect data. By analyzing features such as the complexity of code changes, the type of change, and the developer's historical defect rate, it predicts the probability and severity of defects that may be introduced by the modified code module. Based on the prediction results, the initial impact factor is weighted and adjusted to form a final impact factor that is more in line with the actual risk.

[0048] It's important to understand that defect prediction for modified code modules can be performed using logistic regression. The principle behind this is to build a data-driven risk assessment model that objectively predicts the probability of potential defects by quantifying the correlation between code characteristics and historical defect patterns. Specifically, multidimensional features are extracted from modified code modules, including metrics such as code complexity (e.g., complexity reflects control flow complexity), change activity (change frequency reflects module stability), development quality metrics (unit test coverage reflects verification adequacy), and developer experience factors, to form a feature vector matrix. These features collectively characterize the risk attributes of code modules. For example, modules with high complexity, frequent changes, and low test coverage typically have higher defect rates.

[0049] Use normalization and other methods to map features of different dimensions to the [0, 1] range to ensure that the logistic regression model has consistent sensitivity to each feature. For example, the number of lines of code (which may be in the hundreds or thousands) and unit test coverage (percentage) can be unified to the same scale to prevent features with large numerical ranges from dominating model decisions.

[0050] The historical data was divided into a training set and a test set at an 80% / 20% ratio. The training set was used to fit a logistic regression model, and the model parameters were determined through maximum likelihood estimation. The test set was used to verify the generalization ability of the logistic regression model. The output of the logistic regression model served as the basis for adjusting the initial impact factor.

[0051] This prediction mechanism, based on learning from historical data, converts subjective "impact levels" into quantifiable "defect prediction results," effectively making up for the shortcomings of relying solely on code structure analysis, making test resource allocation more in line with actual risk distribution and improving test accuracy.

[0052] This mechanism, which combines defect prediction with impact analysis, enables the impact factor to not only reflect code dependencies but also incorporate the dimension of potential defect risks, providing a more accurate quantitative basis for subsequent test case generation and avoiding test coverage deviations caused by relying solely on code structure analysis.

[0053] like Figure 3 In an exemplary embodiment, the initial impact factor is adjusted according to the prediction result to obtain the final impact factor, including: in response to the prediction result indicating that the modified code module has a defect, determining the initial impact factor of the impact code module corresponding to the modified code module as the final impact factor of the impact code module; in response to the prediction result indicating that the modified code module does not have a defect, setting the final impact factor of the impact code module directly functionally coupled to the modified code module to the maximum impact factor, and setting the final impact factor of the impact code module indirectly functionally associated with the modified code module to the minimum impact factor.

[0054] Specifically, a dynamic impact factor adjustment mechanism is constructed based on defect prediction results, enabling the impact factor to more accurately reflect the actual risk level of code changes. Specifically, when the defect prediction model determines that a modified code module contains a potential defect, indicating that the change itself may introduce a fault, the initial impact factor of the affected code module is directly determined as the final impact factor to maintain the original impact range calculated based on code dependencies and ensure that all modules that may be directly or indirectly affected are covered by the test. When the prediction results show that the modified module is defect-free, it indicates that the change itself has a low risk. In this case, the initial impact factor is adjusted differently based on the tightness of functional coupling. The final impact factor of the affected code module with direct functional coupling to the modified code module (such as direct function calls and data sharing) is assigned the maximum impact factor. Since they are more likely to be directly affected by the change, they require focused testing. The final impact factor of the module with indirect functional coupling (such as the impact transmitted through intermediate modules) is set to the minimum impact factor. Since their impact is weaker, the test coverage intensity can be appropriately reduced.

[0055] This mechanism of adjusting the impact factor based on the dual dimensions of defect prediction results and functional coupling levels can not only ensure the comprehensiveness of testing in high-risk scenarios, but also optimize the allocation of test resources in low-risk scenarios, avoid redundant testing, and make the final impact factor more in line with the actual risk distribution, providing a more scientific quantitative basis for test case generation.

[0056] Furthermore, in an exemplary embodiment, the operating data of the SSD firmware and the resource usage of the test environment (such as the utilization rate of the central processing unit and the memory bandwidth) are collected in real time; based on the preset environment optimization rules (such as adjusting the storage bus frequency when the fluctuation of the number of read and write operations per second exceeds 20%), combined with the final influencing factors affecting the code module, the configuration parameters of the test environment (such as the main frequency of the central processing unit of the test machine and the SSD interface protocol version) are dynamically adjusted; after the adjustment, the remaining test cases are continued to be executed, and the optimized test results are compared with the historical baseline data. If an abnormality is found, a secondary environment optimization or test case re-run mechanism is triggered.

[0057] Specifically, by building a closed-loop mechanism of "test execution-feedback-optimization", the dynamic adaptation of the test environment is driven by real-time data. Specifically, the environment optimization module automatically identifies the matching degree between the current environment configuration and the test requirements based on the mapping relationship between the influencing factors and the environmental parameters, combined with the real-time collected firmware running status and resource usage data. For example, when a high-impact factor module encounters a performance bottleneck during testing, the CPU frequency of the test machine is automatically increased to simulate a high-load scenario, or the interface protocol version is lowered to verify the compatibility boundary. This dynamic optimization mechanism breaks the static limitations of the traditional pre-configured environment, allowing the test environment to be adjusted in real time according to the actual running performance of the firmware and the impact of code changes. It not only avoids the insufficient test scenario coverage caused by fixed environment configuration (such as the inability to simulate extreme loads), but also can promptly correct the environmental parameter deviation through the feedback mechanism to ensure the reliability of the test results and the comprehensiveness of the scenario coverage. It is especially suitable for precise testing requirements in complex firmware change scenarios.

[0058] In an exemplary embodiment, the method further includes: when at least two modified code modules correspond to the same impact code module, the final impact factor of the impact code module takes the maximum value of the final impact factors corresponding to the at least two modified code modules.

[0059] Specifically, when at least two modified code modules correspond to the same impact code module, the maximum value strategy is used to ensure that the risk assessment of the impact code module is not underestimated. When multiple modified code modules act on the same impact code module at the same time, the final impact factors corresponding to each modified module may vary due to different defect prediction results and functional coupling strengths. At this time, if the average value or weighted summation is adopted, the actual impact of high-risk changes may be weakened. The maximum value strategy is essentially based on the risk superposition logic. That is, any high-risk attribute of the modified code module (such as defect prediction showing the existence of potential defects, direct functional strong coupling) may cause the impact code module to fail. Therefore, using the maximum impact factor as the final value can not only ensure that the highest-risk scenarios are covered first when generating test cases, but also avoid risk omissions caused by the computational complexity under multi-source influences.

[0060] This strategy in this embodiment improves processing efficiency while ensuring assessment accuracy, enabling testing resources to be precisely focused on the highest-risk areas under the combined effects of multiple change sources, effectively addressing test coverage requirements in complex code change scenarios.

[0061] In an exemplary embodiment, before generating a set of test cases for the firmware to be tested according to the influencing factors and preset rules, it also includes: constructing test cases, wherein the test cases include basic test cases for necessary test items for verifying core functional logic, extended test cases for supplementary test items for verifying non-core functions, and abnormal test cases for verifying fault-tolerant behavior in registration scenarios, wherein the execution priority of the basic test cases is higher than the execution priority of the extended test cases; generating a set of test cases for the firmware to be tested according to the influencing factors and preset rules, including: selecting from the test cases according to the influencing factors and preset rules to generate a set of test cases for the firmware to be tested.

[0062] Specifically, a hierarchical, configurable test case pool is constructed, and automated dynamic test case screening is implemented using impact factors. First, test cases are divided into basic test cases covering core functional logic, extended test cases verifying non-core functionality, and exception test cases verifying fault-tolerant behavior in abnormal scenarios. Basic test cases are prioritized over extended test cases. Subsequently, pre-set rules establish screening logic based on impact factor thresholds: For modules with high impact factors (e.g., those with high defect prediction probability or strong direct functional coupling), basic test cases, extended test cases, and exception test cases are selected to ensure comprehensive coverage of risk points. For modules with low impact factors, only basic test cases or selectively selected extended test cases with high correlation are executed to avoid redundant testing. Furthermore, when multiple modified code modules jointly affect the same affected code module, the highest impact factor is used as the screening criterion to ensure that risks are not underestimated. Furthermore, candidate test cases are ranked using pre-set rules (e.g., the functional relevance of the test case to the affected code module and the test execution cost), prioritizing test items with low execution cost but high coverage, thus achieving a balance between test efficiency and quality.

[0063] This design transforms the test case generation process from one dominated by manual experience to one driven by data and intelligent decision-making. By precisely matching the layered test pool with influencing factors, it not only ensures the full verification of core functions, but also dynamically adjusts the test scope based on the actual impact of code changes, significantly improving the targetedness and efficiency of testing.

[0064] In an exemplary embodiment, a test case set for the firmware to be tested is generated based on the influencing factors and preset rules, including: for each influencing code module, determining a subset of test cases corresponding to the influencing code module from basic test cases, extended test cases and abnormal test cases based on the corresponding final influencing factor; wherein the final influencing factor is positively correlated with the number of elements in the test case set; integrating the subset of test cases corresponding to all influencing code modules to generate a test case set for the firmware to be tested.

[0065] Specifically, impact factors are used to quantify risk levels and drive the dynamic generation of test cases, establishing a test case generation mechanism that precisely matches risk and coverage. Specifically, for each affected code module, a subset of basic, extended, and exception test cases is dynamically selected and combined based on the magnitude of its final impact factor. The impact factor is positively correlated with the number of test cases within the subset. Modules with high impact factors (e.g., those with high defect prediction probability, strong direct functional coupling, or the cumulative impact of multiple change sources) are assigned more test cases. In addition to full coverage with basic test cases, these include more extended test cases (to verify non-core functionality) and exception test cases (to verify fault tolerance) to provide in-depth coverage of potential risk points. Modules with low impact factors are matched only with basic test cases or a small number of extended test cases to avoid test redundancy. After generating the corresponding test case subsets for each affected code module, all test case sets are merged into a complete set through integration strategies such as deduplication and prioritization, ensuring that test coverage strictly corresponds to the actual impact of the code change.

[0066] This mechanism, which dynamically adjusts the number and types of test cases based on influencing factors, not only optimizes resource allocation by focusing on testing high-risk modules and streamlining testing low-risk modules, but also avoids the subjectivity of manual case selection through standardized subset generation and integration processes, making the test case collection generation process more scientific and automated.

[0067] In an exemplary embodiment, for each influencing code module, a subset of test cases corresponding to the influencing code module is determined from basic test cases, extended test cases and abnormal test cases according to the corresponding final impact factor, including: for each influencing code module, if the final impact factor of the influencing code module is greater than or equal to a first threshold, determining that the subset of test cases corresponding to the influencing code module includes basic test cases, extended test cases and abnormal test cases; if the final impact factor of the influencing code module is greater than or equal to a second threshold and less than the first threshold, determining that the subset of test cases corresponding to the influencing code module includes basic test cases and abnormal test cases; the first threshold is greater than the second threshold; if the final impact factor of the influencing code module is greater than zero and less than the second threshold, determining that the subset of test cases corresponding to the influencing code module includes basic test cases.

[0068] Specifically, risk levels are divided by impact factor thresholds, and corresponding differentiated test coverage strategies are established to achieve precise allocation of testing resources. Specifically, two impact factor thresholds are preset (first threshold > second threshold > 0), dividing affected code modules into three risk levels. High-risk modules (final impact factor ≥ first threshold) are assigned a comprehensive testing strategy, with a subset of test cases consisting of basic test cases (verifying core functionality), extended test cases (verifying non-core functionality), and exception test cases (verifying fault tolerance), ensuring in-depth coverage of potential defects. Medium-risk modules (second threshold ≤ final impact factor < first threshold) are assigned a moderate testing strategy, with a subset consisting of only basic and exception test cases, focusing on core functionality verification and boundary condition testing. Low-risk modules (0 < final impact factor < second threshold) are assigned a basic testing strategy, with a subset consisting of only basic test cases, ensuring the correctness of core functionality with minimal testing cost. For example, the first threshold is 0.5 and the second threshold is 0.2.

[0069] This threshold-based grading mechanism essentially converts the quantitative impact of code changes into executable testing decisions. It automatically matches test intensity and risk level through preset rules, avoiding both the omission of defects caused by insufficient testing of high-risk modules and the waste of resources caused by excessive testing of low-risk modules, thus achieving the optimal balance between test coverage and execution efficiency.

[0070] For example, a specific embodiment is: version management module, the developer submitted a part of the code (firmware to be tested) and generated firmware version A, which is located in the error handling module and the garbage collection module (modified code module); in the process of submitting the code, the developer needs to enter the modified code module, the affected code module and the impact factor, for example, the impact factor of the error handling module code for the error handling module is 1, the impact factor for the bad block management module is 0.8, the impact factor for the data recovery module is 0.6, the impact factor for the performance module is 0.5, and the impact factor for the compatibility module is 0.2; the impact factor of the garbage collection module code for the garbage collection module is 1, the impact factor for the data transmission module is 0.8, and the impact factor for the performance module is 0.8; after the review is passed, enter the test case management module.

[0071] In the test case management module, test case mapping is generated based on the influencing factors.

[0072] Test case mapping table 1 = {(error handling module, [error handling test case, 1; bad block management test case, 0.8; data recovery test case, 0.6; performance test case, 0.5; compatibility test case, 0.2]), (garbage collection module, [garbage collection test case, 1; output transmission test case, 0.8; performance test case, 0.8])}.

[0073] In the Test Case Management module, click the Defect Prediction button. Assuming the defect prediction result for the error handling module is 1 (defective), Test Case Mapping Table 1 will not be updated and will be executed according to the current policy. Assuming the defect prediction result for the garbage collection module is 0 (no defect), the initial impact factors will be revised. Specifically, the impact factors for garbage collection test cases directly coupled to its functionality remain unchanged, while the impact factors for other code modules are revised to 0.1. This means that the test cases corresponding to the garbage collection module must be fully run, while the test cases for other code modules only require basic test cases.

[0074] The updated test case mapping table 2 is: Test case mapping table 2 = {(error handling module, [error handling test case, 1; bad block management test case, 0.8; data recovery test case, 0.6; performance test case, 0.5; compatibility test case, 0.2]), (garbage collection module, [garbage collection test case, 1; data transmission test case, 0.1; performance test case, 0.1])}.

[0075] For the performance test case, both modified code modules are related to it and are treated as having a high impact factor of 0.5.

[0076] The final test case set is: {error handling test case A, error handling test case B, error handling test case C, bad block management test case A, bad block management test case B, bad block management test case C, data recovery test case A, data recovery test case B, data recovery test case C, performance test case A, performance test case B, performance test case C, compatibility test case A, compatibility test case B, garbage collection test case A, garbage collection test case B, garbage collection test case C, data transmission test case A}; It should be understood that A here represents the basic test case, B represents the abnormal test case, and C represents the extended test case.

[0077] The test system sends the corresponding test case set to the corresponding test platform, updates the test platform's test firmware, and clicks the Start Test button to execute the test. For example, the error handling module and performance testing module are tested on the error handling test platform and performance test platform, respectively. After the test is completed, the test results are submitted to the platform for analysis and the generation of a test report.

[0078] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0079] like Figure 4 An embodiment of the present application also provides a testing device, including: a first acquisition module 41, used to obtain a firmware to be tested, the firmware to be tested includes at least one modified code module; a second acquisition module 42, used to obtain an impact code module corresponding to the modified code module, and an impact factor corresponding to at least one impact code module; a use case generation module 43, used to generate a test case set for the firmware to be tested according to the impact factor and preset rules; a testing module 44, used to execute the test case set in a preconfigured test environment to test the firmware to be tested.

[0080] In an exemplary embodiment, the second acquisition module 42 includes: an initialization module for obtaining the affecting code modules affected by the modified code module and the initial impact factor corresponding to at least one affecting code module; an adjustment module for performing defect prediction on each modified code module, adjusting the initial impact factor according to the prediction result, and obtaining the final impact factor; a use case generation module 43, specifically used to generate a set of test cases for the firmware to be tested according to the final impact factor and preset rules.

[0081] In an exemplary embodiment, the use case generation module 43 includes: a first generation module for determining, in response to a prediction result indicating that a defect exists in the modified code module, an initial impact factor of the influencing code module corresponding to the modified code module as a final impact factor of the influencing code module; and a second generation module for setting, in response to a prediction result indicating that the modified code module does not exist, the final impact factor of the influencing code module that is directly functionally coupled to the modified code module to a maximum impact factor value, and setting the final impact factor of the influencing code module that is indirectly functionally associated with the modified code module to a minimum impact factor value.

[0082] In an exemplary embodiment, the method further includes: a third generating module configured to, when at least two modified code modules correspond to the same impact code module, take the maximum value of the final impact factors corresponding to the at least two modified code modules as the final impact factor of the impact code module.

[0083] In an exemplary embodiment, before generating a set of test cases for the firmware to be tested according to the influencing factors and preset rules, it also includes: a use case construction module for constructing test cases, wherein the test cases include basic test cases for necessary test items for verifying the core functional logic, extended test cases for supplementary test items for verifying non-core functions, and abnormal test cases for verifying fault-tolerant behavior in registration scenarios, wherein the execution priority of the basic test cases is higher than the execution priority of the extended test cases; a use case generation module 43, specifically for selecting from the test cases according to the influencing factors and preset rules to generate a set of test cases for the firmware to be tested.

[0084] In an exemplary embodiment, the use case generation module 43 includes: a subset determination module, which is used to determine, for each influencing code module, a subset of test cases corresponding to the influencing code module from basic test cases, extended test cases and abnormal test cases according to the corresponding final influencing factor; wherein the final influencing factor is positively correlated with the number of elements in the test case subset; and an integration module, which is used to integrate the test case subsets corresponding to all influencing code modules to generate a test case set for the firmware to be tested.

[0085] In an exemplary embodiment, for each influencing code module, the subset determination module includes: a first determination module, which is used to determine, for each influencing code module, if the final impact factor of the influencing code module is greater than or equal to a first threshold, that the subset of test cases corresponding to the influencing code module includes basic test cases, extended test cases and abnormal test cases; a second determination module, which is used to determine, if the final impact factor of the influencing code module is greater than or equal to a second threshold and less than the first threshold, that the subset of test cases corresponding to the influencing code module includes basic test cases and abnormal test cases; the first threshold is greater than the second threshold; and a third determination module, which is used to determine, if the final impact factor of the influencing code module is greater than zero and less than the second threshold, that the subset of test cases corresponding to the influencing code module includes basic test cases.

[0086] For the description of the features in the embodiment corresponding to the testing device, please refer to the relevant description of the embodiment corresponding to the testing method, and no further details will be given here.

[0087] like Figure 5 An embodiment of the present application further provides an electronic device, including a memory 101 and a processor 102, wherein the memory 101 stores a computer program 202, and the processor 102 is configured to run the computer program 202 to perform the steps in any of the above-mentioned test method embodiments.

[0088] like Figure 6 An embodiment of the present application further provides a computer-readable storage medium 201, in which a computer program 202 is stored, wherein the computer program 202 is configured to execute the steps of any of the above-mentioned test method embodiments when running.

[0089] In an exemplary embodiment, the computer-readable storage medium 201 may include, but is not limited to, various media that can store the computer program 202, such as a USB flash drive, a read-only memory 101 (ROM), a random access memory 101 (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0090] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above-mentioned test method embodiments are implemented.

[0091] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned test method embodiments are implemented.

[0092] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0093] The above is a detailed introduction to a test method, device, electronic device and storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A testing method, characterized in that: include: Acquire a firmware to be tested, wherein the firmware to be tested includes at least one modified code module; Obtaining an impact code module corresponding to the modified code module and an initial impact factor corresponding to at least one of the impact code modules; determining the initial impact factor by identifying the impact code modules directly and indirectly affected by the modified code module through static code dependency analysis and dynamic call chain tracing, and assigning the initial impact factor based on the call frequency and data coupling between the modules; Defect prediction is performed on the modified code module to obtain a prediction result; the prediction result is obtained by performing a risk assessment on each modified code module based on a defect prediction algorithm. The defect prediction algorithm trains a machine learning model based on historical firmware defect data and predicts the probability and severity of defects introduced by the modified code module by analyzing the complexity of the code change, the type of change, and the developer's historical defect rate characteristics; In response to the prediction result indicating that the modified code module has a defect, determining an initial impact factor of an impact code module corresponding to the modified code module as a final impact factor of the impact code module; In response to the prediction result indicating that the modified code module does not have defects, setting the final impact factor of the impact code module directly functionally coupled to the modified code module to a maximum impact factor, and setting the final impact factor of the impact code module indirectly functionally associated with the modified code module to a minimum impact factor; Generating a test case set for the firmware to be tested according to the influencing factors and preset rules; Executing the test case set in a preconfigured test environment to test the firmware to be tested; Generating a test case set for the firmware to be tested according to the influencing factors and preset rules, including: A test case set for the firmware to be tested is generated according to the final influencing factor and the preset rule.

2. The testing method according to claim 1, wherein: Also includes: When at least two of the modified code modules correspond to the same influencing code module, the final influencing factor of the influencing code module takes the maximum value of the final influencing factors corresponding to the at least two modified code modules.

3. The testing method according to any one of claims 1 to 2, characterized in that: Before generating a test case set for the firmware to be tested according to the influencing factors and preset rules, the method further includes: Construct test cases, wherein the test cases include basic test cases for verifying required test items of core functional logic, extended test cases for verifying supplementary test items of non-core functions, and abnormal test cases for verifying fault-tolerant behavior in registration scenarios, wherein the execution priority of the basic test cases is higher than the execution priority of the extended test cases; Generating a test case set for the firmware to be tested according to the influencing factors and preset rules, including: According to the influencing factors and preset rules, test cases are selected from the test cases to generate a test case set for the firmware to be tested.

4. The testing method according to claim 3, wherein: Generating a test case set for the firmware to be tested according to the influencing factors and preset rules, including: For each of the impact code modules, determine a subset of test cases corresponding to the impact code module from the basic test cases, the extended test cases, and the abnormal test cases according to the corresponding final impact factor; wherein the final impact factor is positively correlated with the number of elements in the subset of test cases; Integrate the test case subsets corresponding to all the impact code modules to generate a test case set for the firmware to be tested.

5. The testing method according to claim 4, characterized in that: For each of the impact code modules, determining a subset of test cases corresponding to the impact code module from the basic test cases, the extended test cases, and the abnormal test cases according to the corresponding final impact factor, including: For each of the influencing code modules, if the final influencing factor of the influencing code module is greater than or equal to a first threshold, determining that the subset of test cases corresponding to the influencing code module includes the basic test case, the extended test case, and the abnormal test case; If the final impact factor of the influencing code module is greater than or equal to a second threshold and less than the first threshold, determining that the subset of test cases corresponding to the influencing code module includes the basic test case and the abnormal test case; and the first threshold is greater than the second threshold; If the final impact factor of the influencing code module is greater than zero and less than the second threshold, it is determined that the subset of test cases corresponding to the influencing code module includes the basic test case.

6. A testing device, characterized in that: include: A first acquisition module is used to acquire a firmware to be tested, wherein the firmware to be tested includes at least one modified code module; A second acquisition module is configured to acquire an impact code module corresponding to the modified code module and an initial impact factor corresponding to at least one of the impact code modules; performing defect prediction on the modified code module to obtain a prediction result; in response to the prediction result indicating that the modified code module has a defect, determining an initial impact factor of an impact code module corresponding to the modified code module as a final impact factor of the impact code module; In response to the prediction result indicating that the modified code module does not have defects, setting the final impact factor of the impact code module directly functionally coupled to the modified code module to a maximum impact factor, and setting the final impact factor of the impact code module indirectly functionally associated with the modified code module to a minimum impact factor; The initial impact factor is determined by: identifying the code modules directly and indirectly affected by the modified code module through static code dependency analysis and dynamic call chain tracking, and assigning the initial impact factor based on the call frequency and data coupling between modules; The prediction results are obtained by performing a risk assessment on each modified code module using a defect prediction algorithm. The defect prediction algorithm trains a machine learning model based on historical firmware defect data. By analyzing the complexity and type of code changes and the developer's historical defect rate, the algorithm predicts the probability and severity of defects introduced by the modified code module. A test case generation module, configured to generate a test case set for the firmware to be tested according to the influencing factors and preset rules; A testing module, configured to execute the test case set in a preconfigured test environment to test the firmware to be tested; The use case generation module is specifically used to generate a test case set for the firmware to be tested according to the final influencing factor and the preset rules.

7. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the testing method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the testing method according to any one of claims 1 to 5 when executed by a processor.

Images