Test sequence optimization method and device, equipment and storage medium

Abstract

The invention discloses a test sequence optimization method and device, equipment and a storage medium. Comprising the steps of obtaining function point related data, and generating a function point test sequence based on an upper confidence bound algorithm; defining a global resource occupancy matrix, and according to the function point test sequence, sequentially importing the function points, traversing and querying the global resource occupancy matrix, and generating a parallel scheduling task; and executing the parallel scheduling task, and monitoring resource conflict conditions and test failure information in real time. By comprehensively considering related data of the function points, intelligent sorting of the function points is achieved, and the problems that in a traditional sorting method, exploration and utilization are difficult to balance and global dynamic optimization is lacked are solved. By reasonably distributing parallel tasks, resource conflicts are avoided, the utilization rate of limited test resources is improved, and efficiency reduction caused by unreasonable resource planning is reduced. Resource conflicts are found and processed in time through real-time monitoring, the conflicts are prevented from influencing the test process, test failures can be quickly responded, and the test efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of automated testing technology, and in particular to a test sequence optimization method, apparatus, device, and storage medium. Background Technology

[0002] In the field of automated testing technology, especially for testing complex systems with multiple functions, such as complex embedded products, system-on-a-chip (SoC) and industrial software, the rationality of the test sequence and the effectiveness of resource scheduling directly affect testing efficiency and resource utilization.

[0003] In existing technologies, traditional methods for arranging test sequences for complex systems often employ fixed-order testing or heuristic sorting. Regarding resource scheduling, they frequently rely on simple parallel test planning; and in terms of test process monitoring, they lack dynamic handling mechanisms for resource conflicts and test failures. Meanwhile, some intelligent sorting algorithms, such as deep reinforcement learning and parallel scheduling algorithms, such as genetic algorithms, attempt to optimize the testing process.

[0004] However, traditional sorting methods do not consider resource conflicts and time efficiency, and heuristic sorting lacks global dynamic optimization capabilities, leading to resource idleness or frequent conflicts. Intelligent sorting algorithms and some parallel scheduling algorithms have high computational overhead, making real-time optimization difficult and unable to meet the dynamic testing requirements of complex systems. Summary of the Invention

[0005] This invention provides a test sequence optimization method, apparatus, device, and storage medium. By using function point sorting based on the upper confidence bound algorithm, resource conflict assessment and parallel scheduling, and dynamic pruning and sequence reorganization, it solves the problems in complex system testing, such as resource idleness or conflict caused by many test cases, low efficiency of parallel test resource competition, redundancy and waste caused by repeated testing of related function points due to function point test failures, as well as the problems of insufficient global dynamic optimization, large computational overhead, and lack of dynamic pruning mechanism in traditional sorting and intelligent algorithms.

[0006] According to one aspect of the present invention, a test sequence optimization method is provided, the method comprising:

[0007] Acquire relevant data for function points and generate a test sequence for function points based on the upper confidence bound algorithm;

[0008] Define a global resource usage matrix, import function points sequentially according to the function point test sequence, traverse and query the global resource usage matrix, and generate parallel scheduling tasks;

[0009] Execute parallel scheduling tasks and monitor resource conflicts and test failure information in real time.

[0010] Optionally, the method also includes: when a test failure is detected, triggering dynamic pruning and sequence reorganization to generate a new test sequence; and regenerating the parallel scheduling task based on the new test sequence.

[0011] Optionally, dynamic pruning and sequence recombination are triggered to generate new test sequences, including: calculating the error propagation probability of failed function points to other function points under test; removing function points under test with error propagation probabilities greater than a threshold; and recombining the remaining function points to generate new test sequences.

[0012] Optionally, acquire relevant data for each function point and generate a function point test sequence based on the upper confidence bound algorithm. This includes: acquiring relevant data for each function point, including historical error rate, average test time, test time standard deviation, resource requirement vector, importance level, degree of recent design changes, and error propagation probability; setting algorithm parameters for the upper confidence bound algorithm to generate an upper confidence bound score calculation model including utilization items, exploration items, and priority items, wherein the algorithm parameters include initial exploration intensity, minimum exploration intensity, decay constant, utilization item weight coefficient, priority item weight coefficient, and adjustment coefficient; calculating the priority score of each function point using the upper confidence bound score calculation model, and sorting each function point in descending order of priority score to generate a function point test sequence.

[0013] Optionally, according to the function point test sequence, import function points sequentially, traverse and query the global resource occupancy matrix, and generate parallel scheduling tasks, including: importing function points sequentially according to the function point test sequence, obtaining the resource demand vector of the currently imported function point and the resource occupancy data in the global resource occupancy matrix, as well as the total number of instances of each type of resource; taking each type of resource as a target resource; determining whether the sum of the current demand of the imported function point for the target resource and the current occupancy of the target resource in the global resource occupancy matrix exceeds the total number of instances of the target resource to determine the first resource conflict situation; and generating parallel scheduling tasks based on the first resource conflict situation.

[0014] Optionally, a parallel scheduling task is generated based on the first resource conflict situation, including: when the first resource conflict situation is that a resource conflict exists, a new parallel queue is created, the currently imported function point is added to the new parallel queue, and the global resource occupancy matrix is ​​updated with the resource requirements of the new parallel queue; when the first resource conflict situation is that no resource conflict exists, the currently imported function point is added to the current parallel queue, and the corresponding resource occupancy in the global resource occupancy matrix is ​​updated; when all function points have been imported, a parallel scheduling task consisting of multiple parallel queues without resource conflicts is generated.

[0015] Optionally, execute parallel scheduling tasks and monitor resource conflicts in real time, including: executing parallel scheduling tasks, monitoring changes in the resource occupancy matrix in real time to obtain the second resource conflict situation during the execution of parallel tasks; determining whether the second resource conflict situation is a resource conflict, if so, waiting for the test resources to be released; otherwise, continuing to import subsequent tests according to the parallel scheduling tasks until all functional tests are completed.

[0016] According to another aspect of the present invention, a test sequence optimization apparatus is provided, the apparatus comprising:

[0017] The function point test sequence generation module is used to acquire function point-related data and generate function point test sequences based on the upper confidence bound algorithm.

[0018] The parallel scheduling task generation module is used to define the global resource usage matrix, import function points in sequence according to the function point test sequence, traverse and query the global resource usage matrix, and generate parallel scheduling tasks.

[0019] The resource conflict monitoring module is used to execute parallel scheduling tasks and monitor resource conflicts and test failure information in real time.

[0020] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0021] At least one processor;

[0022] and a memory communicatively connected to the at least one processor;

[0023] The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform a test sequence optimization method according to any embodiment of the present invention.

[0024] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement a test sequence optimization method according to any embodiment of the present invention.

[0025] The technical solution of this invention, by comprehensively considering relevant data of functional points, achieves intelligent sorting of functional points, solving the problem of balancing exploration and utilization and the lack of global dynamic optimization in traditional sorting methods. By rationally allocating parallel tasks, it avoids resource conflicts, improves the utilization rate of limited test resources, and reduces efficiency reduction caused by unreasonable resource planning. Through real-time monitoring, it promptly detects and handles resource conflicts, preventing conflicts from affecting the testing process, and enables rapid response to test failures, thus improving testing efficiency.

[0026] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of a test sequence optimization method provided in Embodiment 1 of the present invention;

[0029] Figure 2 This is a flowchart of another test sequence optimization method provided in Embodiment 2 of the present invention;

[0030] Figure 3 This is a schematic diagram of a test sequence optimization device according to Embodiment 3 of the present invention;

[0031] Figure 4 This is a schematic diagram of the structure of an electronic device that implements a test sequence optimization method according to an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] Example 1

[0035] Figure 1 This is a flowchart of a test sequence optimization method provided in Embodiment 1 of the present invention. This embodiment is applicable to automated testing scenarios. The method can be executed by a test sequence optimization device, which can be implemented in hardware and / or software and can be configured in a computer controller. Figure 1 As shown, the method includes:

[0036] S110. Obtain function point related data and generate function point test sequences based on the upper confidence bound algorithm.

[0037] In this context, a function point refers to a module or unit with a specific function in a complex system, such as an automotive ECU controller or a complex embedded product. Examples include CAN communication and sensor calibration in an automotive ECU, which are the fundamental objects of testing. Function point-related data includes historical error rates, average test times, test time standard deviations, resource requirement vectors, importance levels, recent design change severity, and error propagation probability. This data is used to calculate the Upper Confidence Bound (UCB) algorithm to achieve a reasonable ranking of function points. The UCB algorithm is an algorithm for balancing exploration and exploitation. In this embodiment, it is used for function point ranking. Its formula is: Exploration item, Exploration item, Priority item. It comprehensively considers historical data effectiveness, exploration needs, and function point priorities, solving the problem of balancing exploration and exploitation and the lack of global dynamic optimization in traditional ranking methods. The function point test sequence refers to the test order queue formed after ranking function points using the UCB algorithm, such as {Fault Diagnosis, CAN Communication, Sensor Calibration…}. The function point test sequence is the basis for subsequent resource scheduling and test execution.

[0038] Optionally, acquire relevant data for each function point and generate a function point test sequence based on the upper confidence bound algorithm. This includes: acquiring relevant data for each function point, including historical error rate, average test time, test time standard deviation, resource requirement vector, importance level, degree of recent design changes, and error propagation probability; setting algorithm parameters for the upper confidence bound algorithm to generate an upper confidence bound score calculation model including utilization items, exploration items, and priority items, wherein the algorithm parameters include initial exploration intensity, minimum exploration intensity, decay constant, utilization item weight coefficient, priority item weight coefficient, and adjustment coefficient; calculating the priority score of each function point using the upper confidence bound score calculation model, and sorting each function point in descending order of priority score to generate a function point test sequence.

[0039] Specifically, when acquiring data related to functional points, the information to be collected includes: the historical error rate of the functional point, which is the probability that the functional point has encountered errors in previous tests; the average test time, which is the average time spent testing the functional point historically; the standard deviation of test time, which is the standard test time, usually the sum of the maximum time taken to test each test case of the functional point; the resource requirement vector, which is a vector representing the resource requirements of each functional point, such as the specific quantity of resources required by a functional point, such as USB ports and CAN channels; the importance level, which measures the importance of the functional point in the system, defaulting to between 0 and 1; the degree of recent design changes, which reflects the recent design changes to the functional point, also defaulting to between 0 and 1; and the error propagation probability, which is the average probability that an error in the functional point will cause errors in other unsorted functional points.

[0040] Furthermore, when setting the algorithm parameters for the upper confidence bound algorithm, the initial exploration intensity is configured to 2.5 by default, which determines the exploration intensity of unknown functional points in the early stage of the algorithm; the minimum exploration intensity is configured to 0.5 by default, which is the lower limit of the exploration intensity; the decay constant is configured to 0.1 times the total number of sequences that can be formed by all functional items by default, which is used to control the decay rate of the exploration intensity with the number of iterations; the utilization item weight coefficient includes error rate weight, time-saving rate weight, and error propagation rate weight, and the sum of the three coefficients is 1, which is used to balance the influence of historical error rate, test time-saving rate, and average error propagation probability in the utilization item; the priority item weight coefficient includes importance level weight and recent design change degree weight, and the sum of the two is 1, which is used to balance the role of functional importance level and recent design change degree in the priority item; the adjustment coefficient is used to adjust the influence of the priority item in the overall upper confidence bound score.

[0041] Specifically, the constructed upper confidence boundary score calculation model consists of three parts: utilization term, exploration term, and priority term. It is expressed using the following formula (1):

[0042]

[0043] Among them, UCB(f i ) represents the priority score, U a Indicates the use item, Denotes the exploration term, λ·P(f) i () indicates a priority item. The utilization item is represented by the following formula (2):

[0044]

[0045] Among them, U a Represents the utilization term, E(f) i ) represents function point f i Historical error rate Function point fi Historical average test time Function point f i The standard test time is usually the sum of the maximum time taken to test each test case for each functional point individually, P. epa (f i ) represents function point f i The average probability that an error will cause errors in other unsorted function points, i.e., the error propagation probability, w. e w t and w a The weighting coefficient is used to balance the weights among the three main considerations of the three utilization items: functional error rate, time-saving rate, and error propagation rate. The sum of the three coefficients is 1. The utilization item formula integrates three factors: historical error rate (quality-oriented), test time-saving rate (time-oriented), and average error propagation probability (risk-oriented). It prioritizes test items with high error rates, shorter test times, and higher error propagation rates to the front of the test sequence, thereby improving the overall functional testing efficiency. The exploration factor is expressed by the following formula (3):

[0046]

[0047] Where c(t) represents the exploration factor, c base The initial exploration intensity is set to 2.5 by default. min The minimum exploration intensity is set to 0.5 by default, where t represents the total number of iterations of the current test sequence, and k represents the decay constant, with the default setting being the total number of sequences that can be formed by all functional items * 0.1. The exploration factor aims to dynamically adjust the intensity of test sequence optimization exploration in the early and late stages of a large number of repetitive tests (such as the factory testing of batch products), so as to achieve effective exploration and optimization of the best test sequence as much as possible under the limited computational overhead of the test system. The priority item is represented by the following formula (4):

[0048] P(f i )=α·c(f i )+β·D(f i (4)

[0049] Wherein, P(f i ) represents the priority term, C(f) i ) represents function point f i Importance level, default setting range is between 0 and 1, D(f i ) represents function point f iThe recent design change level is set by default between 0 and 1. α and β are weighting coefficients used to balance the weights of the two priority items: the importance level of the function and the recent design change level. The priority item considers both the importance level of the product function and the recent design change level to effectively balance the functional importance of multiple product features with the impact of version iterations, thus compensating for the shortcomings of the utilization item which only considers historical data analysis.

[0050] Specifically, using the aforementioned upper confidence boundary score calculation model, the priority score for each function point is calculated. This priority score integrates the results of utilization, exploration, and priority metrics. Then, all function points are sorted from highest to lowest according to their calculated priority scores. The resulting queue is the function point test sequence. For example, in automotive ECU controller testing, after calculating the scores for each function point, functions such as fault diagnosis and CAN communication are arranged according to their scores, forming a specific test sequence.

[0051] S120. Define the global resource usage matrix, import the function points in sequence according to the function point test sequence, traverse and query the global resource usage matrix, and generate parallel scheduling tasks.

[0052] The global resource occupancy matrix, defined as Ocp = [oc1, oc2, ..., ocm], is used to monitor the occupancy of various resources in the testing system in real time. Here, ocj represents the number of currently occupied resources rj, 0 ≤ ocj ≤ cj, and cj is the total number of instances of resource rj. Resources refer to various resources required during the testing process, including physical devices such as USB ports and spectrum analyzers, software resources such as license certificates and virtual machine instances, and network resources such as bandwidth channels and IP address pools, denoted as the set R = {r1, r2, ..., rm}. Function point resource requirements refer to the resource requirements of each function point i, represented as the vector Rqi = [qi1, qi2, ..., qim], where qij is the resource requirement of function point i for resource rj. Parallel scheduling tasks refer to the group of parallel-executable tasks formed by splitting the function point test sequence, considering resource conflicts, to improve testing efficiency.

[0053] Optionally, according to the function point test sequence, import function points sequentially, traverse and query the global resource occupancy matrix, and generate parallel scheduling tasks, including: importing function points sequentially according to the function point test sequence, obtaining the resource demand vector of the currently imported function point and the resource occupancy data in the global resource occupancy matrix, as well as the total number of instances of each type of resource; taking each type of resource as a target resource; determining whether the sum of the current demand of the imported function point for the target resource and the current occupancy of the target resource in the global resource occupancy matrix exceeds the total number of instances of the target resource to determine the first resource conflict situation; and generating parallel scheduling tasks based on the first resource conflict situation.

[0054] Specifically, when importing function points sequentially according to the function point test sequence, the function points are first imported into the resource scheduling system one by one, following the previously generated test sequence order. For each currently imported function point, its resource requirement vector needs to be obtained. This vector contains the specific requirements of the function point for each type of resource, such as how many are needed for USB ports, CAN channels, and other types of resources. Simultaneously, data from the global resource occupancy matrix needs to be obtained. This matrix records the current number of occupied resources of each type and the total number of instances of each type of resource, i.e., how many available instances of each type of resource. Then, the controller performs conflict detection on each type of resource as a target resource. Specifically, for each type of resource, the sum of the current imported function point's requirement for that type of resource and the current occupancy of that type of resource in the global resource occupancy matrix is ​​calculated. Then, it is determined whether the total occupancy exceeds the total number of instances of that type of resource. If, for a certain type of target resource, the total occupancy exceeds its total number of instances, it indicates that the currently imported function point conflicts with the already scheduled function points on that type of resource, i.e., a first resource conflict has occurred. If the sum of the demand and current usage of all target resources does not exceed the total number of instances of each category, there is no resource conflict.

[0055] Optionally, a parallel scheduling task is generated based on the first resource conflict situation, including: when the first resource conflict situation is that a resource conflict exists, a new parallel queue is created, the currently imported function point is added to the new parallel queue, and the global resource occupancy matrix is ​​updated with the resource requirements of the new parallel queue; when the first resource conflict situation is that no resource conflict exists, the currently imported function point is added to the current parallel queue, and the corresponding resource occupancy in the global resource occupancy matrix is ​​updated; when all function points have been imported, a parallel scheduling task consisting of multiple parallel queues without resource conflicts is generated.

[0056] Specifically, when a resource conflict is detected (meaning the sum of the current imported feature's demand for a certain type of target resource and the current occupancy of that type of resource in the global resource occupancy matrix exceeds the total number of instances of that type of resource), a new parallel queue needs to be created. The controller then adds the currently imported feature to the newly created parallel queue and updates the global resource occupancy matrix with the resource demand vector of that feature in the new parallel queue, setting the occupancy of the corresponding resource in the matrix to the feature's demand for all types of resources. Conversely, when no resource conflict is detected (meaning the sum of the current imported feature's demand for all types of target resources and the current occupancy of the corresponding resource in the global resource occupancy matrix does not exceed the total number of instances of any type of resource), the currently imported feature is directly added to the currently being built parallel queue, and the occupancy of the corresponding resource in the global resource occupancy matrix is ​​updated based on the feature's resource demand vector, adding the feature's demand for all types of resources to the existing occupancy. Finally, once all functionalities have been imported, a parallel scheduling task consisting of multiple parallel queues will be generated. These parallel queues do not have resource conflicts and can be tested simultaneously, thereby improving testing efficiency.

[0057] For example, in automotive ECU controller testing, if the current parallel queue already occupies 2 CAN channels, and there are a total of 3 CAN channels, and a newly imported function point requires 2 CAN channels, then the sum of the demand and the current occupancy exceeds the total number of instances. In this case, a new parallel queue will be created, and the function point will be added. At the same time, the CAN channel occupancy in the global resource occupancy matrix will be updated to 2. If the newly imported function point only requires 1 CAN channel, and the sum of the demand and the current occupancy does not exceed the total number of instances, then it will be added to the current parallel queue, and the CAN channel occupancy in the global resource occupancy matrix will be updated to 3. Once all function points have been imported, these parallel queues constitute the final parallel scheduling task.

[0058] S130: Execute parallel scheduling tasks and monitor resource conflicts and test failure information in real time.

[0059] Parallel scheduling task execution refers to the process of simultaneously testing multiple functionalities according to the generated parallel scheduling tasks, making full use of test resources and shortening the overall test time. Test failure information refers to information related to a functionality failing the test during the testing process. When a test failure is detected, subsequent operations such as error propagation analysis, dynamic pruning, and sequence reassembly are triggered. Real-time monitoring refers to continuously tracking changes in resource usage and test results of functionalities during the execution of parallel scheduling tasks, promptly identifying resource conflicts and test failures, ensuring the smooth progress of the testing process, and enabling dynamic adjustments.

[0060] Optionally, execute parallel scheduling tasks and monitor resource conflicts in real time, including: executing parallel scheduling tasks, monitoring changes in the resource occupancy matrix in real time to obtain the second resource conflict situation during the execution of parallel tasks; determining whether the second resource conflict situation is a resource conflict, if so, waiting for the test resources to be released; otherwise, continuing to import subsequent tests according to the parallel scheduling tasks until all functional tests are completed.

[0061] Specifically, after generating parallel scheduling tasks based on the first resource conflict situation, resource usage needs to be monitored in real time during the execution of these tasks. Specifically, executing parallel scheduling tasks involves simultaneously testing the functionalities within multiple generated parallel queues without resource conflicts. During execution, the controller continuously monitors changes in the global resource usage matrix to identify potential second resource conflicts during parallel task execution. This means checking if any functionalities are causing the total usage of a certain type of resource to exceed its total number of instances. If resource conflicts exist, it means the currently executing task has exceeded the system's total resource capacity for certain tasks. In this case, it's necessary to wait for test resources to be released until a sufficient number are available and conflicts cease before continuing execution. If no resource conflicts exist, subsequent test functionalities are continuously imported and tested according to the order of the parallel scheduling tasks until all functionalities have been tested.

[0062] For example, in automotive ECU controller testing, when executing parallel task 1 (comprising fault diagnosis and CAN communication) and parallel task 2 (comprising sensor calibration, etc.), the occupancy matrix of resources such as power supply and CAN channels is monitored in real time. If, during execution, the total power consumption of the two tasks exceeds the total number of power supply instances at a certain moment, a second resource conflict is detected. At this point, it is necessary to wait for one of the tasks to release some power resources. If no resource conflict is detected during monitoring, the next functional point is imported sequentially for testing until all ECU functional point tests are completed.

[0063] The technical solution of this invention, by comprehensively considering relevant data of functional points, achieves intelligent sorting of functional points, solving the problem of balancing exploration and utilization and the lack of global dynamic optimization in traditional sorting methods. By rationally allocating parallel tasks, it avoids resource conflicts, improves the utilization rate of limited test resources, and reduces efficiency reduction caused by unreasonable resource planning. Through real-time monitoring, it promptly detects and handles resource conflicts, preventing conflicts from affecting the testing process, and enables rapid response to test failures, thus improving testing efficiency.

[0064] Example 2

[0065] Figure 2This is a flowchart of a test sequence optimization method provided in Embodiment 2 of the present invention. This embodiment adds a dynamic pruning and sequence recombination process to the above-described Embodiment 1. The specific content of steps S210-S230 is largely the same as steps S110-S130 in Embodiment 1, therefore, it will not be described again in this embodiment. Figure 2 As shown, the method includes:

[0066] S210. Obtain function point related data and generate function point test sequences based on the upper confidence bound algorithm.

[0067] Optionally, acquire relevant data for each function point and generate a function point test sequence based on the upper confidence bound algorithm. This includes: acquiring relevant data for each function point, including historical error rate, average test time, test time standard deviation, resource requirement vector, importance level, degree of recent design changes, and error propagation probability; setting algorithm parameters for the upper confidence bound algorithm to generate an upper confidence bound score calculation model including utilization items, exploration items, and priority items, wherein the algorithm parameters include initial exploration intensity, minimum exploration intensity, decay constant, utilization item weight coefficient, priority item weight coefficient, and adjustment coefficient; calculating the priority score of each function point using the upper confidence bound score calculation model, and sorting each function point in descending order of priority score to generate a function point test sequence.

[0068] S220. Define the global resource usage matrix, import the function points in sequence according to the function point test sequence, traverse and query the global resource usage matrix, and generate parallel scheduling tasks.

[0069] Optionally, according to the function point test sequence, import function points sequentially, traverse and query the global resource occupancy matrix, and generate parallel scheduling tasks, including: importing function points sequentially according to the function point test sequence, obtaining the resource demand vector of the currently imported function point and the resource occupancy data in the global resource occupancy matrix, as well as the total number of instances of each type of resource; taking each type of resource as a target resource; determining whether the sum of the current demand of the imported function point for the target resource and the current occupancy of the target resource in the global resource occupancy matrix exceeds the total number of instances of the target resource to determine the first resource conflict situation; and generating parallel scheduling tasks based on the first resource conflict situation.

[0070] Optionally, a parallel scheduling task is generated based on the first resource conflict situation, including: when the first resource conflict situation is that a resource conflict exists, a new parallel queue is created, the currently imported function point is added to the new parallel queue, and the global resource occupancy matrix is ​​updated with the resource requirements of the new parallel queue; when the first resource conflict situation is that no resource conflict exists, the currently imported function point is added to the current parallel queue, and the corresponding resource occupancy in the global resource occupancy matrix is ​​updated; when all function points have been imported, a parallel scheduling task consisting of multiple parallel queues without resource conflicts is generated.

[0071] S230 executes parallel scheduling tasks and monitors resource conflicts and test failure information in real time.

[0072] Optionally, execute parallel scheduling tasks and monitor resource conflicts in real time, including: executing parallel scheduling tasks, monitoring changes in the resource occupancy matrix in real time to obtain the second resource conflict situation during the execution of parallel tasks; determining whether the second resource conflict situation is a resource conflict, if so, waiting for the test resources to be released; otherwise, continuing to import subsequent tests according to the parallel scheduling tasks until all functional tests are completed.

[0073] S240. When a test failure is detected, dynamic pruning and sequence recombination are triggered to generate a new test sequence.

[0074] Specifically, when a test failure is detected, a dynamic pruning and sequence recombination process is immediately triggered. First, the controller starts the correlation analysis module to calculate the error propagation probability, which is derived by combining the design expected risk probability and the historical occurrence conditional probability. It is expressed by the following formula (5):

[0075] Error propagation probability = a·P DFEMA +b·P historical (5)

[0076] Where a and b represent weighting coefficients, P DFEMA P represents the expected risk probability in the design. historical This represents the conditional probability of a historical event. When the error propagation probability exceeds a threshold, high-probability related functional points are pruned, meaning they are removed from subsequent test sequences. The remaining unpruned functional points are then recombine to generate new test sequences.

[0077] Optionally, dynamic pruning and sequence recombination are triggered to generate new test sequences, including: calculating the error propagation probability of failed function points to other function points under test; removing function points under test with error propagation probabilities greater than a threshold; and recombining the remaining function points to generate new test sequences.

[0078] Specifically, the error propagation probability integrates two dimensions: the design expected risk probability and the historical occurrence conditional probability. The design expected risk probability (P_DFEMA) is determined based on the Design Failure Mode and Effects Analysis (DFEMA) document and is expressed by the following formula (6):

[0079]

[0080] Here, S represents the severity assessment value of a failed test function point causing other test functions to fail in the DFEMA file, and O represents the probability assessment value of such a failure occurring in other test functions. Typically, S and O range from 1 to 10. The historical occurrence conditional probability is calculated from historical data, specifically the number of times the failed function point failed simultaneously with other test functions, divided by the number of times the failed function point failed alone; its value ranges from 0 to 1.

[0081] Furthermore, the controller determines whether the calculated error propagation probability exceeds a set pruning threshold. This pruning threshold can be adjusted based on the product testing phase or test completion time requirements. When the error propagation probability of a function under test exceeds the preset threshold, that function is removed from the subsequent test sequence—this is pruning. Finally, the controller reassembles all remaining unpruned function points in a specific order to generate a new test sequence for subsequent resource scheduling and testing. For example, in automotive ECU controller testing, if the CAN communication function test fails, and the calculated error propagation probability for the sensor calibration function is 0.81, while the pruning threshold is 0.8, the sensor calibration function will be removed, and the remaining function points, such as the barometric pressure sensor acquisition function, will be reassembled into a new test sequence.

[0082] S250, Regenerate parallel scheduling tasks based on the new test sequence.

[0083] Specifically, after generating a new test sequence, the controller re-enters the resource conflict assessment and parallel scheduling phase based on the new test sequence. This involves importing function points sequentially according to the new test sequence, iterating through the global resource occupancy matrix, detecting resource conflicts, creating new parallel queues or adding them to the current queue based on the conflict status, updating the resource occupancy matrix, and finally regenerating a scheduleable task that can be executed in parallel, continuing the test until completion. For example, in automotive ECU controller testing, if the CAN communication function test fails, and the calculated error propagation probability to the sensor calibration function exceeds a threshold, the sensor calibration function will be pruned, and the remaining function points, such as barometric pressure sensor acquisition, will be reorganized into a new sequence, and resource scheduling will be re-performed to generate new parallel tasks.

[0084] The technical solution of this invention reduces redundant testing caused by logical connections between functional points by calculating the error propagation probability and removing highly probable related functional points under test, thereby avoiding test redundancy and resource waste, and improving the relevance and efficiency of testing. Regenerating parallel scheduling tasks based on new test sequences allows the new sequences to adapt to the current resource state, ensuring the rationality of test tasks at the resource level, continuing to leverage the resource utilization improvement effect of parallel scheduling, and guaranteeing efficient test progress.

[0085] Example 3

[0086] Figure 3 This is a schematic diagram of a test sequence optimization device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: a function point test sequence generation module 310, used to acquire function point related data and generate a function point test sequence based on the upper confidence bound algorithm;

[0087] Parallel scheduling task generation module 320 is used to define the global resource occupancy matrix, import function points sequentially according to the function point test sequence, traverse and query the global resource occupancy matrix, and generate parallel scheduling tasks.

[0088] The resource conflict monitoring module 330 is used to execute parallel scheduling tasks and monitor resource conflicts and test failure information in real time.

[0089] Optionally, the device further includes a dynamic pruning and sequence reassembly module, used to: trigger dynamic pruning and sequence reassembly when a test failure is detected to generate a new test sequence; and regenerate the parallel scheduling task based on the new test sequence.

[0090] Optionally, a dynamic pruning and sequence recombination module is used to: calculate the error propagation probability of a failed function point to other function points under test; remove function points under test whose error propagation probability is greater than a threshold; and recombine the remaining function points to generate a new test sequence.

[0091] Optionally, the function point test sequence generation module 310 is specifically used for: acquiring function point-related data, including the historical error rate, average test time, test time standard deviation, resource requirement vector, importance level, recent design change degree, and error propagation probability of the function point; setting the algorithm parameters of the upper confidence bound algorithm to generate an upper confidence bound score calculation model including utilization items, exploration items, and priority items, wherein the algorithm parameters include initial exploration intensity, minimum exploration intensity, decay constant, utilization item weight coefficient, priority item weight coefficient, and adjustment coefficient; calculating the priority score of each function point through the upper confidence bound score calculation model, and sorting the function points in descending order of priority score to generate a function point test sequence.

[0092] Optionally, the parallel scheduling task generation module 320 is specifically used for: importing function points sequentially according to the function point test sequence; obtaining the resource demand vector of the currently imported function point and the resource occupancy data in the global resource occupancy matrix, as well as the total number of instances of each type of resource; taking each type of resource as a target resource; determining whether the sum of the current demand of the imported function point for the target resource and the current occupancy of the target resource in the global resource occupancy matrix exceeds the total number of instances of the target resource to determine the first resource conflict situation; and generating a parallel scheduling task based on the first resource conflict situation.

[0093] Optionally, the parallel scheduling task generation module 320 specifically includes: a parallel scheduling task generation unit, used to: when the first resource conflict situation is that a resource conflict exists, create a new parallel queue, add the currently imported function point to the new parallel queue, and update the global resource occupancy matrix with the resource requirements of the new parallel queue; when the first resource conflict situation is that there is no resource conflict, add the currently imported function point to the current parallel queue, and update the corresponding resource occupancy in the global resource occupancy matrix; when all function points have been imported, generate a parallel scheduling task composed of multiple parallel queues without resource conflicts.

[0094] Optionally, the resource conflict monitoring module 330 is specifically used to: execute parallel scheduling tasks, monitor changes in the resource occupancy matrix in real time to obtain the second resource conflict situation during the execution of parallel tasks; determine whether the second resource conflict situation is a resource conflict, if so, wait for the test resources to be released; otherwise, continue to import subsequent tests according to the parallel scheduling tasks until all functional tests are completed.

[0095] The technical solution of this invention, by comprehensively considering relevant data of functional points, achieves intelligent sorting of functional points, solving the problem of balancing exploration and utilization and the lack of global dynamic optimization in traditional sorting methods. By rationally allocating parallel tasks, it avoids resource conflicts, improves the utilization rate of limited test resources, and reduces efficiency reduction caused by unreasonable resource planning. Through real-time monitoring, it promptly detects and handles resource conflicts, preventing conflicts from affecting the testing process, and enables rapid response to test failures, thus improving testing efficiency.

[0096] The test sequence optimization device provided in this embodiment of the invention can execute a test sequence optimization method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0097] Example 4

[0098] Figure 4A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0099] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0100] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0101] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a test sequence optimization method.

[0102] In some embodiments, a test sequence optimization method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the test sequence optimization method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform a test sequence optimization method by any other suitable means (e.g., by means of firmware).

[0103] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0104] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0105] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0106] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0107] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0108] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0109] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and no limitation is imposed herein.

[0110] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A test sequence optimization method, characterized in that, include: Acquire relevant data for function points and generate a test sequence for function points based on the upper confidence bound algorithm; Define a global resource usage matrix, import function points sequentially according to the function point test sequence, traverse and query the global resource usage matrix, and generate parallel scheduling tasks; Execute the parallel scheduling task and monitor resource conflicts and test failure information in real time.

2. The method according to claim 1, characterized in that, The method further includes: When a test failure is detected, dynamic pruning and sequence recombination are triggered to generate a new test sequence; The parallel scheduling task is regenerated based on the new test sequence.

3. The method according to claim 2, characterized in that, The triggering of dynamic pruning and sequence recombination to generate new test sequences includes: Calculate the error propagation probability of a failed function point to other function points under test; Remove the functional points under test whose error propagation probability is greater than the threshold, and reorganize the remaining functional points to generate a new test sequence.

4. The method according to claim 1, characterized in that, The step of acquiring function point-related data and generating a function point test sequence based on the upper confidence bound algorithm includes: Acquire function point-related data, including the function point's historical error rate, average test time, test time standard deviation, resource requirement vector, importance level, degree of recent design changes, and error propagation probability; Set the algorithm parameters for the upper confidence bound algorithm to generate an upper confidence bound score calculation model that includes utilization items, exploration items, and priority items. The algorithm parameters include initial exploration intensity, minimum exploration intensity, decay constant, utilization item weight coefficient, priority item weight coefficient, and adjustment coefficient. The priority score of each function point is calculated using the upper confidence boundary score calculation model. The function points are then sorted in descending order of priority score to generate a function point test sequence.

5. The method according to claim 1, characterized in that, The process of importing function points sequentially according to the function point test sequence, traversing and querying the global resource usage matrix, and generating parallel scheduling tasks includes: According to the functional point test sequence, import the functional points in sequence, and obtain the resource requirement vector of the currently imported functional point, the resource usage data in the global resource usage matrix, and the total number of instances of various resources. Each type of resource is designated as a target resource. Determine whether the sum of the current demand for target resources by the imported function and the current occupancy of target class resources in the global resource occupancy matrix exceeds the total number of instances of target class resources to determine the first resource conflict situation; Parallel scheduling tasks are generated based on the first resource conflict situation.

6. The method according to claim 5, characterized in that, The step of generating parallel scheduling tasks based on the first resource conflict includes: When the first resource conflict situation is that a resource conflict exists, a new parallel queue is created, the currently imported function point is added to the new parallel queue, and the global resource occupancy matrix is ​​updated with the resource requirements of the new parallel queue. When the first resource conflict situation is that there is no resource conflict, the currently imported function point is added to the current parallel queue, and the corresponding resource occupancy in the global resource occupancy matrix is ​​updated; Once all functionalities have been imported, a parallel scheduling task consisting of multiple parallel queues without resource conflicts is generated.

7. The method according to claim 6, characterized in that, The execution of the parallel scheduling task and real-time monitoring of resource conflicts include: Execute the parallel scheduling task and monitor the changes in the resource occupancy matrix in real time to obtain the second resource conflict situation during the parallel task execution process; Determine whether the second resource conflict situation constitutes a resource conflict; if so, wait for the test resource to be released. Otherwise, continue importing subsequent tests as parallel scheduling tasks until all functional tests are completed.

8. A test sequence optimization device, characterized in that, include: The function point test sequence generation module is used to acquire function point-related data and generate function point test sequences based on the upper confidence bound algorithm. The parallel scheduling task generation module is used to define the global resource usage matrix, import function points in sequence according to the function point test sequence, traverse and query the global resource usage matrix, and generate parallel scheduling tasks. The resource conflict monitoring module is used to execute the parallel scheduling task and monitor resource conflict status and test failure information in real time.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer storage medium stores computer instructions that are used to cause a processor to execute the method of any one of claims 1-7.

Images