Test methods, apparatus, electronic devices and computer-readable storage media

Abstract

This application provides a testing method, apparatus, electronic device, and computer-readable storage medium. The method includes: generating test cases based on the current test status and historical feedback information; executing the test cases and collecting execution process data; determining execution feedback results based on the execution process data, wherein the execution feedback results include at least one of conflict events and abnormal behaviors; and updating the test case generation strategy based on the execution feedback results until a preset test termination condition is met. By generating test cases, collecting test process data, analyzing the test process data, and optimizing the test case generation strategy, the testing process is made intelligent. Compared with testing methods that rely on fixed scripts or random generation, the method of this application saves manpower and improves testing efficiency.

Description

Technical Field

[0001] This application relates to the field of testing technology, and in particular to a testing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] CAN (Controller Area Network) bus is widely used in automotive electronics, industrial control, aerospace, and robotics control. As system complexity continues to increase, the number of ECUs (Electronic Control Units) connected to the CAN bus network is also growing, leading to higher requirements for CAN bus testing and verification.

[0003] In related technologies, testers pre-write test case scripts based on prior experience or manual design to test the CAN bus.

[0004] The above methods rely on human experience, and fixed test cases cannot cover complex bus interaction scenarios. When the system under test is updated or new testing requirements arise, test scripts need to be rewritten manually, resulting in low testing efficiency. Summary of the Invention

[0005] In view of the above problems, embodiments of the present invention are proposed to provide a testing method, apparatus, electronic device and computer-readable storage medium that overcomes or at least partially solves the above problems.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, embodiments of this application disclose a testing method, the method comprising: Generate test cases based on the current test status and historical feedback information; Execute the test cases and collect execution process data; The execution feedback result is determined based on the execution process data, and the execution feedback result includes at least one of conflict events and abnormal behavior; Based on the execution feedback results, update the test case generation strategy until the preset test termination conditions are met.

[0007] Secondly, embodiments of this application disclose a testing apparatus, the apparatus comprising: The generation module is used to generate test cases based on the current test status and historical feedback information; The execution module is used to execute the test cases and collect execution process data; The feedback module is used to determine the execution feedback result based on the execution process data, wherein the execution feedback result includes at least one of conflict events and abnormal behavior; The update module is used to update the test case generation strategy based on the execution feedback results until the preset test termination conditions are met.

[0008] Thirdly, embodiments of this application disclose an electronic device, including: a processor connected to a memory; the memory being used to store a computer program; and the processor being used to execute the computer program stored in the memory to implement the steps in the method described in the first aspect.

[0009] Fourthly, embodiments of this application disclose a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0010] This application discloses a testing method that generates test cases based on the current test state and historical feedback information. This links the generation of test cases with historical execution results and the current test state, allowing the testing process to dynamically adjust subsequent testing strategies based on conflict events and abnormal behaviors, thereby improving test coverage. Simultaneously, the test case generation strategy is optimized based on execution feedback results, improving the accuracy of the generated test cases and ensuring they match current actual testing requirements. By generating test cases, collecting and analyzing test process data, and optimizing the test case generation strategy, the testing process is made intelligent. Compared to testing methods that rely on fixed scripts or random generation, this application's method saves manpower and improves testing efficiency. Attached Figure Description

[0011] Figure 1 This is a system architecture diagram provided in an embodiment of this application; Figure 2 This is a flowchart of the steps of a testing method provided in an embodiment of this application; Figure 3 This is a block diagram of a testing device provided in an embodiment of this application; Figure 4 This is a block diagram of an electronic device provided in an embodiment of this application; Figure 5 This is a schematic diagram of another electronic device provided in the embodiments of this application. Detailed Implementation

[0012] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0013] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and are not limited in number; for example, a first object can be one or more. Furthermore, the term "and / or" in the specification and claims is used to describe the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. In the embodiments of this application, the term "multiple" refers to two or more, and other quantifiers are similar.

[0014] refer to Figure 1 , Figure 1 This application discloses a system architecture diagram provided by an embodiment. In terms of hardware, a 1-to-N architecture is adopted: the host control board is connected to a CAN bus, the CAN bus connects to multiple ECUs, driving multiple slave test nodes (ECUs) to achieve parallel data transmission and reception; each test node is interconnected via the CAN bus, and each listens to and records data on the bus. The host control board has high-speed signal processing capabilities and an interface for communication with the AI ​​module. Testing of the CAN bus can be performed by a test device. Depending on the object under test, the test device can be any of an industrial control computer, test server, vehicle computing platform, or cloud server. The test device communicates with the CAN bus under test, and the CAN bus network under test connects to multiple ECUs. The test device can run a test software system, which is internally divided into a control layer, a scheduling layer, and an AI-assisted layer. The control layer is responsible for issuing test cases, acquiring CAN signals, and interacting with the CAN driver. The scheduling layer is responsible for task queue management, thread pool scheduling, and allocating threads to execute tasks according to task priority and resource status, while also receiving instructions from the AI-assisted layer (such as dynamically adjusting priorities and inserting new test steps). The AI-assisted layer includes modules such as intelligent test case generation, conflict prediction, anomaly detection, and scheduling optimization. These modules, based on big data and machine learning technologies, analyze and optimize the testing process, and feed the results back to the scheduling or control layer.

[0015] During system runtime, the testing process is as follows: The scheduling layer fills the task queue according to a predefined test plan or intelligently generated test cases (provided by the intelligent test case module). The thread pool allocates threads to read and execute tasks, such as sending a CAN frame to an ECU and receiving reply data. The received data is passed to the anomaly detection module for real-time analysis; the scheduler simultaneously monitors system load and task completion status, and if potential conflicts or resource bottlenecks are detected, it calls the conflict prediction or scheduling optimization module to make decision adjustments; the intelligent test case module continuously updates and supplements subsequent test cases dynamically based on historical execution results. The entire system forms a closed-loop feedback loop, and the AI ​​module continuously optimizes the testing strategy through experience learning and online computation.

[0016] refer to Figure 2 , Figure 2 This application discloses a flowchart of a testing method provided in an embodiment, including: Step 101: Generate test cases based on the current test status and historical feedback information.

[0017] In this embodiment, the current test state reflects the current state of the test environment, including the operating status of each actuator (ECU) (e.g., active, dormant, offline), and the list of covered functions. Historical feedback information can be historical test data, including historical test case coverage information, historical failure rate, error distribution, etc. Test cases can be test operations or configurations, such as selecting a test path or inserting a signal sequence at a specific time.

[0018] For example, in an automotive scenario, the list of covered functions could include tested engine control, transmission control, and lathe control, but untested steering control and braking control, as well as the operating status of each actuator, such as the headlight ECU being in sleep mode or the gateway being active. Historical feedback information could include: historical test case coverage information, such as 60%; historical failure rate, such as 10% anomaly rate out of 100 tests; and error distribution, such as 50% validation errors and 50% data errors. For a robotic scenario, the list of covered functions could include tested motor temperature and end effector control, but untested emergency stop response; the operating status of each actuator, such as joint controller 1 being active and joint controller 2 being in sleep mode; and historical feedback information could include: historical test case coverage information, such as 70%; historical failure rate, such as 5% anomaly rate out of 100 tests; and error distribution, such as 50% timeout errors and 50% data loss errors.

[0019] Furthermore, to improve test coverage and defect detection capabilities, reinforcement learning or deep learning methods can be used to automatically generate and select test cases. This involves using the current test state and historical feedback information as output to construct a test case generation strategy and automatically generate test cases. The process of generating test cases can involve encoding the current test state and historical feedback information into state vectors, selecting test actions from a defined test action space based on these state vectors, and then generating test cases according to these actions. This allows the generated test cases to reference historical data and experience, improving testing efficiency.

[0020] Step 102: Execute the test cases and collect execution process data.

[0021] In this embodiment, execution process data can be collected during test case execution. For example, according to the test case configuration information, a CAN message is sent to the target actuator on the CAN bus, and response data is received. The execution process data can be data collected from the CAN bus, including message frames, timestamps, retransmission events, etc. During the execution of the test cases, bus data is collected in real time for subsequent analysis of the execution process data, enabling updates to the test case generation strategy.

[0022] Step 103: Determine the execution feedback result based on the execution process data. The execution feedback result includes at least one of conflict events and abnormal behavior.

[0023] In this embodiment, the execution process data is data collected from the CAN bus during the execution of test cases, and the execution feedback result is data used to reflect the execution effect after the test cases are executed. Conflict events are conflicts that occur during the execution of test cases, such as message retransmissions and errors. Abnormal behaviors include signal loss, abnormal frame intervals, etc.

[0024] Step 104: Update the test case generation strategy based on the execution feedback results until the preset test termination conditions are met.

[0025] In this embodiment, the execution status of the test case can be evaluated based on the execution feedback results. For example, a reward value can be calculated based on the execution feedback results, and a reinforcement learning algorithm can be used to update the test case generation strategy based on the reward value. This strategy prioritizes test actions that can obtain high reward values. Supplementary test cases are generated based on the updated test case generation strategy, and the test process continues until the preset test termination condition is met. The test termination condition can be a pre-set condition, such as reaching the coverage target or reaching the upper limit of the number of test cases.

[0026] This application updates the test case generation strategy by executing feedback results, enabling the test case generation strategy to continuously learn and optimize subsequent test case generation, thereby improving test coverage and testing efficiency.

[0027] Optionally, step 101 includes: Sub-step 1011: Encode the current test state and the historical feedback information into a state vector; the current test state includes the running state of each executor and at least one item from the list of covered functions; the historical feedback information includes at least one item from historical test case coverage information, historical failure rate, and error distribution. Sub-step 1012: Based on the state vector, determine the test action; the test action includes at least one of selecting a test path, inserting a signal sequence, and configuring test case parameters; Sub-step 1013: Generate test cases based on the test actions.

[0028] In this embodiment, for sub-steps 1011 to 1013, the current test state and historical feedback information are encoded into a state vector. The current test state includes real-time information such as the running status of each executor and the list of covered functions. The historical feedback information includes historical test case coverage information, historical failure rate, and error distribution, among other empirical data from historical test case execution. Test actions are determined based on the state vector. The state vector is input into a test case generation strategy, which outputs test actions. The test case generation strategy can be a reinforcement learning policy network. Test actions can be selected from a preset test action space, including at least one of selecting a test path, inserting a signal sequence, or configuring test case parameters. Test cases are generated through these test actions. For example, if the test action is selecting a test path, a test case sequence corresponding to that path is generated; if the action is inserting a signal sequence, test cases containing a specific signal sequence are generated.

[0029] The test case generation strategy includes state representation, action space, and reward function. State representation refers to describing the current test state and historical feedback information using vectors or other structures, such as historical test case coverage vectors, activation status of each ECU, historical failure rate, or error distribution. Action space defines optional test actions, such as "selecting the next test path" or "inserting a signal sequence at a specific time." Each action corresponds to a specific test case or step configuration. The reward function assigns a reward value based on the test execution result. A higher reward is given if the new test case improves signal coverage or triggers an anomaly; otherwise, a lower reward or penalty is given. The reward design can comprehensively consider metrics such as coverage improvement rate, anomaly detection rate, and test resource consumption.

[0030] Furthermore, reinforcement learning algorithms such as Deep Q-Network (DQN), Proximal Policy Optimization (PPO), and Deep Deterministic Policy Gradient (DDPG) can be used to train the test case generation strategy. The model continuously explores different test case sequences, obtains feedback through interaction with the environment (testing platform), and updates its decision-making strategy based on the feedback. The model update rule can adopt the Bellman equation:

[0031] in This is the current state. For the selected test cases, The reward obtained after executing the test cases. As a discount factor, For learning rate, This refers to the rewards that can be obtained in the future under the new state.

[0032] The policy gradient method can directly optimize the policy function. Its objective is to maximize the expected cumulative reward, and the policy gradient updates the policy parameter θ using the following formula:

[0033] in Let be the policy function, representing the state. Select test cases below The probability, Let be the advantage function. If the advantage of a certain action is positive, the probability of that action being selected is increased; if the advantage is negative, the probability of that action being selected is decreased. In this way, the policy network gradually learns the optimal decision policy.

[0034] After training with a large amount of simulated or real-world test data, the intelligent test case module can predict which test cases are more likely to cover untested paths or trigger exceptions, and thus prioritize their selection. Reinforcement learning-based test generation can significantly improve defect detection and coverage.

[0035] This application generates test cases based on the current test status and the historical feedback information, so that the generated test cases can match the current test requirements and improve test efficiency.

[0036] Optionally, step 103 includes: Sub-step 1031: The execution process data is segmented and encoded according to time windows to form a time-series input sequence; Sub-step 1032: Based on the time-series input sequence, detect conflict events and abnormal behaviors, and use the detected conflict events and abnormal behaviors as the execution feedback results.

[0037] In this embodiment, for sub-steps 1031 to 1033, the execution process data is segmented and encoded according to time windows to form a time-series input sequence. The execution process data is, for example, a CAN data stream. The acquired CAN data stream is segmented according to time windows, and the signal sequence within each window is converted into model input. Different signal IDs and data domains can be encoded to form a time-series input sequence. Conflict events and abnormal behaviors are detected based on the time-series input sequence. Further, conflict events are detected based on the time-series input sequence. During training, conflict events recorded in historical operation logs (such as message retransmission counts) are used as labels to identify bus conflicts. For abnormal behaviors, an autoencoder constructed using a Long Short-Term Memory (LSTM) network is used for anomaly detection. During the training phase, only normal operating data is used to compress and reconstruct the signal sequence. The reconstruction error is defined as e = ||x - x'||². When the reconstruction error exceeds a threshold, it is marked as an anomaly, thereby detecting abnormal behavior. The detected conflict events and abnormal behaviors are used as execution feedback results for subsequent updates to the test case generation strategy, enabling the system to iterate in a closed loop based on historical execution results.

[0038] Optionally, sub-step 1032 includes: Sub-step 10321: The time-series input sequence is reconstructed using an autoencoder to obtain a reconstructed sequence; the autoencoder is trained using normal operating data. Sub-step 10322: Calculate the reconstruction error based on the time-series input sequence and the reconstruction sequence; Sub-step 10323: When the reconstruction error exceeds the first preset threshold, it is determined that there is abnormal behavior.

[0039] In this embodiment, for sub-steps 10321 to 10323, the autoencoder can be an LSTM (Long Short-Term Memory) autoencoder. During the training phase, only normal operating data is used, allowing the autoencoder to learn the pattern characteristics of normal signals through compressed reconstruction of the signal sequence. When an abnormal time-series input sequence is input, since the autoencoder has not learned the abnormal pattern, it cannot accurately reconstruct the original sequence, resulting in a significant increase in reconstruction error. The reconstruction error is defined as e = ||x - x'||², where x is the original input sequence and x' is the reconstructed sequence. When e exceeds a first preset threshold, it is marked as abnormal.

[0040] LSTM can capture long-term dependencies in sequences, making it suitable for time-series analysis of CAN signals. The internal state update of an LSTM cell is implemented through a gating mechanism, and the internal state update of an LSTM cell can be represented as:

[0041]

[0042]

[0043]

[0044]

[0045]

[0046] in, For activation function, This is the current input (a timing input sequence, such as CAN). , These represent the hidden state and the cellular state at the previous moment, respectively. , , These are the activation values ​​for the forget gate, input gate, and output gate, respectively. The decision of which information to discard from the cell state is made by examining the previous hidden state. and current input Output a vector between 0 and 1, where each number represents the degree to which information corresponding to the cell state is retained or discarded. and It refers to the weights and biases of the forget gate. Hidden state The weights of the input gate. Determine which new information will be stored in the cell state, and receive and As input, , , These are the corresponding weights and biases. Used to determine which information to output from the current cell state. , , These are the corresponding weights and biases. Candidate cell states are used to combine the cell state from the previous time step with the current information to update the cell state. , , These are the corresponding weights and biases. To combine the cell state from the previous moment with the latest information and update the cell state, This represents the cell state at the previous moment. Generate the hidden state at the current moment, as the encoding result at the current moment. The activation function is hyperbolic tangent. Through a gating mechanism, LSTM can selectively retain or forget historical information, capturing long-term dependencies in CAN signal sequences, thereby achieving anomaly detection.

[0047] Optionally, sub-step 1032 includes: Sub-step 10324: Classify the temporal input sequence using an attention mechanism to obtain a classification result; Sub-step 10325: Determine whether there is abnormal behavior based on the classification results.

[0048] In the embodiments of this application, for sub-steps 10324 to 10325, anomaly detection can also incorporate an attention mechanism into the encoder or classification model to assign dynamic weights to different signal components, thereby improving the accuracy of anomaly detection and assisting in root cause analysis.

[0049] Furthermore, an attention layer can be added to the output layer of the LSTM encoder or the Transformer model. The attention weights are then calculated. :

[0050] in, This represents the hidden representation of each time step in the sequence. For context vectors, The hidden representation of the current moment. This is a trainable weight matrix. The attention weight distribution is obtained by calculating the similarity between the hidden representations at the current time step and those at each historical time step; a higher weight indicates that the historical time step is more important to the current decision. Based on the attention weights, the context vector is further calculated. This involves weighted summation of the hidden representations at each historical moment to reinforce learning decision-making. Through attention mechanisms, it's possible to identify historical signal segments that have the greatest impact on the current judgment at a specific moment, thus providing a certain root cause cue.

[0051] In any of the above schemes, the model output can be a binary classification result (normal / abnormal) or anomaly probability. During model training, cross-entropy loss (for classification) or mean squared error loss (for reconstruction) can be used, etc.

[0052] Optionally, the method further includes: Step 105: Calculate the influence weight of each historical moment on the classification result; Step 106: Generate an attention heatmap based on the influence weights to locate the source of the abnormal behavior.

[0053] In this embodiment, for steps 105 to 106, the system utilizes an attention mechanism to identify historical signal segments that have the greatest impact on the current judgment at a specific moment, thus providing root cause indications. For example, when an anomaly is detected, the CAN signal segment corresponding to a historical moment with a high impact weight is the main source of the anomaly. An attention heatmap can be generated based on the impact weight to visualize the source of the anomaly. That is, after an anomaly is detected, an alarm can be generated and an attention heatmap can be displayed to help users analyze the source of the anomaly. Through the visualized attention heatmap, testers can intuitively see which signal segments led to the anomaly judgment, thereby quickly locating the root cause of the problem.

[0054] Optionally, the test case is associated with at least one target executor or message flow, and step 102 includes: Sub-step 1021: During execution, the target executor or the message stream is constructed as graph nodes, and the relationships between nodes are constructed as graph edges; Sub-step 1022: Obtain the node features of each graph node, and generate a node representation by aggregating the features of neighboring nodes; the node features include at least one of message frequency, load index, or message type distribution; Sub-step 1023: Predict the conflict probability of each node based on the node representation; Sub-step 1024: Adjust the message sending strategy based on the predicted conflict probability.

[0055] In this embodiment, for sub-steps 1021 to 1024, during parallel stress testing, multiple ECUs may simultaneously send messages to the bus, leading to conflicts and delays. To reduce such conflicts, graph neural networks (GNNs) or related models can be used to model and predict CAN bus communication. Specifically, the implementation can be as follows: abstract each ECU (actuator) or message flow participating in communication in the CAN bus network as a node in a graph structure, and construct the relationships between nodes (such as signals with a high probability of simultaneous transmission being considered as edges) as edges in the graph, forming a graph network G=(V,E) describing the CAN bus topology and interaction relationships, where each node v∈V represents an ECU or message flow, and edge E represents the relationship between ECUs or messages.

[0056] For each graph node, acquire its node features, including at least one of the following: message frequency, load metric, or message type distribution of the corresponding ECU or message stream. Using a graph neural network, each node aggregates the feature information of its neighbors to form a node representation that reflects the node's context within the network.

[0057] Graph Neural Networks (GNNs) are used for feature propagation and learning on graphs. Taking Graph Convolutional Networks (GCNs) as an example, the update process can be described as follows:

[0058] in For adjacency matrices with self-loops, It is a degree matrix. For the first Layer nodes represent matrices. For trainable weights, The activation function is . After several layers of propagation, the GNN can generate node representations that consider neighborhood relationships. Based on the node representations, the model can predict the collision probability or communication congestion of each node or message stream, thereby identifying which nodes have a high risk of collision. Conflict prediction can be treated as a classification or regression task: during training, collision events recorded in historical logs (such as message retransmission counts) are used as labels. The message sending strategy is dynamically adjusted based on the predicted collision probability. When the predicted collision probability of a node exceeds a preset threshold, the system reduces message priority by increasing its CAN identifier value or actively postpones message transmission to avoid collision peaks; when a critical message stream (such as security-related control signals) is identified, the system increases its priority by decreasing its CAN identifier value or pre-allocates collision-free transmission slots to ensure reliable transmission of critical messages.

[0059] For example, in a robot bus communication scenario, each joint controller and end effector is treated as a node in a graph, and the relationships between nodes are constructed as edges in the graph. These relationships include frequent interaction of commands between coaxial joints and the timing of motion commands. Features such as message frequency, load indicators, and message type distribution of each node are obtained. By aggregating the features of adjacent joint controllers, node representations are generated. Based on the node representations, the probability of message conflicts occurring between nodes in the future is predicted. When the predicted conflict probability of a certain joint controller exceeds a threshold, the priority of the controller's messages is reduced or its transmission time is postponed to avoid critical commands, such as emergency stop signals, being delayed or lost due to bus conflicts.

[0060] Optionally, sub-step 1024 includes: Sub-step 10241: If the predicted conflict probability exceeds the second preset threshold, reduce the priority of the corresponding message or postpone the sending time. Sub-step 10242: If a key message stream is identified, increase the priority of the key message stream or reserve a transmission time slot.

[0061] In this embodiment, regarding sub-steps 10241 to 10242, when the predicted conflict probability of a certain ECU or message stream exceeds a second preset threshold, it indicates that the message may conflict with other messages on the bus. To avoid conflict, the priority of the corresponding message can be reduced. Specifically, this can be done by increasing the CAN identifier value of the message (a smaller CAN identifier value indicates higher priority; increasing the identifier value reduces priority), or by delaying the transmission time of the message until after the current peak conflict period. By proactively intervening before a conflict occurs, bus conflicts can be avoided, improving the success rate of test execution.

[0062] When a message to be sent is identified as a critical message stream—for example, in the automotive field, signals with high real-time requirements related to braking control and steering control—it is determined to be a critical message stream. In robotics scenarios, signals such as speed control commands, joint position commands, and stop commands are identified as critical information streams. In this case, the priority of the critical message stream can be increased. For example, this can be done by reducing the CAN identifier value of the message (reducing the identifier value increases priority), making it prioritized during bus arbitration, or by reserving a transmission time slot and pre-allocating a conflict-free transmission window, ensuring that the critical message stream can be sent within the specified time. Through conflict prediction, the real-time performance and reliability of the critical message stream can be guaranteed, avoiding message delays or loss caused by bus conflicts.

[0063] This application automatically adjusts the testing strategy based on real-time feedback during the testing process, without manual intervention. For example, when frequent collisions are detected in a signal cluster, the system automatically adjusts the priority or inserts a waiting state to reduce the error rate. This closed-loop intelligent control scheme makes the testing process more automated.

[0064] Optionally, step 104 includes; Sub-step 1041: Calculate the reward value based on the execution feedback result; Sub-step 1042 involves updating the test case generation strategy based on the reward value using a reinforcement learning algorithm, and generating supplementary test cases based on the updated test case generation strategy.

[0065] In this embodiment, for sub-steps 1041 to 1042, a reward value is calculated based on the execution feedback result. If the test case executed this time covers a new signal path or triggers abnormal behavior, a higher positive reward is given; if the test case causes excessive bus resource consumption or repeated testing, a lower reward is given. The reward function can be designed by comprehensively considering throughput (number of tasks completed), latency (queue waiting time), or resource utilization.

[0066] The calculated reward values ​​are used to update the test case generation strategy. When a test action receives a high reward, the probability of that action being selected is increased; when it receives a low or negative reward, the probability of that action being selected is decreased. Through continuous learning, the policy network gradually learns which types of test actions are more likely to cover untested paths or trigger exceptions, thereby continuously optimizing decisions. After the update is complete, supplementary test cases are generated based on the updated test case generation strategy and added to the task queue for execution, forming a closed-loop iteration.

[0067] Optionally, step 102 includes: Sub-step 1025: Obtain the scheduling status, which includes the task queue length, the load of each thread, etc. Sub-step 1026: Based on the scheduling state, determine the scheduling action, which includes at least one of allocating execution threads to test cases, adjusting the thread pool size, or setting task priorities; Sub-step 1027: Perform resource allocation for the test cases according to the scheduling action.

[0068] In this embodiment of the application, for sub-steps 1025 to 1027, the scheduling status is collected in real time before or during the execution of test cases. The scheduling status includes at least one of the following: task queue length, load of each thread, response latency of each executor, or system computing resource utilization. The task queue length may be the number of test cases to be executed currently; the load of each thread is used to describe the busyness of each execution thread; the response latency of each executor is used to reflect the real-time response capability of the system under test; and the system computing resource utilization is used to reflect the resource reserve of the test equipment itself.

[0069] The optimal scheduling action can be determined using deep reinforcement learning algorithms (such as deep Q-networks or deep deterministic policy gradient algorithms). Scheduling actions include at least one of the following: allocating execution threads to test cases, adjusting the thread pool size, or setting task priorities. The scheduler continuously explores different scheduling actions and optimizes the scheduling strategy based on the reward feedback obtained from the execution results.

[0070] For example, a scheduling policy can be trained using a Deep Q-Network (DQN) or a Deep Policy Gradient (DDPG) system. Taking DQN as an example, its iterative formula is:

[0071] Let the action value function be... This is the current state. For the current action, For instant rewards, Next state, Next move, For learning rate, As a discount factor, For the maximum future reward.

[0072] For example, in the testing scenario of a robot control system, deep Q-networks learn the optimal scheduling strategy through continuous trial and error: current state This can include task queue length, CPU load, bus load rate, and the status of each joint controller. Under the current status, the scheduler selects an action. ,action To assign a joint testing task to a specific thread, and to provide an immediate reward based on task completion efficiency, response latency, and bus conflict status after execution. and enter the next state. By updating the action value function through the above iterative formula, the scheduler learns to prioritize the execution of critical tasks in similar states, such as the robotic arm emergency stop response test, thereby improving the overall test throughput. At the same time, it can prioritize the processing of high real-time tasks.

[0073] This application dynamically adjusts thread allocation based on real-time load, avoiding resource waste caused by over- or under-allocation. Simultaneously, the conflict prediction module ensures more balanced bus usage, reduces retransmissions, and minimizes unnecessary time consumption. This allows for the support of more concurrent test tasks with the same hardware resources.

[0074] Furthermore, the strategy can be trained offline or online: offline training can be based on existing scheduling logs, while online training allows for real-time model adjustments. The scheduler trained through reinforcement learning can automatically learn efficient resource allocation strategies for different operating scenarios (task size, urgency, etc.), prioritizing the allocation of more computing threads to real-time tasks and increasing parallelism to improve throughput when the overall load is high. In actual simulations, compared to static scheduling strategies, DRL scheduling can significantly improve system throughput and reduce average task latency.

[0075] This application utilizes machine learning or reinforcement learning algorithms to intelligently generate and filter test cases based on historical test data and coverage information, automatically optimizing test paths and coverage blind spots. Graph Neural Networks (GNNs) or related models are used to model the message transmission relationships of multiple nodes on the CAN bus, predicting potential signal conflicts and dynamically adjusting message priorities or transmission timing to reduce bus conflict rates. Deep learning models such as LSTM (Long Short-Term Memory) networks and attention mechanisms are used to analyze collected CAN timing signals, enabling abnormal behavior detection and alarms, and locating abnormal signal sources through attention or feature activation maps output by the model. Deep reinforcement learning methods are employed to optimize task scheduling strategies, allowing the scheduler to adaptively adjust thread / resource allocation based on real-time task queue status and resource utilization, thereby improving overall throughput and resource utilization. The various AI modules work collaboratively to form an intelligent load testing system: firstly, test signals are sent in parallel based on a 1-to-N architecture, and the thread pool retrieves tasks from the task queue for execution; simultaneously, the AI ​​modules monitor task execution in real time and intelligently adjust test plans and system parameters. Through this model, the system achieves a comprehensive improvement in testing efficiency, accuracy, and resource utilization.

[0076] In summary, this application discloses a testing method that generates test cases based on the current test state and historical feedback information. This links test case generation with historical execution results and the current test state, allowing the testing process to dynamically adjust subsequent testing strategies based on conflict events and abnormal behaviors, thereby improving test coverage. Simultaneously, optimizing the test case generation strategy based on execution feedback results enhances the accuracy of the generated test cases, ensuring they match current actual testing requirements. By generating test cases, collecting and analyzing test process data, and optimizing the test case generation strategy, this method achieves intelligent testing. Compared to testing methods relying on fixed scripts or random generation, this method saves manpower and improves testing efficiency.

[0077] refer to Figure 3 It illustrates a testing apparatus 20 provided in an embodiment of this application, the apparatus comprising: The generation module 201 is used to generate test cases based on the current test status and historical feedback information; Execution module 202 is used to execute the test cases and collect execution process data; Feedback module 203 is used to determine execution feedback results based on the execution process data, wherein the execution feedback results include at least one of conflict events and abnormal behavior; The update module 204 is used to update the test case generation strategy based on the execution feedback results until the preset test termination conditions are met.

[0078] Optionally, the generation module includes: The first encoding submodule is used to encode the current test state and the historical feedback information into a state vector; the current test state includes the running state of each executor and at least one item from the list of covered functions; the historical feedback information includes at least one item from historical test case coverage information, historical failure rate and error distribution. The first determining submodule is used to determine the test action based on the state vector; the test action includes at least one of selecting a test path, inserting a signal sequence, and configuring test case parameters; A generation submodule is used to generate test cases based on the test actions.

[0079] Optionally, the feedback module includes: The second encoding submodule is used to segment and encode the execution process data according to a time window to form a time-series input sequence. The detection submodule is used to detect conflict events and abnormal behaviors based on the time-series input sequence, and to use the detected conflict events and abnormal behaviors as the execution feedback results.

[0080] Optionally, the detection submodule includes: The reconstruction unit is used to reconstruct the time-series input sequence using a sequence autoencoder to obtain a reconstructed sequence; the sequence autoencoder is trained using normal operating data. The calculation unit is used to calculate the reconstruction error based on the time-series input sequence and the reconstruction sequence; The judgment unit is used to determine that there is abnormal behavior when the reconstruction error exceeds a first preset threshold.

[0081] Optionally, the detection submodule includes: A classification unit is used to classify the temporal input sequence through an attention mechanism to obtain a classification result; The determining unit is used to determine whether abnormal behavior exists based on the classification results.

[0082] Optionally, the device further includes: The calculation module is used to calculate the influence weight of each historical moment on the classification result; The localization module is used to generate an attention heatmap based on the influence weights to locate the source of the abnormal behavior.

[0083] Optionally, the test case is associated with at least one target executor or message flow, and the execution module includes: A construction submodule is used to construct the target executor or the message flow as graph nodes and the relationships between nodes as graph edges during execution. The aggregation submodule is used to obtain the node features of each graph node and generate a node representation by aggregating the features of neighboring nodes; the node features include at least one of message frequency, load index or message type distribution; The prediction submodule is used to predict the conflict probability of each node based on the node representation; The adjustment submodule is used to adjust the message sending strategy based on the predicted conflict probability.

[0084] Optionally, adjust the submodules, including: The first processing unit is used to reduce the priority of the corresponding message or postpone the sending time when the predicted conflict probability exceeds a second preset threshold. The second processing unit is used to increase the priority of the critical message stream or reserve a transmission time slot when a critical message stream is identified.

[0085] Optionally, update the module, including: The reward calculation submodule is used to calculate the reward value based on the execution feedback result; The update submodule is used to update the test case generation strategy based on the reward value using a reinforcement learning algorithm, and to generate supplementary test cases based on the updated test case generation strategy.

[0086] Optionally, the execution module includes: The acquisition submodule is used to acquire the scheduling status, which includes at least one of the following: task queue length, load of each thread, response latency of each executor, or system computing resource utilization. The second determining submodule is used to determine a scheduling action based on the scheduling state. The scheduling action includes at least one of allocating execution threads to test cases, adjusting the thread pool size, or setting task priorities. The scheduling submodule is used to perform resource allocation for test cases according to the scheduling action.

[0087] In summary, this application discloses a testing method that generates test cases based on the current test state and historical feedback information. This links test case generation with historical execution results and the current test state, allowing the testing process to dynamically adjust subsequent testing strategies based on conflict events and abnormal behaviors, thereby improving test coverage. Simultaneously, optimizing the test case generation strategy based on execution feedback results enhances the accuracy of the generated test cases, ensuring they match current actual testing requirements. By generating test cases, collecting and analyzing test process data, and optimizing the test case generation strategy, this method achieves intelligent testing. Compared to testing methods relying on fixed scripts or random generation, this method saves manpower and improves testing efficiency.

[0088] Reference Figure 4 The electronic device 600 may include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0089] Processing component 602 typically controls the overall operation of electronic device 600, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0090] Memory 604 is used to store various types of data to support the operation of electronic device 600. Examples of this data include instructions for any application or method operating on electronic device 600, contact data, phonebook data, messages, pictures, multimedia, etc. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0091] Power supply component 606 provides power to various components of electronic device 600. Power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 600.

[0092] Multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may not only sense the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a multimedia mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0093] Audio component 610 is used to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) used to receive external audio signals when electronic device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0094] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0095] Sensor assembly 614 includes one or more sensors for providing state assessments of various aspects of electronic device 600. For example, sensor assembly 614 can detect the on / off state of electronic device 600, the relative positioning of components such as the display and keypad of electronic device 600, changes in position of electronic device 600 or a component of electronic device 600, the presence or absence of user contact with electronic device 600, orientation or acceleration / deceleration of electronic device 600, and temperature changes of electronic device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0096] Communication component 616 facilitates wired or wireless communication between electronic device 600 and other devices. Electronic device 600 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0097] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to implement a method provided in the embodiments of this application.

[0098] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, which can be executed by a processor 620 of an electronic device 600 to perform the above-described method. For example, the non-transitory storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0099] Figure 5 A block diagram of an electronic device 700 is shown according to an exemplary embodiment. For example, the electronic device 700 may be provided as a server. (Refer to...) Figure 5 Electronic device 700 includes a processing component 722, which further includes one or more processors, and memory resources represented by memory 732 for storing instructions, such as application programs, that can be executed by the processing component 722. The application programs stored in memory 732 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 722 is configured to execute instructions to perform a method provided in embodiments of this application.

[0100] Electronic device 700 may also include a power supply component 726 configured to perform power management of electronic device 700, a wired or wireless network interface 750 configured to connect electronic device 700 to a network, and an input / output (I / O) interface 758. Electronic device 700 may operate on an operating system stored in memory 732, such as Windows Server™, MacOSX™, Unix™, Linux™, FreeBSD™, or similar.

[0101] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0102] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0103] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A testing method, characterized in that, The method includes: Generate test cases based on the current test status and historical feedback information; Execute the test cases and collect execution process data; The execution feedback result is determined based on the execution process data, and the execution feedback result includes at least one of conflict events and abnormal behavior; Based on the execution feedback results, update the test case generation strategy until the preset test termination conditions are met.

2. The method according to claim 1, characterized in that, The step of generating test cases based on the current test status and historical feedback information includes: The current test state and the historical feedback information are encoded into a state vector; the current test state includes the running state of each executor and at least one item from the list of covered functions; the historical feedback information includes at least one item from historical test case coverage information, historical failure rate, and error distribution. Based on the state vector, the test action is determined; the test action includes at least one of selecting a test path, inserting a signal sequence, and configuring test case parameters. Test cases are generated based on the test actions.

3. The method according to claim 1, characterized in that, Determining the execution feedback result based on the execution process data includes: The execution process data is segmented and encoded according to time windows to form a time-series input sequence; Based on the time-series input sequence, conflict events and abnormal behaviors are detected, and the detected conflict events and abnormal behaviors are used as the execution feedback results.

4. The method according to claim 3, characterized in that, Based on the aforementioned time-series input sequence, abnormal behavior is detected, including: The time-series input sequence is reconstructed using an autoencoder to obtain a reconstructed sequence; the autoencoder is trained using normal operating data. The reconstruction error is calculated based on the time-series input sequence and the reconstructed sequence; When the reconstruction error exceeds a first preset threshold, it is determined that there is abnormal behavior.

5. The method according to claim 3, characterized in that, Based on the aforementioned time-series input sequence, abnormal behavior is detected, including: The time-series input sequence is classified using an attention mechanism to obtain the classification result; The existence of abnormal behavior is determined based on the classification results.

6. The method according to claim 5, characterized in that, The method further includes: Calculate the influence weight of each historical moment on the classification result; An attention heatmap is generated based on the influence weights to locate the source of the abnormal behavior.

7. The method according to claim 1, characterized in that, The test cases are associated with at least one target executor or message flow, and the execution of the test cases includes: During execution, the target executor or the message flow is constructed as nodes of a graph, and the relationships between nodes are constructed as edges of the graph; Obtain the node features of each node, and generate a node representation by aggregating the features of neighboring nodes; the node features include at least one of message frequency, load index, or message type distribution; Based on the node representation, predict the conflict probability of each node; Adjust the message sending strategy based on the predicted conflict probability.

8. The method according to claim 7, characterized in that, The step of adjusting the message sending strategy based on the predicted conflict probability includes: If the predicted probability of conflict exceeds a second preset threshold, the priority of the corresponding message is reduced or the sending time is postponed. If a critical message flow is identified, its priority is increased or a transmission time slot is reserved.

9. The method according to claim 1, characterized in that, Based on the execution feedback results, update the test case generation strategy, including: Calculate the reward value based on the execution feedback results; The test case generation strategy is updated based on the reward value using a reinforcement learning algorithm, and supplementary test cases are generated based on the updated test case generation strategy.

10. The method according to claim 1, characterized in that, Executing the test cases includes: Obtain the scheduling status, which includes at least one of the following: task queue length, thread load, executor response latency, or system computing resource utilization. Based on the scheduling state, a scheduling action is determined, which includes at least one of allocating execution threads to test cases, adjusting the thread pool size, or setting task priorities. Resource allocation for test cases is performed according to the scheduling action.

11. A testing apparatus, characterized in that, The device includes: The generation module is used to generate test cases based on the current test status and historical feedback information; The execution module is used to execute the test cases and collect execution process data; The feedback module is used to determine the execution feedback result based on the execution process data, wherein the execution feedback result includes at least one of conflict events and abnormal behavior; The update module is used to update the test case generation strategy based on the execution feedback results until the preset test termination conditions are met.

12. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the method as described in any one of claims 1 to 10.

Images