A Simulink compiler testing method with diverse configurations based on reinforcement learning.
By updating the Simulink compiler testing method through reinforcement learning-based feature representation and vectorization, diverse CPS models are generated, solving the problem of insufficient model diversity in existing technologies, improving bug detection performance, and ensuring the stability of CPS development tools.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DALIAN MARITIME UNIVERSITY
- Filing Date
- 2022-12-19
- Publication Date
- 2026-05-26
Smart Images
Figure CN116107876B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of software testing, and more particularly to a Simulink compiler testing method based on reinforcement learning to achieve diverse configurations. Background Technology
[0002] Cyber-Physical System (CPS) development tools, such as MathsWorks Simulink, are the basic platform for developers to design and analyze CPS before developing embedded code [1], [2], [3]. In CPS development tools, developers design CPS models, a block diagram with blocks and connections, to simulate the behavior of CPS. As an industry standard, Simulink has been widely used by developers to design CPS models for many safety-critical applications, such as aerospace and healthcare. Simulink analyzes and compiles the CPS model with its compilation system (i.e., the Simulink compiler), and then generates embedded code to be deployed to the target CPS application. However, the Simulink compiler may contain bugs. These bugs can inject unexpected behavior into the CPS model, seriously threatening the correctness and security of the target CPS application.
[0003] Therefore, testing and discovering bugs in them is crucial. Several Simulink compiler testing methods have been proposed to find errors in the Simulink compiler. These methods are based on testing a large number of CPS models generated by a CPS model generator (e.g., SLFORGE). The CPS models are compiled differently with the Simulink compiler, with the expectation of obtaining the same output. By analyzing the inconsistencies in the output, bugs in the Simulink compiler can be discovered based on the idea of differential testing. Furthermore, these CPS models can be mutated to generate equivalent CPS model variants for Equivalent Modular Input (EMI) testing.
[0004] While automatically generated CPS models are important, CPS model generators rely heavily on appropriate configurations (e.g., the probability of generating a certain type of block) to produce CPS models that trigger bugs, as the same configuration tends to generate CPS models with similar block distributions and trigger duplicate bugs. In the realm of general compiler testing (such as GCC and LLVM), two approaches have been proposed to generate different programs through configuration diversification: swarm testing and history-based testing. These methods randomly configure or use historical bug triggers to infer configurations to generate different programs, thereby exploring more of the compiler input space.
[0005] However, due to issues with CPS model representation and configuration learning, swarm testing and history-based testing are not effective at testing the Simulink compiler. This is because these methods generate programs by controlling the probability of generating different native program features (i.e., return statements). However, CPS model generators do not support these features. They also ignore the value of global CPS model features (e.g., CPS model architecture) in guiding CPS model generation. Therefore, it is crucial to mine effective features to represent CPS models in order to measure their diversity. History-based testing relies on thousands of programs that trigger bugs in older compiler versions (generated by the program generator Csmith) to infer configuration. However, as mature commercial software, a CPS model generator cannot possibly find thousands of bugs in the Simulink compiler for configuration learning. Summary of the Invention
[0006] To address the problems existing in the prior art, this invention discloses a Simulink compiler testing method based on reinforcement learning to achieve diverse configurations, specifically including the following steps:
[0007] S1: Collect the configuration file of the CPS model generator and a set of initialization states of the CPS model M0;
[0008] S2: Update the configuration of the CPS model generator and generate a new set of CPS models M1;
[0009] S3: Input CPS model M0 and CPS model M1 into the feature vectorization component and vectorize them into feature vectors v0 and v1 respectively;
[0010] S4: Input the feature vectors into the reinforcement learning component, where the distance between feature vectors v0 and v1 is compared and used as the reward for reinforcement learning.
[0011] S5: The reinforcement learning component records the reward and uses the reward and feature vector as the learned knowledge;
[0012] S6: Based on the learned knowledge, the reinforcement learning component recommends a new configuration and iterates through S2-S6. In S4, the distance between v1 and the CPS model generated in the previous iteration is also compared.
[0013] S7: At the end of each iteration, the newly generated CPS model is used to test the Simulink compiler through differential testing. Differential testing is used to compare the execution trajectory of the CPS model in different simulation modes. If the execution trajectory is the same, the error information is recorded to complete the testing process of the CPS model. If there is a difference, the difference information is recorded and it is considered as a bug discovery.
[0014] Furthermore, when updating the CPS generator configuration: the CPS model generator configuration is updated using reinforcement learning components, and CPS models with diversity are generated using the new CPS configuration.
[0015] Furthermore, when representing the CPS model as a feature vector, the generated CPS model is input into a feature vectorization component, which is based on a set of manually mined bug-triggered features, thereby representing the CPS model as a feature vector used to measure the diversity of the CPS model.
[0016] Furthermore, when training reinforcement learning components: use reinforcement learning components to learn the rewards that will be obtained when certain configuration updates are made, thereby obtaining a diverse CPS model by obtaining the largest possible rewards.
[0017] Furthermore, when using reinforcement learning components to recommend configurations: new configurations are recommended using the knowledge learned by the reinforcement learning components, and it is expected that the generator with the new configuration will generate CPS models with different distributions of bug triggering characteristics.
[0018] Furthermore, during differential testing: the execution results of the CPS model in normal simulation mode and accelerated simulation mode are compared respectively. If the same CPS model has different execution results in different simulation modes, it is considered that a bug has been found; otherwise, no bug has been found.
[0019] By employing the aforementioned technical solutions, this invention provides a Simulink compiler testing method based on reinforcement learning for diversified configurations. This method introduces feature representation and reinforcement learning during test case code generation, addressing the challenges of model representation and insufficient bug-triggering models in existing Simulink compiler testing techniques. It increases model diversity and improves bug detection performance. Through the use of reinforcement learning and feature representation, the impact of different block distributions on the CPS model and the value of global CPS model architecture features in guiding CPS model generation are effectively explored. This ensures the stability of CPS development tools in chip design and high-end manufacturing. Compared to previous technologies, our method has a broader coverage and higher technological maturity, maintaining Simulink stability to a certain extent. This provides guidance for Simulink testing and CPS development toolchain testing. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 This is a flowchart of the method of the present invention;
[0022] Figure 2 This is a schematic diagram of the CPS model features collected in this invention;
[0023] Table 1 shows the pseudocode of the reinforcement learning component in this invention; Detailed Implementation
[0024] To make the technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention:
[0025] like Figure 1 The method shown is a Simulink compiler testing method based on reinforcement learning to achieve diverse configurations, which includes the following steps:
[0026] Step 1: Collect the configuration file of the CPS generator and a set of CPS models M0 in initial state: Collect the initial configuration of the CPS model generator and generate a set of CPS models using a set of configurations with random options. Vectorize this set of models as the initial state of the reinforcement learning components.
[0027] Step 2: Update the CPS model generator configuration and generate a new set of CPS models M1: Modify the CPS model generator configuration to update the CPS model generator configuration file, and under the new configuration, use the CPS model generator to generate a new set of CPS models M1.
[0028] Step 3: Input CPS models M0 and M1 into the feature vector component, vectorizing them into feature vectors v0 and v1 respectively, thus realizing the feature representation of the model: In this component, the CPS model is represented as a feature vector used to measure the diversity of CPS models. Table 2 shows the features used in the component, including feature categories and subcategories, the number of features, descriptions, intuitions for mining these features, and value types. We construct these features by analyzing the semantics of the block and the features of some collected bug-inducing CPS models. These properties can reflect the ability of the CPS model to trigger bugs in the Simulink compiler. If CPS models with different distributions of these properties are generated, then more bug-prone spaces can be explored in the Simulink compiler.
[0029] feature:
[0030] This method constructs two types of features: local features and global features. Local features capture the characteristics of individual blocks, while global features analyze blocks in the CPS model as a whole. Specifically, such as... Figure 2 As shown, we have four subcategories of local features, including features related to inputs and outputs, ports and subsystems, data types, and operations. Within each subcategory, there are multiple features. Each feature represents the frequency of a block in the CPS model. For example, there are 38 input and output-related features that calculate the frequencies of blocks such as displays, clocks, sine waves, constants, ranges, terminators, and output ports in the CPS model. These blocks can be used as features because they frequently interact with the external world. External uncertainties may explore unexpected compilation branches in the Simulink compiler. Utilizing the different distributions of these blocks to generate CPS models allows for thorough testing of the Simulink compiler. Figure 2 The third and fourth columns provide details for each subcategory. As shown in the figure, a total of 113 local features were constructed.
[0031] Vectorization:
[0032] After successfully constructing the features, each CPS model needs to be represented as a feature vector to guide the generation of a group of CPS models with different feature distributions. These CPS models can then be comprehensively tested against the input space of the Simulink compiler.
[0033] We use M = {m1, m2, ..., m} i ,...,m n Let} represent a set of CPS models generated by the CPS model generator using certain configuration option values, where n is the number of CPS models in the set, m iThis is the i-th CPS model. The corresponding feature vector of M is represented as V = {v1, v2, ..., v...} i ,...,v n We define v i Let v be the feature vector of the i-th CPS model. i =[x i1 ,x i2 ,...,xi j ,...,x ir ], where r is the number of features, x ij It is the value of the j-th feature. i All values are normalized using the min-max normalization method. Specifically, we use x ij * represents x ij The original values of the feature vector v (e.g., the frequency of a block), where 1 ≤ i ≤ n, 1 ≤ j ≤ r. i The value x in ij *Normalized to:
[0034]
[0035] The Feature Vectorization component addresses the challenge of representing CPS models in Simulink compiler testing by representing the CPS model as a set of feature vectors.
[0036] Step 4: Input the feature vectors into the reinforcement learning component. Compare the distances between feature vectors and use these distances as the reward for reinforcement learning. Use reinforcement learning to learn the reward to be obtained when certain configuration updates are made.
[0037] This component relies on two basic machine learning techniques:
[0038] Reinforcement learning:
[0039] Reinforcement learning
[18] is a class of machine learning algorithms that take actions in an environment to maximize the concept of cumulative reward. There are four basic concepts in reinforcement learning.
[0040] State (S) is the state of the environment. For example, in a chess game, state can be the current position of a piece on the chessboard.
[0041] An action (A) is an operation performed on a state, such as moving a piece on a chessboard.
[0042] Reward (R) is the immediate reward obtained after taking an action on the current state of the environment (S×A→R). For example, if function f represents the reward obtained immediately after taking an action on the current state of the environment... t If the probability of winning the game is calculated, an immediate reward of r is given. t =f(s)t+1 )-f(s), where s t+1 =a t (s t ), a t It is in s t The actions performed above.
[0043] Cumulative return (G) is the total return of a series of actions. Where γ is a discount factor, used to reduce the impact of a certain behavior on future returns.
[0044] The goal of reinforcement learning is to train a model (without labeling input / output pairs) to learn how to obtain cumulative rewards when performing certain actions in a given state, based on the "feedback" of immediate rewards. This model is then used to predict the action that should be taken in a given state.
[0045] Double Deep Q-Network(DDQN):
[0046] DDQN[9] is one of the most advanced algorithms for training reinforcement learning models. The kernel of DDQN consists of two deep neural networks with the same structure, denoted as Q... φ and Where φ and This represents the parameters of the two networks (i.e., the weights of edges and nodes). Q φ Used for training and predicting actions, Then for Q φ Rectification is performed to reduce fluctuations during training. DDQN aims to achieve Q... φ Optimize φ to accurately predict the cumulative reward for each action.
[0047] The execution of DDQN is iterative. Initially, φ is randomly initialized, and... In each iteration t, based on the current state s t Choose an action based on the ε-greedy strategy.
[0048]
[0049] Where A represents that it is possible in s t All actions taken. DDQN has a probability ε of randomly selecting an action, and through Q... φ The probability of determining an action is 1-ε. For the latter, DDQN uses ε to predict the cumulative reward for each action a∈A and selects the action a with the highest predicted cumulative reward. t . when a t When determined, DDQN calculates from s based on a predefined function f. t Move to s t+1Actual instant reward r t DDQN uses the experience pool P to... t Save as tuple t ,a t ,s t+1 ,r t >
[0050] In each iteration, DDQN selects a subset of tuples in P to optimize φ. Specifically, DDQN assumes that the cumulative reward is only affected by the two most recent instantaneous rewards, i.e., g. t =r t +γr t+1 Given a tuple t ,a t ,s t+1 ,r t The loss function for this tuple is:
[0051] loss(φ)=(g t -Q φ (s t ,a t )) 2 (2)
[0052] Among them, Q φ (s t ,a t ) is Q φ The predicted cumulative return. Here, g t Defined as:
[0053]
[0054] Among them, a t+1 =argmaxQ φ (s t+1 ,a t This means that DDQN first uses Q. φ To estimate state s t+1 Best action a t+1 Then, a t+1 fed to To estimate the immediate reward r t+1 Based on the loss() of each tuple, Q is updated using gradient descent in each iteration. φ The parameter φ is used to better predict g. t .at the same time, In By Q φ Copy the parameter φ in the middle. In the middle, it is updated every C iterations. because The update frequency is lower than Q. φ , It can improve the stability of the optimization process without being affected by iterative fluctuations.
[0055] In this method, the reinforcement learning component comprises the following parts:
[0056] State: This refers to the value of the configuration option. Given an option value combination, we can generate a set of CPS models M and compute their feature vectors V;
[0057] Action: This refers to an operation on the configuration option (e.g., increasing the value of the configuration option by 0.02);
[0058] Reward: The newly generated CPS model group M t+1 Compared with the previously generated CPS model group M1,...,M t The minimum distance between them.
[0059] Cumulative earnings: g t =r t +γr t+1 ;
[0060] The feature vector of the CPS model can be represented by V: V = {v1, v2, ..., v...} i ,...,v n}
[0061] The core of the CPS model can be represented as: in It is the average value of the j-th feature in V.
[0062] Based on vectorized models, the distance between two models can be represented by the Manhattan distance as:
[0063]
[0064] The pseudocode for the reinforcement learning components is shown in Table 1:
[0065]
[0066]
[0067] Table 1
[0068] The specific process is described as follows:
[0069] First, initialize a collection. To save the generated CPS model and an experience pool P (lines 1-2). For N ep Perform iterations. In each episode, randomly initialize the network Q. φ(Line 4) and the configuration of the CPS model generator O (Line 5). Record generates a set of CPS models with initial configuration and adds the corresponding feature vectors V1 to the set V (Lines 6-9).
[0070] After initialization, this component iteratively generates the CPS model. In each iteration t, based on network Q... φ (Line 11) Determines the action on configuration option O. We define three actions A on an option: increasing the value by 0.02, decreasing the value by 0.02, and keeping the value unchanged. The reinforcement learning component uses an ε-greedy policy (Equation 1) to select an action. Specifically, the component generates all combinations of actions on all configuration options. Then, it utilizes the network Q... φ For V-based t Predict the cumulative reward for each action combination. Select the action combination with the highest predicted cumulative reward. Select action a. t Then, update the configuration on line 12.
[0071] Step 5: The reinforcement learning component records the acquired rewards and uses the rewards and feature vectors as learned knowledge. We will... t As "experience," it reflects the situation when the eigenvector is derived from V. t Change to V t+1 At that time, through a t The actual, immediate reward that an action can yield. We store this "experience" as a tuple in the experience pool p. <V t ,a t V t+1 ,r t >
[0072] Step 6: Based on the learned knowledge, the reinforcement learning component suggests new configurations and iterates through steps 2-6. These configurations are often those that can obtain high cumulative rewards. The component then updates the configurations, generating CPS models with different bug-triggering characteristic distributions. In each iteration, the reinforcement learning component learns that when certain actions (i.e., changes) are performed on the configuration, the cumulative reward can be obtained using the CPS model set generated in the previous iteration and the corresponding reward. Utilizing this knowledge, the reinforcement learning component intelligently updates the configurations in each iteration to generate new CPS models and obtain high cumulative rewards.
[0073] Step 7: Perform Differential Testing: At the end of each loop, use the newly generated CPS model to test the Simulink compiler using differential testing. Differential testing compares different simulation modes of the CPS model, namely the normal simulation mode and the accelerated simulation mode. By design, the trajectory lines in different simulation modes should be equivalent, meaning they are expected to produce the same result values. If the results are different, a Simulink compiler error may have been found. If the execution results of the two simulation modes differ, a bug is considered found. Otherwise, no bug is found. We will compile the test cases that produce bugs and submit them to the official documentation. More formally, for the program Base and its corresponding simulation modes c1, c2, ..., c... n And given that the program input parameter m is based on our understanding of programming language notation [.], we expect to produce the same execution result, namely:
[0074] [Base(m)]=[C1(m)]=[C2(m)]=[C n (m)]
[0075] If a program produces different results when compiled and run under different compiler configurations, we consider that a bug in the toolchain has been found.
[0076] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A Simulink compiler testing method based on reinforcement learning to achieve diverse configurations, characterized in that... include: S1: Collect the configuration file of the CPS model generator and a set of initialization states of the CPS model M0; S2: Update the configuration of the CPS model generator and generate a new set of CPS models M1; S3: Input CPS model M0 and CPS model M1 into the feature vectorization component and vectorize them into feature vectors v0 and v1 respectively; S4: Input the feature vectors into the reinforcement learning component, where the distance between feature vectors v0 and v1 is compared and used as the reward for reinforcement learning. S5: The reinforcement learning component records the reward and uses the reward and feature vector as the learned knowledge; S6: Based on the learned knowledge, the reinforcement learning component recommends a new configuration and iterates through S2-S6. In S4, the distance between v1 and the CPS model generated in the previous iteration is also compared. S7: At the end of each iteration, the newly generated CPS model is used to test the Simulink compiler through differential testing. Differential testing is used to compare the execution trajectory of the CPS model in different simulation modes. If the execution trajectory is the same, the error information is recorded to complete the testing process of the CPS model. If there is a difference, the difference information is recorded and it is considered that a bug has been found. When representing features of the CPS model: The generated CPS model is input into the feature vectorization component, which is based on a set of manually mined bug-triggered features, thereby representing the CPS model as a feature vector used to measure the diversity of the CPS model.
2. The method according to claim 1, characterized in that: Update the configuration of the CPS model generator using reinforcement learning components, and generate CPS models with diversity using the new CPS configuration.
3. The method according to claim 1, characterized in that: Use reinforcement learning components to learn the rewards that will be obtained when certain configuration updates are made, and obtain diverse CPS models by obtaining the largest possible rewards.
4. The method according to claim 1, characterized in that: By leveraging the knowledge learned from reinforcement learning components, new configurations are recommended. It is expected that the generator with the new configuration will generate CPS models with different distributions of bug triggering characteristics.
5. The method according to claim 1, characterized in that: Compare the execution results of the CPS model in normal simulation mode and accelerated simulation mode respectively. If the same CPS model has different execution results in different simulation modes, it is considered that a bug has been found; otherwise, no bug has been found.