A data augmentation method and system for embedded system storage function testing

Abstract

The present invention relates to the field of testing of embedded system storage design, and is a data augmentation method for embedded system storage function testing. The method comprises the following steps: generating a system specification formula data set according to an input system specification document description; training a sequence-to-sequence neural network using the system specification formula data set; initializing a sampling formula using the trained sequence-to-sequence neural network, and gradually sampling to obtain a copy of the system specification formula data set; fine-tuning a pre-trained neural network for detection using the copy of the system specification formula data set, and testing the fine-tuned neural network for detection using the system specification formula data set to verify the validity of the generated system specification formula and whether the embedded system has realized the corresponding storage function. The present invention solves the problem that it is difficult to use a neural network to directly learn the satisfiability bias for a small number of embedded system linear temporal logic formulas with unknown distributions.

Description

Technical Field

[0001] The invention relates to the field of testing of embedded system storage design, in particular to a data augmentation method and system for testing storage function of embedded system. Background Art

[0002] The testing of embedded system storage function design is a crucial link, which ensures the stability and reliability of embedded systems in practical applications. Embedded systems are usually designed to perform specific tasks, such as controlling machines, collecting data, performing communications, etc. They are widely used in industrial control, automotive electronics, medical equipment, smart homes and other fields. Testing is a necessary means to ensure that the memory design meets the system performance and reliability requirements.

[0003] It is very necessary to test the storage function design of embedded systems because it helps to ensure that the system can operate in the expected manner and meet specific requirements and functions. Through specification test verification, it can be ensured that the quality of the system meets the requirements, thereby reducing the possibility of system errors and defects and improving the reliability and stability of the system. Specification test verification helps to ensure that the system can meet the needs and expectations of users and improve user satisfaction. The test of the storage function design of embedded systems also contributes to the continuous improvement of the system. The results of test verification can provide directions for improvement and optimization for subsequent versions of the system. The general test method for the storage function design of embedded systems is to convert the system function specification described in natural language into a linear temporal logic formula, and then call the linear temporal logic satisfiability solver to verify the system function.

[0004] Most existing methods are based on expert judgment or post-use analysis of system logs and user logs to verify the storage function of embedded systems. Existing methods are highly subjective, costly, and require use before correction, so some minor problems can lead to huge economic losses. Summary of the invention

[0005] In order to solve the problems existing in the prior art, the present invention provides a data augmentation method and system for embedded system storage function testing, which generates more linear temporal logic formulas that meet the embedded system specifications according to unknown distributions, and these formulas are used to allow a neural network to learn the satisfiability bias of the embedded system, and then obtain a satisfiability neural network solver that can be used in the field of embedded system specification description verification and solution; using the satisfiability neural network solver, the storage function design of the embedded system can be tested quickly, and the problem that it is difficult to use a neural network to directly learn the satisfiability bias for a small number of embedded system linear temporal logic formulas with unknown distributions is solved.

[0006] The method of the present invention is implemented by the following technical solution: A data augmentation method for embedded system storage function testing, comprising the following steps:

[0007] S1. Generate a system specification formula data set based on the input system specification document description;

[0008] S2, train a sequence-to-sequence neural network using the system specification formula dataset;

[0009] S3, using the trained sequence-to-sequence neural network, initialize the sampling formula, and gradually sample to obtain a copy of the system specification formula data set. The formulas in the copy of the system specification formula data set have the same distribution as the formulas in the system specification formula data set in step S1, but the two do not overlap, that is, the formulas in the copy of the system specification formula data set will not appear in the system specification data set in step S1;

[0010] S4. Fine-tune the pre-trained neural network for detection using the copy of the system specification formula data set, and test the fine-tuned neural network for detection using the system specification formula data set to verify the validity of the generated system specification formula and whether the embedded system implements the corresponding storage function;

[0011] Step S1 includes:

[0012] S11, according to the input system specification document description, identifying the atomic event set involved in the system specification document, replacing the atomic events in the system specification document description with atomic proposition symbols, thereby defining an atomic proposition set according to the atomic event set;

[0013] S12. According to the description of the system specification document, an event set is defined using an atomic proposition set and a temporal operator set, where each event in the event set is presented as a linear temporal logic formula composed of an atomic proposition and a temporal operator;

[0014] Among them, events are used to describe the specifications that the hardware buffer must meet, including atomic events and non-atomic events; atomic events are basic events that cannot be decomposed any further, and non-atomic events are a collection of single atomic events or a combination of multiple atomic events; atomic propositions are either true or false at a certain moment, and the event referred to by the atomic proposition cannot be equivalent to the combinational logic of other atomic propositions and timing operators;

[0015] When describing system specifications, events are presented in the form of linear temporal logic formulas consisting of atomic propositions and timing operators; timing operators are used to describe changes in system states, including describing the system's current state, next state, and future state.

[0016] The system of the present invention is implemented by the following technical solution: A data augmentation system for embedded system storage function testing, comprising the following modules:

[0017] A data set generation module, used to generate a system specification formula data set according to an input system specification document description;

[0018] Neural network training module, which trains sequence-to-sequence neural networks using system specification formula datasets;

[0019] The dataset replica sampling module uses the trained sequence-to-sequence neural network to initialize the sampling formula and gradually samples to obtain a replica of the system specification formula dataset;

[0020] A testing module, which uses a copy of the system specification formula data set to fine-tune the pre-trained neural network for detection, and uses the system specification formula data set to test the fine-tuned neural network for detection to verify the validity of the generated system specification formula and whether the embedded system implements the corresponding storage function;

[0021] The process of generating a system standard formula data set by the data set generation module includes:

[0022] According to the input system specification document description, an atomic event set involved in the system specification document is identified, and the atomic events in the system specification document description are replaced with atomic proposition symbols, so as to define an atomic proposition set according to the atomic event set;

[0023] According to the system specification document description, an event set is defined using an atomic proposition set and a temporal operator set. Each event in the event set is presented as a linear temporal logic formula composed of an atomic proposition and a temporal operator.

[0024] Among them, events are used to describe the specifications that the hardware buffer must meet, including atomic events and non-atomic events; atomic events are basic events that cannot be decomposed any further, and non-atomic events are a collection of single atomic events or a combination of multiple atomic events; atomic propositions are either true or false at a certain moment, and the event referred to by the atomic proposition cannot be equivalent to the combinational logic of other atomic propositions and timing operators;

[0025] When describing system specifications, events are presented in the form of linear temporal logic formulas consisting of atomic propositions and timing operators; timing operators are used to describe changes in system states, including describing the system's current state, next state, and future state.

[0026] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0027] The data augmentation method of the present invention uses a sequence-to-sequence abstract description process for a small number of data sets with unknown distributions, uses a Transformer neural network to learn the distribution of the data set, and iterates sampling based on a conditional probability model to obtain a new formula for the current distribution. The existing technology for formula generation mainly focuses on traditional sequence generation and graph generation, and there is no research on tree-based formula generation for this task; the present invention uses fine-tuning of the task of satisfiability solution to evaluate the effectiveness of the generated formula. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is an overall flow chart of a data augmentation method for embedded system storage function testing in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The present invention studies and develops a neural network-based data augmentation method to implement the test of embedded system storage function design, and models this task as a linear temporal logic satisfiability checking problem.

[0030] The essence of linear temporal logic satisfiability checking is to check the satisfiability of a given linear temporal logic formula. The development of a practical neural network-based linear temporal logic satisfiability checker relies heavily on induction of data with the same deviations as the test formula. Unfortunately, in the test scenario of embedded system storage function design, the availability of data sets is still limited. Usually, an embedded system only has hundreds of formally described linear temporal logic formula data sets, which is far from meeting the application of large-scale neural networks with large amounts of training data.

[0031] In addition, since it takes expertise to accurately characterize system properties in embedded systems using linear temporal logic formulas, it may be daunting to collect enough high-quality formulas for specific industrial scenarios. Therefore, the data sets of embedded systems are usually small and their distribution is difficult to determine. Data augmentation provides a solution to this situation, such as modifying existing data according to certain rules to obtain noisy copies. However, the formulation of such rules requires expertise, and it is difficult to grasp the similarity between the copies and the original data sets. Excessive similarity and difference will lead to more serious overfitting and model non-convergence.

[0032] The present invention realizes a technical solution for data augmentation, and its application scenario has the characteristics of small data sets and unknown distribution, and existing data augmentation rules cannot be applied.

[0033] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0034] Example

[0035] An important component of the embedded storage system is the hardware buffer, the main function of which is to cache the content sent by n transmitters and then forward it to m receivers. The embedded storage system ensures the reliable implementation of functions by ensuring communication with receivers, communication with transmitters and multiplexing. In the hardware buffer of the embedded storage system, a total of 13 guarantees are required, and a total of about 200 system specification formulas are generated. For example, it is necessary to ensure that the request sent to the buffer by the first sender must be confirmed. In this embodiment, the above guarantees reflect the specifications that the hardware buffer must meet, also called events, and are described using linear temporal logic formulas.

[0036] Before introducing the technical solution of this embodiment in detail, the relevant terms involved are introduced as follows:

[0037] 1. Event: used to describe the specifications that the embedded storage system must meet. Events are divided into atomic events and non-atomic events.

[0038] An atomic event refers to a basic event that cannot be decomposed any further. It is a single element or the simplest event in the sample space. An atomic event is usually a basic result that cannot be decomposed any further. For example, in the hardware buffer in this embodiment, there is an atomic event: the request sent by the first sender to the buffer arrives. A non-atomic event is an event composed of one or more atomic events.

[0039] In order to simplify the description, the present invention uses variables to describe atomic events, which are also called atomic propositions. The atomic propositions are either true or false at a certain moment, and the events referred to by the atomic propositions cannot be equivalent to the combinational logic of other atomic propositions and timing operators.

[0040] The present invention will subsequently use events to refer to non-atomic events, and use atomic propositions to refer to atomic events. Non-atomic events can be a collection of single atomic events or a combination of multiple atomic events, for example: the request sent by the first sender to the buffer must be confirmed. This event is composed of a combination of multiple atomic events, one of which is "the request sent by the first sender to the buffer arrives".

[0041] When describing system specifications, events are presented in the form of linear temporal logic formulas consisting of atomic propositions and temporal operators.

[0042] In addition to specifically referring to atomic propositions, the present invention also sets a placeholder for a general atomic proposition in the linear temporal logic formula. The position where the atomic proposition placeholder appears in the formula indicates that this position is an atomic proposition, but does not specifically specify which atomic proposition in the atomic proposition set it is.

[0043] 2. Time series operators: used to describe changes in system states, including the current state, next state, and future state of the system. Time series operators include X (the next state is established), U (until a certain state is established), F (a certain state is established in the future), G (all states are established in the future), (No, this sub-formula does not hold in the current state), (or, in the current state, the first sub-formula or the second sub-formula is established), (implies that if the current sub-formula is true, then the following sub-formula is also true) and (And, in the current state, both the first subformula and the second subformula are valid.) The time series operators involved in this embodiment are divided into two categories according to the number of connected subformulas. One category is a one-dimensional time series operator (X, F, G, ), only one sub-formula is connected; the other type is the binary time series operator (U, , ), connecting the two sub-formulas.

[0044] In the data augmentation problem for embedded system storage function design test in this embodiment, the input of the problem is a given standard formula data set, and the output is whether the system function is satisfied. When using a neural network for data augmentation, it is necessary to train the neural network with a known data set, and the trained neural network can generate a new standard formula only after learning the distribution of the data set.

[0045] like Figure 1 As shown, the present embodiment provides a data augmentation method for embedded system storage function testing, comprising the following steps:

[0046] S1. Generate a system specification formula data set based on the input system specification document description.

[0047] Step S1 includes the following steps:

[0048] S11. According to the input system specification document description, identify the atomic event set involved in the system specification document, replace the atomic events in the system specification document description with atomic proposition symbols, and thus define an atomic proposition set based on the atomic event set.

[0049] S12. According to the system specification document, use the atomic proposition set and the timing operator set (including timing operators X, U, F, G, , , ) defines an event set, each event in the event set is presented as a linear temporal logic formula composed of atomic propositions and temporal operators.

[0050] In this embodiment, events are used to describe the specifications that the hardware buffer must meet, and events include atomic events and non-atomic events. For example, one of the non-atomic events is: "The request sent by the first sender to the buffer must be confirmed." The atomic event set consists of multiple atomic events.

[0051] This step defines the set of atomic propositions as , where the first atomic proposition symbol The atomic event representing the arrival of a request from the first sender to the buffer, the second atomic proposition symbol The atomic event representing the confirmation of the request sent by the first sender to the buffer, the third atomic proposition symbol The atomic event representing the arrival of the request sent by the second sender to the buffer, the fourth atomic proposition symbol The atomic event representing the confirmation of the request sent by the second sender to the buffer. It should be noted that the atomic proposition set here is the set of atomic propositions related to the entire hardware buffer, and an event may involve one or more atomic propositions.

[0052] Each event can be represented by a linear temporal logic formula. For example, the event "the first sender's request to the buffer must be confirmed" can be represented by a linear temporal logic formula To express it, the formula represents: "When the first atomic proposition Established (i.e., the request sent by the first sender to the buffer arrives), a certain state F in the future is established, i.e., the second atomic proposition The request sent by the first sender to the buffer is confirmed, and this process is established in all future states (G).

[0053] In order to reduce the time of solving the linear temporal logic formula in the later stage and improve the solving efficiency, this embodiment simplifies the temporal operator set, and needs to solve the problems except X, U, F, G, , , These time series operators other than the time series operators are equivalently converted. For example, in this embodiment, according to the implication ( ) and or ( ),no( ), the linear temporal logic formula of the above event Equivalent conversion to formula The expression of the linear temporal logic formula formed after equivalent transformation is a tree data structure in semantics, and its prefix expression is , which is the pre-order order: root node - left subtree - right subtree.

[0054] Assume that after processing in this step, a total of 200 system specification formulas are obtained, and the system specification formula data set D is obtained.

[0055] S2. Training a sequence-to-sequence neural network using the system specification formula dataset.

[0056] In this embodiment, the sequence-to-sequence neural network includes a sequence-to-sequence Transformer decoding model TD1 for predicting a system specification formula and a sequence-to-sequence Transformer decoding model TD2 for predicting an instantiation of a system specification formula.

[0057] Since the sequence-to-sequence neural network cannot be used to retain long-distance information, that is, when the length of the system specification formula is too long, the neural network will lose the long-distance dependency information. Therefore, the formula that is too long cannot be used directly to train the sequence-to-sequence neural network. This embodiment uses a moving window strategy to divide the system specification formula and generate a local formula data set. While retaining the long-distance dependency information of the system specification, the size of the training set is increased, which is conducive to the sequence-to-sequence neural network to better learn the distribution characteristics and avoid overfitting of the sequence-to-sequence neural network.

[0058] Step S2 includes the following steps:

[0059] S21. Slice the system standard formula data set to obtain the sliced ​​local formulas and form a local formula data set with a fixed length of 300.

[0060] In this embodiment, the formulas in the system standard formula data set D are sliced ​​with a maximum length of 5 to obtain a local formula data set. These local formulas in the local formula data set represent the standard requirements that the embedded system needs to meet under a preset state length. For example, for formula After slicing, we can get two local formulas and The former means that the request sent by the first sender to the buffer will arrive at a certain time in the future; the latter means that at a certain time in the future, if the request arrives, then a certain status request after this time will be confirmed.

[0061] S22. Use the local formula dataset to train a sequence-to-sequence Transformer decoding model TD1 for predicting the system's canonical formula.

[0062] The system specification formula in this step is not instantiated, that is, the atomic proposition in this step is an abstract atomic proposition placeholder, which represents all possible atomic propositions defined in step S11.

[0063] S23. Use the local formula dataset to train a sequence-to-sequence Transformer decoding model TD2 for predicting system specification formula instantiations.

[0064] The system specification formula of this step has been instantiated, and the atomic proposition in this step is one of all possible atomic propositions defined in step S11.

[0065] The sequence-to-sequence neural network in step S2 learns the distribution characteristics of the system specification formula data set in step S12, and then generates a new copy of the system specification formula data set based on these distribution characteristics. The formulas in the copy conform to the same distribution as the formulas in the system specification formula data set in step S12, but the two do not overlap, that is, the formulas in the copy will not appear in the system specification data set in step S12.

[0066] S3. Use the trained sequence-to-sequence neural network to initialize the sampling formula and gradually sample to obtain a copy of the system specification formula data set.

[0067] Step S3 includes:

[0068] S31. Input two trained sequence-to-sequence neural networks and the maximum number of time series operators, initialize the sampling formula to be empty, and initialize the number of symbols to be predicted.

[0069] Among them, the maximum number of timing operators is the maximum value of the number of timing operators in the sampling formula, which is used to control the termination of steps S32 and S33, and is intuitively understood as controlling the length of the sampling formula; the number of symbols to be predicted is the number of symbols that have not yet been predicted in the sampling formula, which is used to control the formula obtained by sampling to be a legal linear temporal logic formula. When the number of symbols to be predicted is 0, a legal linear temporal logic formula has been sampled, and the sampling can be terminated in advance to prevent the sampling formula from being too long and causing redundancy, thereby improving efficiency; when the sampling has reached the maximum number of timing operators, and the number of symbols to be predicted is not 0, it means that the formula is too short and needs to be filled in step S34 to obtain a legal linear temporal logic formula.

[0070] In this embodiment, the maximum number of timing operators is set to 4, and the sampling formula is initialized to an empty formula, that is, zero formula symbols are sampled initially; and the number of symbols to be predicted is initialized to 1.

[0071] S32. According to the current sampling formula, the probability distribution of the next symbol of the sampling formula is predicted by the decoding model TD1, the next symbol of the sampling formula is sampled therefrom, and the sampling formula is updated; and according to the type of the next symbol, the number of symbols to be predicted is increased by 1, decreased by 1, or remains unchanged.

[0072] Among them, the next symbol of the sampling formula can be a temporal operator or an atomic proposition placeholder. The atomic proposition placeholder is used to refer to the atomic proposition that appears in the linear temporal logic formula. The atomic proposition placeholder needs to be sampled in step S35 to determine which atomic proposition it is. In this step, the sampling can only be determined as an atomic proposition, but it is still impossible to determine which atomic proposition it is. In this embodiment, P is used to represent the atomic proposition placeholder, and P plus a subscript is used to represent a specific atomic proposition, such as .

[0073] If the next symbol of the sampling formula is an atomic proposition placeholder, the number of symbols to be predicted is reduced by 1; if the next symbol of the sampling formula is a binary temporal operator, the number of symbols to be predicted is increased by 1; if the next symbol of the sampling formula is a unary temporal operator (G, F, X, ), the number of symbols to be predicted remains unchanged.

[0074] For example, the current sampling formula is an empty formula (i.e., there is no symbol), and the predicted next symbol is a unary time series operator G (i.e., all future states are established), indicating that the following events are all established in the embedded storage system. Then, the number of symbols to be predicted remains unchanged, and the number of symbols to be predicted is still 1. The sampling formula is updated to the predicted next symbol, i.e., the sampling formula is updated to G.

[0075] S33, continue step S32 until the number of symbols to be predicted is equal to 0 or the maximum number of timing operators is reached, and generate an uninstantiated formula.

[0076] That is, repeat step S32, and sample the next symbol from the probability distribution predicted by the decoding model TD1 according to the updated sampling formula G as a binary timing operator. (i.e., "or", meaning that in the current state, either the first sub-formula or the second sub-formula is valid), the sampling formula is updated to , the number of symbols to be predicted is increased by 1, and the number of symbols to be predicted is updated to 2, indicating that the embedded storage system specification has two sub-specifications, and at least one of the two sub-specifications is established at any time; according to the updated sampling formula , the next symbol is sampled from the probability distribution predicted by the decoding model TD1 as a unary timing operator (i.e. "No", meaning that the sub-formula does not hold in the current state), the updated sampling formula is , the number of symbols to be predicted remains unchanged, and the number of symbols to be predicted is still 2, which means that in the two sub-specifications, the first sub-specification does not occur at any time, or the second sub-specification occurs at any time; according to the updated sampling formula , the next symbol is sampled from the probability distribution predicted by the decoding model TD1 as the atomic proposition placeholder P, and the sampling formula is updated to , the number of symbols to be predicted is reduced by 1, and the number of symbols to be predicted becomes 1, indicating that one of the two sub-specifications is an atomic proposition; according to the updated sampling formula , the next symbol is sampled from the probability distribution predicted by the decoding model TD1 as a unary temporal operator F (i.e., a certain state will be established in the future), indicating that the second sub-specification is an atomic proposition that is established at a certain moment in the future. The sampling formula is updated to , the number of symbols to be predicted remains unchanged, the number of symbols to be predicted is still 1, and the maximum number of timing operators is 4. The sampling formula finally updated is an uninstantiated formula, and step S34 is executed.

[0077] S34. Fill the atomic proposition placeholders at the end of the generated uninstantiated formula. The number of atomic proposition placeholders filled is the number of symbols to be predicted at the end of step S33. If the number of symbols to be predicted is equal to 0 at the end of step S33, do not fill them.

[0078] In this embodiment, the atomic proposition placeholder P is filled at the end of the finally updated sampling formula, and the sampling formula is updated to , indicating that the second sub-specification is also an atomic proposition that is valid at a future time.

[0079] S35. According to the uninstantiated formula generated in step S33, use decoding model TD2 to generate the probability distribution of atomic proposition placeholders on the atomic proposition set, and determine that the atomic proposition placeholder is a specific atomic proposition in the atomic proposition set according to the probability distribution; then sample the leftmost atomic proposition placeholder of the uninstantiated formula, and replace the sampled atomic proposition placeholder with the determined specific atomic proposition to update the uninstantiated formula.

[0080] In this embodiment, the formula obtained after filling in step S34 is: , contains two atomic proposition placeholders P; the atomic proposition placeholders are determined to be atomic propositions from the probability distribution returned by the decoding model TD2 ; Start sampling from the atomic proposition placeholder P on the leftmost side of the formula, and get the updated uninstantiated formula as , indicating that the first sub-specification is specifically the arrival of the request sent by the second sender to the buffer; then the second atomic proposition placeholder P is updated.

[0081] S36. Repeat step S35 until there are no atomic proposition placeholders in the uninstantiated formula to generate a system specification formula.

[0082] In this embodiment, according to the currently uninstantiated formula , the probability distribution of the atomic proposition placeholder returned from the decoding model TD2 on the atomic proposition set, determines that the atomic proposition placeholder on the left of the current uninstantiated formula is an atomic proposition , update the uninstantiated formula to , indicating that the second sub-specification specifically is that the request sent by the second sender to the buffer is confirmed, and it is established at some time in the future.

[0083] S37. Repeat steps S31-S36, check whether the system specification formula generated each time is repeated, record the amount of repetition, and end step S3 when the amount of repetition reaches a threshold or a specified number of copies of the system specification formula data set are generated.

[0084] S4. Fine-tune the pre-trained neural network for detection using a copy of the system specification formula dataset, and test the fine-tuned neural network for detection using the system specification formula dataset to verify the validity of the generated system specification formula and whether the embedded system implements the corresponding storage function.

[0085] The neural network used for detection in this embodiment considers two of the most advanced solvers among the current linear temporal logic formula neural symbolic solvers: tree neural network and graph neural network. In order to avoid randomness when evaluating the validity of the generated system specification formula, this embodiment pre-trains 5 converged models for these two neural symbolic solvers using the same parameters, that is, there are 5 models for the tree neural network and 5 models for the graph neural network.

[0086] Step S4 includes:

[0087] S41. Call the linear temporal logic solver to solve the copy of the system specification formula data set generated in step S3 and record its label.

[0088] In this step, the system specification formula data set copy is divided into a training set and a validation set according to an 8:1 ratio; the system specification formula data set in step S1 is used as the test set.

[0089] S42. Use a linear temporal logic random generator to generate tens of thousands (e.g., 200,000) of linear temporal logic random formulas, perform pre-training on a neural network model for detection, and obtain a fine-tuned graph neural network model and a tree neural network model.

[0090] S43. Fine-tune the graph neural network model and the tree neural network model using the training set and validation set in step S41 respectively.

[0091] S44. Use the system specification formula dataset to test the fine-tuned graph neural network model and tree neural network model.

[0092] The test results show that the graph neural network model and tree neural network model fine-tuned by the newly generated formula have indeed greatly improved the accuracy of the system specification formula data set, providing a certain reference value for subsequent demand conflict detection and verification, indicating the effectiveness of the generated formula. If the result predicted by the neural network is satisfactory, it means that the test of "the request sent by the first sender to the buffer must be confirmed" in step S11 has passed, that is, the software test has passed.

[0093] In the above steps S1-S4, S2-S3 intuitively shows the process of data augmentation, S1 describes the preprocessing stage of data, and S4 uses downstream application tasks to evaluate the effectiveness of augmented data. The entire S1-S4 process can be intuitively understood as: first convert the system specification requirements described in natural language into linear temporal logic formulas (step S1), then steps S2-S3 use data augmentation methods to generate more data, and step S4 uses the augmented data for training, thereby evaluating the effectiveness of the present invention.

[0094] In this embodiment, a data augmentation system for embedded system storage function testing is also provided, comprising the following modules:

[0095] A data set generation module, used to generate a system specification formula data set according to an input system specification document description;

[0096] Neural network training module, which trains sequence-to-sequence neural networks using system specification formula datasets;

[0097] The dataset replica sampling module uses the trained sequence-to-sequence neural network to initialize the sampling formula and gradually samples to obtain a replica of the system specification formula dataset;

[0098] A testing module, which uses a copy of the system specification formula data set to fine-tune the pre-trained neural network for detection, and uses the system specification formula data set to test the fine-tuned neural network for detection to verify the validity of the generated system specification formula and whether the embedded system implements the corresponding storage function;

[0099] The process of generating a system standard formula data set by the data set generation module includes:

[0100] According to the input system specification document description, an atomic event set involved in the system specification document is identified, and the atomic events in the system specification document description are replaced with atomic proposition symbols, so as to define an atomic proposition set according to the atomic event set;

[0101] According to the system specification document description, an event set is defined using an atomic proposition set and a temporal operator set. Each event in the event set is presented as a linear temporal logic formula composed of an atomic proposition and a temporal operator.

[0102] Among them, events are used to describe the specifications that the hardware buffer must meet, including atomic events and non-atomic events; atomic events are basic events that cannot be decomposed any further, and non-atomic events are a collection of single atomic events or a combination of multiple atomic events; atomic propositions are either true or false at a certain moment, and the event referred to by the atomic proposition cannot be equivalent to the combinational logic of other atomic propositions and timing operators;

[0103] When describing system specifications, events are presented in the form of linear temporal logic formulas consisting of atomic propositions and timing operators; timing operators are used to describe changes in system states, including describing the system's current state, next state, and future state.

[0104] The specific implementation process of each of the above modules can refer to the above steps S1-S4 and will not be repeated here.

[0105] The experimental results of the method of the present invention on three unknown distribution data sets and two satisfiability solvers are shown in the following table, where Table 1 is the experimental results of the tree neural network and Table 2 is the experimental results of the graph neural network. The present invention compares the performance of the neural network used for detection in three cases, namely: no fine-tuning (zero-shot), using the random formula generation method to generate formulas for fine-tuning (random) and using the formula generation method of the present invention to generate formulas for fine-tuning (LTLGen). At the same time, the present invention conducted experiments on three common industrial data sets (amba, genbuf and forobots) including the hardware buffer data set (genbuf). The results show that the method of the present invention can effectively improve the performance of a small number of unknown distribution data sets on the satisfiability solving neural network.

[0106] Table 1. Experimental results of tree neural network

[0107]

[0108] Table 2. Experimental results of graph neural network

[0109]

[0110] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A data augmentation method for embedded system storage function testing, characterized in that: The following steps are involved: S1. Generate a system specification formula data set based on the input system specification document description; S2, train a sequence-to-sequence neural network using the system specification formula dataset; S3, using the trained sequence-to-sequence neural network, initialize the sampling formula, and gradually sample to obtain a copy of the system specification formula data set; S4. Fine-tune the pre-trained neural network for detection using the copy of the system specification formula data set, and test the fine-tuned neural network for detection using the system specification formula data set to verify the validity of the generated system specification formula and whether the embedded system implements the corresponding storage function; Step S1 includes: S11, according to the input system specification document description, identifying the atomic event set involved in the system specification document, replacing the atomic events in the system specification document description with atomic proposition symbols, thereby defining an atomic proposition set according to the atomic event set; S12. According to the description of the system specification document, an event set is defined using an atomic proposition set and a temporal operator set, where each event in the event set is presented as a linear temporal logic formula composed of an atomic proposition and a temporal operator; Among them, events are used to describe the specifications that the hardware buffer must meet, including atomic events and non-atomic events; atomic events are basic events that cannot be decomposed any further, and non-atomic events are a collection of single atomic events or a combination of multiple atomic events; atomic propositions are either true or false at a certain moment, and the event referred to by the atomic proposition cannot be equivalent to the combinational logic of other atomic propositions and timing operators; When describing system specifications, events are presented in the form of linear temporal logic formulas consisting of atomic propositions and timing operators; timing operators are used to describe changes in system states, including describing the system's current state, next state, and future state.

2. The data augmentation method according to claim 1, characterized in that: Step S1 defines the set of atomic propositions as , where the first atomic proposition symbol The atomic event representing the arrival of a request from the first sender to the buffer, the second atomic proposition symbol The atomic event representing the confirmation of the request sent by the first sender to the buffer, the third atomic proposition symbol The atomic event representing the arrival of the request sent by the second sender to the buffer, the fourth atomic proposition symbol An atomic event representing the confirmation of the request sent to the buffer by the second sender; In linear temporal logic formulas, placeholders for atomic propositions used for generalization are also set; The position where an atomic proposition placeholder appears in a linear temporal logic formula indicates that the position is an atomic proposition in the set of atomic propositions.

3. The data augmentation method according to claim 1, characterized in that: Step S1 simplifies the set of timing operators, except for the timing operator X used to indicate that the next state is established, the timing operator U used to indicate that until a certain state is established, the timing operator F used to indicate that a certain state is established in the future, the timing operator G used to indicate that all states are established in the future, and the timing operator , time series operator or , time series operator and The timing operators other than are converted equivalently.

4. The data augmentation method according to claim 1, characterized in that: In step S2, the sequence-to-sequence neural network learns the distribution characteristics of the system specification formula data set, and generates a new copy of the system specification formula data set according to the distribution characteristics; The formulas in the replicas conform to the same distribution as the formulas in the system specification formula dataset; The sequence-to-sequence neural network includes a sequence-to-sequence Transformer decoding model TD1 for predicting the system specification formula and a sequence-to-sequence Transformer decoding model TD2 for predicting the instantiation of the system specification formula; Step S2 includes: S21, slicing the system specification formula data set to obtain the sliced ​​local formulas and form a local formula data set of fixed length; the local formulas in the local formula data set represent the specification requirements that the embedded system needs to meet under a preset state length; S22, use the local formula dataset to train the sequence-to-sequence Transformer decoding model TD1 for predicting the system specification formula; S23. Use the local formula dataset to train a sequence-to-sequence Transformer decoding model TD2 for predicting system specification formula instantiations.

5. The data augmentation method according to claim 4, characterized in that: Step S3 includes: S31, input the trained sequence-to-sequence neural network and the maximum number of time series operators, initialize the sampling formula to be empty, and initialize the number of symbols to be predicted; Among them, the maximum number of temporal operators is the maximum number of temporal operators in the sampling formula, which is used to control the length of the sampling formula; the number of symbols to be predicted is the number of symbols that have not been predicted in the sampling formula, which is used to control whether the formula obtained by sampling is a legal linear temporal logic formula; S32, according to the current sampling formula, predict the probability distribution of the next symbol of the sampling formula through the decoding model TD1, sample the next symbol of the sampling formula, and update the sampling formula; and according to the type of the next symbol, increase or decrease the number of symbols to be predicted by 1 or keep it unchanged; S33, continuing step S32 until the number of symbols to be predicted is equal to 0 or the maximum number of timing operators is reached, and generating an uninstantiated formula; S34, filling atomic proposition placeholders at the end of the generated uninstantiated formula, the number of filled atomic proposition placeholders being the number of symbols to be predicted at the end of step S33; if the number of symbols to be predicted is equal to 0 at the end of step S33, no filling is done; S35, according to the uninstantiated formula generated in step S33, using decoding model TD2 to generate the probability distribution of the atomic proposition placeholder on the atomic proposition set, and determining the atomic proposition placeholder as a specific atomic proposition in the atomic proposition set according to the probability distribution; then sampling to obtain the atomic proposition placeholder on the leftmost side of the uninstantiated formula, and replacing the sampled atomic proposition placeholder with the determined specific atomic proposition to update the uninstantiated formula; S36, repeating step S35 until there are no atomic proposition placeholders in the uninstantiated formula, so as to generate a system specification formula; S37. Repeat steps S31-S36, check whether the system specification formula generated each time is repeated, record the amount of repetition, and when the amount of repetition reaches a threshold or a specified number of copies of the system specification formula data set are generated.

6. The data augmentation method according to claim 5, characterized in that: In step S32, the next symbol of the sampled formula is a temporal operator or an atomic proposition placeholder. The temporal operator includes a unary temporal operator and a binary temporal operator. The unary temporal operator only connects one sub-formula, and the binary temporal operator connects two sub-formulas. The atomic proposition placeholder is used to generally refer to the atomic proposition appearing in the linear temporal logic formula. If the next symbol of the sampled formula is an atomic proposition placeholder, the number of symbols to be predicted is reduced by 1; If the next symbol of the sampling formula is a binary timing operator, the number of symbols to be predicted is increased by 1; if the next symbol of the sampling formula is a unary timing operator, the number of symbols to be predicted remains unchanged.

7. The data augmentation method according to claim 1, characterized in that: Step S4 includes: S41, calling a linear temporal logic solver to solve the system specification formula data set copy and record its label; S42, using a linear temporal logic random generator to generate tens of thousands of linear temporal logic random formulas, pre-training a neural network model for detection, and obtaining a fine-tuned graph neural network model and a tree neural network model; S43, fine-tuning the graph neural network model and the tree neural network model using the training set and the validation set in step S41 respectively; S44. Use the system specification formula dataset to test the fine-tuned graph neural network model and tree neural network model.

8. A data augmentation system for embedded system storage function testing, characterized in that: Includes the following modules: A data set generation module, used to generate a system specification formula data set according to an input system specification document description; Neural network training module, which trains sequence-to-sequence neural networks using system specification formula datasets; The dataset replica sampling module uses the trained sequence-to-sequence neural network to initialize the sampling formula and gradually samples to obtain a replica of the system specification formula dataset; A testing module, which uses a copy of the system specification formula data set to fine-tune the pre-trained neural network for detection, and uses the system specification formula data set to test the fine-tuned neural network for detection to verify the validity of the generated system specification formula and whether the embedded system implements the corresponding storage function; The process of generating a system standard formula data set by the data set generation module includes: According to the input system specification document description, an atomic event set involved in the system specification document is identified, and the atomic events in the system specification document description are replaced with atomic proposition symbols, so as to define an atomic proposition set according to the atomic event set; According to the system specification document description, an event set is defined using an atomic proposition set and a temporal operator set. Each event in the event set is presented as a linear temporal logic formula composed of an atomic proposition and a temporal operator. Among them, events are used to describe the specifications that the hardware buffer must meet, including atomic events and non-atomic events; atomic events are basic events that cannot be decomposed any further, and non-atomic events are a collection of single atomic events or a combination of multiple atomic events; atomic propositions are either true or false at a certain moment, and the event referred to by the atomic proposition cannot be equivalent to the combinational logic of other atomic propositions and timing operators; When describing system specifications, events are presented in the form of linear temporal logic formulas consisting of atomic propositions and timing operators; timing operators are used to describe changes in system states, including describing the system's current state, next state, and future state.

9. The data augmentation system according to claim 8, characterized in that: The data set generation module defines the atomic proposition set as , where the first atomic proposition symbol The atomic event representing the arrival of a request from the first sender to the buffer, the second atomic proposition symbol The atomic event representing the confirmation of the request sent by the first sender to the buffer, the third atomic proposition symbol The atomic event representing the arrival of the request sent by the second sender to the buffer, the fourth atomic proposition symbol An atomic event representing the confirmation of the request sent to the buffer by the second sender; In linear temporal logic formulas, placeholders for atomic propositions used for generalization are also set; The position where an atomic proposition placeholder appears in a linear temporal logic formula indicates that the position is an atomic proposition in the set of atomic propositions.

10. The data augmentation system according to claim 8, characterized in that: In the neural network training module, the sequence-to-sequence neural network learns the distribution characteristics of the system specification formula data set, and generates a new copy of the system specification formula data set according to the distribution characteristics; The formulas in the replicas conform to the same distribution as the formulas in the system specification formula dataset; The sequence-to-sequence neural network includes a sequence-to-sequence Transformer decoding model TD1 for predicting the system specification formula and a sequence-to-sequence Transformer decoding model TD2 for predicting the instantiation of the system specification formula; The training process of the neural network training module includes: Slicing the system specification formula data set to obtain the sliced ​​local formulas and form a local formula data set of fixed length; the local formulas in the local formula data set represent the specification requirements that the embedded system needs to meet under a preset state length; Use the local formula dataset to train a sequence-to-sequence Transformer decoding model TD1 for predicting the system's canonical formula; The local formula dataset is used to train a sequence-to-sequence Transformer decoding model TD2 for predicting instantiations of the system's canonical formula.

Images