A neural network-based airborne embedded software test case automatic generation method

Abstract

The application discloses a kind of neural network-based airborne embedded software test case automatic generation method, specifically is: screening test demand, the information of demand statement containing relevant field is completed, and the constant and variable in demand document are replaced;Multiple demand statements are cut into single-line demand statements;All single-line demand statements are identified using named entities identification;The entity tag corresponding to each character in the identified single-line demand statement is processed to generate a triple containing subject, predicate, object element;The key word of triple is replaced;According to the action of predicate element and the value of object element, the subject element is assigned value;Test case sequence is generated.The beneficial effects of the present application are: the present application is especially for airborne embedded software, can automatically generate test case without changing the writing rules of demand document, significantly improve the test efficiency, reduce manual intervention, reduce the test cost.

Description

TECHNICAL FIELD

[0001] The present application relates to test case automatic generation technology, in particular to a neural network-based airborne embedded software test case automatic generation method. BACKGROUND

[0002] Software automatic test technology is a technology for verifying software running conditions by using automatically generated test cases, and its main purpose is to reduce software test cost, reduce the pressure of software test engineers in writing test cases and test scripts, avoid human errors caused by manually writing test cases and test scripts, and improve software test efficiency. Writing test cases by using automatic tools for functional testing can save testing time.

[0003] At present, the research on the theory and technology of airborne embedded software testing is still in its infancy, and the generation of test cases for airborne software needs to be manually written according to the requirement document, which is low in efficiency.

[0004] The existing test case automatic generation method uses genetic algorithm to realize the priority sorting of airborne software test cases, converts the activity diagram of software system functions into a control flow diagram, and after coding, selects, crosses, mutates and other steps to finally obtain the individual with the highest fitness for priority testing. However, this method can only be used for test case reuse when performing regression testing, and cannot directly generate test cases from requirement documents. The named entity recognition technology has achieved good results in public languages and some specific fields, but cannot be directly applied to the field of airborne embedded software, and lacks relevant corpus. SUMMARY

[0005] In order to automatically generate test cases from requirement files and improve software automatic test efficiency, the present application provides a neural network-based airborne embedded software test case automatic generation method.

[0006] In order to achieve the above-mentioned application purposes, the present application provides a neural network-based airborne embedded software test case automatic generation method, which comprises the following steps:

[0007] Step S1: screening test requirements from the software requirement document, supplementing information to the requirement sentences containing relevant fields according to the requirement information supplement table, and replacing the constants and variables in the requirement document according to the corresponding relationship of the constants and variables in the comparison table;

[0008] Step S2: selecting multiple lines of requirement sentences and cutting them into single-line requirement sentences;

[0009]

[0009] Step S3: identifying all single-line requirement sentences using named entity recognition;

[0010] Step S4: processing the entity label corresponding to each character in the identified single-line requirement sentence to generate a triple containing a subject, predicate and object element;

[0011] Step S5: keyword replacement on the triple;

[0012] Step S6: assigning a value to the subject element according to the action of the predicate element and the value of the object element;

[0013] Step S7: generating a test case sequence.

[0014] The step S1 is specifically: first, screening the testable requirements from the software requirement document, supplementing the information of the requirement sentence containing the relevant fields according to the requirement information supplement table, and finally replacing the constants and variables according to the corresponding relationship of the constant and variable in the comparison table to obtain the requirement sentence document that can be directly used for named entity recognition.

[0015] The step S3 is to realize named entity recognition based on the BiLSTM-CRF model, and to mark the subject, predicate and object in the testable inner sentence in the requirement sentence by using the SPO marking method, and to mark the corresponding target element, operation instruction and interaction information; wherein, in the marking process, the BIO marking method based on the character level is used; in the embedding layer of the BiLSTM-CRF model, the one-hot encoding is converted into a low-dimensional dense word vector using the Word2Vec model, which is horizontally spliced with the part-of-speech feature vector to form the final word vector, and the obtained final word vector is input into the BiLSTM model in units of sentences, the relationship between the input word vector and the output label is learned through training, and then the CRF model is used to update the transition matrix parameters to predict the optimal label corresponding to each character.

[0016] In the step S4, the number of B-SUBJECT, B-PREDICATE and B-OBJECT labels is used to represent the number of entities of the corresponding subject element, predicate element and object element in the identified single-line requirement sentence; wherein, the triple generation rule is as follows:

[0017] If the number of B-SUBJECT and B-PREDICATE is 1 and B-OBJECT exists, it means that the requirement sentence is in the subject-predicate-object structure, and a subject-predicate-object triple is generated at this time;

[0018] If the number of B-SUBJECT is 1, the number of B-PREDICATE and B-OBJECT is equal and greater than 1, it means that the requirement sentence is in the subject-predicate-object and predicate-object structure, and a plurality of subject-predicate-object structures with the same subject are generated at this time;

[0019] If the number of B-SUBJECT and B-PREDICATE is 1 and there is no B-OBJECT, it indicates that the requirement sentence is a subject-predicate structure, and a triple without an object element is generated at this time;

[0020] If the number of B-SUBJECT is greater than 1, it indicates that the requirement sentence is a multiple structure containing multiple triples, and the splitting process is performed until the number of B-SUBJECT in the split part is 1.

[0021] If the number of labels is not the above composition, it is considered as not meeting the triple generation rule.

[0022] In the step S6, the keyword mapping table and the keyword antonym mapping table stored in the form of a table are used to identify the same words in the requirement sentence with the same keyword, and the antonym of the keyword which needs to generate the reverse test case is replaced.

[0023] The step S7 is specifically: traversing the requirement document and collecting all variables, and then storing the name and value of the variable in the data dictionary; for the repeatedly appearing variable, updating the variable value in the data dictionary; finally, generating the test case according to the data dictionary and the structured data.

[0024] The application also provides a neural network-based automatic generation method of an airborne embedded software test script, characterized in that the method is further realized based on the test case automatic generation method in any one of claims 1-8, and the steps of the method are as follows:

[0025] Firstly, a file list of a file position storing the test case is read for the parsing of the test case file one by one;

[0026] Secondly, an XML file reading module is used to read an XML file storing model structure and other information, which is used to judge the validity and data type of the variable in the test case;

[0027] Thirdly, the test case is imported, the corresponding information is extracted according to the keyword in the test case through a variable processing module, and a test script file is generated according to the syntax rule of the test script.

[0028] The application has the beneficial effects that the application is particularly for the airborne embedded software, can automatically generate the test case without changing the requirement document writing rule, significantly improves the test efficiency, reduces the manual intervention and reduces the test cost; the requirement sentence which can generate the test case is accurately extracted through the preprocessing of the original requirement document, the requirement sentence generates the triple after the named entity recognition, and the correct test case is generated in combination with the requirement sentence logic, and finally the test case has a unified format and can be directly imported into the simulation test tool. Attached Figure Description

[0029] Figure 1 The flowcharts are for embodiments 1-3 of the present invention.

[0030] Figure 2 The flowcharts are for the preprocessing of requirement documents in embodiments 1 and 2 of the present invention.

[0031] Figure 3 The flowcharts for generating triples in Embodiments 1 and 2 of the present invention are shown.

[0032] Figure 4 This is a flowchart of the automatic generation method for test script files in Embodiment 3 of the present invention. Detailed Implementation

[0033] To clearly illustrate the technical features of this solution, the following detailed implementation method will be used to explain the solution.

[0034] The main objective of this invention is to transform onboard software requirements described in natural language into test cases and test scripts that computers can understand. This invention uses software requirement documents described in natural language as the starting point for automated testing, and combines a deep learning-based named entity recognition method with a keyword-based test case generation method. This enables the automatic generation of test cases, and based on the conversion relationship between test cases and test scripts, generates test script files that can be directly run in SCADE.

[0035] Example 1

[0036] This invention provides a method for automatically generating test cases for airborne embedded software based on neural networks (e.g., Figure 1 As shown), the method includes the following steps:

[0037] Step S1: Select test requirements from the software requirements document, complete the information of the requirement statements containing relevant fields according to the requirement information completion table, and replace the constants and variables in the requirement document according to the correspondence between constants and variables in the comparison table.

[0038] Step S2: Select the multi-line requirement statement and split it into single-line requirement statements;

[0039] Step S3: Use named entity recognition to identify all single-line requirement statements;

[0040] Step S4: Process the entity tags corresponding to each character in the identified single-line requirement statement to generate a triple containing subject, predicate, and object elements;

[0041] Step S5: Replace keywords in the triples;

[0042] Step S6: value the subject element according to the action of the predicate element and the value of the object element;

[0043] Step S7: generate the test case sequence.

[0044] Wherein, step S1 is: first, filter the testable requirements from the software requirement document, complete the information of the requirement statement containing the related fields according to the requirement information completion table, and finally replace the constants and variables according to the corresponding relationship of the constant and variable in the comparison table to obtain the requirement statement document which can be directly used for named entity recognition.

[0045] Step S3 is to realize named entity recognition based on BiLSTM-CRF model, and to mark the subject, predicate and object in the testable requirement statement in the requirement statement by using SPO marking method, and to mark them as the corresponding target elements, operation instructions and interaction information; wherein, in the marking process, the BIO marking method based on character level is used; in the embedding layer of BiLSTM-CRF model, the one-hot encoding is converted into low-dimensional dense word vector by using Word2Vec model, which is transversely spliced with the part-of-speech feature vector to form the final word vector, and the obtained final word vector is input into the BiLSTM model in units of sentences, the relationship between the input word vector and the output label is learned through training, and then the CRF model is used to update the transition matrix parameters to predict the optimal label corresponding to each character.

[0046] In step S4, the number of B-SUBJECT, B-PREDICATE and B-OBJECT labels is used to represent the number of entities of the corresponding subject element, predicate element and object element in the recognized single-line requirement statement; wherein, the triple generation rule is as follows:

[0047] If the number of B-SUBJECT and B-PREDICATE is 1 and B-OBJECT exists, it means that the requirement statement is in the subject-predicate-object structure, and the subject-predicate-object triple is generated at this time;

[0048] If the number of B-SUBJECT is 1, the number of B-PREDICATE and B-OBJECT is equal and greater than 1, it means that the requirement statement is in the subject-predicate-object and predicate-object structure, and multiple subject-predicate-object structures with the same subject are generated at this time;

[0049] If the number of B-SUBJECT and B-PREDICATE is 1 and B-OBJECT does not exist, it means that the requirement statement is in the subject-predicate structure, and the triple without object element is generated at this time;

[0050] If the number of B-SUBJECT is more than 1, it indicates that the requirement sentence is a multiple structure containing multiple triples, at this time, the splitting processing is carried out until the number of B-SUBJECT in the split part is 1.

[0051] If the number of labels is not the above composition, it is considered as not meeting the triple generation rule.

[0052] In step S6, the keyword mapping table and the keyword antonym mapping table stored in the form of table are used to identify the words with the same meaning in the requirement sentence with the same keyword, and the antonym of the keyword needing to generate the reverse test case is replaced.

[0053] Step S7 is specifically: traversing the requirement document and collecting all variables, and then storing the names and values of the variables in the data dictionary; for the repeatedly appearing variables, updating the variable values in the data dictionary; finally generating test cases according to the data dictionary and structured data.

[0054] Embodiment 2

[0055] Referring to Figure 1 The application provides a neural network-based airborne embedded software test case automatic generation method, and the specific steps are as follows:

[0056] Step one: in the process of test case generation, the testable requirements are screened out, and the constants and variables in the requirement document are replaced according to the corresponding relationship of the common variables in the comparison table.

[0057] Referring to Figure 2 The requirement document preprocessing process includes the following steps:

[0058] First, the testable requirements are screened out; then the information of the requirement information is completed to complete the information of the requirement sentence containing the related fields; finally, the constants and variables are replaced according to the corresponding relationship of the common variables in the comparison table, and the requirement sentence document directly used for named entity recognition is obtained.

[0059] Step two: since the named entity recognition model needs to split multiple lines of requirement sentences in the training process, before the entity recognition of the requirement sentence, the single-line requirement sentence and the multi-line requirement sentence need to be processed respectively. For single-line requirement sentence, directly perform named entity recognition processing; for multi-line requirement sentence, first split the sentence according to the line break symbol, and then process sentence by sentence.

[0060] Step three: using the named entity recognition model to recognize the requirement sentence.

[0061] The testable statement in the requirement statement has a clear subject-predicate-object structure, and the SPO (subject, predicate, object) labeling method is used to mark the target elements, operation instructions and interaction information of the test case as three types of labels for the training model.

[0062] The named entity recognition corpus of the embodiment is annotated at the character level based on words, to avoid boundary recognition errors of named entities caused by direct word segmentation. Since the named entities of the airborne software requirement statement are usually not complex, and there is little overlap between adjacent entities, the embodiment uses the BIO labeling method.

[0063] According to the characteristics of the airborne display system requirement statement, the BiLSTM-CRF model widely used in the named entity recognition task and having good effect is adopted, and a BiLSTM-CRF model combined with the part-of-speech feature is constructed, to improve the feature expression ability of the word vector. In the embedding layer, the one-hot encoding is converted into a low-dimensional dense word vector using the Word2Vec model. These word vectors are spliced with the part-of-speech feature vectors obtained by the same method to form the final word vector. Since the order of magnitude of the part-of-speech annotation is very different from that of the character, the dimension of the part-of-speech word vector is set to 10% of the character word vector. The obtained word vector is input into the BiLSTM model in units of sentences, and the relationship between the input word vector and the output label is learned through training. Then, the CRF model is used to update the transition matrix parameters to predict the optimal label corresponding to each character.

[0064] The BiLSTM model uses a forward and backward two-layer LSTM model to capture information from both directions. The CRF model takes the feature vector generated by the BiLSTM model as input, and learns the relationship between the input word vector and the entity label to improve the accuracy of named entity recognition. In this section, the Word2Vec model is used to convert characters into word vectors, and the word vectors are horizontally spliced with the part-of-speech feature vectors, which are then used as input for the BiLSTM-CRF model for neural network training.

[0065] Step four: After identifying the named entities in the requirement statement, the entities need to be further processed to generate triples containing subject, predicate and object elements. For example, the extracted triples of the preprocessed requirement statement “When ipEIIs is 3, and ipEIIs is 1, the N1 label should be displayed in 3 font.” are (ipEIIs, is, 3) and (ipEIIs, is, 1). To obtain the triples, the entity label corresponding to each character in the requirement statement needs to be processed accordingly.

[0066] In the triple, the subject is a variable name, indicating the object that needs to be assigned; the predicate indicates the action related to the assignment of the variable, such as "for", "greater than", "increase", etc.; the object is the value related to the variable. The assignment of the variable in the test case is usually determined by the value in the object.

[0067] Since the B-X label represents the starting label of a certain type of named entity, the number of B-X labels can be used to represent the number of corresponding named entities in the sentence. If the number of B-SUBJECT and B-PREDICATE is 1 and there is B-OBJECT, it means that the requirement sentence is in the subject-predicate-object structure, and the subject-predicate-object triple is generated at this time; if the number of B-SUBJECT is 1, the number of B-PREDICATE and B-OBJECT is equal and greater than 1, it means that the requirement sentence is in the subject-predicate-object, predicate-object structure, and multiple subject-predicate-object structures with the same subject are generated at this time; if the number of B-SUBJECT and B-PREDICATE is 1 and there is no B-OBJECT, it means that the requirement sentence is in the subject-predicate structure, and the triple without the object element is generated at this time; if the number of B-SUBJECT is greater than 1, it means that the requirement sentence is in a multiple structure containing multiple triples, and the splitting process is performed until the number of B-SUBJECT in the split part is 1; if the number of labels is not in the above structure, it is considered not to meet the triple generation rule. The triple generation process is shown in Figure 3 .

[0068] Step five: After generating the triple, the subject element needs to be assigned according to the action of the predicate element and the value of the object element. Generally, the same semantics in the predicate element have multiple different representations, such as "greater than", "higher than", "more than", etc. all represent the meaning of greater than a certain value. In order to facilitate further processing of the triple, the present application uses a keyword mapping table to represent the same keyword for the words with the same meaning in the requirement sentence.

[0069] Step six: Test case sequence generation. In the process of traversing the requirement document and generating the test case, the present application collects all the variables, stores these variables and their values in the data dictionary, and updates the repeated variable values. The function of the data dictionary is to record the data in the currently generated test case, and to provide the pre-sequence for generating the subsequent test case. For example, the requirement sentence "When the value of the ipFADECN1L parameter increases, the left engine N1 pointer should rotate clockwise." cannot generate a correct test case from this requirement sentence. In this case, the value of the ipFADECN1L parameter can be obtained from the data dictionary, and a larger value can be taken on this basis, so as to obtain the test case corresponding to the requirement.

[0070] After the keyword replacement of the triplets, the application uses the structured data to record the relevant information extracted in the above steps. The next step is to generate test case sequences according to the data dictionary and the structured data. That is, different processing methods are used for different predicate keywords to obtain the final test cases.

[0071] Embodiment 3

[0072] The embodiment of the application provides a method for automatically generating an airborne embedded software test script based on a neural network. The method is further implemented based on the test cases obtained in Embodiment 1 or Embodiment 2. Specifically, referring to Figure 4 , the test script file automatic generation step is as follows:

[0073] 1. First, read the file list storing the test case file position, which is used for the individual parsing of the test case file.

[0074] 2. Read the XML file storing the model structure and other information through an XML file reading module, which is used for judging the validity and data type of the variable in the test case.

[0075] 3. Import the test case, extract the corresponding information according to the keyword in the test case through the variable processing module, and generate the script file according to the syntax rules of the test script.

[0076] After obtaining the variable name and value, first, judge whether the variable name is valid in the SCADE model according to the variable information in the XML file. If it is not valid, output the note "not found this signal" in the script file, which is used to prompt the tester to analyze the error. Since the value of the bool type variable is usually represented by the numbers "0" and "1", it is necessary to filter out the bool type value for separate processing. If the variable is a bool type value, assign "false" or "true" according to "0" or "1". If the variable is a numerical value, directly assign the corresponding value.

[0077] The model coverage analysis of the airborne software is realized through MTC. Before the model coverage collection process, the test script file needs to be taken as the input, and the test script needs to be written according to the test case file. Since the number of test cases is huge, manual conversion of the test case to the test script will consume a large amount of manpower and time. The test case automatic generation method provided in Embodiment 1 or Embodiment 2 can save a lot of manpower and time. Therefore, the combination of Embodiment 1 or 2 and the test script file automatic generation tool of the present embodiment (see Figure 1 ) can greatly improve the work efficiency and reduce the labor cost.

[0078] The above merely describes preferred embodiments of the present application and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for automatically generating test cases for airborne embedded software based on neural networks, characterized in that, The method includes the following steps: Step S1: Select test requirements from the software requirements document, complete the information of the requirement statements containing relevant fields according to the requirement information completion table, and replace the constants and variables in the requirement document according to the correspondence between constants and variables in the comparison table. Step S2: Select the multi-line requirement statement and split it into single-line requirement statements; Step S3: Use named entity recognition to identify all single-line requirement statements; Step S4: Process the entity tags corresponding to each character in the identified single-line requirement statement to generate a triple containing subject, predicate, and object elements; Step S5: Replace keywords in the triples; Step S6: Assign a value to the subject element based on the action of the predicate element and the value of the object element; Step S7: Generate a sequence of test cases; Specifically, step S1 involves: first, selecting testable requirements from the software requirements document; then, completing the requirements statements containing relevant fields according to the requirements information completion table; and finally, replacing constants and variables according to the correspondence between constants and variables in the lookup table to obtain a requirements statement document that can be directly used for named entity recognition. Step S3 is based on the BiLSTM-CRF model to realize named entity recognition. The SPO annotation method is used to perform part-of-speech tagging on the subject, predicate and object in the testable internal sentences of the requirement statement, and to clearly mark them as the corresponding target elements, operation instructions and interaction information. During the tagging process, a character-level BIO tagging method is used. In the embedding layer of the BiLSTM-CRF model, the Word2Vec model is used to convert one-hot encoding into low-dimensional dense character vectors, which are then concatenated horizontally with part-of-speech feature vectors to form the final character vectors. The final character vectors are then input into the BiLSTM model in units of sentences. The relationship between the input character vectors and the output labels is learned through training. Then, the CRF model is used to update the parameters of the transition matrix to predict the optimal label for each character. In step S4, the number of B-SUBJECT, B-PREDICATE and B-OBJECT tags respectively represents the number of entities of the corresponding subject element, predicate element and object element in the identified single-line requirement statement; In step S4, the triplet generation rules are as follows: If the number of B-SUBJECT and B-PREDICATE is 1 and B-OBJECT exists, it indicates that the requirement statement has a subject-verb-object structure, and a subject-verb-object triplet is generated. If the number of B-SUBJECT is 1, and the number of B-PREDICATE and B-OBJECT is equal and both greater than 1, it indicates that the requirement statement has a subject-verb-object structure, and multiple subject-verb-object structures with the same subject are generated. If the number of B-SUBJECT and B-PREDICATE is 1 and there is no B-OBJECT, it means that the requirement statement is a subject-verb structure, and a triplet without an object element is generated. If the number of B-SUBJECTs is greater than 1, it means that the requirement statement is a multi-structure containing multiple triples. In this case, it is split until the number of B-SUBJECTs in the split part is 1. If the number of tags is not as described above, it is considered to violate the triple generation rules. In step S6, a keyword mapping table and a keyword antonym mapping table stored in tabular form are used to identify words with the same meaning in the requirement statement using the same keyword, and the keywords that need to generate reverse test cases are antonymed and replaced. Step S7 specifically involves: traversing the requirements document and collecting all variables, then storing the names and values ​​of the variables in the data dictionary; for variables that appear repeatedly, updating their values ​​in the data dictionary; and finally generating test cases based on the data dictionary and structured data.

2. A method for automatically generating test scripts for airborne embedded software based on neural networks, characterized in that, This method is a further implementation based on the test case automatic generation method described in claim 1, and the steps of this method are as follows: First, read the file list where the test cases are stored, and use it to parse the test case files one by one; Secondly, the XML file reading module reads the XML file that stores model structure information, which is used to determine the validity and data type of variables in the test cases; Next, import the test cases, extract relevant information based on the keywords in the test cases using the variable processing module, and generate test script files according to the syntax rules of the test scripts.

Images