Neural network-based airborne embedded software test case automatic generation method

Abstract

The invention discloses an airborne embedded software test case automatic generation method based on a neural network, and the method specifically comprises the steps: screening out a test demand, carrying out the information completion of a demand statement containing a related field, and replacing a constant and a variable in a demand document; segmenting the multi-row demand statements into single-row demand statements; identifying all single-line demand statements by using a named entity; processing an entity tag corresponding to each character in the identified single-row demand statement to generate a triple containing subject, predicate and object elements; carrying out keyword replacement on the triple; assigning the subject element according to the action of the predicate element and the value of the object element; and generating a test case sequence. The method has the advantages that the method is especially for airborne embedded software, the test case can be automatically generated on the premise that the requirement document writing rule is not changed, the test efficiency is remarkably improved, manual intervention is reduced, and the test cost is reduced.

Description

Technical Field

[0001] The invention relates to a test case automatic generation technology, and in particular to a neural network-based airborne embedded software test case automatic generation method. Background Art

[0002] Software automated testing technology is a technology that uses automatically generated test cases to verify the operation of software. Its main purpose is to reduce software testing costs, reduce the pressure on software testing engineers to write test cases and test scripts, avoid human errors caused by manually writing test cases and test scripts, and improve software testing efficiency. Using automated tools to write test cases for functional testing can save testing time.

[0003] At present, the research on the theory and technology of airborne embedded software testing is still in its infancy. The generation of airborne software test cases needs to be manually written according to the requirement documents, which is inefficient.

[0004] The existing test case automatic generation method uses genetic algorithms to prioritize airborne software test cases, converts the activity diagram of the software system function into a control flow diagram, and performs selection, crossover, mutation and other steps after encoding to finally obtain the individuals with the highest fitness for priority testing. However, this method can only be used to reuse test cases when performing regression testing, and cannot achieve direct generation from requirement documents to test cases. Named entity recognition technology has achieved good results in the public language field and some specific fields, but it cannot be directly applied to the field of airborne embedded software, and there is a lack of relevant corpus. Summary of the invention

[0005] In order to automatically generate test cases from requirement documents and improve software automatic testing efficiency, the present invention provides a method for automatically generating test cases for airborne embedded software based on a neural network.

[0006] In order to achieve the above-mentioned object of the invention, the present invention provides a method for automatically generating airborne embedded software test cases based on a neural network, the method comprising the following steps: Step S1: Filter out the test requirements from the software requirement document, complete the information of the requirement statements containing relevant fields according to the requirement information completion table, and replace the constants and variables in the requirement document according to the correspondence between the constants and variables in the comparison table; Step S2: select multiple lines of requirement statements and divide them into single-line requirement statements; Step S3: Use named entity recognition to identify all single-line requirement statements; Step S4: Processing the entity tag corresponding to each character in the recognized single-line requirement sentence to generate a triple containing subject, predicate, and object elements; Step S5: replace keywords on the triples; Step S6: assigning a value to the subject element according to the action of the predicate element and the value of the object element; Step S7: Generate a test case sequence.

[0007] Among them, the step S1 is specifically as follows: first, filter out testable requirements from the software requirement document, and complete the information of the requirement statements containing relevant fields according to the requirement information completion table; finally, replace constants and variables according to the correspondence between constant variables in the comparison table to obtain a requirement statement document that can be directly used for named entity recognition.

[0008] The step S3 implements named entity recognition based on the BiLSTM-CRF model, and uses the SPO tagging method to perform part-of-speech tagging on the subject, predicate and object in the testable inner sentence in the requirement sentence, and clearly marks them as corresponding target elements, operation instructions and interaction information; wherein, in the tagging process, the character-level BIO tagging method is used; in the embedding layer of the BiLSTM-CRF model, the one-hot encoding is converted into a low-dimensional dense word vector using the Word2Vec model, which is horizontally spliced ​​with the part-of-speech feature vector to form a final word vector, and the final word vector obtained is input into the BiLSTM model in units of sentences, and the relationship between the input word vector and the output label is learned through training, and then the transfer matrix parameters are updated using the CRF model to predict the optimal label corresponding to each character.

[0009] In step S4, the number of B-SUBJECT, B-PREDICATE and B-OBJECT tags respectively represents the number of entities of the corresponding subject element, predicate element and object element in the identified single-line requirement statement; wherein the triple generation rule is as follows: If the number of B-SUBJECT and B-PREDICATE are both 1 and there is a B-OBJECT, it means that the demand statement is a subject-predicate-object structure, and a subject-predicate-object triple is generated; If the number of B-SUBJECT is 1, and the numbers of B-PREDICATE and B-OBJECT are equal and both greater than 1, it means that the demand sentence is a subject-predicate-object or predicate-object structure, and multiple subject-predicate-object structures with the same subject are generated; If the number of B-SUBJECT and B-PREDICATE are both 1 and there is no B-OBJECT, it means that the requirement statement is a subject-predicate structure, and a triple without an object element is generated; If the number of B-SUBJECTs is greater than 1, it means that the requirement statement is a multiple structure containing multiple triples. In this case, it is segmented until the number of B-SUBJECTs in the segmented part is 1. If the number of labels is not the above structure, it is considered that it does not comply with the triple generation rules.

[0010] In step S6, a keyword mapping table and a keyword antonym mapping table stored in a table form are used to identify words with the same meaning in the requirement statement with the same keyword, and antonym replacement is performed on the keywords for which reverse test cases need to be generated.

[0011] The step S7 is specifically as follows: traverse the requirement document and collect all variables, and then store the variable names and values ​​in the data dictionary; for repeated variables, update their variable values ​​in the data dictionary; and finally generate test cases based on the data dictionary and structured data.

[0012] The present invention also provides a method for automatically generating airborne embedded software test scripts based on a neural network, characterized in that the method is further implemented based on the test case automatic generation method according to any one of claims 1 to 8, and the steps of the method are as follows: First, read the file list of the file location storing the test case, for parsing the test case files one by one; Secondly, the XML file reading module is used to read the XML file used to store information such as the model structure, which is used to determine the validity and data type of the variables in the test case; Again, the test case is imported, and the corresponding information is extracted according to the keywords in the test case through the variable processing module, and the test script file is generated according to the grammatical rules of the test script.

[0013] The beneficial effects of the present invention are as follows: the present invention is particularly aimed at airborne embedded software, and can automatically generate test cases without changing the requirements document writing rules, which significantly improves the test efficiency, reduces manual intervention, and reduces the test cost; by preprocessing the original requirements document, the requirement statements that can generate test cases are accurately extracted, the requirement statements are generated into triples after named entity recognition, and the correct test cases are generated in combination with the requirement statement logic, and the test cases finally generated have a unified format and can be directly imported into the simulation test tool for execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of embodiments 1-3 of the present invention.

[0015] Figure 2 This is a flow chart of the requirement document preprocessing in Examples 1 and 2 of the present invention.

[0016] Figure 3 This is a flow chart for generating triples in embodiments 1 and 2 of the present invention.

[0017] Figure 4 This is a flowchart of a method for automatically generating a test script file according to Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0018] In order to clearly illustrate the technical features of this solution, this solution is described below through a specific implementation method.

[0019] The main purpose of the present invention is to convert the airborne software requirements described in natural language into test cases and test scripts that can be understood by computers. The present invention takes the software requirement document described in natural language as the starting point of automated testing, and combines the named entity recognition method based on deep learning with the test case generation method based on keywords, so that it can automatically generate test cases, and generate test script files that can be directly run in SCADE according to the conversion relationship between test cases and test scripts.

[0020] Example 1 The embodiment of the present invention provides a method for automatically generating test cases for airborne embedded software based on a neural network (such as Figure 1 As shown), the method comprises the following steps: Step S1: Filter out the test requirements from the software requirement document, complete the information of the requirement statements containing relevant fields according to the requirement information completion table, and replace the constants and variables in the requirement document according to the correspondence between the constants and variables in the comparison table; Step S2: select multiple lines of requirement statements and divide them into single-line requirement statements; Step S3: Use named entity recognition to identify all single-line requirement statements; Step S4: Processing the entity tag corresponding to each character in the recognized single-line requirement sentence to generate a triple containing subject, predicate, and object elements; Step S5: replace keywords on the triples; Step S6: assigning a value to the subject element according to the action of the predicate element and the value of the object element; Step S7: Generate a test case sequence.

[0021] Among them, step S1 is specifically as follows: first, filter out testable requirements from the software requirement document, and complete the information of the requirement statements containing relevant fields according to the requirement information completion table; finally, replace the constants and variables according to the correspondence between the constant variables in the comparison table to obtain a requirement statement document that can be directly used for named entity recognition.

[0022] Step S3 implements named entity recognition based on the BiLSTM-CRF model, and uses the SPO tagging method to tag the subject, predicate and object in the testable inner sentence in the requirement sentence, and clearly marks them as corresponding target elements, operation instructions and interaction information; wherein, in the tagging process, the character-level BIO tagging method is used; in the embedding layer of the BiLSTM-CRF model, the Word2Vec model is used to convert the one-hot encoding into a low-dimensional dense word vector, which is horizontally spliced ​​with the part-of-speech feature vector to form the final word vector, and the final word vector obtained is input into the BiLSTM model in units of sentences, and the relationship between the input word vector and the output label is learned through training, and then the CRF model is used to update the transfer matrix parameters to predict the optimal label corresponding to each character.

[0023] In step S4, the number of B-SUBJECT, B-PREDICATE and B-OBJECT tags respectively represents the number of entities of the corresponding subject element, predicate element and object element in the identified single-line requirement statement; wherein the triple generation rule is as follows: If the number of B-SUBJECT and B-PREDICATE are both 1 and there is a B-OBJECT, it means that the demand statement is a subject-predicate-object structure, and a subject-predicate-object triple is generated; If the number of B-SUBJECT is 1, and the numbers of B-PREDICATE and B-OBJECT are equal and both greater than 1, it means that the demand sentence is a subject-predicate-object or predicate-object structure, and multiple subject-predicate-object structures with the same subject are generated; If the number of B-SUBJECT and B-PREDICATE are both 1 and there is no B-OBJECT, it means that the requirement statement is a subject-predicate structure, and a triple without an object element is generated; If the number of B-SUBJECTs is greater than 1, it means that the requirement statement is a multiple structure containing multiple triples. In this case, it is segmented until the number of B-SUBJECTs in the segmented part is 1. If the number of labels is not the above structure, it is considered that it does not comply with the triple generation rules.

[0024] In step S6, a keyword mapping table and a keyword antonym mapping table stored in a table form are used to identify words with the same meaning in the requirement statement with the same keyword, and antonym replacement is performed on the keywords for which reverse test cases need to be generated.

[0025] Step S7 is specifically as follows: traverse the requirement document and collect all variables, then store the variable names and values ​​in the data dictionary; for repeated variables, update their variable values ​​in the data dictionary; and finally generate test cases based on the data dictionary and structured data.

[0026] Example 2 See also Figure 1 The present invention proposes a method for automatically generating airborne embedded software test cases based on a neural network. The specific steps of the method are as follows: Step 1: In the process of test case generation, it is necessary to screen out testable requirements and replace the constants and variables in the requirements document according to the correspondence between the constant variables in the comparison table.

[0027] See also Figure 2 ,The requirement document preprocessing process is divided into the following steps: First, filter out the testable requirements; then complete the information of the requirement statements containing relevant fields according to the requirement information completion table; finally, replace the constants and variables according to the correspondence between the constant variables in the comparison table to obtain the requirement statement document that can be directly used for named entity recognition.

[0028] Step 2: Since the named entity recognition model needs to segment multi-line requirement statements during the training process, single-line requirement statements and multi-line requirement statements need to be processed separately before entity recognition is performed on the requirement statements. For single-line requirement statements, named entity recognition is performed directly; for multi-line requirement statements, the statements are first segmented according to the line breaks, and then processed sentence by sentence.

[0029] Step 3: Use the named entity recognition model to identify the requirement statements.

[0030] The testable statements in the requirement statements have an obvious subject-predicate-object structure. The SPO annotation method (Subject, Predicate, Object) is used to mark the target elements, operation instructions, and interaction information of the test case as three types of labels for the training model.

[0031] The named entity recognition corpus of this embodiment is annotated at the character level based on words to avoid errors in named entity boundary recognition caused by direct word segmentation. Since the named entities of the onboard software requirement statements are usually not complex and there is little overlap between adjacent entities, this embodiment uses the BIO annotation method.

[0032] According to the characteristics of the airborne display system requirement statement, the present invention adopts the BiLSTM-CRF model, which is currently widely used and has good effects in named entity recognition tasks, and combines the part-of-speech features to construct a BiLSTM-CRF model that integrates the part-of-speech features, thereby improving the feature expression ability of the word vector. In the embedding layer, the Word2Vec model is used to convert the one-hot encoding into a low-dimensional dense word vector. These word vectors are spliced ​​with the part-of-speech feature vectors obtained by the same method to form the final word vector. Since the order of magnitude of the part-of-speech tagging is very different from the order of magnitude of the characters, the word vector dimension of the part-of-speech is set to 10% of the character word vector. The obtained word vector is input into the BiLSTM model in units of sentences, and the relationship between the input word vector and the output label is learned through training. Then, the CRF model is used to update the transfer matrix parameters to predict the optimal label corresponding to each character.

[0033] The BiLSTM model uses a two-layer LSTM model, forward and backward, to capture sentence information from two directions. The CRF model uses the feature vector generated by the BiLSTM model as input and improves the accuracy of named entity recognition by learning the relationship between the input word vector and the entity label. In this section, the Word2Vec model is used to convert characters into word vectors, and the word vectors are horizontally concatenated with the part-of-speech feature vectors, which are then used as the input of the BiLSTM-CRF model for neural network training.

[0034] Step 4: After the named entity recognition of the requirement statement, the entity needs to be further processed to generate triples containing subject, predicate and object elements. For example, the preprocessed requirement statement "When ipEIIs is 3 and ipEIIs is 1, the N1 label should be displayed in font size 3." extracts the triples (ipEIIs, is, 3) and (ipEIIs, is, 1). To obtain the triples, the entity labels corresponding to each character in the requirement statement need to be processed accordingly.

[0035] In a triple, the subject is the variable name, indicating the object to be assigned a value; the predicate indicates the action related to the variable assignment, such as "is", "is greater than", "increase", etc.; the object is the value related to the variable. The assignment of variables in a test case is usually determined by the value in the object.

[0036] Since the BX tag represents the starting tag of a certain type of named entity, the number of BX tags can be used to represent the number of corresponding named entities in the sentence. If the number of B-SUBJECT and B-PREDICATE are both 1 and there is a B-OBJECT, it means that the demand sentence is a subject-predicate-object structure, and a subject-predicate-object triple is generated at this time; if the number of B-SUBJECT is 1, and the number of B-PREDICATE and B-OBJECT is equal and greater than 1, it means that the demand sentence is a subject-predicate-object and predicate-object structure, and multiple subject-predicate-object structures with the same subject are generated at this time; if the number of B-SUBJECT and B-PREDICATE are both 1 and there is no B-OBJECT, it means that the demand sentence is a subject-predicate structure, and a triple without an object element is generated at this time; if the number of B-SUBJECT is greater than 1, it means that the demand sentence is a multiple structure containing multiple triples, and it is segmented until the number of B-SUBJECTs in the segmented part is 1; if the number of tags is not composed as described above, it is deemed to not comply with the triple generation rules. See the triple generation process for details. Figure 3 .

[0037] Step 5: After generating the triple, the subject element needs to be assigned a value according to the action of the predicate element and the value of the object element. Usually, the same semantics in the predicate element has a variety of different representations, for example, "greater than", "higher than", "more than", etc. all mean greater than a certain value. In order to facilitate further processing of the triple, the present invention uses a keyword mapping table to use the same keyword to represent words with the same meaning in the demand statement.

[0038] Step 6: Test case sequence generation. In the process of traversing the requirement document and generating test cases, the present invention collects all the variables that appear, stores these variables and their values ​​in the data dictionary, and updates the values ​​of the variables that appear repeatedly. The function of the data dictionary is to record the data in the currently generated test case and provide a pre-sequence for generating subsequent test cases. For example, the requirement statement "When the value of ipFADECN1L involving display hysteresis increases, the left engine N1 pointer should rotate clockwise." A correct test case cannot be generated from this requirement statement alone. In this case, the value of the ipFADECN1L parameter can be obtained from the data dictionary, and a larger value can be taken on this basis to obtain a test case corresponding to this requirement.

[0039] After replacing the keywords of the triples, the present invention uses structured data to record the relevant information extracted in the above steps. The next step is to generate a test case sequence based on the data dictionary and the structured data. That is, different processing methods are used for different predicate keywords to obtain the final test case.

[0040] Example 3 The embodiment of the present invention provides a method for automatically generating an onboard embedded software test script based on a neural network. The method is further implemented based on the test case obtained in Example 1 or Example 2. Specifically: Figure 4 , the test script file is automatically generated as follows: 1. First, read the file list where the test case files are stored, and parse the test case files one by one; 2. Through the XML file reading module, read the XML file used to store information such as model structure to determine the validity and data type of variables in the test case; 3. Import the test case, extract the corresponding information according to the keywords in the test case through the variable processing module, and generate the script file according to the grammatical rules of the test script.

[0041] After obtaining the variable name and value, first determine whether the variable name is valid in the SCADE model based on the variable information in the XML file. If it is invalid, output the comment "not found this signal" in the script file to prompt the tester to analyze the error. Since the bool type variable value is usually represented by the numbers "0" and "1", it is necessary to filter out the bool type value and process it separately. If the variable is a bool type value, assign "false" or "true" according to "0" or "1". If the variable is a numeric value, assign the corresponding value directly.

[0042] Model coverage analysis of airborne software is implemented through MTC. Before the model coverage acquisition process is performed, a test script file needs to be input, and the test script needs to be written according to the test case file. Due to the huge number of test cases, manual conversion of test cases to test scripts will consume a lot of manpower and time, and the test case automatic generation method provided by Example 1 or Example 2 can save a lot of manpower and time. Therefore, Example 1 or 2 is combined with the tool for automatically generating test script files in this embodiment (see Figure 1 ), can greatly improve work efficiency and reduce labor costs.

[0043] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automatically generating test cases for airborne embedded software based on neural network, characterized in that: The method comprises the following steps: Step S1: Filter out the test requirements from the software requirement document, complete the information of the requirement statements containing relevant fields according to the requirement information completion table, and replace the constants and variables in the requirement document according to the correspondence between the constants and variables in the comparison table; Step S2: select multiple lines of requirement statements and divide them into single-line requirement statements; Step S3: Use named entity recognition to identify all single-line requirement statements; Step S4: Processing the entity tag corresponding to each character in the recognized single-line requirement sentence to generate a triple containing subject, predicate, and object elements; Step S5: replace keywords on the triples; Step S6: assigning a value to the subject element according to the action of the predicate element and the value of the object element; Step S7: Generate a test case sequence.

2. The automatic generation method according to claim 1, characterized in that: The step S1 is specifically as follows: first, testable requirements are screened out from the software requirement document, and information is completed for the requirement statements containing relevant fields according to the requirement information completion table; finally, constants and variables are replaced according to the correspondence between constant variables in the comparison table to obtain a requirement statement document that can be directly used for named entity recognition.

3. The automatic generation method according to claim 1, characterized in that: The step S3 implements named entity recognition based on the BiLSTM-CRF model, and uses the SPO tagging method to tag the subject, predicate and object in the testable inner sentences in the requirement sentence, and clearly marks them as corresponding target elements, operation instructions and interaction information.

4. The automatic generation method according to claim 3, characterized in that: In the tagging process, the character-level BIO tagging method is used; in the embedding layer of the BiLSTM-CRF model, the Word2Vec model is used to convert the one-hot encoding into a low-dimensional dense word vector, which is horizontally concatenated with the part-of-speech feature vector to form the final word vector. The final word vector is input into the BiLSTM model in units of sentences. The relationship between the input word vector and the output label is learned through training, and then the CRF model is used to update the transfer matrix parameters to predict the optimal label corresponding to each character.

5. The automatic generation method according to claim 1, characterized in that: In step S4, the number of B-SUBJECT, B-PREDICATE and B-OBJECT tags are used to represent the number of entities of the corresponding subject element, predicate element and object element in the identified single-line requirement statement.

6. The automatic generation method according to claim 5, characterized in that: In step S4, the triple generation rule is as follows: If the number of B-SUBJECT and B-PREDICATE are both 1 and there is a B-OBJECT, it means that the demand statement is a subject-predicate-object structure, and a subject-predicate-object triple is generated; If the number of B-SUBJECT is 1, and the numbers of B-PREDICATE and B-OBJECT are equal and both greater than 1, it means that the demand sentence is a subject-predicate-object or predicate-object structure, and multiple subject-predicate-object structures with the same subject are generated; If the number of B-SUBJECT and B-PREDICATE are both 1 and there is no B-OBJECT, it means that the requirement statement is a subject-predicate structure, and a triple without an object element is generated; If the number of B-SUBJECTs is greater than 1, it means that the requirement statement is a multiple structure containing multiple triples. In this case, it is segmented until the number of B-SUBJECTs in the segmented part is 1. If the number of labels is not the above structure, it is considered that it does not comply with the triple generation rules.

7. The automatic generation method according to any one of claims 1 to 6, characterized in that: In step S6, a keyword mapping table and a keyword antonym mapping table stored in a table form are used to identify words with the same meaning in the requirement statement with the same keyword, and antonym replacement is performed on the keywords for which reverse test cases need to be generated.

8. The automatic generation method according to any one of claims 1 to 6, characterized in that: The step S7 is specifically as follows: traverse the requirement document and collect all variables, and then store the variable names and values ​​in the data dictionary; for repeated variables, update their variable values ​​in the data dictionary; and finally generate test cases based on the data dictionary and structured data.

9. A method for automatically generating test scripts for airborne embedded software based on neural networks, characterized in that: The method is further implemented based on the test case automatic generation method according to any one of claims 1 to 8, and the steps of the method are as follows: First, read the file list of the file location storing the test case, for parsing the test case files one by one; Secondly, the XML file reading module is used to read the XML file used to store information such as the model structure, which is used to determine the validity and data type of the variables in the test case; Again, the test case is imported, and the corresponding information is extracted according to the keywords in the test case through the variable processing module, and the test script file is generated according to the grammatical rules of the test script.

Images