Test coverage rate evaluation method and device based on fine-grained text understanding

Abstract

The invention discloses a test coverage rate evaluation method and device based on fine-grained text understanding. The method comprises the steps of obtaining a demand document and a test case report of a to-be-tested software project; respectively converting the demand document and the test case report into a plurality of pieces of structured information; based on the structured information, generating a plurality of demand tested events corresponding to the demand document and a plurality of case tested events corresponding to the test case report; according to the similarity between the demand tested events and the use case tested events, obtaining the number of the tested events of which the demands are tested; and obtaining the test coverage rate of the to-be-tested software engineering based on the total number of the required tested events and the number of the tested events of which the requirements are tested. According to the method, software quality evaluation can be completed on the premise of solving the problem of accurate matching for test events.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a test coverage evaluation method and device based on fine-grained text understanding. Background Art

[0002] Software quality is closely related to the life of software products. As software becomes an indispensable infrastructure in people's lives, the requirements for software quality are gradually increasing. Designing and executing test cases to complete software testing is an important measure to ensure software quality. Through the test coverage index, testers can have an intuitive evaluation of software quality. Due to the strong interpretability and adjustability of natural language, software artifacts (software engineering byproducts such as data models, requirement documents and test case reports other than system code) described in natural language (NL) are widely present in actual industrial production environments. Therefore, research based on natural language requirements and use cases remains a focus. However, due to 1) requirements are written for customers, test cases are written for developers, and documents come from different teams, there are large differences in terminology; 2) the requirements description text of large systems is detailed and complex, and has different functions in different contexts, which increases the difficulty of fine-grained understanding of test texts. 3) The inclusion of complex elements related to test tasks in test requirements and test cases brings great challenges to accurate coverage evaluation. Solving the above three challenges and realizing an automated test coverage evaluation method and system is of great significance to ensuring software quality. The test coverage indicator can provide guidance for testers, assist testers in managing and tracking test progress, and provide data support for software system acceptance.

[0003] With the rapid development of Natural Language Processing (NLP) technology, we can obtain key information in complex sentences through information extraction (IE) methods to solve the problem of fine-grained understanding. Information extraction includes two subtasks: named entity recognition (NER) and relation extraction (RE). With the deepening of research, it is found that the pipeline workflow of first entity extraction and then relation extraction is prone to defects such as error transmission that affect the performance of the method. Therefore, researchers have further proposed a joint entity-relation extraction (Joint Entity-Relation Extraction) method, which implements two tasks through one model. The model learns and optimizes entity extraction and relationship extraction at the same time to achieve better performance on the training data set. The goal of the entity-relation joint extraction method is to extract triple structured information related to the task definition from unstructured or semi-structured text data. The triple structured information is represented as<Head,Relation,Tail> , where Head and Tail represent two entities respectively, and Relation is the relationship between the two entities.

[0004] Test coverage evaluation is the process of comparing and statistically analyzing the completion of software requirement tests with test cases. The work of comparing the matching relationship between requirements and use case texts can be achieved based on text similarity research. As a basic method in the field of natural language processing, text similarity calculation methods are widely used in many tasks. For example, in the research related to duplicate report identification in the field of software engineering, by measuring the text similarity between different software problem report documents, similar problem reports can be screened and removed for software maintenance personnel, reducing the time cost of maintenance personnel to screen and read reports. Similarly, due to the importance of requirements in software engineering, it drives the development and testing process of the system. The test tasks in the test case must be mentioned in the requirements. Therefore, the similarity between each requirement and the use case text can be calculated, and the similarity can be used to determine whether the requirement is covered by the test case. Summary of the invention

[0005] In order to solve the fine-grained understanding problem and precise matching challenge mentioned in the background, the present invention discloses a test coverage evaluation method and device based on fine-grained text understanding. The method is based on entity-relationship joint extraction and text similarity calculation research technology or method, which can complete software quality evaluation on the premise of solving the precise matching problem for test events. .

[0006] The main contents of the present invention are as follows:

[0007] A test coverage evaluation method based on fine-grained text understanding, the method comprising:

[0008] Obtain the requirement documents and test case reports of the software project to be tested;

[0009] The requirement document and the test case report are respectively converted into a plurality of structured information; wherein the structure of the structured information is entity-relationship-entity;

[0010] Based on the structured information, a plurality of demand tested events corresponding to the demand document and a plurality of use case tested events corresponding to the test case report are generated; wherein the event templates of the demand tested events and the use case tested events include all categories of entities;

[0011] According to the similarity between the event under test of the requirement and the event under test of the use case, the number of the event under test of the requirement is obtained;

[0012] Based on the total number of tested events of the requirement and the number of tested events of the tested requirement, the test coverage of the software project to be tested is obtained.

[0013] Further, the category of the entity is one of test operation, test object, test tool, test condition and test status; the type of the relationship is one of act on, satisfy, exist in and assist in.

[0014] Furthermore, the requirement document is converted into a plurality of structured information, including:

[0015] Preprocessing the demand document; wherein the preprocessing includes: removing noise in the demand document;

[0016] Obtaining each phrase in the preprocessed demand document through the open source tool HanLP to generate a part-of-speech tag, and expanding the part-of-speech tag according to the length of the phrase to obtain the part-of-speech information corresponding to each word in the preprocessed demand document;

[0017] Based on the part-of-speech information, the sentence word embedding representation vector T1, T2, ..., T of each sentence in the preprocessed demand document is calculated. n , and the expanded part-of-speech tag sequence P1, P2, ..., P n , n represents the sentence length;

[0018] The sentence words are embedded into vectors T1, T2, ..., T n Input the pre-trained Bert model and obtain the output vector CLS of the Bert model and the semantic representation vector of the hidden layer output;

[0019] Under the restriction of the longest span parameter, iteratively select two elements in the semantic representation vector to construct a candidate entity set corresponding to the sentence;

[0020] Combine the output vector CLS and the expanded part-of-speech tag sequence P1, P2, ..., P n , calculating the probability that the type of each candidate entity in the candidate entity set belongs to the entity type, to obtain an entity set;

[0021] Based on the context of the two entities in the entity set, a relationship between the two entities is extracted.

[0022] Furthermore, the preprocessing of the demand document includes:

[0023] Noise in the requirement document is removed; wherein the noise includes: stop words and verbs, and the verbs include: provide, test, and support.

[0024] Furthermore, the output vector CLS is combined with the expanded part-of-speech tag sequence P1, P2, ..., P n , calculating the probability that the type of each candidate entity in the candidate entity set belongs to the entity type, to obtain an entity set, including:

[0025] Use the maximum pooling function to form the semantic representation vector of the candidate entity;

[0026] Based on the expanded part-of-speech tag sequence P1, P2, ..., P n , obtain the part-of-speech representation vector of the candidate entity;

[0027] After concatenating the semantic representation vector, the part-of-speech representation vector and the output vector CLS of the candidate entity, classification is performed based on the concatenation result to obtain the probability that the type of the candidate entity belongs to the entity type;

[0028] The probability is compared with a first set threshold to determine whether the candidate entity is an entity.

[0029] Furthermore, extracting the relationship between the two entities based on the context of the two entities in the entity set includes:

[0030] According to the positions of the two entities in the sentence, at least one context text block is obtained;

[0031] Calculating a semantic vector representation of the context text block;

[0032] The semantic vector representations corresponding to the two entities and the semantic vector representations of all context text blocks are input into a multi-layer neural network model to obtain the relationship between the two entities.

[0033] Further, obtaining the number of tested events for the tested requirement according to the similarity between the tested event for the requirement and the tested event for the use case includes:

[0034] Vectorizing the demand tested events and the use case tested events;

[0035] Using an average pooling function to process the vectors of the demand tested event and the use case tested event, and calculating the similarity between the processing results;

[0036] Comparing the similarity with a second set threshold to determine whether the required measured event is met;

[0037] Count the number of tested events for the requirements that have been tested.

[0038] A test coverage evaluation device based on fine-grained text understanding, the device comprising:

[0039] Data acquisition module, used to obtain the requirement documents and test case reports of the software project to be tested;

[0040] An information extraction module, used to convert the requirement document and the test case report into a plurality of structured information respectively; wherein the structure of the structured information is entity-relationship-entity;

[0041] An event generation module, used to generate a number of demand tested events corresponding to the demand document and a number of use case tested events corresponding to the test case report based on the structured information; wherein the event templates of the demand tested events and the use case tested events include all categories of entities;

[0042] The test evaluation module is used to obtain the number of tested events of the tested requirements based on the similarity between the tested events of the requirements and the tested events of the use cases; and to obtain the test coverage of the software engineering to be tested based on the total number of tested events of the requirements and the number of tested events of the tested requirements.

[0043] A computer device, characterized in that the computer device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, any of the above-mentioned methods is implemented.

[0044] A computer-readable storage medium, characterized in that computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by a processor, any of the methods described above is implemented.

[0045] Compared with the prior art, the present invention has at least the following beneficial effects:

[0046] 1. Aiming at the problem of fine-grained understanding of test text that has not been solved in the prior art, the present invention innovatively defines five entity types and four relationship types in the test field for test field data in software engineering. The above definitions can cover the scope of multiple functional or non-functional requirements of software testing and can fully achieve the goal of test coverage calculation. The five entity categories defined are: object, operation, tool, condition and state. The four relationship categories defined are divided into: operation-acts on-object, tool-assists on-object, condition-satisfies-object and state-exists on-object.

[0047] 2. The present invention extracts and refines the information related to the test in the original natural language test domain text by extracting the entities and relations defined above and implementing a domain entity-relationship joint extraction model with enhanced semantic information, thereby solving the problem of fine-grained understanding. Specifically, it is as follows:

[0048] 1) Provide an entity-relationship joint extraction dataset in a test domain.

[0049] 2) Provide a domain entity-relationship joint extraction model with enhanced semantic information. The model includes word embedding layer, candidate entity generation layer, entity extraction layer and relationship layer. The innovations of the model are a) span-based candidate entity generation strategy; b) innovative introduction of part-of-speech tag information, context information and semantic information of candidate entities for entity extraction; c) innovative use of local context information and entity semantic information to complete relationship extraction. The model continuously optimizes the extraction performance of entities and relationships through the entity-relationship extraction joint training method.

[0050] 3) Provide a test event parsing method to parse and transform the results of the entity relationship joint extraction in the previous step to facilitate the implementation of subsequent test case coverage evaluation.

[0051] 3. Based on the extraction and parsing results - test events, the present invention designs and implements a test coverage evaluation method for test event matching to solve the problem of exact matching. The core of this method is a coverage evaluation model based on domain text similarity calculation. Through the similarity between demand use cases, the parts of all use cases that are associated with the requirements are screened out, and the proportion of requirements with associated use cases in the requirements is statistically calculated to complete the test coverage evaluation process. Specifically, the present invention not only provides a test domain text matching data set and a test domain text similarity calculation model; it also implements an automated test coverage evaluation algorithm based on test events to complete software quality evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 The overall flow chart of the test coverage evaluation method based on fine-grained text understanding.

[0053] Figure 2 Framework diagram of the domain entity-relationship joint extraction model for semantic information enhancement.

[0054] Figure 3 An embodiment of test coverage calculation based on fine-grained text understanding. DETAILED DESCRIPTION

[0055] The present invention is further described in detail below with reference to the accompanying drawings and embodiments:

[0056] 1. Data preprocessing and part-of-speech information addition

[0057] First, through data preprocessing, stop words and other noise data in the text are deleted. In particular, through empirical observation of test field data, there are a large number of verbs such as "provide", "test", and "support" in requirement texts and test case reports. These words do not play a big role in identifying test events. On the contrary, because of their high frequency of appearance, they cause trouble for the use of part-of-speech tags. Therefore, this method deletes the above three phrases.

[0058] On this data set, the open source tool HanLP is used to generate part-of-speech tags for each test text. Since Chinese parts of speech are in phrases, the present invention expands the part-of-speech tags by phrase length. For example, a Chinese phrase "folder" is annotated with a part-of-speech "NN" through an open source tool, and the complete part-of-speech information after length expansion is "NN, NN, NN". The part-of-speech tags refer to the Pennsylvania Chinese Treebank, and the part-of-speech tags are converted into corresponding numbers through a dictionary to facilitate the use of subsequent models. The corresponding table of all part-of-speech numbers in the method is shown in Table 1.

[0059] Table 1 Part-of-speech tag dictionary number correspondence table

[0060]

[0061]

[0062] 2. Construction of Domain Entity-Relation Joint Extraction Model with Semantic Information Enhancement

[0063] The joint extraction model proposed in this invention includes four layers such as Figure 3 As shown in the figure, they are word embedding layer, candidate entity generation layer, entity extraction layer and relationship extraction layer. The main functions of each layer are as follows:

[0064] 1) Word Embedding Layer

[0065] At this layer, the model first converts all sentences and part-of-speech information into digital vectors using word embedding methods. For sentences of length n, the sentence word embedding representation vectors are: T1, T2, ..., T n ; The vector representation of the expanded part-of-speech tag sequence after sentence word embedding is: P1, P2, ..., P n . Converting from the text domain to the digital domain is beneficial to the representation learning of the model.

[0066] After that, the method uses the pre-trained Bert model, which is a commonly used representation model in the NLP field and shows strong robustness in different fields. Through the pre-trained model, the method takes the hidden layer output as the semantic representation vector of the text: V1, V2, ..., V n , the hidden layer output vector after Bert encoding contains the semantic information of the candidate entity.

[0067] 2) Candidate entity generation layer

[0068] The strategy for generating candidate entities is as follows: under the constraint of the longest span parameter, iteratively select all consecutive word blocks in the input semantic representation vector that do not exceed the span length to construct a candidate span phrase set [[V1], [V1, V2], [V2, V3], [V1, V2, V3], …], and each span word block is regarded as a candidate entity represented as CE i , where the subscript i is a number not exceeding the sentence length n, indicating the position of the candidate entity representation in the candidate span phrase set. At the same time, the corresponding candidate phrase set generates a set of part-of-speech span phrases for the candidate entity representation [[P1], [P1, P2], [P2, P3], [P1, P2, P3], …]. Based on empirical observations of the dataset, the span length is empirically set to 30 in this model, because most entities do not have more than 30 words, and a span that is too long will cause a decrease in model training speed and performance.

[0069] 3) Entity extraction layer

[0070] In the next step, the model receives the representation of all candidate entities, and determines whether the candidate entity is a meaningful entity based on various representation information. On the basis of determining that it is an entity, it further determines and classifies which of the five defined entity categories the entity belongs to.

[0071] First, for a candidate entity semantic representation vector in the candidate entity set, such as [V2, V3], the present invention uses the Max Pooling function to retain key semantic information and filter noise to form a candidate entity vector representation CE3. Similarly, the part-of-speech representation vector of the candidate entity is processed, and the corresponding part-of-speech representation vector POS3 is obtained from the part-of-speech tag representation vector [P2, P3] of the candidate entity.

[0072] Afterwards, the model obtains the output vector CLS of the Bert model to represent the global context information of the sentence, which is the weighted sum of the hidden representations of all tags in the test text, and has been proven to effectively improve the performance of classification tasks.

[0073] Finally, the semantic representation vector CE3 of the candidate entity, the part-of-speech representation vector POS3 of the candidate entity, and the context information representation vector CLS are concatenated and input into the multi-layer neural network classification model for learning. The output layer vector of the classification model uses the SoftMax function to determine whether the candidate entity is a meaningful entity, and outputs the probability value that the candidate entity belongs to a certain entity category.

[0074] 4) Relationship extraction layer

[0075] In the relation extraction layer, first, obtain any two entity CEs determined in the entity extraction layer i and CE j The semantic representation vector of (the subscripts i and j represent the positions of the two entities in the candidate entity set). Then, the context information in the sentence other than the entity representation is obtained. Since the two entities divide a sentence into at most three context text blocks C1, C2, and C3, the context information other than the entity representation is subjected to maximum pooling (Max Pooling) to obtain the key information therein. Finally, through the comprehensive use of the above information, all representations (including context information representation and entity representation) are also input into the multi-layer neural network model for learning. The output layer outputs the result after the SoftMax function transforms the result, and the judgment of the relationship category is completed. The output result is the probability of belonging to the relationship category defined in the first content of the invention or the absence of a relationship.

[0076] By reproducing the four-layer structure of the model, a domain entity-relationship joint extraction model with enhanced semantic information was constructed. After multiple rounds of learning on the test domain extraction dataset, all parameter nodes of the model were saved. The obtained model can be loaded during prediction to complete the task requirements of entity-relationship joint extraction.

[0077] In order to achieve the goal of software test coverage evaluation, the present invention defines five entity categories and four relationship categories related to testing through observation of natural language requirement descriptions and test cases. The definitions of entities and relationships are shown in Table 2.

[0078] Regarding the definition of entities in the method, first, the test case is driven by the characteristics in the requirements, and the characteristics specify the behavior of one or more components according to the current conditions. In this sense, the key entities in the characteristics will also be reflected in the test case description. Therefore, the method determines three entity categories "test object", "test operation" and "test condition" respectively. Secondly, in order to meet certain functional or compatibility requirements, the requirements document clearly defines the methods used for testing, such as using a specific test software (for example: using Docker software to test the operation of the virtualization environment), so the method further adds the "test tool" entity category. Finally, when performing performance testing, the use case document clearly defines in which states the system needs to be in to complete the test, such as "testing when the system is in the boot state...", so the "test state" entity category is obtained.

[0079] Regarding the definition of relations in the method, first, the extracted relation categories are related to the categories of four entities in addition to the core entity "test object" category. The appearance of the "test tool" entity is generally accompanied by the existence of the "tool-assists-object" relationship that serves a specific "test object". Similarly, there is a strong association between "test operation" and the relationship "operation-acts-on-object", "test status" and the relationship "status-exists-in-object", and "test condition" and the relationship "condition-satisfied-by-object".

[0080] Table 2 Entity and relationship category definitions

[0081]

[0082] In order to train the domain entity-relationship joint extraction model enhanced by semantic information, the method needs to construct an extraction data set of the test domain. Based on the entity and relationship types defined above, the method collects test case data of eight test projects. These eight test cases belong to three different test types and correspond to three different test requirements. Through data preprocessing, the duplicate data and noise data are deleted. The method collects a total of 4252 valid data. After multiple rounds of annotation by three researchers, the method data set is obtained. On this data set, the performance of the present invention is shown in Table 3.

[0083] Table 3 Performance of the extraction model

[0084]

[0085] 3. Generate test events based on extraction results

[0086] After the joint extraction model, the original text sentence is converted into structured information, but the information at this time is still difficult to understand, so the present invention introduces test events to parse the extraction results. According to the definition in the first invention content, the test text is centered on the test object and is associated with the other four entities under the restriction of the relationship. Among them, the extraction result is for the original sentence. If there is an entity in the sentence, the subscript indicates that the i-th to j-th phrases constitute an entity; if there is a relationship between the entities, the subscript of the entity indicates the relationship between the entities. Therefore, the test event template can be defined as follows:

[0087] Test event = <test operation, test object, test tool, test condition, test status>

[0088] The parsing process of the extraction results is as follows: starting from the relationship, when there is a relationship in the extraction result, the two entities of the relationship are parsed and filled into the test event template according to the relationship type; when there is no relationship, the extracted entities are directly parsed and filled into the test event template. Non-existent entity elements are set to NULL (meaning the content is empty).

[0089] Take a requirement "Whether the system can create a new folder." as an example, after the joint extraction model extraction in Implementation Method 2, the relationship "operation-act-on-object" is obtained, and the two entities associated with the relationship are "new (test operation)" and "folder (test object)". After the above result parsing process, the test event obtained is: <new, folder, NULL, NULL, NULL>.

[0090] 4. Construction of test domain text similarity calculation model

[0091] The test domain text similarity calculation model proposed in the present invention is constructed based on the twin network architecture. It is fine-tuned through computer domain data sets on the basis of the pre-trained model, and the obtained model has good performance in this field.

[0092] The implementation process of the test domain text similarity calculation model is as follows: for any requirement test event and a use case test event input, the requirement text is represented as re, and the use case text is represented as tc. The requirement text and the use case text are input into the Sentence-bert pre-training model, and the requirements and use cases are converted into vector representations through the pre-training model. After that, the average pooling function is used to process the requirement and use case test event vectors. The reason for choosing average pooling is that under the premise that the test event has highly condensed the original text, average pooling can better utilize the information in all test events. Finally, the cosine function is used to calculate the similarity between the two vectors. This function is a commonly used formula for calculating the similarity of text vectors, and has been proven to have good performance in many research works. The similarity calculation formula is described as:

[0093] similarity = cosine(re, tc)

[0094] The domain fine-tuning process of the model is: compare the results of the model calculation with the labels in the computer field text matching dataset, use the mean square error to measure the difference between the two, and achieve the purpose of fine-tuning the model by continuously optimizing the error. The model parameters and nodes with the best performance evaluated through training are saved.

[0095] In order to train the test domain text similarity calculation model, the method needs to build a test text matching dataset. The annotators manually proofread the association management between eight test case reports and the corresponding test requirements, and generated a positive sample (matching data) for the requirements and cases with an associated relationship. For the data without an associated relationship (unmatched data), the negative sampling method was used to generate negative samples. Through the above method, this method collected a test text matching dataset with a data volume of 5125. The performance of the model on the dataset of this method is shown in Table 4.

[0096] Table 4 Performance of matching models

[0097]

[0098] 5. Test Coverage Evaluation Method Based on Test Event Similarity Calculation

[0099] For all requirements and use cases of the same project, use the similarity calculation model in step 4 to calculate the similarity between the two, record the requirements that can be associated with the test cases, these requirements are the tested requirements, and count the number of all tested requirements to calculate the test coverage index. The test coverage calculation formula is described as:

[0100]

[0101] The criterion for determining whether a requirement is tested is that the text similarity calculation result between the requirement and the test event corresponding to the test case is greater than the empirical parameter 0.8.

[0102] The specific implementation of this method is as follows Figure 3 shown.

[0103] Through step one, the input requirements and use case texts are preprocessed, and the part-of-speech tag information of the text is added using HanLP. The stop words in the test text are deleted ("test" in the use case example is deleted) to obtain part-of-speech information of the same length as the original text.

[0104] Through step 2, the domain entity-relationship joint extraction model enhanced by semantic information is used to extract information from requirements and test cases respectively, and the extracted information output in JOSN format is obtained.

[0105] Through step three, the extraction results in JSON format are analyzed and transformed, and the extracted information is filled into the test event template defined in the invention to obtain the requirement test event "<access, files and folders in mobile device, NULL, NULL, NULL, NULL>" and the use case test event "<open, document in storage medium, NULL, NULL, NULL, NULL>".

[0106] Through step 4, the text similarity between the requirement test event and the use case test event is calculated using the test domain similarity calculation model constructed by the invention.

[0107] Through step five, under the limitation of 0.8 similarity threshold, it is judged that the two are similar, which means that requirement 023 and use cases 003-005 are testing similar test targets. Therefore, this requirement is tested by the test case. According to the definition in the invention content, this requirement is met, and the number of tested requirements is increased by 1.

[0108] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the present disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof.

Claims

1. A test coverage evaluation method based on fine-grained text understanding, characterized in that: The method comprises: Obtain the requirement documents and test case reports of the software project to be tested; The requirement document and the test case report are respectively converted into a plurality of structured information; wherein the structure of the structured information is entity-relationship-entity; Based on the structured information, a plurality of demand tested events corresponding to the demand document and a plurality of use case tested events corresponding to the test case report are generated; wherein the event templates of the demand tested events and the use case tested events include all categories of entities; According to the similarity between the event under test of the requirement and the event under test of the use case, the number of the event under test of the requirement is obtained; Based on the total number of tested events of the requirement and the number of tested events of the tested requirement, the test coverage of the software project to be tested is obtained.

2. The method according to claim 1, characterized in that The category of the entity is one of test operation, test object, test tool, test condition and test status; the type of the relationship is one of act on, satisfy, exist in and assist in.

3. The method according to claim 1, characterized in that The converting the requirement document into a plurality of structured information includes: Preprocessing the demand document; wherein the preprocessing includes: removing noise in the demand document; Obtaining each phrase in the preprocessed demand document through the open source tool HanLP to generate a part-of-speech tag, and expanding the part-of-speech tag according to the length of the phrase to obtain the part-of-speech information corresponding to each word in the preprocessed demand document; Based on the part-of-speech information, the sentence word embedding representation vector T1, T2, ..., T of each sentence in the preprocessed demand document is calculated. n , and the expanded part-of-speech tag sequence P1, P2, ..., P n , n represents the sentence length; The sentence words are embedded into vectors T1, T2, ..., T n Input the pre-trained Bert model and obtain the output vector CLS of the Bert model and the semantic representation vector of the hidden layer output; Under the restriction of the longest span parameter, iteratively select two elements in the semantic representation vector to construct a candidate entity set corresponding to the sentence; Combine the output vector CLS and the expanded part-of-speech tag sequence P1, P2, ..., P n , calculating the probability that the type of each candidate entity in the candidate entity set belongs to the entity type, to obtain an entity set; Based on the context of the two entities in the entity set, a relationship between the two entities is extracted.

4. The method according to claim 3, characterized in that The preprocessing of the demand document includes: Noise in the requirement document is removed; wherein the noise includes: stop words and verbs, and the verbs include: provide, test, and support.

5. The method according to claim 3, characterized in that The output vector CLS is combined with the expanded part-of-speech tag sequence P1, P2, ..., P n , calculating the probability that the type of each candidate entity in the candidate entity set belongs to the entity type, to obtain an entity set, including: Use the maximum pooling function to form the semantic representation vector of the candidate entity; Based on the expanded part-of-speech tag sequence P1, P2, ..., P corresponding to the candidate entity n , obtain the part-of-speech representation vector of the candidate entity; After concatenating the semantic representation vector, the part-of-speech representation vector and the output vector CLS of the candidate entity, classification is performed based on the concatenation result to obtain the probability that the type of the candidate entity belongs to the entity type; The probability is compared with a first set threshold to determine whether the candidate entity is an entity.

6. The method according to claim 5, characterized in that The extracting the relationship between the two entities based on the context of the two entities in the entity set includes: According to the positions of the two entities in the sentence, at least one context text block is obtained; Calculating a semantic vector representation of the context text block; The semantic vector representations corresponding to the two entities and the semantic vector representations of all context text blocks are input into a multi-layer neural network model to obtain the relationship between the two entities.

7. The method according to claim 1, characterized in that The obtaining the number of tested events of the tested requirements according to the similarity between the tested events of the requirements and the tested events of the use cases includes: Vectorizing the demand tested events and the use case tested events; Using an average pooling function to process the vectors of the demand tested event and the use case tested event, and calculating the similarity between the processing results; Comparing the similarity with a second set threshold to determine whether the required measured event is met; Count the number of tested events for the requirements that have been tested.

8. A test coverage evaluation device based on fine-grained text understanding, characterized in that: The device comprises: Data acquisition module, used to obtain the requirement documents and test case reports of the software project to be tested; An information extraction module, used to convert the requirement document and the test case report into a plurality of structured information respectively; wherein the structure of the structured information is entity-relationship-entity; An event generation module, used to generate a number of demand tested events corresponding to the demand document and a number of use case tested events corresponding to the test case report based on the structured information; wherein the event templates of the demand tested events and the use case tested events include all categories of entities; The test evaluation module is used to obtain the number of tested events of the tested requirements based on the similarity between the tested events of the requirements and the tested events of the use cases; and to obtain the test coverage of the software engineering to be tested based on the total number of tested events of the requirements and the number of tested events of the tested requirements.

9. A computer device, characterized in that: The computer device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images