English ambiguity aided inspection system

Abstract

The application discloses an English requirement ambiguity auxiliary checking system, which comprises a storage layer, a model layer, a business logic layer and a display layer, wherein the NLP model of the model layer comprises several submodels, the business logic layer comprises an NLP pipeline management module, the NLP pipeline management module selects corresponding submodels from the NLP model as subtask processing modules according to ambiguity checking requirements, and then connects the subtask processing modules in series to form an NLP pipeline; after a requirement document is imported into the NLP pipeline, each subtask processing module in the NLP pipeline labels the requirement document with a specific label, until the ambiguity checking is performed, and finally, the output is the requirement document with the label.

Description

Technical Field

[0001] This invention relates to the field of requirements analysis, and in particular to an auxiliary system for checking the ambiguity of English requirements. Background Technology

[0002] Requirements analysis, situated in the early stages of product development and project implementation, clarifies the product's functional and performance requirements, or the project's implementation process and acceptance criteria. It forms the foundation for product and engineering quality control. Therefore, in the development of complex systems like civil aircraft avionics systems, requirements analysis demands significant effort and holds a crucial position, arguably the most critical stage in the entire product lifecycle. Defects arising during the requirements analysis phase often result in products or product quality failing to meet requirements, leading to project delays or failure. Inaccurate or incomplete understanding of requirements by product manufacturers and project implementers, or unclear descriptions of requirements by designers, can significantly jeopardize the final outcome.

[0003] Requirements analysis leads to requirements documents. While some requirements documents are written in formal languages ​​or UML, the vast majority are written in natural language. This is because natural language is easy to understand, universal, and flexible, making it applicable to requirements descriptions across different domains. However, the inherent ambiguity of natural language introduces uncertainty into the understanding of requirements. According to the Oxford English Dictionary, the 500 most commonly used words in English have an average of 23 meanings each; and a phrase or even a sentence often has multiple interpretations. If requirements contain semantic ambiguity, inaccurate descriptions, or vague meanings, different people will have differing understandings of the same requirement description. Therefore, experienced developers often need to review requirements before they are implemented. However, manual review is costly, time-consuming, labor-intensive, and prone to errors. Missing issues can affect the product development cycle and product quality, causing losses or even serious consequences.

[0004] With the development of artificial intelligence technology, Natural Language Processing (NLP) has been applied to many fields. In the aviation industry, major international system suppliers have begun to utilize NLP technology in engineering applications to improve the quality of requirements documents. Requirements analysis for civil aircraft avionics systems is an important part of the construction of the systems engineering development environment for civil aircraft avionics systems. Addressing the high cost and low accuracy of manual requirements inspection, this paper proposes to utilize the latest NLP research results to establish a semi-automated tool to assist in detecting ambiguities in requirements documents for civil aircraft avionics systems, improving the efficiency and accuracy of requirements defect detection, effectively saving economic and labor costs, shortening product development cycles, and improving product quality.

[0005] Requirements engineering (RE) is a discipline that applies proven techniques and methods to perform requirements analysis, identify customer needs, and help requirements analysts understand problems and define all external characteristics of the target system. Requirements engineering has a wide range of applications.

[0006] Requirements engineering primarily includes requirements development and requirements management. Requirements development mainly includes requirements elicitation, requirements analysis, specification writing, and requirements verification. Its purpose is to ensure developers fully understand customer needs, conduct technical feasibility analyses, and create complete and reliable requirements documents. User requirements iterate continuously during development; the main task of requirements management is to ensure the product can adapt to and control these changes. The purpose of requirements management is to track and modify requirements after development, ensuring that requirements remain consistent and unchanged throughout the development process. Requirements management mainly includes baseline control, change management, and requirements tracking.

[0007] Requirements development is the first step in product or process development, and its outcome is often the completion of a requirements document. A requirements document is a formal statement of what functionalities the product developers or process implementers need to achieve. It describes the expectations of customers or users for the product or process, such as conditions, attributes, goals, and uses. The level of detail in the requirements document depends on the content to be developed and the development processes involved. Effective requirements engineering is a core element in organizing software system development and enhancing technological competitiveness. Numerous studies have found that deficiencies in requirements are a major culprit in product or project failures. Therefore, the quality of the requirements document is crucial to product quality. Requirements quality not only determines whether user requirements are completely, correctly, and logically translated into concrete functional implementations, but also significantly impacts product development costs.

[0008] There is no unified standard for judging requirement quality, and developers often find it difficult to know which requirement characteristics are crucial to project success. Therefore, it is necessary to manually define requirement quality attributes. Common quality attributes include: the correctness of the requirement, i.e., whether the requirement can describe the real world; the accuracy of the requirement, i.e., whether the requirement statement contains logical errors; the completeness of the requirement, i.e., whether the requirement covers all necessary parts and whether the business logic has branching or exception flows; and the ambiguity of the requirement, i.e., whether it introduces natural language ambiguity that leads to unclear requirement statements, etc.

[0009] Referring to IEEE standards, requirements are defined as: statements that define the operation, function, design characteristics, or constraints of a product or process, characterized by being explicit, testable or measurable, and acceptable to the product or process according to consumer or internal quality assurance criteria. This definition outlines several aspects that a high-quality requirements document must meet:

[0010] (1) A requirement should be a description that can be expressed in text or presented in the form of a diagram. For example, in Tom Gilb's P language, requirements are constructed in graphical form. Other methods include notation used in UML, such as the B-method, or domain-specific notation. The key is to identify and designate a set of traceable, manageable elements as a requirement.

[0011] (2) It encompasses complete solutions involving different products and processes. Therefore, requirements can define processes and products. In addition, there may be requirements that define how the product is developed, and their purpose is usually for quality control.

[0012] (3) Requirements can be operational, functional, design features, or constraints. There are many different types of requirements, which may lead to different kinds of requirements elicitation, requirements analysis, requirements modeling, implementation processes, and solutions. Design features include performance aspects such as usability, security, maintainability, and many other features.

[0013] (4) Unambiguity. In short, a requirement should be clear and have a single, unambiguous understanding, so that all participants have the same understanding of the requirement, avoiding discrepancies in understanding the same requirement. This is precisely the problem that this project mainly addresses.

[0014] (5) Testable or measurable. Requirements need to consider whether the design or solution will ultimately be acceptable. To achieve this, requirements should be quantified to provide a solution with measurable requirements.

[0015] (6) The product or process needs to be acceptable. This highlights the multidimensional role of requirements: they define what should be designed and developed, as well as how technical solutions should be tested and accepted. They have an impact on various processes from the initial stages of development to the acceptance process.

[0016] (7) Demand originates from consumers or other sources. Demand has many sources, including but not limited to customers, market research, industry consulting, users, and internal company information.

[0017] In general, the quality of natural language description requirements can be assessed by measuring factors such as correctness, accuracy, ambiguity, completeness, consistency, traceability, legality, readability, testability, and verifiability. Among these, ambiguity is a particularly prominent issue.

[0018] Manually checking for ambiguity in requirements is a costly and inaccurate process. Furthermore, with each update and iteration of requirements, the need for repeated manual checks arises, posing a significant challenge to requirement ambiguity detection. Therefore, there is an urgent need for semi-automated and automated tools to detect ambiguities in requirement specifications, improving the efficiency and accuracy of requirement defect detection. Some research has already been conducted in this area.

[0019] Rule-based methods were initially used to detect ambiguity in requirements. The rules need to cover all states, including all grammatical cases and exceptions. Wilson et al. (1996) were the first to experiment with rule-based methods, and Femmer et al. (2017) continued this research as recently as now. However, relying solely on rules for requirement ambiguity detection is somewhat ineffective and requires contextual knowledge. Statistical methods require large labeled datasets for training, relying heavily on annotation information in the corpus to learn ambiguity detection. Dowser et al. (2003, 2008) initially detected ambiguous words in sentences by examining grammar. They used a parser to parse sentences and perform syntactic analysis, achieving good experimental results in attachment ambiguity and homonymy. Furthermore, checklist-based methods identify ambiguity in statements by comparing requirements. Fagan et al. (2002) surveyed 117 checklists from 24 sources, demonstrating that this method can effectively identify ambiguity in software development. Kamsties et al. (2000) used a scenario-based reading approach and checklists to improve the ambiguity detection rate. Scenario-based reading provides the operational context of the requirements, allowing for the analysis of those requirements and the discovery of ambiguities in the language.

[0020] Describing requirements using specific sentence structures can avoid grammatical and semantic ambiguity, such as using active voice instead of passive voice and avoiding conjunctions. Sentence structure is one method to avoid ambiguity; by modifying the original requirements to match the defined requirement format, ambiguity can be eliminated. Tiong et al. (2008) proposed a tool called SREE to detect potential ambiguity in software requirements. They proposed methods for detecting ambiguity related to vagueness, quantifiers, lack of reference, pronouns, and plurals.

[0021] Song et al. (2004) proposed a machine learning-based method to identify ambiguity in requirements by training on a corpus. Brill et al. (1995) proposed using machine learning to identify and detect ambiguity in prepositional phrases in sentences. Yang (2010) proposed NAI, an automated tool for identifying ambiguity in requirements, which can effectively identify compound ambiguities of nouns and verbs in sentences. This method demonstrates the effectiveness of machine learning in identifying ambiguity.

[0022] Knowledge-based ambiguity checking methods utilize knowledge to detect and resolve ambiguities, which can be viewed as using knowledge from different domains to reduce the scope of the ambiguity search space. Korner (2009) proposed RESI, a knowledge-based semi-automatic requirements engineering specification improvement tool. This tool provides a graphical interface to detect ambiguities and offer modification suggestions, mainly focusing on incomplete and vague requirements. Al-Harbi et al. (2012) proposed a method based on context knowledge and concept ontology to solve the semantic ambiguity problem in natural language. This method is simple, has a low error rate, and achieves an accuracy of 83.4%. Summary of the Invention

[0023] The purpose of this invention is to provide an auxiliary system for checking the ambiguity of English requirements. This system identifies potentially ambiguous sentences by searching the requirement document for words or phrases from an ambiguous dictionary. Simultaneously, it uses machine learning methods to train various tasks in a natural language processing pipeline, such as named entity recognition and coreference resolution, to obtain information on words, sentence structure, semantics, and pronoun reference. Based on this information, potentially ambiguous sentences are examined, and established checking rules are used to ultimately determine whether a sentence is ambiguous. This invention combines machine learning and rule-based methods to achieve requirement ambiguity detection.

[0024] The objective of this invention is achieved through the following technical solution:

[0025] An auxiliary system for checking the ambiguity of English requirements, comprising a storage layer, a model layer, a business logic layer, and a presentation layer, wherein:

[0026] The storage layer is responsible for the persistent storage of data in the system;

[0027] The model layer contains NLP models, and the various sub-models in the NLP models are used as sub-tasks on the NLP pipeline;

[0028] The business logic layer includes an NLP pipeline management module. The NLP pipeline management module selects the corresponding sub-model from the above NLP models as sub-task processing modules according to the ambiguity check requirements and then connects them to form an NLP pipeline. After the requirement document is imported into the NLP pipeline, each sub-task processing module in the NLP pipeline will give the requirement document a specific tag until the ambiguity check is completed. The final output is the requirement document with tags.

[0029] The presentation layer is responsible for various display interfaces and human-computer interaction interfaces.

[0030] Preferably, the storage layer includes a relational database and a file system. The relational database stores user information, a glossary, an ambiguous dictionary, and inspection rules, while the file system stores a corpus, requirement documents, and processed tagged requirement documents.

[0031] Preferably, the model layer also includes the GATE framework. Each sub-model in the NLP model runs on the GATE framework. The NLP model comes from GATE and the Stanford NLP library and is trained using a corpus and a terminology database.

[0032] Ideally, each sub-model in the NLP model also provides corresponding API interfaces for other programs or systems to call.

[0033] Preferably, the NLP pipeline includes the following subtask processing modules in sequence:

[0034] The tagging module breaks down phrases, sentences, paragraphs, or the entire text document into smaller units;

[0035] The sentence segmentation module divides a piece of text into individual sentences.

[0036] The name index module searches for names in the corpus that appear in the requirements document;

[0037] The part-of-speech tagging module identifies the part of speech of each word in a sentence.

[0038] The lemmatization module removes affixes from words and extracts the word's core components.

[0039] The named entity recognition module uses a glossary to identify entities with specific meanings in the requirements document;

[0040] The chunking module, with the help of a glossary, divides a sentence into non-overlapping text chunks based on part-of-speech tagging and named entity recognition.

[0041] The coreference resolution module identifies noun phrases or pronouns that represent the same entity and categorizes them.

[0042] The spell check module uses a glossary to detect spelling errors in words in the requirements document.

[0043] The ambiguity checking module performs ambiguity checks based on the identifiers added to the requirements document by previous tasks, as well as additionally defined detection rules and an ambiguity dictionary.

[0044] Preferably, the business logic layer also includes a corpus management module, which allows developers to manage terminology and corpus, and use terminology and corpus as datasets for training NLP models in machine learning.

[0045] Preferably, the business logic layer also includes a dictionary maintenance module, which is used to maintain an ambiguous dictionary.

[0046] Preferably, the business logic layer also includes an inspection rule management module, which is used to maintain inspection rules.

[0047] Preferably, the business logic layer also includes a report generation module, which is used to generate reports from the tagged requirement documents output by the NLP pipeline.

[0048] The beneficial effects of this invention are as follows: Each system layer of this invention contains multiple modules. The module division is mainly based on the following considerations: functional requirements—code that meets specific functional requirements is divided into a module, facilitating understanding of the function and later maintenance; module independence—each module is relatively independent, and the coupling between modules is relatively small, so that modifying one module has little or no impact on another module, achieving a low-coupling and high-cohesion design for the entire system; abstraction and encapsulation—based on considerations for future functional modifications and expansions, each module can achieve closure to changes but openness to expansion; and finally, module size—the module division is even in terms of the number of functions, complexity, and code volume, avoiding excessively large and complex modules. This is beneficial for code development management and later maintenance. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating an English requirement ambiguity auxiliary inspection system as shown in an embodiment.

[0050] Figure 2 This is a hierarchical diagram of an English requirement ambiguity auxiliary inspection system as shown in the embodiment.

[0051] Figure 3 This is a schematic diagram of the NLP model in the example.

[0052] Figure 4 This is a schematic diagram of the NLP pipeline in the embodiment. Detailed Implementation

[0053] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0054] As an example, this embodiment illustrates an English requirement ambiguity auxiliary inspection system, taking the field of civil aircraft avionics systems as an example, which is required to verify the requirement ambiguity items shown in Table 1.

[0055] Table 1. List of Ambiguous Requirements

[0056]

[0057]

[0058] The following is a detailed description of these ambiguous situations:

[0059] (1) Anaphoric ambiguity

[0060] Demonstrative pronoun ambiguity is a type of semantic ambiguity. It refers to a situation where multiple different noun phrases appear in the context of the same pronoun, and it's unclear which noun the pronoun actually refers to. Demonstrative pronouns in English (e.g., he, it, that, this, which, etc.) can refer to things that appear in the preceding text. The quoted part of the text is usually called the antecedent. If the text provides multiple antecedents in the same sentence, such as the sentence "After some work, the machine will send a signal to the receiver, and then it finishes the task," demonstrative pronoun ambiguity arises because we are unsure whether "it" refers to "machine," "receiver," or something else in the preceding statement.

[0061] To find ambiguous pronouns in a sentence, we first need to identify the presence of pronouns, such as "he, she, they, it, you, they, we, him, her, them, your, his, her, its, our, their, this, that, these, those, such, some, who, whom, whose, which, what, whoever, whichever, whatever, whom". This is relatively easy to do. In English, although there are many types of pronouns (such as personal pronouns, possessive pronouns, reflexive pronouns, reciprocal pronouns, demonstrative pronouns, interrogative pronouns, etc.), they are all closed parts of speech and can be looked up in specific dictionaries. If we assume that all the pronouns we find are ambiguous, we can achieve 100% recall, but the precision will be relatively low. To improve precision, we might try using coreference resolution to find the entity that the pronoun refers to, thus eliminating ambiguity.

[0062] (2) Coordination scope

[0063] Ambiguity of coordinating conjunctions refers to the unclear scope of coordinating conjunctions (such as "or" or "and"), leading to multiple potential interpretations of a sentence. The scope of coordinating conjunctions is related to syntactic structure; therefore, this ambiguity falls under the category of syntactic ambiguity.

[0064] This section considers two types of ambiguity related to coordinating conjunctions. The first type involves sentences that require more than one coordinating conjunction within the same sentence. For example, "There is a hundred-meter distance between monitor 1 and monitor 2 and monitor 3 shall have two hundred-meter distance." Such a description makes it unclear to the reader which monitor is being referred to by the second "and."

[0065] The second type includes sentences where parallel conjunctions are used with modifiers. For example, in "nice decks and tables," "nice" could modify only "decks," or it could modify both "decks" and "tables," which can be misleading. Another example is "We colored the chairs green and black." This sentence can be seen as a shortened form of "We colored the chairs green and some black," meaning we painted some chairs green and some black. However, another interpretation exists: we painted the chairs in a pattern of alternating green and black.

[0066] Both types of ambiguity caused by coordinating conjunctions can be detected by defining matching rules. Therefore, we need to define two matching rules. The first rule matches cases where the conjunction appears twice or more. The second rule matches cases where an adjective appears first, followed by a pair of singular or plural nouns connected by "and" or "or".

[0067] (3) Vague modifiers

[0068] Vague modifiers refer to the use of vague terms that express numerical or conditional expressions, such as "as far as possible" where the true condition cannot be quantitatively determined. Other examples include "as much as possible," "later," "taking into account," "acceptable," "up to," "periodically," "appropriate," "user-friendly," "support," and so on. Vague modifiers are a type of ambiguity in requirements engineering.

[0069] Detecting ambiguity caused by vague modifiers is relatively simple. We just need to create a dictionary for these vague modifiers and then check whether the words or phrases in the dictionary appear in the requirements document.

[0070] (4) Modal adverbs

[0071] Modal adverbs are adverbs that express a certain degree of possibility or uncertainty. For example, "likely," "totally," and "loosely" are commonly used to modify modal verbs. These words convey a sense of possibility, and their very presence introduces a degree of uncertainty. Besides modal adverbs expressing possibility or uncertainty, adverbs like "surely" and "undoubtedly," while attempting to eliminate uncertainty, still carry a subjective element and can thus introduce ambiguity. The ambiguity caused by modal adverbs is a form of requirement engineering ambiguity.

[0072] Other examples of modal adverbs include: “fortunately”, “essentially”, “evidently”, “definitely”, “scarcely”, “probably”, “noticeably”, “arguably”, “all in all”, etc.

[0073] Detecting ambiguity caused by modal adverbs is also relatively simple. There is a fixed list of modal adverbs in English. We just need to build them into a dictionary and then check whether the words or phrases in the dictionary appear in the requirements document.

[0074] (5) Passive voice

[0075] Using the passive voice can make the demand unclear and may lead to ambiguous interpretations when the subject of the verb is unclear. The passive voice is indirect, verbose, and logically weak. If the subject of the passive voice is omitted, the subject of the action is unknown, resulting in ambiguity. For example, in "The machine shall be shutdown," the lack of a subject means we don't know who should shut down the machine. Furthermore, some passive voice sentences, while having a subject, use "it" instead of a first-person pronoun, such as "It is concluded that the treatment is effective." In this sentence, we cannot know who concluded that the treatment was effective. The ambiguity caused by the passive voice falls under the category of semantic ambiguity.

[0076] To detect whether passive voice leads to ambiguity, we first need to identify the passive voice. This can be done by obtaining the sentence's parse tree through syntactic parsing and then inferring the passive voice from it. For a passive voice sentence, we then check whether a subject is missing or whether the subject is the ambiguous "it". If either of these two situations exists, ambiguity is considered to exist.

[0077] (6) Vague terms

[0078] Vague terminology falls under the category of semantic ambiguity. Terms such as "constituent," "element," "ingredient," "component," "part," and "subsystem" are broad concepts with different meanings in different fields. Without clear explanation, their specific meaning is difficult to determine. Even terms within a specific field can be somewhat ambiguous. For example, in the aviation field, "aircrafts" doesn't specify the type or model of the aircraft. Another concrete example is "interface." In the computer field, hardware interfaces and software interfaces are different. Even within the software field, user interfaces, application interfaces, and application programming interfaces are entirely different concepts. Simply mentioning "interface" without any explanation makes its true meaning unclear.

[0079] The detection of fuzzy terms also requires the use of a dictionary, but this dictionary is open-source. Besides manually collecting a subset of words, we also need to utilize natural language processing techniques to find terms that may lead to ambiguity. Specifically, this can be done by obtaining a large set of words from a glossary, extracting nouns from a corpus, and merging the two to obtain a candidate set of fuzzy terms. Then, semantic models such as Word2vec can be used to calculate the semantics of each word. If significant differences are found in the semantics calculated across different documents or corpora, the term is considered ambiguous.

[0080] (7) Subjective adjectives

[0081] Subjective adjectives are easy to understand; they are adjectives that contain a person's subjective intent, such as "simple," "complex," "good," "bad," "efficient," and "inefficient." The semantics or degree expressed by these words are highly subjective, introducing uncertainty into others' understanding of the requirements document's content. Ambiguity caused by subjective adjectives is a type of ambiguity in requirements engineering.

[0082] Detecting ambiguity caused by subjective adjectives requires a dictionary. Currently, collecting these subjective adjectives is done manually. After the dictionary is built, it is necessary to check whether the words or phrases in the dictionary appear in the requirements document.

[0083] (8) Distributive determiners

[0084] Distributive determiners are words that refer to a single person or thing, or a group of people or things, such as "every," "any," "all," "each," "none," "some," "another," "one," "both," and "part of." They provide different ways of referring to groups and individuals, demonstrating how things are distributed, shared, or divided. Distributive determiners cause ambiguity in a similar way to pronoun ambiguity, both stemming from unclear reference. Therefore, ambiguity caused by distributive determiners falls under the category of semantic ambiguity. An example is the sentence: "Half the people have already left." In this sentence, it's unclear which half of the people have already left.

[0085] The allocation qualifiers in English are fixed. We can collect them into a dictionary and then check whether the words or phrases in the dictionary appear in the requirements document to detect such ambiguities.

[0086] (9) Vague comparatives

[0087] The case of fuzzy comparatives is relatively more complex, as it can lead to both syntactic ambiguity and requirements engineering ambiguity.

[0088] For example, the sentence "There are more intelligent boys than I expected in the classroom." Since "more" can be used as the comparative form of "many," the sentence can be understood as "There are more intelligent boys in the classroom than I expected." However, "more" can also be used as the comparative form of "much," so the sentence can also be understood as "The boys in the classroom are more intelligent than I expected." Another example is the sentence "The people send them more important messages than we had imaged." This can be understood as either "These people sent them more important messages than we imagined," or "These people sent them messages that were more important than we imagined."

[0089] Another situation arises because the comparative adjective doesn't specify the exact size or degree of the difference. For example, "Product A is longer than product B" is ambiguous because we don't know exactly how much longer product A is compared to product B. This is a type of requirement engineering ambiguity. However, in the same situation, if we add a specific numerical value, we can eliminate the ambiguity, such as "Product A is 1.3 meters longer than product B." We clearly know that product A is 1.3 meters longer than product B, and there is no ambiguity or vagueness.

[0090] For the first type of ambiguity (syntactic ambiguity), it can be detected by establishing a simple checking rule: check if an adjective appears after the comparative degree. The second type of ambiguity (requirements engineering ambiguity) can also be detected by establishing checking rules, but it is relatively more complex. First, identify the comparative word, then check if a number and unit of measurement appear before the comparative degree. If so, the ambiguity can be eliminated. The number and unit must appear together; having a number without a unit actually introduces another type of ambiguity.

[0091] (10) Vague superlatives

[0092] Vague superlatives are a type of ambiguity in requirements engineering. An example is the previously discussed sentence, "If the bank customer maintains a minimum balance in his or her checking account, there is no monthly service charge." Here, we don't know what the minimum account balance is. In fact, whenever superlative words appear, such as "highest," "largest," "lowest," "smallest," "lightest," "most," "least," "heaviest," "hardest," "fastest," and "slowest," we consider the sentence ambiguous. This differs from comparatives.

[0093] Correspondingly, the method for checking fuzzy superlatives is relatively simple; you just need to find out if there are superlative words in the sentence.

[0094] (11) Unverifiable quantifiers

[0095] Unverifiable quantifiers refer to quantifiers that describe quantities in a vague and ambiguous way, thus introducing ambiguity. These words include "some," "sufficient," "large amount of," "much," "large number of," "little," "a small amount of," "insufficient," and "inadequate." Unverifiable quantifiers are a type of ambiguity in requirements engineering.

[0096] Similarly, a method to check for difficult-to-verify quantifiers is to create a dictionary of them and then search a sentence for the words in the dictionary. The number of difficult-to-verify quantifiers is limited, making them relatively easy to collect and organize.

[0097] (12) Missing condition

[0098] Missing conditions refer to conditional statements such as "if...else", "if...then", "if...otherwise", "when...then", "while...then", and "where...then" where one of the two keywords is missing. For example, a sentence might only contain "if" without "then", or only contain "then" without "if". This situation is common in ordinary writing and taken for granted, but it's considered insufficiently rigorous in requirements documents. Although this ambiguity is related to requirements engineering, because it's directly related to syntax, we classify it as a type of syntactic ambiguity.

[0099] Checking for ambiguity in missing conditions is relatively simple and can be achieved using a simple checking rule.

[0100] See Figure 1 , Figure 2 As shown in this embodiment, an English requirement ambiguity auxiliary inspection system is divided into four functional layers: storage layer, model layer, business logic layer, and presentation layer. These four system layers form a dependency relationship from bottom to top.

[0101] I. Storage Layer

[0102] The storage layer is responsible for the persistent storage of data in the system. The storage containers include a relational database and a file system. The relational database stores system data such as user information, as well as various terminologies, dictionaries, and check rules. The file system stores corpora, requirement documents, and processed tagged requirement documents. This unstructured or unstructured data is more suitable for file storage than a database.

[0103] II. Model Layer

[0104] The model layer includes the GATE framework and NLP models.

[0105] Most of the sub-models in an NLP model are subtasks on the NLP pipeline, such as: tokenization model, part-of-speech tagging model, lexical reconstruction model, sentence segmentation model, named entity parsing model, coreference parsing model, etc.

[0106] Each sub-model in the NLP model runs on the GATE framework. The NLP model comes from libraries such as GATE and StanfordNLP. After being trained with a specific corpus, it is more suitable for processing requirement documents in the field of civil aircraft avionics systems.

[0107] Figure 3 This section demonstrates the specific NLP models selected for each subtask in the NLP pipeline described below. As you can see, for some basic tasks, we used several libraries from the ANNIE processing flow included with GATE; for most of the more complex tasks, we used the StanfordNLP library; and finally, for ambiguity detection, we used GATE's JAPETransducer.

[0108] The choice of these libraries is primarily based on our past experience, but several points are worth noting: we will expand ANNIE Gazetteer to create a richer name index table; some models require retraining on additional corpora, such as Stanford POSTagger and Stanford NER; although many libraries in Stanford NLP perform well (mainly because they use large and high-quality training sets), they are relatively slow. If problems are encountered during implementation, we may consider using other libraries, such as replacing Stanford CorefAnnotator for coreference resolution with ANNIE OrthoMatcher.

[0109] Each sub-model in the NLP model also provides a corresponding API interface for other programs or systems to call.

[0110] III. Business Logic Layer

[0111] The business logic layer contains the system's business logic functions, including: corpus management module, NLP pipeline management module, dictionary maintenance module, inspection rule management module, user management module, report generation module, etc.

[0112] (I) Corpus Management Module

[0113] The corpus management module allows developers to collect and organize terminology lists and corpora during the development of this system, and to use the terminology lists and corpora as datasets for machine learning to train NLP models.

[0114] During the usage phase of this system, if the terminology and corpus are added to or removed, the corpus management module can retrain these NLP models. However, the system needs to be recompiled before it can be used again.

[0115] (II) NLP Pipeline Management Module

[0116] The NLP pipeline management module selects corresponding sub-models from the aforementioned NLP models as sub-task processing modules based on the ambiguity checking requirements, and then connects them to form an NLP pipeline. After the requirement document is imported into the NLP pipeline, each sub-task processing module in the NLP pipeline will tag the requirement document with specific tags until the ambiguity checking is completed. The final output is the tagged requirement document. From the tags in the requirement document, we can identify which sentences are ambiguous, which spellings are incorrect, etc.

[0117] A natural language processing (NLP) system or application typically includes a NLP pipeline. An NLP pipeline is a series of NLP tasks. The requirement document in natural language is input into the NLP pipeline and sequentially passes through each sub-task processing module. Each sub-task adds corresponding tags to the requirement document, forming a pipelined process. Different applications need to build their own NLP pipelines to achieve different goals. Common sub-task processing modules in a natural language pipeline include: tokenization, sentence segmentation, part-of-speech tagging, lemmatization, and named entity recognition.

[0118] See Figure 2 The diagram shows the pipeline designed in this embodiment. The NLP pipeline includes the following sub-task processing modules:

[0119] (1) Identification module

[0120] Tokenization is the most fundamental step in processing text data. The tokenization module breaks down phrases, sentences, paragraphs, or entire text documents into smaller units, such as individual words or terms. Each smaller unit is called a token, which can be a word, a number, or a punctuation mark.

[0121] (2) Sentence segmentation module

[0122] The Sentence Splitting module divides a text into individual sentences. A sentence is one of the basic units of language, and much subsequent processing and analysis is performed on a single, complete sentence.

[0123] (3) Name Index Module

[0124] The term "Gazetteer" literally means a place name index, a dictionary or index of geographical terms used in conjunction with atlases. In natural language processing, however, "Gazetteer" simply refers to a collection of lists of various nouns. These lists contain names of entities such as countries, cities, organizations, and days of the week. The name indexing module is used to find names in a corpus that appear in a requirement document. Each language has some general corpora, while specific corpora are extracted based on the needs of a particular domain for subsequent tasks such as named entity recognition.

[0125] (4) Part-of-speech tagging module

[0126] Part-of-speech tagging (POS tagging) is one of the most common tasks in natural language processing. A POS tagging module identifies the part of speech (POS) of each word in a sentence. A word's POS represents its syntactic role in a sentence, making POS tagging a fundamental task for syntactic analysis. POS are divided into two main categories: closed and open. Open POS are constantly evolving, with new words being created or borrowed, such as nouns like "iPhone" or "fax." This is the challenge of POS tagging.

[0127] There are various general algorithms and models for part-of-speech tagging, such as Hidden Markov Models (HMMs), Maximum Entropy Markov Models (MEMMs), and Recurrent Neural Networks (RNNs). Training these models often requires labeled corpora; for example, an important corpus for English is Penn Treebank, which contains 36 classes of part-of-speech tags.

[0128] (5) Word form restoration module

[0129] Lemmaization is an important part of text preprocessing. The lemmaization module is used to remove affixes from words and extract the core of the word. Usually, the extracted words will be words from the dictionary.

[0130] Lexical reduction and stemming are similar, but not the same. Lexical reduction reduces any form of a word to its general form (that expresses complete meaning), while stemming extracts the stem or root form of a word (which may not necessarily express complete meaning), and the extracted word may not appear in a dictionary.

[0131] The reason we place word form reduction after part-of-speech tagging is that word form reduction depends on the results of part-of-speech tagging. For example, suppose we encounter the word "arms". To determine its meaning, we first need to perform part-of-speech tagging. As a noun meaning "weapons, munitions, ammunition", "arms" is itself. As a noun meaning "arm", it is a plural form, and its original form is "arm". As a verb meaning "to arm", its original form is also "arm".

[0132] (6) Named Entity Recognition Module

[0133] Named entity recognition (NER) is a hot research topic in the field of natural language processing. The NER module is used to identify entities with specific meanings in requirements documents using a glossary, mainly including names of people, places, organizations, and proper nouns. These technical terms need to be distinguished from common nouns and processed accordingly. Simply put, it identifies the boundaries and categories of entity references in natural text.

[0134] This system requires a specific named entity recognition method to identify terms specific to the field of civil aircraft avionics systems. In addition to a general glossary, a glossary specific to civil aircraft avionics systems is also needed to accomplish this task.

[0135] (7) Modular module

[0136] The chunking module is used to divide a sentence into non-overlapping text chunks based on part-of-speech tagging and named entity recognition, using a glossary. Our primary focus is on noun phrases (NPs) and verb phrases (VPs). Distinguishing between these chunks facilitates subsequent spell checking and ambiguity checking tasks. Similar to named entity recognition, chunking also utilizes a glossary from the field of civil aircraft avionics systems.

[0137] (8) Core pointer resolution module

[0138] The coreference resolution module is used to automatically identify noun phrases or pronouns that represent the same entity and categorize them. When these noun phrases or pronouns appear together, we need to analyze them based on existing knowledge or contextual information to determine that they refer to the same entity.

[0139] Coreference resolution plays a specific role in this system, primarily to resolve the problem of identifying referential ambiguity. Wherever a pronoun appears, there is a possibility of referential ambiguity. However, if the pronoun corresponds to only one unique entity in the context, then this suspicion can be ruled out; otherwise, a large number of false positives will occur.

[0140] (9) Spell check module

[0141] The spelling check module is used to check for spelling errors in the requirements document using a glossary. It's listed separately here because it's a specific requirement of this project. The spelling check also uses general glossaries and glossaries specific to civil aircraft avionics systems.

[0142] (10) Ambiguity checking module

[0143] The ambiguity detection module is the core of this project. It performs ambiguity detection based on the identifiers added to the requirements document by previous tasks, as well as additionally defined detection rules and an ambiguity dictionary.

[0144] (III) Dictionary Maintenance Module

[0145] The dictionary maintenance module is used to maintain the ambiguous dictionary, which assists the ambiguity checking module in the NLP pipeline. The dictionary maintenance module should support adding, deleting, modifying, and querying the ambiguous dictionary. During system operation, operators can maintain the ambiguous dictionary, and any changes will be immediately reflected.

[0146] (iv) Inspection Rule Management Module

[0147] The inspection rule management module is used to maintain inspection rules, which assist the ambiguity checking module in the NLP pipeline to perform ambiguity checks. The inspection rule management module should support adding, deleting, modifying, and querying inspection rules. During system operation, operators can maintain the dictionary and rule sets, and any changes will be immediately reflected.

[0148] (V) Report Generation Module

[0149] The report generation module is used to generate reports from the tagged requirements documents output by the NLP pipeline. The report content can be exported in Word or Excel format.

[0150] IV. Display Layer

[0151] The presentation layer is responsible for the system's various display interfaces and human-computer interaction interfaces, including functions such as: displaying requirements documents, query interfaces, displaying inspection results, importing and exporting documents, and displaying reports. The presentation layer needs to provide a user interface to accept requirements documents as input. Requirements documents can be copied and pasted into the system for ambiguity checking, or imported into the system as files of various formats via the interface. After receiving the requirements documents, the system calls the NLP pipeline to process them. The report generation module generates reports and displays the results to the user through a user-friendly interface.

[0152] It is understood that those skilled in the art can make equivalent substitutions or modifications to the technical solution and inventive concept of the present invention, and all such substitutions or modifications should fall within the protection scope of the appended claims.

Claims

1. A system for assisting in checking the ambiguity of English requirements, comprising a storage layer, a model layer, a business logic layer, and a presentation layer, characterized in that: The storage layer is responsible for the persistent storage of data in the system; The model layer contains the GATE framework and the NLP model. The various sub-models in the NLP model are used as sub-tasks in the NLP pipeline. All the sub-models in the NLP model run on the GATE framework. The NLP model comes from GATE and the Stanford NLP library and is trained using a corpus and a terminology database. The business logic layer includes an NLP pipeline management module. Based on the ambiguity checking requirements, this module selects corresponding sub-models from the aforementioned NLP models as sub-task processing modules and connects them to form an NLP pipeline. After the requirement document is imported into the NLP pipeline, each sub-task processing module in the pipeline adds a specific tag to the requirement document until the ambiguity checking is completed. The final output is the tagged requirement document. The NLP pipeline includes the following sub-task processing modules in sequence: The tagging module breaks down phrases, sentences, paragraphs, or the entire text document into smaller units; The sentence segmentation module divides a piece of text into individual sentences. The name index module searches for names in the corpus that appear in the requirements document; The part-of-speech tagging module identifies the part of speech of each word in a sentence. The lemmatization module removes affixes from words and extracts the word's core components. The named entity recognition module uses a glossary to identify entities with specific meanings in the requirements document; The chunking module, with the help of a glossary, divides a sentence into non-overlapping text chunks based on part-of-speech tagging and named entity recognition. The coreference resolution module identifies noun phrases or pronouns that represent the same entity and categorizes them. The spell check module uses a glossary to detect spelling errors in words in the requirements document. The ambiguity checking module implements ambiguity checking based on the identifiers added to the requirements document by previous tasks, as well as additionally defined detection rules and an ambiguity dictionary. The presentation layer is responsible for various display interfaces and human-computer interaction interfaces.

2. The English requirement ambiguity auxiliary inspection system according to claim 1, characterized in that... The storage layer consists of a relational database and a file system. The relational database stores user information, a glossary, an ambiguous dictionary, and inspection rules, while the file system stores the corpus, requirement documents, and processed tagged requirement documents.

3. The English requirement ambiguity auxiliary inspection system according to claim 1, characterized in that... Each sub-model in the NLP model also provides a corresponding API interface for other programs or systems to call.

4. The English requirement ambiguity auxiliary inspection system according to claim 1, characterized in that... The business logic layer also includes a corpus management module, which allows developers to manage terminology lists and corpora, and use terminology lists and corpora as datasets for training NLP models in machine learning.

5. The English requirement ambiguity auxiliary inspection system according to claim 1, characterized in that... The business logic layer also includes a dictionary maintenance module, which is used to maintain ambiguous dictionaries.

6. The English requirement ambiguity auxiliary inspection system according to claim 1, characterized in that... The business logic layer also includes an inspection rule management module, which is used to maintain inspection rules.

7. The English requirement ambiguity auxiliary inspection system according to claim 1, characterized in that... The business logic layer also includes a report generation module, which is used to generate reports from the tagged requirement documents output by the NLP pipeline.

Images