Software demand defect detection method based on large language model and satisfiability problem solver

Abstract

The invention provides a software demand defect detection method based on a large language model and a satisfiability problem solver, and the method comprises the steps: constructing a domain knowledge base, guiding the large language model to extract key elements in a demand text through combining with a cue word project, and recognizing imperfect and ambiguous defects; and meanwhile, the demand is converted into a logic expression, and logic inconsistency is detected by using an SAT solver. According to the method, incremental updating of the knowledge base is supported, efficient retrieval and comparison are achieved through vectorization and Hash coding technologies, experiments show that the method is remarkably superior to a traditional technology in demand detection in the fields of finance and the like, defects can be accurately positioned, correction suggestions can be provided, and the efficiency and accuracy of demand analysis are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the cross - technical field of software engineering and artificial intelligence, and particularly relates to a software requirement defect detection method based on a large - language model and a satisfiability problem solver. Background Art

[0002] In the software development life cycle, it is crucial to define a comprehensive, clear, and non - contradictory target product. Requirement texts, as the main tool for describing this target product, are usually expressed in natural language. Due to the inherent complexity of requirement specifications, especially those involving thousands of independent requirements derived from interdisciplinary cooperation, requirement texts often face problems such as incompleteness, ambiguity, and inconsistency. For safety - critical systems, identifying and solving these problems is particularly crucial. Although existing technologies have made some progress in software requirement defect detection, there are still some limitations. Among them, rule - based methods rely on predefined structured rules, which are difficult to cover the semantic diversity of natural language and have high maintenance costs. Formal methods require manual processing of requirements into formal languages, which have extremely high thresholds for professional users and cannot handle fuzzy expressions. Traditional natural language processing technologies can only perform shallow syntactic parsing and lack the ability to detect logical contradictions. While methods that solely use large - language models have strong semantic understanding capabilities and can solve most of the above problems, they lack the support of a professional knowledge base and are prone to misjudgment.

[0003] Therefore, how to solve the technical problems of low efficiency and low accuracy in existing software requirement defect detection has become an urgent technical problem for those skilled in the art. Summary of the Invention

[0004] The purpose of the present invention is to provide a software requirement defect detection method based on a large - language model and a satisfiability problem solver to solve the problems of low efficiency and low accuracy in existing software requirement defect detection.

[0005] The technical solution adopted by the present invention is a software requirement defect detection method based on a large - language model and a satisfiability problem solver, and the steps include:

[0006] Step S1: Construct a knowledge base matching the field of requirements to be detected;

[0007] Step S2: Incrementally update the domain knowledge in the knowledge base;

[0008] Step S3: Receive the input of the software requirement text to be detected and identify the newly added software requirement text blocks;

[0009] Step S4: Guide the large - language model to extract key elements in the requirement text through prompt engineering;

[0010] Step S5: Identify the incompleteness and ambiguity of requirements by combining the domain knowledge base and the requirement text knowledge base;

[0011] Step S6: Convert the requirement text into a set of logical expressions through a conversion prompt template;

[0012] Step S7: Incrementally update the requirement text information in the knowledge base;

[0013] Step S8: Generate a detection report containing defects of incompleteness, ambiguity and inconsistency.

[0014] The beneficial effects of the present invention are:

[0015] 1. The method of the present invention can automatically detect three types of defect problems, namely incompleteness, ambiguity and inconsistency, in the requirement text. Compared with the prior art that only relies on formal methods, the present invention greatly improves the efficiency of requirement detection and significantly improves the accuracy of detecting inconsistency in the requirement text.

[0016] 2. Through the dynamic knowledge base construction and incremental update mechanism, the present invention realizes the continuous optimization and expansion of domain knowledge. Through the MD5 hash coding and similarity comparison technology, it can quickly identify new requirement text blocks and domain knowledge, avoid repeated calculations and storage redundancy, support cross-domain (such as finance, medical, education) adaptive detection, and significantly reduce the domain migration cost.

[0017] 3. Through the prompt engineering and dynamic template library, the present invention realizes context-aware semantic analysis and defect recognition. Through the dynamic slot filling mechanism and domain-adaptive prompt templates, it guides the large language model to extract key elements (such as entities, attributes and association relationships) in combination with the knowledge base context, effectively improving the accuracy of incompleteness and ambiguity detection.

[0018] 4. Each module of the system of the present invention can work independently and cooperate with each other to form a complete requirement defect detection process. By integrating the natural language understanding ability of the large language model and the formal verification ability of the SAT solver, it provides an innovative solution for software requirement defect detection, which can effectively improve the quality of software requirements and reduce the risks and costs in the subsequent development stage. Description of the Drawings

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0020] Figure 1It is the flowchart of the software requirement defect detection method of the present invention.

[0021] Figure 2 It is the structural diagram of the software requirement defect detection system of the present invention.

[0022] Figure 3 It is the return result of the software requirement defect detection method of the present invention. Specific implementation manners

[0023] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] Embodiment 1

[0025] As Figures 1 to 3 shown, the embodiment of the present invention provides a software requirement defect detection method based on a large language model and a satisfiability problem solver. The steps include:

[0026] Step S1: Construct a knowledge base matching the requirement field to be detected. The specific steps are as follows:

[0027] S11, collect standard specification documents, historical requirement documents, and relevant books in a specific field (such as finance, medical, manufacturing, or education), and establish a document set D = {d1, d2,.., d n}, where d n represents the nth field document.

[0028] S12, establish a domain-specific term dictionary, constraint conditions, and business rules. The term dictionary includes a term set T = {t1, t2,..., t i ,..., t m}, where t i represents the ith term, and m represents the total number of terms. Each term contains the following attributes: term name t i .name, term definition t i .definition, term synonym set t i .synonyms = {s i1 , s i2 ,..., s ik}, where s ik represents the kth term synonym. The constraint conditions are included in the business rules, and a business rule set R = {r1, r2,..., r j ..., rp} where r j represents the j-th business rule, p represents the total number of business rules, and each business rule r j contains the following attributes: r j .description represents the description of the j-th business rule, r j .constraint represents the constraint condition of the j-th business rule, r j .source represents the starting object on which the j-th business rule acts, r j .target represents the target object on which the j-th business rule acts.

[0029] Step S2: Incrementally update the domain knowledge in the knowledge base. The specific steps are as follows:

[0030] S21: Read the domain term dictionary constructed in S1, where each term contains attributes such as name, definition, and synonym set; and the business rule base constructed in S1, where each rule item contains a description, a constraint condition, and the object on which the rule acts, and the object on which the rule acts is the domain entity affected or constrained by the rule. The specific reading method is:

[0031] a. Establish a data access interface and define reading functions LoadTerms():T and LoadRules():R. LoadTerms():T represents a function to read all term data from the domain term dictionary, and LoadRules():R represents a function to read all business rule data from the business rule base;

[0032] b. Store terms and rules in JSON format and read data by parsing the JSON file.

[0033] S22: Use a vector model to vectorize the term names and definitions in the domain term dictionary, as well as the object on which the rule acts, the constraint condition, and the rule description in the business rule base, and use a hash algorithm to generate a unique encoding. The vectorization uses a vector model to convert text into a vector representation of a fixed dimension, and the hash encoding process uses the MD5 (Message-Digest algorithm 5) hash algorithm to ensure the uniqueness of the identifier. The specific implementation method is:

[0034] a. Define a vectorization function Vectorize(text):vector to map text to a vector of a fixed dimension. For each term t in the term dictionary i , t i .content = t i .name + t i .definition, calculate V(t i.content) = Vectorize(t i .content); and perform hash encoding on t i .name to generate a unique identifier for the term;

[0035] where t i .name represents the name of the i-th term, t i .definition represents the definition of the i-th term, t i .content represents the content of the i-th term, including the combination of the term name and definition, V(·) represents the vectorization function used to map text content to a vector of a fixed dimension, Vectorize(·) represents the operation of vectorizing text to convert the input text into a numerical vector of a fixed dimension, text represents the text content to be processed, and vector represents the mathematical representation of the text in a high-dimensional space.

[0036] b. For each business rule item r in the business rule library j , r j .content = r j .source + r j .target + r j .description + r j .constraint, calculate V(r j .content) = Vectorize(r j .content), define r j .name = r j .source + r j .target, and perform hash encoding on r j .name to generate a unique identifier for the rule;

[0037] where r j .description represents the description of the j-th business rule, r j .constraint represents the constraint condition of the j-th business rule, r j .source represents the starting object on which the j-th business rule acts, r j .target represents the target object on which the j-th business rule acts, r j.content represents the content of the j-th business rule item, including the combination of the object of action of the j-th business rule, the j-th rule description, and the j-th constraint condition. V(·) represents a vectorization function used to map text content into a vector of a fixed dimension. Vectorize(·) represents the operation of vectorizing text, converting the input text into a numerical vector of a fixed dimension, r j .name represents the object of action of the j-th business rule, including the combination of the starting object of the j-th business rule and the target object of the j-th business rule.

[0038] S23: Set the similarity threshold parameter. By calculating the cosine similarity of vectors, compare the content t i .content of the term and the content r j .content of the business rule item, identify the newly added domain knowledge, store the identified new terms in the corresponding domain term dictionary vector library, store the identified new rule items in the corresponding domain business rule vector library, and at the same time insert the new knowledge into the knowledge graph, where the starting object of the rule is used as the entity node, the business rule description is used as the edge name, and the constraint condition is used as the edge attribute. Set the similarity thresholds θ term and θ rule , which are respectively used for the similarity judgment of terms and rules;

[0039] For the new term t to be added new : Calculate the cosine similarity with each term in the existing term dictionary. The specific formula is as follows:

[0040]

[0041] If there exists t i such that sim(t new ,t i )≥θ term , it is determined as an existing term, otherwise it is determined as a new term;

[0042] For the new rule r to be added new : Calculate the similarity with each rule in the existing rule library. The specific formula is as follows:

[0043]

[0044] If there exists r j such that sim(r new ,r i )≥θ rule , it is determined as an existing rule, otherwise it is determined as a new rule;

[0045] For the updated new knowledge in the knowledge graph: For the new term t newAs a node in the graph, the new rule r new As an edge in the graph, connecting the entity it acts on, the specific formula is as follows:

[0046] G.addNode(t new .name, definition = t new .definition, synonyms = t new .synonyms)

[0047] G.addEdge(source = r new .source, target = r new .target, label

[0048] = r new .description, constraint = r new .constraint)

[0049] Among them, sim(·) represents the cosine similarity, t new represents the new term, r new represents the new rule, t i represents the i-th term, r j represents the j-th business rule, θ term represents the threshold for judging the term similarity, θ rule represents the threshold for judging the rule similarity, t i .content represents the content of the i-th term, including the combination of the name and definition of the i-th term, t new .content represents the content of the new term, including the combination of the term name and definition, r new .content represents the content of the new business rule, including the combination of the object the business rule acts on, the rule description, and the constraint conditions, r j .content represents the content of the j-th business rule, including the combination of the object the j-th business rule acts on, the rule description, and the constraint conditions, t new .name represents the new term name, t new .definition represents the new term definition, t new .synonyms represents the set of synonyms of the new term, r new .source represents the starting object the new business rule acts on, r new .target represents the target object the new business rule acts on, r new .description represents the description of the new business rule, r new.constraint represents the constraints of the new business rules, V(·) represents the vectorization function that maps text content into vectors of a fixed dimension, G.addNode represents the operation of adding a node in the knowledge graph, G.addEdge represents the operation of adding an edge in the knowledge graph, label represents the label of the knowledge graph, |·| represents the norm of the vector, definition represents the definition, synonyms represents the set of synonyms, source represents the starting object, target represents the target object, and constraint represents the constraint condition.

[0050] Step S3: Receive the input of the software requirement text to be detected, and identify the newly added software requirement text chunks. The specific steps are as follows:

[0051] S31. Receive the input of the software requirement text to be detected, and define the requirement document structure Req = {id, text, chunks[]}, where id represents the unique identifier of the requirement document, text represents the requirement text content, and chunks[] is an array of requirement text chunks; establish storage indexes for each software requirement text, including entity indexes, relationship indexes, and text chunk indexes.

[0052] S32. Perform chunking processing on the software requirement text, vectorize each requirement text chunk using a vector model, and at the same time perform hash coding processing on each requirement text chunk to generate a unique identifier for the text chunk. The vectorization uses the same vector model as S22 to convert the text into a vector representation of a fixed dimension. The vector calculation formula is as follows:

[0053] V(chunk k ) = Vectorize(chunk k .text)

[0054] The hash coding processing uses the MD5 hash algorithm to generate a unique identifier for the text content, specifically as follows:

[0055] chunk k .hash = Hash(chunk k .text)

[0056] Among them, V(·) represents the vectorization function used to convert the text chunk into a vector representation of a fixed dimension, chunk k represents the k-th text chunk, chunk k .text represents the content of the k-th text chunk, Vectorize(·) represents the vectorization operation that converts the text content into a vector of a fixed dimension, and chunk k.hash represents the hash-encoded value of the k-th text chunk, and Hash(·) represents the hash function used to generate a unique identifier for the text content.

[0057] S33. By comparing whether the hash encoding of each requirement text chunk exists in the repository of this requirement text, identify the newly added requirement text chunks. Maintain a set of hashes of processed text chunks ProcessedHashes = {hash1, hash2,...} for each requirement document, where hash1 represents the hash of the first processed text chunk; and define an array of new chunks new_chunks[]. For the currently processed text chunk chunk k , check the hash-encoded value of the text chunk chunk k .hash ∈ ProcessedHashes. If it exists, mark it as a processed text chunk; if it does not exist, mark it as a newly added text chunk, add it to the new_chunks array, and add the hash value to ProcessedHashes;

[0058] Step S4: Guide the large language model to extract key elements from the requirement text through prompt engineering. The key elements include entities, attributes, and association relationships. The specific steps are as follows:

[0059] S41. Construct a prompt framework system for software requirement analysis.

[0060] S42. Develop key element extraction prompts according to the entity types in the domain knowledge base.

[0061] S43. Select a small number of entity and relationship extraction examples according to the domain to which the software requirements belong and splice them into the prompts.

[0062] S44. Based on the prompt framework system, use the large language model to evaluate and optimize the prompt effect.

[0063] S45. Traverse the new_chunks array in S33, loop to extract the key elements from the text chunks, and construct an entity set entities = {e1, e2,.., e u} and a relationship set relationships = {rs1, rs2,..., rs z}; where e u represents the u-th entity extracted, and rs z represents the z-th relationship extracted.

[0064] For example, the prompt template for identifying requirement text entities and relationships is as follows:

[0065] "You are a professional software requirements analysis expert with rich software engineering experience. You are good at accurately identifying the key elements in software requirements. Your task is to analyze the given requirements text and the relevant knowledge retrieved from the knowledge base, identify all entities in the text, identify the attributes of each entity, and identify the relationships between entities."

[0066] Requirements:

[0067] 1. Entities must be specific business concepts or system components.

[0068] 2. Assign a unique ID (format: E + number) to each entity.

[0069] 3. Determine the type of the entity. The optional entity types include: {entities_type}

[0070] 4. Identify the status, behavior, and constraints of the entity.

[0071] 5. Identify the logical dependencies between entities.

[0072] Requirements text: {text}

[0073] Relevant knowledge retrieved from the knowledge base: {knowledge}

[0074] Output format: JSON format, without excessive description.

[0075]

[0076] Step S5: Combine the domain knowledge base and the requirements text knowledge base to identify requirement incompleteness and ambiguity. The specific steps are as follows:

[0077] S51. Receive the defect query problem of the software requirements text, and use the model to identify whether the query problem is relevant to software requirements defect analysis. If it is not relevant to software requirements defect analysis, then reply that the analysis of the query problem cannot be performed; if it is relevant to software requirements defect analysis, then continue with the subsequent steps.

[0078] S52. Traverse and extract the entities in the requirements text block, and select the similarity threshold θ entity , for each entity e u , calculate the similarity with the terms in the domain knowledge base. The specific formula is as follows:

[0079] For the cosine similarity sim of the term name name (e u ,t i ):

[0080]

[0081] Cosine similarity sim for term synonyms syn (e u ,t i ):

[0082]

[0083] Comprehensive similarity σ(e u ,t i ):

[0084] σ(e u ,t i ) = max{sim name (e u ,t i ), sim syn (e u ,t i )}

[0085] Among them, σ(·) represents the comprehensive similarity, V(·) represents the vectorization function, which is used to convert the text block into a vector representation of a fixed dimension, e u .name represents the name of the u-th entity, represents taking the maximum value in the synonym similarity between the entity and the i-th term, s represents the synonym in the synonym set of the i-th term, sim name (·) represents the cosine similarity function of the term name, sim syn (·) represents the cosine similarity function of the term synonym, e u represents the u-th entity, t i represents the i-th term, t i .name represents the name of the i-th term, |·| represents the modulus of the vector, max{·} represents taking the maximum value in the set, θ entity is the threshold for judging the comprehensive similarity between the entity and the term.

[0086] By comparing the similarities, identify similar entities in the domain knowledge base. When σ(e u ,t i ) ≥ θ entity , add t i to the list of similar entities of entity e u .

[0087] S53. Use the similar entities to find relevant business rules and constraints in the domain knowledge graph.

[0088] S54. Traverse the similar entities to obtain the corresponding requirement text blocks.

[0089] S55. Use the retrieved information and the newly added requirement text block as context information to establish a prompt engineering to guide the large language model to identify the incompleteness and ambiguity in the requirement text. If relevant data can be retrieved from the knowledge base, process it through the large language model and feedback the retrieved data, clearly marking that the retrieved data comes from the knowledge base. If no relevant data can be retrieved from the knowledge base, use the large language model to generate response data and mark that the response data is generated by the model.

[0090] S56. Make corrective annotations for the identified incompleteness and ambiguity.

[0091] For example, the prompt templates for identifying incompleteness and ambiguity are as follows:

[0092] "You are a professional software requirements analysis expert. Now please carefully analyze the given requirement text and its entity analysis results to detect any incompleteness and ambiguity issues.

[0093] Requirements:

[0094] 1. Incompleteness detection:

[0095] Check if the entity is missing necessary attributes.

[0096] Check if the relationships between entities are complete.

[0097] Check if the process or state transitions are complete.

[0098] Check if boundary conditions or exception handling are missing.

[0099] 2. Ambiguity detection:

[0100] Check if there are cases of polysemy in terms.

[0101] Check if the relationships between entities are ambiguous.

[0102] Check if quantifiers are ambiguous (such as "some", "multiple").

[0103] Check if conditional statements are clear.

[0104] 3. Defect correction and annotation:

[0105] Assign a unique ID (format: D + number) to each discovered defect.

[0106] Mark the specific location of the defect in the original text.

[0107] Describe the type (incompleteness or ambiguity) and cause of the defect.

[0108] Provide specific and feasible corrective suggestions.

[0109] Requirement text: {text}

[0110] Entity analysis result: {entity_analysis}

[0111] Output format: JSON format

[0112]

[0113] Step S6: Convert the requirement text into a set of logical expressions through a conversion hint template, and use a SAT solver to detect requirement inconsistencies; splice the newly added requirement text block with the associated existing text block; map the natural language description to a first-order logical expression through a logical conversion template; convert the logical expression into conjunctive normal form (CNF), and its form is:

[0114]

[0115] where A, B, and C are Boolean variables, B and C are literals, and the literal values can be True or False, and (B ∨ C) are clauses; use a SAT solver to detect the unsatisfiability of the CNF formula. If it is not satisfied, it indicates that there is an inconsistency in the requirements.

[0116] For example, the hint word template for converting requirement text into logical expressions is as follows:

[0117] "You are a professional software requirement analysis expert. Please convert the given requirement text into a first-order logical expression and a clause set. I will input the clause set you output into a SAT solver to detect requirement inconsistencies.

[0118] Requirements:

[0119] 1. Identify variables, constants, predicates, and functions in the requirements

[0120] 2. Use standard first-order logic symbols: Use standard first-order logic symbols: universal quantifier existential quantifier implication (→), conjunction (∧), disjunction (∨), negation

[0121] 3. Decompose complex requirements into simple clauses

[0122] 4. Ensure that the converted expression preserves the original semantics of the requirements

[0123] 5. When outputting, convert the first-order logical expression into CNF normal form

[0124] Requirement text: {text}

[0125] Previous analysis result: {analysis_result}

[0126] Output format: JSON format

[0127]

[0128] Step S7: Incrementally update the requirement text information in the knowledge base. The specific steps are as follows:

[0129] S71. Incrementally update the entity set entities = {e1, e2,.., e u} and the relationship set relationships = {rs1, rs2,..., rs z}, where u represents the u-th entity extracted, and z represents the z-th relationship extracted. For the requirement text information established for the first time, in addition to the text block index established in step S3, establish two vector library indexes, which are respectively used to store entities, relationships, and their attributes;

[0130] S72. For the entity set, use the vector model to vectorize the entity name and calculate the vector:

[0131] V(e u .name) = Vectorize(e u .name)

[0132] Use the MD5 hash algorithm to calculate the hash value of e u .name as the unique identifier:

[0133] e u .hash = Hash(e u .name)

[0134] For the relationship set, rs z .content = rs z .source + rs z .target + rs z .description +

[0135] rs z .constraint, and calculate the vector:

[0136] V(rs z .content) = Vectorize(rs z .content)

[0137] Use the MD5 hash algorithm to calculate rs z .source and rsz The hash value of.target, which serves as a unique identifier.

[0138] Among them, V(·) represents a vectorization function used to convert a text block into a vector representation of a fixed dimension, e u .name represents the name of the u-th entity, and Vectorize(·) represents a vectorization operation that converts text content into a vector of a fixed dimension, e u .hash represents the hash value of the name of the u-th entity, rs z .source represents the starting entity in the z-th relationship, rs z .target represents the target entity in the z-th relationship, rs z .description represents the description of the z-th relationship, rs z .constraint represents the constraint condition of the z-th relationship, rs z .content represents the content of the z-th relationship, including the combination of its starting entity, target entity, description, and constraint condition. Hash(·) represents a hash function used to generate a unique identifier for the text content.

[0139] S73. Maintain a set of processed entity hashes ProcessedEntityHashes = {hash e1 , hash e2 ,...} for each requirement document; define a set of new entities new_entity. For the currently processed entity e u , check if e u .hash ∈ ProcessedEntityHashes. If it exists, mark it as a processed entity; if not, mark it as a new entity and add it to the new_entity set.

[0140] S74. For the entity set entities, if the entity e u already exists in the knowledge base, then update the attributes of the entity e u . If the entity e u does not exist in the knowledge base, then add the entity e u . The same applies to the relationship set.

[0141] Compare with the existing knowledge base and only update the changed parts. Establish a version record for each update to facilitate tracking the history of requirement changes.

[0142] Step S8: Generate a detection report containing defects of incompleteness, ambiguity, and inconsistency. The report includes defect location information and correction suggestions. Incompleteness refers to the lack of comprehensive information related to the expected functions in the requirements, that is, the requirement description fails to include all necessary elements. This may manifest as the requirements not covering all required states, behaviors, and outputs under various possible constraints; Ambiguity means that the information or vocabulary contained in the requirement description can have multiple interpretations, making the requirement content subjective. For example, using two different terms to refer to the same concept, resulting in the ambiguity of the requirement; Inconsistency occurs when two or more requirements in the specification are contradictory to each other, or when two or more stakeholders propose inconsistent requirements. In this case, the inconsistency between requirements may lead to implementation difficulties or misunderstandings.

[0143] S81: Receive the user's feedback query on the generated defect detection report.

[0144] S82: Use the model to identify whether the user's query is a problem related to software requirement defects. If so, according to the query content, use the large language model to retrieve relevant knowledge from the knowledge base and give defect correction suggestions. When relevant knowledge cannot be retrieved from the knowledge base, use the large language model to generate correction suggestions and mark that the suggestion is generated by the model; When relevant knowledge is retrieved from the knowledge base, provide the retrieved knowledge through the large language model and mark that the suggestion is sourced from the knowledge base data. If not, reply that no relevant suggestions can be provided.

[0145] Through a similar core prompt template as above, this method can systematically extract entities and relationships from the requirement text, identify incompleteness and ambiguity, and convert the requirements into formal expressions for inconsistency detection.

[0146] After converting the requirement text into logical expressions through the conversion prompt template, the system inputs the generated clause set into the SAT solver. The SAT solver will conjoin all clauses to form a SAT instance to be solved, and then try to assign values to all variables to determine whether the formula is satisfiable. If Result = false, it indicates the existence of a logical contradiction, indicating that the requirements are inconsistent. In this way, the present invention can accurately locate the inconsistency problems in the requirement text.

[0147] The entity category slot in the prompt template can be dynamically configured according to specific domain requirements to improve the domain adaptability and analysis accuracy of the system. This configurable prompt engineering design significantly enhances the flexibility and scalability of the method and can adapt to the requirement defect detection of different types of software domains.

[0148] Embodiment 2

[0149] A software requirement defect detection method based on a large language model and a satisfiability problem solver of the present invention can be implemented through the following modules:

[0150] G1 Domain Knowledge Base Module: It includes a vector database and a knowledge graph database. The vector database is used to store the vector representations of domain term dictionaries and business rules, and these vectors are generated by the M3E (Moka Massive Mixed Embedding) model; the knowledge graph database is used to store structured domain knowledge, including term entity nodes, rule relationship edges, and constraint attributes.

[0151] G2 Requirement Text Knowledge Base Module: It is respectively connected to the text processing module and the prompt engineering control module. It includes a vector database, a knowledge graph database, and a MYSQL (MySQL Database Management System) database. The vector database stores the vector representations of entities and relationships extracted from the requirement text. The knowledge graph database constructs a structured graph according to the requirement version, with nodes representing requirement entities and edges representing cross-text block association relationships. The MYSQL database stores the MD5 hash code of the requirement text block, the text content, and its mapping relationship table with entities.

[0152] The vector database of the requirement text knowledge base module is implemented using Elasticsearch, which supports fast retrieval and similarity calculation of high-dimensional vectors. For each entity and relationship extracted from the requirement text, the system generates a vector representation using the same M3E model as the domain knowledge base and stores it in the vector database for subsequent semantic similarity analysis. The knowledge graph database of the requirement text knowledge base is also implemented using Neo4j.

[0153] G3 Text Processing Module: It is connected to the requirement text knowledge base module and the prompt engineering control module. The text processing module performs text parsing and text chunking on the input requirement document, and generates a globally unique version identifier and location index for each text chunk.

[0154] G4 Prompt Engineering Control Module: It is connected to the text processing module, the domain knowledge base module, and the defect detection module. The prompt engineering control module includes a dynamic template library, which stores extensible entity-relationship extraction templates, logical transformation templates, and defect identification templates. The templates contain domain adaptation parameters. The interaction interface between the prompt engineering control module and the large language model supports multiple large language models, including the GPT series and domestic large models, to adapt to different deployment environments and performance requirements.

[0155] G5 Defect Detection Module: Integrates two core components, namely large language model analysis and SAT solver. The large language model analysis is based on prompt templates to perform term detection, context analysis, and preliminary identification of incompleteness and ambiguity; the SAT solver is responsible for handling the verification of formal logical constraints to achieve strict consistency checking. The two parts fuse the results, ensuring both the breadth of detection (covering various types of defects) and the depth of detection in specific domains (especially logical consistency).

[0156] The data interaction process between modules of this system is as follows:

[0157] 1. The text processing module receives the requirement document, parses it to generate structured text blocks, and transmits them through the interface to the requirement text knowledge base module for storage. At the same time, the text processing module provides the processed text blocks to the prompt engineering control module as the input for subsequent defect detection.

[0158] 2. The prompt engineering control module selects appropriate entity-relationship extraction templates from the dynamic template library, combines them with the text blocks received from the text processing module, and guides the large language model to identify the entities and relationships in the requirements. The recognition results are returned to the prompt engineering control module through the standardized interface, and the latter then transmits this information to the requirement text knowledge base module for storage.

[0159] 3. The requirement text knowledge base module receives and stores the text blocks transmitted from the text processing module, as well as the entity and relationship information transmitted from the prompt engineering control module. For entities and relationships, the requirement text knowledge base module generates vector representations using the same M3E model, stores them in the vector database, and constructs a requirement knowledge graph and stores it in the graph database. At the same time, the requirement text knowledge base module maintains the mapping relationship between text blocks and entities and stores it in the MYSQL database.

[0160] 4. During the defect detection process, the prompt engineering control module selects defect recognition templates from the dynamic template library, combines them with the text blocks received from the text processing module, the entity relationship information queried from the requirement text knowledge base module, and the information retrieved from the domain knowledge base to generate complete prompts. The prompt engineering control module sends these prompts to the defect detection module to guide the latter to perform the defect detection task.

[0161] 5. When performing ambiguity and incompleteness detection, the defect detection module needs to query the standard term vectors and business rule knowledge graphs in the domain knowledge base module.

[0162] 6. For requirement inconsistency detection, the defect detection module first uses the logical transformation template in the prompt engineering control module to convert natural language requirements into formal logical expressions. Then, the SAT solver component of the defect detection module performs satisfiability analysis on these logical expressions to identify potential logical conflicts.

[0163] 7. After the detection is completed, the defect detection module generates a defect report, including the location, type, and repair suggestions of the defect. The defect location information depends on the text block identifiers generated by the text processing module and the entity mapping relationship maintained by the requirement text knowledge base module, ensuring that the defect can be accurately located to the specific location in the original document.

[0164] Experimental verification

[0165] By selecting software requirement documents from the financial domain and comparing the capabilities of the method of the present invention and the prior art in detecting three types of defects: incompleteness, ambiguity, and inconsistency, the prior art mainly includes: traditional rule-based methods, methods that only use large language models, and traditional formal methods.

[0166] The method of the present invention shows significant advantages in the detection of all three types of defects:

[0167] a. Incompleteness detection:

[0168] In the requirement document of a bank's cross-border payment system, there is the following description: "The system should support customers to initiate cross-border transfers and notify customers after the transaction is completed." Traditional methods can only check whether the requirement contains basic elements and cannot identify the incompleteness therein; methods that only use large language models may prompt the lack of description of the notification method and the lack of scenarios for transaction failures and cannot give complete professional suggestions; while the method of the present invention, with the assistance of the financial domain knowledge base, not only points out the key elements missing in this requirement, such as the notification of the intermediate state, exchange rate description, handling fee calculation method, and compliance inspection process, but also provides specific modification suggestions that meet regulatory requirements, such as "The system should support customers to initiate cross-border transfers, provide real-time status updates during the transaction process, clearly display the applicable exchange rate and calculation method, details of all handling fees, perform necessary anti-money laundering and counter-terrorism financing inspections, and notify customers through multiple channels after the transaction is completed." This ability of in-depth analysis based on professional domain knowledge gives the present invention an obvious advantage in incompleteness detection.

[0169] With the incremental update of the financial domain knowledge base, when the system incorporates the relevant regulations of the latest cross-border payment service management measures, the method of the present invention can further identify the compliance elements missing in the original requirement and make full use of the real-time updated knowledge.

[0170] b. Ambiguity detection:

[0171] In a certain financial product requirement, there is the following description: "The system shall allow authorized users to freeze suspicious accounts and unfreeze them after the account issues are resolved." Traditional methods rely on predefined rule sets for defect identification and are difficult to identify the ambiguity problem here. Methods that only use large language models may point out the problem that the definitions of "authorized users" and "suspicious accounts" are not clear, but lack the depth of understanding of financial domain-specific terms. The method of the present invention, with the support of a financial knowledge base, accurately identifies multiple ambiguities. For example, "freeze" in the financial system may refer to temporary freezing or full freezing; the criteria for determining "suspicious accounts" are not clear and may be based on abnormal transaction patterns, risk scores, or external reports; the confirmation process and criteria for "issue resolution" are not clear and may require multiple levels of approval. The present invention can not only identify these ambiguities but also provide correction suggestions that conform to financial industry norms.

[0172] With the incremental update of the knowledge base, when the system incorporates the latest financial account risk management norms, the understanding of the "freeze" operation becomes more refined, capable of distinguishing different levels such as "transaction freeze", "withdrawal freeze", and "full-function freeze", and the accuracy of ambiguity detection is further improved.

[0173] c. Inconsistency detection:

[0174] In the requirement document of an investment trading platform, it is stipulated on page 5 that: "All investment transactions with an amount exceeding 100,000 yuan must be manually reviewed before execution, and the review duration is 1 - 3 days"; while in the VIP customer service section on page 25, it is also stipulated that: "Diamond-level VIP customers enjoy the privilege of one-key fast trading, and the system shall complete the transaction confirmation within 10 seconds." Traditional methods are difficult to discover the requirement conflicts scattered in different sections. Methods that only use large language models are difficult to discover the problems of these two requirement statements due to the large text span. Even if they may suspect that there are conflicts between these two requirements, they cannot conduct strict logical verification. The method of the present invention accurately identifies the defect by converting these two requirements into formal logical expressions and combining the incremental update of the requirements. By analyzing the satisfiability through a SAT solver, the system determines that when a diamond-level VIP customer initiates a transaction exceeding 100,000 yuan, these two requirements cannot be logically satisfied simultaneously because the manual review process cannot be completed within 10 seconds. The present invention not only accurately locates this logical contradiction but also provides a solution suggestion, such as adding a condition: "For large transactions of diamond-level VIP customers, the system shall implement a risk score pre-review. Low-risk transactions shall be directly passed, and medium- and high-risk transactions still require manual review."

[0175] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, reference can be made to the corresponding description in the method embodiment.

[0176] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are all included in the protection scope of the present invention.

Claims

1. A software requirement defect detection method based on a large language model and a satisfiability problem solver, characterized by the steps Including: Step S1: Construct a knowledge base that matches the area of requirements to be detected; Step S2: Incrementally update the domain knowledge in the knowledge base; Step S3: Receive the input of the software requirements text to be detected, and identify the newly added software requirements text blocks; Step S4: Guide the large language model to extract the key elements in the requirements text through prompt engineering; Step S5: Combine the domain knowledge base and the requirements text knowledge base to identify requirement incompleteness and ambiguity; Step S6: Convert the requirements text into a set of logical expressions through a conversion prompt template; Step S7: Incrementally update the requirements text information in the knowledge base; Step S8: Generate a detection report containing defects such as incompleteness, ambiguity, and inconsistency.

2. A software requirement defect detection method based on a large language model and a satisfiability problem solver according to claim 1, characterized in that The specific steps of S1 are as follows: S11, Collect standard specification documents, historical requirement documents, and relevant books in a specific domain, and establish a document set D = {d1, d2,.., d n}, where d n represents the nth domain document; S12. Establish a domain-specific term dictionary, constraint conditions, and business rules. The term dictionary includes a set of terms T = {t1, t2,..., t i ,..., t m}, where t i represents the i-th term, and m represents the total number of terms. Each term contains the following attributes: term name t i .name, term definition t i .definition, set of term synonyms t i .synonyms = {s i1 , s i2 ,..., s ik}, where s ik represents the k-th term synonym. The constraint conditions are included in the business rules, defining a set of business rules R = {r1, r2,..., r j ..., r p}, where r j represents the j-th business rule, and p represents the total number of business rules. Each business rule r j contains the following attributes: r j .description represents the description of the j-th business rule, r j .constraint represents the constraint condition of the j-th business rule, r j .source represents the starting object on which the j-th business rule acts, and r j .target represents the target object on which the j-th business rule acts.

3. A software requirement defect detection method based on a large language model and a satisfiability problem solver according to claim 1, characterized in that The specific steps of S2 are as follows: S21: Read the domain term dictionary constructed in S1 and the business rule base constructed in S1. The specific reading method is: a. Establish a data access interface, and define the reading functions LoadTerms():T and LoadRules():R. LoadTerms():T represents the function of reading all term data from the domain term dictionary, and LoadRules():R represents the function of reading all business rule data from the business rule base; b. Store terms and rules in JSON format, and read data by parsing JSON files; S22: Use a vector model to vectorize the term names and definitions of the domain term dictionary, as well as the rule application objects, constraint conditions, and rule descriptions of the business rule base, and use a hash algorithm to generate unique encodings. The specific implementation method is: a. Define the vectorization function Vectorize(text): vector, which maps the text into a vector of a fixed dimension. For each term t in the term dictionary i , t i .content = t i .name + t i .definition, calculate V(t i .content) = Vectorize(t i .content); and perform hash encoding processing on t i .name to generate a unique identifier for the term; where, t i .name represents the name of the i-th term, t i .definition represents the definition of the i-th term, t i .content represents the content of the i-th term, including the combination of the term name and definition. V(·) represents a vectorization function used to map the text content into a vector of a fixed dimension. Vectorize(·) represents the operation of vectorizing the text, converting the input text into a numerical vector of a fixed dimension. text represents the text content to be processed, and vector represents the mathematical representation of the text in a high-dimensional space; b. For each business rule item r in the business rule library j , r j .content = r j .source + r j .target + r j .description + r j .constraint, calculate V(r j .content) = Vectorize(r j .content), define r j .name = r j .source + r j .target, and perform hash encoding processing on r j .name to generate a unique identifier for the rule; where r j .description represents the description of the j-th business rule, r j .constraint represents the constraint condition of the j-th business rule, r j .source represents the starting object on which the j-th business rule acts, r j .target represents the target object on which the j-th business rule acts, r j .content represents the content of the j-th business rule item, including the combination of the object on which the j-th business rule acts, the description of the j-th rule, and the j-th constraint condition. V(·) represents a vectorization function used to map text content into a vector of a fixed dimension, and Vectorize(·) represents an operation to vectorize text, converting the input text into a numerical vector of a fixed dimension, r j .name represents the object on which the j-th business rule acts, including the combination of the starting object on which the j-th business rule acts and the target object on which the j-th business rule acts; S23: Set the similarity threshold parameter. By calculating the cosine similarity of vectors, compare the content t i .content of the term with the content r j .content of the business rule item to identify the newly added domain knowledge. Store the identified new terms in the corresponding domain term dictionary vector library, store the identified new rule items in the corresponding domain business rule vector library, and at the same time insert the new knowledge into the knowledge graph, where the starting object of the rule action is used as the entity node, the business rule description is used as the edge name, and the constraint condition is used as the edge attribute.

4. A software requirement defect detection method based on a large language model and a satisfiability problem solver according to claim 1, characterized in that, The specific steps of S3 are as follows: S31, Receive the input of the software requirements text to be detected, and define the requirements document structure Req = {id, text, chunks[]}, where id represents the unique identifier of the requirements document, text represents the content of the requirements text, and chunks[] is an array of requirements text blocks. Establish a storage index for each software requirements text, including entity index, relationship index, and text block index; S32, Perform chunking processing on the software requirements text, use a vector model to vectorize each requirements text block, and at the same time perform hash encoding processing on each requirements text block to generate a unique identifier for the text block; S33. Identify newly added requirement text chunks by comparing whether the hash code of each requirement text chunk exists in the repository of the requirement text. Maintain a set of hashes of processed text chunks ProcessedHashes = {hash1, hash2,...} for each requirement document, where hash1 represents the hash of the first processed text chunk. Define an array of newly added text chunks new_chunks[]. For the currently processed text chunk chunk k , check whether the hash code value of the text chunk chunk k .hash ∈ ProcessedHashes. If it exists, mark it as a processed text chunk; if it does not exist, mark it as a newly added text chunk, add it to the new_chunks array, and add the hash value to ProcessedHashes.

5. A software requirement defect detection method based on a large language model and a satisfiability problem solver according to claim 1, characterized in that, The specific steps of S4 are as follows: S41, Construct a prompt word framework system for software requirements analysis; S42, Develop key element extraction prompt words according to the entity types in the domain knowledge base; S43, Select a small number of entity and relationship extraction examples according to the area to which the software requirements belong and splice them into the prompt words; S44, Based on the prompt word framework system, use the large language model to evaluate and optimize the effect of the prompt words; S45, traverse the new_chunks array described in S33, loop to extract key elements from text chunks, and construct an entity set entities = {e1, e2,.., e u} and a relationship set relationships = {rs1, rs2,..., rs z}; where e u represents the u-th entity extracted, and rs z represents the z-th relationship extracted.

6. A software requirement defect detection method based on a large language model and a satisfiability problem solver according to claim 1, characterized in that The specific steps of S5 are as follows: S51, Receive the defect query question of the software requirements text, and use the model to identify whether the query question is relevant to the analysis of software requirements defects. If it is not relevant to the analysis of software requirements defects, then reply that the analysis of the query question cannot be performed; if it is relevant to the analysis of software requirements defects, then continue to execute the subsequent steps; S52. Traverse and extract entities in the requirement text block, and select the similarity threshold θ entity , for each entity e u , calculate the similarity with the terms in the domain knowledge base. Through similarity comparison, identify similar entities in the domain knowledge base. When σ(e u , t i ) ≥ θ entity , add t i to the list of similar entities of entity e u ; where σ(e u , t i ) represents the comprehensive similarity between the entity and the term, and θ entity represents the threshold for judging the comprehensive similarity between the entity and the term; S53. Use similar entities to find relevant business rules and constraints in the domain knowledge graph; S54. Traverse the similar entities to obtain the corresponding requirement text blocks; S55. Use the information found and the newly added requirement text blocks as context information to build a prompt engineering to guide the large language model to identify the incompleteness and ambiguity in the requirement text; S56. Correctly annotate the identified incompleteness and ambiguity.

7. A software requirement defect detection method based on a large language model and a satisfiability problem solver according to claim 1, characterized in that In step S6, after converting the requirement text into a set of logical expressions, use a SAT solver to detect requirement inconsistencies, then splice the newly added requirement text blocks with the associated existing text blocks, and then map the natural language description to a first-order logical expression through a logical conversion template, and convert the logical expression into conjunctive normal form (CNF), whose form is: where A, B, and C are Boolean variables, B and C are literals, and the literal values can be True or False, and (B ∨ C) is a clause; a SAT solver is used to detect the unsatisfiability of the CNF formula. If it is unsatisfiable, it indicates that there is an inconsistency in the requirements.

8. A software requirement defect detection method based on a large language model and a satisfiability problem solver according to claim 1, characterized in that, The specific steps of S7 are as follows: S71. Incrementally update the set of entities entities = {e1, e2,.., e u} and the set of relationships relationships = {rs1, rs2,..., rs z}, where u represents the u-th entity extracted and z represents the z-th relationship extracted. For the requirement text information established for the first time, in addition to the text block index established in step S3, establish two vector library indexes, which are respectively used to store entities, relationships, and their attributes; S72. For the entity set, use a vector model to vectorize the entity names and calculate the vectors: V(e u .name) = Vectorize(e u .name) Calculate e u .name hash value as the unique identifier: e u .hash = Hash(e u .name) For the relationship set, rs z .content = rs z .source + rs z .target + rs z .description + rs z .constraint, calculate the vector: V(rs z .content) = Vectorize(rs z .content) Calculate the hash value of rs using the MD5 hash algorithm z .source and rs z .target as the unique identifier; Among them, V(·) represents the vectorization function, which is used to convert the text block into a vector representation of a fixed dimension, e u .name represents the name of the u-th entity, Vectorize(·) represents the vectorization operation, which converts the text content into a vector of a fixed dimension, e u .hash represents the hash value of the name of the u-th entity, rs z .source represents the starting entity in the z-th relationship, rs z .target represents the target entity in the z-th relationship, rs z .description represents the description of the z-th relationship, rs z .constraint represents the constraint condition of the z-th relationship, rs z .content represents the content of the z-th relationship, including the combination of its starting entity, target entity, description and constraint condition. Hash(·) represents the hash function, which is used to generate a unique identifier for the text content; S73, maintain a set of processed entity hashes ProcessedEntityHashes = {hash e1 , hash e2 ,...} for each requirements document; define a set of new entities new_entity, for the entity e being currently processed u , check if e u .hash ∈ ProcessedEntityHashes; S74. For the entity set entities, if entity e already exists in the knowledge base u , then update the attributes of entity e u . If entity e does not exist in the knowledge base u , then add entity e u .

9. A software requirement defect detection method based on a large language model and a satisfiability problem solver according to claim 1, characterized in that, The detection report in S8 contains defect location information and correction suggestions. The specific steps to generate the detection report are as follows: S81: Receive the user's feedback query on the generated defect detection report; S82: Use the model to identify whether the user's query is related to software requirement defect problems. If so, according to the query content, use the large language model to retrieve relevant knowledge from the knowledge base and give defect correction suggestions. When relevant knowledge cannot be retrieved from the knowledge base, use the large language model to generate correction suggestions and mark that the suggestion is generated by the model. When relevant knowledge is retrieved from the knowledge base, provide the retrieved knowledge through the large language model and mark that the suggestion comes from the knowledge base data; if not, reply that no relevant suggestions can be provided.

Images