Software demand quality evaluation data processing method and device, equipment and medium

Abstract

The invention discloses a software demand quality evaluation data processing method and device, equipment and a medium, and the method comprises the steps: obtaining an initial software demand text, and recognizing a candidate demand text containing a predefined defect indication word through semantic analysis; for each candidate text, matching or generating a high-quality demand text corresponding to semantics from a historical demand library or a domain knowledge library, and constructing a difference sample pair; and labeling the sample pair input into the arbitration system based on the large language model, and automatically generating a quality label corresponding to the demand text. And generating a high-quality evaluation data set based on all the labeled difference sample pairs. According to the technical scheme, automation of data set construction can be achieved, the structural design can effectively reveal context ambiguity of the demand text, and a reliable reference is provided for training and evaluating a high-precision demand quality detection model. The method can be widely applied to the technical field of software.

Description

Technical Field

[0001] This application relates to the field of software technology, and in particular to a method, apparatus, equipment and medium for processing software requirements quality assessment data. Background Technology

[0002] In the field of software development, the software requirements specification is the core document defining the functions, performance, and other constraints of a software system, and its quality directly determines the success or failure of a software project. To ensure the quality of software requirements information description, automated inspection technology has emerged. It uses artificial intelligence models to identify ambiguities and unverifiable defects in software requirements, thereby facilitating the standardized use of software requirements and becoming a key means of improving software development efficiency. However, the detection capabilities of these AI-based inspection tools are highly dependent on the quality and structure of the underlying dataset used for training and evaluation (i.e., software requirements quality assessment data). If the quality of the software requirements quality assessment data is poor, the performance of the trained AI model will also be difficult to achieve the desired results.

[0003] In related technologies, the construction of software requirements quality assessment datasets generally faces technical bottlenecks, making it difficult to meet the high requirements of industrial applications for assessment tools. For example, existing methods often result in datasets with scattered sample distributions and a lack of internal logical connections, making trained detection models insensitive to complex contextual semantic differences in requirement texts and exhibiting weak generalization ability. Furthermore, the dataset construction process typically relies heavily on inefficient and subjective manual annotation procedures, limiting the dataset's scale, consistency, and objectivity. These factors collectively make existing datasets unsuitable as effective benchmarks for training artificial intelligence models, thus affecting the accuracy and reliability of assessment tools.

[0004] In summary, the problems with the relevant technologies urgently need to be addressed. Summary of the Invention

[0005] The purpose of this application is to at least partially solve one of the technical problems existing in the related art.

[0006] Therefore, one object of the embodiments of this application is to provide a method, apparatus, device and medium for processing software requirements quality assessment data.

[0007] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of this application include: On the one hand, embodiments of this application provide a method for processing software requirements quality assessment data, the method comprising: Obtain multiple initial requirement texts to describe the software requirements; Semantic recognition is performed on the initial requirement text, and candidate requirement texts containing predefined defect indicator words are determined from the initial requirement text based on the recognition results; For each candidate requirement text, a corresponding positive requirement text is determined, and the candidate requirement text and the positive requirement text are combined into a difference sample pair; wherein, the positive requirement text is a requirement text that corresponds semantically to the candidate requirement text and does not contain the defect indicator word; Each set of difference sample pairs is input into an arbitration system based on a large language model. The arbitration system annotates the candidate requirement text and the positive requirement text to obtain the labels corresponding to the candidate requirement text and the positive requirement text. The labels are used to characterize whether the requirement text contains defects or not. Based on the candidate requirement text, the positive requirement text, and the tags, a set of software requirement quality evaluation data is constructed.

[0008] In addition, the software requirements quality assessment data processing method according to the above embodiments of this application may also have the following additional technical features: Furthermore, in one embodiment of this application, obtaining multiple initial requirement texts for describing software requirements includes: Monitor and capture requirement change events from at least one external requirement management system through predefined application interfaces or webhooks; Based on the requirement change event, the initial requirement text used to describe the software requirements is extracted.

[0009] Furthermore, in one embodiment of this application, the step of performing semantic recognition on the initial requirement text and determining candidate requirement texts containing predefined defect indicator words from the initial requirement text based on the recognition result includes: Convert the initial requirement text into a vector representation; Based on the vector representation, the candidate requirement text is identified using a natural language processing model.

[0010] Furthermore, in one embodiment of this application, determining a corresponding positive demand text for each candidate demand text includes at least one of the following: If the candidate requirement text is derived from a historical requirement text, the requirement text containing clear quantitative indicators that existed in the previous version is retrieved and extracted by comparing the requirement version history as the positive requirement text. Alternatively, based on the vectorized representation of the candidate demand texts, demand texts that are semantically and syntactically similar and contain clear quantitative indicators can be retrieved from a vector database or historical demand database as the positive demand texts. Alternatively, based on the domain involved in the candidate requirement text, practice rules or failure safety principles from the domain knowledge base can be invoked to generate a requirement text that meets the requirements of being clear and verifiable, which can then be used as the positive requirement text.

[0011] Furthermore, in one embodiment of this application, the arbitration system includes a coordinator agent and multiple heterogeneous expert agents; the step of annotating the candidate demand text and the positive demand text through the arbitration system includes: The coordinator agent distributes the difference sample pairs to each of the expert agents. Each expert agent, based on a preset domain knowledge role, independently and in parallel analyzes the demand text in the difference sample pairs to obtain preliminary original labels and judgment criteria. Based on the original tags and the judgment criteria, the candidate demand text and the tags corresponding to the positive demand text are determined.

[0012] Furthermore, in one embodiment of this application, determining the candidate demand text and the corresponding tags for the positive demand text based on the original tags and the determination criteria includes: The coordinator agent detects whether there are conflicts in the original labels generated by each of the expert agents. If there is a conflict in the original labels, a structured debate process is initiated, in which the coordinator agent distributes the judgment criteria of each expert agent and guides the expert agents to conduct cross-evaluation and rebuttal of arguments. After at least one round of debate, the label is output based on the consensus reached or according to the preset adjudication rules.

[0013] Furthermore, in one embodiment of this application, after constructing the set of software requirements quality assessment data, the method further includes: The software requirements quality assessment data is anonymized. Generate and publish the version number information corresponding to the set of software requirement quality evaluation data.

[0014] On the other hand, embodiments of this application provide a software requirements quality assessment data processing apparatus, the apparatus comprising: The acquisition unit is used to acquire multiple initial requirement texts that describe software requirements; The identification unit is used to perform semantic recognition on the initial requirement text and determine candidate requirement texts containing predefined defect indicator words from the initial requirement text based on the recognition result. A combination unit is configured to determine a corresponding positive demand text for each candidate demand text, and combine the candidate demand text and the positive demand text into a difference sample pair; wherein, the positive demand text is a demand text that semantically corresponds to the candidate demand text and does not contain the defect indicator word; An arbitration unit is used to input the pairs of differing samples from each group into an arbitration system based on a large language model. The arbitration system annotates the candidate requirement text and the positive requirement text to obtain labels corresponding to the candidate requirement text and the positive requirement text. The labels are used to characterize whether the requirement text contains defects or not. The construction unit is used to construct a set of software requirement quality evaluation data based on the candidate requirement text, the positive requirement text, and the tags.

[0015] On the other hand, embodiments of this application provide an electronic device, including: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the above-described method for processing software requirement quality assessment data.

[0016] On the other hand, embodiments of this application also provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to implement the above-described method for processing software requirements quality assessment data.

[0017] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program stored in a computer-readable storage medium. The processor of an electronic device reads the computer program from the computer-readable storage medium and executes the computer program, causing the electronic device to perform the processing method for implementing the software requirements quality assessment data described above.

[0018] The advantages and beneficial effects of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application: This application discloses a method, apparatus, device, and medium for processing software requirement quality assessment data. It acquires initial software requirement text and identifies candidate requirement texts containing predefined defect indicator words through semantic analysis. For each candidate text, it matches or generates a semantically corresponding high-quality requirement text from a historical requirement database or domain knowledge base, constructing a difference sample pair. The sample pairs are input into an arbitration system based on a large language model for annotation, automatically generating quality labels corresponding to the requirement texts. A high-quality assessment dataset is generated based on all labeled difference sample pairs. The technical solution of this application automates dataset construction, and its structured design effectively reveals the contextual ambiguity of requirement texts, providing a reliable benchmark for training and evaluating high-precision requirement quality detection models. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of this application or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions in this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0020] Figure 1 This is a schematic diagram of the implementation environment for a software requirements quality assessment data processing method provided in this application embodiment; Figure 2 This is a flowchart illustrating a method for processing software requirements quality assessment data provided in an embodiment of this application. Figure 3 This is a schematic diagram of a process for obtaining initial requirement text provided in an embodiment of this application; Figure 4 This is a schematic diagram of a process for identifying candidate requirement text provided in an embodiment of this application; Figure 5 This is a schematic diagram illustrating a process of annotation via an arbitration system provided in an embodiment of this application; Figure 6 This is a system architecture diagram of a method for processing software requirements quality assessment data provided in the embodiments of this application; Figure 7 This is a flowchart illustrating the specific implementation of a method for processing software requirements quality assessment data provided in this application embodiment; Figure 8 This is a schematic diagram of an automated debate process for agent consensus arbitration provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a software requirements quality assessment data processing device provided in the embodiments of this application; Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] The present application will be further described below with reference to the accompanying drawings and specific embodiments. The described embodiments should not be considered as limitations on the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0022] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0023] Before introducing the technical solution of this application, let's first introduce some of the technical terms involved in this application: 1) Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0024] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interactive systems, and mechatronics. Among these, pre-trained models, also known as large-scale models or foundational models, can be widely applied to downstream tasks across various AI fields after fine-tuning. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0025] 2) Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and pre-trained learning. Pre-trained models are the latest development in deep learning, integrating the above techniques.

[0026] In the field of software development, the software requirements specification is the core document defining the functions, performance, and other constraints of a software system, and its quality directly determines the success or failure of a software project. To ensure the quality of software requirements information description, automated inspection technology has emerged. It uses artificial intelligence models to identify ambiguities and unverifiable defects in software requirements, thereby facilitating the standardized use of software requirements and becoming a key means of improving software development efficiency. However, the detection capabilities of these AI-based inspection tools are highly dependent on the quality and structure of the underlying dataset used for training and evaluation (i.e., software requirements quality assessment data). If the quality of the software requirements quality assessment data is poor, the performance of the trained AI model will also be difficult to achieve the desired results.

[0027] In related technologies, the construction of software requirements quality assessment datasets generally faces technical bottlenecks, making it difficult to meet the high requirements of industrial applications for assessment tools. For example, existing methods often result in datasets with scattered sample distributions and a lack of internal logical connections, making trained detection models insensitive to complex contextual semantic differences in requirement texts and exhibiting weak generalization ability. Furthermore, the dataset construction process typically relies heavily on inefficient and subjective manual annotation procedures, limiting the dataset's scale, consistency, and objectivity. These factors collectively make existing datasets unsuitable as effective benchmarks for training artificial intelligence models, thus affecting the accuracy and reliability of assessment tools.

[0028] Specifically, the inventors of this application, through in-depth research, have discovered that current benchmark dataset construction methods suffer from the following technical bottlenecks, preventing them from meeting the requirements of modern industrial-grade software development for evaluation tools: 1. The sample structure fails to reveal the technical challenge of "contextual ambiguity": Existing datasets mostly employ random sampling or simple keyword matching strategies. This approach fundamentally fails to address the "contextual ambiguity" problem prevalent in software requirements texts. For example, the same defect indicator may be an acceptable description in one technical context, but constitute a vague, unverifiable, and fatal quality defect in another. Existing methods collect samples randomly and without order, resulting in datasets filled with a large amount of "contextual noise." The trained evaluation models have extremely weak ability to perceive contextual semantics and cannot accurately distinguish the quality connotations of the same word in different scenarios, leading to persistently high false positive and false negative rates in industrial applications.

[0029] 2. The dataset structure does not support effective differential regression testing of evaluation tools: A powerful evaluation tool must not only be able to "find" defects correctly, but also be able to "distinguish" similar non-defect scenarios. An ideal regression test set must be able to quantify this fine-grained discrimination capability of the tool. However, existing datasets cannot provide "minimum difference sample pairs" for the same defect indicator word, under similar grammatical structures, where the quality judgment results are opposite only due to differences in contextual semantics. This lack of structured samples makes it impossible to perform differential, fine-grained regression testing on evaluation tools, and to assess whether their ability to distinguish subtle semantic differences deteriorates after model upgrades. This is a key technical bottleneck in the current field of automated requirement quality assessment.

[0030] 3. The manual annotation mechanism has inherent bottlenecks in efficiency and objectivity: Even when human experts are used for annotation, the process itself has inherent technical limitations: 1) It is costly and inefficient, making it difficult to adapt to the needs of large-scale, rapid iteration software development; 2) The consensus-building process is highly dependent on synchronous communication, resulting in serious process bottlenecks; 3) The judgment of human experts is affected by subjective bias, knowledge limitations, and fatigue, leading to inconsistencies in the annotation results that are difficult to quantify.

[0031] Therefore, how to construct a consensus mechanism that can both simulate expert-level knowledge and achieve high efficiency, objectivity, and automation is another major technical challenge that urgently needs to be solved in this field.

[0032] In view of this, this application provides a method, apparatus, device, and medium for processing software requirement quality assessment data, which improves the technical problem of fragmented dataset samples in the prior art, which cannot support effective assessment. The method of this application includes: acquiring initial software requirement text; identifying candidate requirement texts containing predefined defect indicator words through semantic analysis; for each candidate text, matching or generating a semantically corresponding high-quality requirement text from a historical requirement database or domain knowledge base to construct a difference sample pair; inputting the sample pair into an arbitration system based on a large language model for annotation, automatically generating quality labels corresponding to the requirement text; and generating a high-quality assessment dataset based on all labeled difference sample pairs. The technical solution of this application can automate dataset construction, and its structured design can effectively reveal the contextual ambiguity of requirement texts, providing a reliable benchmark for training and evaluating high-precision requirement quality detection models.

[0033] Please refer to Figure 1 , Figure 1 This diagram illustrates an implementation environment for a software requirements quality assessment data processing method provided in this embodiment. In this implementation environment, the main hardware and software components involved include a terminal device 110 and a backend server 120. The terminal device 110 and the backend server 120 are connected by communication.

[0034] The software requirement quality assessment data processing method provided in this application embodiment can be configured to be executed separately on the terminal device 110, or separately on the backend server 120, or it can be executed based on the data interaction between the terminal device 110 and the backend server 120.

[0035] Terminal device 110 is typically a client for performance testing engineers or system maintenance personnel to operate and interact with. Its specific form can be a personal computer, laptop, workstation, or dedicated monitoring terminal. Backend server 120 is the server that carries the core processing logic and data computation. It can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0036] The terminal device 110 and the backend server 120 can establish a communication connection via a wireless network or a wired network. This wireless or wired network uses standard communication technologies and / or protocols. The network can be the Internet or any other network, including but not limited to any combination of Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), mobile, wired or wireless networks, private networks, or virtual private networks.

[0037] The software functional modules related to the software requirement quality evaluation data processing method provided in this application embodiment can be configured on the above-mentioned hardware in different ways according to different application scenarios and resource allocation requirements, and this application does not limit them.

[0038] Of course, this is understandable. Figure 1 The implementation environment described in this application is merely one of the optional application scenarios for the software requirements quality assessment data processing method provided in this embodiment. Actual applications are not fixed. Figure 1 The software and hardware environment shown.

[0039] Below, in conjunction with the aforementioned description of the implementation environment, a method for processing software requirements quality assessment data provided in this application embodiment will be introduced and explained.

[0040] Please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating a method for processing software requirement quality assessment data according to an embodiment of this application. Specifically, the method for processing software requirement quality assessment data provided in this embodiment includes, but is not limited to: Step 210: Obtain multiple initial requirement texts to describe the software requirements; Step 220: Perform semantic recognition on the initial requirement text, and determine candidate requirement texts containing predefined defect indicator words from the initial requirement text based on the recognition results; Step 230: For each candidate requirement text, determine a corresponding positive requirement text, and combine the candidate requirement text and the positive requirement text into a difference sample pair; wherein, the positive requirement text is the requirement text that corresponds semantically to the candidate requirement text and does not contain defect indicator words. Step 240: Input the pairs of differing samples into the arbitration system based on the large language model. The arbitration system will label the candidate requirement text and the positive requirement text to obtain the corresponding labels for the candidate requirement text and the positive requirement text. The labels are used to characterize whether the requirement text contains defects or not. Step 250: Based on the candidate requirement text, positive requirement text, and tags, construct a set of software requirement quality evaluation data.

[0041] This application provides an automated processing method for software requirement quality assessment data, which constructs a high-quality, structured benchmark dataset through a design-driven strategy. The method first acquires initial requirement text and intelligently identifies candidate texts that may contain quality defects. Then, it systematically matches or generates a semantically corresponding but clearly descriptive high-quality text as a comparison for each candidate text, thus forming structured difference sample pairs. Most importantly, the method introduces an arbitration system based on a large language model. By simulating parallel judgment and consensus arbitration mechanisms from multiple expert perspectives, it automatically generates objective and reliable quality labels for the sample pairs, thereby compiling them into a standard dataset for training and evaluating requirement quality detection models.

[0042] The methods and datasets provided in this application have broad application prospects in the field of software development. For example, in some embodiments, it can be applied to the field of artificial intelligence-assisted development as a core training data source to build intelligent requirement quality inspection plugins that can be integrated into IDEs or requirement management tools, providing developers with real-time quality feedback. In some embodiments, in software process improvement and quality management scenarios, it can serve as a link in the CI / CD pipeline to perform automated quality gate checks on submitted requirement documents, improving requirement quality from the source. In some embodiments, in the fields of academic research and tool evaluation, the high-quality benchmark dataset generated by this application can provide a fair and authoritative performance evaluation benchmark for different requirement quality detection algorithms or commercial tools, and can also serve as an ideal teaching case library for analyzing requirement defects and learning high-quality requirement writing standards. This application does not limit its specific application scenarios.

[0043] The following section provides a detailed description and explanation of the software requirements quality assessment data processing method provided in the embodiments of this application, with specific steps.

[0044] Step 210 involves acquiring multiple initial requirement texts to describe software requirements. This step marks the starting point of the data processing flow, aiming to collect raw requirement description materials from various sources. These initial requirement texts are the foundational raw materials for building a high-quality evaluation dataset. In practice, initial requirement texts can originate from various channels, such as user manuals entered in project requirement management tools, functional clauses extracted from software requirement specifications, feature descriptions in product design documents, or functional discussion records on online collaboration platforms. By extensively acquiring these initial requirement texts, a rich data foundation is laid for subsequent in-depth processing and analysis, ensuring that the final dataset covers diverse business scenarios and requirement expression styles.

[0045] In this embodiment, the specific content of the initial requirements text is not limited. Its content can be a highly structured and formalized expression, such as independent functional clauses, performance indicator definitions, or security constraints derived from a software requirements specification; or it can be a relatively unstructured description, such as user instructions extracted from agile development management tools, product backlog items, or feature point discussion records on online collaboration platforms. This text can cover various levels, from high-level business objectives to specific implementation details.

[0046] In step 220, after obtaining the initial requirement text, semantic recognition is performed on it. Based on the recognition results, candidate requirement texts containing predefined defect indicator words are identified from the initial requirement text. The main task of this step is to intelligently filter out candidate objects that may have quality problems from a massive amount of initial requirements, i.e., candidate requirement texts, which is key to improving the efficiency of subsequent processing.

[0047] Specifically, firstly, a deep semantic analysis can be performed on the initial requirement text to understand its core intent and descriptive logic. Based on this, the system will perform matching based on a predefined feature library containing common quality defect patterns. This feature library covers typical patterns such as vague vocabulary, unverifiable expressions, incomplete logic, or violations of domain-specific rules. Through this refined identification process, candidate requirement texts that may contain ambiguity, inaccuracy, untestability, or other quality risks can be accurately located from a large number of requirements, thus providing precise input for subsequently building targeted comparison sample pairs.

[0048] It should be noted that in this embodiment, the candidate requirement text is text that may have quality issues, and does not imply that it actually has quality issues. Identifying an initial requirement text as a candidate requirement text only indicates that the text contains specific defect indicator words (such as vague terms), thereby triggering the system's subsequent analysis process and qualifying it for in-depth review. However, in certain specific technical contexts or business scenarios, the use of these indicator words may be reasonable and acceptable. For example, when describing high-level business objectives or non-functional requirements, the word "high" in "the system should have high availability" may be an appropriate and widely accepted expression.

[0049] In step 230, for each candidate requirement text, a corresponding positive requirement text is determined, and the candidate requirement text and the positive requirement text are combined into a difference sample pair. The purpose of this step is to construct the smallest contrast unit that can highlight the quality difference.

[0050] Specifically, for each candidate requirement text that may have defects identified in the previous step, this step actively seeks out or constructs a positive requirement text that highly corresponds in semantic intent but meets high standards of clarity and verifiability in terms of expression quality. These two requirement texts, one positive and one negative, constitute a "difference sample pair." This pairing design ensures that each sample unit in the dataset contains clear quality comparison information, thereby greatly enhancing the pedagogical and evaluative capabilities of the dataset and providing ideal material for training a detection model that can keenly distinguish subtle quality differences.

[0051] In step 240, the pairs of differing samples are input into an arbitration system based on a large language model. The arbitration system annotates the candidate requirement texts and positive requirement texts to obtain corresponding labels. This step introduces an automated annotation mechanism to ensure the objectivity and authority of the quality labels. In this embodiment, the arbitration system utilizes the deep language understanding and domain knowledge reasoning capabilities of the large language model to intelligently analyze and evaluate the input pairs of differing samples. Specifically, the system comprehensively considers the clarity, verifiability, completeness, and conformity with domain best practices of the requirement description, performs a quality assessment on each requirement text in the sample pair, and outputs a final label indicating whether it contains defects.

[0052] It's easy to understand that this automated annotation process effectively overcomes the bottlenecks of traditional manual annotation, such as strong subjectivity, low efficiency, and difficulty in scaling, providing technical support for generating large-scale, highly consistent gold standard datasets. Furthermore, submitting structured pairs of differing samples, rather than isolated single requirements, to the arbitration system for adjudication brings deeper advantages: this side-by-side comparison provides valuable contextual information to the arbitration system (especially large language models), enabling it to precisely focus its analysis on the key semantic units that cause quality differences, thus greatly promoting the interpretability of the judgment process. By comparing and analyzing the subtle differences in the expression of positive and negative samples, the arbitration system can more clearly understand what is "fuzzy" versus "clear," and what is "unverifiable" versus "measurable." This comparison-based learning and judgment mechanism not only improves the accuracy of the arbitration results themselves, but the detailed judgment criteria and logical reasoning processes generated also constitute highly valuable metadata, providing profound guidance for subsequent optimization of requirement writing specifications and even training more accurate detection models, achieving a sublimation from simple annotation to in-depth analysis and knowledge extraction.

[0053] In step 250, all processing results are integrated into a structured dataset that can be directly used for downstream tasks. Specifically, all discrepancy sample pairs that have undergone automated arbitration and are labeled with quality tags are systematically organized, formatted, and serialized. The resulting evaluation dataset is a well-structured, accurately labeled resource collection rich in comparative information. This dataset can be directly used to train next-generation intelligent software requirement quality inspection models, and can also serve as a benchmark set for objectively evaluating the performance of different inspection tools or algorithms, thereby promoting the automation and intelligent development of software requirement quality assurance technology.

[0054] It is understood that the software requirement quality assessment data processing method provided in this application involves obtaining initial software requirement text and identifying candidate requirement texts containing predefined defect indicator words through semantic analysis. For each candidate text, a semantically corresponding high-quality requirement text is matched or generated from a historical requirement database or domain knowledge base, constructing a difference sample pair. The sample pair is input into an arbitration system based on a large language model for annotation, automatically generating quality labels corresponding to the requirement text. A high-quality assessment dataset is generated based on all labeled difference sample pairs. This method automates dataset construction, and its structured design effectively reveals the contextual ambiguity of requirement texts, providing a reliable benchmark for training and evaluating high-precision requirement quality detection models.

[0055] Specifically, in some embodiments, reference is made to Figure 3 Obtain multiple initial requirement texts used to describe software requirements, including: Monitor and capture requirement change events from at least one external requirement management system through predefined application interfaces or webhooks; Based on the requirement change event, the initial requirement text used to describe the software requirements is extracted.

[0056] In this embodiment, the initial requirements text is not obtained through traditional manual collection or batch import methods, but rather through an event-driven, automated capture mechanism deeply integrated with the development process. This mechanism establishes a real-time connection with one or more external requirements management systems (such as Jira, ZenTao, Confluence, etc.) through predefined application interfaces or webhooks, continuously monitoring requirement change events occurring in the system.

[0057] Once a new requirement is created or an existing requirement is modified, the system immediately captures the event notification. The processing module then parses and extracts the core content of the requirement description from the event payload, resulting in a structured initial requirement text. This method ensures the high timeliness of the acquired requirement data, accurately reflecting the latest status of requirements during software development, and laying a solid foundation for building a dynamically updated, high-quality evaluation dataset synchronized with project progress. Simultaneously, this automated data inflow method significantly reduces manual intervention, improving the efficiency and consistency of the entire dataset construction process.

[0058] Specifically, in some embodiments, reference is made to Figure 4 The initial requirement text undergoes semantic recognition. Based on the recognition results, candidate requirement texts containing predefined defect indicator words are identified from the initial requirement text, including: Convert the initial requirements text into a vector representation; Candidate requirement texts are obtained by using vector representation and natural language processing models.

[0059] In this embodiment, the semantic recognition process of the initial requirement text can employ a vectorized analysis method based on deep semantic understanding. This method first uses a natural language processing model to transform the initial requirement in text format into a vector representation in a high-dimensional space. This process can deeply capture the semantic connotation and contextual relationships of the requirement text. Subsequently, based on the generated vector representation, precise matching and pattern recognition are performed in the semantic space, thereby filtering out candidate requirement texts containing predefined defect indicator words from a massive number of initial requirements.

[0060] Understandably, this vector representation-based recognition method can effectively overcome the problems of low accuracy and poor generalization ability in traditional keyword matching, significantly improve the recall ability and recognition accuracy of potential quality defects, and facilitate the subsequent construction of high-quality differential sample pairs.

[0061] Specifically, in some embodiments, for each candidate demand text, a corresponding positive demand text is determined, including at least one of the following: If the candidate requirement text is derived from a historical requirement text, the requirement text containing clear quantitative indicators in the previous version that exists in the previous version is retrieved and extracted as the positive requirement text by comparing the requirement version history. Alternatively, based on the vectorized representation of candidate demand texts, demand texts that are semantically and syntactically similar and contain clear quantitative indicators can be retrieved from a vector database or historical demand database as positive demand texts. Alternatively, based on the domain involved in the candidate requirement text, the practice rules or failure safety principles in the domain knowledge base can be invoked to generate a requirement text that meets clear and verifiable requirements as the positive requirement text.

[0062] In this application, the process of determining the corresponding positive requirement text for candidate requirement text can be implemented in various ways. For example, in some embodiments, the version history of the requirement itself can be utilized to find explicit expressions lost due to modifications by comparing change records. This approach can preserve the original business context to the greatest extent possible. In some embodiments, semantic and structural retrieval can be used, such as in vector databases and historical requirement databases, to retrieve samples that are highly similar to negative samples (i.e., candidate requirement texts) in semantic and syntactic structure but contain explicit, quantifiable indicators as positive requirement texts. In some embodiments, a knowledge-based synthesis approach can also be adopted, under the guidance of a domain knowledge base, to automatically synthesize a high-quality requirement that conforms to "failure safety" or best practice principles as positive requirement text. This application does not impose any limitations on this approach.

[0063] Specifically, in some embodiments, the arbitration system includes a coordinator agent and multiple heterogeneous expert agents; see reference Figure 5 The arbitration system is used to annotate candidate requirement texts and positive requirement texts, including: The coordinator agent distributes the discrepancy sample pairs to the various expert agents. Each expert agent, based on its pre-defined domain knowledge role, independently and in parallel analyzes the demand text in the differential sample pairs to obtain preliminary original labels and judgment criteria. Based on the original tags and judgment criteria, determine the tags corresponding to the candidate requirement text and the positive requirement text.

[0064] In this embodiment, the annotation process of the arbitration system for discrepancy sample pairs can simulate the collaborative mode of an expert review meeting. A highly structured parallel processing and consensus decision-making mechanism ensures the authority and objectivity of the annotation results. A coordinator agent acts as the process scheduler, distributing the discrepancy sample pairs to be adjudicated to expert agents representing different professional perspectives. Each expert agent, based on its pre-defined domain knowledge role (e.g., requirements analysis, software testing, or system development), performs independent and parallel in-depth analysis of the samples, generating preliminary quality judgments and their detailed logical basis. This parallel mechanism not only significantly improves processing efficiency but, more importantly, lays a solid foundation for subsequent in-depth analysis by introducing multi-dimensional professional perspectives. Finally, based on these parallel analysis results and their judgment criteria, the system synthesizes the final quality label through a specific decision-making mechanism. This process effectively integrates multi-domain professional knowledge, making the annotation conclusions both comprehensive and profound, significantly surpassing the limitations of a single model or manual annotation.

[0065] For example, an arbitration system may include a coordinator agent and three heterogeneous expert agents, such as: an analytical agent (infused with knowledge of requirements engineering theory and formal standards), a testing agent (infused with knowledge of software testing theory and a large number of historical defect reports), and a development agent (infused with knowledge of software architecture and specific implementation logic). Each expert agent is associated with a domain knowledge base and equipped with a "judgment criterion generation module" based on a large language model.

[0066] Specifically, in some embodiments, the tags corresponding to candidate demand text and positive demand text are determined based on the original tags and the determination criteria, including: The coordinator agent detects whether there are conflicts in the original labels generated by each expert agent. If there is a conflict in the original labels, the structured debate process is initiated, where the coordinator agent distributes the judgment criteria of each expert agent and guides the expert agents to conduct cross-evaluation and rebuttal of arguments. After at least one round of debate, a label is output based on the consensus reached or according to the pre-set adjudication rules.

[0067] In this embodiment of the application, the arbitration system can use an automated arbitration protocol to output labels. Its specific logic is a structured multi-round debate mechanism, including the following process: a) Independent Parallel Labeling: The coordinator agent distributes the requests to be arbitrated to all expert agents, which then output preliminary labeling results (defect / non-defect) and their judgment criteria in parallel and independently.

[0068] b) Automated conflict detection: The coordinator agent collects all results and automatically initiates a debate process if inconsistencies are found.

[0069] c) Multi-round structured debate: The coordinating agent broadcasts all decision criteria to all expert agents and guides them through multiple rounds of cross-evaluation and argument rebuttal according to a pre-set debate strategy. During this process, expert agents are allowed to cite specific provisions from the domain knowledge base to strengthen their arguments or change their positions after evaluating valid arguments from other expert agents.

[0070] d) Consensus Reached and Final Decision: The debate terminates upon reaching a consensus or after the preset maximum number of rounds. If no consensus is reached, the coordinating agent will make a final decision based on predefined decision rules (such as assigning higher weights to analytical agents related to functional safety).

[0071] Specifically, in some embodiments, after constructing the set of software requirements quality assessment data, the method further includes: Anonymize the software requirements quality assessment data; Generate and publish the version number information corresponding to the set of software requirements quality assessment data.

[0072] In this embodiment of the application, after the collection of software requirement quality evaluation data is constructed, a data post-processing and publishing process can be executed to ensure the usability, security and maintainability of the dataset.

[0073] Specifically, the relevant data can be anonymized to remove or obfuscate all sensitive information that may involve trade secrets or personal privacy, such as specific project names, internal codes, or personnel information. This allows the dataset to be shared and used while protecting intellectual property rights. Next, the system standardizes and encapsulates the dataset. The resulting structured data not only includes the original requirements, final quality labels, and minimum difference pair identifiers, but also fully records the decision-making path metadata of the multi-agent debate during the arbitration process, greatly enhancing the traceability and research value of the dataset. In this embodiment, the system also generates a unique version number for the dataset, clearly identifying its iteration sequence, and publishes it to a designated repository. This complete release management mechanism ensures clear and controllable dataset versions, providing a stable, reliable, and standardized data foundation for downstream model training, tool evaluation, and academic research.

[0074] The following section, in conjunction with specific application embodiments, introduces and explains the processing method of software requirement quality evaluation data provided in the embodiments of this application.

[0075] In this application embodiment, the application of the technical solution of this application is described in a typical application scenario in the field of industrial automation - building a requirement quality evaluation benchmark dataset for ladder diagram programming software of "programmable logic controller (PLC)".

[0076] Specifically, please refer to Figure 6 , Figure 6 This illustration shows a system architecture diagram of a method for processing software requirements quality assessment data provided in an embodiment of this application. In a CI / CD process of a PLC programming software, when an engineer is in the requirements management system (… Figure 6 This system is automatically triggered when a request is submitted or updated in the DMS (Data Management System). The request processed in this instance is: "PLC-REQ-TIMER-112: When the network connection between the PLC and the host computer is disconnected, the running TON timer should be 'properly handled' to prevent unexpected behavior on the production line."

[0077] Reference Figure 7 The specific processes involved in the embodiments of this application include: Step S101: Data acquisition and association with domain knowledge; like Figure 6 As shown, the data acquisition module in the system retrieves PLC-REQ-TIMER-112 from DMS. The feature extraction module within the system identifies keywords such as "PLC," "TON timer," and "network connection disconnected." Based on these keywords, relevant contextual information, including rule sets from IEC 61131-3 (PLC programming language standard) and IEC 61508 (functional safety standard), is automatically loaded from a pre-set industrial domain knowledge base to prepare for subsequent agent analysis.

[0078] Step S102: Differential sampling based on failure modes; The differential sampling strategy module begins operation: Negative sample identification: Analyzing the requirement text, the system finds that the phrase "handle properly" is an undefined and extremely vague expression. In the field of industrial control, this ambiguity directly violates the basic principles of functional safety. Therefore, the system marks PLC-REQ-TIMER-112 as a high-risk negative context sample. Positive sample matching: The module queries the historical requirement library and domain knowledge base for standard handling methods related to "network interruption." It finds a high-quality historical requirement that conforms to the "fail-safe" design principle: "PLC-REQ-TIMER-045: When Profinet bus communication is interrupted for more than 500ms, all running TON timers must be immediately reset (their accumulated values ​​are cleared to zero, and the output position is set to zero)." This requirement is clear, quantifiable, and unambiguous, and is selected as the paired positive context sample. Sample pair generation: The system pairs these two requirements to form a "minimum difference sample pair" and sends it to the heterogeneous intelligent agent arbitration module for processing.

[0079] Step S103: Heterogeneous intelligent agent consensus arbitration; Please refer to Figure 8 , Figure 8 The automated debate process of consensus arbitration among heterogeneous intelligent agents is described in detail using a time sequence diagram.

[0080] Independent parallel annotation: such as Figure 7 As shown in "Phase 1," the coordinator agent distributes the fuzzy requirement PLC-REQ-TIMER-112 to three parallel expert agents. Based on their respective knowledge bases and reasoning preferences, they provide conflicting initial judgments: Test agent: Determined as "defective" because "proper handling" cannot define verifiable test cases.

[0081] Development agent: judged as "non-defective" on the grounds of "industry practice", which usually means "reset timer", which is technically feasible.

[0082] Analytical intelligent agent: judged as "defective" because the requirement violates the principle in functional safety standards that failure states must be clearly defined.

[0083] Automated conflict detection and multi-round debate: Conflict detection: When the coordinator agent collects a 2:1 conflict result, it automatically enters "Phase 2: Multi-round debate".

[0084] Round 1 of the debate: The coordinating agent broadcasts the initial rationale to all expert agents. It also issues specific instructions, requiring analytical agents to cite concrete safety standards and development agents to reassess the risks of their "industry practice" assumptions.

[0085] Second round of debate (position update): For example Figure 8 As shown, the analytical agent, based on its knowledge base, provides stronger arguments, explicitly citing "IEC 61508-2, Clause 7.4.2.2," and presenting a specific counterexample—a "chemical feed pump"—to demonstrate that the "reset" operation is not always safe. Faced with strong rebuttals from authoritative standards and specific risk scenarios, the development agent updates its judgment logic, ultimately changing its stance and also determining the requirement to be a "defect."

[0086] Consensus and ruling reached: After the second round of debate, the coordinator agent detected that all agents had reached a 3:0 consensus on the "flaw," and the debate ended.

[0087] Step S104: Generate a dataset with detailed debate transcripts.

[0088] Finally, the data post-processing module formats the arbitration results, which are then saved by the dataset storage module. The final generated JSON entry not only contains the final "defect" label, but its metadata field also records detailed information. Figure 7 The entire debate process shown includes the voting results for each round, the original reasons, changes in stance, and the industry standards cited. This makes each annotation in the dataset highly reliable and traceable.

[0089] It is understood that the technical solution of this application has at least the following technical effects: 1. Implemented dataset construction to support differential regression testing, enabling accurate evaluation of the performance of the evaluation tool: Through a unique differential sampling strategy, this application constructs a benchmark dataset containing a large number of "minimum difference pairs". This unique structure allows users to perform fine-grained, quantitative differential regression tests on automated evaluation tools, thereby accurately assessing their ability to distinguish true contextual ambiguity. This overcomes the technical challenge of existing flat datasets failing to deeply evaluate the robustness of tools.

[0090] 2. It fundamentally improves the efficiency, objectivity, and consistency of the annotation process: By replacing human experts with a heterogeneous intelligent agent arbitration system, this application shortens the annotation process, which originally took days or even weeks and required expensive expert resources for coordination, to an automated process in minutes or hours, achieving an order-of-magnitude efficiency improvement. At the same time, through cross-validation from multiple expert perspectives and automated debate, the subjective bias, knowledge blind spots, and state fluctuations of a single expert are effectively eliminated, making the annotation results more objective, stable, and reproducible.

[0091] 3. It provides unprecedented transparency and traceability in decision-making, enhancing the credibility of the dataset: By generating "decision path metadata," this application makes the origin and development of each labeled result clearly traceable. Users not only know "what" it is, but also "why." This greatly enhances the credibility of the dataset as the "gold standard," elevating it from a simple "label set" to a trustworthy knowledge base with a complete "audit log."

[0092] 4. Enhanced quality assurance capabilities for software requirements in key areas (such as industry and healthcare): By deeply integrating domain knowledge bases (such as functional safety standards) into the arbitration logic of intelligent agents, this application can identify, earlier and more accurately, vague or ambiguous requirements that may have serious consequences in a specific industry context. This directly improves early quality assurance capabilities in safety-critical software development, helping to reduce later risks and costs.

[0093] Reference Figure 9 This application also provides a software requirement quality assessment data processing apparatus, which includes: Acquisition unit 910 is used to acquire multiple initial requirement texts that describe software requirements; The identification unit 920 is used to perform semantic recognition on the initial requirement text and determine candidate requirement texts containing predefined defect indicator words from the initial requirement text based on the recognition results. The combination unit 930 is used to determine a corresponding positive requirement text for each candidate requirement text and combine the candidate requirement text and the positive requirement text into a difference sample pair; wherein, the positive requirement text is the requirement text that corresponds semantically to the candidate requirement text and does not contain defect indicator words. Arbitration unit 940 is used to input the pairs of difference samples into the arbitration system based on the large language model. The arbitration system annotates the candidate requirement text and the positive requirement text to obtain the corresponding labels for the candidate requirement text and the positive requirement text. The labels are used to characterize whether the requirement text has defects or not. Building unit 950 is used to construct a set of software requirement quality evaluation data based on candidate requirement text, positive requirement text, and tags.

[0094] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0095] This application also provides an electronic device, including: At least one processor; At least one memory for storing at least one program; When at least one program is executed by at least one processor, the at least one processor implements the above-described method for processing software requirements quality assessment data.

[0096] The electronic device in the embodiments of this application may be a terminal device, a computer device, or a server device.

[0097] For example, taking a server device as an example, refer to Figure 10 The server device 1000 can vary significantly due to differences in configuration or performance. It may include one or more central processing units (CPUs) and memory 1060, and one or more storage media 1030 (e.g., one or more mass storage devices) for storing application programs 1033 or data 1032. The memory 1060 and storage media 1030 may be temporary or persistent storage. The program stored in the storage media 1030 may include one or more units or modules, each of which may include a series of operation instructions for the server device 1000. Furthermore, the CPU 1010 may be configured to communicate with the storage media 1030 and execute the series of operation instructions stored in the storage media 1030 on the server device 1000.

[0098] Server device 1000 may also include one or more power supplies 1020, one or more wired or wireless network interfaces 1040, one or more input / output interfaces 1050, and one or more operating systems 1031.

[0099] The central processing unit 1010 in server device 1000 can be used to perform, for example, Figure 2 The example shown illustrates a method for processing software requirements quality assessment data.

[0100] Similarly, the content of the above method embodiments is applicable to the embodiments of this electronic device. The specific functions implemented by the embodiments of this electronic device are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0101] This application embodiment also provides a computer-readable storage medium storing a program executable by a central processing unit 1010. When executed by the central processing unit 1010, the program executable by the central processing unit 1010 is used to perform the above-described method for processing software requirement quality assessment data.

[0102] Similarly, the content of the above method embodiments is applicable to the present computer-readable storage medium embodiments. The specific functions implemented by the present computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0103] This application also discloses a computer program product or computer program, which includes computer instructions stored in the aforementioned computer-readable storage medium. Figure 10 The processor of the illustrated electronic device can read the computer instructions from the aforementioned computer-readable storage medium, and the processor executes the computer instructions, causing the electronic device to perform... Figure 2 The example shown illustrates a method for processing software requirements quality assessment data.

[0104] Understandable Figure 2 The methods for processing software requirements quality assessment data shown in the embodiments are all applicable to this computer program product or computer program embodiment. The specific functions implemented by this computer program product or computer program embodiment are the same as those described above. Figure 2 The method for processing software requirements quality assessment data shown in the embodiment is the same, and the beneficial effects achieved are the same. Figure 2 The beneficial effects achieved by the software requirements quality assessment data processing method shown in the embodiment are also the same.

[0105] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0106] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional technology for an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0107] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0108] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0109] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0110] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0111] In the foregoing description of this specification, the references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0112] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0113] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for processing software requirements quality assessment data, characterized in that, The method includes: Obtain multiple initial requirement texts to describe the software requirements; Semantic recognition is performed on the initial requirement text, and candidate requirement texts containing predefined defect indicator words are determined from the initial requirement text based on the recognition results; For each candidate requirement text, a corresponding positive requirement text is determined, and the candidate requirement text and the positive requirement text are combined into a difference sample pair; wherein, the positive requirement text is a requirement text that corresponds semantically to the candidate requirement text and does not contain the defect indicator word; Each set of difference sample pairs is input into an arbitration system based on a large language model. The arbitration system annotates the candidate requirement text and the positive requirement text to obtain the labels corresponding to the candidate requirement text and the positive requirement text. The labels are used to characterize whether the requirement text contains defects or not. Based on the candidate requirement text, the positive requirement text, and the tags, a set of software requirement quality evaluation data is constructed.

2. The method for processing software requirements quality assessment data according to claim 1, characterized in that, The process of obtaining multiple initial requirement texts to describe software requirements includes: Monitor and capture requirement change events from at least one external requirement management system through predefined application interfaces or webhooks; Based on the requirement change event, the initial requirement text used to describe the software requirements is extracted.

3. A method for processing software requirements quality assessment data according to claim 1 or 2, characterized in that, The step of performing semantic recognition on the initial requirement text and determining candidate requirement texts containing predefined defect indicator words from the initial requirement text based on the recognition results includes: Convert the initial requirement text into a vector representation; Based on the vector representation, the candidate requirement text is identified using a natural language processing model.

4. The method for processing software requirements quality assessment data according to claim 1, characterized in that, For each candidate demand text, determining a corresponding positive demand text includes at least one of the following: If the candidate requirement text is derived from a historical requirement text, the requirement text containing clear quantitative indicators that existed in the previous version is retrieved and extracted by comparing the requirement version history as the positive requirement text. Alternatively, based on the vectorized representation of the candidate demand texts, demand texts that are semantically and syntactically similar and contain clear quantitative indicators can be retrieved from a vector database or historical demand database as the positive demand texts. Alternatively, based on the domain involved in the candidate requirement text, practice rules or failure safety principles from the domain knowledge base can be invoked to generate a requirement text that meets the requirements of being clear and verifiable, which can then be used as the positive requirement text.

5. The method for processing software requirements quality assessment data according to claim 1, characterized in that, The arbitration system includes a coordinator agent and multiple heterogeneous expert agents. The annotation of the candidate demand text and the positive demand text through the arbitration system includes: The coordinator agent distributes the difference sample pairs to each of the expert agents. Each expert agent, based on a preset domain knowledge role, independently and in parallel analyzes the demand text in the difference sample pairs to obtain preliminary original labels and judgment criteria. Based on the original tags and the judgment criteria, the candidate demand text and the tags corresponding to the positive demand text are determined.

6. The method for processing software requirements quality assessment data according to claim 5, characterized in that, The step of determining the candidate demand text and the corresponding tags for the positive demand text based on the original tags and the determination criteria includes: The coordinator agent detects whether there are conflicts in the original labels generated by each of the expert agents. If there is a conflict in the original labels, a structured debate process is initiated, in which the coordinator agent distributes the judgment criteria of each expert agent and guides the expert agents to conduct cross-evaluation and rebuttal of arguments. After at least one round of debate, the label is output based on the consensus reached or according to the preset adjudication rules.

7. The method for processing software requirements quality assessment data according to claim 1, characterized in that, After constructing the set of software requirements quality assessment data, the method further includes: The software requirements quality assessment data is anonymized. Generate and publish the version number information corresponding to the set of software requirement quality evaluation data.

8. A device for processing software requirements quality assessment data, characterized in that, The device includes: The acquisition unit is used to acquire multiple initial requirement texts that describe software requirements; The identification unit is used to perform semantic recognition on the initial requirement text and determine candidate requirement texts containing predefined defect indicator words from the initial requirement text based on the recognition result. A combination unit is configured to determine a corresponding positive demand text for each candidate demand text, and combine the candidate demand text and the positive demand text into a difference sample pair; wherein, the positive demand text is a demand text that semantically corresponds to the candidate demand text and does not contain the defect indicator word; An arbitration unit is used to input the pairs of differing samples from each group into an arbitration system based on a large language model. The arbitration system annotates the candidate requirement text and the positive requirement text to obtain labels corresponding to the candidate requirement text and the positive requirement text. The labels are used to characterize whether the requirement text contains defects or not. The construction unit is used to construct a set of software requirement quality evaluation data based on the candidate requirement text, the positive requirement text, and the tags.

9. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a method for processing software requirement quality assessment data as described in any one of claims 1-7.

10. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to implement a method for processing software requirements quality assessment data as described in any one of claims 1-7.

Images