Demand processing method and system based on natural language processing and data mining

Abstract

The present disclosure relates to a demand processing method and system based on natural language processing and data mining, which can realize automatic demand processing and analysis through natural language processing and knowledge graph structure, and improve the efficiency of demand processing and analysis. The method comprises: performing word segmentation on a business demand document through a word segmenter to obtain atomic demands; analyzing the atomic demands through a syntax layer to obtain function point triples, analyzing the atomic demands through a semantic layer to obtain demand constraint conditions, and analyzing the atomic demands through a pragmatic layer to obtain demand quantification information; taking the atomic demands as nodes, and constructing a knowledge graph structure based on at least the function point triples, the demand constraint conditions and the demand quantification information; and outputting a demand analysis result of the business demand document based on the knowledge graph structure.

Description

Technical Field

[0001] This disclosure relates to the fields of natural language processing and data mining technology, specifically to a demand processing method and system based on natural language processing and data mining. Background Technology

[0002] In today's digital age, the scale and complexity of software systems are constantly increasing, highlighting the growing importance of requirements management. During the requirements management process, manual analysis of requirements documents consumes a significant amount of time, with approximately 40% of the time spent on requirements clarification. As project scale expands and requirements become increasingly complex, the limitations of manual analysis become more apparent, not only being inefficient but also prone to oversights. Summary of the Invention

[0003] To overcome the problems existing in the related technologies, this disclosure provides a demand processing method and system based on natural language processing and data mining to solve the defects in the related technologies.

[0004] According to a first aspect of the present disclosure, a demand processing method based on natural language processing and data mining is provided, comprising:

[0005] The business requirement document is segmented by a word segmenter to obtain atomic requirements, wherein the word segmenter is adapted to the business domain corresponding to the business requirement document.

[0006] The atomic requirements are parsed through a syntactic layer to obtain functional point triples, and through a semantic layer to obtain requirement constraints. Finally, the atomic requirements are parsed through a pragmatic layer to obtain requirement quantification information. Specifically, the syntactic layer identifies key entities corresponding to the atomic requirements using a fusion model combining bidirectional long short-term memory networks and conditional random fields; the semantic layer performs semantic mapping using a domain ontology-based context disambiguation algorithm and extracts the constraint relationships in the semantically mapped business requirement document through dependency parsing; and the pragmatic layer quantifies the natural language content in the atomic requirements into numerical values ​​based on semantics.

[0007] Based on the knowledge graph structure, the requirements analysis results of the business requirements document are output.

[0008] In one embodiment, it also includes:

[0009] Based on the business requirements document, determine the stakeholder information and change history of the atomic requirements;

[0010] The step of constructing a knowledge graph structure, using the atomic requirements as nodes, based at least on the function point triples, the requirement constraints, and the requirement quantification information, includes:

[0011] Using the atomic requirements as nodes, a knowledge graph structure is constructed based on the functional point triples, the requirement constraints, the requirement quantification information, the stakeholder information, and the change history. The weight values ​​of the nodes in the knowledge graph structure are obtained based on the number of times the atomic requirements appear in the preset business document library and the stakeholder information. The relationship strength between nodes in the knowledge graph structure is obtained based on the co-occurrence information of the corresponding atomic requirements in the business requirement documents and the semantic similarity of the corresponding atomic requirements.

[0012] In one embodiment, the weight values ​​of nodes in the knowledge graph structure are obtained in the following way:

[0013] The frequency of occurrence of the atomic requirement in the preset business document library is normalized to obtain the normalized frequency of occurrence.

[0014] Based on the responsible person information marked in the business requirement document and the historical business behavior of each stakeholder corresponding to the stakeholder information, data analysis is performed to obtain the relevance of each stakeholder to the atomic requirement. Based on the preset global weight of each stakeholder to the business requirement document, the relevance of each stakeholder to the atomic requirement is weighted and summed to obtain the stakeholder influence score of the atomic requirement.

[0015] The weight value of the node corresponding to the atomic requirement is obtained by weighted summation of the normalized occurrence frequency and the stakeholder influence score based on the first preset weight.

[0016] In one embodiment, the relationship strength between nodes in the knowledge graph structure is obtained in the following way:

[0017] The co-occurrence counts of the first atomic requirement and the second atomic requirement in the same chapter of the business requirement document are normalized to obtain the normalized co-occurrence counts.

[0018] The function point triples of the first atomic requirement are vectorized to obtain a first vector, and the function point triples of the second atomic requirement are vectorized to obtain a second vector. The similarity between the first vector and the second vector is determined as the semantic similarity between the first atomic requirement and the second atomic requirement.

[0019] The normalized co-occurrence count and the semantic similarity are weighted and summed based on the second preset weight to obtain the relationship strength between the node corresponding to the first atomic requirement and the node corresponding to the second atomic requirement.

[0020] In one embodiment, the requirements analysis results are displayed as a force-directed graph and / or Sankey diagram corresponding to the knowledge graph structure, and further include:

[0021] In response to a requirement change operation on the business requirement document, the target atomic requirement and requirement change information corresponding to the requirement change operation are determined, and the scope of the change impact of the target atomic requirement is predicted by an automated impact analysis algorithm based on the change propagation tree and the requirement change information, wherein the scope of the change impact represents other atomic requirements affected by the change of the target atomic requirement;

[0022] Based on the scope of the change, the change impact flow of the target atom requirement is displayed in the force-directed graph and / or the Sankey diagram.

[0023] In one embodiment, the requirements analysis results are displayed as a force-directed graph and / or Sankey diagram corresponding to the knowledge graph structure, and further include:

[0024] In response to the version comparison operation of the business requirement document, the first requirement version and the second requirement version corresponding to the version comparison operation are determined, and the difference nodes corresponding to the atomic requirements of the difference between the first requirement version and the second requirement version are highlighted in the force-directed graph and / or the Sankey diagram.

[0025] And / or, in response to a triggering operation on a node in the force-directed graph and / or the Sankey graph, display at least one of the functional coverage, constraint satisfaction, change impact tree, and associated test case coverage of the atomic requirement corresponding to the node, wherein the change impact tree represents other atomic requirements affected by changes to the atomic requirement corresponding to the node.

[0026] In one embodiment, outputting the requirements analysis results of the business requirements document based on the knowledge graph structure includes:

[0027] The business requirement document is subjected to requirement conflict detection by the rule engine based on the preset conflict detection rules and the knowledge graph structure to obtain a first conflict detection result. The business requirement document is subjected to requirement conflict detection by the machine learning engine based on the conflict prediction model and the knowledge graph structure to obtain a second conflict detection result.

[0028] Based on the first conflict detection result and the second conflict detection result, a conflict heatmap is output, wherein the radius of the heatmap points is proportional to the conflict severity.

[0029] According to a second aspect of the present disclosure, a demand processing system based on natural language processing and data mining is provided, comprising:

[0030] The first processing module is used to perform word segmentation on the business requirement document through a word segmenter to obtain atomic requirements, wherein the word segmenter is adapted to the business domain corresponding to the business requirement document;

[0031] The parsing module is used to parse the atomic requirements through the syntactic layer to obtain function point triples, parse the atomic requirements through the semantic layer to obtain requirement constraints, and parse the atomic requirements through the pragmatic layer to obtain requirement quantification information. Specifically, the syntactic layer is used to identify the key entities corresponding to the atomic requirements by combining a bidirectional long short-term memory network and a conditional random field fusion model; the semantic layer is used to perform semantic mapping through a context disambiguation algorithm based on domain ontology and extract the constraint relationships in the semantically mapped business requirement document through dependency parsing; and the pragmatic layer is used to quantify the natural language content in the atomic requirements into numerical values ​​according to preset quantization rules.

[0032] The second processing module is used to construct a knowledge graph structure based on the atomic requirements as nodes, at least on the function point triples, the requirement constraints, and the requirement quantification information.

[0033] The output module is used to output the requirements analysis results of the business requirements document based on the knowledge graph structure.

[0034] According to a third aspect of the present disclosure, an electronic device is provided, the electronic device including a memory and a processor, the memory being configured to store computer instructions executable on the processor, and the processor being configured to implement the method described in any one of the first aspects when executing the computer instructions.

[0035] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the first aspects.

[0036] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0037] The requirement processing method provided in this disclosure can automatically process and analyze requirements through natural language processing and knowledge graph structures, reducing human intervention in the process and thus improving efficiency. Specifically, extracting atomic requirements using a domain-adaptive word segmenter better adapts to different business domains, improving the efficiency and accuracy of subsequent requirement analysis. Furthermore, the three-layer parsing structure based on syntax, semantics, and pragmatics not only better adapts to the language characteristics of specific domains and accurately identifies key entities in business requirement documents, but also accurately determines the specific meaning of synonyms by combining domain ontology and contextual information, avoiding semantic confusion. Additionally, it can quantify ambiguous requirement descriptions, facilitating subsequent requirement processing and analysis, thereby improving the efficiency and accuracy of further requirement processing and analysis. Moreover, outputting requirement analysis results based on a knowledge graph structure allows for a visual representation of the complex relationships between requirements, facilitating comprehensive understanding and analysis of requirements by different personnel. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0039] Figure 1 This is a flowchart illustrating an exemplary embodiment of the requirements processing method based on natural language processing and data mining disclosed herein;

[0040] Figure 2 This is a schematic diagram illustrating the structure of a demand processing system based on natural language processing and data mining, as shown in an exemplary embodiment of this disclosure.

[0041] Figure 3 This is a structural block diagram of an electronic device illustrated in an exemplary embodiment of the present disclosure. Detailed Implementation

[0042] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure.

[0043] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used herein are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0044] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0045] As mentioned in the background section, manual analysis of requirements documents consumes a significant amount of time in the requirements management process, with approximately 40% of the time spent on requirements clarification. As project scale increases and requirements become more complex, the limitations of manual analysis become increasingly apparent; it is not only inefficient but also prone to oversights. Furthermore, traditional regular expression matching methods cannot handle semantic ambiguity. For example, the quantifiable criteria for a description like "rapid response" are difficult to determine using traditional regular expression matching, leading to inaccurate understanding of the requirements and consequently affecting the accuracy of requirements analysis.

[0046] Based on this, in a first aspect, at least one embodiment of this disclosure provides a demand processing method based on natural language processing and data mining, please refer to the appendix. Figure 1 It illustrates the process of the method, including steps S101 to S103.

[0047] In step S101, the business requirement document is segmented by a word segmenter to obtain atomic requirements, wherein the word segmenter is adapted to the business domain corresponding to the business requirement document.

[0048] In step S102, the atomic requirements are parsed through the syntactic layer to obtain function point triples, through the semantic layer to obtain requirement constraints, and through the pragmatic layer to obtain requirement quantification information. Specifically, the syntactic layer identifies key entities corresponding to the atomic requirements using a fusion model combining bidirectional long short-term memory networks and conditional random fields; the semantic layer performs semantic mapping using a domain ontology-based context disambiguation algorithm and extracts constraint relationships from the semantically mapped business requirement document through dependency parsing; and the pragmatic layer quantifies the natural language content in the atomic requirements into numerical values ​​according to preset quantification rules.

[0049] In step S103, atomic requirements are used as nodes, and a knowledge graph structure is constructed based at least on function point triples, requirement constraints, and requirement quantification information.

[0050] In step S104, based on the knowledge graph structure, the requirements analysis results of the business requirements document are output.

[0051] To facilitate understanding, the steps described above will be further explained below.

[0052] For example, in step S101, atomic requirements are the smallest indivisible business units (such as T+0 settlement and real-time reporting), which serve as input units for semantic parsing to ensure controllable requirement granularity. Taking financial system requirement management as an example, using a domain-adaptive word segmenter to process bank business requirement documents can accurately identify technical terms, such as identifying "T+0 settlement" as an atomic requirement, providing accurate basic data for subsequent requirement analysis.

[0053] For example, in step S102, the grammar layer employs an improved BiLSTM-CRF (Bidirectional Long Short-Term Memory Network-Conditional Random Field) model to achieve domain-adaptive named entity recognition, increasing the accuracy to 92.3%. This better adapts to the language characteristics of specific domains, accurately identifying key entities in requirement documents and laying the foundation for subsequent semantic analysis. The function point triples output by the grammar layer can include subject-action-object. The BiLSTM-CRF model can be trained using sample business requirement documents and their corresponding key entities.

[0054] Continuing with the example of financial system requirements management, the business requirements document includes: "The monitoring system should report suspicious transactions in real time," with the atomic requirement being: "Real-time reporting." Using the BiLSTM-CRF model, key entities are identified as: monitoring system (the subject), reporting (the action), and suspicious transaction (the object). Therefore, the functional triple corresponding to this atomic requirement is: {monitoring system, reporting, suspicious transaction}. Thus, the semantic triple extraction technology based on domain ontology can accurately extract semantic triples from the requirements document, standardizing the content of the business requirements document into a subject-action-object structure. This provides foundational data for knowledge graph construction, facilitating subsequent requirements processing and analysis.

[0055] For example, in step S102, the semantic layer can solve the synonym mapping problem based on the context disambiguation algorithm of the domain ontology, improving accuracy by 37%. Furthermore, by introducing the domain ontology, it can accurately determine the specific meaning of synonyms by combining contextual information, avoiding semantic confusion. For instance, continuing with the example of financial system requirement management, the "real-time" in the atomic requirement "real-time report" can be mapped to the financial domain ontology "response time ≤ 200ms". The semantic layer can also extract the constraint relationships in the semantically mapped business requirement document through dependency parsing. For example, it can extract the constraint relationship between "monitoring" and "reporting" from "monitoring should report suspicious transactions in real time," which helps to deeply understand the meaning of the requirements.

[0056] For example, in step S103, the pragmatic layer can convert fuzzy descriptions into computable values ​​in the [0,1] range using a demand intensity quantification model. For instance, fuzzy demand descriptions such as "satisfy as much as possible" and "must be satisfied" can be quantified, facilitating subsequent demand processing and analysis.

[0057] Therefore, by using a three-layer parsing structure of syntax, semantics, and pragmatics, the efficiency and accuracy of requirement processing and analysis can be improved.

[0058] For example, in step S103, atomic requirements can be used as nodes, and function point triples, requirement constraints, and requirement quantification information can be used as node information to construct a knowledge graph structure.

[0059] In some embodiments, stakeholder information and change history of atomic requirements can also be determined based on business requirement documents. Accordingly, in step S103, atomic requirements can be used as nodes to construct a knowledge graph structure based on function point triples, requirement constraints, requirement quantification information, stakeholder information, and change history. The weight values ​​of nodes in the knowledge graph structure are obtained based on the number of times the atomic requirement appears in a preset business document library and the stakeholder information. The relationship strength between nodes in the knowledge graph structure is obtained based on the co-occurrence information of the corresponding atomic requirement in the business requirement document and the semantic similarity of the corresponding atomic requirement.

[0060] In other words, the embodiments of this disclosure can perform four-dimensional vector space modeling:

[0061] Functionality: Standardizes functional descriptions using the requirement element triple extractor (subject-action-object), and records parameters such as lines of code and number of interfaces.

[0062] Constraints: Logical constraints (such as response time and concurrency) are extracted based on dependency parsing and associated with the corresponding functional points.

[0063] Stakeholders: The influence of stakeholders is quantified using a weighted allocation algorithm (such as AHP), with weights ranging from [0,1].

[0064] Change history: Records metadata such as version number, change timestamp, and change type (add / modify / delete).

[0065] Therefore, by modeling requirements from four dimensions—functional points, constraints, stakeholders, and change history—relevant information about requirements can be comprehensively recorded, facilitating requirement tracking and management.

[0066] In some embodiments, the weight values ​​of nodes in the knowledge graph structure are obtained as follows: the frequency of occurrence of atomic requirements in a preset business document library is normalized to obtain normalized frequency; data analysis is performed on the historical business behaviors of each stakeholder corresponding to the responsible person information and stakeholder information marked in the business requirement document to obtain the relevance of each stakeholder to the atomic requirement; and based on the preset global weight of each stakeholder to the business requirement document, the relevance of each stakeholder to the atomic requirement is weighted and summed to obtain the stakeholder influence score of the atomic requirement; and the normalized frequency and stakeholder influence score are weighted and summed based on the first preset weight to obtain the weight value of the node corresponding to the atomic requirement.

[0067] It should be understood that the weight value of a node in a knowledge graph structure can represent the importance of the atomic requirement corresponding to the node. The higher the weight value, the more important the corresponding atomic requirement is, and thus the atomic requirement can be satisfied first.

[0068] For example, a pre-defined business document library may include historical business requirement documents, industry standard documents, etc. The frequency of an atomic requirement appearing in the pre-defined business document library reflects its business necessity; the more frequent the occurrence, the higher the business necessity, and thus the greater the importance of the corresponding atomic requirement. Stakeholder influence scores reflect the decision-making influence of stakeholders; the higher the decision-making influence, the greater the importance of the corresponding atomic requirement. Therefore, the weight value of the node corresponding to the atomic requirement can be obtained by weighted summing the normalized frequency of occurrence and the stakeholder influence score based on a first pre-defined weight.

[0069] For example, the atomic requirement "User Interface Upgrade" appears in 6 documents in the preset business document library, meaning it appears 6 times. The highest frequency of appearance among all atomic requirements is 8. Dividing this frequency by the highest frequency yields a normalized frequency of 0.75. Data analysis based on the historical requirement modification behavior of the Compliance Department, Technology Department, and Product Department corresponding to the responsible party and stakeholder information marked on the business requirement document reveals the following relevance scores for this atomic requirement: 0.1, 0.4, and 0.9, respectively. The preset global weights for the business requirement document from these departments are 0.5, 0.3, and 0.2, respectively. Therefore, the stakeholder influence score is: (0.5 × 0.1) + (0.3 × 0.4) + (0.2 × 0.9) = 0.35. Finally, the weight value of the node corresponding to this atomic requirement can be obtained as: (0.6×0.75) + (0.4×0.35) =0.59.

[0070] The first preset weights include the weights corresponding to the normalized occurrence count and the stakeholder influence score, which can be set according to the actual situation. For example, in the above example, the weights corresponding to the normalized occurrence count and the stakeholder influence score are 0.6 and 0.4, respectively. This embodiment of the present disclosure does not limit this.

[0071] In some embodiments, the relationship strength between nodes in the knowledge graph structure is obtained as follows: the co-occurrence frequency of the first atomic requirement and the second atomic requirement in the same chapter of the business requirement document is normalized to obtain the normalized co-occurrence frequency; the function point triples of the first atomic requirement are vectorized to obtain the first vector, and the function point triples of the second atomic requirement are vectorized to obtain the second vector; the similarity between the first vector and the second vector is determined as the semantic similarity between the first atomic requirement and the second atomic requirement; the normalized co-occurrence frequency and the semantic similarity are weighted and summed based on the second preset weight to obtain the relationship strength between the node corresponding to the first atomic requirement and the node corresponding to the second atomic requirement.

[0072] It should be understood that relation strength can quantify the degree of interaction between atomic requirements. The higher the relation strength, the greater the degree of interaction between the corresponding atomic requirements, which makes it easier to show the complex relationships between atomic requirements.

[0073] For example, the first atomic requirement is "real-time reporting," and the second atomic requirement is "system maintenance." First, these two atomic requirements appear simultaneously in three chapters of the business requirements document, meaning they co-occur 3 times in the same chapter. Dividing this co-occurrence count by the total number of chapters in the business requirements document (10) yields a normalized co-occurrence count of 0.3. The function point triples corresponding to the first atomic requirement are: {monitoring system, reporting, suspicious transactions}, and the function point triples corresponding to the second atomic requirement are: {maintaining system, execution, maintenance}. Vectorizing these two function point triples using the BERT model and calculating their cosine similarity yields a semantic similarity of 0.37. Finally, the relationship strength between the nodes corresponding to the first and second atomic requirements is: (0.7 × 0.3) + (0.3 × 0.37) = 0.321.

[0074] The second preset weight includes the weights corresponding to the normalized co-occurrence count and semantic similarity, which can be set according to the actual situation. For example, in the above example, the weights corresponding to the normalized co-occurrence count and semantic similarity are 0.7 and 0.3, respectively. This embodiment of the present disclosure does not limit this.

[0075] Therefore, in the process of constructing a knowledge graph, a dynamic node weight algorithm can be used to dynamically adjust the weight of nodes in the knowledge graph based on the frequency of demand occurrence and stakeholder influence scores, highlighting important demands and stakeholders, improving the expressive power of the knowledge graph, and accurately calculating the strength of relationships between demands by integrating co-occurrence analysis and semantic similarity through a relationship strength calculation model, providing richer information for demand analysis and conflict detection.

[0076] In some embodiments, the requirements analysis results are displayed as a force-directed graph and / or Sankey diagram corresponding to a knowledge graph structure. Furthermore, in response to a requirement change operation on a business requirements document, the target atomic requirement and requirement change information corresponding to the requirement change operation are determined. Using an automated impact analysis algorithm based on a change propagation tree and the requirement change information, the scope of the change impact of the target atomic requirement is predicted, where the scope of the change impact represents other atomic requirements affected by the change in the target atomic requirement. Based on the scope of the change impact, the change impact flow of the target atomic requirement is displayed in the force-directed graph and / or Sankey diagram.

[0077] It should be understood that requirement changes are inevitable in project development, but the version chaos problem, with an industry average issue rate as high as 32%, seriously affects project progress and quality. Poor version management can lead to misunderstandings of requirements by the development team, increasing development costs and risks. Template-based requirement tools in related technologies lack dynamic evolution capabilities, requiring manual backtracking when conducting change impact analysis. This makes them unable to adapt to dynamic changes in requirements, increasing labor costs and the risk of errors.

[0078] This disclosure provides a real-time impact calculation algorithm that improves the accuracy of change propagation path prediction by 68%. This algorithm enables rapid and accurate prediction of the impact of requirement changes on other parts of the system, providing strong support for change management and reducing labor costs and error risks caused by requirement changes.

[0079] For example, taking financial system requirements management as an example, when adding a requirement to "support fingerprint verification," the system can automatically label the three affected modules and two interfaces, clearly showing the scope of the requirement change's impact, facilitating targeted development and testing by team members. Thus, the automated impact analysis algorithm based on the change propagation tree can quickly and accurately analyze the scope of the requirement change's impact, providing strong support for change management. Furthermore, this embodiment can also generate a test case coverage suggestion list, providing guidance for testing work and ensuring that requirement changes are comprehensively tested and verified.

[0080] In some embodiments, the requirements analysis results are displayed as a force-directed graph and / or Sankey diagram corresponding to the knowledge graph structure. Furthermore, in response to a version comparison operation on the business requirements document, a first requirement version and a second requirement version corresponding to the version comparison operation are determined, and the difference nodes corresponding to the atomic requirements differing between the first and second requirement versions are highlighted in the force-directed graph and / or the Sankey diagram; and / or, in response to a trigger operation on a node in the force-directed graph and / or the Sankey diagram, at least one of the functional coverage, constraint satisfaction, change impact tree, and associated test case coverage of the atomic requirement corresponding to the node is displayed, wherein the change impact tree represents other atomic requirements affected by changes to the atomic requirement corresponding to the node.

[0081] It should be understood that visualization systems in related technologies often remain at the document tree level, lack semantic association mining, and cannot intuitively display the inherent relationships between requirements, which is not conducive to team members' overall understanding of the requirements.

[0082] This embodiment of the disclosure can use a combination of force-directed graphs and Sankey diagrams to visualize requirement relationships, intuitively displaying the complex relationships between requirements and facilitating a comprehensive understanding and analysis of requirements by team members. Furthermore, it supports highlighting and comparing differences between multiple versions of the requirement graph, allowing team members to clearly understand requirement changes and improving the efficiency of requirement management. Additionally, a real-time requirement completeness index calculation model can calculate the completeness of requirements in real time, enabling team members to promptly grasp the completion status of requirements. This real-time completeness index calculation model can be a pre-trained large model, and its input can be development progress information (such as lines of code) and testing information provided by team members.

[0083] For example, comparing the requirement graphs of versions v2.3 and v2.4: the new node is "Fingerprint Verification Module" (highlighted in green), and the modified node is "Face Recognition Threshold changed from 0.7 to 0.75" (with a yellow border). The Sankey diagram shows the impact flow of the changes and indicates that 75% of the test cases need to be refactored.

[0084] In some embodiments, a multi-dimensional radar chart can also be used to display requirement coverage, intuitively reflecting the satisfaction status of each requirement dimension, making it easier for team members to quickly understand the overall requirement status of the project. For example, functional coverage can be used as the X-axis, constraint satisfaction as the Y-axis, stakeholder influence score as the Z-axis, and color mapping change impact value.

[0085] In some embodiments, you can also double-click a node in the force-directed graph or Sankey diagram to expand a penetrating details panel, which displays: the change impact tree corresponding to the node (using a Collapsible Tree layout); and the coverage of the test cases associated with the node (displayed as a circular progress bar).

[0086] In some embodiments, the version timeline can also be dragged to trigger the difference comparison engine and use the LCS (Longest Common Subsequence) algorithm to highlight the changed content.

[0087] In some embodiments, in step S104, a first conflict detection result can be obtained by using a rule engine to perform requirement conflict detection on the business requirement document based on preset conflict detection rules and a knowledge graph structure, and a second conflict detection result can be obtained by using a machine learning engine to perform requirement conflict detection on the business requirement document based on a conflict prediction model and a knowledge graph structure. Based on the first and second conflict detection results, a conflict heatmap is output, wherein the radius of the heatmap points is proportional to the severity of the conflict.

[0088] It should be understood that cross-departmental collaboration is a common practice in large projects, but the discovery of requirement conflicts is delayed by an average of 14.5 days. Delayed conflict discovery leads to significant rework later in the project, increasing costs and potentially delaying delivery.

[0089] This disclosure provides a dual-engine conflict detection system. The rule engine boasts a database of over 200 industry-specific conflict rules, enabling rapid identification of common requirement conflicts, such as the mutual exclusion between "real-time synchronization" and "batch processing." The rule database is built upon industry experience and best practices, ensuring high accuracy and reliability. The machine learning engine utilizes an XGBoost conflict prediction model based on historical project data, achieving an F1 score of 0.89. This model learns conflict patterns from historical projects, accurately predicting potential requirement conflicts and improving the efficiency and accuracy of conflict detection. Therefore, this hybrid verification mechanism combining rule-based reasoning and machine learning fully leverages the accuracy of rule-based reasoning and the adaptability of machine learning, enhancing the efficiency and accuracy of requirement conflict detection. Furthermore, displaying the severity and distribution of requirement conflicts in the form of a heatmap allows for timely alerts to team members regarding potential conflicts.

[0090] For example, taking financial system demand management as an example, a dual-engine conflict detection system can identify logical contradictions between demands, such as the contradiction between "7x24-hour service" and "daily system maintenance window of 2 hours," promptly identifying and resolving potential conflicts to avoid impacting the normal operation of the system. Furthermore, it can predict potential conflicts, such as potential compatibility issues between mobile facial recognition functionality and older devices, facilitating proactive prevention and resolution. After conflict detection, the results can be visualized.

[0091] X-axis: Functional dimensions (e.g., the degree of conflict between service time and maintenance time).

[0092] Y-axis: Resource usage (e.g., CPU (Central Processing Unit) utilization conflict value).

[0093] The radius of the hot spot is directly proportional to the severity of the conflict, and a solution suggestion can be popped up by clicking: "It is recommended to adopt a segmented maintenance mechanism".

[0094] In some embodiments, mobile adaptation functionality is also provided, allowing 3D topological rotation to be implemented using Three.js: two-finger zoom control of the viewing angle (scaleFactor). [0.5,2.0]); Long press the node to display an AR (Augmented Reality) floating label (integrated ARKit coordinate mapping).

[0095] The demand processing method based on natural language processing and data mining provided in this disclosure can achieve at least one of the following beneficial effects:

[0096] 1. Improved efficiency in requirements analysis: Requirements analysis efficiency has been improved by 4-6 times, with actual measurements showing a reduction from an average of 8 person-days / 10,000 words to 1.5 person-days / 10,000 words, shortening the time required for requirements analysis and improving project development efficiency;

[0097] 2. Reduced demand omission rate: The demand omission rate was reduced to 2.1%, lower than the industry average (15%-20%), which effectively improved the quality of demand management and reduced rework and risks in the later stages of the project.

[0098] 3. Reduced time for change impact analysis: The time spent on change impact analysis has been reduced by 83%, from an average of 6.2 hours / time to 1.1 hours / time, enabling rapid and accurate assessment of the impact of requirement changes and ensuring the smooth progress of the project.

[0099] According to a second aspect of the embodiments of this disclosure, a demand processing system based on natural language processing and data mining is provided. Please refer to the appendix. Figure 2 A demand processing system 200 based on natural language processing and data mining, including:

[0100] The first processing module 201 is used to perform word segmentation on the business requirement document through a word segmenter to obtain atomic requirements, wherein the word segmenter is adapted to the business domain corresponding to the business requirement document;

[0101] The parsing module 202 is used to parse the atomic requirement through the syntactic layer to obtain the function point triplet, parse the atomic requirement through the semantic layer to obtain the requirement constraint conditions, and parse the atomic requirement through the pragmatic layer to obtain the requirement quantification information. The syntactic layer is used to identify the key entities corresponding to the atomic requirement by combining a fusion model of bidirectional long short-term memory network and conditional random field. The semantic layer is used to perform semantic mapping by a context disambiguation algorithm based on domain ontology and extract the constraint relationship in the business requirement document after semantic mapping by dependency parsing. The pragmatic layer is used to quantify the natural language content in the atomic requirement into numerical values ​​according to preset quantization rules.

[0102] The second processing module 203 is used to construct a knowledge graph structure based on the atomic requirements as nodes, at least on the function point triplet, the requirement constraints and the requirement quantification information.

[0103] Output module 204 is used to output the requirements analysis results of the business requirements document based on the knowledge graph structure.

[0104] In some embodiments of this disclosure, the requirements processing system 200 based on natural language processing and data mining further includes a determination module, used to determine the stakeholder information and change history of the atomic requirements based on the business requirements document;

[0105] The second processing module 203 is further used for:

[0106] Using the atomic requirements as nodes, a knowledge graph structure is constructed based on the functional point triples, the requirement constraints, the requirement quantification information, the stakeholder information, and the change history. The weight values ​​of the nodes in the knowledge graph structure are obtained based on the number of times the atomic requirements appear in the preset business document library and the stakeholder information. The relationship strength between nodes in the knowledge graph structure is obtained based on the co-occurrence information of the corresponding atomic requirements in the business requirement documents and the semantic similarity of the corresponding atomic requirements.

[0107] In some embodiments of this disclosure, the weight values ​​of nodes in the knowledge graph structure are obtained through a third processing module, which is used to:

[0108] The frequency of occurrence of the atomic requirement in the preset business document library is normalized to obtain the normalized frequency of occurrence.

[0109] Based on the responsible person information marked in the business requirement document and the historical business behavior of each stakeholder corresponding to the stakeholder information, data analysis is performed to obtain the relevance of each stakeholder to the atomic requirement. Based on the preset global weight of each stakeholder to the business requirement document, the relevance of each stakeholder to the atomic requirement is weighted and summed to obtain the stakeholder influence score of the atomic requirement.

[0110] The weight value of the node corresponding to the atomic requirement is obtained by weighted summation of the normalized occurrence frequency and the stakeholder influence score based on the first preset weight.

[0111] In some embodiments of this disclosure, the relationship strength between nodes in the knowledge graph structure is obtained through a fourth processing module, which is used to:

[0112] The co-occurrence counts of the first atomic requirement and the second atomic requirement in the same chapter of the business requirement document are normalized to obtain the normalized co-occurrence counts.

[0113] The function point triples of the first atomic requirement are vectorized to obtain a first vector, and the function point triples of the second atomic requirement are vectorized to obtain a second vector. The similarity between the first vector and the second vector is determined as the semantic similarity between the first atomic requirement and the second atomic requirement.

[0114] The normalized co-occurrence count and the semantic similarity are weighted and summed based on the second preset weight to obtain the relationship strength between the node corresponding to the first atomic requirement and the node corresponding to the second atomic requirement.

[0115] In some embodiments of this disclosure, the requirements analysis results are displayed as force-directed graphs and / or Sankey diagrams corresponding to the knowledge graph structure. The requirements processing system 200 based on natural language processing and data mining further includes a first display module, used for:

[0116] In response to a requirement change operation on the business requirement document, the target atomic requirement and requirement change information corresponding to the requirement change operation are determined, and the scope of the change impact of the target atomic requirement is predicted by an automated impact analysis algorithm based on the change propagation tree and the requirement change information, wherein the scope of the change impact represents other atomic requirements affected by the change of the target atomic requirement;

[0117] Based on the scope of the change, the change impact flow of the target atom requirement is displayed in the force-directed graph and / or the Sankey diagram.

[0118] In some embodiments of this disclosure, the requirements analysis results are displayed as force-directed graphs and / or Sankey diagrams corresponding to the knowledge graph structure. The requirements processing system 200 based on natural language processing and data mining further includes a second display module, used for:

[0119] In response to the version comparison operation of the business requirement document, the first requirement version and the second requirement version corresponding to the version comparison operation are determined, and the difference nodes corresponding to the atomic requirements of the difference between the first requirement version and the second requirement version are highlighted in the force-directed graph and / or the Sankey diagram.

[0120] And / or, in response to a triggering operation on a node in the force-directed graph and / or the Sankey graph, display at least one of the functional coverage, constraint satisfaction, change impact tree, and associated test case coverage of the atomic requirement corresponding to the node, wherein the change impact tree represents other atomic requirements affected by changes to the atomic requirement corresponding to the node.

[0121] In some embodiments of this disclosure, the output module 204 is further configured to:

[0122] The business requirement document is subjected to requirement conflict detection by the rule engine based on the preset conflict detection rules and the knowledge graph structure to obtain a first conflict detection result. The business requirement document is subjected to requirement conflict detection by the machine learning engine based on the conflict prediction model and the knowledge graph structure to obtain a second conflict detection result.

[0123] Based on the first conflict detection result and the second conflict detection result, a conflict heatmap is output, wherein the radius of the heatmap points is proportional to the conflict severity.

[0124] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments of the method in the first aspect, and will not be elaborated upon here.

[0125] According to a third aspect of the embodiments of this disclosure, please refer to the appendix. Figure 3 The diagram illustrates an exemplary block diagram of an electronic device 700, which may include a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0126] The processor 701 controls the overall operation of the electronic device 700 to complete all or part of the steps in any of the methods described above. The memory 702 stores various types of data to support the operation of the electronic device 700. This data may include, for example, instructions for any application or method operating on the electronic device 700, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 703 may include a screen and audio components. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 702 or transmitted via communication component 705. The audio component also includes at least one speaker for outputting audio signals. I / O interface 704 provides an interface between processor 701 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination thereof; therefore, the corresponding communication component 705 may include a Wi-Fi module, a Bluetooth module, or an NFC module.

[0127] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the aforementioned demand processing method based on natural language processing and data mining.

[0128] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of any of the methods described above. For example, the computer-readable storage medium may be the memory 702 including the program instructions described above, which may be executed by the processor 701 of the electronic device 700 to perform any of the methods described above.

[0129] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a processor, wherein the computer program, when executed by the processor, implements the steps of any of the methods described above.

[0130] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0131] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0132] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A demand processing method based on natural language processing and data mining, characterized in that, The method comprises the following steps: performing word segmentation on a business requirement document by a word segmenter to obtain atomic requirements, wherein the word segmenter is adapted to a business field corresponding to the business requirement document; performing parsing on the atomic requirements by a syntax layer to obtain function point triples, performing parsing on the atomic requirements by a semantic layer to obtain requirement constraint conditions, and performing parsing on the atomic requirements by a pragmatic layer to obtain requirement quantification information, wherein the syntax layer is configured to identify key entities corresponding to the atomic requirements by a fusion model combining a bidirectional long short-term memory network and a conditional random field, the semantic layer is configured to perform semantic mapping by a context disambiguation algorithm based on a domain ontology and extract constraint relationships in the business requirement document after semantic mapping by a dependency syntax analysis, and the pragmatic layer is configured to quantify natural language content in the atomic requirements into numerical values according to semantics; constructing a knowledge graph structure based on at least the function point triples, the requirement constraint conditions and the requirement quantification information, with the atomic requirements as nodes; outputting a requirement analysis result of the business requirement document based on the knowledge graph structure; The method further comprises the following steps: determining stakeholder information and change history of the atomic requirements based on the business requirement document; constructing the knowledge graph structure based on the function point triples, the requirement constraint conditions, the requirement quantification information, the stakeholder information and the change history, with the atomic requirements as nodes, wherein a weight value of a node in the knowledge graph structure is obtained based on the number of occurrences of the atomic requirement in a preset business document library and the stakeholder information, and a relationship strength between nodes in the knowledge graph structure is obtained based on co-occurrence information of corresponding atomic requirements in the business requirement document and semantic similarity of the corresponding atomic requirements. The weight value of the node in the knowledge graph structure is obtained in the following manner: 2.The demand processing method based on natural language processing and data mining of claim 1, wherein, normalizing the number of occurrences of the atomic requirement in the preset business document library to obtain a normalized occurrence number; performing data analysis on historical business behaviors of each stakeholder corresponding to the responsibility information and the stakeholder information annotated based on the business requirement document to obtain an association degree of the atomic requirement for each stakeholder, and performing weighted summation on the association degree of the atomic requirement for each stakeholder based on a preset global weight of each stakeholder for the business requirement document to obtain a stakeholder influence score of the atomic requirement; performing weighted summation on the normalized occurrence number and the stakeholder influence score based on a first preset weight to obtain a weight value of a node corresponding to the atomic requirement. The relationship strength between nodes in the knowledge graph structure is obtained in the following manner: 3.The demand processing method based on natural language processing and data mining of claim 1, wherein, normalizing the co-occurrence number of a first atomic requirement and a second atomic requirement in the same chapter in the business requirement document to obtain a normalized co-occurrence number; ​ vectorize the function point triplets of the first atomic requirement to obtain a first vector, vectorize the function point triplets of the second atomic requirement to obtain a second vector, and determine a similarity between the first vector and the second vector as a semantic similarity between the first atomic requirement and the second atomic requirement; weight and sum the normalized co-occurrence number and the semantic similarity based on a second preset weight to obtain a relationship strength between the node corresponding to the first atomic requirement and the node corresponding to the second atomic requirement.

4. The demand processing method based on natural language processing and data mining according to any one of claims 1-3, characterized in that, The requirement analysis result is displayed as a force-directed graph and / or a Sankey diagram corresponding to the knowledge graph structure, and further includes: In response to a requirement change operation on the business requirement document, target atomic requirements and requirement change information corresponding to the requirement change operation are determined, and a change influence range of the target atomic requirements is predicted through an automatic influence analysis algorithm based on a change propagation tree and the requirement change information, where the change influence range represents other atomic requirements affected by the change of the target atomic requirements; Based on the change influence range, a change influence flow of the target atomic requirements is displayed in the force-directed graph and / or the Sankey diagram.

5. The demand processing method based on natural language processing and data mining according to any one of claims 1-3, characterized in that, The requirement analysis result is displayed as a force-directed graph and / or a Sankey diagram corresponding to the knowledge graph structure, and further includes: In response to a version comparison operation on the business requirement document, a first requirement version and a second requirement version corresponding to the version comparison operation are determined, and difference nodes corresponding to difference atomic requirements between the first requirement version and the second requirement version are highlighted in the force-directed graph and / or the Sankey diagram; And / or, in response to a trigger operation on a node in the force-directed graph and / or the Sankey diagram, at least one of a function coverage, a constraint satisfaction degree, a change influence tree, and an associated test case coverage of an atomic requirement corresponding to the node is displayed, where the change influence tree represents other atomic requirements affected by the change of the atomic requirement corresponding to the node.

6. The demand processing method based on natural language processing and data mining according to any one of claims 1-3, characterized in that, The requirement analysis result of the business requirement document is output based on the knowledge graph structure, including: A first conflict detection result is obtained by performing requirement conflict detection on the business requirement document based on a preset conflict detection rule and the knowledge graph structure through a rule engine, and a second conflict detection result is obtained by performing requirement conflict detection on the business requirement document based on a conflict prediction model and the knowledge graph structure through a machine learning engine; A conflict heat map is output based on the first conflict detection result and the second conflict detection result, where the radius of a heat point in the conflict heat map is proportional to the conflict severity.

7. A requirement processing system based on natural language processing and data mining, characterized by, It includes: The first processing module is configured to perform word segmentation processing on the business requirement document through a word segmenter to obtain atomic requirements, where the word segmenter is adapted to a business field corresponding to the business requirement document. The parsing module is configured to parse the atomic requirements through a syntax layer to obtain function point triplets, through a semantic layer to obtain requirement constraint conditions, and through a pragmatic layer to obtain requirement quantification information. The syntax layer is configured to identify key entities corresponding to the atomic requirements by using a fusion model combining a bidirectional long short-term memory network and a conditional random field. The semantic layer is configured to perform semantic mapping by using a context disambiguation algorithm based on a domain ontology and extract constraint relationships in the business requirement document after semantic mapping by using dependency syntax analysis. The pragmatic layer is configured to quantize natural language content in the atomic requirements into numerical values according to a preset quantization rule. The determining module is configured to determine stakeholder information and change history of the atomic requirements based on the business requirement document. The second processing module is configured to construct a knowledge graph structure based on the function point triplets, the requirement constraint conditions, the requirement quantification information, the stakeholder information, and the change history, with the atomic requirements as nodes. A weight value of a node in the knowledge graph structure is obtained based on the number of occurrences of the atomic requirement in a preset business document library and the stakeholder information. A relationship strength between nodes in the knowledge graph structure is obtained based on co-occurrence information of corresponding atomic requirements in the business requirement document and semantic similarity of the corresponding atomic requirements. The output module is configured to output a requirement analysis result of the business requirement document according to the knowledge graph structure.

8. An electronic device, comprising: The electronic device includes a memory and a processor. The memory is configured to store computer instructions executable on the processor. The processor is configured to implement the steps of the method in any one of claims 1-6 when executing the computer instructions.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by the processor, implements the steps of the method in any one of claims 1-6.

Images