Multi-source document mapping method and system based on software development field

Abstract

The invention discloses a multi-source document mapping method and system based on the field of software development, and the method comprises the steps: 1) converting project documents of each stage of a target project into a unified format, and obtaining a multi-source document set; according to the structured data structure of each document in the multi-source document set, storing the corresponding document as an XML tree; 2) for each document information node in the XML tree, identifying a function point node to which the document information node belongs; extracting the content of each function point node and splicing the content with the context of the function point node; correlation is formed between each internal node of the function point nodes and other function point nodes except the area where the function point node is located, and candidate function point nodes of the internal nodes are determined according to correlation scores; and 3) establishing a mapping matrix according to a matching result of the function point node corresponding to each document and the function point nodes corresponding to other documents, and obtaining an optimal mapping sequence of each document according to the mapping matrix to form a mapping table.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence for software project management, and in particular relates to a multi-source document mapping method and system based on the field of software development. Background Art

[0002] With the development of network and information technology, the number of software equipment is increasing, and the task of software R&D acceptance is becoming heavier. At present, there are many software development project management software, but they are basically process node management and work decomposition management. Due to the physical isolation between the demander and the developer, although Party A or Party B has different project management software to monitor the process, the results of the developer are generally delivered to the demander in the form of documents. Based on the mutual verification of documents between multi-source documents, especially during the final acceptance, a large amount of manual means are required to compare, track and verify each type of material submitted during the software development process. The lack of support from automated methods and means leads to a variety of problems such as heavy management work, low efficiency, poor quality, and easy omissions, which seriously affects the quality and efficiency of software acceptance. It is urgent to introduce advanced technical means such as big data and artificial intelligence in the process of software project management acceptance, and to assist in matching each functional point and acceptance standard of software project demonstration, feasibility analysis, bidding function requirements, technical design books, and demand function specifications through a combination of technology and manual methods. Summary of the invention

[0003] The present invention provides a multi-source document mapping method and system based on the field of software development, which makes up for the shortcomings of the traditional method of relying on manual keyword retrieval, such as inconsistent terms adopted by different documents and inability to match, typos or synonyms of words and inability to match, etc. The present invention uses a variety of technical means and special data processing techniques, combined with different standard formats and writing characteristics of various process documents in the software development process, to map similar functional points within and between documents, meet the requirements of locating one place and automatically associating and expanding multiple places, and form a functional mapping statistical list, so as to achieve project acceptance or other needs when alignment of various documents is required, conduct a comprehensive evaluation of the functional point relationship between the documents, and verify the completion of the function and the integrity of the document.

[0004] The technical solution adopted by the present invention is as follows:

[0005] A multi-source document mapping method based on the field of software development, the steps of which include:

[0006] 1) Converting the project documents of each stage of the target project into a unified format to obtain a multi-source document set; based on the structured data structure of each document in the multi-source document set, storing the corresponding document as an XML tree, wherein the XML tree includes two parts: a document information node and a document content node, wherein the document information node includes the file name, creation date, author, and original document file path of the document, and the document content node is the starting root node of the XML tree;

[0007] 2) For each document information node in the XML tree, identify the function point node to which the document information node belongs; extract the content of each function point node and concatenate it with the context of the function point node as the context of the function point node; associate each internal node of the function point node with other function point nodes outside the region where the function point node is located, calculate the association score, and determine the candidate function point node of the internal node according to the association score;

[0008] 3) If function point nodes with the same name appear in multiple places in the XML tree, the contexts of the function point nodes are merged; a mapping matrix is ​​established based on the matching results between the function point nodes corresponding to each document and the function point nodes corresponding to other documents, and the optimal mapping sequence for each document is obtained based on the mapping matrix to form a mapping table.

[0009] Furthermore, for each document, the function point nodes in the document are listed and described, and the same function points in various areas of the document are linked to each other.

[0010] Furthermore, the method for establishing the mapping matrix is:

[0011] 31) The XML tree corresponding to the document A is recorded as tree A; for each function point node d in tree A, the function outline tree corresponding to the function point node d is matched with the outline trees in the XML trees corresponding to other documents, and the candidate set of mapping nodes for the function point node d is determined according to the matching results;

[0012] 32) Calculate the name similarity S1 between the function point node d and its child nodes and each node in the mapping node candidate set;

[0013] 33) According to the superior-subordinate relationship of the function point node d in the tree A, obtain the child nodes of the function point node d; according to the mapping node candidate set, obtain the child nodes of each child node d of the function point node d. jMatch the target child node, obtain all the parent nodes of the target child node; take the name similarity between the nodes as the basic weight, multiply it by the hierarchical coefficient function of the function point node d, and add it upward to the parent node to determine the optimal mapping node of tree B corresponding to the first-level node of tree A; calculate the score of each optimal mapping node of the function point node d and record it as S2; tree B is the XML tree corresponding to document B;

[0014] 34) Calculate the semantic similarity between the function point node d and each optimal mapping node, and select the node with the highest semantic similarity as the mapping node of the function point node d; the semantic similarity score between the function point node d and its mapping node is recorded as S3;

[0015] 35) Calculate the total score S of the function point node d according to S1, S2, and S3 as the mapping score between the function point node d and the corresponding mapping node;

[0016] 36) Establishing the mapping matrix according to the mapping nodes and mapping scores corresponding to each function point node.

[0017] Furthermore, according to the mapping table and the context of each function point node, the description and definition of the corresponding function point in each stage of the target project development are described.

[0018] Furthermore, the method for obtaining the multi-source document collection is: pre-processing the project document format and uniformly converting it into OpenXml format and parsing it to obtain the hierarchical outline, table content, text content, and illustrations of each project document, forming a multi-source document collection that reflects the project demand process at each stage of the project process.

[0019] Furthermore, based on the title feature, sequence number feature, and all parent node titles corresponding to the document information node, the function point node to which the document information node belongs is comprehensively identified.

[0020] Furthermore, a total analysis is performed on the mapping table to comprehensively describe the degree of completion of the functions involved; the total analysis includes function point scale, time coverage, function point coverage rate, test rate, project establishment rate, design rate, and change rate.

[0021] A multi-source document mapping system based on the field of software development, characterized by comprising a document preprocessing module, a correlation analysis module, and a mapping module;

[0022] The document preprocessing module is used to convert the project documents of each stage of the target project into a unified format to obtain a multi-source document set; according to the structured data structure of each document in the multi-source document set, the corresponding document is stored as an XML tree, the XML tree includes two parts: a document information node and a document content node, the document information node includes the file name, creation date, author, and original document file path of the document, and the document content node is the starting root node of the XML tree;

[0023] The association analysis module is used to identify, for each document information node in the XML tree, the function point node to which the document information node belongs; extract the content of each function point node and splice it with the context of the function point node as the context of the function point node; associate each internal node of the function point node with other function point nodes outside the region where the function point node is located, calculate the association score, and determine the candidate function point node of the internal node according to the association score;

[0024] The mapping module is used to merge the contexts of function point nodes with the same name appearing in multiple places in the XML tree; calculate the matching results of the function point nodes corresponding to each document and the function point nodes corresponding to other documents, establish a mapping matrix according to the matching results, and obtain the optimal mapping sequence of each document according to the mapping matrix to form a mapping table.

[0025] A server, characterized in that it comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing each step in the above method.

[0026] A computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the above method when executed by a processor.

[0027] In a first aspect, the present invention provides a method for multi-source document mapping in the field of software development, which realizes the analysis of various types of documents in the software project development process but not limited to the software development field, and extracts the functional requirements involved in each type of document, forms a series connection for different sources of the functional requirements in a single document, and also forms a series connection between various documents. After the entire project document is connected in series, a comprehensive description of the historical evolution, functional description, functional definition, functional parameters, functional acceptance criteria, etc. of all functions involved in the project is finally formed. The method comprises the following steps:

[0028] S1: Project document analysis, collect project documents, pre-process the project document format and convert it into OpenXml format. First, read the document stream through the document reading component, and then parse the hierarchical outline, table content, text content, illustrations, etc. of the project document through the document parsing engine, including the project report parsing engine, feasibility study report parsing engine, requirement specification parsing engine, etc., to form a multi-source document that reflects the project requirement process at each stage of the project process, complete the OpenXml stream parsing and call the structured analysis engine to convert it into a structured data structure of the document and store it in xml. Each document is represented by an XML tree, and each tree contains two parts: document information node and document content node. The document information node includes the file name, creation date, author, and original document file path of the document. The document content node serves as the starting root node for storing the parsed document.

[0029] S2: Single document function point mapping engine, further identifies the document content nodes of the document XML tree, completes the processing of the document information nodes of each document, and comprehensively identifies which level of nodes belong to the function point node based on the title feature, sequence number feature, all parent node titles, etc., and extracts the content of its function point node, splices the context of the corresponding node, including the parent directory, the description inside the node area, such as the table involved, the multi-level header of the Y axis, the multi-level header of the X axis, etc. As the context of the function point node, the internal node forms an association with other areas outside the area, calculates the association score, and uses other function point nodes as candidate links, so that the function point nodes of each area are connected in series with the same function point nodes in other places.

[0030] S3: Multi-document function point mapping engine, first of all, for each document in S2, if the same node appears in multiple places in different areas, merge the context of the corresponding function point node, such as the function performance parameter table that appears in the text or attachment; secondly, match the function point node list of each document with the function point nodes in other documents to complete the mapping matrix, and obtain the optimal mapping sequence of each document in other documents to form a mapping table.

[0031] S4: Conduct a total analysis of the mapping table, including function point scale, time coverage, function point coverage rate, test rate, project establishment rate, design rate, change rate, etc., and provide a comprehensive description of the degree of completion of the functions involved.

[0032] S5: For each document, the function point nodes in the document are listed and described, and the same function points in each area of ​​the document are linked to each other, jump connections are set in the document, and can be output to other system platforms.

[0033] S6: Based on the mapping table and the context of each function point node, describe the description and definition of each function point in each stage of project development. And can output the project management platform to track the coverage of each function point in each stage, whether the project establishment, feasibility study, design, development, testing, training, etc. of each function point in the project have been completed. Different document types and formats correspond to their professional document parsing engines. A comprehensive document type parsing engine pre-parses what type of document, and then calls its dedicated engine for parsing.

[0034] Furthermore, the step S1 comprises:

[0035] (1) Reading of project establishment report, including preprocessing of project establishment report, converting it into standard OpenXML format, and then forming a document flow. Secondly, the outline and content of the project establishment report are parsed through the project establishment report parsing engine, including the purpose, background, and premise of feasibility study. The functional requirements involved, such as "functional requirements", will be dynamically identified in different report formats or reference standards, without relying on fixed names or chapters. The recognition algorithm can adopt rule mode, regular mode, sequence mode, contextual semantic artificial intelligence classification model, etc.

[0036] (2) Reading of project feasibility reports, including preprocessing of project establishment reports, converting them into the standard OpenXML format, and then forming a document stream. Secondly, the feasibility report parsing engine is used to parse the outline and content of the feasibility report, including writing the project overview, project name, construction objectives, main construction contents, investment estimates, etc. The functional requirements involved, such as "basic functions", will be dynamically identified in different formats or reference standards of the feasibility report, without relying on fixed names or chapters. The recognition algorithm can adopt rule patterns, regular patterns, sequence patterns, contextual semantic artificial intelligence classification models, etc.

[0037] (3) Reading of requirement specifications, including preprocessing of the requirement specifications, converting them into the standard OpenXML format, and then forming a document stream. Secondly, the requirement specification parsing engine is used to parse the outline and content of the requirement specifications, including the introduction, project overview, system functional design, non-functional requirements, acceptance criteria, attachments, etc. The functional requirements involved, such as "system functional design", will be dynamically identified in different formats or reference standards of the actual document, without relying on fixed names or chapters. The recognition algorithm can adopt rule patterns, regular patterns, sequence patterns, contextual semantic artificial intelligence classification models, etc.

[0038] (4) Reading of the outline design document, including preprocessing of the outline design document, converting it into the standard OpenXML format, and then forming a document stream. Secondly, the outline design document parsing engine is used to parse the outline and content of the outline design document, including the introduction, design summary, system logic design, system error handling design, technical design, database design, vocabulary, attachments, etc. The functional requirements involved, such as "technical design", will be dynamically identified in different formats or reference standards of the actual document report, without relying on fixed names or chapters. The recognition algorithm can adopt rule mode, regular mode, sequence mode, contextual semantic artificial intelligence classification model, etc.

[0039] (5) Reading of detailed design documents, including preprocessing of detailed design documents, converting them into standard OpenXML format, and then forming a document stream. Secondly, the detailed design document parsing engine is used to parse the outline and content of the detailed design document, including introduction, project overview, program design description, functions, performance, input, output, algorithms, attachments, etc. The functional requirements involved, such as "program design description. Function", will be dynamically identified in different formats or reference standards of the actual document report, without relying on fixed names or chapters. The recognition algorithm can adopt rule mode, regular mode, sequence mode, contextual semantic artificial intelligence classification model, etc.

[0040] (6) Functional specification reading, including preprocessing of the functional specification, converting it into the standard OpenXML format, and then forming a document stream. Secondly, the functional specification parsing engine parses the outline and content of the functional specification, including the introduction, project overview, operating environment, system functional description, subsystem X functional description, attachments, etc. The functional requirements involved, such as "subsystem X functional description", will be dynamically identified in different formats or reference standards of the actual document, without relying on fixed names or chapters. The recognition algorithm can adopt rule mode, regular mode, sequence mode, contextual semantic artificial intelligence classification model, etc.

[0041] Furthermore, the step S2 comprises:

[0042] (1) Outline navigation analysis. Based on the outline level numbers of the document, including the levels of humanities manuscripts and scientific and technological manuscripts, as well as the mixed compilation of the outlines of the two types of manuscripts. The first method is to use the hierarchical structure recorded in Outline, restore its outline serial numbers, and form a serial number outline. If there is marked content of OutLine in the original OpenXml document stream stored in the document, the LevelNumbering of the document will be recognized and converted into standard outline text, such as (1), (2), etc., and output according to the hierarchical relationship of the outline. The second method is that if there is no OutLine content, the hierarchical relationship of the outline in the paragraph will be recognized using the rules of Chinese outline writing. It is necessary to intercept the serial number part according to the serial number of each paragraph of the document and exclude the mixed content that conforms to the serial number format but is not an actual serial number, that is, the outline recognition algorithm, including sequence consistency, sequence continuity, etc. Using the hierarchical characteristics of the outlines of humanities manuscripts and scientific and technological manuscripts, corrections are made to form an outline paragraph with an upper and lower hierarchical structure. Combining the above two methods, the outlines in the original document are uniformly output as automatically corrected and patched ordered outline subtree nodes. And each outline node is uniformly re-coded, including a unique sequence code and a unique hierarchical code. Among them, the sequence code is a sequence code starting from 1, such as 1, 2, 3, 4, 5, etc., and the hierarchical code is 1.1, 1.2, 1.2.1, 1.2.2, 2.1, 2.2, etc., which is recursively pushed according to the hierarchy to form a surrogate key to avoid discontinuity, non-uniqueness, and non-standardization of the outlines in the original document, which are not conducive to subsequent algorithm calculations. For example, if the actual outline sequence in the original document is typed as "3.2, 3.3, 3.4...", the sequence code will be re-coded as "1, 2, 3...", and the hierarchical code will be re-coded as "1.1, 1.2, 1.3...".

[0043] (2) Serial number recognition. First, it includes using "1." for the first layer, "(1)" for the second layer, "1." for the third layer, "(1)" for the fourth layer, or 1.1, 1.1.1, a), b), etc. The common writing order of capital numbers, full parentheses, right parentheses, dots, etc. is followed between various levels, and it does not strictly depend on a fixed writing method. It is possible to ignore one or more intermediate levels. For example, the lower level of "1." can directly be "1)" or "1". The algorithm determines the possible serial number level layout that may be adopted globally based on the serial number level and the characteristics of the first serial number appearance. And using this layout as a constraint, the possible outline serial numbers to be pre-recognized are adjusted, and the suspected serial numbers that do not conform to the characteristics are excluded. Secondly, the non-continuity of the serial numbers is recognized in a compatible manner. Sequences that meet the threshold of credibility, such as "(1), (2), (4), (5), (6)", can be judged that there is a "(X)" outline level.

[0044] (3) Identification of function points in the outline. In the function point node tree, the nodes that are identified as outlines are called outline nodes. For each outline node, the title of the outline node + the title or content of the descendant nodes below the outline node form a new paragraph and input it into the sentence classification model. The model can use the Transformer classification model or a similar text deep learning classification model to determine whether the outline node is a function point node and output the score of the node as a function point node. Finally, according to all the outlines, for the same level sequence, if the score sequence exceeds the threshold, it is identified as a function point sequence, such as 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.4.1, 2.4.2, 2.4.3, 2.5.1, 2.5.2, 2.5.3. The model can use a deep learning model and continuously iterate training. When the expression of different document content is met, it has a good recognition effect. The model also has good performance in contract outlines, regulatory outlines, paper outlines, etc.

[0045] (4) Table extraction: Identify the header part of the table, including the content on the left side of the X-axis and the content on the top level of the Y-axis. Software function descriptions are mostly in the form of tables in various documents. They are read by merging cells to distinguish the hierarchical relationships. If a dimension of the X-axis or Y-axis in the table is identified as a software function requirement, the table is structured, the function points in it are flattened, and the other dimensional parts, along the X-axis direction or the Y-axis direction, are used as the description or feature part of the function point.

[0046] (5) Dimension identification, also known as title identification, uses the nouns and terms that appear at other outline levels in the text, as well as features that are strongly related to functional points such as "functional requirements", "functional requirements", and "functional descriptions" to identify possible dimensional directions in the table.

[0047] (6) Identify function points in the table. Extract the corresponding cell content along the axis corresponding to the title of the function point definition to form a sequence, and verify the sequence with the title content of other outline levels. If it exceeds a certain threshold, such as a coverage rate of more than 80%, it is considered a successful function description table, and the corresponding sequence is also the function point {Function1, Function2, Function3}. Use the following steps to match the function point nodes in the table and their extended information with the function point nodes appearing in the document body.

[0048] Furthermore, the above steps also include intra-document matching:

[0049] (1) According to the above steps, the outline of the document and the nodes marked as function points and the function points that may be included in the table can be obtained. For all the function points in the document, title matching and description semantic matching are adopted, where the description is the content under the title or the collection of sub-level points under the title and their content, or in the table, it is the horizontal or vertical direction adjacent to the function point. The title and description are semantically converted, including SBert semantic vector conversion or Word2Vector semantic vector conversion, into representation vectors that can be used for distance calculation.

[0050] (2) Summarize the frequency of the title appearing in the outline. If it occurs more than once, the title = parent title + current title. If it is still repeated, the title = parent title + parent title + current title until there is no ambiguity. Segment the title, remove single words and stop words, and only keep verbs and nouns. Use the Jaccard calculation formula to calculate the TFIDF weighted distance value, and fuse the semantic distance score and the Jaccard distance part. D(m,n) = p1*V1(m,n)+(1-p1)V2(m,n), where V1 represents the semantic distance and V2 represents the Jaccard distance. p1 is the fusion weight of the semantic distance, 1-p1 is the fusion weight of the Jaccard distance of the words in the two paragraphs, m represents the source function point node, and n represents the candidate target function point node. Function points with D(m,n) exceeding 0.8 are set as semantically similar function points.

[0051] Furthermore, the step S3 comprises:

[0052] (1) Tree node matching: traverse each tree of multiple document nodes, match the function point outline tree in the tree with the function point outline tree of other documents. The matching algorithms include name matching, subtree matching, and description semantic matching. After weighting multiple matching algorithms, the top 10 nodes with the highest matching degree are used as the candidate set of mapping nodes linked to the node in another document according to the comprehensive weight. The function point outline tree is a subset of the outline tree. For the content identified as function point nodes, its parent-child structure relationship is retained. The function point outline tree extracted from document A is recorded as tree A, and the function point outline tree extracted from document B is recorded as tree B.

[0053] (2) Name matching: For each candidate matching pair, the name similarity is compared, including the TF-IDF Weight + Jaccard weighting algorithm, and the formula is: Where W i is each word in the intersection of A and B i TF-IDF value of j is each word in the intersection of A and B jThe TF-IDF value of each tree node is compared with each node of the tree in other documents. The score of each node is recorded as S1.

[0054] (3) Subtree matching: According to the node hierarchy, the child node is matched with the target child node, and all parent nodes of the target child node are taken as a set statistics. The weight of each level takes the name match as the basic weight, which is multiplied by the level coefficient function of the current tree node. The level coefficient function can be a logarithm or other mathematical formula, and is accumulated upward to the parent node. Finally, the mapping node of tree B corresponding to the first-level node of tree A is determined. Tree A and tree B use the subtree matching method to calculate the weight, so as to obtain the optimal mapping node between any node of tree A and tree B. The score of each node of the mapped tree B is the sum of all scores of all child mapping bubbles of tree A, such as Node A1 The candidate mapping set is {Node B2.1(w1) ,Node B2.1.1(w2) ,Node B4.2(w3) ,Node B2.3(w4) ,Node B2.3.1(w5) ,Node B2.4(w6),…}, get Node B2 The matching score of Node is w1+s2+w4+s5+w6. B4＝w3 ,Node B2.3＝w4+w5 ,Node B2.1(w1) ,Node B2.1.1(w2) ,Node B4.2(w3) ,Node B2.3(w4) ,Node B2.3.1(w5) ,Node B2.4(w6) , rearrange in reverse order, and cut off the data below the threshold to reduce the amount of calculation. Each group of mappings retains the best 10 candidate pairs, and their score set is recorded as S2.

[0055] (4) Describe semantic matching. Read the source description and the mapped target description of the mapping set generated by the above steps, convert them into vectors using a semantic model, and calculate the distance between the two vectors using cosine or other distance calculation methods. And convert this distance into a standard score. The calculation formula is S3(m,n)=p1*V1(m,n)+(1-p1)V2(m,n), where V1 represents the semantic distance and V2 represents the Jaccard distance. Among them, p1 is the fusion weight of the semantic distance, 1-p1 is the fusion weight of the Jaccard distance of the words in the two paragraphs, m represents the source function point node, and n represents the function point node of the candidate target. And integrate it into the score of the above tree node mapping set. Get a new ranking. Finally, select the node with the highest comprehensive score as the Node A1 The mapping node.

[0056] (5) Parameter search, select a set of items {P1, P2, P3, ... P p}, P p The pth item; each item corresponds to a document set M = {M1, M2, M3, ... M m},M m is the mth document; the document set M of each project includes project justification report, functional requirement document, outline design document, detailed design document, test document, functional specification, etc. Calculate the document set {M1,M2,M3,…M m}The function point node set in each document {N1, N2, N3, ... N k The total score of each node is S=a1*S1+a2*S2+a3*S3. The average score of each node and the node mapping of other documents is taken as the average score of the current node. If the mapping node of the maximum value is the actual correct node Defining the expectation function Where P is each project set, M is the document set in the project, (n∈M) and (n≠m) are the comparisons between the document and other documents, k is the function point node in the current document, S1, S2, S3 are the calculation score formulas for the comparison of different feature algorithms between each node and other nodes, and grid search is used to determine a1, a2, and a3; if the expectation is the largest, the corresponding a1, a2, and a3 are optimal. In order to improve the stability and generalization of parameters, K-fold cross validation is used for parameter learning.

[0057] (6) Mapping matrix: Based on the hyperparameters a1, a2, and a3, the model is determined. For a set of documents of a new project, namely, the project feasibility report, functional requirement document, outline design document, detailed design document, test document, functional specification, etc., after calculation, the mapping nodes and mapping scores of each functional requirement point in this set of documents and other materials in the group are obtained.

[0058] Furthermore, the step S5 comprises:

[0059] (1) Comprehensive analysis: conduct an overall analysis of the contents of each document of a set of materials for a new project, including single-document analysis and multi-document analysis.

[0060] (2) Single document analysis: analyze the content of each document, including the function point list and trust score, take the internal function point association mapping relationship and content of the document as input, call the module and sub-module analysis of external function points, and analyze the cross-validation quantity between function points and attachments.

[0061] (3) Multi-document analysis: First, conduct a joint analysis of the differences and associations between documents, including verification matching analysis of function points with other documents, mismatch analysis of function points with other documents, outlier function points, functions with multiple descriptions, function points with simplified and traditional descriptions, etc. Secondly, conduct a global total description, and describe the changes in the functional requirements of each function point node during its time period, including feasibility study, project establishment, requirements, development, testing, implementation, and other different links, as well as the differences and existence of the implementation process.

[0062] (4) Software iteration function point analysis report: Based on the results of the previous analysis, an analysis report is generated and output in the form of documents and tables. It can also be output to other business systems in more detailed ways such as XML, JSON, or databases.

[0063] The beneficial effects and contributions of the present invention are:

[0064] 1) Through technical means, various documents in the traditional project management process are structured and analyzed.

[0065] 2) Identify and locate the functional requirements in the document, and associate various descriptions of the functional requirements in the document, including those that appear in the outline, tables, or attachments. If you can locate one place, you can expand information in multiple places.

[0066] 3) Focus on solving different outline styles and different writing styles in different documents, connect the whole process documents centered on functional requirements, and track the changes in the life cycle of each functional point.

[0067] 4) Analyze the entire series of documents to form an iterative function point analysis report, which is conducive to the macro understanding of the entire project during project acceptance and project management, and strengthens the control of the granularity of acceptance. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a flow chart of a method and system for multi-source document mapping based on the software development field of the present invention.

[0069] Figure 2 It is a schematic diagram of the project document analysis calling parsing analysis engine of the present invention.

[0070] Figure 3 It is a schematic diagram of the document parsing engine functional requirement mapping process of the present invention.

[0071] Figure 4 It is a schematic diagram of the algorithm, features and process processing adopted by the internal document matching of the present invention.

[0072] Figure 5It is a schematic diagram of the algorithm, features and process processing adopted by the multi-document matching of the present invention.

[0073] Figure 6 It is a schematic diagram of the comprehensive analysis process of the present invention. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the technical solution and specific implementation of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0075] Embodiment 1

[0076] Sentence Embedding is a vector representation of the semantics of text content. It refers to the framework of the twin network model, inputs different sentences into two BERT models (but these two BERT models share parameters, and can also be understood as the same BERT model), and obtains the sentence representation vector of each sentence; and the sentence representation vector finally obtained can be used for semantic similarity calculation and unsupervised clustering tasks. Word2Vector is designed to convert each word into a word vector through the network model, and the word vector can also represent contextual information. There are two ways of Word2Vector: CBOX and Sikp-gram. K-folder is to divide the training / test data set into n mutually exclusive subsets, use one of the subsets as the validation set each time, and the remaining n-1 as the training set, perform n training and testing, and obtain n results. Grid search is an exhaustive search method that finds the optimal hyperparameter by traversing all possible combinations of hyperparameters. Grid search first sets a set of candidate values ​​for each hyperparameter, and then generates the Cartesian product of these candidate values ​​to form a combination grid of hyperparameters. Next, the grid search will train and evaluate the model for each hyperparameter combination to find the best hyperparameter combination. This patent starts with document analysis, identification of document functional requirements, mapping of functional requirements, and hyperparameter search, and finally builds a development and iteration report in the document throughout the life cycle based on functional requirements. This completes the management and control of functional requirements in the project management process. The entire data processing process is as follows: Figure 1 shown.

[0077] Furthermore, the step S1 comprises:

[0078] (1) Reading of project establishment report, including preprocessing of project establishment report, parsing the outline and content of project establishment report through project establishment report parsing engine, including the purpose, background, premise of feasibility study, and functional requirements involved, such as "functional requirements" xml structured content.

[0079] (2) Reading of project feasibility reports, including preprocessing of project approval reports, parsing of the outline and content of the feasibility report through the feasibility report parsing engine, including writing of project overview, project name, construction objectives, main construction contents, investment estimates, etc., and the functional requirements involved, such as "basic functions", are formed into XML structured content.

[0080] (3) Reading of requirement specifications, including preprocessing of requirement specifications. The requirement specification parsing engine is used to parse the outline and content of the requirement specifications, including the introduction, project overview, system functional design, non-functional requirements, acceptance criteria, attachments, etc. The functional requirements involved, such as "system functional design", are converted into XML structured content.

[0081] (4) Reading of the outline design document, including preprocessing of the outline design document. The outline and content of the outline design document are parsed through the outline design document parsing engine, including the introduction, design summary, system logic design, system error handling design, technical design, database design, vocabulary, attachments, etc. The functional requirements involved, such as "technical design", are formed into XML structured content.

[0082] (5) Reading of detailed design documents, including preprocessing of the detailed design documents. The detailed design document parsing engine is used to parse the outline and content of the detailed design document, including the introduction, project overview, program design description, functions, performance, input, output, algorithms, attachments, etc. The functional requirements involved, such as "program design description.function", are formed into XML structured content.

[0083] (6) Functional specification reading, including preprocessing of the functional specification. The functional specification parsing engine is used to parse the outline and content of the functional specification, including the introduction, project overview, operating environment, system functional description, subsystem X functional description, attachments, etc. The functional requirements involved, such as "subsystem X functional description", are converted into XML structured content.

[0084] Furthermore, the step S2 comprises:

[0085] (1) Outline navigation analysis. Based on the outline level numbers of the document, including the levels of humanities manuscripts and scientific and technological manuscripts, as well as the mixed compilation of the outlines of the two types of manuscripts. The first method is to use the hierarchical structure recorded in Outline, restore its outline serial numbers, and form a serial number outline. If there is marked content of OutLine in the original OpenXml document stream referenced in the O document tree, the LevelNumbering of the document will be recognized and converted into standard outline text, such as (1), (2), etc., and output according to the hierarchical relationship of the outline. The second method is that if there is no OutLine content, the hierarchical relationship of the outline in the paragraph will be recognized using the rules of Chinese outline writing. It is necessary to intercept the serial number part according to the serial number of each paragraph of the document and exclude the mixed content that conforms to the serial number format but is not an actual serial number, that is, the outline recognition algorithm, including sequence consistency, sequence continuity, etc. Using the hierarchical characteristics of the outlines of humanities manuscripts and scientific and technological manuscripts, corrections are made to form an outline paragraph with an upper and lower hierarchical structure. Combining the above two methods, the outlines in the original document are uniformly output as an ordered outline subtree node that has been automatically corrected and patched. And each outline node is uniformly recoded, including a unique sequence code and a unique hierarchical code. The sequence code is a sequence code starting from 1, such as 1, 2, 3, 4, 5, etc., and the hierarchical code is 1.1, 1.2, 1.2.1, 1.2.2, 2.1, 2.2, etc., which is recursively deduced according to the hierarchy to form a surrogate key to avoid discontinuity, non-uniqueness, and non-standardization of the outlines in the original document, which are not conducive to subsequent algorithm calculations. For example, if the actual outline sequence in the original document is typed as "3.2, 3.3, 3.4...", the sequence code will be recoded as "1, 2, 3...", and the hierarchical code will be recoded as "1.1, 1.2, 1.3...".

[0086] (2) Serial number recognition. First, it includes using "1." for the first layer, "(1)" for the second layer, "1." for the third layer, "(1)" for the fourth layer, or 1.1, 1.1.1, a), b), etc. The common writing order of capital numbers, full parentheses, right parentheses, dots, etc. is followed among various levels, and it does not strictly depend on a fixed writing method. It is possible to ignore one or more intermediate levels. For example, the lower level of "1." can directly be "1)" or "1". The algorithm determines the possible serial number level layout that may be adopted globally based on the serial number level and the characteristics of the first serial number appearance. And using this layout as a constraint, the possible outline serial numbers to be pre-recognized are adjusted, and the suspected serial numbers that do not conform to the characteristics are excluded. Secondly, the discontinuity of the serial numbers is recognized in a compatible manner. Sequences that meet the confidence threshold or above, such as "(1), (2), (4), (5), (6)", can be judged as the existence of the "(X)" outline level.

[0087] (3) Identification of function points in the outline. In the function point node tree, the nodes that are identified as outlines are called outline nodes. For each outline node, the title of the node + the title or content of the descendant nodes below the node form a new paragraph and input it into the sentence classification model. The model can use the Transformer classification model or a similar text deep learning classification model to determine whether the outline node is a function point node and output the score of the node as a function point node. Finally, according to the entire outline, for the same level sequence, if the score sequence exceeds the threshold, it is identified as a function point sequence.

[0088] (4) Table extraction: Identify the header part of the table, including the content on the left side of the X-axis and the content on the top of the Y-axis. Software function descriptions are mostly in the form of tables in various documents. They are read by merging cells to distinguish the hierarchical relationships. If a dimension of the X-axis or Y-axis in the table does not identify the software function requirements, the table is structured, the function points are flattened, and the other dimensional parts along the X-axis or Y-axis are used as the description or feature part of the function point. In the software requirements specification and the function module matrix, identify the function module name, function module description, function module indicators, etc.

[0089] (5) Dimension identification: According to the table header, the identification table is a top-down structure. The header is in the first column, the first column is the serial number, the second column is the system name, the third column is the function name, the fourth column is the function description, and the fifth column is the function indicator.

[0090] (6) Function point identification in the table, where the second and third columns may be the names of software function requirements, but because the third column has a clear and consistent relationship with the title directory of other outlines and appears frequently, the coverage is higher than the second column, and the sequence proximity is high, the third column is selected as the function requirement name column, and the other columns are extended columns.

[0091] Furthermore, the above steps also include intra-document matching:

[0092] (3) According to the above steps, the outline of the document and the nodes marked as function points and the function points that may be included in the table can be obtained. For all the function points in the document, the title matching and description semantic matching are used to perform semantic conversion on the title and description, including SBert semantic vector conversion or Word2Vector semantic vector conversion, and convert them into representation vectors that can perform distance calculation.

[0093] (4) Summarize the frequency of the title appearing in the outline. If it occurs more than once, then the title = parent title + current title. If it is still repeated, then the title = parent title + parent title + current title until there is no ambiguity. Segment the title, remove single words and stop words, etc., and only retain verbs and nouns. Use the Jaccard calculation formula to calculate the TFIDF weighted distance value, and fuse the semantic distance score and the Jaccard distance part. D(m,n) = p1*V1(m,n)+(1-p1)V2(m,n), where V1 represents the semantic distance and V2 represents the Jaccard distance. Among them, p1 is the fusion weight of the semantic distance, 1-p1 is the fusion weight of the Jaccard distance of the words in the two paragraphs, m represents the source function point node, and n represents the candidate target function point node. D(m,n) exceeding 0.8 is set as a semantically similar function point.

[0094] Furthermore, the step S3 comprises:

[0095] (1) Tree node matching: traverse each XML function point node tree of multiple documents, match the function outline tree with the function outline trees of other documents, and use matching algorithms including name matching, subtree matching, and description semantic matching. After weighting multiple matching algorithms, the top 10 nodes with the highest matching degree are used as the candidate set of mapping nodes linked to the node in another document according to the comprehensive weights, and each function requirement node is matched. The function point outline tree is a subset of the outline tree, and the parent-child structure relationship is retained for the content identified as function point nodes. The function point outline tree extracted from document A is recorded as tree A, and the function point outline tree extracted from document B is recorded as tree B.

[0096] (2) Name matching: For each candidate matching pair, calculate the score of each node and record it as S1.

[0097] (3) Subtree matching: According to the node hierarchy, the child node is matched with the target child node, and all the parent nodes of the target child node are collected for statistics. The weight of each level takes the name match as the basic weight, which is multiplied by the coefficient function of the level of the current tree node. The level coefficient function can be a logarithm or other mathematical formula, and is accumulated upward to the parent node. Finally, the optimal mapping node of tree B corresponding to the first-level node of tree A is determined. The score of each node of the mapped tree B is the sum of all scores of all the child mapping bubbles of tree A, such as Node A1 The candidate mapping set is {Node B2.1(w1) ,Node B2.1.1(w2) ,Node B4.2(w3) ,Node B2.3(w4) ,Node B2.3.1(w5) ,Node B2.4(w6),…}, get Node B2The matching score of Node is w1+s2+w4+s5+w6. B4＝w3， Node B2.3＝w4+w5， Node B2.1(w1) ,Node B2.1.1(w2) ,Node B4.2(w3) ,Node B2.3(w4) ,Node B2.3.1(w5) ,Node B2.4(w6) , rearrange them in reverse order, and cut off the data below the threshold to reduce the amount of calculation. The best 10 candidate pairs are retained for each mapping, and their scores are recorded as S2.

[0098] (4) Describe semantic matching. Read the source description and the mapped target description of the mapping set generated by the above steps, convert them into vectors using a semantic model, and calculate the distance between the two vectors using cosine or other distance calculation methods. And convert this distance into a standard score. The calculation formula is S3(m,n)=p1*V1(m,n)+(1-p1)V2(m,n), where V1 represents the semantic distance and V2 represents the Jaccard distance. Among them, p1 is the fusion weight of the semantic distance, 1-p1 is the fusion weight of the Jaccard distance of the words in the two paragraphs, m represents the source function point node, and n represents the function point node of the candidate target. And integrate it into the score of the above tree node mapping set. Get a new ranking. Finally, select the node with the highest comprehensive score as the Node A1 The mapping node.

[0099] (5) Parameter search, organize several projects Project1, Project2, Project3, etc., and their collection is {P1, P2, P3, ... Pp}. The document collection M of each project is project feasibility report, functional requirements document, outline design document, detailed design document, test document, functional specification, etc. Calculate the total score of each node in the document collection {M1, M2, M3, ... Mm} of each project and the function point node collection {N1, N2, N3, ... Nk} of each document M. The node formula is S = a1*S1+a2*S2+a3*S3. The average score of each node mapped with the nodes of other documents is taken as the average score of the current node. If the mapped node with the maximum value is the actual correct node, p represents the mapped function point node, p^ is the function point of another document that is approximately equivalent to p. If it is truly approximately equivalent, then p = p^. If it is mapped to another function point node with a different concept, then p ≠ p. Define the expectation function Where P is each project set, M is the document set in the project, (n∈M) and (n≠m) are the comparisons between the document and other documents, k is the function point node in the current document, S1, S2, S3 are the calculation score formulas for the comparison of each node with other nodes by different feature algorithms, grid search is used, each group of parameters step=0.1 is set, a1, a2, a3 are determined, and the expectation is the largest, then the corresponding a1, a2, a3 are the best. In order to improve the stability and generalization of parameters, parameter learning uses K-fold cross validation, and the actual content a1=0.7, a2=0.6, a3=0.2.

[0100] (6) Mapping matrix: Based on the hyperparameters a1, a2, and a3, the model is determined. For a set of documents of a new project, namely, the project feasibility report, functional requirement document, outline design document, detailed design document, test document, functional specification, etc., after calculation, the mapping nodes and mapping scores of each functional requirement point in this set of documents and other materials in the group are obtained.

[0101] Furthermore, the step S5 comprises:

[0102] (1) Comprehensive analysis: conduct an overall analysis of the contents of each document of a set of materials for a new project, including single-document analysis and multi-document analysis.

[0103] (2) Single document analysis: analysis of the content of each document, including function point lists and trust scores, analysis of external function point modules and submodules, and analysis of the number of cross-validations between function points and attachments.

[0104] (3) Multi-document analysis: First, conduct a joint analysis of the differences and associations between documents, including verification matching analysis of function points with other documents, mismatch analysis of function points with other documents, outlier function points, functions with multiple descriptions, function points with simplified and traditional descriptions, etc. Secondly, conduct a global total description, and describe the changes in the functional requirements of each function point node during its time period, including feasibility study, project establishment, requirements, development, testing, implementation, and other different links, as well as the differences and existence of the implementation process.

[0105] (4) Software iteration function point analysis report: Based on the results of the previous analysis, an analysis report is generated and output in the form of documents and tables. It can also be output to other business systems in more detailed ways such as XML, JSON, or databases.

[0106] The specific implementation process of each module refers to the above description of the method of the present invention.

[0107] Another embodiment of the present invention provides an electronic device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.

[0108] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk), wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the steps of the method of the present invention are implemented.

[0109] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. A person skilled in the art may modify or make equivalent substitutions for the technical solutions of the present invention without departing from the principle and scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A multi-source document mapping method based on the field of software development, the steps of which include: 1) Convert the project documents of each stage of the target project into a unified format to obtain a multi-source document set; According to the structured data structure of each document in the multi-source document collection, the corresponding document is stored as an XML tree, wherein the XML tree includes two parts: a document information node and a document content node, wherein the document information node includes the file name, creation date, author, and original document file path of the document, and the document content node is the starting root node of the XML tree; 2) for each document information node in the XML tree, identifying the function point node to which the document information node belongs; extracting the content of each function point node and concatenating it with the context of the function point node as the context of the function point node; Associating each internal node of the function point node with other function point nodes outside the region where the function point node is located, calculating the association score, and determining the candidate function point node of the internal node according to the association score; 3) If function point nodes with the same name appear in multiple places in the XML tree, the contexts of the function point nodes are merged; a mapping matrix is ​​established based on the matching results between the function point nodes corresponding to each document and the function point nodes corresponding to other documents, and the optimal mapping sequence for each document is obtained based on the mapping matrix to form a mapping table.

2. The method according to claim 1, characterized in that For each document, the function point nodes in the document are listed and described, and the same function points in each area of ​​the document are linked to each other.

3. The method according to claim 1, characterized in that The method of establishing the mapping matrix is: 31) The XML tree corresponding to the document A is recorded as tree A; for each function point node d in tree A, the function outline tree corresponding to the function point node d is matched with the outline trees in the XML trees corresponding to other documents, and the candidate set of mapping nodes for the function point node d is determined according to the matching results; 32) Calculate the name similarity S1 between the function point node d and its child nodes and each node in the mapping node candidate set; 33) According to the superior-subordinate relationship of the function point node d in the tree A, the child nodes of the function point node d are obtained; and each child node d corresponding to the function point node d is obtained according to the mapping node candidate set. j Match the target child node, obtain all the parent nodes of the target child node; use the name similarity between the nodes as the basic weight, multiply it by the level coefficient function of the function point node d, and accumulate it upward to the parent node to determine the optimal mapping node of tree B corresponding to the first-level node of tree A; The scores of the optimal mapping nodes of the function point node d are calculated and recorded as S2; tree B is the XML tree corresponding to document B; 34) Calculate the semantic similarity between the function point node d and each optimal mapping node, and select the node with the highest semantic similarity as the mapping node of the function point node d; the semantic similarity score between the function point node d and its mapping node is recorded as S3; 35) Calculate the total score S of the function point node d according to S1, S2, and S3 as the mapping score between the function point node d and the corresponding mapping node; 36) Establishing the mapping matrix according to the mapping nodes and mapping scores corresponding to each function point node.

4. The method according to claim 1, 2 or 3, characterized in that: According to the mapping table and the context of each function point node, the description and definition of the corresponding function point in each stage of the target project development are described.

5. The method according to claim 1, 2 or 3, characterized in that: The method for obtaining the multi-source document collection is: pre-process the project document format and convert it into OpenXml format and parse it to obtain the hierarchical outline, table content, text content, and illustrations of each project document, forming a multi-source document collection that reflects the project demand process at each stage of the project process.

6. The method according to claim 1, 2 or 3, characterized in that: According to the title feature, sequence number feature and all parent node titles corresponding to the document information node, the function point node to which the document information node belongs is comprehensively identified.

7. The method according to claim 1, 2 or 3, characterized in that: A total analysis is performed on the mapping table to comprehensively describe the degree of completion of the functions involved; the total analysis includes function point scale, time coverage, function point coverage rate, test rate, project establishment rate, design rate, and change rate.

8. A multi-source document mapping system based on the field of software development, characterized in that: It includes document preprocessing module, association analysis module and mapping module; The document preprocessing module is used to convert the project documents of each stage of the target project into a unified format to obtain a multi-source document set; according to the structured data structure of each document in the multi-source document set, the corresponding document is stored as an XML tree, the XML tree includes two parts: a document information node and a document content node, the document information node includes the file name, creation date, author, and original document file path of the document, and the document content node is the starting root node of the XML tree; The association analysis module is used to identify, for each document information node in the XML tree, a function point node to which the document information node belongs; Extract the content of each function point node and concatenate it with the context of the function point node as the context of the function point node; Associating each internal node of the function point node with other function point nodes outside the region where the function point node is located, calculating the association score, and determining the candidate function point node of the internal node according to the association score; The mapping module is used to merge the contexts of function point nodes with the same name appearing in multiple places in the XML tree; The matching results of the function point nodes corresponding to each document and the function point nodes corresponding to other documents are calculated, a mapping matrix is ​​established according to the matching results, and the optimal mapping sequence of each document is obtained according to the mapping matrix to form a mapping table.

9. A server, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing each step in the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images