Document consistency comparison method based on semantic analysis and keyword driving

Abstract

The invention provides a document consistency comparison method based on semantic analysis and keyword driving. The method comprises the following steps: firstly, constructing a directory structure and a weight model by analyzing a directory of an input document, preferentially positioning key chapters and extracting contents; then, a semantic vector is generated by utilizing a pre-training language model, and keyword capture and similar semantic search are carried out on the document in combination with a multi-layer semantic analysis technology; multi-level semantic comparison is carried out at three levels of vocabularies, sentences and paragraphs, and semantic link tracking and an intelligent adjustment mechanism are combined, so that differences among documents are accurately identified, and content consistency is ensured. And carrying out matching and verification on contents by utilizing methods such as cosine similarity and Euclidean distance, identifying logic contradictions and information omission, and finally outputting a structured text. According to the invention, the accuracy and efficiency of document comparison can be effectively improved, and the accuracy and consistency of document contents are ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing and information retrieval, and in particular relates to a semantic analysis and keyword-driven document consistency comparison method. Background Art

[0002] During the project life cycle management process, a large number of complex documents will be generated at different stages (such as project planning, procurement management, contract signing, project execution, and results acceptance). These documents are often large in content, complex in structure, and involve multiple aspects of information. Consistency is crucial to the successful implementation of the project. However, existing manual review and traditional text comparison methods are inefficient and prone to missing details. It is difficult to accurately identify potential differences and logical contradictions between documents. Especially when dealing with complex plans, clauses and other key contents, fast and accurate content comparison is particularly important.

[0003] Traditional comparison methods mostly rely on simple keyword matching and manual inspection, which makes it difficult to capture the deep semantic connections between documents and cannot effectively handle complex sentence structures and semantic drift. This deficiency makes it difficult for existing systems to ensure efficiency and accuracy when dealing with large-scale text comparisons and document consistency verification, affecting the smooth progress of the project. With the rapid development of natural language processing technology, comparison technology based on semantic understanding has gradually become the key to solving this problem. However, there is currently a lack of systems on the market that integrate multi-layer semantic parsing, keyword-driven and optimization strategies. There is an urgent need for a solution that can achieve multi-level, intelligent comparison to cope with complex document consistency challenges in various process scenarios. Summary of the invention

[0004] The present invention aims to provide a document consistency comparison method driven by multi-layer semantic analysis and keywords, so as to improve the efficiency and accuracy of document comparison at all stages of the project. Through catalog analysis, semantic embedding, keyword capture and multi-layer semantic comparison, document differences and logical contradictions can be automatically identified, structured reports and visualization results can be generated, and content review and optimization can be assisted.

[0005] The technical solution of the present invention is as follows:

[0006] The semantic analysis and keyword-driven document consistency comparison method includes the following steps:

[0007] S1. Intelligent analysis and weight distribution through document directory to achieve accurate positioning of key content;

[0008] S2, use the pre-trained language model to perform deep semantic analysis on the document, generate a multi-dimensional semantic vector, and then perform dimensionality reduction optimization to continue semantic computing;

[0009] S3, extract the core keywords of the document and assign weights, combine with similar semantic search algorithm, optimize the keyword-driven semantic matching process, and identify potential differences between documents;

[0010] S4, through the multi-level comparison algorithm, the content comparison at different levels is refined semantically compared;

[0011] S5. Conduct in-depth modeling of the contextual semantics of the document content to detect logical contradictions and information omissions to ensure the semantic coherence and accuracy of the document in the overall process;

[0012] S6. Combine intelligent prompts and visualization tools to generate differential analysis results.

[0013] Furthermore, the specific steps of step S1 include the following:

[0014] S11, directory extraction and parsing: Use text parsing algorithms to automatically identify the directory part in the document, extract chapter titles, numbers, and page number information, form a preliminary directory framework, convert the identified directory framework into a tree structure, and generate a corresponding index table;

[0015] S12, catalog weighting model and automatic content positioning: The catalog weighting model is used to assign weights to each chapter and sub-chapter. The weights are set according to the relevance of the chapter title and the importance of the chapter content. Based on the catalog weighting model, high-weight chapter content is extracted first, and a local semantic vector matrix is ​​generated.

[0016] S13, key chapter location and content segment division: quickly locate relevant chapters from the directory framework based on predefined keywords, and directly extract the target chapter content based on the page number information in the directory framework, and then refine the extracted chapter content and divide the content into smaller semantic units.

[0017] Further, the specific steps of step S2 include the following:

[0018] S21, application of pre-trained language model: use the pre-trained language model BERT to capture the contextual relationship and deep semantic information of the document, input the extracted content fragments into the pre-trained language model, use its encoder part to process the document, and generate the corresponding high-dimensional semantic vector;

[0019] S22, multi-dimensional semantic vector generation;

[0020] S23, Dimensionality reduction and optimization: Dimensionality reduction technology is used to reduce the dimension of semantic vectors. During the dimension reduction process, the feature dimensions that have the greatest impact on semantic calculation are retained to ensure that the vector after dimension reduction can still accurately reflect the semantic features of the original document.

[0021] S24, semantic vector storage and management: the generated semantic vector is stored in a vector database, the vector database supports vector retrieval and similarity calculation, and the vector database indexes the semantic vector to achieve efficient approximate nearest neighbor search.

[0022] Further, the specific steps of step S3 include the following:

[0023] S31, keyword extraction and weight assignment;

[0024] S311, keyword extraction: Based on the generated semantic vectors, the pre-trained language model is used to deeply analyze the text and automatically identify the core keywords in the document. The specific steps include: first, the language model is used to perform context analysis on the vocabulary to generate vectors that reflect the semantic relationship; then, combined with the word frequency information, the high-frequency and topic-related words in the full text or key chapters are screened out; then, the occurrence positions of these words are examined in detail; finally, the word parts and semantic relevance are comprehensively considered to finally select the keywords that best represent the core content of the text;

[0025] S312, keyword weight allocation: according to the importance of the keyword in the document and its influence in different semantic levels, an initial weight is allocated to each keyword. If a keyword appears in multiple important paragraphs or is highly relevant to the key topic, its weight is increased. In the subsequent matching process, the keyword weight is dynamically adjusted according to the matching effect and difference recognition results.

[0026] S32, similar semantic keyword search;

[0027] S321, similar semantic space construction: map the extracted keywords to the semantic space, generate the semantic vector representation of the keywords using semantic embedding technology, construct a multidimensional space containing the keyword semantic vectors, capture the potential semantic associations between keywords, and provide a basis for similar keyword search;

[0028] S322, similarity calculation and matching: by calculating the similarity between the semantic vectors of the keywords, the semantic proximity of the keywords is quantified, and in the compared documents, words or phrases with similar semantics to the target keywords are searched to identify similar semantic contents in different expressions;

[0029] S323, Dynamic Semantic Expansion and Associative Search: Use word embedding and semantic network to expand the target keywords to synonyms, near synonyms and related words, and combine the usage of keywords in context to perform semantic associative search, capture concepts and topics related to keywords, and expand the matching range;

[0030] S33, keyword-driven semantic matching optimization;

[0031] S331, multi-level semantic comparison: Multi-level comparison includes comparison of vocabulary, sentence and paragraph:

[0032] Lexical comparison: Use word embeddings to semantically match and compare keywords, quickly identifying the consistency or difference between two documents at the terminology and concept level.

[0033] Sentence-level comparison: Calculate the similarity between sentences through sentence embedding to determine whether the two documents have the same expression of the same topic sentence, and discover differences in expression and potential information missing.

[0034] Paragraph-level comparison: compare paragraphs as whole semantic units to ensure that the document is logically consistent and thematically coherent at the high-level semantic units, and to detect information additions, deletions, and topic deviations in a timely manner.

[0035] S332, difference identification and intelligent adjustment: based on the comparison results, automatically identify the difference in keyword matching, mark the inconsistent parts, generate intelligent prompts for the identified differences, point out the specific difference content and location, and dynamically adjust the keyword weight and search strategy based on the comparison feedback;

[0036] S333, semantic link tracking: Track the semantic trajectory of keywords in documents, record their usage and meaning changes in different contexts, ensure that the meaning of keywords in different documents and different locations is consistent through semantic link tracking, and identify any semantic drift or logical contradictions;

[0037] S34, combined with similar semantic search algorithm;

[0038] S341, Optimize the matching process: organically combine keyword extraction, weight allocation and similar semantic search algorithm to form an efficient keyword-driven matching mechanism;

[0039] S342, Identification of potential differences and optimization of verification process: Use similar semantic search algorithm to deeply compare the potential semantic differences between documents, find the content that cannot be identified by keyword matching alone, classify the identified differences, distinguish terminology differences, expression differences, and content missing types, and provide a basis for subsequent processing;

[0040] The extracted and matched keywords and similar semantic contents will serve as the basic data for the next step of multi-level semantic comparison.

[0041] 5. The semantic analysis and keyword-driven document consistency comparison method according to claim 4 is characterized in that: the specific steps of step S4 include the following:

[0042] S41, semantic comparison at the lexical level: using the extracted keywords, perform direct lexical matching in two documents, identify the same words and terms, use word embedding technology to calculate the semantic similarity between words, identify words with different word forms but similar semantics, and expand the matching range;

[0043] S42, sentence-level semantic comparison: Use the Sentence-BERT model to convert sentences into semantic vectors, capture the overall semantic information of sentences, calculate the semantic similarity of corresponding sentences in two documents, and identify sentences with the same meaning but different expressions and sentences with different meanings;

[0044] S43, paragraph-level semantic comparison: semantically embed the paragraph content, obtain the overall semantic representation of the paragraph, consider the context and logical structure, compare the semantic vectors of corresponding paragraphs, evaluate the consistency of paragraph content, and find out the information increase or decrease or logical inconsistency.

[0045] S44, comprehensive comparison and difference identification: The comparison results at the vocabulary, sentence, and paragraph levels are integrated to form an overall assessment of document consistency. Based on the comparison results, the specific vocabulary, sentence, or paragraph where the difference exists is accurately located;

[0046] S45, comparison result optimization: assign different weights to the comparison results at different levels according to project requirements.

[0047] 6. The semantic analysis and keyword-driven document consistency comparison method according to claim 5 is characterized in that: the specific steps of step S5 include the following:

[0048] S51, Contextual Semantic Modeling and Logical Contradiction Detection: Use the pre-trained language model to perform global semantic modeling of document content, capture the semantic associations and logical relationships between content through semantic embedding of paragraphs and chapters, and build the semantic network structure of the document;

[0049] Based on semantic modeling, natural language inference (NLI) technology and logical rules are applied to detect logical contradictions in documents and automatically mark the contradictory parts to prompt users to review and correct them.

[0050] S52, Information omission detection and semantic consistency assurance: According to project management and document standards, establish a key information list, use information extraction technology to extract core entities and events from the document, compare them with the key information list, identify missing information, and prompt users for possible missing content;

[0051] By analyzing the semantic flow of the document, evaluating the coherence between paragraphs and chapters, calculating the semantic similarity and logical association of adjacent content, identifying topic mutations, content jumps or semantic faults, and providing modification suggestions for detected coherence issues.

[0052] 7. The semantic analysis and keyword-driven document consistency comparison method according to claim 6 is characterized in that: the specific steps of step S6 include the following:

[0053] S61, Generation of a difference analysis report: All detected differences, logical contradictions and information omissions are summarized and a detailed difference analysis report is generated. The difference analysis report is presented in a structured form, including consistent, partially consistent and inconsistent content, and the chapter, paragraph or sentence position of the difference is marked. For each difference, a detailed description is provided, indicating the specific problem, the impact of the problem and the recommended solution. The difference analysis report also includes overall statistical data, such as the number of differences, type distribution and severity;

[0054] S62, intelligent prompts and modification suggestions: For each detected problem, semantic analysis and generative technology are used to refer to the industry knowledge base and relevant standards to automatically generate simplified and compliant modification suggestions for users. For inconsistent terminology, standard definitions are directly quoted, and logically coherent sentence structures and logically coherent content are given for logically incoherent parts. In the process of users reviewing and selecting modification plans, suggestions are continuously optimized based on user feedback;

[0055] S63, intuitive display of visualization tools: Use visualization tools to intuitively display the comparison results.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] 1. The present invention significantly improves the efficiency and accuracy of document comparison at all stages of the project by introducing a multi-level document comparison method driven by semantic parsing and keywords. By using intelligent parsing of the document directory, key chapters can be accurately located, and the deep semantic associations of the document can be captured in depth in combination with the multi-dimensional semantic vectors generated by the pre-trained language model. Through refined semantic comparisons at different levels such as vocabulary, sentences, and paragraphs, subtle differences and potential problems between documents, including logical contradictions and information omissions, can be automatically identified. Contextual semantic analysis ensures the semantic coherence and accuracy of the document in the overall process. Finally, a difference analysis report is generated, and combined with intelligent prompts and visualization tools, users can intuitively understand the comparison results, and quickly conduct content review and decision optimization. The present invention is suitable for a variety of scenarios such as full-process project management, and can effectively improve the accuracy and efficiency of document comparison, and ensure the accuracy and consistency of document content. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of Example 1 of the present application;

[0059] Figure 2 This is a flow chart of step S2 of Example 1 of the present application;

[0060] Figure 3 This is a flowchart of step S3 of Example 1 of the present application;

[0061] Figure 4 This is a flowchart of step S4 of Example 1 of the present application. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] Example 1

[0064] like Figure 1-4 As shown in FIG. 1 , the semantic parsing and keyword-driven document consistency comparison method includes the following steps:

[0065] S1, intelligently analyzes and assigns weights through document directories to accurately locate key content;

[0066] By parsing the document directory, building the directory structure and weighted model, the key content can be accurately located, thereby improving the efficiency and accuracy of the comparison.

[0067] S11, directory extraction and parsing: Use text parsing algorithms to automatically identify the directory part in the document, extract chapter titles, numbers, page numbers and other information, and form a preliminary directory framework. Convert the identified directory information into a tree structure to clearly represent the hierarchical relationship between chapters and sub-chapter. Generate a corresponding index table to quickly locate specific content in the document.

[0068] S12, directory weighting model and automatic content positioning: Assign weights to each chapter and sub-chapter, and the weights are set according to the relevance of the chapter title and the importance of the chapter content. Prioritize important chapters according to the weights to reduce the waste of computing resources and improve the overall document parsing efficiency. Based on the directory weighting model, high-weight chapter content is extracted first, and a local semantic vector matrix is ​​generated to avoid processing irrelevant content, significantly improving the processing efficiency of large documents and avoiding unnecessary full-text scanning.

[0069] S13, key chapter location and content segmentation: Based on predefined keywords, such as "construction content", "technical solution", "implementation plan", etc., quickly locate relevant chapters from the directory structure, and directly extract the target chapter content based on the page number information in the directory, avoiding full-text scanning and saving time. Refine the extracted chapter content and divide the content into smaller semantic units (such as paragraphs and sentences) to prepare for subsequent semantic vector generation and multi-level semantic comparison.

[0070] S2, applies the pre-trained language model to perform deep semantic analysis on the document, generates a multi-dimensional semantic vector, and then performs dimensionality reduction optimization to continue semantic computing;

[0071] After successfully locating and extracting key content, the deep semantic analysis phase begins. This step aims to convert the document content into a high-dimensional semantic vector representation, capture the deep semantic features of the document, and provide a basis for subsequent keyword-driven comparison and consistency verification.

[0072] S21, Application of pre-trained language model: The pre-trained language model BERT is used to capture the contextual relationship and deep semantic information of the document through its semantic understanding ability. The extracted content fragments are input into the pre-trained language model, and the document is processed using its encoder part to generate the corresponding high-dimensional semantic vectors (Embeddings). These vectors can effectively represent the semantic features of the document and support subsequent semantic calculation and matching.

[0073] S22, multi-dimensional semantic vector generation: In the process of generating semantic vectors, the model not only considers information at the lexical level, but also captures multi-dimensional features such as sentence structure, context association, and semantic dependency. Through this deep semantic analysis, a comprehensive semantic representation of the document can be obtained. The bidirectional encoding mechanism of the pre-trained model enables it to consider the contextual information of the document at the same time, enhances the ability to understand polysemous words and complex sentences, and ensures the accuracy of the semantic vector.

[0074] S23, Dimensionality reduction and optimization: Although high-dimensional semantic vectors can fully represent the semantic information of documents, they will bring about a large computational overhead in actual calculations. In order to improve computational efficiency, dimensionality reduction technology (principal component analysis PCA) is used to reduce the dimensionality of semantic vectors. In the dimensionality reduction process, the focus is on retaining the feature dimensions that have the greatest impact on semantic calculations to ensure that the vectors after dimensionality reduction can still accurately reflect the semantic features of the original document. Through dimensionality reduction processing, the dimension of the semantic vector is reduced, the complexity of subsequent semantic matching and similarity calculations is reduced, and the overall performance and response speed are improved.

[0075] S24, semantic vector storage and management: The generated semantic vectors are stored in an efficient vector database to support fast vector retrieval and similarity calculation. To speed up the vector matching process, the semantic vectors are indexed (using FAISS vector retrieval technology) to achieve efficient approximate nearest neighbor search. The generated multi-dimensional semantic vectors provide a data basis for subsequent keyword extraction, similar semantic search, and multi-level semantic comparison, ensuring a deep understanding of the document content and accurate identification of potential differences between documents.

[0076] S3, extracts the core keywords of the document and assigns weights, combines similar semantic search algorithms, optimizes the keyword-driven semantic matching process, and identifies potential differences between documents;

[0077] After completing the deep semantic analysis of the document and generating the multi-dimensional semantic vector, the keyword extraction and similar semantic search phase begins. The core goal of this step is to automatically extract core keywords from the document, reasonably assign weights, and combine similar semantic search algorithms to optimize the keyword-driven semantic matching process, thereby identifying potential differences between documents.

[0078] S31, keyword extraction and weight assignment

[0079] S311, keyword extraction: Based on the generated semantic vectors, the pre-trained language model is used to deeply analyze the text and automatically identify the core keywords in the document. The specific steps include: first, the language model is used to perform contextual analysis on the vocabulary to generate vectors that reflect semantic relationships; then, combined with word frequency information, the high-frequency and topic-related words in the full text or key chapters are screened out; then, the locations where these words appear are examined, such as high-weight areas such as titles, abstracts, and the beginning of chapters; finally, the word parts and semantic relevance are combined to finally select the keywords that best represent the core content of the text. Through this process, it is ensured that the extracted keywords accurately reflect the main ideas and themes of the document.

[0080] S312, keyword weight assignment: Assign initial weights to each keyword based on its importance in the document and its influence at different semantic levels (vocabulary, sentence, paragraph). Consider the relevance of keywords to the context. If a keyword appears in multiple important paragraphs or is highly relevant to key topics, increase its weight. In the subsequent matching process, dynamically adjust keyword weights based on matching effects and difference recognition results to optimize matching performance.

[0081] S32, similar semantic keyword search

[0082] S321, similar semantic space construction: map the extracted keywords into the semantic space, and use semantic embedding technology (Sentence-BERT) to generate semantic vector representations of keywords. Construct a multidimensional space containing keyword semantic vectors to capture the potential semantic associations between keywords and provide a basis for similar keyword search.

[0083] S322, similarity calculation and matching: quantify the semantic proximity of keywords by calculating the similarity between keyword semantic vectors. Search for words or phrases with similar semantics to the target keyword in the compared documents, and identify similar semantic contents in different expressions.

[0084] S323, Dynamic semantic expansion and associative search: Using word embedding and semantic network (WordNet), the target keywords are expanded to synonyms, near synonyms and related words to enrich the keyword set. Combined with the usage of keywords in the context, semantic associative search is performed to capture concepts and topics related to keywords (for example, "construction plan" is associated with "construction progress", "construction schedule", etc.), and the matching range is expanded.

[0085] S33, keyword-driven semantic matching optimization

[0086] S331, multi-level semantic comparison: Multi-level comparison includes comparison of vocabulary, sentence and paragraph:

[0087] Lexical comparison: Use word embeddings to semantically match and compare keywords, quickly identifying the consistency or difference between two documents at the terminology and concept level.

[0088] Sentence-level comparison: Calculate the similarity between sentences through sentence embedding to determine whether the two documents have the same expression of the same topic sentence, and discover differences in expression and potential information missing.

[0089] Paragraph-level comparison: Compare paragraphs as whole semantic units to ensure that the document maintains logical consistency and thematic coherence at the high-level semantic units, and promptly detect information additions and deletions and thematic deviations. The high-level semantic units refer to elements that can represent the core meaning and content of a text or discourse in language processing and information retrieval. High-level semantic units are often abstracted from specific vocabulary and sentence structures, and can capture the deep meaning of the text. They regard paragraphs (or larger structures, such as chapters) as a whole and complete semantic carrier. High-level semantic units combine all sentences, contextual associations, and thematic contexts in the paragraph for analysis, rather than just looking at a single word or sentence. By comparing paragraphs as a whole, you can more comprehensively evaluate the consistency of the content, logical coherence, and whether the theme in the document has shifted.

[0090] S332, Difference Identification and Intelligent Adjustment: Based on the comparison results, automatically identify the differences in keyword matching and mark the inconsistent parts. Generate intelligent prompts for the identified differences, point out the specific content and location of the differences, and facilitate user review. Based on the comparison feedback, dynamically adjust the keyword weight and search strategy. For example, for keywords with low matching but high importance, their weight can be increased to enhance their influence in the matching process.

[0091] S333, semantic link tracking: Track the semantic trajectory of keywords in documents and record their usage and changes in meaning in different contexts. Through semantic link tracking, ensure that the meaning of keywords in different documents and different locations remains consistent, and identify semantic drift or logical contradictions.

[0092] S34, combined with similar semantic search algorithm

[0093] S341, Optimize the matching process: organically combine keyword extraction, weight allocation and similar semantic search algorithm to form an efficient keyword-driven matching mechanism. By optimizing the algorithm, the accuracy and speed of matching are improved, and the number of mismatches and missed matches is reduced.

[0094] S342, Identification of potential differences and optimization of confirmatory processes:

[0095] Using similar semantic search algorithms, we can deeply compare the potential semantic differences between documents and find content that cannot be identified by surface keyword matching alone. We can classify the identified differences and distinguish between types such as terminology differences, expression differences, and content missing, providing a basis for subsequent processing.

[0096] The extracted and matched keywords and similar semantic contents will serve as the basic data for the next multi-level semantic comparison. Through accurate keyword matching and difference identification, the complexity of subsequent comparisons can be reduced and the efficiency and effectiveness of consistency verification can be improved.

[0097] S4, through a multi-level comparison algorithm, performs refined semantic comparison of content at different levels such as vocabulary, sentences, and paragraphs to improve the verification accuracy of document consistency;

[0098] Perform refined semantic comparison of document content at different levels (vocabulary, sentences, paragraphs) to comprehensively improve the verification accuracy of document consistency.

[0099] S41, semantic comparison at the vocabulary level: Using the keywords extracted in the previous step, perform direct vocabulary-level matching in the two documents to identify the same words and terms. Use word embedding technology (Word2Vec) to calculate the semantic similarity between words, identify words with different word forms but similar semantics, and expand the matching range.

[0100] S42, sentence-level semantic comparison: Use models such as Sentence-BERT to convert sentences into semantic vectors to capture the overall semantic information of sentences. Calculate the semantic similarity of corresponding sentences in two documents, and identify sentences with the same meaning but different expressions, as well as sentences with differences.

[0101] S43, paragraph-level semantic comparison: semantically embed the paragraph content to obtain the overall semantic representation of the paragraph, taking into account the context and logical structure. Compare the semantic vectors of corresponding paragraphs, evaluate the consistency of the paragraph content, and discover possible information additions or deletions or logical inconsistencies.

[0102] S44, comprehensive comparison and difference identification: The comparison results at the vocabulary, sentence, and paragraph levels are integrated to form an overall assessment of the consistency of the document. Based on the comparison results, the specific vocabulary, sentence, or paragraph with differences is accurately located to facilitate the user's focused review.

[0103] S45, optimization of comparison results: according to project requirements, different weights are assigned to the comparison results at different levels. For example, important paragraph comparisons can be assigned higher weights. Set a similarity threshold, and when the comparison result is below the threshold, it is automatically marked as "possibly inconsistent" or "needs review".

[0104] S5, conducts deep modeling of the contextual semantics of the document content, detects logical contradictions and information omissions, and ensures the semantic consistency and accuracy of the document in the overall process; after completing the multi-level semantic comparison, further conducts deep modeling of the contextual semantics of the document, examines the document content from a global perspective, detects potential logical contradictions and information omissions, and ensures the semantic consistency and accuracy of the document.

[0105] S51, contextual semantic modeling and logical contradiction detection: Use pre-trained language models (BERT, GPT series models) to perform global semantic modeling of document content. By embedding the semantics of paragraphs and chapters, the semantic associations and logical relationships between content are captured, and the semantic network structure of the document is constructed. The model can understand the topic, sentiment, and intent of the document, providing a basis for subsequent logical analysis.

[0106] Based on semantic modeling, natural language inference (NLI) technology and logical rules are applied to detect possible logical contradictions in documents. For example, it can identify contradictory statements in previous and subsequent content, inconsistencies between conditions and conclusions, and logical conflicts caused by antonyms or negative expressions. It can automatically mark parts that may contain contradictions and prompt users to review and correct them.

[0107] S52, Information omission detection and semantic consistency assurance: According to project management and document standards, establish a key information list (such as important parameters, regulations, time nodes, etc.). Use information extraction technology to extract core entities and events from the document, compare with the list, and identify missing necessary information (key entities, core data or parameters, necessary constraints, required process steps, etc.). Prompt users for possible missing content to ensure the completeness and accuracy of the document.

[0108] By analyzing the semantic flow of the document, the coherence between paragraphs and chapters is evaluated. The semantic similarity and logical association of adjacent content are calculated to identify problems such as sudden changes in the topic, content jumps, or semantic faults. For the detected coherence problems, modification suggestions are provided to help users optimize the structure and content connection of the document and ensure that the entire document is semantically coherent.

[0109] S6, generate a difference analysis report, combine intelligent prompts and visualization tools to intuitively display the comparison results, and assist users in rapid content review and decision optimization; In order to enable users to intuitively and efficiently understand the comparison results and conduct rapid content review and decision optimization, a difference analysis report is generated in this step, combined with intelligent prompts and visualization tools to comprehensively display the comparison results.

[0110] S61, Generation of a difference analysis report: All differences, logical contradictions and information omissions detected in the previous steps are summarized to generate a detailed difference analysis report. The difference analysis report is presented in a structured form, including consistent, partially consistent and inconsistent content, and clearly marks the chapter, paragraph or sentence location where the difference is located. For each difference, provide a detailed description, point out the specific problem, possible impact (such as affecting the project schedule, affecting the allocation of project resources, reducing the feasibility of the work arrangement corresponding to the document, increasing communication and coordination costs, and generating compliance and legal risks, etc.) and recommended solutions. The difference analysis report also includes overall statistics, such as the number of differences, type distribution and severity, to help users fully understand the consistency of the document.

[0111] S62, intelligent prompts and modification suggestions: For each detected problem, semantic analysis and generative technology are used to refer to the industry knowledge base and relevant standards to automatically generate simplified and compliant modification suggestions for users. For content with inconsistent terminology, standard definitions are directly quoted, and logically coherent sentence structures and logically coherent content are given for logically incoherent parts. In the process of users reviewing and selecting modification plans, suggestions will be continuously optimized based on user feedback.

[0112] S63, intuitive display of visualization tools: To improve the user experience, visualization tools are used to intuitively display the comparison results. Using the similarity heat map, the similarity of different parts of the document is presented in the form of color depth, which intuitively reflects the similarity level and helps users quickly identify areas where differences are concentrated. Through the relationship network diagram, the relationship between keywords and topics in the document is displayed to understand the logical structure and semantic association of the content. An interactive interface is provided, and users can directly view the highlighted difference content in the document view and click to obtain detailed information and modification suggestions. It also supports filtering and sorting functions, and users can filter the differences of concern according to their needs to improve review efficiency.

[0113] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A semantic analysis and keyword-driven document consistency comparison method, characterized in that: The steps include: S1. Intelligent analysis and weight distribution through document directory to achieve accurate positioning of key content; S2, use the pre-trained language model to perform deep semantic analysis on the document, generate a multi-dimensional semantic vector, and then perform dimensionality reduction optimization to continue semantic computing; S3, extract the core keywords of the document and assign weights, combine with similar semantic search algorithm, optimize the keyword-driven semantic matching process, and identify potential differences between documents; S4, through the multi-level comparison algorithm, the content comparison at different levels is refined semantically compared; S5. Conduct in-depth modeling of the contextual semantics of the document content to detect logical contradictions and information omissions to ensure the semantic coherence and accuracy of the document in the overall process; S6. Combine intelligent prompts and visualization tools to generate differential analysis results.

2. The semantic parsing and keyword-driven document consistency comparison method according to claim 1, characterized in that: The specific steps of step S1 include the following: S11, directory extraction and parsing: Use text parsing algorithms to automatically identify the directory part in the document, extract chapter titles, numbers, and page number information, form a preliminary directory framework, convert the identified directory framework into a tree structure, and generate a corresponding index table; S12, catalog weighting model and automatic content positioning: The catalog weighting model is used to assign weights to each chapter and sub-chapter. The weights are set according to the relevance of the chapter title and the importance of the chapter content. Based on the catalog weighting model, high-weight chapter content is extracted first, and a local semantic vector matrix is ​​generated. S13, key chapter location and content segment division: quickly locate relevant chapters from the directory framework based on predefined keywords, and directly extract the target chapter content based on the page number information in the directory framework, and then refine the extracted chapter content and divide the content into smaller semantic units.

3. The semantic parsing and keyword-driven document consistency comparison method according to claim 2, characterized in that: The specific steps of step S2 include the following: S21, application of pre-trained language model: use the pre-trained language model BERT to capture the contextual relationship and deep semantic information of the document, input the extracted content fragments into the pre-trained language model, use its encoder part to process the document, and generate the corresponding high-dimensional semantic vector; S22, multi-dimensional semantic vector generation; S23, Dimensionality reduction and optimization: Dimensionality reduction technology is used to reduce the dimension of semantic vectors. During the dimension reduction process, the feature dimensions that have the greatest impact on semantic calculation are retained to ensure that the vector after dimension reduction can still accurately reflect the semantic features of the original document. S24, semantic vector storage and management: the generated semantic vector is stored in a vector database, the vector database supports vector retrieval and similarity calculation, and the vector database indexes the semantic vector to achieve efficient approximate nearest neighbor search.

4. The semantic parsing and keyword-driven document consistency comparison method according to claim 3, characterized in that: The specific steps of step S3 include the following: S31, keyword extraction and weight assignment; S311, keyword extraction: Based on the generated semantic vectors, the pre-trained language model is used to deeply analyze the text and automatically identify the core keywords in the document. The specific steps include: first, the language model is used to perform context analysis on the vocabulary to generate vectors that reflect the semantic relationship; then, combined with the word frequency information, the high-frequency and topic-related words in the full text or key chapters are screened out; then, the occurrence positions of these words are examined in detail; finally, the word parts and semantic relevance are comprehensively considered to finally select the keywords that best represent the core content of the text; S312, keyword weight allocation: according to the importance of the keyword in the document and its influence in different semantic levels, an initial weight is allocated to each keyword. If a keyword appears in multiple important paragraphs or is highly relevant to the key topic, its weight is increased. In the subsequent matching process, the keyword weight is dynamically adjusted according to the matching effect and difference recognition results. S32, similar semantic keyword search; S321, similar semantic space construction: map the extracted keywords to the semantic space, generate the semantic vector representation of the keywords using semantic embedding technology, construct a multidimensional space containing the keyword semantic vectors, capture the potential semantic associations between keywords, and provide a basis for similar keyword search; S322, similarity calculation and matching: by calculating the similarity between the semantic vectors of the keywords, the semantic proximity of the keywords is quantified, and in the compared documents, words or phrases with similar semantics to the target keywords are searched to identify similar semantic contents in different expressions; S323, Dynamic Semantic Expansion and Associative Search: Use word embedding and semantic network to expand the target keywords to synonyms, near synonyms and related words, and combine the usage of keywords in context to perform semantic associative search, capture concepts and topics related to keywords, and expand the matching range; S33, keyword-driven semantic matching optimization; S331, multi-level semantic comparison: Multi-level comparison includes comparison of vocabulary, sentence and paragraph: Lexical comparison: Use word embeddings to semantically match and compare keywords, quickly identifying the consistency or difference between two documents at the terminology and concept level. Sentence-level comparison: Calculate the similarity between sentences through sentence embedding to determine whether the two documents have the same expression of the same topic sentence, and discover differences in expression and potential information missing. Paragraph-level comparison: compare paragraphs as whole semantic units to ensure that the document is logically consistent and thematically coherent at the high-level semantic units, and to detect information additions, deletions, and topic deviations in a timely manner. S332, difference identification and intelligent adjustment: based on the comparison results, automatically identify the difference in keyword matching, mark the inconsistent parts, generate intelligent prompts for the identified differences, point out the specific difference content and location, and dynamically adjust the keyword weight and search strategy based on the comparison feedback; S333, semantic link tracking: Track the semantic trajectory of keywords in documents, record their usage and meaning changes in different contexts, ensure that the meaning of keywords in different documents and different locations is consistent through semantic link tracking, and identify any semantic drift or logical contradictions; S34, combined with similar semantic search algorithm; S341, Optimize the matching process: organically combine keyword extraction, weight allocation and similar semantic search algorithm to form an efficient keyword-driven matching mechanism; S342, Identification of potential differences and optimization of verification process: Use similar semantic search algorithm to deeply compare the potential semantic differences between documents, find the content that cannot be identified by keyword matching alone, classify the identified differences, distinguish terminology differences, expression differences, and content missing types, and provide a basis for subsequent processing; The extracted and matched keywords and similar semantic contents will serve as the basic data for the next step of multi-level semantic comparison.

5. The semantic analysis and keyword-driven document consistency comparison method according to claim 4, characterized in that: The specific steps of step S4 include the following: S41, semantic comparison at the lexical level: using the extracted keywords, perform direct lexical matching in two documents, identify the same words and terms, use word embedding technology to calculate the semantic similarity between words, identify words with different word forms but similar semantics, and expand the matching range; S42, sentence-level semantic comparison: Use the Sentence-BERT model to convert sentences into semantic vectors, capture the overall semantic information of sentences, calculate the semantic similarity of corresponding sentences in two documents, and identify sentences with the same meaning but different expressions and sentences with different meanings; S43, paragraph-level semantic comparison: semantically embed the paragraph content, obtain the overall semantic representation of the paragraph, consider the context and logical structure, compare the semantic vectors of corresponding paragraphs, evaluate the consistency of paragraph content, and find out the information increase or decrease or logical inconsistency. S44, comprehensive comparison and difference identification: The comparison results at the vocabulary, sentence, and paragraph levels are integrated to form an overall assessment of document consistency. Based on the comparison results, the specific vocabulary, sentence, or paragraph where the difference exists is accurately located; S45, comparison result optimization: assign different weights to the comparison results at different levels according to project requirements.

6. The semantic analysis and keyword-driven document consistency comparison method according to claim 5, characterized in that: The specific steps of step S5 include the following: S51, Contextual Semantic Modeling and Logical Contradiction Detection: Use the pre-trained language model to perform global semantic modeling of document content, capture the semantic associations and logical relationships between content through semantic embedding of paragraphs and chapters, and build the semantic network structure of the document; Based on semantic modeling, natural language inference (NLI) technology and logical rules are applied to detect logical contradictions in documents and automatically mark the contradictory parts to prompt users to review and correct them. S52, Information omission detection and semantic consistency assurance: According to project management and document standards, establish a key information list, use information extraction technology to extract core entities and events from the document, compare them with the key information list, identify missing information, and prompt users for possible missing content; By analyzing the semantic flow of the document, evaluating the coherence between paragraphs and chapters, calculating the semantic similarity and logical association of adjacent content, identifying topic mutations, content jumps or semantic faults, and providing modification suggestions for detected coherence issues.

7. The semantic analysis and keyword-driven document consistency comparison method according to claim 6, characterized in that: The specific steps of step S6 include the following: S61, Generation of a difference analysis report: All detected differences, logical contradictions and information omissions are summarized and a detailed difference analysis report is generated. The difference analysis report is presented in a structured form, including consistent, partially consistent and inconsistent content, and the chapter, paragraph or sentence position of the difference is marked. For each difference, a detailed description is provided, indicating the specific problem, the impact of the problem and the recommended solution. The difference analysis report also includes overall statistical data, such as the number of differences, type distribution and severity; S62, intelligent prompts and modification suggestions: For each detected problem, semantic analysis and generative technology are used to refer to the industry knowledge base and relevant standards to automatically generate simplified and compliant modification suggestions for users. For inconsistent terminology, standard definitions are directly quoted, and logically coherent sentence structures and logically coherent content are given for logically incoherent parts. In the process of users reviewing and selecting modification plans, suggestions are continuously optimized based on user feedback; S63, intuitive display of visualization tools: Use visualization tools to intuitively display the comparison results.

Images