Method for automatically identifying difference between domestic and overseas standard files

Abstract

The invention relates to the field of standard comparison analysis, in particular to a method for automatically identifying differences of domestic and overseas standard files. A method for automatically identifying differences of standard files at home and abroad comprises the following four steps of S1, standard file analysis and information extraction, S2, semantic understanding and information integration, S3, standard difference comparison, S4, compliance analysis and suggestion, and in combination with a rule engine and a natural language generation technology, evaluating difference influences and outputting adjustment suggestions. According to the method, comprehensive analysis of the standard file is realized through multi-modal integration and structured processing, and a knowledge base with cross-language and format is constructed; clause association reasoning is realized through deep semantic analysis and knowledge graph construction; according to the method, accurate difference recognition is achieved through hybrid embedding and a dynamic threshold strategy, the accuracy and efficiency of standard comparison are remarkably improved, the manual intervention cost is reduced, and an efficient solution is provided for standard system analysis in international trade and technical cooperation.

Description

Technical Field

[0001] The present invention relates to the field of standard comparison and analysis, and particularly to a method for automatically identifying differences between domestic and foreign standard documents. Background Art

[0002] With the acceleration of the global economic integration process, the differences in standardization systems have become a key challenge in international trade and technological cooperation. Current standard comparison technologies mainly rely on rule-driven methods and text similarity calculations: the former analyzes standard texts through predefined rules or keyword matching, and the latter realizes automated analysis based on word frequency statistics or word embedding models. However, these methods have fundamental limitations in dealing with multi-modal information integration, deep semantic understanding, and complex structure parsing.

[0003] The existing technologies face four core problems: First, the ability to process multi-modal information is insufficient, and it is difficult to comprehensively integrate unstructured content such as tables, charts, and formulas in standard texts through OCR or manual annotation; second, the depth of semantic understanding is limited, and rule matching and lexical co-occurrence cannot capture the implicit logical relationships between articles; third, the ability to parse complex structures is weak, and there is a lack of effective modeling means for multi-level references, cross-references, and other relationships; fourth, the dynamic adaptability is poor, the cost of maintaining the rule base is high, and the threshold setting depends on manual experience, making it difficult to cope with standard updates and diverse expression methods.

[0004] The existing patent CN113408965B discloses a method and system for comparing standards of textile products, which optimizes the process through key index extraction, weight assignment, and redundancy judgment: extracting technical parameters and test methods based on preset rules, setting weights and similarity thresholds in combination with expert experience, and filtering duplicate information. However, this patent still has significant defects: index extraction depends on fixed rules and is difficult to cover the diverse expressions of standards in different regions; the setting of weights and thresholds lacks a data-driven dynamic adjustment mechanism, which is prone to evaluation biases; the processing of non-text content such as charts and formulas still relies on template matching and cannot achieve cross-modal semantic alignment; the lack of in-depth semantic analysis of nested references and cross-dependencies between standard articles affects the comparison accuracy. Therefore, there is an urgent need for a standard comparison method that can break through the bottlenecks of traditional technologies. Summary of the Invention

[0005] The purpose of the present invention is to overcome the above problems and provide a method for automatically identifying differences between domestic and foreign standard documents. To achieve the above purpose, the present invention adopts the following technical solutions: A method for automatically identifying differences between domestic and foreign standard documents, comprising the following steps: Step S1: Standard document parsing and information extraction, automatically parsing domestic and foreign standard documents, and extracting text, charts, formulas, tables, and reference relationship information; Step S2: Semantic Understanding and Information Integration. Through multimodal semantic analysis and knowledge graph technology, a unified knowledge base is constructed to establish cross-modal semantic associations and reference networks; Step S3: Standard Difference Comparison. Based on semantic vector similarity calculation and structured data comparison, identify the differences in technical indicators, parameters, and methods between standards; Step S4: Compliance Analysis and Recommendations. Combine rule engines and natural language generation technology to evaluate the impact of differences and output adjustment recommendations.

[0006] Furthermore, in Step S1, the standard document parsing and information extraction include the following steps: Step S11: Text Information Processing. Construct a structured field extraction rule base based on regular expressions, automatically identify the core chapters in the standard document, segment the text into sentence units, and perform syntactic analysis and entity recognition using natural language processing technology; Step S12: Chart and Formula Information Processing. Use the multimodal model Vision-Parse to identify charts and formulas in the standard text and generate corresponding text descriptions; normalize the formulas and convert them into LaTeX format; Step S13: Table Information Processing. Use the Camlot tool to convert the tables in the standard document into structured data and perform semantic annotation on the cell contents in the tables; Step S14: Annotation and Reference Relationship Processing. Identify the annotations and references in the standard through natural language processing technology, extract the reference targets, construct a reference network between standards, and record multi-level dependency relationships.

[0007] Furthermore, in Step S2, the semantic understanding and information integration include the following steps: Step S21: Semantic Analysis. Extract semantic features and perform correlation analysis on the standard provisions through a language model; Step S22: Information Integration. Use knowledge graph technology to integrate text, charts, formulas, tables, annotations, and reference relationships to construct a unified knowledge base; assign a unique identifier to each information entity to support cross-modal knowledge retrieval; Step S23: Reference Relationship Processing. Use a graph database to store multi-level reference relationships and analyze nested dependencies through graph traversal algorithms.

[0008] Furthermore, in Step S21, the semantic analysis includes the following steps: Step S211: Use the BERT large language model for in-depth semantic analysis. Through text classification, entity recognition, and relationship extraction, understand the deep meaning of the standard provisions; Step S212: Semantically complete the chart description and formula meaning, and use multimodal embedding technology to jointly model text, charts, and formulas to establish cross-modal semantic associations; Step S213: Analyze the logical relationships between standard provisions based on the context window to complete context reasoning.

[0009] Furthermore, in step S3, the standard difference comparison includes the following steps: Step S31: Use a sentence embedding model to convert the standard content into semantic vectors and separately embed key entities; Step S32: Calculate semantic similarity through cosine similarity, and set a threshold to filter out difference points. ; where and respectively represent the semantic vectors of two standard provisions, represents the dot product of vectors, represents the vector modulus, represents the modulus of the vector.

[0010] Step S33: Combine string comparison and semantic analysis to identify differences in objects, indicators, and methods, and output a difference list.

[0011] Furthermore, in step S4, the compliance analysis and suggestions include the following steps; Step S41: Impact assessment, through a rule engine, combined with preset compliance rules, evaluate the impact of standard differences on product compliance; Step S42: Suggestion generation, use a large language model to generate natural language suggestions, and combine the existing data of the enterprise to provide specific adjustment plans; Step S43: Result output, generate a compliance report including difference points, impact assessment, and adjustment suggestions.

[0012] The advantages of the present invention are as follows: Through multimodal information extraction and structured processing technology, the present invention realizes the comprehensive parsing and cross-modal integration of standard documents, develops a dedicated parsing framework to automatically identify and extract text, charts, formulas, tables, and citation relationships in domestic and foreign standard documents, converts unstructured content into structured data in a unified format, establishes a multimodal data association mechanism to ensure semantic alignment between text descriptions and chart formulas, forms a complete standard content knowledge base, supports the processing of standard documents across languages and formats, and provides a standardized and complete data source for in-depth analysis.

[0013] Through deep semantic understanding and knowledge graph construction techniques, the present invention uses a large language model to perform deep semantic analysis on standard content, combines multi-modal embedding techniques to establish cross-modal semantic associations, constructs a knowledge graph to store the graph structure of elements such as standards, articles, and entities, clearly shows the citation relationships between standards and the logical associations between articles, can identify implicit semantic relationships between articles, provides semantic-level support for standard difference comparison, significantly improves the analysis ability of complex standard systems, and realizes multi-level semantic association analysis of standard articles.

[0014] Through a hybrid embedding technique and a dynamic threshold strategy, the present invention constructs a hybrid embedding system to convert standard content into a high-dimensional vector representation, combines a dynamic threshold adjustment strategy to optimize the comparison threshold, automatically calculates the semantic similarity between standards and identifies differences in technical requirements, parameters, and methods, generates a structured comparison report to support the automated comparison of cross-regional and cross-industry standard systems, eliminates the need for manual verification item by item, effectively reduces the labor intervention cost of standard analysis, provides efficient support for enterprises to quickly adjust technical solutions, and realizes the accurate identification of standard differences and compliance assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings constituting a part of this application are used to provide a further understanding of this application, making other features, objectives, and advantages of this application more obvious. The schematic embodiments and descriptions thereof of this application are used to explain this application and do not constitute an improper limitation of this application.

[0016] In the drawings: Figure 1 It is a flowchart of an automatic method for identifying differences between domestic and foreign standard documents in Embodiment 1. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0018] The present invention will be introduced in detail and specifically below through specific embodiments to better understand the present invention. However, the following embodiments do not limit the protection scope of the present invention.

[0019] Embodiment:

[0020] As Figure 1 shown, an automatic method for identifying differences between domestic and foreign standard documents includes the following steps: Step S1: Standard Document Parsing and Information Extraction. Automatically parse domestic and international standard documents to extract text, charts, formulas, tables, and reference relationship information; The standard documents include international standards, EU standards, national standards, industry standards, etc., laying a foundation for subsequent standard difference comparison.

[0021] Step S2: Semantic Understanding and Information Integration. Through multimodal semantic analysis and knowledge graph technology, construct a unified knowledge base, and establish cross-modal semantic associations and reference networks; Step S3: Standard Difference Comparison. Based on semantic vector similarity calculation and structured data comparison, identify differences in technical indicators, parameters, and methods between standards; Step S4: Compliance Analysis and Recommendations. Combine rule engines and natural language generation technology to evaluate the impact of differences and output adjustment recommendations.

[0022] Furthermore, in Step S1, the standard document parsing and information extraction include the following steps: Step S11: Text Information Processing. Construct a structured field extraction rule base based on regular expressions, automatically identify the core chapters in the standard document, split the text into sentence units, and use natural language processing technology for syntactic analysis and entity recognition; Step S12: Chart and Formula Information Processing. Use the multimodal model Vision-Parse to identify charts and formulas in the standard text and generate corresponding text descriptions; normalize the formulas and convert them into LaTeX format; Adopt the Vision-Parse multimodal parsing system, which includes an object detection and formula parsing module. The object detection module uses the Faster R-CNN architecture to locate the chart area and identifies the chart type through the position-sensitive ROI pooling layer. The formula parsing module constructs a structure tree through 7 convolutional layers, bidirectional LSTM, and CRF layers, and converts the formula into LaTeX format. Cross-modal alignment uses the CLIP model, and the contrast loss function is: ; Among them, represents the temperature parameter, represents the visual feature vector of the chart, represents the text description vector of the corresponding chart, represents the negative sample text vector.

[0023] Step S13: Table Information Processing. Use the Camlot tool to convert the tables in the standard document into structured data, and perform semantic annotation on the cell content in the table. The semantic annotation includes test methods, technical indicators, etc.

[0024] Step S14: Processing of Annotations and Citation Relationships: Identify the annotations and citations in the standard through natural language processing techniques, extract the citation targets, construct a citation network among standards, and record multi-level dependency relationships.

[0025] Further, in step S2, the semantic understanding and information integration include the following steps: Step S21: Semantic analysis, extracting semantic features and performing correlation analysis on the standard provisions through a language model; Step S22: Information integration, using knowledge graph technology to integrate text, charts, formulas, tables, annotations and citation relationships to construct a unified knowledge base; assign a unique identifier to each information entity to support cross-modal knowledge retrieval; Step S23: Processing of citation relationships, storing multi-level citation relationships using a graph database, and analyzing nested dependencies through a graph traversal algorithm.

[0026] Use the Neo4j graph database to store the standard citation network. The node types include standard documents, provisions, and technical entities, and the relationship types are inter-standard citations, provision dependencies, and entity definitions. Graph traversal uses the breadth-first search (BFS) algorithm to analyze multi-level citations, such as finding all the standards directly or indirectly cited by the ISO 9001 standard. Optimization strategies include caching common paths and using bidirectional BFS to accelerate the shortest path query to ensure rapid parsing of the complex dependencies among standards. Among them, the standard document node stores metadata such as the standard name, number, and release date, the provision node stores the specific clause content and the standard to which it belongs, and the technical entity node stores key technical concepts such as parameters, methods, and requirements; the inter-standard citation relationship represents a direct citation of one standard to another, the provision dependency relationship represents the hierarchical dependency between provisions, and the entity definition relationship represents the definition of a technical entity by a provision. The BFS algorithm traverses the standard citation chain by expanding nodes layer by layer to ensure coverage of the deep nested relationships in the standard system.

[0027] Further, in step S21, the semantic analysis includes the following steps: Step S211: Use the BERT large language model for in-depth semantic analysis, and understand the deep meaning of the standard provisions through text classification, entity recognition, and relationship extraction. The deep meaning includes technical requirements, compliance conditions, etc.

[0028] Adopt an improved BERT-Large model for pre-training on the standard text corpus. Entity recognition uses a BERT-CRF joint model, and relationship extraction constructs a dependency tree based on the multi-head attention mechanism. The domain adaptation layer fuses domain features through a dynamic weight matrix: ; Among them, represents the dynamic weight matrix (between 0 and 1), represents the input weight matrix, represents the bias vector, represents the current layer hidden state, represents the base model hidden state.

[0029] Step S212: Semantically complete the chart description and formula meaning, and jointly model text, charts, and formulas using multi-modal embedding technology to establish cross-modal semantic associations; The cross-modal semantic fusion system adopts a CLIP extended architecture, ViT-B / 16 image encoder + BERT-Base text encoder. The formula embedding model processes character-level and structure-level features through a hierarchical Transformer. The alignment loss function is: ; where, represents the temperature parameter, represents the image feature vector, represents the corresponding text vector, represents the training data distribution.

[0030] Step S213: Analyze the logical relationship between standard provisions based on the context window to complete context reasoning.

[0031] Furthermore, in step S3, the standard difference comparison includes the following steps: Step S31: Use a sentence embedding model to convert the standard content into semantic vectors and separately embed key entities; Adopt the Sentence-BERT model to fine-tune on the basis of BERT-Base to generate 768-dimensional semantic vectors. The training parameters of the TransE entity embedding model are: dimension 200, learning rate 0.001, loss function: ; where, represents the head entity embedding vector, represents the relationship embedding vector, represents the tail entity embedding vector, represents the set of positive sample triples, represents the set of negative sample triples, represents the margin parameter, set to 1.0, represents the distance function.

[0032] Step S32: Calculate semantic similarity through cosine similarity and set a threshold to filter out difference points, ; where, and respectively represent the semantic vectors of two standard provisions, represents the dot product of vectors, represents a vector modulus, represents the modulus of the vector.

[0033] If the threshold is set to , when the calculated is obtained, it indicates that the semantic similarity of the two texts is high and the difference is small; when , it shows that the semantic similarity is low and it belongs to the difference points that need attention.

[0034] Calculating the cosine similarity of the semantic vectors of the standard provisions, comparing the results with the preset threshold, and screening out the difference points below the threshold solve the technical problems of insufficient semantic understanding and difficulty in accurately capturing deep semantic differences in traditional standard document difference recognition, and avoid misjudgment or missed judgment caused by relying solely on surface text or rule matching. By quantifying the semantic similarity degree, it can more accurately identify the semantic differences in the standard provisions, improve the accuracy and efficiency of difference recognition, lay a foundation for the subsequent output of a reliable difference list and compliance analysis, and ensure the effective detection of the deep semantic differences of the standard documents under the integration of multi-modal information.

[0035] Step S33: Combine string comparison and semantic analysis to identify the differences in objects, indicators, and methods, and output a difference list.

[0036] Construct a three-dimensional difference detection system, including surface difference, structural difference, and semantic difference detection. The surface difference uses the Levenshtein distance to calculate the text similarity: ; wherein, represents the original string, represents the comparison string, represents the edit distance, and represent the string lengths.

[0037] The text similarity directly reflects the similarity degree of the text surface. The larger the value, the smaller the edit distance of the two strings relative to the string length, the higher the text surface similarity, and the smaller the difference; the smaller the value, the lower the text surface similarity and the larger the difference. This calculation method provides a quantitative basis for identifying the surface differences in the standard documents, making the surface difference detection more accurate, and then jointly constituting a complete three-dimensional difference detection system with the structural difference and semantic difference detections, improving the accuracy and comprehensiveness of the overall difference recognition, and laying a foundation for the output of a reliable difference list.

[0038] The structural differences analyze the changes in chapter order through XML tree edit distance, and the costs of insertion / deletion / replacement operations are 1 / 1 / 2 respectively. Semantic differences use Datalog rule reasoning to identify implicit conflicts.

[0039] Further, in step S4, the compliance analysis and suggestions include the following steps; Step S41: Impact assessment. Through the rule engine, combined with the preset compliance rules, evaluate the impact of the standard differences on product compliance; For example, if the new standard requires a certain parameter of the product to be less than 10, and the parameter of the enterprise product is 12, the system will mark it as "high impact".

[0040] Step S42: Suggestion generation. Use a large language model to generate natural language suggestions, and combine the existing data of the enterprise to provide specific adjustment plans. The existing data includes product parameters, supply chain information, etc.

[0041] For example, the system may suggest "adjust the product parameter from 12 to 9 to meet the new standard".

[0042] Step S43: Result output. Generate a compliance report containing the difference points, impact assessment and adjustment suggestions to help the enterprise quickly understand the problem and take actions.

[0043] In this embodiment, through multi-modal information extraction and structured processing, standard documents at home and abroad are automatically parsed, such as text, charts, formulas, tables and citation relationships in formats such as PDF and DOCX. The large language model and multi-modal embedding technology are used to deeply understand the semantics and extract features of the standard content, generating a unified high-dimensional vector representation. Subsequently, combined with knowledge graph technology and embedding technology, the complex citation relationships and technical requirements between standards are accurately compared, the differences between domestic and foreign standards are automatically identified, and a clear comparison report is generated. Compared with the traditional methods that rely on rule matching or simple text similarity calculation, this embodiment can comprehensively capture the multi-modal information and its deep semantic relationships of standard documents, significantly improve the accuracy and efficiency of comparison, without the need for manual extraction and comparison of standard content one by one, effectively reducing the manual intervention and maintenance costs in the analysis of cross-language, cross-region and cross-industry standard systems, and is particularly suitable for the standard comparison requirements in international trade, technical cooperation and export product development, providing efficient support for enterprises to quickly adjust the technology chain, supply chain and production chain.

[0044] The specific embodiments of the present invention have been described in detail above, but they are only examples, and the present invention is not equivalent to the specific embodiments described above. For those skilled in the art, any equivalent modifications and substitutions to the present invention are also within the scope of the present invention. Therefore, all equivalent transformations and modifications made without departing from the spirit and scope of the present invention should be covered within the scope of the present invention.

Claims

1. An automatic recognition method for differences in domestic and foreign standard documents, characterized in that: It includes the following steps: Step S1: Standard document parsing and information extraction. Automatically parse domestic and foreign standard documents, and extract text, charts, formulas, tables, and reference relationship information; Step S2: Semantic understanding and information integration. Through multimodal semantic analysis and knowledge graph technology, build a unified knowledge base, and establish cross-modal semantic associations and reference networks; Step S3: Standard difference comparison. Based on semantic vector similarity calculation and structured data comparison, identify differences in technical indicators, parameters, and methods between standards; Step S4: Compliance analysis and suggestions. Combine rule engines and natural language generation technology to evaluate the impact of differences and output adjustment suggestions.

2. The automatic recognition method for differences between domestic and foreign standard documents according to claim 1, characterized in that: In step S1, the standard document parsing and information extraction include the following steps: Step S11: Text information processing. Build a structured field extraction rule base based on regular expressions, automatically identify the core chapters in the standard document, split the text into sentence units, and use natural language processing technology for syntactic analysis and entity recognition; Step S12: Chart and formula information processing. Use the multimodal model Vision-Parse to identify charts and formulas in the standard text, and generate corresponding text descriptions; Normalize the formulas and convert them into LaTeX format; Step S13: Table information processing. Use the Camlot tool to convert the tables in the standard document into structured data, and perform semantic annotation on the cell content in the tables; Step S14: Annotation and reference relationship processing. Identify annotations and references in the standard through natural language processing technology, extract reference targets, build a reference network between standards, and record multi-level dependency relationships.

3. The automatic recognition method for differences between domestic and foreign standard documents according to claim 2, characterized in that: In step S2, the semantic understanding and information integration include the following steps: Step S21: Semantic analysis. Extract semantic features and perform correlation analysis on standard provisions through a language model; Step S22: Information integration. Use knowledge graph technology to integrate text, charts, formulas, tables, annotations, and reference relationships to build a unified knowledge base; Assign a unique identifier to each information entity to support cross-modal knowledge retrieval; Step S23: Reference relationship processing. Use a graph database to store multi-level reference relationships, and analyze nested dependencies through graph traversal algorithms.

4. The automatic recognition method for differences between domestic and foreign standard documents according to claim 3, wherein: In step S21, the semantic analysis includes the following steps: Step S211: Use the BERT large language model for in-depth semantic analysis. Through text classification, entity recognition, and relationship extraction, understand the deep meaning of standard provisions; Step S212: Semantically complete the chart description and formula meaning. Adopt multimodal embedding technology to jointly model text, charts, and formulas, and establish cross-modal semantic associations; Step S213: Analyze the logical relationship between standard provisions based on the context window to complete context reasoning.

5. The automatic recognition method for differences between domestic and foreign standard documents according to claim 4, characterized in that: In step S3, the standard difference comparison includes the following steps: Step S31: Use a sentence embedding model to convert the standard content into semantic vectors, and separately embed key entities; Step S32: Calculate semantic similarity through cosine similarity, and set a threshold to filter out different points. ; Among them, and respectively represent the semantic vectors of two standard clauses, represents the dot product of vectors, represents the vector modulus, represents the modulus of the vector; Step S33: Combine string comparison and semantic analysis to identify differences in objects, indicators, and methods, and output a list of differences.

6. The automatic recognition method for differences between domestic and foreign standard documents according to claim 5, characterized in that: In step S4, the compliance analysis and suggestions include the following steps; Step S41: Impact assessment. Through the rules engine, combined with the preset compliance rules, evaluate the impact of the standard differences on product compliance; Step S42: Recommendation generation. Use the large language model to generate natural language recommendations and provide specific adjustment plans in combination with the existing data of the enterprise; Step S43: Result output. Generate a compliance report containing the difference points, impact assessment, and adjustment recommendations.

Images