Real-time data analysis and visualization method and system in big data environment

Abstract

The present invention provides a real-time data analysis and visualization method and system in a big data environment, which relates to the field of data processing technology, including obtaining and preprocessing documents to be processed; performing hierarchical structure analysis and semantic understanding analysis; establishing semantic index mapping; receiving natural language processing instructions input by users; performing intention understanding and semantic parsing; matching semantic understanding tasks; processing corresponding semantic fragment units; obtaining target documents and visualizing them. The method can realize intelligent processing and analysis of different types of documents and improve data processing efficiency.

Description

Technical Field

[0001] The present invention relates to data processing technology, and in particular to a real-time data analysis and visualization method and system in a big data environment. Background Art

[0002] With the rapid development of informatization and digitalization, enterprises and organizations have accumulated a massive amount of unstructured document data. These documents are of various types, including contracts, papers, news, and resumes, which contain important business value and knowledge information. However, how to quickly and accurately extract valuable information from massive heterogeneous documents and realize intelligent processing and analysis of documents has become a key issue that needs to be solved in the era of big data.

[0003] Existing document processing systems often separate functions such as document preprocessing, semantic understanding, and information retrieval, making it difficult to achieve end-to-end intelligent processing. At the same time, the system has a high threshold for use and requires professional technicians to configure and maintain it, which is not conducive to daily use by business personnel. Therefore, it is of great practical significance to develop a technical solution that can achieve intelligent document processing, deep semantic understanding, and interactive visualization. Summary of the invention

[0004] The embodiments of the present invention provide a real-time data analysis and visualization method and system in a big data environment, which can solve the problems in the prior art.

[0005] According to a first aspect of the embodiments of the present invention,

[0006] Provides real-time data analysis and visualization methods in a big data environment, including:

[0007] Obtaining a document to be processed, performing denoising and normalization processing on the document to be processed, and according to a preset document type determination rule, extracting format features, keyword features, and layout structure features of the document, using a classification model based on a combination of rules and machine learning to determine the type of the document to be processed, and using corresponding preprocessing parameters to perform preprocessing on different types of documents to be processed to obtain a preprocessed document, wherein the type of the document to be processed includes at least one of a contract, a paper, news, and a resume;

[0008] Performing hierarchical structural analysis on the preprocessed document, extracting structured information from the document, and constructing a structured document model based on the structured information; performing semantic understanding analysis on the preprocessed document using a pre-trained semantic understanding analysis model to obtain document semantic information, wherein the semantic understanding analysis model adopts a neural network framework that combines a pre-trained language model based on knowledge enhancement with an attention mechanism, and is constructed by introducing domain ontology knowledge for fine-tuning; establishing a semantic index mapping based on the structured document model and the document semantic information, wherein the semantic index mapping includes a mapping relationship between document content and semantic information and structural information;

[0009] Receive automated processing instructions input by the user in natural language, use a natural language understanding model based on deep learning to understand the intention and semantically parse the instructions, extract semantic elements in the instructions, match a predefined semantic understanding task library according to the semantic elements, determine one or more semantic understanding tasks corresponding to the instructions, match one or more semantic fragment units of corresponding granularity from the semantic index mapping according to the granularity of the semantic understanding task; process the matched semantic fragment units for the semantic understanding task to obtain a target document, and visualize the target document.

[0010] Performing hierarchical structural analysis on the preprocessed document, extracting structured information in the document, and constructing a structured document model based on the structured information, including:

[0011] The preprocessed document is subjected to structural feature extraction to obtain layout areas including text area, image area, table area and formula area; the text area is further analyzed using the connected domain analysis algorithm, and text objects of different granularities are extracted through adaptive threshold segmentation and morphological filtering processing; the image area is detected and identified using the target detection model based on regional convolutional neural network to obtain the location and content information of the image object; the table area is analyzed using a method based on table structure analysis, and the structured representation of the table is obtained through table line extraction, cell positioning and content recognition; the formula area is extracted from the formula area using a method based on symbolic analysis and syntax parsing to obtain the document layout structure features including text, images, tables and formulas;

[0012] Different layout areas are abstracted as nodes of the graph model, and the node attributes contain the document layout structure characteristics of different layout areas; according to the document layout structure characteristics between different layout areas, corresponding edges are added to the graph model, and the attributes of the edges represent the logical relationship type between different layout areas; the graph model is updated through iterative optimization until the graph model converges; the spectral clustering algorithm is used to cluster the nodes of the graph model to obtain the clustered logical structure units; the hierarchical and sequential relationships between the clustered logical structure units are obtained through topological sorting to obtain the document logical structure tree;

[0013] Combined with the document logical structure tree, the structured feature vector representation of each logical structure unit is extracted, wherein the geometric and format features including position, font, font size and alignment are extracted for the text-type logical structure units, and the geometric features and content features including position, size and shape are extracted for the non-text-type logical structure units; the document logical structure tree and the structured feature vector representation are matched according to the predefined structure template rules, and the structured feature vectors that meet the predefined structure template rules are combined to form a structure framework that meets the document structure specifications; a document ontology knowledge base for a specific field is constructed, and a structured document model is constructed based on the document ontology knowledge base for a specific field and the structure framework that meets the document structure specifications.

[0014] The pre-processed document is subjected to semantic understanding analysis using a pre-trained semantic understanding analysis model to obtain document semantic information, including:

[0015] Performing semantic understanding analysis on the preprocessed document using a pre-trained semantic understanding analysis model, wherein the semantic understanding analysis model normalizes the professional terms and abbreviations in the document to obtain the semantic feature vectors of each structural element of the document, and obtaining the semantic relevance between different structural elements by calculating the similarity between the semantic feature vectors; processing the preprocessed document using a pre-trained semantic understanding analysis model to obtain word semantic feature vectors, sentence semantic feature vectors and document semantic feature vectors, wherein the semantic understanding analysis model includes a word vector model, a bidirectional recurrent neural network and a hierarchical attention model; generating semantic feature vectors of the document structural elements based on the word semantic feature vectors, the sentence semantic feature vectors and the document semantic feature vectors;

[0016] A cosine similarity calculation method is used to map the semantic feature vectors of two different document structure elements into a vector space of the same dimension, and the cosine value of the angle between the semantic feature vectors of the two document structure elements is calculated, and the smaller the cosine value of the angle, the higher the semantic similarity; a Euclidean distance calculation method is used to calculate the Euclidean distance between the semantic feature vectors of the two document structure elements, and the smaller the Euclidean distance, the higher the semantic similarity; a semantic similarity calculation model is constructed using a twin network, and the twin network includes two sub-networks with shared parameters, and nonlinear transformations are performed on the semantic feature vectors of the two document structure elements respectively, and a cosine similarity layer is used at the top layer to calculate the similarity score of the twin network of the semantic feature vectors of the two document structure elements;

[0017] A semantic similarity calculation model is constructed using a matching network, and the semantic feature vectors of the document structure elements to be matched are matched with the sample vectors in the support set through an attention mechanism, a matching vector is generated, and the similarity score of the matching network between the semantic feature vectors of the document structure elements to be matched and the sample vectors in the support set is calculated through a fully connected layer; the semantic relevance of the two document structure elements is calculated based on the cosine similarity, the Euclidean distance, the similarity score of the twin network and the similarity score of the matching network; the semantic feature vectors of the document structure elements are traversed to calculate the semantic relevance between the semantic feature vectors of any two different document structure elements.

[0018] The pre-processed documents are processed using a pre-trained semantic understanding analysis model to obtain structural element semantic feature vectors including word semantic feature vectors, sentence semantic feature vectors, and document semantic feature vectors, including:

[0019] For the professional terms and abbreviations in the preprocessed documents, a domain-adaptive word vector enhancement method is used. The domain text corpus is used to perform secondary training on the word vector model. The word vector model is adjusted through a supervised probability prediction task. The adjusted word vector model is used to embed the words in the preprocessed documents to obtain the word semantic feature vector; for the unregistered words in the preprocessed documents, a character-based word embedding method is used to obtain the word semantic feature vector;

[0020] Based on the word semantic feature vector, a bidirectional recurrent neural network is used to encode the sentence to obtain the sentence semantic feature vector, wherein the bidirectional recurrent neural network includes a long short-term memory network LSTM and a gated recurrent unit GRU; in the process of encoding the sentence, the word semantic feature vector in the sentence is encoded from left to right and from right to left respectively; an attention mechanism is superimposed on the basis of the bidirectional recurrent neural network; the output of the bidirectional recurrent neural network is weightedly summed after attention to obtain the sentence semantic feature vector;

[0021] Use the fully connected layer in the hierarchical attention model to map the sentence semantic vectors in the paragraph to the attention space, use the normalization function to normalize the vectors in the attention space to obtain the attention weight distribution, multiply the sentence semantic vectors in the paragraph with the corresponding attention weights and sum them to obtain the paragraph semantic vector; use the hierarchical attention model to integrate the paragraph semantic vectors to obtain the chapter semantic vector; use the attention mechanism to perform weighted fusion on the chapter semantic vectors to generate the document semantic feature vector.

[0022] According to the structured document model and the document semantic information, a semantic index mapping is established, wherein the semantic index mapping contains a mapping relationship between document content and semantic information and structural information, including:

[0023] According to the structured document model and semantic relevance, a sequence annotation model based on conditional random field (CRF) is used, combined with predefined semantic segmentation rules, to automatically segment the document into semantic segments to obtain semantic segment units; a semantic index mapping is established for the structural features, semantic features and knowledge features of the semantic segment units, wherein the semantic index mapping includes a mapping relationship between document content and semantic information and structural information; the semantic relevance between different structural element sequences in the structured document model is calculated to obtain a document structural element semantic relevance matrix;

[0024] The document structure element sequence is used as the observation sequence of the sequence annotation model based on the conditional random field (CRF), and the semantic fragment label is used as the hidden state sequence; the transfer features between the observation sequence and the hidden state sequence, as well as the emission features of the observation sequence are calculated; wherein the transfer features between the observation sequence and the hidden state sequence include: the semantic relevance between adjacent structure elements and the compatibility between the semantic features of the structure elements and the semantic fragment labels; the emission features of the observation sequence include: the hierarchical depth, context structure and format structure features of the structure elements; the transfer features and the emission features are combined through the feature function to construct a conditional random field model; the Viterbi algorithm is used to decode the conditional random field model to obtain the optimal semantic fragment label sequence as the semantic fragment unit;

[0025] For the semantic fragment unit, its structural features, semantic features and knowledge features are extracted; wherein the structural features of the semantic fragment unit include: the position, context structure and format of the semantic fragment unit in the document; the semantic features of the semantic fragment unit include: the keywords, themes and sentiment tendency of the semantic fragment unit; the knowledge features of the semantic fragment unit include: the entities, events and concepts contained in the semantic fragment unit; an inverted index of the semantic index mapping is constructed, with the semantic features, structural features and knowledge features of the semantic fragment unit as index fields, and the semantic fragment unit number and position information as an inverted list; based on the inverted index, a vectorized representation of the semantic fragment unit is constructed to establish a semantic index mapping, wherein the semantic index mapping includes a mapping relationship between the content of the semantic fragment unit and its structural features, semantic features and knowledge features.

[0026] For the semantic understanding task, the matched semantic fragment units are processed to obtain the target document, including:

[0027] For the semantic understanding task, the matching semantic fragment units are preprocessed to obtain a bag-of-words representation, and the latent Dirichlet allocation (LDA) model is used to estimate parameters through the Gibbs sampling algorithm to obtain the topic distribution vector and topic keyword list of the semantic fragment unit, and the topic distribution vector and the keyword list are combined to form the topic representation of the semantic fragment unit; the bag-of-words representation is converted into the input format of the deep bidirectional encoder representation BERT model, and the context vector of each word in the semantic fragment unit is calculated through the forward propagation of the pre-trained BERT model, and the vector corresponding to the special tag is extracted and combined with the attention mechanism and pooling operation to generate the deep semantic representation of the semantic fragment unit;

[0028] The topic representation and deep semantic representation of the semantic fragment unit are fused by any one of connection fusion, attention fusion and gated fusion to obtain the fused semantic representation of the semantic fragment unit; the domain knowledge base is introduced to enhance the knowledge of the semantic fragment unit of the fused semantic representation, and the links between the entities in the semantic fragment unit and the standard entities in the domain knowledge base are established by constructing a domain entity dictionary and using co-occurrence statistics. Based on the links, the entity attributes and the relationship between entities are extracted from the semantic fragment unit by using template-based and machine learning methods to obtain the knowledge triples in the fragment;

[0029] The reasoning method based on logical rules and knowledge representation learning is used to utilize the domain knowledge base to perform knowledge reasoning on the knowledge triples in the fragment to obtain the semantic information after knowledge enhancement; any one of the connection fusion, attention fusion, and gated fusion methods is used to integrate the semantic information after knowledge enhancement with the fused semantic representation of the semantic fragment unit to obtain the semantic fragment unit representation after knowledge enhancement; according to the requirements of the semantic understanding task, the semantic fragment unit representation after knowledge enhancement is aggregated, sorted, filtered, and filled into the corresponding position of the document generation template to generate the target document.

[0030] According to the requirements of the semantic understanding task, the semantic fragment unit representation after knowledge enhancement is aggregated, sorted, filtered, and filled into the corresponding position of the document generation template to generate the target document, including:

[0031] The semantic fragment unit representation after knowledge enhancement is subjected to topic aggregation, entity aggregation and relationship aggregation. The topic aggregation uses the topic distribution similarity to calculate the topic similarity between the semantic fragment unit representations after knowledge enhancement, and the semantic fragment unit representations after knowledge enhancement with a topic similarity higher than a preset threshold are aggregated into the same topic class; the entity aggregation uses the same entity inclusion to determine whether the semantic fragment unit representation after knowledge enhancement contains the same entity, and the semantic fragment unit representations after knowledge enhancement containing the same entity are aggregated into the same entity class; the relationship aggregation uses the relationship type consistency to determine whether the relationship type between the semantic fragment unit representations after knowledge enhancement is consistent, and the semantic fragment unit representations after knowledge enhancement with the same relationship type are aggregated into the same relationship class;

[0032] For each topic in the topic aggregation results, the semantic fragment unit and task relevance scoring and sorting method is used to calculate the relevance score of each knowledge-enhanced semantic fragment unit representation and the semantic understanding task, and the knowledge-enhanced semantic fragment unit representations are sorted from high to low according to the task relevance score; for each entity class in the entity aggregation results, the importance evaluation scoring and sorting method is used to calculate the entity importance score contained in each knowledge-enhanced semantic fragment unit representation, and the knowledge-enhanced semantic fragment unit representations are sorted from high to low according to the entity importance score; for each relationship class in the relationship aggregation results, the logical order between the knowledge-enhanced semantic fragment unit representations is calculated based on the logical order relationship sorting method, and the knowledge-enhanced semantic fragment unit representations are arranged in sequence according to the logical order;

[0033] In the sorting results of topic aggregation results, the semantic fragment unit representations enhanced with redundant knowledge whose similarity is higher than a preset threshold in each topic class are eliminated by using the similarity calculation method between semantic fragment unit representations; in the sorting results of entity aggregation results, the semantic fragment unit representations enhanced with knowledge irrelevant to the entity in each entity class are eliminated by using the keyword matching method; in the sorting results of relationship aggregation results, the semantic fragment unit representations enhanced with knowledge whose readability and coherence are lower than a preset quality threshold in each relationship class are eliminated by using the readability and coherence quality assessment method to obtain optimized semantic fragment units; a document generation template that meets the requirements of the semantic understanding task is designed, and the optimized semantic fragment units are filled into the corresponding positions in the document generation template. When filling, the semantic fragment units are matched with the template positions for semantic similarity, the length of the semantic fragment units is trimmed and the grammatical style is adjusted and adapted according to the template specifications, and transition words inserted between the semantic fragment units are generated by a template-based or deep learning method to obtain the target document.

[0034] According to a second aspect of the embodiments of the present invention,

[0035] Provides real-time data analysis and visualization systems in a big data environment, including:

[0036] The first unit is used to obtain a document to be processed, perform denoising and normalization processing on the document to be processed, and according to a preset document type determination rule, extract the format features, keyword features and layout structure features of the document, and use a classification model based on a combination of rules and machine learning to determine the type of the document to be processed, and use corresponding preprocessing parameters to preprocess different types of documents to obtain preprocessed documents, wherein the type of the document to be processed includes at least one of a contract, a paper, news, and a resume;

[0037] The second unit is used to perform hierarchical structural analysis on the preprocessed document, extract structured information in the document, and construct a structured document model based on the structured information; perform semantic understanding analysis on the preprocessed document using a pre-trained semantic understanding analysis model to obtain document semantic information, wherein the semantic understanding analysis model adopts a neural network framework that combines a pre-trained language model based on knowledge enhancement with an attention mechanism, and is constructed after fine-tuning by introducing domain ontology knowledge; establish a semantic index mapping based on the structured document model and the document semantic information, wherein the semantic index mapping includes a mapping relationship between document content and semantic information and structural information;

[0038] The third unit is used to receive automated processing instructions input by the user in natural language, use a natural language understanding model based on deep learning to understand the intention and semantically parse the instructions, extract semantic elements in the instructions, match a predefined semantic understanding task library according to the semantic elements, determine one or more semantic understanding tasks corresponding to the instructions, match one or more semantic fragment units of corresponding granularity from the semantic index mapping according to the granularity of the semantic understanding task; process the matched semantic fragment units for the semantic understanding task to obtain a target document, and visualize the target document.

[0039] According to a third aspect of the embodiments of the present invention,

[0040] An electronic device is provided, comprising:

[0041] processor;

[0042] a memory for storing processor-executable instructions;

[0043] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0044] A fourth aspect of the embodiments of the present invention is:

[0045] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0046] The beneficial effects of this application are as follows:

[0047] 1. Improve the accuracy and efficiency of document processing:

[0048] The present invention performs denoising and normalization on the documents to be processed, and uses a classification model based on rules and machine learning to determine the document type, and uses corresponding preprocessing parameters for different types. This method can effectively improve the accuracy of document classification and provide clearer and more standardized input for subsequent processing, thereby improving the efficiency of overall document processing.

[0049] 2. Enhance the semantic understanding of documents:

[0050] The present invention uses a pre-trained semantic understanding analysis model to perform semantic understanding analysis on documents. The model adopts a neural network framework that combines a knowledge-enhanced pre-trained language model with an attention mechanism, and introduces domain ontology knowledge for fine-tuning. This method can significantly improve the ability to deeply understand the semantics of document content, making subsequent document processing more accurate and intelligent.

[0051] 3. Enable flexible natural language interaction and visualization:

[0052] The present invention supports users to input processing instructions in natural language, and performs intention understanding and semantic analysis through a natural language understanding model based on deep learning, thus realizing a more intuitive and flexible human-computer interaction method. At the same time, by establishing semantic index mapping and matching semantic fragment units, user needs can be accurately located, and the processing results can be presented in a visual manner, greatly improving the user experience and the practicality of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flowchart of a real-time data analysis and visualization method in a big data environment according to an embodiment of the present invention;

[0054] Figure 2 It is a structural diagram of a real-time data analysis and visualization system in a big data environment according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0057] Figure 1 Schematic diagram of the process of real-time data analysis and visualization method in a big data environment according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0058] S101. Obtaining a document to be processed, performing denoising and normalization processing on the document to be processed, and according to a preset document type determination rule, extracting format features, keyword features, and layout structure features of the document, using a classification model based on rules and machine learning to determine the type of the document to be processed, and using corresponding preprocessing parameters for different types of documents to be processed to obtain a preprocessed document, wherein the type of the document to be processed includes at least one of a contract, a paper, news, and a resume;

[0059] S102. Performing hierarchical structural analysis on the preprocessed document, extracting structured information from the document, and constructing a structured document model based on the structured information; performing semantic understanding analysis on the preprocessed document using a pre-trained semantic understanding analysis model to obtain document semantic information, wherein the semantic understanding analysis model is constructed by introducing domain ontology knowledge and fine-tuning a neural network framework that combines a pre-trained language model based on knowledge enhancement with an attention mechanism; establishing a semantic index mapping based on the structured document model and the document semantic information, wherein the semantic index mapping includes a mapping relationship between document content and semantic information and structural information;

[0060] S103. Receive automated processing instructions input by the user in natural language, use a natural language understanding model based on deep learning to understand the intent and semantically parse the instructions, extract semantic elements in the instructions, match a predefined semantic understanding task library according to the semantic elements, determine one or more semantic understanding tasks corresponding to the instructions, match one or more semantic fragment units of corresponding granularity from the semantic index mapping according to the granularity of the semantic understanding task; process the matched semantic fragment units for the semantic understanding task to obtain a target document, and visualize the target document.

[0061] This embodiment provides a real-time data analysis and visualization method in a big data environment. The method first obtains the document to be processed, and performs denoising and normalization processing on it. The denoising process includes removing irrelevant information in the document, such as headers, footers, watermarks, etc.; the normalization process converts the document into a standard format, such as PDF or TXT format.

[0062] Then, according to the preset document type determination rules, the type of the document to be processed is determined by extracting the format features, keyword features and layout structure features of the document, using a classification model based on rules and machine learning. Specifically, the format features include the file format and page layout of the document; the keyword features are obtained by extracting high-frequency words and professional terms in the document; the layout structure features analyze the chapter division and paragraph organization of the document. The classification model uses a combination of decision trees and support vector machines (SVM) to comprehensively analyze the extracted features and output the probability distribution of the document type.

[0063] The document types supported by this method include contracts, papers, news, resumes, etc. For different types of documents, corresponding preprocessing parameters are used for processing. For example, for contract documents, the focus is on extracting information such as contract terms and signatories; for papers, the focus is on analyzing abstracts, keywords, chapter structures, etc. Through type-related preprocessing, key information in documents can be extracted more specifically.

[0064] Next, the preprocessed documents are subjected to hierarchical structural analysis to extract the structured information in the documents. Specifically, a rule-based method is used to identify the document's title, paragraph, list and other structural elements, and to construct a hierarchical structure tree of the document. For example, for an academic paper, the main chapters such as "Abstract", "Introduction", "Methods", "Results", and "Discussion" can be identified, as well as the sub-chapters and paragraphs under each chapter. Based on the extracted structured information, a structured document model is constructed, which represents the hierarchical relationship of the document in a tree structure.

[0065] At the same time, the pre-trained semantic understanding analysis model is used to perform semantic understanding analysis on the pre-processed documents to obtain the document semantic information. The semantic understanding analysis model adopts a neural network framework that combines a pre-trained language model based on knowledge enhancement with an attention mechanism. Specifically, based on pre-trained language models such as BERT, domain ontology knowledge is introduced for fine-tuning. For example, for documents in the legal field, domain knowledge such as legal terminology dictionaries and case knowledge bases can be introduced. Through the multi-head self-attention mechanism, the model can capture long-distance dependencies in the document and improve the accuracy of semantic understanding.

[0066] The output of semantic understanding analysis includes document topic classification, key information extraction, sentiment tendency, etc. For example, for a commercial contract, the model can identify the contract type (such as sales contract, labor contract, etc.), extract key information (such as the parties to the contract, contract amount, fulfillment period, etc.), and analyze the tendency of the contract terms (such as whether it is beneficial to the buyer / seller, etc.).

[0067] Based on the structured document model and document semantic information, a semantic index mapping is established. The mapping contains the correspondence between the document content and the semantic information and structural information. Specifically, a unique identifier is assigned to each structural unit of the document (such as paragraphs and sentences), and its corresponding semantic label and structural position information are associated. For example, for a clause in a contract document, its semantic index may include: clause ID, chapter to which it belongs, clause type (such as liability for breach of contract), key entities (such as the defaulting party, compensation amount), and other information.

[0068] After receiving the automatic processing instructions input by the user in natural language, the natural language understanding model based on deep learning is used to understand the intention and semantically parse the instructions. The model adopts a bidirectional long short-term memory network (Bi-LSTM) structure and combines it with an attention mechanism to effectively capture the key information in the instructions. By preprocessing the instructions such as word segmentation, part-of-speech tagging, and dependency syntax analysis, the model extracts semantic elements in the instructions, such as operation type, target object, and conditional restrictions.

[0069] For example, for the user input command "Extract the customer name and contract amount from all sales contracts signed in 2022", the model can identify key semantic elements: operation type (extraction), target object (customer name, contract amount), and conditional restrictions (signed in 2022, sales contract).

[0070] According to the extracted semantic elements, the system matches the predefined semantic understanding task library and determines one or more semantic understanding tasks corresponding to the instruction. The semantic understanding task library contains a variety of predefined task types, such as information extraction, text classification, question answering, etc. For the above example, the system may match the two tasks of "contract information extraction" and "time filtering".

[0071] Next, according to the granularity of the semantic understanding task, one or more semantic fragment units of the corresponding granularity are matched from the semantic index mapping. For example, for the contract information extraction task, the system will locate the relevant paragraphs or sentences in the contract document that contain customer information and amount information; for the time filtering task, it is necessary to retrieve the contract signing date information.

[0072] For the matching semantic understanding tasks, the system processes the corresponding semantic fragment units. During the processing, the corresponding algorithm module is called according to the task type. For example, for information extraction tasks, the named entity recognition (NER) model can be used to extract customer names and regular expressions can be used to match contract amounts; for time filtering tasks, rule matching or time expression recognition models can be used to determine whether the contract was signed in 2022.

[0073] Finally, the processing results are integrated into the target document and visualized. The visualization method can be flexibly selected according to the data characteristics and user needs, such as tables, charts, relationship maps, etc. For the above example, the system may generate a table containing customer names, contract amounts, and signing dates, and display the distribution of contract amounts for different customers in the form of a bar chart.

[0074] The beneficial effects of this method include the following three aspects:

[0075] 1. Improved the intelligence and automation level of document processing. By combining the document classification method based on rules and machine learning, as well as the semantic understanding model based on knowledge enhancement, it can accurately identify different types of documents and conduct in-depth semantic analysis, greatly reducing manual intervention and improving processing efficiency.

[0076] 2. Enhanced flexibility and adaptability of document analysis. The semantic index mapping mechanism is used to associate document content with semantic and structural information, so that the system can quickly locate relevant information based on the user's natural language instructions and achieve flexible document retrieval and analysis. This method is suitable for various types of documents and diverse analysis needs.

[0077] 3. The document analysis results are presented intuitively. Through visualization technology, complex document analysis results are converted into easy-to-understand charts or graphs, helping users quickly grasp key information and improve decision-making efficiency. At the same time, the visualization results also facilitate further interactive exploration and analysis.

[0078] In an optional implementation, the preprocessed document is subjected to hierarchical structure analysis, structured information in the document is extracted, and a structured document model is constructed based on the structured information, including:

[0079] The preprocessed document is subjected to structural feature extraction to obtain layout areas including text area, image area, table area and formula area; the text area is further analyzed using the connected domain analysis algorithm, and text objects of different granularities are extracted through adaptive threshold segmentation and morphological filtering processing; the image area is detected and identified using the target detection model based on regional convolutional neural network to obtain the location and content information of the image object; the table area is analyzed using a method based on table structure analysis, and the structured representation of the table is obtained through table line extraction, cell positioning and content recognition; the formula area is extracted from the formula area using a method based on symbolic analysis and syntax parsing to obtain the document layout structure features including text, images, tables and formulas;

[0080] Different layout areas are abstracted as nodes of the graph model, and the node attributes contain the document layout structure characteristics of different layout areas; according to the document layout structure characteristics between different layout areas, corresponding edges are added to the graph model, and the attributes of the edges represent the logical relationship type between different layout areas; the graph model is updated through iterative optimization until the graph model converges; the spectral clustering algorithm is used to cluster the nodes of the graph model to obtain the clustered logical structure units; the hierarchical and sequential relationships between the clustered logical structure units are obtained through topological sorting to obtain the document logical structure tree;

[0081] Combined with the document logical structure tree, the structured feature vector representation of each logical structure unit is extracted, wherein the geometric and format features including position, font, font size and alignment are extracted for the text-type logical structure units, and the geometric features and content features including position, size and shape are extracted for the non-text-type logical structure units; the document logical structure tree and the structured feature vector representation are matched according to the predefined structure template rules, and the structured feature vectors that meet the predefined structure template rules are combined to form a structure framework that meets the document structure specifications; a document ontology knowledge base for a specific field is constructed, and a structured document model is constructed based on the document ontology knowledge base for a specific field and the structure framework that meets the document structure specifications.

[0082] This embodiment provides a method for performing hierarchical structure analysis on a preprocessed document, extracting structured information, and constructing a structured document model. The method comprises the following steps:

[0083] First, the preprocessed document is subjected to structured feature extraction to obtain layout areas including text area, image area, table area and formula area. Specifically, a layout segmentation algorithm based on deep learning, such as Mask R-CNN, can be used to perform pixel-level semantic segmentation on the document image and divide the document into different layout areas. For example, for an academic paper containing text, pictures, tables and formulas, the corresponding mask images of the text area, image area, table area and formula area can be obtained.

[0084] Next, different layout areas are further analyzed. For the text area, the connected domain analysis algorithm is used for processing. First, the text area is binarized by adaptive threshold segmentation, then the morphological opening operation is applied to remove noise, and finally the connected domain is marked to obtain text objects of different granularities, such as paragraphs, sentences, and words. For example, for a piece of text "This is a paragraph. It contains two sentences.", 1 paragraph object, 2 sentence objects, and 7 word objects can be obtained.

[0085] For image regions, we use a target detection model based on a regional convolutional neural network for detection and recognition. We can use the Faster R-CNN model to detect targets in image regions and obtain the bounding box and category information of the image objects. We then use convolutional neural networks such as ResNet to extract features and recognize the content of the detected image objects. For example, for a picture containing a cat and a dog, we can get two target boxes and the category labels of "cat" and "dog".

[0086] For the table area, we use a method based on table structure analysis to process it. First, we extract the table lines through Hough transform and other methods, then locate the cells based on the table line information, and finally perform text recognition on each cell. For example, for a 3x3 table, we can get 9 cell objects, each of which contains location and content information.

[0087] For the formula area, a method based on symbol analysis and syntax parsing is used to extract the structured representation of the formula. First, the formula image is symbolically recognized to obtain the sequence of mathematical symbols contained in the formula. Then, according to the grammatical rules of mathematical expressions, the symbol sequence is parsed to obtain the tree structure representation of the formula. For example, for the formula "a^2 + b^2 =c^2", a syntax tree containing addition and square operations can be obtained.

[0088] Through the above steps, we can get the document layout structure features including text, images, tables and formulas. These features include the location, size, content and other information of each layout area.

[0089] Next, different layout areas are abstracted into nodes of the graph model. The attributes of each node contain the structural features of the corresponding layout area. For example, the attributes of text nodes include text content, font, font size, etc.; the attributes of image nodes include image position, size, category, etc. According to the spatial relationship and semantic association between different layout areas, corresponding edges are added to the graph model. The attributes of the edges represent the type of logical relationship between different layout areas, such as "include", "reference", etc.

[0090] Next, the graph model is updated through iterative optimization. The graph model can be optimized using a graph convolutional network (GCN), and the feature representations of nodes and edges can be updated through multiple rounds of message passing until the graph model converges. For example, for a graph model containing 10 nodes, 5 rounds of iterative optimization can be performed, and the features of all nodes and edges are updated in each round of iteration.

[0091] Then, the spectral clustering algorithm is used to cluster the nodes of the optimized graph model to obtain the clustered logical structure units. Specifically, the Laplacian matrix of the graph is constructed, its eigenvectors are calculated, the first k eigenvectors are selected to form the feature matrix, and the row vectors of the feature matrix are clustered using K-means. For example, for a graph model containing 20 nodes, 5 logical structure units can be clustered.

[0092] Next, the hierarchical and sequential relationships between the clustered logical structure units are obtained through topological sorting to obtain the document logical structure tree. First, a directed acyclic graph between the logical structure units is constructed, and then the graph is topologically sorted to obtain a linear sequence of the logical structure units. Finally, a tree structure is constructed based on the linear sequence. For example, for 5 logical structure units, a 3-layer tree structure can be obtained.

[0093] Combined with the document logical structure tree, the structured feature vector representation of each logical structure unit is extracted. For text-type logical structure units, geometric and typographic features including position, font, font size and alignment are extracted. For example, for a paragraph unit, its coordinates in the page, the name of the font used, the font size, left alignment / center alignment / right alignment and other features can be extracted. For non-text-type logical structure units, geometric features and content features including position, size and shape are extracted. For example, for an image unit, its coordinates in the page, aspect ratio, area ratio, as well as image category, theme and other features can be extracted.

[0094] The document logical structure tree and the structured feature vector representation are matched according to the predefined structure template rules. The predefined structure template rules may include the structural composition of different types of documents, the feature requirements of each component, etc. For example, for academic papers, a structure template including the title, abstract, keywords, text, references, etc. may be defined. Through the matching process, the structured feature vectors that meet the predefined structure template rules are combined to form a structure framework that meets the document structure specifications.

[0095] Finally, construct a document ontology knowledge base for a specific domain. The ontology knowledge base contains domain concepts, relationships, attributes and other knowledge. For example, for the field of medical literature, an ontology knowledge base containing concepts such as diseases, symptoms, and treatment methods can be constructed. Based on the document ontology knowledge base for a specific domain and a structural framework that conforms to the document structure specification, a structured document model is constructed. Specifically, the various components in the structural framework are associated with the concepts in the ontology knowledge base to form a semantically rich structured representation.

[0096] The beneficial effects of this embodiment include:

[0097] 1. Improved the accuracy and robustness of document structure analysis. Through multimodal feature extraction and graph model optimization, it can effectively capture the complex relationship between different types of content in the document, improving the accuracy of structural analysis. At the same time, the deep learning-based method has strong generalization ability and can adapt to documents of different types and formats.

[0098] 2. Achieved deep semantic understanding of documents. By introducing the domain ontology knowledge base and combining the document structure with domain knowledge, not only the surface structure of the document is obtained, but also the deep semantic information is mined, laying the foundation for subsequent intelligent applications.

[0099] 3. Provides a flexible and scalable document modeling method. With a modular design, each processing module can be flexibly adjusted or replaced according to actual needs. At the same time, by updating the domain ontology knowledge base, it can be easily expanded to new fields and application scenarios.

[0100] In an optional implementation, the pre-trained semantic understanding analysis model performs semantic understanding analysis on the pre-processed document to obtain document semantic information, including:

[0101] Performing semantic understanding analysis on the preprocessed document using a pre-trained semantic understanding analysis model, wherein the semantic understanding analysis model normalizes the professional terms and abbreviations in the document to obtain the semantic feature vectors of each structural element of the document, and obtaining the semantic relevance between different structural elements by calculating the similarity between the semantic feature vectors; processing the preprocessed document using a pre-trained semantic understanding analysis model to obtain word semantic feature vectors, sentence semantic feature vectors and document semantic feature vectors, wherein the semantic understanding analysis model includes a word vector model, a bidirectional recurrent neural network and a hierarchical attention model; generating semantic feature vectors of the document structural elements based on the word semantic feature vectors, the sentence semantic feature vectors and the document semantic feature vectors;

[0102] A cosine similarity calculation method is used to map the semantic feature vectors of two different document structure elements into a vector space of the same dimension, and the cosine value of the angle between the semantic feature vectors of the two document structure elements is calculated, and the smaller the cosine value of the angle, the higher the semantic similarity; a Euclidean distance calculation method is used to calculate the Euclidean distance between the semantic feature vectors of the two document structure elements, and the smaller the Euclidean distance, the higher the semantic similarity; a semantic similarity calculation model is constructed using a twin network, and the twin network includes two sub-networks with shared parameters, and nonlinear transformations are performed on the semantic feature vectors of the two document structure elements respectively, and a cosine similarity layer is used at the top layer to calculate the similarity score of the twin network of the semantic feature vectors of the two document structure elements;

[0103] A semantic similarity calculation model is constructed using a matching network, and the semantic feature vectors of the document structure elements to be matched are matched with the sample vectors in the support set through an attention mechanism, a matching vector is generated, and the similarity score of the matching network between the semantic feature vectors of the document structure elements to be matched and the sample vectors in the support set is calculated through a fully connected layer; the semantic relevance of the two document structure elements is calculated based on the cosine similarity, the Euclidean distance, the similarity score of the twin network and the similarity score of the matching network; the semantic feature vectors of the document structure elements are traversed to calculate the semantic relevance between the semantic feature vectors of any two different document structure elements.

[0104] The specific implementation is as follows:

[0105] First, the document is preprocessed, including word segmentation, stop word removal, part-of-speech tagging, etc. Then, the pre-trained semantic understanding analysis model is used to perform semantic understanding analysis on the preprocessed document to obtain the document semantic information.

[0106] The semantic understanding analysis model includes a word vector model, a bidirectional recurrent neural network, and a hierarchical attention model. The word vector model maps words in a document into low-dimensional dense vectors to capture the semantic information of the words. The bidirectional recurrent neural network can encode contextual information and obtain sentence-level semantic representations. The hierarchical attention model adaptively focuses on important information through word-level attention and sentence-level attention to generate semantic representations of documents.

[0107] Specifically, we first use the word vector model to convert each word in the document into a word vector. For professional terms and abbreviations, we use the method of synonym replacement and expansion to normalize the representation. For example, "artificial intelligence" can be replaced by "AI", "machine learning" can be expanded to "machine learning algorithm", etc. Then, the word vector sequence is input into the bidirectional LSTM network to obtain the semantic representation of the sentence. Then, through the hierarchical attention mechanism, the importance weights of words and sentences are calculated to obtain the semantic representation of the document. Finally, the semantic feature vectors of three levels, word, sentence and document, are obtained.

[0108] Next, based on the obtained semantic feature vectors, the semantic feature vectors of each structural element of the document (such as chapters, paragraphs, etc.) are generated. For each structural element, the semantic feature vectors of the words and sentences it contains are weighted averaged to obtain the semantic feature vector of the structural element.

[0109] Then the semantic relevance between different structural elements is calculated. First, the cosine similarity method is used to map the semantic feature vectors of two structural elements to the vector space of the same dimension, and the cosine value of the angle between the vectors is calculated. The closer the cosine value is to 1, the higher the semantic similarity. For example, vector a=(1,2,3), b=(2,4,6), the cosine similarity is 0.9959, indicating that the semantics are very similar.

[0110] At the same time, the Euclidean distance calculation method is used to calculate the Euclidean distance between two semantic feature vectors. The smaller the distance, the higher the semantic similarity. For example, the Euclidean distance of vector a=(1,2,3) and b=(2,4,6) is 3.7417, which is relatively small, indicating semantic similarity.

[0111] In addition, a semantic similarity calculation model is constructed using a twin network. The network contains two sub-networks with shared parameters, which perform nonlinear transformations on the semantic feature vectors of the two structural elements respectively. The cosine similarity layer is used at the top layer to calculate the similarity score of the transformed vectors. For example, after the input vectors a and b are nonlinearly transformed to obtain a' and b', the cosine similarity of a' and b' is calculated as the final score.

[0112] You can also use the matching network to build a semantic similarity calculation model. The network matches the structural elements to be matched with the samples in the support set through the attention mechanism. Specifically, first, the attention weights are calculated for the vector to be matched and each sample vector in the support set, and then the support set is weighted and summed to obtain the matching vector. Finally, the similarity score between the vector to be matched and the matching vector is calculated through the fully connected layer.

[0113] Based on the similarity results obtained by the above methods, the final semantic relevance of the two structural elements can be calculated by weighted average. For example, the weights can be set as: cosine similarity 0.3, Euclidean distance 0.2, twin network 0.3, matching network 0.2. Assume that the similarities obtained by the four methods are 0.9, 0.8, 0.85, 0.9,

[0114] The final semantic relevance is 0.9*0.3+0.8*0.2+0.85*0.3+0.9*0.2=0.87.

[0115] Finally, all structural elements are traversed, the semantic relevance between any two different structural elements is calculated, and the semantic relationship network of the document structural elements is obtained.

[0116] The beneficial effects of this method include:

[0117] 1. Through the comprehensive application of multiple semantic similarity calculation methods, the accuracy and robustness of semantic relevance calculation are improved.

[0118] 2. The use of pre-trained semantic understanding analysis models can effectively extract the deep semantic information of documents, overcoming the limitations of traditional keyword-based matching.

[0119] 3. Normalization of professional terms and abbreviations solves the semantic inconsistency problem caused by different expressions and improves the accuracy of semantic analysis.

[0120] In an optional implementation, the pre-trained semantic understanding analysis model processes the pre-processed document to obtain a structural element semantic feature vector including a word semantic feature vector, a sentence semantic feature vector and a document semantic feature vector, including:

[0121] For the professional terms and abbreviations in the preprocessed documents, a domain-adaptive word vector enhancement method is used. The domain text corpus is used to perform secondary training on the word vector model. The word vector model is adjusted through a supervised probability prediction task. The adjusted word vector model is used to embed the words in the preprocessed documents to obtain the word semantic feature vector; for the unregistered words in the preprocessed documents, a character-based word embedding method is used to obtain the word semantic feature vector;

[0122] Based on the word semantic feature vector, a bidirectional recurrent neural network is used to encode the sentence to obtain the sentence semantic feature vector, wherein the bidirectional recurrent neural network includes a long short-term memory network LSTM and a gated recurrent unit GRU; in the process of encoding the sentence, the word semantic feature vector in the sentence is encoded from left to right and from right to left respectively; an attention mechanism is superimposed on the basis of the bidirectional recurrent neural network; the output of the bidirectional recurrent neural network is weightedly summed after attention to obtain the sentence semantic feature vector;

[0123] Use the fully connected layer in the hierarchical attention model to map the sentence semantic vectors in the paragraph to the attention space, use the normalization function to normalize the vectors in the attention space to obtain the attention weight distribution, multiply the sentence semantic vectors in the paragraph with the corresponding attention weights and sum them to obtain the paragraph semantic vector; use the hierarchical attention model to integrate the paragraph semantic vectors to obtain the chapter semantic vector; use the attention mechanism to perform weighted fusion on the chapter semantic vectors to generate the document semantic feature vector.

[0124] The specific implementation is as follows:

[0125] First, the input document is preprocessed, including word segmentation, stop word removal, part-of-speech tagging, etc. Then, the preprocessed document is processed using a pre-trained semantic understanding analysis model to obtain a structural element semantic feature vector including a word semantic feature vector, a sentence semantic feature vector, and a document semantic feature vector.

[0126] For professional terms and abbreviations in the preprocessed documents, a domain-adaptive word vector enhancement method is used. Specifically, a large-scale domain text corpus is used to perform secondary training on the pre-trained word vector model. During the secondary training process, the parameters of the word vector model are adjusted through supervised probability prediction tasks. For example, a professional dictionary can be used to construct synonym pairs as positive samples, and random word pairs as negative samples, and the training model can predict whether two words are synonyms. The adjusted word vector model can better capture the semantic information of words in the domain. The adjusted word vector model is used to embed the words in the document to obtain the semantic feature vector of the words.

[0127] For unknown words in the document, a character-based word embedding method is used. First, the words are split into character sequences, then the character-level CNN or RNN is used to encode the character sequences, and finally the vector representation of the words is obtained through the pooling layer. This method can handle new and rare words and improve the robustness of the model.

[0128] Based on the word semantic feature vector, a bidirectional recurrent neural network is used to encode the sentence to obtain the sentence semantic feature vector. The bidirectional recurrent neural network used includes a long short-term memory network LSTM and a gated recurrent unit GRU. During the encoding process, the word semantic feature vectors in the sentence are processed from left to right and from right to left respectively. Specifically, for each word vector in the sentence, the forward LSTM / GRU reads in and updates the hidden state from left to right, and the reverse LSTM / GRU reads in and updates the hidden state from right to left. In this way, the context information of the word in the sentence can be captured.

[0129] The attention mechanism is superimposed on the bidirectional recurrent neural network. First, the forward and reverse hidden states of the bidirectional LSTM / GRU are concatenated to obtain the encoding vector of each word. Then, a feedforward neural network is used to calculate the attention score of each word, and the score is normalized by the softmax function to obtain the attention weight. Finally, the encoding vector of the word is multiplied by the corresponding attention weight and summed to obtain the semantic feature vector of the sentence. This method can adaptively weight the contribution of different words according to the semantic importance of the sentence.

[0130] For paragraphs consisting of multiple sentences, a hierarchical attention model is used for processing. First, a fully connected layer is used to map the sentence semantic vector to the attention space. Then, the vector in the attention space is normalized using the softmax function to obtain the attention weight of each sentence. Next, the sentence semantic vector is multiplied by the corresponding attention weight and summed to obtain the paragraph semantic vector. This method can adaptively weight the contribution of different sentences according to their importance in the paragraph.

[0131] For chapters consisting of multiple paragraphs, the hierarchical attention model is also used for processing. First, a fully connected layer is used to map the paragraph semantic vector to the attention space. Then, the softmax function is used to calculate the attention weight of each paragraph. Finally, the paragraph semantic vector is multiplied by the corresponding attention weight and summed to obtain the chapter semantic vector.

[0132] Finally, the chapter semantic vectors are weighted and fused through the attention mechanism to generate the document semantic feature vector. Specifically, a feedforward neural network is used to calculate the attention score of each chapter vector, and the attention weight is normalized by the softmax function. Then, the chapter vector is multiplied by the corresponding attention weight and summed to obtain the final document semantic feature vector.

[0133] The following is a specific data processing example:

[0134] Suppose there is a technical document about artificial intelligence, which contains multiple chapters, paragraphs and sentences. First, the document is segmented and preprocessed. For professional terms such as "deep learning" and "convolutional neural network" that appear in the document, a domain-adaptive word vector enhancement method is used. For example, a large amount of text corpus in the field of artificial intelligence is collected to fine-tune the pre-trained word vector model. During the fine-tuning process, "deep learning" and "neural network" can be used as positive sample pairs, and "deep learning" and "database" can be used as negative sample pairs to train the model to predict the relevance of word pairs. After fine-tuning, the word vectors of "deep learning" and "neural network" will be closer, while the distance from the word vector of "database" will be farther.

[0135] For new words such as "GPT-3" that appear in the document, a character-based word embedding method is used. "GPT-3" is split into a character sequence ['G', 'P', 'T', '-', '3'], the character sequence is encoded using a character-level CNN, and then the vector representation of the word is obtained through maximum pooling.

[0136] Next, use bidirectional LSTM to encode the sentence. For example, for the sentence "Deep learning is one of the core technologies of artificial intelligence", first obtain the word vector representation of each word. Then, the forward LSTM reads the word vector from left to right and updates the hidden state, and the reverse LSTM reads the word vector from right to left and updates the hidden state. The hidden states of the forward and reverse LSTMs are concatenated to obtain the encoding vector of each word.

[0137] On this basis, an attention mechanism is added. A single-layer feedforward neural network is used to calculate the attention score of each word. For example, "deep learning" and "core technology" may get higher attention scores. The scores are normalized by the softmax function to get the attention weight. Finally, the encoding vector of the word is multiplied by the corresponding attention weight and summed to get the semantic feature vector of the sentence.

[0138] For paragraphs containing multiple sentences, a hierarchical attention model is used. First, the semantic vector of each sentence is mapped to the attention space through a fully connected layer. Then the softmax function is used to calculate the attention weight of each sentence, and important sentences will get higher weights. The sentence semantic vector is multiplied by the corresponding attention weight and summed to obtain the paragraph semantic vector.

[0139] The same method is used to process chapters containing multiple paragraphs and entire documents containing multiple chapters. Finally, the semantic feature vector of the document is obtained, which comprehensively considers the semantic information and importance of words, sentences, paragraphs and chapters in the document.

[0140] The beneficial effects of this method include:

[0141] 1. Through domain-adaptive word vector enhancement and character-based word embedding, the model's ability to handle professional terms, abbreviations, and new words is improved, so that the generated semantic feature vectors more accurately reflect the professional content of the document.

[0142] 2. The use of bidirectional recurrent neural network and multi-level attention mechanism can effectively capture the long-distance dependencies and hierarchical semantic structures in the document, thereby generating richer and more representative semantic feature vectors.

[0143] 3. By integrating semantic information at different levels in an adaptive weighted manner, the generated document semantic feature vector can highlight key content and ignore irrelevant information, thereby improving the effectiveness of subsequent text analysis tasks.

[0144] In an optional implementation, the structured document model and the document semantic information establish a semantic index mapping, wherein the semantic index mapping contains a mapping relationship between document content and semantic information and structural information, including:

[0145] According to the structured document model and semantic relevance, a sequence annotation model based on conditional random field (CRF) is used, combined with predefined semantic segmentation rules, to automatically segment the document into semantic segments to obtain semantic segment units; a semantic index mapping is established for the structural features, semantic features and knowledge features of the semantic segment units, wherein the semantic index mapping includes a mapping relationship between document content and semantic information and structural information; the semantic relevance between different structural element sequences in the structured document model is calculated to obtain a document structural element semantic relevance matrix;

[0146] The document structure element sequence is used as the observation sequence of the sequence annotation model based on the conditional random field (CRF), and the semantic fragment label is used as the hidden state sequence; the transfer features between the observation sequence and the hidden state sequence, as well as the emission features of the observation sequence are calculated; wherein the transfer features between the observation sequence and the hidden state sequence include: the semantic relevance between adjacent structure elements and the compatibility between the semantic features of the structure elements and the semantic fragment labels; the emission features of the observation sequence include: the hierarchical depth, context structure and format structure features of the structure elements; the transfer features and the emission features are combined through the feature function to construct a conditional random field model; the Viterbi algorithm is used to decode the conditional random field model to obtain the optimal semantic fragment label sequence as the semantic fragment unit;

[0147] For the semantic fragment unit, its structural features, semantic features and knowledge features are extracted; wherein the structural features of the semantic fragment unit include: the position, context structure and format of the semantic fragment unit in the document; the semantic features of the semantic fragment unit include: the keywords, themes and sentiment tendency of the semantic fragment unit; the knowledge features of the semantic fragment unit include: the entities, events and concepts contained in the semantic fragment unit; an inverted index of the semantic index mapping is constructed, with the semantic features, structural features and knowledge features of the semantic fragment unit as index fields, and the semantic fragment unit number and position information as an inverted list; based on the inverted index, a vectorized representation of the semantic fragment unit is constructed to establish a semantic index mapping, wherein the semantic index mapping includes a mapping relationship between the content of the semantic fragment unit and its structural features, semantic features and knowledge features.

[0148] This embodiment provides a method for establishing a semantic index mapping based on a structured document model and document semantic information. The method comprises the following steps:

[0149] Firstly, according to the structured document model and semantic relevance, a sequence annotation model based on conditional random field (CRF) is used to automatically divide the document into semantic segments. Specifically, the sequence of document structure elements is used as the observation sequence of the CRF model, and the semantic segment labels are used as the hidden state sequence. The transfer features between the observation sequence and the hidden state sequence, as well as the emission features of the observation sequence, are calculated.

[0150] Transfer features include the semantic relevance between adjacent structural elements and the compatibility between the semantic features of structural elements and semantic fragment labels. For example, for two adjacent paragraphs, their semantic similarity is calculated; for the subject word of a paragraph and the "background" semantic fragment label, their semantic compatibility is calculated. Emission features include the hierarchical depth, context structure and format structure features of structural elements. For example, the depth of a paragraph in the document hierarchy is 2, the previous level is "Chapter 1", and it has a bold format, etc.

[0151] These features are combined through feature functions to construct a conditional random field model. Then the Viterbi algorithm is used to decode the CRF model to obtain the optimal semantic segment label sequence as the semantic segment unit. For example, for a scientific document, the possible semantic segment unit sequence is: abstract, introduction, method, experiment, conclusion, etc.

[0152] Next, for the semantic fragment units obtained by division, their structural features, semantic features and knowledge features are extracted. Structural features include the position, context structure, format, etc. of the semantic fragment unit in the document. For example, a semantic fragment unit is located at 20%-30% of the document, the previous text is "Chapter 2", the following text is "Section 2.1", and it has an indentation format. Semantic features include keywords, topics, emotional tendencies, etc. of the semantic fragment unit. For example, the extracted keywords are "machine learning" and "neural network", the topic is "deep learning algorithm", and the emotional tendency is neutral. Knowledge features include entities, events, concepts, etc. contained in the semantic fragment unit. For example, the entity "convolutional neural network", the event "ImageNet competition", and the concept "image classification" are identified.

[0153] Then, an inverted index of the semantic index mapping is constructed, with the semantic features, structural features and knowledge features of the semantic fragment unit as index fields, and the semantic fragment unit number and position information as the inverted list. For example, the following index can be established:

[0154] Keyword index: "machine learning" -> {fragment 1: [20%, 30%], fragment 4: [60%, 65%]};

[0155] Subject index: "Deep Learning Algorithms" -> {Fragment 2: [35%, 45%], Fragment 5: [70%, 80%]};

[0156] Entity Index: "Convolutional Neural Network" -> {Fragment 3: [50%, 55%], Fragment 6: [85%, 90%]};

[0157] Finally, on the basis of the inverted index, the vectorized representation of the semantic fragment unit is constructed to establish a semantic index mapping. Word embedding, sentence embedding and other technologies can be used to map the content, structural features, semantic features and knowledge features of the semantic fragment unit to the same vector space. For example, for a certain semantic fragment unit, a 300-dimensional vector representation can be obtained, in which the first 100 dimensions represent the content semantics, the middle 100 dimensions represent the structural features, and the last 100 dimensions represent the knowledge features. In this way, a mapping relationship between the content of the semantic fragment unit and its structural features, semantic features and knowledge features is established.

[0158] Through the above steps, the semantic index mapping based on the structured document model and document semantic information is completed. This method has the following beneficial effects:

[0159] 1. Improved the accuracy and comprehensiveness of document semantic understanding. By combining the structural information and semantic information of the document, it can more accurately identify the semantic units of the document and capture the overall semantics of the document.

[0160] 2. Enhanced accuracy and semantic relevance of document retrieval. Semantic fragment-based indexing can support more fine-grained retrieval, taking into account multiple dimensions such as structure, semantics, and knowledge, thus improving the relevance of retrieval results.

[0161] 3. It provides a rich semantic foundation for intelligent document processing. The established semantic index mapping contains the multi-dimensional features of the document, providing strong support for subsequent intelligent applications such as document summarization, question answering, and recommendation.

[0162] In an optional implementation, the semantic understanding task processes the matched semantic segment units to obtain a target document, including:

[0163] For the semantic understanding task, the matching semantic fragment units are preprocessed to obtain a bag-of-words representation, and the latent Dirichlet allocation (LDA) model is used to estimate parameters through the Gibbs sampling algorithm to obtain the topic distribution vector and topic keyword list of the semantic fragment unit, and the topic distribution vector and the keyword list are combined to form the topic representation of the semantic fragment unit; the bag-of-words representation is converted into the input format of the deep bidirectional encoder representation BERT model, and the context vector of each word in the semantic fragment unit is calculated through the forward propagation of the pre-trained BERT model, and the vector corresponding to the special tag is extracted and combined with the attention mechanism and pooling operation to generate the deep semantic representation of the semantic fragment unit;

[0164] The topic representation and deep semantic representation of the semantic fragment unit are fused by any one of connection fusion, attention fusion and gated fusion to obtain the fused semantic representation of the semantic fragment unit; the domain knowledge base is introduced to enhance the knowledge of the semantic fragment unit of the fused semantic representation, and the links between the entities in the semantic fragment unit and the standard entities in the domain knowledge base are established by constructing a domain entity dictionary and using co-occurrence statistics. Based on the links, the entity attributes and the relationship between entities are extracted from the semantic fragment unit by using template-based and machine learning methods to obtain the knowledge triples in the fragment;

[0165] The reasoning method based on logical rules and knowledge representation learning is used to utilize the domain knowledge base to perform knowledge reasoning on the knowledge triples in the fragment to obtain the semantic information after knowledge enhancement; any one of the connection fusion, attention fusion, and gated fusion methods is used to integrate the semantic information after knowledge enhancement with the fused semantic representation of the semantic fragment unit to obtain the semantic fragment unit representation after knowledge enhancement; according to the requirements of the semantic understanding task, the semantic fragment unit representation after knowledge enhancement is aggregated, sorted, filtered, and filled into the corresponding position of the document generation template to generate the target document.

[0166] For the semantic understanding task, the matching semantic segment units are first preprocessed to obtain a bag-of-words representation. Specifically, natural language processing techniques such as word segmentation, stop word removal, and part-of-speech tagging can be used to convert the semantic segment units into word sequences, and the frequency of occurrence of each word is counted to form a bag-of-words vector. For example, for the semantic segment "Today's weather is sunny, suitable for going out to play", after preprocessing, a bag-of-words representation {"Today": 1, "Weather": 1, "Sunny": 1, "Suitable": 1, "Going out": 1, "Play": 1} can be obtained.

[0167] Next, the Latent Dirichlet Allocation (LDA) model is used to estimate parameters through the Gibbs sampling algorithm to obtain the topic distribution vector and topic keyword list of the semantic segment unit. The LDA model assumes that the document is a mixture of multiple topics, and each topic is represented by a multinomial distribution of words. Through iterative sampling, the distribution probability of each semantic segment unit on different topics and the keyword list of each topic can be obtained. For example, assuming that the number of topics is set to 3, the topic distribution vector [0.6, 0.3, 0.1] may be obtained, indicating that the semantic segment unit has a 60% probability of belonging to topic 1, a 30% probability of belonging to topic 2, and a 10% probability of belonging to topic 3. At the same time, the keyword list of topic 1 is obtained ["weather", "sunny", "outing"], the keyword list of topic 2 is ["suitable", "travel", "activity"], and the keyword list of topic 3 is ["today", "time", "date"].

[0168] Then, the topic distribution vector is combined with the keyword list to form the topic representation of the semantic fragment unit. The topic distribution vector and the top-k keyword vectors of each topic can be connected by vector concatenation to obtain a richer topic semantic representation. For example, a topic representation vector such as [0.6, 0.3, 0.1, "weather", "sunny", "outing", "suitable", "travel", "activity", "today", "time", "date"] can be obtained.

[0169] At the same time, the bag-of-words representation is converted to the input format of the Deep Bidirectional Encoder Representation (BERT) model. The BERT model requires the input to be a sequence of words, so the bag-of-words representation needs to be restored to the original text sequence and special tags [CLS] and [SEP] need to be added. For example, the converted input sequence is "[CLS] Today is sunny and suitable for going out to play [SEP]".

[0170] Through the forward propagation calculation of the pre-trained BERT model, the context vector representation of each word in the semantic segment unit can be obtained. The BERT model uses a multi-layer Transformer structure, which can capture the long-distance dependencies between words and generate word vectors that take into account the context. For example, for the word "today", a 768-dimensional vector representation can be obtained, which contains the semantic information of "today" in the entire semantic segment.

[0171] Next, we extract the vector corresponding to the special tag [CLS], and combine the attention mechanism and pooling operation to generate a deep semantic representation of the semantic segment unit. The [CLS] vector can be regarded as a semantic summary of the entire semantic segment. By taking a weighted average of all word vectors, we can get a more comprehensive semantic representation. For example, we can get a 768-dimensional vector as the deep semantic representation of the semantic segment unit.

[0172] Then, the topic representation and the deep semantic representation of the semantic segment unit are fused by one of the connection fusion, attention fusion or gated fusion methods to obtain the fused semantic representation of the semantic segment unit. For example, the connection fusion method can be used to directly splice the topic representation vector and the deep semantic representation vector together to obtain a higher-dimensional fused semantic representation vector.

[0173] Next, the domain knowledge base is introduced to enhance the knowledge of the semantic fragment unit of the fusion semantic representation. First, a domain entity dictionary is constructed, which contains the standard entity names and aliases of the domain. Then, the co-occurrence statistics method is used to establish the link between the entities in the semantic fragment unit and the standard entities in the domain knowledge base. For example, for the phrase "the weather is clear", it can be linked to the "weather conditions" entity in the knowledge base.

[0174] Based on the established entity links, template-based and machine learning methods are used to extract entity attributes and inter-entity relationships from semantic fragment units to obtain knowledge triples within the fragment. For example, a triple such as (weather conditions, attributes, sunny) can be extracted.

[0175] Then, the reasoning method based on logical rules and knowledge representation learning is used to perform knowledge reasoning on the knowledge triples in the fragment using the domain knowledge base to obtain semantic information after knowledge enhancement. For example, based on "clear weather", implicit semantic information such as "suitable for outdoor activities" can be inferred.

[0176] Afterwards, the semantic information after knowledge enhancement is integrated with the fused semantic representation of the semantic fragment unit by using one of the methods of connection fusion, attention fusion or gated fusion to obtain the semantic fragment unit representation after knowledge enhancement. For example, the new semantic information vector obtained by reasoning can be spliced ​​with the previous fused semantic representation vector to obtain a richer semantic representation.

[0177] Finally, according to the requirements of the semantic understanding task, the semantic fragment unit representation after knowledge enhancement is aggregated, sorted, filtered, and filled into the corresponding position of the document generation template to generate the target document. For example, for the summary generation task, the fragments can be sorted according to semantic importance, and the top-k important fragments can be selected and filled into the summary template to generate the final document summary.

[0178] The beneficial effects of this technical solution are mainly reflected in the following three aspects:

[0179] 1. Integrate multiple semantic representation methods to fully explore the semantic information of the text. By combining multiple representation methods such as topic models and deep semantic models, it is possible to capture the semantic features of the text from different angles and obtain a more comprehensive and rich semantic representation, which is conducive to improving the performance of subsequent semantic understanding tasks.

[0180] 2. Introduce domain knowledge to enhance semantic representation and make up for the limitations of relying solely on text. Integrating external knowledge into semantic representation through knowledge base linking, knowledge extraction and reasoning techniques can supplement the implicit semantics that are not clearly expressed in the text and improve the depth and breadth of semantic understanding.

[0181] 3. Adopt flexible fusion strategies to achieve effective integration of semantic information. Using different fusion methods in multiple links, such as connection fusion and attention fusion, can select the best fusion strategy according to specific task requirements, effectively integrate semantic information from different sources, and improve the quality of semantic representation and task adaptability.

[0182] In an optional implementation, the semantic understanding task requires aggregating, sorting, filtering, and filling the semantic fragment unit representations after knowledge enhancement into corresponding positions of the document generation template to generate a target document, including:

[0183] The semantic fragment unit representation after knowledge enhancement is subjected to topic aggregation, entity aggregation and relationship aggregation. The topic aggregation uses the topic distribution similarity to calculate the topic similarity between the semantic fragment unit representations after knowledge enhancement, and the semantic fragment unit representations after knowledge enhancement with a topic similarity higher than a preset threshold are aggregated into the same topic class; the entity aggregation uses the same entity inclusion to determine whether the semantic fragment unit representation after knowledge enhancement contains the same entity, and the semantic fragment unit representations after knowledge enhancement containing the same entity are aggregated into the same entity class; the relationship aggregation uses the relationship type consistency to determine whether the relationship type between the semantic fragment unit representations after knowledge enhancement is consistent, and the semantic fragment unit representations after knowledge enhancement with the same relationship type are aggregated into the same relationship class;

[0184] For each topic in the topic aggregation results, the semantic fragment unit and task relevance scoring and sorting method is used to calculate the relevance score of each knowledge-enhanced semantic fragment unit representation and the semantic understanding task, and the knowledge-enhanced semantic fragment unit representations are sorted from high to low according to the task relevance score; for each entity class in the entity aggregation results, the importance evaluation scoring and sorting method is used to calculate the entity importance score contained in each knowledge-enhanced semantic fragment unit representation, and the knowledge-enhanced semantic fragment unit representations are sorted from high to low according to the entity importance score; for each relationship class in the relationship aggregation results, the logical order between the knowledge-enhanced semantic fragment unit representations is calculated based on the logical order relationship sorting method, and the knowledge-enhanced semantic fragment unit representations are arranged in sequence according to the logical order;

[0185] In the sorting results of topic aggregation results, the semantic fragment unit representations enhanced with redundant knowledge whose similarity is higher than a preset threshold in each topic class are eliminated by using the similarity calculation method between semantic fragment unit representations; in the sorting results of entity aggregation results, the semantic fragment unit representations enhanced with knowledge irrelevant to the entity in each entity class are eliminated by using the keyword matching method; in the sorting results of relationship aggregation results, the semantic fragment unit representations enhanced with knowledge whose readability and coherence are lower than a preset quality threshold in each relationship class are eliminated by using the readability and coherence quality assessment method to obtain optimized semantic fragment units; a document generation template that meets the requirements of the semantic understanding task is designed, and the optimized semantic fragment units are filled into the corresponding positions in the document generation template. When filling, the semantic fragment units are matched with the template positions for semantic similarity, the length of the semantic fragment units is trimmed and the grammatical style is adjusted and adapted according to the template specifications, and transition words inserted between the semantic fragment units are generated by a template-based or deep learning method to obtain the target document.

[0186] The specific implementation is as follows:

[0187] First, the semantic fragment unit representation after knowledge enhancement is subjected to topic aggregation, entity aggregation and relationship aggregation. Topic aggregation uses the topic distribution similarity calculation method to calculate the topic distribution similarity between each two semantic fragment unit representations, such as using Jensen-Shannon divergence to calculate the similarity of two topic distributions. Semantic fragment unit representations with similarity higher than a preset threshold (such as 0.8) are aggregated into the same topic class. For example, there are two semantic fragment unit representations A and B, whose topic distributions are [0.3, 0.5, 0.2] and [0.25, 0.55, 0.2] respectively. The calculated similarity of their topic distribution is 0.95, which is higher than the preset threshold 0.8, so A and B are aggregated into the same topic class.

[0188] Entity aggregation is performed by judging whether the semantic fragment unit representation contains the same entity. First, the entities contained in each semantic fragment unit representation are extracted, and then the entities contained in different semantic fragment unit representations are compared to see if they overlap, and the semantic fragment unit representations containing the same entity are aggregated into the same entity class. For example, semantic fragment unit representation C contains the entities "apple" and "mobile phone", and semantic fragment unit representation D contains the entities "apple" and "computer". Since they both contain the entity "apple", C and D are aggregated into the same entity class.

[0189] Relation aggregation is performed by judging whether the relationship types between semantic fragment units are consistent. First, identify the relationship type contained in each semantic fragment unit, such as causal relationship, parallel relationship, etc., and then aggregate the semantic fragment units with consistent relationship types into the same relationship class. For example, if the semantic fragment units E and F both contain causal relationships, then E and F are aggregated into the same relationship class.

[0190] Next, sort the aggregation results. For each topic class in the topic aggregation results, the semantic fragment unit and task relevance scoring method is used to calculate the relevance score of each semantic fragment unit representation with the semantic understanding task. Specifically, a similarity calculation method based on word vectors can be used to convert the semantic understanding task description and the semantic fragment unit representation into word vectors respectively, and then calculate their cosine similarity as the relevance score. Sort the semantic fragment unit representations from high to low according to the relevance score. For example, for the topic class "artificial intelligence", the relevance score of the semantic fragment unit representation G is 0.8, and the score of H is 0.6, then G is ranked before H.

[0191] For each entity class in the entity aggregation result, the entity importance score contained in each semantic fragment unit representation is calculated using the entity importance evaluation method. The entity importance can be evaluated based on the entity's connectivity in the knowledge graph, PageRank value and other features. The semantic fragment unit representations are sorted from high to low according to the entity importance score. For example, in the entity class "apple", the "apple" entity importance score contained in the semantic fragment unit representation I is 0.9, and the "apple" entity importance score contained in J is 0.7, then I is ranked before J.

[0192] For each relationship class in the relationship aggregation result, a sorting method based on logical order is adopted. First, the logical relationship between the semantic fragment unit representations is analyzed, such as time order, causal order, etc., and then the semantic fragment unit representations are sorted according to the logical order. For example, in the causal relationship class, the semantic fragment unit representation K describes the cause and L describes the result, then K is ranked before L.

[0193] Then filter the sorting results. In the sorting results of topic aggregation, the similarity calculation method between semantic fragment unit representations is used to eliminate redundant content. The cosine similarity of word vectors can be used to calculate the similarity between semantic fragment unit representations. If the similarity is higher than the preset threshold (such as 0.9), it is considered that there is redundancy and the one with the higher score is retained. For example, the similarity between semantic fragment unit representations M and N is 0.95, which is higher than the threshold 0.9, and the score of M is higher than N, then M is retained and N is eliminated.

[0194] In the ranking results of entity aggregation, keyword matching method is used to eliminate content irrelevant to the entity. The core keywords of each entity class are extracted, and the semantic fragment unit representation is checked to see whether it contains these keywords. If not, it is eliminated. For example, for the entity class "Apple mobile phone", the core keywords may include "iPhone", "iOS", etc. If a semantic fragment unit representation does not contain these keywords, it is eliminated.

[0195] In the ranking results of relation aggregation, readability and coherence quality assessment methods are used to eliminate low-quality content. A language model-based method can be used to evaluate the readability and coherence of semantic fragment unit representations, set a quality threshold, and eliminate those below the threshold. For example, a pre-trained language model such as BERT is used to calculate the perplexity of the semantic fragment unit representation as a quality indicator. If the perplexity is higher than the preset threshold, the semantic fragment unit representation is eliminated.

[0196] Finally, a document generation template that meets the requirements of the semantic understanding task is designed, and the optimized semantic fragment units are filled into the corresponding positions of the template. First, the template structure is designed according to the task requirements, such as introduction, main body, conclusion, etc. Then the semantic similarity between the semantic fragment unit and each part of the template is calculated, and the semantic fragment unit is assigned to the most similar template position. When filling, the length of the semantic fragment unit needs to be trimmed and the grammatical style needs to be adjusted to meet the template specifications. For example, if the template stipulates that each paragraph does not exceed 100 words, the overlong semantic fragment units need to be trimmed. At the same time, a template-based or deep learning method is used to generate transition words between semantic fragment units to improve the coherence of the document. Finally, the target document that meets the task requirements is obtained.

[0197] The beneficial effects of this method include:

[0198] 1. Through multi-dimensional aggregation and sorting, the semantic fragment unit representation after knowledge enhancement is effectively organized and optimized, and the structured degree and logic of the generated document are improved.

[0199] 2. Use multiple filtering methods to remove redundant and irrelevant content, while ensuring the simplicity and relevance of the document and improving the quality of the generated document.

[0200] 3. The use of document generation templates and transition language generation technology ensures the standardization and coherence of the generated documents, making the generated documents more in line with human reading habits.

[0201] Figure 2 Schematic diagram of the structure of the real-time data analysis and visualization system in a big data environment according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0202] The first unit is used to obtain a document to be processed, perform denoising and normalization processing on the document to be processed, and according to a preset document type determination rule, extract the format features, keyword features and layout structure features of the document, and use a classification model based on a combination of rules and machine learning to determine the type of the document to be processed, and use corresponding preprocessing parameters to preprocess different types of documents to obtain preprocessed documents, wherein the type of the document to be processed includes at least one of a contract, a paper, news, and a resume;

[0203] The second unit is used to perform hierarchical structural analysis on the preprocessed document, extract structured information in the document, and construct a structured document model based on the structured information; perform semantic understanding analysis on the preprocessed document using a pre-trained semantic understanding analysis model to obtain document semantic information, wherein the semantic understanding analysis model adopts a neural network framework that combines a pre-trained language model based on knowledge enhancement with an attention mechanism, and is constructed after fine-tuning by introducing domain ontology knowledge; establish a semantic index mapping based on the structured document model and the document semantic information, wherein the semantic index mapping includes a mapping relationship between document content and semantic information and structural information;

[0204] The third unit is used to receive automated processing instructions input by the user in natural language, use a natural language understanding model based on deep learning to understand the intention and semantically parse the instructions, extract semantic elements in the instructions, match a predefined semantic understanding task library according to the semantic elements, determine one or more semantic understanding tasks corresponding to the instructions, match one or more semantic fragment units of corresponding granularity from the semantic index mapping according to the granularity of the semantic understanding task; process the matched semantic fragment units for the semantic understanding task to obtain a target document, and visualize the target document.

[0205] According to a third aspect of the embodiments of the present invention,

[0206] An electronic device is provided, comprising:

[0207] processor;

[0208] a memory for storing processor-executable instructions;

[0209] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0210] A fourth aspect of the embodiments of the present invention is:

[0211] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0212] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0213] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time data analysis and visualization method in a big data environment, characterized in that: include: Obtaining a document to be processed, performing denoising and normalization processing on the document to be processed, and according to a preset document type determination rule, extracting format features, keyword features, and layout structure features of the document, using a classification model based on a combination of rules and machine learning to determine the type of the document to be processed, and using corresponding preprocessing parameters to perform preprocessing on different types of documents to be processed to obtain a preprocessed document, wherein the type of the document to be processed includes at least one of a contract, a paper, news, and a resume; Performing hierarchical structural analysis on the preprocessed document, extracting structured information from the document, and constructing a structured document model based on the structured information; performing semantic understanding analysis on the preprocessed document using a pre-trained semantic understanding analysis model to obtain document semantic information, wherein the semantic understanding analysis model adopts a neural network framework that combines a pre-trained language model based on knowledge enhancement with an attention mechanism, and is constructed by introducing domain ontology knowledge for fine-tuning; establishing a semantic index mapping based on the structured document model and the document semantic information, wherein the semantic index mapping includes a mapping relationship between document content and semantic information and structural information; Receive an automated processing instruction input by a user in natural language, use a natural language understanding model based on deep learning to understand the intention and semantically parse the instruction, extract semantic elements in the instruction, match a predefined semantic understanding task library according to the semantic elements, determine one or more semantic understanding tasks corresponding to the instruction, match one or more semantic segment units of corresponding granularity from the semantic index mapping according to the granularity of the semantic understanding task; process the matched semantic segment units for the semantic understanding task to obtain a target document, and visualize the target document; For the semantic understanding task, the matching semantic fragment units are preprocessed to obtain a bag-of-words representation, and the latent Dirichlet allocation (LDA) model is used to estimate parameters through the Gibbs sampling algorithm to obtain the topic distribution vector and topic keyword list of the semantic fragment unit, and the topic distribution vector and the keyword list are combined to form the topic representation of the semantic fragment unit; the bag-of-words representation is converted into the input format of the deep bidirectional encoder representation BERT model, and the context vector of each word in the semantic fragment unit is calculated through the forward propagation of the pre-trained BERT model, and the vector corresponding to the special tag is extracted and combined with the attention mechanism and pooling operation to generate the deep semantic representation of the semantic fragment unit; The topic representation and deep semantic representation of the semantic fragment unit are fused by any one of connection fusion, attention fusion and gated fusion to obtain the fused semantic representation of the semantic fragment unit; the domain knowledge base is introduced to enhance the knowledge of the semantic fragment unit of the fused semantic representation, and the links between the entities in the semantic fragment unit and the standard entities in the domain knowledge base are established by constructing a domain entity dictionary and using co-occurrence statistics. Based on the links, the entity attributes and the relationship between entities are extracted from the semantic fragment unit by using template-based and machine learning methods to obtain the knowledge triples in the fragment; The reasoning method based on logical rules and knowledge representation learning is used to utilize the domain knowledge base to perform knowledge reasoning on the knowledge triples in the fragment to obtain the semantic information after knowledge enhancement; any one of the connection fusion, attention fusion, and gated fusion methods is used to integrate the semantic information after knowledge enhancement with the fused semantic representation of the semantic fragment unit to obtain the semantic fragment unit representation after knowledge enhancement; according to the requirements of the semantic understanding task, the semantic fragment unit representation after knowledge enhancement is aggregated, sorted, filtered, and filled into the corresponding position of the document generation template to generate the target document.

2. The method according to claim 1, characterized in that Performing hierarchical structural analysis on the preprocessed document, extracting structured information in the document, and constructing a structured document model based on the structured information, including: The preprocessed document is subjected to structural feature extraction to obtain layout areas including text area, image area, table area and formula area; the text area is further analyzed using the connected domain analysis algorithm, and text objects of different granularities are extracted through adaptive threshold segmentation and morphological filtering processing; the image area is detected and identified using the target detection model based on regional convolutional neural network to obtain the location and content information of the image object; the table area is analyzed using a method based on table structure analysis, and the structured representation of the table is obtained through table line extraction, cell positioning and content recognition; the formula area is extracted from the formula area using a method based on symbolic analysis and syntax parsing to obtain the document layout structure features including text, images, tables and formulas; Different layout areas are abstracted as nodes of the graph model, and the node attributes contain the document layout structure characteristics of different layout areas; according to the document layout structure characteristics between different layout areas, corresponding edges are added to the graph model, and the attributes of the edges represent the type of logical relationship between different layout areas; the graph model is updated through iterative optimization until the graph model converges; the spectral clustering algorithm is used to cluster the nodes of the graph model to obtain the clustered logical structure units; the hierarchical and sequential relationships between the clustered logical structure units are obtained through topological sorting to obtain the document logical structure tree; Combined with the document logical structure tree, the structured feature vector representation of each logical structure unit is extracted, wherein the geometric and format features including position, font, font size and alignment are extracted for the text-type logical structure units, and the geometric features and content features including position, size and shape are extracted for the non-text-type logical structure units; the document logical structure tree and the structured feature vector representation are matched according to the predefined structure template rules, and the structured feature vectors that meet the predefined structure template rules are combined to form a structure framework that meets the document structure specifications; a document ontology knowledge base for a specific field is constructed, and a structured document model is constructed based on the document ontology knowledge base for a specific field and the structure framework that meets the document structure specifications.

3. The method according to claim 1, characterized in that The pre-processed document is subjected to semantic understanding analysis using a pre-trained semantic understanding analysis model to obtain document semantic information, including: Performing semantic understanding analysis on the preprocessed document using a pre-trained semantic understanding analysis model, wherein the semantic understanding analysis model normalizes the professional terms and abbreviations in the document to obtain the semantic feature vectors of each structural element of the document, and obtaining the semantic relevance between different structural elements by calculating the similarity between the semantic feature vectors; processing the preprocessed document using a pre-trained semantic understanding analysis model to obtain word semantic feature vectors, sentence semantic feature vectors and document semantic feature vectors, wherein the semantic understanding analysis model includes a word vector model, a bidirectional recurrent neural network and a hierarchical attention model; generating semantic feature vectors of the document structural elements based on the word semantic feature vectors, the sentence semantic feature vectors and the document semantic feature vectors; A cosine similarity calculation method is used to map the semantic feature vectors of two different document structure elements into a vector space of the same dimension, and the cosine value of the angle between the semantic feature vectors of the two document structure elements is calculated, and the smaller the cosine value of the angle, the higher the semantic similarity; a Euclidean distance calculation method is used to calculate the Euclidean distance between the semantic feature vectors of the two document structure elements, and the smaller the Euclidean distance, the higher the semantic similarity; a semantic similarity calculation model is constructed using a twin network, and the twin network includes two sub-networks with shared parameters, and nonlinear transformations are performed on the semantic feature vectors of the two document structure elements respectively, and a cosine similarity layer is used at the top layer to calculate the similarity score of the twin network of the semantic feature vectors of the two document structure elements; A semantic similarity calculation model is constructed using a matching network, and the semantic feature vectors of the document structure elements to be matched are matched with the sample vectors in the support set through an attention mechanism, a matching vector is generated, and the similarity score of the matching network between the semantic feature vectors of the document structure elements to be matched and the sample vectors in the support set is calculated through a fully connected layer; the semantic relevance of the two document structure elements is calculated based on the cosine similarity, the Euclidean distance, the similarity score of the twin network and the similarity score of the matching network; the semantic feature vectors of the document structure elements are traversed to calculate the semantic relevance between the semantic feature vectors of any two different document structure elements.

4. The method according to claim 3, characterized in that The pre-processed documents are processed using a pre-trained semantic understanding analysis model to obtain structural element semantic feature vectors including word semantic feature vectors, sentence semantic feature vectors, and document semantic feature vectors, including: For the professional terms and abbreviations in the preprocessed documents, a domain-adaptive word vector enhancement method is used. The domain text corpus is used to perform secondary training on the word vector model. The word vector model is adjusted through a supervised probability prediction task. The adjusted word vector model is used to embed the words in the preprocessed documents to obtain the word semantic feature vector; for the unregistered words in the preprocessed documents, a character-based word embedding method is used to obtain the word semantic feature vector; Based on the word semantic feature vector, a bidirectional recurrent neural network is used to encode the sentence to obtain the sentence semantic feature vector, wherein the bidirectional recurrent neural network includes a long short-term memory network LSTM and a gated recurrent unit GRU; in the process of encoding the sentence, the word semantic feature vector in the sentence is encoded from left to right and from right to left respectively; an attention mechanism is superimposed on the basis of the bidirectional recurrent neural network; the output of the bidirectional recurrent neural network is weightedly summed after attention to obtain the sentence semantic feature vector; Use the fully connected layer in the hierarchical attention model to map the sentence semantic vectors in the paragraph to the attention space, use the normalization function to normalize the vectors in the attention space to obtain the attention weight distribution, multiply the sentence semantic vectors in the paragraph with the corresponding attention weights and sum them to obtain the paragraph semantic vector; use the hierarchical attention model to integrate the paragraph semantic vectors to obtain the chapter semantic vector; use the attention mechanism to perform weighted fusion on the chapter semantic vectors to generate the document semantic feature vector.

5. The method according to claim 1, characterized in that According to the structured document model and the document semantic information, a semantic index mapping is established, wherein the semantic index mapping contains a mapping relationship between document content and semantic information and structural information, including: According to the structured document model and semantic relevance, a sequence annotation model based on conditional random field (CRF) is used, combined with predefined semantic segmentation rules, to automatically segment the document into semantic segments to obtain semantic segment units; a semantic index mapping is established for the structural features, semantic features and knowledge features of the semantic segment units, wherein the semantic index mapping includes a mapping relationship between document content and semantic information and structural information; the semantic relevance between different structural element sequences in the structured document model is calculated to obtain a document structural element semantic relevance matrix; The document structure element sequence is used as the observation sequence of the sequence annotation model based on the conditional random field (CRF), and the semantic fragment label is used as the hidden state sequence; the transfer features between the observation sequence and the hidden state sequence, as well as the emission features of the observation sequence are calculated; wherein the transfer features between the observation sequence and the hidden state sequence include: the semantic relevance between adjacent structure elements and the compatibility between the semantic features of the structure elements and the semantic fragment labels; the emission features of the observation sequence include: the hierarchical depth, context structure and format structure features of the structure elements; the transfer features and the emission features are combined through the feature function to construct a conditional random field model; the Viterbi algorithm is used to decode the conditional random field model to obtain the optimal semantic fragment label sequence as the semantic fragment unit; For the semantic fragment unit, its structural features, semantic features and knowledge features are extracted; wherein the structural features of the semantic fragment unit include: the position, context structure and format of the semantic fragment unit in the document; the semantic features of the semantic fragment unit include: the keywords, themes and sentiment tendency of the semantic fragment unit; the knowledge features of the semantic fragment unit include: the entities, events and concepts contained in the semantic fragment unit; an inverted index of the semantic index mapping is constructed, with the semantic features, structural features and knowledge features of the semantic fragment unit as index fields, and the semantic fragment unit number and position information as an inverted list; based on the inverted index, a vectorized representation of the semantic fragment unit is constructed to establish a semantic index mapping, wherein the semantic index mapping includes a mapping relationship between the content of the semantic fragment unit and its structural features, semantic features and knowledge features.

6. The method according to claim 1, characterized in that According to the semantic understanding task requirements, the semantic fragment unit representations after knowledge enhancement are aggregated, sorted, and filtered, and filled into the corresponding positions of the document generation template to generate the target document, including: The semantic fragment unit representation after knowledge enhancement is subjected to topic aggregation, entity aggregation and relationship aggregation. The topic aggregation uses the topic distribution similarity to calculate the topic similarity between the semantic fragment unit representations after knowledge enhancement, and the semantic fragment unit representations after knowledge enhancement with a topic similarity higher than a preset threshold are aggregated into the same topic class; the entity aggregation uses the same entity inclusion to determine whether the semantic fragment unit representation after knowledge enhancement contains the same entity, and the semantic fragment unit representations after knowledge enhancement containing the same entity are aggregated into the same entity class; the relationship aggregation uses the relationship type consistency to determine whether the relationship type between the semantic fragment unit representations after knowledge enhancement is consistent, and the semantic fragment unit representations after knowledge enhancement with the same relationship type are aggregated into the same relationship class; For each topic in the topic aggregation results, the semantic fragment unit and task relevance scoring and sorting method is used to calculate the relevance score of each knowledge-enhanced semantic fragment unit representation and the semantic understanding task, and the knowledge-enhanced semantic fragment unit representations are sorted from high to low according to the task relevance score; for each entity class in the entity aggregation results, the importance evaluation scoring and sorting method is used to calculate the entity importance score contained in each knowledge-enhanced semantic fragment unit representation, and the knowledge-enhanced semantic fragment unit representations are sorted from high to low according to the entity importance score; for each relationship class in the relationship aggregation results, the logical order between the knowledge-enhanced semantic fragment unit representations is calculated based on the logical order relationship sorting method, and the knowledge-enhanced semantic fragment unit representations are arranged in sequence according to the logical order; In the sorting results of topic aggregation results, the semantic fragment unit representations enhanced with redundant knowledge whose similarity is higher than a preset threshold in each topic class are eliminated by using the similarity calculation method between semantic fragment unit representations; in the sorting results of entity aggregation results, the semantic fragment unit representations enhanced with knowledge irrelevant to the entity in each entity class are eliminated by using the keyword matching method; in the sorting results of relationship aggregation results, the semantic fragment unit representations enhanced with knowledge whose readability and coherence are lower than a preset quality threshold in each relationship class are eliminated by using the readability and coherence quality assessment method to obtain optimized semantic fragment units; a document generation template that meets the requirements of the semantic understanding task is designed, and the optimized semantic fragment units are filled into the corresponding positions in the document generation template. When filling, the semantic fragment units are matched with the template positions for semantic similarity, the length of the semantic fragment units is trimmed and the grammatical style is adjusted and adapted according to the template specifications, and transition words inserted between the semantic fragment units are generated by a template-based or deep learning method to obtain the target document.

7. A real-time data analysis and visualization system in a big data environment, used to implement the method according to any one of claims 1 to 6, characterized in that: include: The first unit is used to obtain a document to be processed, perform denoising and normalization processing on the document to be processed, and according to a preset document type determination rule, extract the format features, keyword features and layout structure features of the document, and use a classification model based on a combination of rules and machine learning to determine the type of the document to be processed, and use corresponding preprocessing parameters to preprocess different types of documents to obtain preprocessed documents, wherein the type of the document to be processed includes at least one of a contract, a paper, news, and a resume; The second unit is used to perform hierarchical structural analysis on the preprocessed document, extract structured information in the document, and construct a structured document model based on the structured information; perform semantic understanding analysis on the preprocessed document using a pre-trained semantic understanding analysis model to obtain document semantic information, wherein the semantic understanding analysis model adopts a neural network framework that combines a pre-trained language model based on knowledge enhancement with an attention mechanism, and is constructed after fine-tuning by introducing domain ontology knowledge; establish a semantic index mapping based on the structured document model and the document semantic information, wherein the semantic index mapping includes a mapping relationship between document content and semantic information and structural information; The third unit is used to receive automated processing instructions input by the user in natural language, use a natural language understanding model based on deep learning to understand the intention and semantically parse the instructions, extract semantic elements in the instructions, match a predefined semantic understanding task library according to the semantic elements, determine one or more semantic understanding tasks corresponding to the instructions, match one or more semantic fragment units of corresponding granularity from the semantic index mapping according to the granularity of the semantic understanding task; process the matched semantic fragment units for the semantic understanding task to obtain a target document, and visualize the target document.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images