541 results about "Document analysis" patented technology

According to the research and development document processing method and device provided by the embodiment of the invention, unified processing of text, voice and image information is realized by constructing the multi-mode document analysis engine. Different types of data are converted into standardized feature representations through cross-modal preprocessing and a semantic alignment network. The system adopts a deep learning model to identify research and development elements, establishes a multi-modal relation graph, and realizes intelligent extraction and correlation analysis of various types of information in research and development documents. According to the method, the defects of the traditional technology in the aspects of multi-modal information fusion and knowledge system construction are effectively overcome, powerful support is provided for research and development process management and knowledge asset accumulation, and the standardized management level of research and development documents is remarkably improved.

Example systems and methods for analyzing and summarizing legal documents using artificial intelligence are disclosed. A system receives a legal document such as a contract or agreement, and utilizes multiple AI models in combination to process and summarize the document. This includes identifying the type of legal document using a natural language model, and then extracting key information like parties, dates, and monetary values. Related clauses within the document are consolidated using a clause consolidation algorithm even if they are not contiguous. Summaries are generated for each section of the document. Additionally, a question-answering module preemptively generates common questions and answers about the agreement to highlight key considerations for the parties involved. By leveraging an ensemble of customized AI models, the system aims to simplify comprehension of legal documents.

An Artificial Intelligence (AI) & Generative AI-driven cross-domain document analysis system enables accurate and consistent narratives across a longitudinal timeline for an entity regarding communications in different operational aspects. The document analysis and insight system includes an Artificial Intelligence (AI) powered Search Interface (AIPS) and an Advanced Intelligent Knowledge Engine (AIKE). The AIPS is configured to pre-process documents from structured and unstructured data sources to generate data taxonomies and custom synonym files. The AIKE generates a preliminary evaluation of the various Large Language Models (LLMs) and uses the data taxonomies and custom synonym files to generate prompts that are configured to address limitations of the various LLMs to obtain accurate replies to user requirements.

The invention provides a semi-structured file processing method based on an LLMs (Language Language Model), which is used for efficiently analyzing a semi-structured file and converting the semi-structured file, and belongs to the technical field of intelligent document processing and multi-modal learning. The method is characterized by comprising the following steps: multi-modal fusion analysis: adopting a hierarchical cross-modal attention mechanism HCMA, and combining a multi-head attention mechanism with a graph neural network GNN to realize multi-level feature fusion of a text and an image; adaptive structured conversion: adopting a dynamic structure analysis network DPN, and combining reinforcement learning and a generation model to realize adaptive analysis of a document structure; and model generation and multi-modal reasoning: adopting a multi-modal generation reasoning network MGRN, and combining a diffusion model and neural symbol reasoning to realize joint generation and reasoning of the text and the image. According to the method, a remarkable technical breakthrough is brought to the aspects of document analysis, information extraction and intelligent reasoning, and powerful support is provided for data-driven innovation of various industries.

The embodiment of the invention discloses an RAG multi-mode document analysis method and device and a medium, relates to the technical field of natural language processing and is used for solving the problem that an existing analysis mode is low in processing capacity. The method comprises the following steps: identifying a document type of a multi-modal document, and distributing an analysis task of the multi-modal document to a corresponding Kubernetes task node based on the document type; based on the analysis requirement of the multi-modal document, the analysis process of each Kubernetes task node is adjusted, the analysis task of the multi-modal document is executed based on the updated analysis process, and analysis data is obtained; preprocessing the parsed data, and performing semantic slicing on the processed parsed data in a dynamic text windowing mode based on a preset large language model to obtain semantic blocks corresponding to the processed parsed data; and converting the semantic blocks into embedded vectors, and storing the embedded vectors in a preset vector database so as to carry out rapid retrieval based on the preset vector database.

The invention belongs to the technical field of artificial intelligence. According to the automatic evaluation method and system for the retrieval enhancement generation system, different answer evaluation algorithms are adopted for evaluation for answers generated by the retrieval enhancement generation system and answers in an evaluation data set, and a plurality of algorithm evaluation indexes are obtained; and calculating a plurality of overall evaluation indexes of each retrieval enhancement generation system, performing principal component analysis on all the algorithm evaluation indexes and the overall evaluation indexes, and further evaluating the RAG system. According to the method, the automation problem of document analysis and question and answer data generation is solved, high-quality question and answer pairs are provided, the question and answer quality and retrieval quality of the RAG system are automatically evaluated, and a more accurate automatic evaluation result is obtained by combining multiple evaluation indexes with a principal component analysis method.

The invention discloses a domain-adaptive retrieval enhancement generation method and system, and the method comprises the steps: carrying out the document analysis and semantic partitioning of a document through a semantic understanding model and a document rule, and storing document fragments; on the basis of the large language model and a preset question library, generating a question and disassembling the question; performing correlation evaluation and sorting on the initial retrieval results in combination with a retrieval technology; candidate content most matched with the cue words is screened out through a large language model, and question-retrieval fragments are scored and marked; performing fusion on the initial retrieval result and the scoring and labeling result to construct positive and negative samples, and constructing a supervised data set; and based on a preset loss function, utilizing the supervised data set to carry out supervised training on the knowledge semantic search model, replacing the RAG mixed search module, and carrying out continuous iteration. By means of the technical scheme, new domain knowledge and operation feedback data can be flexibly applied, and domain self-adaptive retrieval enhancement generation is achieved.

Example document analysis and management systems and methods are described. In one implementation, a document is identified for processing. An artificial intelligence engine extracts information from the document and creates multiple chunks of data associated with the document. Embeddings are performed for the multiple chunks of data to create chunk embeddings, where the chunk embeddings are represented as numerical vectors. The chunk embeddings are stored in a vector database. A large language model (LLM) generates document content insights based on the multiple chunks of data and the chunk embeddings.

The invention provides a multi-modal document content cross-platform analysis system, and relates to the technical field of data processing, and the system comprises a document preprocessing module which is used for receiving multi-source heterogeneous document input data, and generating a preprocessed document through format conversion, page segmentation, noise reduction and optical character recognition; the feature extraction module is used for extracting three types of features, including spatial layout features, semantic features and logic structure features, based on the preprocessed document; according to the method, the multi-source heterogeneous document is cooperatively processed, accurate extraction, calibration and association of features in cross-platform analysis are realized, finally structured data are generated, and the accuracy, consistency and cross-platform applicability of multi-modal document analysis are improved.

The invention discloses a character recognition and automatic auditing method and system, and the method comprises the steps: obtaining all to-be-audited document data including semi-structured and non-structured document data, guaranteeing the comprehensiveness of an auditing process, accurately recognizing the document content through an OCR technology, converting the document content into operable structured data, and carrying out the automatic auditing of the to-be-audited document data. Key information in a document is selected through two modes of rule extraction and model extraction, the efficiency and accuracy of information processing are improved, the information is compared with pre-established target structured text data, compliance check is automatically completed, the compliance of an auditing result is ensured, a confidence coefficient threshold value is updated in real time according to the complexity of the document, and the accuracy of information processing is improved. The method is suitable for document data with different complexities, the automatic auditing process remarkably improves the auditing efficiency, when a large amount of document data is processed, a large amount of time and manpower resources can be saved, the auditing result and the corresponding confidence rate can be output, a user can see the specific result of document analysis, and the auditing reliability and the credibility of the user are improved.

The invention discloses a document analysis and query method based on a knowledge graph, and the method comprises the steps: carrying out the part-of-speech tagging of a received to-be-analyzed document, and obtaining a part-of-speech tagging result; extracting knowledge element information from the part-of-speech tagging result based on a preset power grid domain ontology knowledge base, and constructing an initial knowledge graph based on the knowledge element information; combining nodes in the initial knowledge graph to obtain a fused knowledge graph; constructing a mapping table according to the fused knowledge graph, and generating a candidate query template set based on the mapping table; receiving a natural language query statement input by a user, selecting a target query template from the candidate query template set based on the natural language query statement, and generating a target query statement; querying from the fused knowledge graph by using the target query statement, and outputting a query result; according to the method, the accuracy and comprehensiveness of information analysis can be effectively improved, the query intention of the user can be accurately understood, and the accurate query and analysis requirements of professionals on project documents are met.

The invention relates to the technical field of artificial intelligence, in particular to an agent decision path optimization method fusing explicit topology and implicit semantics. The method comprises the following steps: acquiring multi-modal data; performing document analysis according to the multi-modal data to obtain document analysis data; performing knowledge injection according to the document analysis data to obtain knowledge injection data; carrying out implicit semantic detection according to the knowledge injection data to obtain implicit semantic data; constructing a user demand agent blueprint according to the implicit semantic data; constructing an agent workflow according to the agent blueprint required by the user; performing task execution scheduling according to the agent workflow to obtain task execution data; performing node explicit topology construction according to the task execution data to obtain node explicit topology data; and performing communication anomaly analysis based on the node explicit topology data to obtain node communication anomaly data. According to the invention, agent task scheduling and path optimization are realized based on an artificial intelligence technology, and the resource utilization rate of the system is improved.

The invention discloses a semantic recognition system and method based on a heterogeneous graph attention network and dynamic normalization, and belongs to the technical field of natural language processing and artificial intelligence. The system adopts a dual-channel architecture and comprises a general semantic channel and a domain semantic channel, semantic feature extraction is performed through a DIFF attention mechanism and a ToST statistical attention mechanism, and training stability is improved by adopting a DyT dynamic normalization module. Adaptive fusion of cross-channel semantic features is realized through a GeGLU gating mechanism, and high-precision semantic recognition is realized by combining an improved SimCSE + + comparison learning loss and a local minimization editing strategy of semantic perception. According to the method, the problems of inaccurate semantic expression, poor context adaptability and the like in the prior art are solved, the accuracy and applicability of cross-domain semantic recognition are remarkably improved, and the method can be widely applied to scenes of legal document processing, financial document analysis and the like.

The invention discloses an intelligent retrieval management method and system based on OCR and document analysis, and relates to the technical field of intelligent retrieval management. The system comprises a document scanning and processing module, a semantic analysis and label generation module, a feature extraction and mapping module and a user demand analysis and retrieval module. The document scanning and processing module is used for converting a paper document into an image file by using a scanner and forming a text data set; the semantic analysis and label generation module performs semantic analysis on each text in the text data set to form a semantic label; the feature extraction and mapping module extracts keywords in the text data, generates word vectors of a keyword set, and constructs a mapping matrix of tags and text word vectors; and the user demand analysis and retrieval module analyzes the user demand data, generates a feature matrix queried by the user, evaluates the matching degree between the text data and the user demand, and sorts and outputs retrieval results.

The invention discloses a multi-literature intelligent classification and automatic review generation method based on a large language model, which is characterized by comprising the following steps of: retrieving literatures or uploading local literatures, and extracting text contents; identifying keywords by utilizing an algorithm or a large language model; converting the keyword into a vector form by using a vectorization model; summarizing the research direction of the clustering result by using a large language model; comprehensively summarizing the single literature and literatures in different research directions by using a large language model; and executing the steps of reviewing, quotation insertion and the like through a large language model and a vectorization semantic matching technology. Compared with the prior art, the method has the advantages that the efficiency and quality of document classification and review generation are remarkably improved, technologies such as text vectorization, topic clustering and large language model generation are well fused, efficient and accurate multi-document review automatic generation is achieved, the method is suitable for interdisciplinary academic document analysis and information integration application, and the method is suitable for popularization and application. Wide application potential is shown.

The invention discloses a document parsing method and device, a medium and a computer program product, and relates to the technical field of data processing.The document parsing method comprises the steps that a to-be-parsed document is obtained, layout detection is conducted on the to-be-parsed document, and element blocks are obtained; determining a content extraction rule corresponding to the element block according to the layout element of the element block; processing the element block through a content extraction rule to obtain initial content of the element block; and correcting the preliminary content by using a large language model to obtain a target document. According to the method, different element blocks are divided, different content extraction methods are adopted for the element blocks, and the extracted content is corrected by utilizing the large language model, so that the correctness of document analysis is improved.

The invention relates to the technical field of artificial intelligence, and discloses an education data report content interaction method and system generated based on retrieval enhancement, and the method comprises the steps: judging whether a natural language problem is an education field problem or not through a large language model, and if yes, carrying out semantic analysis to generate a structured query instruction; when the problem relates to cross-document association analysis, retrieving the structured semantic index database to generate a retrieval result set; if policy association analysis is involved, matching a policy knowledge graph by combining semantic similarity calculation and an entity linking technology, and then performing cross-modal fusion processing to obtain a retrieval result set; and inputting the retrieval result set into the retrieval enhancement generation model, and calling an education field language model to generate an analysis report. According to the method, the industrial pain points of inaccurate intention recognition, low cross-document analysis efficiency, incapability of dynamically combining with latest policies and the like in a traditional interaction mode can be solved, the efficiency and quality of data report interaction in the education field are remarkably improved, and the user interaction experience is optimized.

A data processing system implements obtaining an image of a document and an indication of one or more content items to generate; analyzing the image of the document to generate a textual representation of contents of the document in the image; constructing a query based on the textual representation; analyzing the query using a second machine learning model to obtain embeddings representing one or more categories of information represented in the query; searching a knowledge graph based on the query embeddings to obtain results of the query; providing the query results to a content generation unit to generate the one or more content items based on the results of the query; and obtaining the one or more content items from the content generate unit.

The invention discloses an enterprise data link management and value management method and system, and the method comprises the steps: carrying out the real-time capturing of multi-source heterogeneous data of all business systems in an enterprise through a distributed data collection engine, and recognizing the types of original data in different formats through a preset data source adapter; if structured data is detected, a relational database connector is adopted for extraction, and if the structured data is recognized as unstructured data, a document analysis module is started for content extraction, and an initial data set containing metadata tags is obtained; performing format conversion and field mapping on the initial data set according to a pre-established data standardization rule base, eliminating duplicate records and abnormal values through a data cleaning algorithm, performing automatic evaluation on a data quality grade by adopting a naive Bayes classifier, if a data quality score is lower than a preset threshold value, triggering a data recovery process, and if the data quality score is lower than the preset threshold value, performing data recovery. And standardized data meeting a unified standard is obtained. The normativity and value utilization efficiency of data management are effectively improved.

The embodiment of the invention provides a document-based intelligent question and answer method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: performing document analysis on a target electronic document to obtain document attributes and document contents; based on the document attribute and the document content, document segmentation is conducted on the target electronic document, document segmentation units and unit position information of the document segmentation units are obtained, and the document segmentation units comprise segmentation unit identifiers; based on the document attribute, performing network construction on the target electronic document to obtain a document structure tree network; on the basis of the unit position information, the document segmentation units are arranged in sequence, and a document linear list is obtained; performing document query on the document structure tree network or the document linear list based on the document problem information and the segmentation unit identifier to obtain document query information; and based on the document query information, performing question reply on the document question information. According to the embodiment of the invention, the intelligent question and answer accuracy based on the document can be improved.

The invention discloses a document analysis method based on an RAG model and a dynamic knowledge graph, and relates to the technical field of artificial intelligence. The method is combined with an RAG model and a dynamic knowledge graph technology, and is realized by the following steps of: performing entity relationship joint extraction on an input document, generating a structural triple, and constructing a dynamically updatable knowledge graph; based on the knowledge graph, mapping entities and relationships into low-dimensional vectors by adopting a graph embedding model, and constructing a local vector knowledge base with a topological structure; receiving user questions in real time, encoding the user questions into query vectors, executing approximate nearest neighbor search based on the vector knowledge base, and matching related map fragments; and combining the retrieved graph fragments with the large language model, and generating a structured answer through path constraint of the injection knowledge graph. The method is used for solving the problem that in the prior art, a model cannot capture document deep semantics and dynamic relations insufficiently.

The invention provides a document analysis method and device, equipment and a storage medium, and relates to the technical field of computers, in particular to the technical field of deep learning, data processing and document analysis. According to the specific implementation scheme, at least one layout area divided by an article to which the document image belongs is determined according to the layout of the document image; identifying element contents of a plurality of layout elements in the document image; sorting the reading sequence of the layout elements in the same layout area; and obtaining structured document information according to the sorting result and the corresponding element content. According to the technical scheme, different article areas on the same page can be accurately identified and separated, the respective reading sequence is correctly reconstructed on the basis, and the content in the document image is converted into structured information.

Systems and methods of scoring quality of open source software documentation are disclosed. In one aspect, a method includes downloading the open source software project code and documentation of the open source software project code, analyzing the open source software project code and the documentation, parsing the open source software project documentation into sections, validating the sections of the documentation with project or stack metrics, assessing a quality of sections of the documentation, assessing a quality of language of the documentation; and computing the quality score of the documentation based on the assessed quality of sections and assessed quality of language.

The invention discloses a method for improving long text processing efficiency and accuracy, and relates to the technical field of natural language processing and large language models.According to the method, text word segmentation embedding, sliding block preprocessing, YaRN position code injection, dynamic sparse attention calculation, multi-level attention fusion, graded KV cache management and output generation are sequentially executed; position drift is inhibited through logarithmic scaling, and key contexts are adaptively screened according to the attention activeness, so that the attention calculation complexity is close to linearity; in million-level Token reasoning, the video memory occupation of the method is reduced, the remote dependency recall rate is improved, and the method is suitable for scenes such as document analysis, code auditing and multi-mode streaming understanding.

Document type specific adapters of a document analysis system are used to provide for additional document types or document specialization when generating answers to user submitted questions targeting information included in a document image provided with the user submitted question. The document analysis system receives a visual question answering (VQA) prompt comprising a document image and a question defining information to be extracted from the document image, generates tokens based on the document image and the question, and adjusts encoding of the tokens using document type specific adapters of a transformer model to extract the information from the document image. A classifier of the document analysis system may determine whether the document image matches a document type supported by the document type specific adapters.

The invention relates to a vertical industry document intelligent analysis and question-answering method based on a multi-modal retrieval enhancement generation technology, which can comprehensively process various data in a document and remarkably improve the overall analysis capability of a large model on the document. By deeply analyzing multi-modal data such as texts, tables and charts in the document, the method enables document analysis to be more comprehensive and accurate. According to the method, a vector database and a graph database are combined for efficiently storing and retrieving information. The combination not only improves the ability of the system to understand the context, but also significantly improves the accuracy of retrieval. Meanwhile, the output retrieval result can provide evidence for the result obtained by the user by using the method, so that the output result of the method has acceptable accuracy in various vertical fields.

The invention discloses a document analysis and index construction method. The method comprises the following steps of layout detection, table image analysis and extraction, page recovery, content splitting and index construction and storage. According to layout detection, a document layout analysis model is used for carrying out area detection and classification on images, tables, titles and text elements of pdf document pages; the table image analysis extraction is to extract and process formats and contents of tables and images existing in each page of the pdf obtained based on layout detection; according to the method for analyzing the document and constructing the index, intelligent analysis is provided, texts, tables and images in the pdf document can be accurately extracted, and perfect index elements can be constructed.

The invention discloses a high-precision contract document analysis method based on a large model, relates to the technical field of contract document analysis, and solves the problem that an existing intelligent contract management system analyzes contract documents by utilizing advanced technologies such as computer vision and natural language processing and provides a certain automation function. However, the systems still have many limitations in the aspects of contract document processing accuracy, contract content intelligent comparison, intelligent question and answer and the like. The method comprises the steps of obtaining input data of a user; performing fine tuning on the pre-training model to obtain a query question and answer generation model, and performing processing through the query question and answer generation model based on the input data to obtain a query result; carrying out difference content identification on the input data to obtain an identification result; the technical problem is solved by performing risk assessment on the input data to obtain the assessment result.

The invention provides a multivariate document analysis method and system, relates to the field of computer information processing, and solves the technical problems of low information extraction efficiency and low accuracy caused by incapability of uniformly analyzing and integrating formats of various types of documents. The method comprises the following steps: identifying the type of a to-be-processed document; the document types comprise a table type, a text type and a demonstration type; the text class comprises a Word format and a PDF format; calling a corresponding analysis function according to the type of the to-be-processed document, and analyzing the to-be-processed document to obtain an analysis result, the analysis result comprising the extracted structure information and content data of the to-be-processed document; and converting an analysis result into a standard JSON format and outputting the standard JSON format. The method and the device are used in a document analysis process.

The invention discloses a Java-based large language model dynamic adaptation method and system, belongs to the technical field of artificial intelligence, and aims to solve the technical problems of overcoming the interface difference of multiple model interfaces, simplifying the development process and providing standardized and high-expansibility large model management. Comprising the following steps: providing a model registration and dynamic loading service, a protocol conversion and parameter standardization service and a load balancing and failover service; a streaming transmission protocol service, a function call dynamic injection service and a global error processing mechanism are provided; when a user provides a document analysis and partitioning service, a vectorization index construction service and an RAG enhanced generation service to ask questions, relevant document blocks in the Pinecone are retrieved through the RAG enhanced generation service to serve as contexts to be injected into cue words of the large language model, and answers generated by the large language model are returned.