307 results about "Document analysis" patented technology

A system and method for searching documents in a data source and more particularly, to a system and method for analyzing and clustering of documents for a search engine. The system and method includes analyzing and processing documents to secure the infrastructure and standards for optimal document processing. By incorporating Computational Intelligence (CI) and statistical methods, the document information is analyzed and clustered using novel techniques for knowledge extraction. A comprehensive dictionary is built based on the keywords identified by the these techniques from the entire text of the document. The text is parsed for keywords or the number of its occurrences and the context in which the word appears in the documents. The whole document is identified by the knowledge that is represented in its contents. Based on such knowledge extracted from all the documents, the documents are clustered into meaningful groups in a catalog tree. The results of document analysis and clustering information are stored in a database.

A method and system for retrieving information in response to an information retrieval request comprises extracting additional information from a first corpus of data elements based on the request. The request is modified based on the additional information to refine the scope of information to be retrieved from a second corpus of data elements. The information is retrieved from the second corpus of data elements based on the modified request.

A method of semantic profiling of documents comprises receiving a document to be profiled, the document comprising a plurality of terms, for each of at least a portion of the plurality of terms in the document determining a part of speech and a grammatical function of the term, obtaining senses of the term, selecting a sense as a most likely meaning of the term, and calculating an information value of the term, and generating a semantic profile of the document comprising at least some of the calculated information values.

An enterprise content management system such as an electronic contract system manages a large number of secure documents for many organizations. The search of these private documents for different organizational users with role-based access control is a challenging task. A content-based extensible mark-up language (XML)-annotated secure-index search mechanism is provided that provides an effective search and retrieval of private documents with document-level security. The search mechanism includes a document analysis framework for text analysis and annotation, a search indexer to build and incorporate document access control information directly into a search index, an XML-based search engine, and a compound query generation technique to join user role and organization information into search query. By incorporating document access information directly into the search index and combining user information in the search query, search and retrieval of private contract documents can be achieved very effectively and securely with high performance.

Systems and methods are disclosed for performing duplicate document analyses to identify texturally identical or similar documents, which may be electronic documents stored within an electronic discovery platform. A process is described which includes representing each of the documents, including a target document, as a relatively large n-tuple vector and also as a relatively small m-tuple vector, performing a series of one-dimensional searches on the set of m-tuple vectors to identify a set of documents which are near-duplicates to the target document, and then filtering the near set of near duplicate documents based upon the distance of their n-tuple vectors from that of the target document.

A document management system applies relevant document analysis, metadata extraction, and business process association algorithms and methodology to automatically and dynamically classify documents for routing, processing, and executing customized business logic. The document management system accepts documents from one or more channels, classifies the document and extracts metadata, executes customized application profiles and triggers business logic associated with the process. The document management system comprises a rules engine to detect and classify unstructured forms as well as structured forms, where the locations of attributes and visual layout are not fixed. The document management system provides automatic linkage between disparate systems that manages documents for the complete execution of a business process.

The embodiment of the invention discloses a decentralized identifier management system based on an Ethereum block chain. The system comprises an intelligent contract on an Ethereum block chain and a decentralized identifier document analysis module under the block chain, a representation form of the distributed identifier is specified in the smart contract, an attribute analysis function of the decentralized identifier is realized through a decentralized identifier document analysis module; the user can use the public key which is publicly and autonomously used by the smart contract to realizekey alternation and management; the third-party service provider website and other users can verify the authenticity of the digital signature of the user under the block chain by inquiring the data in the decentralized identifier document so as to determine the user identity; and the entity can freely use any shared trust root to manage the decentralized identifier of the entity, so that the entity has no centralized authority and no single-point fault, and has high information management security and privacy.

The application borrows terminology from data mining, association rule learning and topology. A geometric structure represents a collection of concepts in a document set. The geometric structure has a high-frequency keyword set that co-occurs closely which represents a concept in a document set. Document analysis seeks to automate the understanding of knowledge representing the author's idea. Granular computing theory deals with rough sets and fuzzy sets. One of the key insights of rough set research is that selection of different sets of features or variables will yield different concept granulations. Here, as in elementary rough set theory, by “concept” we mean a set of entities that are indistinguishable or indiscernible to the observer (i.e., a simple concept), or a set of entities that is composed from such simple concepts (i.e., a complex concept).

A document analysis, commenting, and reporting system provides tools that automate quality assurance analysis tailored to specific document types. As one example, the specific document type may be a requirements specification and the system may tag different parts of requirements, including actors, entities, modes, and a remainder. However, the flexibility of the system permits analysis of any other document type, such as instruction manuals and best practices guides. The system helps avoid confusion over the document when it is delivered because of non-standard terms, ambiguous language, conflicts between document sections, incomplete or inaccurate descriptions, size and complexity of the document, and other issues.

A document analyzer receives a collection of text-based terms associated with a document. The document analyzer performs a statistical analysis on the text-based terms to identify a distribution of where the text-based terms appear in the document and relative frequency indicating how often the text-based terms appear in the document. The document analyzer utilizes the distribution and relative frequency information derived from the statistical analysis to rank multiple themes associated with the document. For example, a received listing of multiple themes may not be presented in any useful order, although it can be assumed that the themes in the listing are present in the document. Based on application of distribution and relative frequency information derived from the analysis, the document analyzer can identify which themes are most relevant to the document as a whole and/or which of themes correspond to different portions (e.g., pages or sections) of the document.

A display apparatus including a communicator configured to perform data communication with a content server and to receive at least one of a main document and a sub document related to the main document; a document analyzer configured to extract a keyword having a high frequency of occurrence from the main document and to determine a head keyword for generating a summarized document from the extracted keyword with reference to the received sub document; and a processor configured to determine a reliability of each sentence of the main document based on the head keyword, extract a sentence that matches a predetermined condition with reference to the determined reliability, and analyze a structural format of the extracted sentence so as to re-configure a word that forms the sentence and generate a summarized sentence, thereby generating a summarized document where information and logical cohesion have been obtained.

A document analysis, commenting, and reporting system provides tools that automate quality assurance analysis tailored to specific document types. As one example, the specific document type may be a requirements specification and the system may tag different parts of requirements, including actors, entities, modes, and a remainder. The system also includes tools for visualizing the relationships between entities in a requirements specification and for identifying whether the requirements specification provides for attributes specified by a non-functional attribute glossary. The system facilitates the visualization of interactions of individual entities, of a system of entities, or entities identified for a specific use. The different types of visualizations distinguish between interacting and non-interacting entities, and highlight where a set of requirements may be deficient with respect to the non-interacting entities. However, the flexibility of the system permits analysis of any other document type, such as instruction manuals and best practices guides.

A document analysis system which can execute a layout analysis intended by a document provider and an exhaustive title analysis and output the analysis result which can be used by a third person is provided by the present invention. The input unit (11) obtains a structured or semi-structured document and renders it. The basic layout analysis unit (14) obtains the rendering result and analyzes the layout by grouping document description elements juxtaposed in a determined direction by referencing an arrangement of the document description elements. The title analysis unit (15) obtains the rendering result and a title analysis rule from the title analysis rule storing unit (23) and analyzes the title by comparing the name, attribute, style or the content of the document analysis elements with the title analysis rule. The layout analysis unit (16) obtains the layout components and the hierarchical relationship thereof and the titles for generating a new layout by grouping the layout components. The output unit (13) obtains the layout components and the hierarchical relationship thereof, the relationship between the components and the titles, shapes them into a format having an expression which uses the reference to the document description elements and output them.

A document analysis system that automatically classifies documents by recognizing in each document distinctive features comprises a document acquisition system, a document recognition training system, a document classification system, a document recognition system, and a job organization system. The document acquisition system receives jobs wherein each job containing at least one electronic document. The document feature recognition system automatically extracts image and text features from each received document. The document classification system automatically classifies recognized electronic documents by finding the best match between the extracted features of each of the document and feature sets associated with each category of document. The document recognition training system automatically trains the feature set for each corresponding category of documents, wherein the training system using extracted features of unrecognized documents automatically modifies the feature set for a document category. The job organization system automatically organizes each job according to the document categories it contains.

In a document analysis system that receives and processes jobs from a plurality of users, in which each job may contain multiple electronic documents, to extract data from the electronic documents, a method of automatically extracting data from each received electronic document using a plurality of character recognition engines is provided. The method includes: automatically processing each received electronic document page using each of a plurality of recognition engines to extract data; comparing quality of data extracted from each of the recognition engines to assign a confidence score to the extracted data; and selecting extracted data having highest confidence score as the correct extracted data.

In a document analysis system that receives and processes jobs from a plurality of users, in which each job may contain multiple electronic documents, to extract data from the electronic documents, a method of automatically pre-processing each received electronic document using a plurality of image transformation algorithms to improve subsequent data extraction from said document is provided. The method includes: electronically partitioning each received electronic document page into pieces; automatically processing each piece of the received electronic document page using each of a plurality of image pre-processing algorithms to produce a plurality of image variations of each piece; and analyzing the outputs of subsequent processing and data extraction, on each of the image variations of the pieces to determine which output is best, from the plurality of outputs for each piece.

A method in a document analysis system automatically extracts image and text features from each received electronic document and compares the extracted features with feature sets associated with each category of document to determine whether the document is recognizable as belonging to a document category. If an electronic document is recognized as belonging to one of the document categories, the method classifies the electronic document as belonging to that document category. If, however, an electronic document is unrecognized, the method submits the unrecognized document to a learning phase, in which the unrecognized document is presented to a human trainer for manual classification of the unrecognized electronic document into a document category, and automatically modifies at least one of the features and the weights of the feature set of the document category corresponding to the manually-classified electronic document using the automatically extracted features of the manually-classified document.

In a document analysis system that receives and processes jobs from a plurality of users, in which each job may contain multiple electronic documents, to extract data from the electronic documents, a method of extracting data from a received electronic document page that includes multiple copies of a form is provided. The method comprising: automatically processing a received electronic document page that includes multiple copies of a form to group the multiple copies into corresponding number of records; automatically extracting data from each of the multiple copies of the form and saving the extracted data into the corresponding record; automatically comparing the extracted data in the records to determine which copy of the extracted data to select; if all extracted data instances are identical, assigning a high confidence score to the extracted data; and, if all extracted data instances are not identical, flagging the extracted data for a further processing.

A method of training a document analysis system to extract data from documents is provided. The method includes: automatically analyzing images and text features extracted from a document to associate the document with a corresponding document category; comparing the extracted text features with a set of text features associated with corresponding category of the document, in which the set of text features includes a set of characters, words, and phrases; if the extracted features are found to consist of the characters, words, and phrases belonging to the set of text features associated with the corresponding document category, storing the extracted text features as the data contained in the corresponding document; and, if the extracted text features are found to include at least one text feature that does not belong to the set of text features associated with the corresponding document category, submitting the unrecognized text features to a training phase.