38 results about "Document representation" patented technology

The invention discloses an RAG intelligent retrieval question-answering system and method based on enhanced metadata, and relates to the technical field of information processing and intelligent retrieval, multi-source heterogeneous knowledge data is preprocessed to obtain unified knowledge data, and structured metadata is extracted from the unified knowledge data based on different text forms; vectorizing a document text in the structured metadata by combining with embedding of the knowledge graph to obtain document representation, and outputting the document representation, the structured metadata and the enhanced keyword set as an enhanced metadata object; labeling a display relationship between different enhanced metadata objects, and constructing to obtain a knowledge database; according to the intelligent knowledge service system and method, restrictive conditions and question intentions are extracted from user questions, mixed retrieval is performed from a knowledge database based on the restrictive conditions and the question intentions, a candidate literature semantic set is output, then structured statistical visualization reports and structured answers are output, and accurate, explainable and multifunctional intelligent knowledge services are achieved.

The invention provides a full-text retrieval method and system fusing various types of documents, and relates to the technical field of information retrieval, and the method comprises the following steps: obtaining document representation through document content extraction and structure recognition, generating a cross-modal semantic vector by using word embedding and nonlinear transformation, constructing a hierarchical index and a cross-document association graph, and obtaining a full-text retrieval result; the basic correlation score is calculated after the query request is received, and the comprehensive score of the candidate content segments is calculated based on the association graph to determine the optimal retrieval result, so that unified representation and retrieval of heterogeneous documents are realized, the cross-document retrieval precision and relevance are improved, and the processing capability of a retrieval system on complex queries is enhanced.

The present invention discloses a system and method for processing trade finance documents and performing automated compliance screening. The system comprises a computing device, and a database for storing trade finance documents. The system processes documents using OCR to extract text and positional data, generating structured document representations via a layout-aware AI model. An AI classifier module categorizes documents based on content, layout, and domain-specific roles, while a semantic verification module aligns document data with master Letter of Credit templates. A rule management module validates compliance against international trade standards, and a financial crime risk control module performs real-time checks against external sanctions, vessel, and dual-use goods databases. The system further determines and reports discrepancies, anomalies, and compliance issues. The system supports heterogeneous layouts, multi-language documents, and integration with banking APIs.

The invention relates to the technical field of mathematical modeling, and discloses a modeling method for a power business decision model, and the method comprises the steps: constructing a power corpus block library and a power vector knowledge library, and determining a power business problem text; performing mixed retrieval on the power business problem text by means of a power corpus block library and a power vector knowledge library to obtain a first similarity score corresponding to a candidate corpus block and a second similarity score corresponding to a candidate document representation vector; screening the candidate corpus blocks and the candidate document representation vectors based on the first similarity score and the second similarity score, and determining a target reference text; inputting the power business problem text and the target reference text into a target large language model to obtain a target decision model; the target decision model comprises a description text of the power business problem, a target function and constraint conditions thereof. The method has the beneficial effects that the reliability and the accuracy of the target decision model are remarkably improved based on standard output of the hybrid retrieval and the large language model.

Systems and methods for document processing that can process and understand the layout, text size, text style, and multimedia of a document can generate more accurate and informed document representations. The layout of a document paired with text size and style can indicate what portions of a document are possibly more important, and the understanding of that importance can help with understanding of the document. Systems and methods utilizing a hierarchical framework that processes the block-level and the document-level of a document can capitalize on these indicators to generate a better document representation.

The invention discloses a multi-page bidding document-oriented classification and key information extraction method and system, and the method comprises the steps: obtaining bidding document image data containing a plurality of pages, and constructing a corresponding document representation data set; performing structured modeling on bidding document image data corresponding to each page in the document representation data set, extracting and fusing multi-modal features, and constructing unified page representation; based on the unified page representation, document module type classification is carried out on the pages, and structure category labels which all the pages belong to are obtained; on the basis of the structure category labels of the pages, in combination with the multi-modal features of the corresponding pages, key information extraction operation is executed, and target field information contained in the corresponding pages is extracted; and outputting a structured result, wherein the structured result comprises the structure category label to which each page belongs and the corresponding target field information. According to the method, accurate classification of the pages in the bidding and tendering document is realized, targeted key information extraction is performed, and the method has relatively high accuracy, maintainability and actual landing capability.

The invention discloses a scientific literature topic discovery method and device based on LLM. The method comprises the following steps: 1) acquiring text representation of each scientific literature sample and encoding the text representation by using a text encoder to obtain a document representation matrix corresponding to the scientific literature sample; 2) clustering the scientific literature samples to obtain clustering results of different themes; calculating an entropy value of each scientific literature sample, and selecting a high-uncertainty sample; 3) calculating semantic similarity between each high-uncertainty sample and other scientific literature samples, and constructing a plurality of triple tasks; fine-tuning the text encoder by using each triple task through a comparative learning method; 4) encoding the text representation of each scientific literature sample by using a text encoder to obtain a document representation matrix corresponding to the scientific literature sample; and 5) performing topic clustering on each scientific literature sample by using the document representation matrix of each scientific literature sample, and generating a clustering tag and a topic division result of each scientific literature sample.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for video storage optimization. One of the methods includes obtaining data for a plurality of video files of a content delivery system, wherein the plurality of video files corresponds to a set of video ladders, each video ladder identifying a respective transcoding version of video content represented by a video file of the plurality of video file and having different parameters; for each video file, executing one or more respective storage strategies to compute one or more respective output video scores, wherein each storage strategy uses trained models to evaluate characteristics of the respective video file and estimate future viewing of the video file; and in response to evaluating the output video scores computed for each of the plurality of video files, determining one or more actions to reduce storage for the plurality of video files.

The invention discloses a multi-modal file label automatic generation method, and relates to the technical field of label generation, and the method comprises the steps: firstly constructing a multi-level file label category set as a label matching reference library, then uploading a target multi-modal file, preprocessing a text extracted according to the file, and constructing a file representation vector; finally, the computing power of the current equipment is judged, labels are generated according to scenes, if the computing power is limited, the labels are extracted step by step, and a multi-level candidate label set is generated; if the computing power is sufficient, combining the multi-level label category sets, and directly extracting a candidate label set; thirdly, modifying the word segmentation result by calculating the cohesion degree of adjacent words in the initial word segmentation result, and automatically extracting the multi-level file label again; and finally, calculating a comprehensive cosine similarity value of the generated candidate file tags, and outputting an optimal file tag. According to the invention, effective utilization of resources can be realized, and the accuracy of the generated file tag is improved.

The invention discloses an English reading understanding automation problem generation method, and relates to the technical field of problem generation, and the method comprises the following steps: S1, data collection and extraction, S2, dependency syntax graph generation, S3, document representation determination, S4, problem generator training, and S5, problem statement determination. According to the method, feature fusion is carried out on the document and the syntactic graph, so that the final document representation can better conform to semantics expressed by the document, when a decoder analyzes the document representation to generate problem statements, the actual content of the document can be more accurately known, and the user experience is improved. Meanwhile, a parameter fine tuning algorithm is used for continuously adjusting the problem generator, so that the problem generator can be optimized in continuous operation, the problem statements provided during specified document analysis can be adopted by most users, a manual auditing function is provided, and the user experience is improved. Problem statements and documents meeting conditions can be stored in the data set, and it is guaranteed that the data set can be updated on time.

The invention discloses a long text abstract generation method based on hierarchical graph comparison themes, which comprises the following steps of: 1, preprocessing an original document, dividing sentence sequences, and obtaining global context-aware sentence and document representation through a hierarchical encoder network; 2, deducing document-level and sentence-level topic distribution by using a neural topic model; and 3, constructing a supervision graph based on a standard abstract to perform graph comparison learning so as to close the topic representation of a document and a key sentence and push redundant information. According to the method, the deep semantic structure of the long document can be effectively captured, so that the theme consistency and the information coverage degree of the abstract can be improved, and the redundancy is reduced.

A document processing method and apparatus. The method comprises: determining the frequency x of the qth sentence in the N sentences in the i-th document based on the similarity between the sentence in the i-th document and N sentences in M ​​documents. i,q , where M and N are both integers greater than 1, q is a positive integer not greater than N, and i is a positive integer; based on the frequency x corresponding to the qth sentence in the i-th document i,q , determine the document representation of the i-th document. This can improve the efficiency of document representation.

This application relates to the field of mathematical modeling technology and discloses a modeling method for a power business decision-making model. The method includes: constructing a power corpus segmentation library and a power vector knowledge base to determine the power business problem text; performing a hybrid retrieval of the power business problem text using the power corpus segmentation library and the power vector knowledge base to obtain a first similarity score corresponding to candidate corpus segments and a second similarity score corresponding to candidate document representation vectors; filtering the candidate corpus segments and candidate document representation vectors based on the first and second similarity scores to determine the target reference text; and inputting the power business problem text and the target reference text into a target large language model to obtain the target decision model. The target decision model includes a description of the power business problem, an objective function, and its constraints. Its beneficial effect is that, based on hybrid retrieval and the standardized output of the large language model, the reliability and accuracy of the target decision model are significantly improved.

Systems and methods are provided for learning a classifier for annotating documents with predicted labels under an extreme classification where there are more than one million labels. The learning includes receiving a joint graph comprising documents and labels as nodes. A multi-dimensional vector representation of the documents (i.e., document representations) is generated based on graph convolution of the joint graph. Each document representation varies in dependence on adjacent nodes to accommodate context. The document representations are feature transformed using a residual layer. A per-label document representation is generated from the transformed document representations based on adjacent label attention. The classifier is trained for each of the more than one million labels based on joint learning using training data and the per-label document representations. The trained classifier performs highly efficiently compared to other classifiers trained using disjoint graphs of documents and labels.

The invention belongs to the technical field of document retrieval, and particularly relates to a multi-modal long document retrieval method based on hierarchical representation and double-branch retrieval. According to the method, three-level hierarchical document representation composed of an element level, a page level and a chapter and section level is constructed by designing hierarchical features oriented to a document structure; a double-branch retrieval mechanism is used for conducting semantic-level matching and feature-level matching on queries, and a final evidence set is generated through a fusion strategy. According to the method, the cross-page semantic association and the fine-grained visual clues can be utilized at the same time, effective retrieval and information positioning of the multi-modal long document are achieved, the retrieval requirements of documents with different layout forms and different lengths in a real scene are met, and the method has important practical application value.

The invention discloses an intellectual property information analysis and management method based on multi-dimensional data, and the method comprises the following steps: obtaining patent original data, constructing a patent record, and carrying out the preprocessing of an intellectual property data set; dividing the patent text data into a plurality of coding units according to a preset field range, and constructing vectorization representation for each patent; generating an adjusted sub-query set for the retrieval input based on a query decomposition algorithm; obtaining a sub-query mark sequence and a comprehensive candidate result set; and calculating a comprehensive sorting score, and generating a retrieval result list. According to the method, by introducing a patent retrieval framework combining multi-dimensional query decomposition and ConstBERT fixed multi-vector document representation, executable sub-query configuration can be stably generated and fine-grained semantic matching can be realized in a scene in which a long retrieval type, multiple limits and exclusion conditions coexist; and the accuracy, controllability, traceability and large-scale application performance of patent retrieval are improved.

The application provides a patent automatic classification method and device, a terminal and a storage medium, and comprises the following steps: performing dependency syntax analysis on a patent document to be classified to obtain pretreatment information; performing word embedding on the pretreatment information to obtain document representation; and obtaining a flow category prediction result, a function category prediction result and a meta-function category prediction result based on the document representation and trained flow classifiers, function classifiers and meta-function classifiers. Through dependency syntax analysis and word embedding on the patent document to be classified, the flow category prediction result, the function category prediction result and the meta-function category prediction result are obtained by using machine learning technology, the comprehensiveness of the classification of the patent document to be classified is realized, and the problem that the patent automatic classification method in the prior art is very limited and cannot comprehensively classify patents is solved.

This application relates to the field of document retrieval technology, and particularly to a multimodal document retrieval method and apparatus based on a cross-modal mutual attention mechanism. The method includes: modeling a multimodal representation of a document; obtaining a target document-perceived multimodal document representation based on a multimodal mutual attention mechanism; fusing the document's self-attention vectorized representation and multimodal enhanced vectorized representation to obtain a unified multimodal enhanced representation of the document; calculating and ranking the relevance scores between the target document and at least one candidate document; and retrieving relevant documents. This application's embodiments, based on a cross-modal mutual attention mechanism, can obtain matching documents by acquiring a unified multimodal enhanced representation of the document and calculating relevance scores, thereby fully utilizing the multimodal information of the document, enhancing the relevance between different modalities of the document, and thus improving the matching degree of the document retrieval results, making the retrieval results more accurate and reliable.

The application provides an event-based document retrieval method and device, electronic equipment and storage medium, wherein the method comprises: obtaining a user query statement of a user for a document set to be retrieved; inputting the user query statement into a pre-trained large language model to obtain a document retrieval result; wherein the large language model is obtained by training and optimizing a training sample data set composed of a document representation and a document identifier, the document representation is obtained by events and event relationships in the document set to be retrieved, and the document identifier is obtained by mapping the events in the document set to be retrieved to an event hierarchy. The method considers the relevance between document contents, effectively represents the document to be retrieved by using events and event relationships, and significantly improves the document retrieval performance of the large language model; the event hierarchy is used to construct a document identifier with a clear semantic structure, effectively strengthening the connection between the document identifier and the document content.

The present application relates to a kind of medical data index query sentence generation method and system based on statistical index;The present application is defined by attribute to statistical index, query syntax is defined, using ANTLR4 is expressed using.g4 file to query syntax, and.g4 File is generated Java syntax analysis code, the syntax in it is converted into logic execution operator, defines executor interface specification, encapsulates into the platform executor plug-in of each platform, defines task manager, identifies and registers platform executor, calls platform executor and obtains intermediate data set IDS from platform, processes into final index data set KDS with intermediate data set IDS, corresponding statistical index attribute and logic execution operator;Through the present application, the standard generation of the query language of different platform data is realized, and the programming workload of query language is reduced.

The application discloses an RAG intelligent retrieval question and answer system and method based on enhanced metadata, relates to the technical field of information processing and intelligent retrieval, and obtains unified knowledge data after preprocessing of multi-source heterogeneous knowledge data, extracts structured metadata from the unified knowledge data based on the difference in text form; the literature text in the structured metadata is vectorized in combination with embedding of a knowledge graph to obtain document representation, and then the document representation, the structured metadata and an enhanced keyword set are output as an enhanced metadata object; a display relationship between different enhanced metadata objects is marked, and a knowledge database is constructed; restrictive conditions and a question intention are extracted from a user question, mixed retrieval is performed from the knowledge database based on the restrictive conditions and the question intention, a candidate literature semantic set is output, and then a structured statistical visual report and a structured answer are output, so that precise, interpretable and multifunctional intelligent knowledge service is realized.

This invention discloses a knowledge encoding and activation method and apparatus based on a large language model, belonging to the field of natural language processing technology. First, the invention generates pseudo-queries for the input document set, describing the document content from multiple semantic dimensions to construct a document semantic representation similar to the user query. Then, the pseudo-queries are used to train the large language model, encoding the document semantic knowledge into the model parameters to form a parameterized knowledge representation. Further, reinforcement learning training guides the language model to activate internal knowledge related to the user query during the generation process. Finally, in the inference stage, the model's internal knowledge is activated based on the user query to generate the output result. This method reduces reliance on external retrieval modules, improves the semantic matching ability between document representation and user query, and thus enhances the accuracy and stability of the generated results.

The application discloses a document multi-level label generation method and device, computer equipment and a readable storage medium, relates to the technical field of natural language processing, and can be specifically applied to the financial and medical fields. Through multi-level clustering marking processing, different granularity feature labels of the document can be accurately captured, and the demand of the financial and medical fields for generating document labels that can finely represent the document is better met. The method comprises the following steps: obtaining a target document to be generated with a document label, and obtaining a pre-generated multi-level label codebook; generating an initial document representation vector for the target document based on a document content summary and a document internal matching diagram of the target document; performing hierarchical iterative processing on the initial document representation vector by using the multi-level label codebook, determining the nearest clustering center for the initial document representation vector from a plurality of clustering centers included in each level, generating a multi-level document label corresponding to the target document, and outputting the multi-level document label.

The invention discloses an information extraction traceability method and system based on a native multi-modal large model and a readable storage medium, and belongs to the technical field of artificial intelligence and multi-modal learning. Comprising the steps of determining a file representation form for a to-be-processed original image, performing coordinate labeling by adopting a mantissa outward rounding mode, and constructing a space labeling data set; inputting an image by using the spatial annotation data set and adopting a dynamic resolution input strategy, and training a multi-modal large language model; inputting a sample image to the trained multi-modal large language model, outputting text content and coordinate information of the sample image, and performing coordinate post-processing on the coordinate information to realize edge compensation; and outputting the text content and the post-processed coordinate information. According to the method, the problems of coordinate dislocation, information loss and the like in a traditional OCR + language model two-stage architecture can be solved, and meanwhile, the defects of precision deviation and non-uniform format of an existing MLLM in space coordinate generation are overcome.

A technique for consistent entity tagging is provided for data access utilizing multiple protocols. In one example, a file storage system is configured to process data according to (multiple) file storage protocols and (multiple) object storage protocols. Object storage protocols can utilize entity tags indicating whether an object (using a file representation in the file storage system) has been modified. In the case of modifying a file using a file storage protocol, an indication that the file lacks a valid entity tag can be stored. If an object storage operation is performed to retrieve an object, and if the object corresponds to a valid entity tag, that entity tag can be returned as part of the response. If the object does not correspond to a valid entity tag, the file storage system can generate a new entity tag and return the newly generated entity tag as part of the response.

The invention discloses a multi-label classification method and device for collaborative attention and prototype alignment, and belongs to the technical field of natural language processing. The method comprises the following steps: firstly, obtaining lexical element level context representation by utilizing a pre-training semantic encoder, calculating a maximum correlation score of lexical elements and a label space based on label embedding, and generating a filtering mask to sparise a sequence and suppress redundant noise; then constructing a label attention branch and a sentence-level hierarchical self-attention branch in parallel, realizing cross-branch fine-grained semantic alignment through bidirectional collaborative attention, and completing feature fusion by adopting adaptive weighting; on the basis of fusion representation, a label prototype gating fusion and momentum type online updating mechanism is introduced, a learnable label prototype is used as a semantic center to continuously guide document representation to gather towards related labels, and long-tail distribution and semantic drift are relieved. According to the method, the robustness of low-frequency label prediction can be enhanced while the classification precision and the sorting quality are improved, and the unnecessary attention calculation overhead is reduced.

Methods, systems, and computer-readable medium for protecting the identity of users and entities. One method includes receiving an entity profile request, receiving a plurality of attributes associated with one or more members of the entity, and receiving / identifying a plurality of access credentials of the one or more members of the entity. The method includes generating an entity profile including a plurality of member identity passes, generating a GUI including a representation of the entity profile, and receiving a request and at least one of the plurality of access credentials. The request is for an exchange or providing at least one of the plurality of attributes. The method includes verifying that access credentials correspond to an access level to perform the request, accessing a data storage, transmitting or permitting access to information to satisfy the request, and updating a usage log.

Embodiments of the present application provide a data migration detection method, device, system and storage medium, the method comprising: obtaining information digests of next layer sub-files of a source file and a target file respectively; the source file represents a file package of migrated data; the target file represents a file package obtained after migration of the source file; the information digest of the sub-file comprises a size of the sub-file, a storage location of the sub-file and a number of next layer files of the sub-file; generating an information digest of the source file based on the information digests of the next layer sub-files of the source file; generating an information digest of the target file based on the information digests of the next layer sub-files of the target file; and determining a data migration detection result according to the information digest of the source file and the information digest of the target file.

The embodiment of the application provides a deep learning model explanation method, device and equipment, the method comprises the following steps: loading a target deep learning model according to a loading mode corresponding to the target deep learning model, obtaining a model file of the target deep learning model; performing convolution, uniformization, pooling and activation processing on the model file of the target deep learning model, obtaining an analysis file corresponding to the model file; reconstructing the analysis file, obtaining a reconstructed file corresponding to the analysis file; and representing the reconstructed file as a structure description file containing only structure description and a weight file arranged according to the order of operators in the structure description file. The method provided by the application can express models under different training frameworks as unified network structure representation and unified parameter arrangement order, improving the efficiency of deep learning model deployment.

Recovering symbols from a stripped binary includes representing the stripped binary as a plurality of graph of graphs (GoG) representations, converting the plurality of GoGs into a plurality of expressive representations of each function in the stripped binary, training a machine learning (ML) model using the expressive representations, and determining a missing symbol of at least one of the functions based on an output of the ML model. Information relating to functions is and interactions between functions are used to train an ML to determine similarity of two functions. Based on similarities, missing symbols may be inferred and used to enable updates to the stripped binary file where source code is not available.