1388 results about "Collation" patented technology

The invention discloses a power plant operation and maintenance knowledge intelligent query method based on a large language model and an RAG technology. The method comprises the following steps: constructing a power plant operation and maintenance knowledge vector library covering structured, semi-structured and unstructured data; receiving a natural language question of a user, inputting an improved instruction to align a preprocessor, and generating a question semantic vector and an intention tag; relevant knowledge fragments are retrieved and sorted through a semantic matching retriever in combination with the intention labels; constructing a large language model cue word structure based on the retrieval result and the original question, generating candidate answers and recording a reference path; and finally, performing term specification and consistency verification according to the expert rule base, and outputting a structured and traceable final answer. According to the invention, the improved RAG technology is fused to realize intelligent query of the operation and maintenance knowledge of the power plant.

The invention discloses a hybrid enhanced index and system based on vector retrieval and a BM2 algorithm, and the method comprises the steps: uploading a document by a user, analyzing the document by the system to obtain text content, segmenting the text content, converting the segmented text content into dense vectors, storing the dense vectors in a vector library, extracting keywords of segmented texts by using a large model, processing the keywords, and inserting the keywords into a word list. Word segmentation is carried out on the text based on the constructed word list and a word segmentation device; in response to the received user query, vectorizing the user query, and calculating the similarity with each vector in the vector library to obtain a preliminary query result; performing word segmentation on user query, performing keyword retrieval by using a BM25 algorithm according to a word list and a word segmentation device to obtain query results, and filtering the similarity of the two query results by the reordering model according to a set threshold value; selecting a result with the highest similarity according to large model parameter limitation and an upper limit set by business requirements; and splicing the document content corresponding to the result and the user query into a cue word, inputting the cue word into the large model, and analyzing a semantic relationship and a logic structure in the cue word to generate an answer.

The invention relates to a PDF extraction method and system based on deep learning and layout analysis, and the method comprises the steps: obtaining a PDF file, carrying out the element recognition of a scanning version file and a text version file through an OCR technology and a deep learning model, and enabling the elements to comprise a text, a picture, a table and a formula; dividing a page of the PDF file into a plurality of logic blocks according to the identified elements, and determining a hierarchical relationship and a sequence; deleting elements without substantial help from the identified elements, and extracting the content of the document; performing page layout analysis according to the document content, and reordering and organizing each logic block to obtain a layout; document contents are classified and analyzed, and then classified storage is carried out; and according to user requirements, splicing the classified and stored document contents in a corresponding layout to generate a document in a target format. Compared with the prior art, efficient and accurate PDF extraction is achieved, and meanwhile the important structure and content of an original document are reserved.

The invention relates to the technical field of data management and the field of financial science and technology, and discloses a data management method based on cold data migration, which comprises the following steps: extracting metadata in a distributed file system, and importing the metadata into a metadata storage library of a distributed data processing platform. And analyzing file information in the metadata storage library through an offline analysis task, identifying cold data, and storing file path information of the cold data in a database. And according to the identified cold data, generating a data migration plan comprising a migration target position, migration priority ranking and migration time arrangement. And migrating the cold data from the original storage position to the target far-end storage, and updating a table partition path in the metadata storage library to point to a new storage position. And after migration is completed, deleting the cold data in the original storage position. According to the method, the cold data and the computing resources are effectively separated, the utilization efficiency of the storage resources is optimized, the overall resource consumption of the system is reduced, and meanwhile the expansibility and the performance of the system are improved.

The invention provides a multi-knowledge-granularity text retrieval method and device for RAG, and the method comprises the steps: sorting an original text into a JSON file, carrying out the recursive character and semantic dicing processing, generating a target JSON file, and storing the target JSON file in a BM25 database and a vector database; when a user queries, preprocessing to generate a keyword list and a query embedded vector; searching for a target text in the BM25 database, and calculating a score of correlation with the keyword to determine a first to-be-selected text; determining a second to-be-selected text according to the query embedding vector; and performing duplicate removal processing on all the first to-be-selected texts and the second to-be-selected texts, and determining and outputting a final recommended text by utilizing a sorting model. By considering the problem of different search granularities, the keyword granularity is solved by adopting a BM25 algorithm. According to the semantic retrieval strategy combining sentence granularity and segment granularity, challenges of different text information lengths and different retrieval purposes are effectively handled, and therefore the advantages of traditional retrieval and semantic retrieval are fully utilized.

The invention discloses a multi-modal literature data extraction method and device and a medium. The method comprises the following steps: firstly, standardizing literatures to obtain a structured document comprising a literature directory, author information, a paragraph text, a paragraph layout, a table LaTeX code, a formula LaTeX code and / or image description; then, related retrieval of fragments is performed on texts, formulas, tables and images from the structured document according to keywords and / or vectorization retrieval strategies. The retrieved fragments are optimized through correlation sorting, and the number of the fragments needing to enter large model processing is reduced. And finally, performing question and answer configuration on the large language model based on a user extraction demand and a sorting fragment, generating a structured answer through a prompt engineering technology, ensuring that the results can be traced through an original fragment, ensuring the availability of question and answer results, and reminding the user to verify and process a part which cannot be traced when necessary.

The invention provides a cross-domain book recommendation method and system based on a neural network, and relates to the technical field of intelligent book recommendation, and the method comprises the steps: obtaining first information containing user reading history, retrieval behaviors, research directions and paper information, and converting the first information into an initial interest topological structure and an evolution trend of a user; then, a cross-domain interest migration graph of the user is obtained through Markov clustering; and inputting the interest migration graph and the structured book text information into an extended probability latent variable semantic model to obtain a semantic embedding expression fusing user interests. Potential association scores between the user and the books are calculated through association analysis of the expression and the interest migration graph, a cross-domain recommendation list is generated according to sorting, and personalized and structured book recommendation is achieved. According to the method, deep alignment of the interest migration structure and the semantic space is realized, and the accuracy and practicability of cross-domain recommendation are remarkably improved.

The invention relates to a tree neighbor context-based big language model RAG optimization method, which comprises the following steps of: reading a directory structure and content of a document, generating a corresponding directory tree, converting the directory tree into a Markdown format containing a title and a level, and automatically extracting according to a document template if no title exists; segmenting the document into knowledge blocks according to the priority; after the knowledge blocks are recalled, the context is expanded by using a tree adjacent context method, and a new knowledge block list is formed; grouping and sorting the expanded knowledge blocks according to document identifiers, splicing reference data based on different fields and confidence coefficients, and generating a character string set from the original document; and splicing prompt words by using the constructed reference data and the new knowledge block list, and sending the spliced prompt words to the large model for processing. According to the scheme, the performance and practicability of the large model in the information retrieval and answer generation process are comprehensively enhanced by optimizing context information, emphasizing key content, adapting to various document formats, improving user experience and the like.

The invention discloses a file retrieval and management method fusing an AI large model and graph data, and belongs to the technical field of retrieval management, the retrieval and management method comprises the following specific steps: I, a user imports a multi-format file into a file retrieval and management system, and the system extracts characters and structured content through a file analysis module; iI, the file retrieval and management system gradually constructs a multi-level database based on the analyzed file content and metadata; iII, performing full-process management on the file from creation to archiving, and providing an intelligent tool to optimize a document processing process; according to the method, the automation and efficiency of file management can be greatly improved, a high-precision semantic search and sorting result is provided, complex association analysis and multi-dimensional query are supported, professional document translation requirements in an internationalized scene are met, document content management is optimized, and the knowledge precipitation capacity is improved.

The invention relates to a knowledge retrieval method, system and device based on document structure context enhancement and a medium, and the method comprises the following steps: obtaining a multi-source heterogeneous structured document, carrying out unified structure analysis and formatting, and extracting metadata and hierarchical structure information of the document; constructing a document directory tree and chapter association according to the extracted document metadata and hierarchical structure information, and segmenting chapters and paragraphs into knowledge blocks; performing hierarchical context enhancement on the knowledge blocks according to the document directory tree to form enhanced knowledge blocks; vectorizing the enhanced knowledge block by using the embedded model, and storing the enhanced knowledge block and a vectorization result thereof into a knowledge base; performing multi-path and multi-layer retrieval in the knowledge base according to a user query text, and recalling related knowledge blocks; and sorting the recalled knowledge blocks, and extracting the most relevant knowledge block to obtain a retrieval result. Compared with the prior art, the method has the advantages that the document retrieval efficiency is improved, and the like.

The invention relates to the technical field of knowledge maps, in particular to a legal knowledge map automatic construction system and method based on machine learning, and the system comprises a metadata extraction module which reads text data from a legal document, carries out the content analysis through a machine learning model, and generates a legal entity list; carrying out attribute association analysis on the legal entity list, and establishing a preliminary metadata index; according to the method, through automatic analysis and content extraction of legal document text data, recognition and efficient induction of legal entities are achieved in combination with a machine learning model, so that a legal entity list is constructed, and a legal knowledge set which is clear in structure and convenient to retrieve is formed through attribute association analysis and metadata indexing. Meanwhile, according to intelligent evaluation of the importance of the legal document content, differential sorting and storage of the documents are achieved, the archiving structure is optimized, and the convenience and response speed of data calling are improved.

The invention discloses an intelligent report generation method and system based on multi-source heterogeneous data fusion, and particularly relates to the technical field of data fusion, and the method comprises the steps: constructing a fact anchor point containing a stable identifier in a fusion layer, and solidifying an aperture version signature and a time interval; packaging a screening condition, a sorting rule, an access plan, time window limitation and the like into a controlled context packet and freezing the controlled context packet into a unique fact source; in the range, generating sentences by using a limited template and writing sentence-level reference marks, so that each sentence can be traced back to a data slice and an evidence fragment; performing consistency verification according to a source, an aperture and time, performing conservative degradation on the attribution expression according to a consistency threshold value, and outputting a degradation strategy description; and finally, page two-way pointer and cross-sentence consistency recording and audit playback are realized through an evidence chain pointer table and a published version signature, and the problems of caliber drift, cross-window access and difficulty in evidence tracing are solved.

The invention provides a distributed storage method based on source code semantic partitioning, and particularly relates to the technical field of cloud data distributed storage. The method comprises the steps of performing semantic partitioning on a source code, and segmenting the source code into a plurality of semantic blocks according to dimensions such as functional semantics, an abstract syntax tree structure, author information and version information; generating metadata containing information such as grammar type tags, file paths, line number ranges, author identifiers, version identifiers and access popularity for each semantic block; constructing a weighted directed acyclic graph (DAG) based on the semantic chunks and the dependency relationship thereof; superposing a metadata layer in the DAG structure, and recording information such as function call dependency, inter-block reference relationship and version evolution chain; blocks with relatively high access frequency and close semantics are aggregated into super blocks, the traversal depth is reduced, and meanwhile, hot data and cold data are differentiated for hierarchical storage by adopting a cold and hot data management strategy; and evaluating a parent block aggregation degree through a BDS algorithm, determining a block sorting priority, and optimizing super block boundary division. Compared with the prior art, the method has the advantages that the semantic retrieval efficiency, the incremental updating capability and the distributed query performance of the source code storage system are improved.

The invention belongs to the technical field of text data processing, and particularly discloses an LLM-based contract element extraction and auditing method and device, and the method comprises the steps: analyzing a target contract to obtain text block data, carrying out the vectorization processing of the text block data to obtain a first multi-dimensional vector, and constructing a text vector database; extracting feature information in the to-be-processed text and taking the feature information as a contract index, generating a label of the contract index according to a category to which the feature information belongs, and vectorizing the feature information and the corresponding label to obtain a second multi-dimensional vector; searching first multi-dimensional vectors similar to the second multi-dimensional vectors in a text vector database, and sorting all the second multi-dimensional vectors to generate a vector sequence; generating a to-be-checked contract according to the vector sequence and the text block data, and labeling the to-be-checked contract through the difference between the first multi-dimensional vector and the second multi-dimensional vector; the method has the advantages that the indexes can be automatically extracted, the verification result containing explanation can be given, and the working efficiency of project management personnel can be remarkably improved.

In a method for providing a visual representation of search results, a natural language query may be obtained from a user. The natural language query may be processed to obtain key query terms. Based on the key query terms, a relevance-ranked search on a plurality of documents may be performed to obtain a subset of the plurality of documents relevant to the natural language query. Answers to the natural language query may be extracted from relevant documents among the subset of documents, and graph data may be obtained using document links associated with the relevant documents from which the answers were extracted. The graph data may include entities and relationships identified in the relevant documents. The graph data may be used to generate a graph that may be displayed to the user to enable the user to discover information previously unknown to the user.

The invention discloses an extensible structured text generation method, device and equipment in the field of PLC and a medium. The method comprises the following steps: constructing a case library and an instruction library; based on semantic similarity of task description, similar task cases are retrieved from a case library through a language model and sorted according to similarity scores; for each task case, common programming instructions used by the task case are retrieved from an instruction library, and an instruction information set is generated according to corresponding instruction information to serve as knowledge of a specific field; constructing a context prompt, inputting a language model, and generating a structured text draft code; taking the structured text draft code as the input of an ANTLR analyzer for grammatical analysis, obtaining grammatical error information and constructing an abstract syntax tree; the abstract syntax tree is traversed through ANTLR Walker for semantic analysis, and semantic error information is obtained; the error information is provided for the language model, a corresponding code repairing scheme is generated, and codes are improved. According to the method, the ST code specific to the supplier can be automatically generated, and the quality of the generated code is ensured.

A method includes applying a document ranking layer of a document selection large language model (LLM) to a document list including multiple reference documents to obtain a ranked document list. The method further includes selecting a subset of reference documents from the ranked document list and processing a user prompt and the document subset by a field LLM to generate an answer. The method further includes ranking the answer with an answer score by a ranking LLM. The method further includes ranking the document subset by the ranking LLM to obtain a ranked document subset. The method further includes calculating a loss function of a preference optimization layer of the document selection LLM based on the answer score and updating at least one training parameter of a foundation model of the document selection LLM based on the loss function of the preference optimization layer.

The invention discloses a retrieval enhancement generation method and device based on reciprocal ranking fusion and a medium, and relates to the technical field of intelligent question answering and document processing. The method comprises the following steps: receiving an original query input by a user, and processing the original query by utilizing a preset first large language model to generate a plurality of extended queries; using a semantic vectorization method to retrieve corresponding related documents in a document database for each extended query; according to a reciprocal ranking fusion algorithm, calculating reciprocal ranking scores of the related documents corresponding to the extended queries, and comprehensively ranking the related documents according to the reciprocal ranking scores; and inputting the related documents, the extended queries and the original queries after comprehensive sorting into a preset second large language model, and generating question and answer output.

The invention provides a document analysis method and device, equipment and a storage medium, and relates to the technical field of computers, in particular to the technical field of deep learning, data processing and document analysis. According to the specific implementation scheme, at least one layout area divided by an article to which the document image belongs is determined according to the layout of the document image; identifying element contents of a plurality of layout elements in the document image; sorting the reading sequence of the layout elements in the same layout area; and obtaining structured document information according to the sorting result and the corresponding element content. According to the technical scheme, different article areas on the same page can be accurately identified and separated, the respective reading sequence is correctly reconstructed on the basis, and the content in the document image is converted into structured information.

The invention belongs to the technical field of text processing, and particularly relates to a method and system for retrieving DOCX document content based on keywords, which comprises the following steps: analyzing an Office Open XML structure of a DOCX document, combining with multi-dimensional features such as style names, and utilizing a title classification score model to accurately distinguish a title and a text, so that a semantic hierarchical structure of the document is effectively reserved; and secondly, a multi-level semantic extension mechanism is introduced, and a Sension-BERT, a HowNet knowledge base and a Word2Vec model are fused, so that intelligent extension of synonyms and synonyms of keywords is realized, and the recall rate and semantic understanding ability of retrieval are remarkably improved. And in addition, a BM25 model is combined with paragraph length normalization and structure position weight to calculate a correlation score, so that retrieval results are sorted more accurately and reasonably. The construction of the reverse index is combined with the position coding and compression optimization strategy, and the retrieval efficiency and the storage performance are both considered.

The invention relates to the technical field of large models, in particular to a large model cue word automatic optimization method and device and electronic device.The large model cue word automatic optimization method comprises the steps that a question input by a user is obtained, and a plurality of candidate cue words are generated based on the question; performing effect evaluation on each candidate cue word by utilizing a large model, performing descending sorting on each candidate cue word according to an effect evaluation result to determine a plurality of optimal cue words, and storing the optimal cue words; carrying out iteration on the optimal cue word in the cue word bank, carrying out distillation evaluation on the optimal cue word after iteration, and if a distillation evaluation result reaches a set threshold value, updating the number of iterations; and performing descending sorting on the optimal cue words completing the set number of iterations according to the distillation evaluation result, and outputting the first optimal cue word as a final cue word. According to the method, the cue word can be automatically optimized, the cue word quality is improved, and the key problem generation accuracy of a large model is improved.

The invention discloses an intelligent retrieval method and system for electronic archives, and belongs to the field of electric digital data processing.The intelligent retrieval method for the electronic archives comprises the following steps that search content input by a user is received, spelling errors are automatically corrected, and related words are expanded; preliminarily screening the candidate text electronic archive set based on a reverse index mechanism to obtain an initial sorting result, performing correlation resorting and extension recall on the initial sorting result through a semantic similarity model, and outputting a final sorting result; and after determining the target text electronic archive in the final sorting result, according to the uploading information of the target text electronic archive, the method has the beneficial effects that the non-text electronic archives are matched through the determined uploading information of the target text electronic archive, and the non-text electronic archives are sorted according to the association strength, so that the efficiency of sorting the non-text electronic archives is improved. The non-text format content can be identified during keyword retrieval; and the tendency of the user is predicted, so that the use experience of the user in downloading or checking the electronic file is improved.

The invention discloses a reading sequence detection method and system combining layout analysis and a language model. The method comprises the following steps: collecting a historical handwritten document image and preprocessing the historical handwritten document image; performing layout analysis on the document image, and identifying and classifying interested text areas in the document image; performing semantic similarity evaluation on different types of layout entities in combination with a language model; and detecting a reading sequence by using a sorting algorithm through document layout and semantic information. The method has the advantages that the reading sequence detection method for the historical handwritten document image is provided through combined recognition of the text and the layout, and conversion of the document from the image to the text content conforming to the reading sequence is achieved.

Systems and methods disclosed herein are enabled to dynamically generate structured documents using one or more artificial intelligence models. A computing device receives an output generation request and uses a first AI model to retrieve data chunks from source documents and applicable templates. A second AI model ranks the retrieved chunks based on one or more metrics, such as vector similarity, keyword density, and temporal relevance. A third AI model subsequently generates a response using the ranked chunks, templates, and predefined operational boundaries for each chunk. The generated response is tagged with source identifiers to enable the traceability of the response back to corresponding chunks. The system transmits, via the computing device, the response, the retrieved chunks, and / or the source identifiers.

The invention relates to a quick file retrieval method, a product, computer equipment and a storage medium. The method comprises the following steps: performing vectorization representation on text content in an original document to generate a document vector; clustering and grouping the original documents according to the similarity between the document vectors to form a plurality of document clusters; setting the dynamic priority of the original documents in each document cluster according to the updating frequency and / or query frequency of the original documents in the document clusters; establishing an index structure for the document vectors, and adjusting the order of the document vectors in the index structure according to the dynamic priority of the original document; when the query initiated by the user is responded, the query of the user is converted into a query vector, and the original document related to the query of the user is obtained by calculating the similarity of the document vectors in the index structure after the query vector and the document vectors are sequenced. Retrieval efficiency can be effectively improved, documents with high dynamic priorities can be quickly searched, and system response delay or resource consumption is reduced.

The invention provides a household appliance knowledge question-answering method and system based on retrieval enhancement generation. The method comprises the following steps: acquiring household appliance field multi-modal data from a multi-format document library; extracting text information, table information and chart information in the multi-modal data; performing domain term injection processing on the extracted information, and constructing a packet domain enhancement index; receiving a natural language question input by a user; the natural language problem is analyzed through a query optimizer, and semantic retrieval and keyword retrieval are executed in parallel; carrying out fusion processing on the semantic retrieval result and the keyword retrieval result; selecting matched document fragments by adopting a relevancy sorting algorithm; inputting the matched document fragments into a large language model to generate candidate answers; verifying the compliance and traceability of the candidate answers through a credibility evaluation module; outputting a final answer with a reference source; and storing the high-frequency questions and the final answers into a cache library to solve the problems that the answer accuracy of a knowledge question-answering system is reduced and the response efficiency is limited.

The invention discloses a multi-level batch text parallel deduplication method, system and device and a storage medium, and the method comprises the steps: carrying out the preprocessing of text data, and carrying out the similarity text representation and content deduplication of sensitive hashing at a chapter level, a paragraph level and a statement level; adding a priority score and a sorting rule in a duplicate removal process to eliminate duplicate removal randomness; during content deduplication, a graph structure is created according to similarity text representation of sensitive Hash, a plurality of connected components are established on the graph structure by utilizing combination and separation of union sets, and a target text is selected and reserved in each connected component by adopting a dynamic Hash threshold value; in the process, data frame objects are respectively constructed for chapter-level, paragraph-level and statement-level texts, unique identifiers are allocated, and the time complexity and the required memory space are optimized by adopting a parallel deduplication strategy of block delay calculation and pointer storage. According to the method, the duplicate removal speed and accuracy can be improved, and meanwhile, the problem of memory shortage faced by mass duplicate removal is solved.

The invention relates to the technical field of anti-counterfeiting watermarks, in particular to a digital intelligent water affair form text anti-counterfeiting watermark generation method based on an open source gap, which comprises the following steps of: acquiring field difference and repetition frequency, screening jitter fields to generate a perturbation table, indexing and marking sensitive segments in sections, establishing a weight structure and adjusting bit width, and sequencing nodes to generate a binding list. And constructing a watermark path chain, and matching track consistency to generate a drift label. According to the method, a field disturbance weight structure is established through proportional mapping, dynamic affinity sorting is performed in combination with node processing capacity and field task attributes, watermark track sequence binding is realized in a field path structure, and a track consistency detection mechanism is introduced in time sequence matching of field filling and node execution. The concealment of field embedding disturbance and the verifiability of a watermark writing path can be enhanced under the condition of not changing a form external display structure, and it is ensured that node selection in the field content embedding process has logical continuity and a track chain can be traced back.

The present application relates to a method for searching for a building information model. The method comprises: performing multi-modal feature extraction on building information models in a model library to be searched, and obtaining multi-modal features corresponding to various building information models, the multi-modal features comprising semantic features, topological features, and geometric features; acquiring a search text inputted by a user, parsing the search text, and obtaining search intention information corresponding to the search text, the search intention information comprising search intention semantic features, search intention topological features, and search intention geometric features; on the basis of deep embedding learning, calculating comprehensive similarities between the search intention information and the multi-modal features of the building information models in said model library; and, on the basis of a sorting result of the comprehensive similarities between the search intention information and the multi-modal features of the building information models in said model library, determining a building information model that serves as a search result corresponding to the search text, and recommending the search result to the user.

The invention belongs to the technical field of power equipment operation inspection auxiliary decision making, and particularly relates to a power operation inspection knowledge service method and device, equipment and a medium. The method comprises the following steps: analyzing semantics of query input of a user by utilizing a large semantic model, and determining a corresponding target sub-knowledge base in combination with an intention classification sample library and an intention set; performing sentence segmentation on the document of the target sub-knowledge base, generating document fragments containing continuous sentences through context supplementation, and enabling semantics of the document fragments to be overlapped; calculating a keyword correlation score of the query word and the document fragment; generating a text vector through a semantic coding model, and calculating a semantic correlation score of the query input and the document fragments; fusing the keyword correlation score and the semantic correlation score, and outputting each related document fragment according to a comprehensive score sequence; and inputting query input of a user and related document fragments into the large semantic model, and generating a structured answer in combination with the logic in the operation and inspection field of the power equipment. And the electric power operation and maintenance knowledge service with high accuracy and strong generalization is realized.