5212results about "Text database indexing" patented technology

The invention belongs to the technical field of multi-modal data processing, and particularly relates to a multi-modal document retrieval enhancement generation method based on a large model, which comprises the following steps: receiving query content input by a user for a multi-modal document; processing the query content by adopting an embedded model, generating a query vector representing user query semantic information, and storing the query vector in a vector database; analyzing the multi-modal document to obtain long text information, segmenting the long text information into data blocks by adopting a recursive partitioning strategy, and numbering and marking the data blocks; carrying out vectorization processing on the data blocks by adopting an embedded model to generate document vectors, storing the document vectors into a vector database, and constructing a hierarchical index structure; retrieving in a vector database based on the query vector, and returning a retrieval result; and processing a retrieval result by utilizing a large language model to generate response content conforming to the query intention of the user. According to the method, the multi-modal document can be effectively analyzed and processed, and the accuracy and comprehensiveness of analysis are improved.

The invention discloses a traffic engineering large model intelligent question answering system and method based on a knowledge graph, and the method comprises the steps: extracting a structured degree feature, a semantic ambiguity feature and a context association feature through receiving and analyzing a natural language query statement inputted by a user, generating a retrieval intention vector, and carrying out the retrieval of the retrieval intention vector; and dynamically selecting a retrieval path according to the intention classification model. And according to the retrieval path, constructing a structured query statement or a semantic vector, and respectively retrieving in the knowledge graph and the vector database to obtain a first retrieval result and a second retrieval result. Further performing bidirectional verification through entity consistency, semantic similarity and relation connectivity indexes, screening a candidate result set, and constructing a reasoning chain; if the inference chain is broken, a large model inference gap complementation mechanism is adopted to generate relay nodes, a complete inference chain is formed, and inference type answer output is generated based on the complete chain. According to the method, the retrieval accuracy and reasoning continuity of the question-answering system are improved.

The invention discloses an interactive question intelligent agent system based on a large language model, and relates to the technical field of dialogue interaction systems. Aiming at the technical defects of the traditional BI tool, the technical scheme is as follows: a heterogeneous data management engine realizes unified access and secure access of cross-source data; the NLP semantic recognition engine converts a natural language into structured semantics through multiple steps, drives the hybrid SQL generation engine and provides visual parameters; the generation engine realizes precise generation from semantics to SQL through a two-stage architecture based on metadata and authority rules; and the visual management console is combined with multi-party configuration and parameters to visually build a number-asking agent. The method is used for realizing intelligent conversion from a natural language to a structured query language (SQL) and a data service interface, and automatically generating an interactive data visualization result.

The invention provides an enhanced document generation and retrieval method based on a knowledge graph, and relates to the technical field of knowledge graphs, and the method comprises the following steps: receiving heterogeneous document input, and carrying out dynamic segmentation on the heterogeneous document; constructing a Graph-RAG model, wherein the Graph-RAG model comprises a knowledge graph-document index extraction module and a design document generation module; constructing and updating a knowledge graph based on document segmentation through a knowledge graph-document index extraction module; extracting feature vectors of the document segments by using a feature extraction model, establishing a mapping relationship between the document segments and entities in the knowledge graph through a matching strategy, and constructing a retrieval index; and receiving a query sequence of a user, retrieving the related entities and document segments through the retrieval index, and generating a target design document in combination with a preset template. According to the method, accurate retrieval and high-quality document generation based on semantic understanding are realized, and the technical problems of insufficient semantic understanding, low knowledge association degree, poor generation quality and the like in a traditional document processing method can be solved.

The invention provides a meteorological metadata storage method and system based on machine learning, and the method comprises the steps: obtaining a meteorological metadata set of a target region, inputting the meteorological metadata set into a pre-trained time sequence feature extraction model, generating the time sequence distribution features of all meteorological file files, and inputting a spatial topology mapping model, generating spatial correlation topological features of the meteorological elements; based on a cross fusion result of the time sequence distribution features and the space correlation topological features, a hierarchical index structure of meteorological metadata is constructed, a meteorological storage optimization model is called to generate a storage node distribution strategy of a meteorological file, and the meteorological file is stored according to the storage node distribution strategy. And writing each meteorological file file in the meteorological metadata set into the corresponding distributed storage node, and dynamically adjusting the mapping relationship between the physical position and the index of the stored file based on a real-time updating mechanism of the hierarchical index. The meteorological metadata storage efficiency and retrieval accuracy can be improved, and storage resource waste and operation and maintenance cost are reduced.

The invention provides a knowledge graph generation method and system for science and technology project risk control, and the method comprises the steps: obtaining a multi-source heterogeneous data set of a target science and technology project, converting structured index data into a standard vector sequence through a heterogeneous data fusion mechanism, and converting unstructured text data into a semantic vector sequence; converting the time sequence behavior data into a behavior pattern vector sequence, inputting the three into a risk quantitative evaluation model, generating a risk entity feature matrix and a risk association strength matrix, and determining a node distribution topology of the knowledge graph according to entity feature vectors in the risk entity feature matrix; and according to association strength values in the risk association strength matrix, determining an entity relationship topology of the knowledge graph, generating a dynamic knowledge graph of the target science and technology project, and identifying a potential risk propagation path in the dynamic knowledge graph. According to the invention, the risk identification result has the dynamic characteristic of real-time updating, and the traceability of the multi-dimensional risk characteristic is maintained.

A system, such as an online system, allows users to ask natural language questions requesting information stored in a database. The system receives a natural language question. The system determines database tables and database queries associated with the natural language question. The system generates a prompt for input to a machine learned language model. The prompt specifies the natural language question, information describing database tables, and the example database queries. The system sends the prompt to the machine learned language model for execution and receives a response generated by the machine learned language model. The response includes a database query corresponding to the natural language question. The system sends the database query for execution on a database system and provides the result of execution of the database query to the client device.

The embodiment of the invention discloses a retrieval generation method and device based on a large language model and a knowledge graph. The problems that retrieval semantic relevance is weak, and recalled data lacks global knowledge can be solved. The method comprises the following steps: performing vector analysis on a plurality of preset domain documents, and constructing a vector database; constructing a knowledge graph according to the entity description information and the relationship description information extracted from the plurality of field documents, and storing the knowledge graph into a graph database; performing feature extraction on a target question input by a user to obtain a multi-dimensional keyword corresponding to the target question; retrieving the multi-dimensional keywords through a preset search engine, a vector database and a graph database to obtain a target retrieval result; and determining target input information according to the target retrieval result, and inputting the target input information into the target language model to obtain target output response information.

Disclosed herein are systems and methods for graph-based dynamic information retrieval and synthesis, particularly for enhancing the response generation of chatbot applications using large language models (LLMs). The methods involve receiving a user query, generating a query embedding, and accessing a knowledge graph comprising nodes representing documents with associated document embeddings. A set of documents is retrieved based on similarity scores, and a distance matrix is constructed to calculate shortest-path distances between nodes. Documents are clustered to identify a largest cluster from which content is retrieved to formulate a response. The disclosed embodiments improve upon retrieval augmented generation (RAG) by leveraging the hierarchical relationships of a knowledge graph and node interconnectedness to provide contextually coherent information and mitigate the inclusion of information from irrelevant or superficially related documents in LLM responses.

The invention discloses an archive knowledge base construction and retrieval method and system based on multi-modal data fusion. The method comprises the steps that heterogeneous archive data are cleaned, image features are extracted through CNN, text features are extracted through Transform, audio is converted into text and then subjected to similarity, a unified feature vector is generated, and metadata is constructed according to archive code association; creating a graph database instance, defining nodes and relationship types, importing entities and relationships, and storing feature vectors and metadata; the features are mapped to a high-dimensional shared semantic space, positive and negative sample pairs are constructed to update embedded layer parameters, self-attention is used in modalities, a shared attention mechanism is used between modalities, weights are adjusted according to archive features, and unified knowledge representation is generated; segmenting the steering quantity of the multi-modal data, storing the steering quantity into a database, and adopting hierarchical indexing and optimizing as required; related document fragments are retrieved through RAG technology vectors, answers are generated with the help of a large language model, and session feedback is provided. The file retrieval efficiency and accuracy are improved.

The invention provides a large model deployment method and system based on a multi-level cache mechanism, and the method comprises the steps: building a first-level dynamic semantic cache, preferentially feeding back a reconstruction answer when a new request is matched, starting a second-level context association cache for a missed request, and starting a second-level context association cache for a missed request; the method comprises the following steps: extracting an implicit semantic trajectory vector and constructing a dynamic association graph to identify a potential context dependency relationship, realizing progressive answer splicing, if coverage is still not performed, activating a third-level intention decision cache, performing scene classification and state marking by utilizing a multi-level analysis structure, and finally performing classification and state marking by utilizing a multi-level analysis structure. Based on a knowledge topology network, a query instruction is generated, multi-modal response elements are extracted to generate a combined response, a system synchronously executes multi-level cache collaborative optimization, a cache strategy is adjusted according to user feedback, cache data are adaptively expanded and eliminated, and a multi-level cache is obtained. According to the technical scheme provided by the embodiment of the invention, the response speed, the context understanding capability and the personalized service level of large model deployment are remarkably improved.

The invention relates to the technical field of archive management, and discloses an AI and encrypted storage-based archive management method, which comprises the following steps of: preprocessing original archive information by using a natural language processing algorithm to generate structured archive data, and performing multi-level classification through a dynamic classification tree model. Data are encrypted by adopting a hybrid encryption technology, and encrypted archive data blocks are constructed and stored in a distributed database. And performing parallel retrieval by using a multi-modal retrieval model, performing decryption according to user permission and scenes based on a dynamic decryption strategy, and finally transmitting data through a secure channel. In addition, the system also has the functions of file data updating and version control, log access and anomaly detection, cross-mechanism file sharing and the like. The efficiency, safety and intelligent level of archive management are improved, and the problems existing in traditional archive management are effectively solved.

A question-answering method and apparatus based on a large language model. The method comprises: acquiring a current dialogue state, wherein the current dialogue state comprises query text in a current dialogue; on the basis of the query text, performing retrieval from a memory chain database, so as to obtain target memory chain data, wherein the memory chain database comprises a plurality of pieces of memory chain data, the plurality of pieces of memory chain data being obtained by means of performing knowledge extraction on historical dialogue data; obtaining prompt text on the basis of the query text and the target memory chain data; and by taking the prompt text as an input of a large language model, outputting an answer corresponding to the query text. In the question-answering method based on a large language model provided in the present application, memory chain data is constructed by means of performing knowledge extraction on historical dialogue data, and the memory chain data assists a large language model with generating better answers, thereby improving the usage experience of users.

The invention discloses a hybrid enhanced index and system based on vector retrieval and a BM2 algorithm, and the method comprises the steps: uploading a document by a user, analyzing the document by the system to obtain text content, segmenting the text content, converting the segmented text content into dense vectors, storing the dense vectors in a vector library, extracting keywords of segmented texts by using a large model, processing the keywords, and inserting the keywords into a word list. Word segmentation is carried out on the text based on the constructed word list and a word segmentation device; in response to the received user query, vectorizing the user query, and calculating the similarity with each vector in the vector library to obtain a preliminary query result; performing word segmentation on user query, performing keyword retrieval by using a BM25 algorithm according to a word list and a word segmentation device to obtain query results, and filtering the similarity of the two query results by the reordering model according to a set threshold value; selecting a result with the highest similarity according to large model parameter limitation and an upper limit set by business requirements; and splicing the document content corresponding to the result and the user query into a cue word, inputting the cue word into the large model, and analyzing a semantic relationship and a logic structure in the cue word to generate an answer.

The invention relates to a multi-data standard definition conflict resolution method based on a large model and a knowledge graph, and belongs to the field of public data governance and data standardization. The method comprises the following steps: extracting and preprocessing a multi-data standard definition; defining semantic representation and large model analysis: performing semantic analysis on the definition of each data item by using a large model, and generating a semantic embedding vector and a key feature; knowledge graph construction and entity alignment: constructing a knowledge graph containing all data items, and connecting data item nodes with similar semantics in different standards by adopting an entity alignment algorithm to form a candidate alignment relationship; conflict detection and type identification: aiming at the aligned data item definition, comparing attributes and values, identifying definition conflict points, and classifying and marking conflict types; and carrying out conflict resolution and unified definition generation by using a globally optimized conflict resolution algorithm. According to the method, the problems of data islands and semantic conflicts caused by inconsistent data standard definitions in the prior art are solved.

The invention provides an agent-based large language model retrieval enhancement generation system and method, and the system comprises a planning layer which is used for receiving user query, carrying out the multi-round iterative decomposition of a complex task through a task planning agent, and generating an atomic query or a direct response; the execution layer is used for executing the atomic query generated by the planning layer in parallel, calling a search module to obtain external knowledge base data, and caching an intermediate result through a memory module; the answer detection module is used for performing multi-dimensional detection on the generated result, including preference, accuracy, integrity and logicality; the dynamic decision-making module is used for adaptively adjusting a subsequent retrieval strategy and a task planning process according to a detection result and user feedback; and a cross-layer interaction mechanism enables the planning layer and the execution layer to realize collaborative optimization through context sharing and iterative feedback.

Certain aspects of the disclosure provide techniques for information retrieval for large datasets. A method comprises receiving input text files and a query for the files; obtaining an index associated with the input text files to process the query, wherein: the index comprises key-value mappings, each key of a respective mapping identifying a voronoi cell of the index, and each value of a respective mapping identifying vector embeddings associated with text files associated with a voronoi cell of the index; creating a query embedding based on the query; identifying a first key-value mapping having a first key associated with a first voronoi cell in the index and corresponding to the query embedding; obtaining a set of vector embeddings associated with the first value; comparing the query embedding to the set of vector embeddings to determine closest vector embeddings; and generating a textual output based on the closest vector embeddings.

The invention discloses a large language model knowledge base question answering system based on a multi-path fusion recall retrieval algorithm, and relates to the technical field of artificial intelligence application, and the system comprises the steps of S1, data collection, S2, data preprocessing, S3, knowledge base construction, S4, analysis and clustering, S5, knowledge recall, S6, weight adjustment, and S7, weight fusion. According to the method, a dynamic weight distribution module is arranged, so that various paths of recall weights such as keyword matching, semantic similarity and a knowledge graph can be flexibly adjusted according to different semantic scenes such as a technical scene and a product query scene, a recall result can be effectively optimized, knowledge related to problems can be accurately screened, and the method is high in practicability and high in practicability. And irrelevant information interference is reduced, the answer quality and the system efficiency are improved, and the user satisfaction is improved.

The invention discloses a knowledge base generation and retrieval method based on the RAG technology and an intelligent question answering system, and belongs to the technical field of natural language processing, and the knowledge base generation and retrieval method based on the RAG technology and the intelligent question answering system based on the RAG technology have the beneficial effects that the accuracy and the real-time performance of information processing are improved through efficient combination of retrieval and generation. High-dimensional vector representation is generated through text slicing and vectorization, a vocabulary index is constructed, and hierarchical retrieval and keyword matching are achieved; meanwhile, a question generation and question-question retrieval method is utilized, a result is optimized and generated in combination with a large language model (LLM), the reasoning ability of the large model is further improved through chain reasoning (COT) and a cue word optimization module, and it is ensured that the generated answer has high accuracy and logicality; the method can be widely applied to the fields of intelligent question answering, knowledge base construction, real-time information retrieval and the like.

The invention relates to the technical field of document retrieval and information processing in the data processing technology, in particular to a document retrieval method based on multistage indexing and feature clustering, which comprises the following steps: performing high-dimensional space mapping on multi-modal features such as texts and images through a quantum embedding layer to generate cross-modal joint feature representation; a first-level index of a multi-level index architecture is dynamically initialized based on a meta-clustering algorithm, and semantic blocks of a second-level index are divided in combination with a multi-head self-attention mechanism. And an optimal transmission matrix is generated by using a Sinkhorn algorithm to align cross-node feature distribution. The multi-target mixed retrieval strategy is fused with vector retrieval, keyword retrieval and graph retrieval results, and weight distribution is dynamically adjusted. Through collaborative optimization of quantum calculation, federated learning and causal reasoning, a closed-loop technical architecture from feature analysis to dynamic index construction is formed, the problems of insufficient cross-modal fusion, static clustering deviation and semantic association deficiency are solved, and the precision, efficiency and dynamic adaptability of heterogeneous document retrieval are improved.

The invention provides a prospecting prediction method based on a multi-agent technology. The method specifically comprises the steps that S11, keyword extraction is conducted on mineral problems input by a user; s12, establishing a vector database of the mineral knowledge text and a graph database of the mineral knowledge graph, and optimizing the extracted keywords to generate cue words; s13, constructing a large prospecting model to think the cue words, expanding the cue words, and obtaining a complete statement for professional expression of the user question again; s14, performing multi-task decomposition on the complete statement to obtain a plurality of sub-tasks, and scheduling the plurality of sub-tasks to corresponding agents based on a graph database of the mineral knowledge graph to perform step-by-step prediction; and S15, integrating prediction results of the intelligent agents and feeding back the integrated prediction results to a user. According to the method, prospecting prediction is promoted to be converted from manual driving to intelligent driving, and emerging technical support is provided for prospecting breakthrough.

The invention belongs to the technical field of electric power data security management and intelligent question answering, and particularly relates to an electric power data security policy large model question answering system and method based on relation pooling. The system comprises a power data security policy knowledge base, a language model, a question processing module, an answer generation module and a user output module. Data of the electric power data security policy knowledge base comes from an electric power data security policy file; the language model converts a natural language problem into a structured query for knowledge base retrieval; the question processing module retrieves policy terms related to the natural language question to serve as retrieval results; the answer generation module generates a final answer according to the natural language question and a retrieval result of the knowledge base; by introducing the intelligent question-answering system combining the large language model and the knowledge base, the problems of low efficiency, inaccurate interpretation, incapability of dynamically updating and the like in the existing power data security policy query and management are solved.

The invention discloses an intelligent automobile question and answer method based on a large model and retrieval enhancement generation, and the method comprises the steps: 1, receiving a user query, carrying out the preliminary retrieval of a user query vector through a mixed index retrieval module, and generating a candidate result; 2, judging whether the relevancy of the retrieval result meets a preset threshold value or not, and triggering query conversion enhancement and re-retrieval when the relevancy of the retrieval result does not meet the preset threshold value; thirdly, retrieval results are rearranged according to the correlation degree; and step 4, the system generates a query related prompt word, combines the original user query as input, and generates and returns a question and answer result. The invention further discloses a system for implementing the method, and the system has wide application value.

The invention provides a multi-modal intelligent question-answering system based on a large model and a construction method and device, and the method comprises the steps: carrying out the region recognition of a PDF document, and obtaining a table region picture, a text region picture and / or an image region picture; the Word document is analyzed, paragraph content of the Word document is obtained, and the paragraph content is composed of the first text, table data and / or Word pictures; constructing a knowledge base; wherein the knowledge base comprises vector representation of document slices; the document slice is determined based on the table area picture, the character area picture and / or the image area picture, or the document slice is determined based on the paragraph content and the corresponding chapter information thereof. Visibly, according to the method, layout recognition can be carried out on the PDF document, catalog chapter recognition can be carried out on the Word document, the knowledge base is constructed based on the layout recognition, the semantic integrity and coherence of multi-modal text question and answer recall can be improved, and the answer accuracy is improved.

The invention provides a potential relation reasoning-based medical knowledge graph retrieval system and a potential relation reasoning-based medical knowledge graph retrieval method. The system comprises a medical data preprocessing module, a knowledge extraction and medical graph generation module, a medical knowledge graph potential relation reasoning complementation module, a keyword network and semantic analysis retrieval module and a semantic mapping and query response module. The system can pre-process and standardize the medical data to generate the knowledge graph containing abundant medical information, not only can complement the knowledge graph through graph structure analysis and semantic reasoning technologies to ensure the integrity and accuracy of the knowledge graph, but also can carry out efficient retrieval based on user query to improve the user experience. And the user query intention and the related entity relationship are accurately matched, and finally an accurate and comprehensive medical information query result is provided.

The present invention proposes a vector database retrieval method and system, which relates to the technical field of big data processing and intelligent information retrieval. The method includes preprocessing high-dimensional multimodal data to obtain a dimension reduction feature vector; constructing a cluster center based on the dimension reduction feature vector; establishing an inverted file and an intra-cluster plane index based on the cluster center, compressing the inverted file and the intra-cluster plane index to obtain a compressed inverted file and an intra-cluster plane index; obtaining a user's query vector, using the inverted file to screen candidate clusters for the query vector, introducing a hierarchical navigable small-world graph in the intra-cluster plane index, performing an intra-cluster search on the cluster cluster, and obtaining a vector database retrieval result. The present invention effectively takes into account the efficiency and accuracy of high-dimensional data retrieval, and improves the retrieval efficiency and accuracy.

The invention provides a document knowledge questioning and answering method, a document knowledge storage method and a document knowledge questioning and answering device, and relates to the technical field of intelligent questioning and answering. All layers of nodes of a knowledge tree are traversed according to a target retrieval mode on the basis of the query statement to obtain a query result, the knowledge tree comprises multiple layers of nodes, the multiple layers of nodes at least comprise first-layer nodes, second-layer nodes and third-layer nodes, semantics of the first-layer nodes are semantics of text contents of knowledge point levels in the document, and semantics of the third-layer nodes are semantics of text contents of knowledge point levels in the document; the semantics of the second-layer nodes are the semantics of the paragraph-level text content in the document, and the semantics of the third-layer nodes are the semantics of the full-text-level text content in the document; and outputting an answer text corresponding to the query statement based on the query result. The method provided by the invention not only improves the recall rate of knowledge and the correlation of answers, but also enhances the overall experience of the user, so that the user can obtain more comprehensive and accurate answers in complex queries.

The invention provides an intelligent data production method and system based on a graph neural network and adaptive learning, and the method comprises the steps: collecting a text, a time sequence, an image and sensor data in real time, and converting unstructured data into a structured JSON format through a format analysis tool; extracting text entity features based on a large language model, modeling a cross-modal entity association relationship in combination with a graph neural network, and fusing multi-source heterogeneous data through a dynamic knowledge graph; the detection threshold is dynamically optimized by adopting a reinforcement learning algorithm, and the false alarm rate is reduced in combination with a double-track verification mechanism; and predicting a data trend by using the space-time diagram convolutional network, and outputting an interpretability analysis report through the generative large model. According to the invention, a multi-protocol adaptation engine and a distributed stream processing architecture are adopted, and a protocol analysis layer automatically identifies a plurality of industrial protocols, so that the manual adaptation time is reduced; and real-time synchronization of production line-level data is realized through edge node parallel acquisition and compression transmission.

The invention provides a data reordering retrieval method and system based on RAG, and the method comprises the steps: carrying out the dynamic semantic partitioning processing of an original document, generating corresponding document blocks, storing the document blocks in a vector database, constructing a hierarchical index, analyzing a received query request, extracting keywords in the query request, and carrying out the retrieval of the query request. Boolean keyword matching is carried out through an inverted index in the hierarchical index, semantic retrieval of keywords is carried out in a vector database, an initial candidate set is generated after multi-source retrieval results are fused, a query request and the initial candidate set are input into a generative reordering model, and a query result is obtained; according to the method, the initial candidate set is selected, the corresponding score is generated through the cross-modal attention mechanism in the generative reordering model, the initial candidate set is reordered according to the score, the final retrieval result is obtained, and the reordered document blocks are displayed, so that the overall retrieval speed is increased, the limitation of a single retrieval mode is avoided, and the accuracy of the retrieval result is improved.

The invention discloses a reverse question guide question answering implementation method and system, and belongs to the technical field of artificial intelligence and natural language processing. Context-aware intention dynamic correction is realized through a three-level intention classification system, and cross-modal knowledge matching is realized by adopting a distributed semantic index technology; based on the reinforcement learning strategy, optimizing a cooperative work mechanism of the dialogue strategy and the knowledge base; comprising the steps of intention recognition: analyzing a session of a user by using an intention recognition model, and constructing a three-level intention classification system based on deep semantic understanding, including main class recognition, fine-grained analysis and context perception; question rewriting: constructing a dynamic rewriting engine to rewrite the user question; recalling and cleaning multi-source item knowledge; generating a reverse question; locking items and acquiring item data; generating questions and answers. According to the method, the robustness, the real-time performance and the scene adaptation capability of a professional question answering system can be improved, and the government affair service question answering accuracy, the intention recognition precision and the cross-region recommendation adoption rate are improved.