61 results about "Graph indexing" patented technology

Systems, devices, and techniques are disclosed for efficient knowledge graph indexing and retrieval. Document chunks may be generated from documents. Summarizations may be generated from document chunks. Entity types, entity properties, relations, and relation properties may be generated from a subset of the summarizations. A schema including entity types, entity properties, relations, and relation properties may be generated. Entity property triplets and entity relation triplets may be generated from the summarizations based on the schema and linked to the document chunks. A knowledge graph including nodes representing entities from the entity property triplets and entity relation triplets and edges representing the entity property triplets and the entity relation triplets may be generated. A search query may be received. Nodes and edges of the knowledge graph that include the entities, the entity property triplets and the entity relation triplets most similar to keywords of the search query may be determined.

The invention relates to the technical field of information retrieval, in particular to a large-scale semantic retrieval optimization method based on deep learning, which comprises the following steps: acquiring to-be-retrieved text data, and converting the to-be-retrieved text data into semantic vector representation through a deep learning model; carrying out topological structure analysis on the semantic vector representation to obtain topological characteristic information of a semantic space; on the basis of the topological characteristic information, a topological perception hierarchical navigable small-world graph index with a multi-layer structure is constructed, and each layer comprises a node set and a connection relation thereof; dynamically optimizing and adjusting the topology-aware hierarchical navigable small-world graph index according to an actual query mode and system performance feedback; and receiving a query request, converting the query request into a query vector, executing multi-layer navigation search in the topology-aware hierarchical navigable small-world graph index, and returning a retrieval result related to the semantics of the query vector.

The invention provides a cross-database approximate nearest neighbor search method and system and a calculation framework, and the method comprises the following steps: constructing a graph index, storing the graph index in a relation table, and obtaining a graph index table; acquiring and storing a data set and a query set in a structured relation table form; the dismantling approximate nearest neighbor search process comprises a plurality of SQL operation stages including a candidate node screening stage, a neighbor expansion stage, a distance calculation stage, a result combination stage and a priority queue maintenance stage; and based on the graph index table, executing an iterative search process of the plurality of SQL operation stages on each query point in the query set, and finally outputting an approximate nearest neighbor search result of the query set. According to the method, the graph index is combined with the relational database, and the approximate nearest neighbor search is realized by adopting a plurality of SQL operation stages, so that the high-dimensional vector retrieval efficiency and the cross-database compatibility are remarkably improved, and the large-scale application of the vector data in a multi-element scene is promoted.

The invention discloses a book resource data security collection retrieval monitoring system based on big data, and relates to the technical field of information technology and data security, and the system comprises a permission-aware association reasoning retrieval module which is in communication connection with a knowledge graph and dynamic index construction module and is used for receiving a user retrieval request and sending the user retrieval request to a database; analyzing a retrieval intention and obtaining a real-time permission context of a user to determine a security level threshold value, verifying a security attribute voucher of the data when retrieving the mixed index, only putting the data with the security level not higher than the threshold value into a result set, and performing association reasoning based on the knowledge graph under permission constraint to expand a result. According to the method, the mixed index structure fusing the inverted index, the vector index and the graph index is constructed, the security attribute voucher is associated, efficient full-text retrieval, semantic retrieval and association reasoning are supported, meanwhile, it is ensured that the retrieval process and result are strictly constrained by permission, and maximum mining of data values on the premise of security is achieved.

The application relates to the technical field of network task allocation, and discloses a heterogeneous computing power network task allocation and path optimization system based on a neural network, which comprises a graph construction module, a node task allocation module and a path optimization module.The graph construction module is used for constructing a traffic dispersion graph, modeling network topology and node data through the traffic dispersion graph, and modeling nodes and edges in combination with graph indexing and graph displacement, and extracting network communication traffic and node features.The node task allocation module is used for inputting graph indexes of all nodes into an improved graph neural network model to intelligently predict and allocate node tasks, and outputting node task allocation prediction results.The path optimization module is used for obtaining path optimization results by minimizing transmission delay and balancing node computing power in combination with node task prediction results.The system optimization module is used for optimizing the overall performance of the system based on the path optimization results and the task allocation prediction results.The application improves the task allocation and path optimization efficiency of the heterogeneous computing power network.

The specification provides a vector graph index merging method and device, the method comprising: obtaining a plurality of vector graph indexes to be merged; wherein each node in the vector graph index represents a vector, and each edge represents that the vectors represented by the nodes connected by the edge are similar; for each target vector in each target vector graph index, determining the vectors similar to the target vector from each vector graph index based on an approximate nearest neighbor search algorithm to form a cross-graph candidate neighbor set of the target vector; based on the cross-graph candidate neighbor set of each vector in the plurality of vector graph indexes, constructing a global neighbor graph; wherein each node in the global neighbor graph represents each vector in the plurality of vector graph indexes, and each edge represents that the vectors represented by the nodes connected by the edge are similar; and converting the global neighbor graph into a merged graph index that can be used for approximate nearest neighbor search.

The invention discloses a GPU-based graph vector index construction method and device and a storage medium, and relates to the technical field of graph index construction. The method comprises the following steps: fragmenting an original data set based on a clustering algorithm to obtain a first preset number of fragmented data subsets; sending the ith fragmented data subset to the GPU, wherein i is a positive integer; constructing a local graph vector index of the ith fragmented data subset in the GPU; sending the (i + 1) th fragmented data subset to the GPU until all local graph vector indexes are obtained; and constructing a hierarchical global graph vector index according to all the local graph vector indexes. According to the method, the production line of the producer-consumer model is constructed, so that the data transmission time of the PCIe bus is completely covered by the calculation time of the GPU, the I / O bottleneck is eliminated, and the construction speed of the graph index is improved.

A GPU-based graph vector index construction method, apparatus, and storage medium are disclosed, relating to the field of graph index construction technology. The method includes: partitioning the original dataset using a clustering algorithm to obtain a first preset number of partitioned data subsets; sending the i-th partitioned data subset to the GPU, where i is a positive integer; constructing a local graph vector index for the i-th partitioned data subset on the GPU; sending the (i+1)-th partitioned data subset to the GPU, until all local graph vector indices are obtained; and constructing a hierarchical global graph vector index based on all local graph vector indices. This method, by constructing a producer-consumer model pipeline, completely masks the PCIe bus data transfer time with GPU computation time, eliminating I / O bottlenecks and improving the graph index construction speed.

The application belongs to the technical field of databases, and specifically discloses an efficient GPU near neighbor graph index construction method based on data locality, which comprises the following steps: dividing index original data into multiple subsets, and performing data processing on the multiple subsets to obtain rearranged data; constructing subgraphs corresponding to each subset in the rearranged data, and constructing a graph index based on the subgraphs; and refining the structure of the graph index to obtain a final graph index. The method can effectively alleviate the problems of insufficient memory resources and excessive calculation cost in the subgraph merging stage, significantly reduce the construction cost of the graph index, and effectively extend to large-scale data sets to improve scalability.

The invention discloses a heterogeneous time series data processing method and system, electronic equipment and a storage medium, and belongs to the technical field of industrial data processing. The method comprises the following steps: acquiring original working condition data of each device in a target area in real time, and converting the original working condition data into intermediate data in a unified format; performing standardization processing on the intermediate data in sequence, outputting standardized data, and performing quality verification on the standardized data; performing data fusion processing on the standardized data according to the quality verification result, and outputting and storing fused data; constructing a hierarchical bitmap index structure of fused data according to the basic index key; and obtaining and analyzing a query request, inputting an analysis result into the hierarchical bitmap index structure, performing accelerated query operation according to an index result, and outputting target data. According to the method, continuous and efficient data fusion and query processing in a large data volume state can be realized, and the expansibility is high.

Embodiments of the present application disclose a data query method, an electronic device and a computer readable storage medium. The electronic device stores a first data table including a first tuple and a first bitmap index corresponding to the first tuple. The first tuple includes one or more queryable attribute values. The first bitmap index includes a plurality of bits arranged in sequence. The method includes receiving a query request for the first data table. The query request includes a query item including an attribute value to be queried. A second bitmap index corresponding to the query item is generated based on the attribute value to be queried. The second bitmap index corresponding to the query item and values of the plurality of bits at the same positions in the first bitmap index corresponding to the first tuple are compared simultaneously to determine whether the first tuple satisfies the query item. Embodiments of the present application can improve the efficiency of data query.

The application discloses a GPU-based multi-modal index integration method. The method comprises the following steps: acquiring multi-modal original data and extracting a feature vector; mapping the feature vector to a unified common semantic space; constructing a double-layer hybrid index structure on a GPU, wherein a first layer adopts IVF-PQ index for coarse screening, and a second layer adopts graph index for fine screening; after receiving a query request, performing two-stage parallel search on the GPU, first screening a candidate set through IVF-PQ index, and then fine screening a final result through graph index. The application solves the cross-modal semantic alignment problem through the unified semantic space, realizes high-precision and low-delay retrieval of massive multi-modal data by using GPU parallel computing and a double-layer index structure, and effectively overcomes the technical bottleneck that precision and efficiency are difficult to be considered in the prior art.

The application discloses a communication product configuration method based on multi-modal analysis and vector matching, and particularly relates to the technical field of intelligent configuration of communication products, and is used for solving the problem that the efficiency of manually reading documents and manually selecting and configuring is low, errors are easy to occur, and it is difficult to trace back when the customer demand is proposed in the form of multi-source and multi-format documents, the field caliber is complex, and the rule constraints are numerous in the group and government and enterprise customer scenarios of operators. By taking the field dictionary and structured constraints as the basis, the multi-source and multi-format communication product demand document is sequentially converted into demand parameters, demand configuration graphs and multi-channel feature vectors, and vector matching of rule perception such as field mandatory, mutual exclusion of rights and interests, region, contract period and cost is performed on the hierarchical near neighbor graph index in combination with the template graph, so that the communication product configuration scheme meeting the business constraints is automatically output under the unified caliber, the workload of manually reading documents and selecting by experience is significantly reduced, and the accuracy and consistency of the configuration result in the complex scenario are improved.

The invention provides a method and a device for constructing a distributed approximate neighbor search graph index. The method comprises the following steps of: initializing each process, dividing batches for a read-only vector data set, and distributing a message buffer area; if the value of the counter is an even number, unprocessed data set batches are taken out, an adjacency list of each point is calculated in parallel, and a tuple is generated; if the counter value is an odd number, determining the affected points in the graph index updating process, and performing parallel pruning again and updating; the adjacency list length information is written into an adjacency list length array in parallel in the process; copying the information to a message sending buffer area in parallel by utilizing a parallel prefix and a calculation offset; broadcasting a sending message buffer area of each process; deserializing a tuple from the received message buffer area by each process, and updating a graph index in the process; generating tuples for the adjacency table of the updated points; printing related results by one process and outputting a graph index. According to the method, the index construction time is greatly shortened; and higher query accuracy can be provided in a query stage.

This invention belongs to the field of big data processing and computer storage architecture technology, specifically involving a software-hardware collaborative vector retrieval method and system based on near-data processing. The method includes the following steps: 1. PQ quantization and graph index construction; 2. Optimization of the index data structure and alignment of physical pages; 3. Index deployment and hotspot cache pool construction; 4. Pre-computation and task deployment; 5. Near-data graph traversal and distance estimation; 6. Candidate node selection and reordering. The system includes an offline construction module, an index data structure optimization module, a retrieval offloading module, a host hardware collaborative retrieval module, and a speculative prefetching module. This method, while ensuring recall and throughput, offloads the space-consuming index structure to SSDs, significantly reducing DRAM requirements; it also facilitates PQ encoding calculations performed by weak-core CPUs, making it suitable for scenarios with limited space and computing power.

The invention provides a federated learning-based neighbor graph indexing method and system and electronic equipment. The method comprises the following steps: each client participating in federated learning trains a local performance prediction model used for predicting the retrieval performance of the neighbor graph based on a local sample data set of the client to obtain a corresponding local model parameter; the federation server participating in federation learning obtains local model parameters and test data sets of all the clients, a global test data set is obtained based on the test data sets of all the clients, and contribution quantized values corresponding to all the clients are calculated through the global test data set; and the federation server side aggregates all the local model parameters based on the contribution quantized value to obtain global model parameters, so that each client side updates a local performance prediction model based on the global model parameters, and constructs a corresponding neighbor graph index for the to-be-queried data set by using the updated local performance prediction model. According to the technical scheme, neighbor graph intelligent indexing supporting distributed data can be realized.

The application provides a kind of digital main line processing method, medium and system based on industrial data atlas, belongs to industrial data atlas technical field, the application is realized unified access of cross-system data by constructing multi-source data acquisition adapter, carries out cleaning standardization processing to heterogeneous data, constructs integrated data set, abstracts data entity as node, constructs industrial data atlas by abstracting cross-system association as edge, adopts graph index optimization strategy and layered storage architecture to improve cross-system query performance, realizes the dynamic adjustment of atlas structure using atlas self-adaptive evolution model, realizes cross-system depth correlation analysis function by graph traversal algorithm, establishes access control and data security mechanism to complete system protection, solve the technical problem that industrial data processing system is difficult to realize cross-system depth correlation analysis.

This invention relates to a retrieval enhancement generation method based on multi-level knowledge fusion, belonging to the field of natural language processing technology. It aims to solve two core problems in existing retrieval enhancement generation technologies: fragmented knowledge levels and interference from noise information. The invention includes the following steps: Graph index construction: Segmenting documents into fragments and extracting entities and relations to generate a knowledge graph with key-value pairs, followed by deduplication and optimization; Multi-level knowledge retrieval: Acquiring local and global knowledge separately, and constructing cross-level semantic associations by connecting knowledge modules through an intermediate layer; Dual-dimensional filtering: Filtering core knowledge based on a weighted score of semantic relevance and knowledge importance, combined with a maximum boundary relevance strategy to remove redundant noise; Response generation: Inputting the filtered knowledge into a large-scale language model to generate accurate and coherent responses. This invention significantly improves the generation quality of knowledge-intensive tasks through cross-level knowledge bridging and precise noise filtering, possessing significant theoretical and practical application value.

The application particularly relates to a similar data efficient retrieval method based on a ciphertext HNSW, and belongs to the technical field of data security and information retrieval. The scheme comprises the following steps: a data owner initializes system parameters, and performs dimension expansion on each vector in an original data set; an HNSW graph index structure is constructed based on the expanded vectors; each node on the index is encrypted by using a system master key and a public parameter matrix, an encrypted HNSW index is generated and outsourced to a cloud server; a query vector is expanded and encrypted at a user end, a query ciphertext is generated and uploaded to the cloud server; and the cloud server performs an approximate nearest neighbor search on the encrypted index, and efficiently returns the nearest neighbor results by calculating the inner product of the ciphertext. The application combines the fast retrieval capability of the HNSW and a specific encryption mechanism, realizes the approximate nearest neighbor efficient retrieval of large-scale vector data under the premise of guaranteeing the privacy and safety of the whole data flow.

In response to a user invoking a graph operation on a graph in an in-memory graph algorithm (IMGA) runtime in a relational database management system (RDBMS), the RDBMS generates a set of one or more graph indexes in memory. The graph is represented as one or more vertex tables and one or more edge tables. The RDBMS generates the set of one or more graph indexes by generating a mapping of each database table vertex identifier to a corresponding internal identifier. For each edge table of the one or more edge tables, the RDBMS generates a graph index data structure representing edges of the edge table. The graph index data structure represents each edge in the edge table as a source internal identifier and a destination internal identifier. The RDBMS can then execute the graph operation in the IMGA runtime using the set of one or more graph indexes.

The application discloses a kind of based on layered joint graph index's house rental multi metric space search method, comprising: obtaining new house object, find the k nearest neighbor house object of the new house object in each metric space, under each metric space, add undirected edge in new house object and its k nearest neighbor house object, adopt splitting and hierarchical update strategy based on tree, dynamically adjust graph structure, improve node distribution, to build layered joint graph index, wherein the metric space includes price metric space, house type metric space, geographical position metric space and comment metric space;Based on the pruning technique of best priority search strategy, multi metric space approximate nearest neighbor search is carried out using the layered joint graph index, and the approximate-k nearest neighbor house object of given query house object is obtained.

The invention provides a large language model retrieval enhancement method and system based on a spaceflight domain knowledge graph, and belongs to the technical field of artificial intelligence and spaceflight information systems. The method comprises the following steps: firstly, carrying out block vectorization on a spaceflight field text, automatically extracting entities and relationships through a cue word project of field optimization, and constructing a knowledge graph; and establishing a graph index and vector index fused retrieval system. During retrieval, semantic retrieval based on query vectors, atlas relation retrieval based on keywords and entity retrieval are synchronously executed, semantic and graph structure fusion reordering is carried out on multiple paths of results, and structured contexts are dynamically constructed according to the semantic and graph structure fusion reordering to be input into a large language model to generate answers. According to the large language model retrieval enhancement method and system based on the spaceflight domain knowledge graph, through deep fusion of the graph and semantic retrieval, the problems that spaceflight knowledge retrieval is weak in association and low in fragmentization and credibility are solved, and systematicness, accuracy and traceability response to complex queries is achieved.

The invention relates to the technical field of communication, and discloses a dynamic allocation method and system for communication network slice resources. The method comprises the following steps: constructing a mapping database of a quantization network state and an optimal resource template in an off-line manner; collecting a full-dimension network state in real time and synthesizing a state vector; a nearest neighbor quantization state is matched at a high speed through a hierarchical navigable small-world graph index; and calling the corresponding resource template to generate a scheduling instruction, and issuing and executing. The system comprises an offline construction module, a state sensing module, a state matching module, a template extraction module and an instruction execution module. According to the method, complex online decisions are converted into deterministic table lookup operation, sub-millisecond response is realized, and the service quality and resource utilization efficiency of time delay sensitive services are remarkably improved.

The invention provides a method and a system for constructing a POI knowledge base, which are characterized in that on the basis of an existing POI dictionary and POI data in the same field, related knowledge of a POI is acquired from a POI field dictionary by RAG retrieval to correct and complement the relationship and attribute of the POI data subjected to entity recognition, so that data nodes for constructing a POI knowledge graph are uniform in format, and the POI knowledge graph can be constructed more accurately. The method comprises the following steps of: establishing a multi-layer POI knowledge base in a multi-layer POI database, and establishing a multi-layer retrieval category on different granularity levels on the basis of multi-modal vectorization processing and HNSW, so that the final multi-layer POI knowledge base can be combined with an RAG retrieval technology, high-level features can be firstly matched on the basis of the retrieval category in a subsequent retrieval process, and layer-by-layer downward retrieval can be carried out on the basis of the retrieval category in the subsequent retrieval process. According to the method, the problems that retrieval needs to be carried out in different databases in the POI fusion technology, the retrieval range is large, and time consumption is long are solved, and the knowledge storage and retrieval efficiency is improved.

The application belongs to the field of data retrieval, and discloses a difference maximization data table combination query method for associated data sets. In the data processing stage, the given table data set is first processed, a feature-data column index is established, connectable tables in the table data set are found according to the index, a data table connection graph index is constructed, and the connection information between each table is pre-calculated. In the data query stage, the candidate connection column set is searched in the feature-data column index according to the given sample query table and the connection column set, the candidate data table set is obtained, and the data table set that can be connected with the sample query table and has the maximum difference is selected according to the given budget. The application first proposes a method for searching connectable data table combinations in associated data sets, filters connectable data tables by using a feature index, establishes a data table connection graph to find the connection paths between data tables, and returns the data table set that has the maximum difference under the budget constraint.

The application discloses a high-dimensional vector graph index construction method based on GPU acceleration, and comprises the following steps: obtaining a high-dimensional vector dataset, using the parallel computing capacity of GPU, and adopting a two-stage k-neighbor graph construction algorithm to quickly initialize a k-neighbor graph; accelerating various neighbor graph pruning strategies through a unified GPU acceleration framework, and designing special GPU kernel processing serial and unbalanced calculation to finally generate a neighbor graph index; for a large-scale dataset, proposing an asynchronous GPU-CPU-disk index framework and a cluster-aware cache mechanism, and adopting a multi-GPU parallel mode to construct a large-scale index and reduce I / O overhead in the index construction process. The application solves the problems of poor generality and unbalanced load of the existing GPU method, significantly reduces the high-dimensional vector graph index construction time and memory overhead, and can maintain high query accuracy and efficiency.

A computer-implemented method for vector retrieval includes, in response to a query vector, obtaining, from a first-level storage resource, at least one target compression vector having a highest similarity with the query vector based on a graph index constructed based on an original vector and a compression vector. After obtaining the at least one target compression vector, obtaining, from a second-level storage resource, at least one target original vector corresponding to the at least one target compression vector. Separately, a similarity between the query vector and the at least one target original vector is calculated; the at least one target original vector is sorted based on the calculated similarity; and, based on the sorted at least one target original vector, a vector retrieval result corresponding to the query vector is determined.