129 results about "Graph indexing" patented technology

The invention relates to the technical field of natural language processing, in particular to an intelligent search engine system and method based on NLP and vector hybrid retrieval, and the system comprises a query analysis module which is used for receiving a query statement input by a user and obtaining structured query information based on the query statement; the symbiotic index module comprises a sparse extension index unit which is used for constructing a sparse inverted index table based on keywords in the structured query information; the vectorization hypergraph index unit is used for forming a hypergraph index based on the business data; the recall arrangement module is used for acquiring a candidate object set according to the structured query information; the constraint rearrangement module is used for calculating a joint priority score between query and candidate objects based on the candidate object set, and obtaining a sorting result under constraint conditions of meeting a supplier proportion, a category proportion and a price interval; and the evidence generation module is used for generating evidence information based on the sorting result and outputting the evidence information to the user interface.

The invention provides a retrieval method and system suitable for different data densities, and the method comprises the steps: distributing each data point to a corresponding hash bucket based on a locality sensitive hash algorithm, and forming an initial inverted index based on an identifier of the hash bucket and the data points; processing data points in each hash bucket based on a hierarchical navigable small world insertion algorithm to form an HNSW graph, and forming an inverted index containing an HNSW graph index; obtaining a candidate bucket set of the query point; obtaining an HNSW graph corresponding to each candidate bucket in the candidate bucket set based on the constructed inverted index; retrieving from the HNSW graph corresponding to the candidate bucket based on a hierarchical navigable small world search algorithm to obtain preliminary retrieval results, and determining a global retrieval result according to the preliminary retrieval results and returning the global retrieval result. According to the method, efficient retrieval of dense vectors and sparse vectors can be carried out uniformly, the retrieval efficiency and precision are improved, the universality is high, and the complexity and development cost of the system are reduced.

The invention relates to a distance perception-based graph division retrieval method and system and a readable storage medium. The method comprises the following steps of: constructing original data into an initial graph structure; distance measurement is selected, a weight is allocated to each edge, and the nearest neighbor node pairs are combined based on the weights to obtain a coarsened graph structure; performing hierarchical segmentation on the coarsened image structure, and projecting the coarsened image structure back to the initial image structure based on the segmentation condition to obtain a plurality of initial cut edge sets; finding a corresponding node according to each initial cut edge in the initial cut edge set, randomly putting the nodes into different partitions, and optimizing each initial cut edge set by taking the sum of minimum cut edge weights as a target function to obtain a graph index structure; and inputting a query, calling the graph index structure, respectively calculating the distance between the query and the central node of each partition of the graph index structure, and selecting the nearest partition for retrieval. According to the method, the number of times of cross-partition query is reduced, and the query efficiency is improved.

The present invention provides an adaptive drone image matching pair selection method and system, comprising: S1: acquiring drone images, performing feature extraction on the drone images using the SIFT algorithm, and obtaining a local feature set of the drone images; S2: constructing a vector codebook, performing feature aggregation on the local feature set using the vector codebook, and obtaining a feature matrix set; vectorizing the feature matrix set to obtain a global feature descriptor vector set; S3: performing image retrieval on the global feature descriptor vector set based on a graph index structure to obtain a drone image matching pair set; S4: performing three-dimensional reconstruction on the drone image matching pair set to obtain a three-dimensional reconstruction model. The present invention uses global feature descriptor vectors to replace word frequency calculation based on local features, and provides a graph-based indexing strategy to achieve efficient overlapping image search, making drone image matching pairs more efficient and accurate.

Systems, devices, and techniques are disclosed for efficient knowledge graph indexing and retrieval. Document chunks may be generated from documents. Summarizations may be generated from document chunks. Entity types, entity properties, relations, and relation properties may be generated from a subset of the summarizations. A schema including entity types, entity properties, relations, and relation properties may be generated. Entity property triplets and entity relation triplets may be generated from the summarizations based on the schema and linked to the document chunks. A knowledge graph including nodes representing entities from the entity property triplets and entity relation triplets and edges representing the entity property triplets and the entity relation triplets may be generated. A search query may be received. Nodes and edges of the knowledge graph that include the entities, the entity property triplets and the entity relation triplets most similar to keywords of the search query may be determined.

The invention discloses a large-scale vector approximate nearest neighbor search heterogeneous system based on a GPU-NVMe framework, which is characterized in that the system is based on a direct GPU-NVMe framework, and a hybrid index combining a clustering algorithm and a graph algorithm is designed. According to the hybrid index, vectors with good clustering are efficiently processed by utilizing the parallel processing capacity of the GPU, and meanwhile, the search accuracy of noise vectors is improved by taking a graph index as a compensation mechanism. According to the method, collaborative optimization of hardware and software is fully considered, vector index data deployment is implemented at an NVMe driving level, meanwhile, CPU and GPU hardware are cooperatively utilized to carry out a gigabit-level vector retrieval calculation process, and an extensible large-scale retrieval system on a direct GPU-NVMe architecture is constructed. Compared with the prior art, the method has the advantages that the performance bottleneck in the gigabit-level vector data search is effectively solved, the search throughput and accuracy are remarkably improved, and the search delay is reduced.

The invention provides an unmanned aerial vehicle image matching method and device, equipment, a storage medium and a program product, and relates to the technical field of photogrammetry and computer vision, and the method comprises the steps: obtaining a global feature vector of an unmanned aerial vehicle image, and carrying out the image index processing of the global feature vector, and obtaining an image matching pair; performing data scheduling processing of iterative matrix bandwidth reduction on the image matching pair to generate an image scheduling block; and performing feature matching on the image scheduling block, and eliminating matching error data in a feature matching result to obtain an unmanned aerial vehicle image matching result. According to the method, based on a data scheduling strategy of matrix bandwidth reduction, the sparsely connected image matching graph is divided into compact sub-blocks, the sub-blocks can be loaded and processed more efficiently under the condition that the GPU memory is limited, meanwhile, the feature matching speed is increased by combining feature matching and gross error elimination of cascade hash, and the matching efficiency is improved. Therefore, the rapid feature matching requirement of large-scale unmanned aerial vehicle images is met.

The present application relates to the field of information retrieval, and in particular relates to a data processing method, an electronic device, a readable storage medium, and a program product. In the method, when a query result of a query vector needs to be searched in a constructed graph index, neighbor nodes of at least some nodes among multiple nodes included in the graph index satisfy at least one of the following: the included angle between two vectors obtained by subtracting a node from any two neighbor nodes of the node is greater than or equal to a first angle threshold value, the distance between any two neighbor nodes of the node is greater than or equal to a first distance threshold value, and the number of neighbor nodes of the node within a preset space range is smaller than a first number. Thus, the neighbor nodes of the nodes in the constructed graph index are relatively dispersed in space, and the problem that a query result falls into a local optimum due to being unable to find nodes at a certain portion of space during a node-based neighbor node search process, because the nodes are densely distributed at a certain position, is avoided.

The invention relates to a self-adaptive retrieval method based on multi-modal graph indexing and privacy calculation, and belongs to the technical field of safe and intelligent retrieval. The method comprises the steps that retrieval problems are classified in real time through a complexity discriminator; the method comprises the following steps: disassembling a retrieval problem into multi-modal metadata through atomic memory nodes, and constructing a semantic association map based on a predefined meta-path rule; through a semantic matching module driven by a neural network model or a large language model, multi-type association is established for the newly-added nodes and historical nodes; triggering index field rewriting and topological structure adjustment of old nodes according to newly input retrieval concepts and relationships, and generating an optimal retrieval path; injecting reversible differential privacy noise into the local embedding model; and establishing a homomorphic encryption cache pool for a high-frequency query result. And the structure is flexibly adjusted in the retrieval process, so that the retrieval path better meets the actual demand, the privacy protection strength can be intelligently adjusted according to the content sensitivity, and the retrieval response speed is further accelerated.

The invention discloses a similar text quick retrieval method and system, and belongs to the technical field of natural language processing and information retrieval, and the similar text quick retrieval method comprises the following specific steps: step 1, vectorizing semantics, and storing the vectorized semantics into a database; using a pre-training language model to respectively encode the query text and the documents in the text library into low-dimensional semantic vectors with fixed dimensions; step 2, hierarchical index retrieval: N candidate documents closest to the query vector are retrieved through a first-level index structure to form a first candidate set; according to the method, millisecond coarse screening is realized through parallel design of the HNSW graph index and the semantic cluster inverted index, the semantic cluster inverted index reduces a search space to a local optimal domain, global precision loss is avoided, a newly added text only needs one-time cluster center distance calculation, and index updating complexity is greatly reduced.

The embodiment of the invention discloses a video duplicate checking method and device, electronic equipment, a storage medium and a product. The method comprises the following steps: acquiring at least one to-be-processed video frame in a video to be subjected to duplicate checking; determining at least one similar video frame associated with the to-be-processed video frame according to the image features of the to-be-processed video frame and a pre-constructed graph index; wherein the graph index comprises a plurality of nodes and edges connected with the plurality of nodes, the nodes correspond to frame features of the key frame, the edges are used for representing that the frame features of the two nodes meet preset similar attributes, and the key frame is a video frame extracted based on shot segmentation of a historical video; and according to the at least one target video to which the at least one similar video frame belongs, determining a duplicate checking result corresponding to the video to be subjected to duplicate checking. According to the technical scheme, the image index is determined based on the shot segmentation algorithm, so that the data volume of the image index is greatly reduced, and the processing efficiency can be improved when the target video is determined according to the image index.

The invention discloses an intelligent AI interactive question-answering method and system applied to science popularization and a storage medium, and the method comprises the steps: constructing a science popularization knowledge spectrogram based on multi-source science popularization knowledge; extracting key feature data based on a voice or text question input by a user, and generating a question feature vector; similarity matching is carried out on the basis of the feature vectors and a node embedding space of the science popularization knowledge spectrogram, and accurate knowledge sub-graphs related to questions are extracted; after completing the response of the current question, recording a storage positioning index of the knowledge sub-graph corresponding to the question in the popular science knowledge spectrogram, and generating knowledge sub-graph index sequence data; and based on the next question of the user, carrying out correlation analysis, including questions of the same and different knowledge types, and carrying out sub-graph matching or full-graph matching, and if a plurality of knowledge fields are involved, carrying out sub-graph matching and full-graph matching. The method has the advantages that efficient knowledge matching is realized by fusing multi-source popular science knowledge and feature extraction, user questions are quickly responded, and the matching precision is continuously optimized.

The invention discloses a text duplicate checking method, a vehicle, a computer readable storage medium and a computer program product, and relates to the technical field of information processing. The method comprises the following steps: carrying out standardized preprocessing on an input text to be subjected to duplicate checking to obtain a current text fragment; performing semantic coding processing on the current text fragment by utilizing the target coding model to obtain a query vector; a similarity retrieval interface corresponding to the target search engine is called, a plurality of candidate vectors corresponding to the query vector are recalled from a semantic vector index, and the semantic vector index is constructed based on a historical text set and a graph index algorithm in the target search engine; sorting analysis is conducted on the multiple candidate vectors based on the multi-level duplicate judgment threshold values, a duplicate checking result is obtained, and the duplicate checking result is used for representing the duplicate situation between the historical texts corresponding to the multiple candidate vectors and the input text. The technical problems of low accuracy and low efficiency of a text duplicate checking method in related technologies are solved.

The invention provides a database vector index construction method and a related product. The construction method comprises the following steps: clustering a vector set to be constructed to obtain a plurality of clusters; wherein the vector number of each cluster is smaller than or equal to a preset vector threshold value; generating hierarchical navigable small-world graph indexes corresponding to a plurality of inverted files according to the center vectors of the plurality of clustering clusters; wherein the inverted files are in one-to-one correspondence with the center vectors, and the hierarchical navigable small-world graph index comprises all the center vectors. According to the construction method, the retrieval efficiency and the retrieval accuracy of the database vector index can be improved, and the effect of considering the memory use and the retrieval accuracy of the database vector index is achieved.

The invention provides a digital main line processing method based on an industrial data atlas, a medium and a system, and belongs to the technical field of the industrial data atlas. Unified access of cross-system data is realized by constructing a multi-source data acquisition adapter, cleaning and standardization processing are performed on heterogeneous data, an integrated data set is constructed, and the data collection efficiency is improved. Data entities are abstracted as nodes, cross-system association relationships are abstracted as edges to construct an industrial data graph, a graph index optimization strategy and a hierarchical storage architecture are adopted to improve cross-system query performance, a graph adaptive evolution model is utilized to realize dynamic adjustment of a graph structure, and a cross-system deep association analysis function is realized through a graph traversal algorithm. And an access control and data security mechanism is established to complete system protection, so that the technical problem that the industrial data processing system is difficult to realize cross-system deep correlation analysis is solved.

The invention relates to the technical field of information retrieval, in particular to a large-scale semantic retrieval optimization method based on deep learning, which comprises the following steps: acquiring to-be-retrieved text data, and converting the to-be-retrieved text data into semantic vector representation through a deep learning model; carrying out topological structure analysis on the semantic vector representation to obtain topological characteristic information of a semantic space; on the basis of the topological characteristic information, a topological perception hierarchical navigable small-world graph index with a multi-layer structure is constructed, and each layer comprises a node set and a connection relation thereof; dynamically optimizing and adjusting the topology-aware hierarchical navigable small-world graph index according to an actual query mode and system performance feedback; and receiving a query request, converting the query request into a query vector, executing multi-layer navigation search in the topology-aware hierarchical navigable small-world graph index, and returning a retrieval result related to the semantics of the query vector.

The present application relates to the technical field of data retrieval, and provides a vector graph index processing method and apparatus. The method comprises: on the basis of the number of nodes and an identifier of a logically deleted node, scanning a vector graph index to obtain in-degree information of the logically deleted node, wherein the in-degree information indicates an in-degree node of the logically deleted node; and on the basis of the in-degree information of the logically deleted node, deleting the in-degree node of the logically deleted node and the edge of the logically deleted node. According to the present application, a memory resource occupied by a related edge of the logically deleted node can be released, the efficiency and accuracy of vector retrieval are improved, and the adequacy of a recall result is ensured. The present application is used for deletion of the vector graph index.

The invention provides a cross-database approximate nearest neighbor search method and system and a calculation framework, and the method comprises the following steps: constructing a graph index, storing the graph index in a relation table, and obtaining a graph index table; acquiring and storing a data set and a query set in a structured relation table form; the dismantling approximate nearest neighbor search process comprises a plurality of SQL operation stages including a candidate node screening stage, a neighbor expansion stage, a distance calculation stage, a result combination stage and a priority queue maintenance stage; and based on the graph index table, executing an iterative search process of the plurality of SQL operation stages on each query point in the query set, and finally outputting an approximate nearest neighbor search result of the query set. According to the method, the graph index is combined with the relational database, and the approximate nearest neighbor search is realized by adopting a plurality of SQL operation stages, so that the high-dimensional vector retrieval efficiency and the cross-database compatibility are remarkably improved, and the large-scale application of the vector data in a multi-element scene is promoted.

The invention provides a vector diagram index processing method and device, and belongs to the technical field of data retrieval, and the method comprises the steps that a vector diagram index is scanned based on the number of nodes and identifiers of logic deletion nodes, in-degree information of the logic deletion nodes is obtained, and the in-degree information indicates in-degree nodes of the logic deletion nodes; and deleting the in-degree node of the logic deletion node and the edge of the logic deletion node based on the in-degree information of the logic deletion node. According to the method, the memory resources occupied by the related edges of the logic deletion nodes can be released, the vector retrieval efficiency and accuracy are improved, and the sufficiency of recall results is ensured. The method and the device are used for deleting the vector graph index.

A digital image comparator includes an Faster R-CNN configured to generate a filtered set of local feature maps of an input image, a match head, and logic to preserve the local feature maps in an indexed data structure and to make the local feature maps retrievable by the match head via local feature map indexes.

The invention discloses a book resource data security collection retrieval monitoring system based on big data, and relates to the technical field of information technology and data security, and the system comprises a permission-aware association reasoning retrieval module which is in communication connection with a knowledge graph and dynamic index construction module and is used for receiving a user retrieval request and sending the user retrieval request to a database; analyzing a retrieval intention and obtaining a real-time permission context of a user to determine a security level threshold value, verifying a security attribute voucher of the data when retrieving the mixed index, only putting the data with the security level not higher than the threshold value into a result set, and performing association reasoning based on the knowledge graph under permission constraint to expand a result. According to the method, the mixed index structure fusing the inverted index, the vector index and the graph index is constructed, the security attribute voucher is associated, efficient full-text retrieval, semantic retrieval and association reasoning are supported, meanwhile, it is ensured that the retrieval process and result are strictly constrained by permission, and maximum mining of data values on the premise of security is achieved.

The invention discloses a high-dimensional space-oriented distributed graph index nearest neighbor search method, which comprises the following steps of: firstly, designing a boundary sensing balanced partition strategy, and measuring boundary characteristics of quantized nodes through a connection ratio; secondly, constructing a hierarchical mixed index architecture, vertically integrating a sparse global navigation layer and a fine-grained local precision layer, managing high-importance boundary nodes by adopting an M-Tree structure, and remarkably improving the cross-partition retrieval efficiency by utilizing the relative insensitivity of a tree structure to a curse in a boundary region; finally, a distributed multi-starting-point parallel search framework is provided, complementary entry points are dynamically generated based on query features and graph topology, vector space multi-region parallel exploration is achieved, irrelevant partitions are filtered through a navigation layer, and then accurate similarity calculation is executed in selected partitions. According to the method, on the premise that the high recall rate is guaranteed, the throughput of high-dimensional vector distributed search is remarkably improved, and an efficient and reliable solution is provided for large-scale high-dimensional vector retrieval.

The invention relates to the field of data storage, and discloses a big data compression storage and rapid retrieval method and system.The method comprises the following steps that dimension weight vectors are dynamically generated based on multi-dimensional information entropy, and rotating Hilbert codes are generated by adjusting the direction of a space filling curve; adaptively determining the number of segments and an error threshold value by using a time sequence prediction model, and compressing and storing a coding residual error by using a linear model; a double-layer collaborative index structure containing a space bitmap index and a value domain B + tree index is constructed, and local index reconstruction during weight change is achieved through an incremental updating mechanism; mapping a query range into a coding interval, screening candidate data segments in combination with an index, and fusing model parameters and residual decoding to return a result; newly added data code correction and expired data elimination are asynchronously processed, and the storage space consistency is optimized. According to the method, the unification of high compression ratio, quick retrieval and self-adaptive updating is realized.

The application relates to the technical field of network task allocation, and discloses a heterogeneous computing power network task allocation and path optimization system based on a neural network, which comprises a graph construction module, a node task allocation module and a path optimization module.The graph construction module is used for constructing a traffic dispersion graph, modeling network topology and node data through the traffic dispersion graph, and modeling nodes and edges in combination with graph indexing and graph displacement, and extracting network communication traffic and node features.The node task allocation module is used for inputting graph indexes of all nodes into an improved graph neural network model to intelligently predict and allocate node tasks, and outputting node task allocation prediction results.The path optimization module is used for obtaining path optimization results by minimizing transmission delay and balancing node computing power in combination with node task prediction results.The system optimization module is used for optimizing the overall performance of the system based on the path optimization results and the task allocation prediction results.The application improves the task allocation and path optimization efficiency of the heterogeneous computing power network.

The specification provides a vector graph index merging method and device, the method comprising: obtaining a plurality of vector graph indexes to be merged; wherein each node in the vector graph index represents a vector, and each edge represents that the vectors represented by the nodes connected by the edge are similar; for each target vector in each target vector graph index, determining the vectors similar to the target vector from each vector graph index based on an approximate nearest neighbor search algorithm to form a cross-graph candidate neighbor set of the target vector; based on the cross-graph candidate neighbor set of each vector in the plurality of vector graph indexes, constructing a global neighbor graph; wherein each node in the global neighbor graph represents each vector in the plurality of vector graph indexes, and each edge represents that the vectors represented by the nodes connected by the edge are similar; and converting the global neighbor graph into a merged graph index that can be used for approximate nearest neighbor search.

The invention discloses a GPU-based graph vector index construction method and device and a storage medium, and relates to the technical field of graph index construction. The method comprises the following steps: fragmenting an original data set based on a clustering algorithm to obtain a first preset number of fragmented data subsets; sending the ith fragmented data subset to the GPU, wherein i is a positive integer; constructing a local graph vector index of the ith fragmented data subset in the GPU; sending the (i + 1) th fragmented data subset to the GPU until all local graph vector indexes are obtained; and constructing a hierarchical global graph vector index according to all the local graph vector indexes. According to the method, the production line of the producer-consumer model is constructed, so that the data transmission time of the PCIe bus is completely covered by the calculation time of the GPU, the I / O bottleneck is eliminated, and the construction speed of the graph index is improved.

A GPU-based graph vector index construction method, apparatus, and storage medium are disclosed, relating to the field of graph index construction technology. The method includes: partitioning the original dataset using a clustering algorithm to obtain a first preset number of partitioned data subsets; sending the i-th partitioned data subset to the GPU, where i is a positive integer; constructing a local graph vector index for the i-th partitioned data subset on the GPU; sending the (i+1)-th partitioned data subset to the GPU, until all local graph vector indices are obtained; and constructing a hierarchical global graph vector index based on all local graph vector indices. This method, by constructing a producer-consumer model pipeline, completely masks the PCIe bus data transfer time with GPU computation time, eliminating I / O bottlenecks and improving the graph index construction speed.

The application belongs to the technical field of databases, and specifically discloses an efficient GPU near neighbor graph index construction method based on data locality, which comprises the following steps: dividing index original data into multiple subsets, and performing data processing on the multiple subsets to obtain rearranged data; constructing subgraphs corresponding to each subset in the rearranged data, and constructing a graph index based on the subgraphs; and refining the structure of the graph index to obtain a final graph index. The method can effectively alleviate the problems of insufficient memory resources and excessive calculation cost in the subgraph merging stage, significantly reduce the construction cost of the graph index, and effectively extend to large-scale data sets to improve scalability.

The invention belongs to the technical field of disk storage, and provides a monotonous path aware vector diagram index disk layout optimization method and system, and the method comprises the steps: preferentially distributing target nodes at the two ends of a front edge to a same target disk page through the sorting of target weights of edges corresponding to all nodes, according to the method, the data locality when each node in the vector diagram index is stored in the disk can be improved, so that the number of I / O units used when specified data in the disk is queried can be effectively reduced, and the query efficiency when the disk is queried is improved; and obtaining a total contribution value of each edge through the target disk layout optimization function and the target weight of the edge of each node, and sequentially distributing each out-of-neighbor node into the target disk page according to the total contribution value of the edge corresponding to the out-of-neighbor node of each target node in the target disk page to which the target node is distributed. And furthermore, the locality of the single target disk page data can be improved, so that the query efficiency when the target disk is queried can be further improved.