98 results about "Graph indexing" patented technology

The invention relates to the technical field of natural language processing, in particular to an intelligent search engine system and method based on NLP and vector hybrid retrieval, and the system comprises a query analysis module which is used for receiving a query statement input by a user and obtaining structured query information based on the query statement; the symbiotic index module comprises a sparse extension index unit which is used for constructing a sparse inverted index table based on keywords in the structured query information; the vectorization hypergraph index unit is used for forming a hypergraph index based on the business data; the recall arrangement module is used for acquiring a candidate object set according to the structured query information; the constraint rearrangement module is used for calculating a joint priority score between query and candidate objects based on the candidate object set, and obtaining a sorting result under constraint conditions of meeting a supplier proportion, a category proportion and a price interval; and the evidence generation module is used for generating evidence information based on the sorting result and outputting the evidence information to the user interface.

Systems, devices, and techniques are disclosed for efficient knowledge graph indexing and retrieval. Document chunks may be generated from documents. Summarizations may be generated from document chunks. Entity types, entity properties, relations, and relation properties may be generated from a subset of the summarizations. A schema including entity types, entity properties, relations, and relation properties may be generated. Entity property triplets and entity relation triplets may be generated from the summarizations based on the schema and linked to the document chunks. A knowledge graph including nodes representing entities from the entity property triplets and entity relation triplets and edges representing the entity property triplets and the entity relation triplets may be generated. A search query may be received. Nodes and edges of the knowledge graph that include the entities, the entity property triplets and the entity relation triplets most similar to keywords of the search query may be determined.

The present application relates to the field of information retrieval, and in particular relates to a data processing method, an electronic device, a readable storage medium, and a program product. In the method, when a query result of a query vector needs to be searched in a constructed graph index, neighbor nodes of at least some nodes among multiple nodes included in the graph index satisfy at least one of the following: the included angle between two vectors obtained by subtracting a node from any two neighbor nodes of the node is greater than or equal to a first angle threshold value, the distance between any two neighbor nodes of the node is greater than or equal to a first distance threshold value, and the number of neighbor nodes of the node within a preset space range is smaller than a first number. Thus, the neighbor nodes of the nodes in the constructed graph index are relatively dispersed in space, and the problem that a query result falls into a local optimum due to being unable to find nodes at a certain portion of space during a node-based neighbor node search process, because the nodes are densely distributed at a certain position, is avoided.

The invention discloses a similar text quick retrieval method and system, and belongs to the technical field of natural language processing and information retrieval, and the similar text quick retrieval method comprises the following specific steps: step 1, vectorizing semantics, and storing the vectorized semantics into a database; using a pre-training language model to respectively encode the query text and the documents in the text library into low-dimensional semantic vectors with fixed dimensions; step 2, hierarchical index retrieval: N candidate documents closest to the query vector are retrieved through a first-level index structure to form a first candidate set; according to the method, millisecond coarse screening is realized through parallel design of the HNSW graph index and the semantic cluster inverted index, the semantic cluster inverted index reduces a search space to a local optimal domain, global precision loss is avoided, a newly added text only needs one-time cluster center distance calculation, and index updating complexity is greatly reduced.

The invention discloses an intelligent AI interactive question-answering method and system applied to science popularization and a storage medium, and the method comprises the steps: constructing a science popularization knowledge spectrogram based on multi-source science popularization knowledge; extracting key feature data based on a voice or text question input by a user, and generating a question feature vector; similarity matching is carried out on the basis of the feature vectors and a node embedding space of the science popularization knowledge spectrogram, and accurate knowledge sub-graphs related to questions are extracted; after completing the response of the current question, recording a storage positioning index of the knowledge sub-graph corresponding to the question in the popular science knowledge spectrogram, and generating knowledge sub-graph index sequence data; and based on the next question of the user, carrying out correlation analysis, including questions of the same and different knowledge types, and carrying out sub-graph matching or full-graph matching, and if a plurality of knowledge fields are involved, carrying out sub-graph matching and full-graph matching. The method has the advantages that efficient knowledge matching is realized by fusing multi-source popular science knowledge and feature extraction, user questions are quickly responded, and the matching precision is continuously optimized.

The invention discloses a text duplicate checking method, a vehicle, a computer readable storage medium and a computer program product, and relates to the technical field of information processing. The method comprises the following steps: carrying out standardized preprocessing on an input text to be subjected to duplicate checking to obtain a current text fragment; performing semantic coding processing on the current text fragment by utilizing the target coding model to obtain a query vector; a similarity retrieval interface corresponding to the target search engine is called, a plurality of candidate vectors corresponding to the query vector are recalled from a semantic vector index, and the semantic vector index is constructed based on a historical text set and a graph index algorithm in the target search engine; sorting analysis is conducted on the multiple candidate vectors based on the multi-level duplicate judgment threshold values, a duplicate checking result is obtained, and the duplicate checking result is used for representing the duplicate situation between the historical texts corresponding to the multiple candidate vectors and the input text. The technical problems of low accuracy and low efficiency of a text duplicate checking method in related technologies are solved.

The invention provides a digital main line processing method based on an industrial data atlas, a medium and a system, and belongs to the technical field of the industrial data atlas. Unified access of cross-system data is realized by constructing a multi-source data acquisition adapter, cleaning and standardization processing are performed on heterogeneous data, an integrated data set is constructed, and the data collection efficiency is improved. Data entities are abstracted as nodes, cross-system association relationships are abstracted as edges to construct an industrial data graph, a graph index optimization strategy and a hierarchical storage architecture are adopted to improve cross-system query performance, a graph adaptive evolution model is utilized to realize dynamic adjustment of a graph structure, and a cross-system deep association analysis function is realized through a graph traversal algorithm. And an access control and data security mechanism is established to complete system protection, so that the technical problem that the industrial data processing system is difficult to realize cross-system deep correlation analysis is solved.

The invention relates to the technical field of information retrieval, in particular to a large-scale semantic retrieval optimization method based on deep learning, which comprises the following steps: acquiring to-be-retrieved text data, and converting the to-be-retrieved text data into semantic vector representation through a deep learning model; carrying out topological structure analysis on the semantic vector representation to obtain topological characteristic information of a semantic space; on the basis of the topological characteristic information, a topological perception hierarchical navigable small-world graph index with a multi-layer structure is constructed, and each layer comprises a node set and a connection relation thereof; dynamically optimizing and adjusting the topology-aware hierarchical navigable small-world graph index according to an actual query mode and system performance feedback; and receiving a query request, converting the query request into a query vector, executing multi-layer navigation search in the topology-aware hierarchical navigable small-world graph index, and returning a retrieval result related to the semantics of the query vector.

The invention provides a cross-database approximate nearest neighbor search method and system and a calculation framework, and the method comprises the following steps: constructing a graph index, storing the graph index in a relation table, and obtaining a graph index table; acquiring and storing a data set and a query set in a structured relation table form; the dismantling approximate nearest neighbor search process comprises a plurality of SQL operation stages including a candidate node screening stage, a neighbor expansion stage, a distance calculation stage, a result combination stage and a priority queue maintenance stage; and based on the graph index table, executing an iterative search process of the plurality of SQL operation stages on each query point in the query set, and finally outputting an approximate nearest neighbor search result of the query set. According to the method, the graph index is combined with the relational database, and the approximate nearest neighbor search is realized by adopting a plurality of SQL operation stages, so that the high-dimensional vector retrieval efficiency and the cross-database compatibility are remarkably improved, and the large-scale application of the vector data in a multi-element scene is promoted.

A digital image comparator includes an Faster R-CNN configured to generate a filtered set of local feature maps of an input image, a match head, and logic to preserve the local feature maps in an indexed data structure and to make the local feature maps retrievable by the match head via local feature map indexes.

The invention discloses a book resource data security collection retrieval monitoring system based on big data, and relates to the technical field of information technology and data security, and the system comprises a permission-aware association reasoning retrieval module which is in communication connection with a knowledge graph and dynamic index construction module and is used for receiving a user retrieval request and sending the user retrieval request to a database; analyzing a retrieval intention and obtaining a real-time permission context of a user to determine a security level threshold value, verifying a security attribute voucher of the data when retrieving the mixed index, only putting the data with the security level not higher than the threshold value into a result set, and performing association reasoning based on the knowledge graph under permission constraint to expand a result. According to the method, the mixed index structure fusing the inverted index, the vector index and the graph index is constructed, the security attribute voucher is associated, efficient full-text retrieval, semantic retrieval and association reasoning are supported, meanwhile, it is ensured that the retrieval process and result are strictly constrained by permission, and maximum mining of data values on the premise of security is achieved.

The invention discloses a high-dimensional space-oriented distributed graph index nearest neighbor search method, which comprises the following steps of: firstly, designing a boundary sensing balanced partition strategy, and measuring boundary characteristics of quantized nodes through a connection ratio; secondly, constructing a hierarchical mixed index architecture, vertically integrating a sparse global navigation layer and a fine-grained local precision layer, managing high-importance boundary nodes by adopting an M-Tree structure, and remarkably improving the cross-partition retrieval efficiency by utilizing the relative insensitivity of a tree structure to a curse in a boundary region; finally, a distributed multi-starting-point parallel search framework is provided, complementary entry points are dynamically generated based on query features and graph topology, vector space multi-region parallel exploration is achieved, irrelevant partitions are filtered through a navigation layer, and then accurate similarity calculation is executed in selected partitions. According to the method, on the premise that the high recall rate is guaranteed, the throughput of high-dimensional vector distributed search is remarkably improved, and an efficient and reliable solution is provided for large-scale high-dimensional vector retrieval.

The invention relates to the field of data storage, and discloses a big data compression storage and rapid retrieval method and system.The method comprises the following steps that dimension weight vectors are dynamically generated based on multi-dimensional information entropy, and rotating Hilbert codes are generated by adjusting the direction of a space filling curve; adaptively determining the number of segments and an error threshold value by using a time sequence prediction model, and compressing and storing a coding residual error by using a linear model; a double-layer collaborative index structure containing a space bitmap index and a value domain B + tree index is constructed, and local index reconstruction during weight change is achieved through an incremental updating mechanism; mapping a query range into a coding interval, screening candidate data segments in combination with an index, and fusing model parameters and residual decoding to return a result; newly added data code correction and expired data elimination are asynchronously processed, and the storage space consistency is optimized. According to the method, the unification of high compression ratio, quick retrieval and self-adaptive updating is realized.

The application relates to the technical field of network task allocation, and discloses a heterogeneous computing power network task allocation and path optimization system based on a neural network, which comprises a graph construction module, a node task allocation module and a path optimization module.The graph construction module is used for constructing a traffic dispersion graph, modeling network topology and node data through the traffic dispersion graph, and modeling nodes and edges in combination with graph indexing and graph displacement, and extracting network communication traffic and node features.The node task allocation module is used for inputting graph indexes of all nodes into an improved graph neural network model to intelligently predict and allocate node tasks, and outputting node task allocation prediction results.The path optimization module is used for obtaining path optimization results by minimizing transmission delay and balancing node computing power in combination with node task prediction results.The system optimization module is used for optimizing the overall performance of the system based on the path optimization results and the task allocation prediction results.The application improves the task allocation and path optimization efficiency of the heterogeneous computing power network.

The specification provides a vector graph index merging method and device, the method comprising: obtaining a plurality of vector graph indexes to be merged; wherein each node in the vector graph index represents a vector, and each edge represents that the vectors represented by the nodes connected by the edge are similar; for each target vector in each target vector graph index, determining the vectors similar to the target vector from each vector graph index based on an approximate nearest neighbor search algorithm to form a cross-graph candidate neighbor set of the target vector; based on the cross-graph candidate neighbor set of each vector in the plurality of vector graph indexes, constructing a global neighbor graph; wherein each node in the global neighbor graph represents each vector in the plurality of vector graph indexes, and each edge represents that the vectors represented by the nodes connected by the edge are similar; and converting the global neighbor graph into a merged graph index that can be used for approximate nearest neighbor search.

The invention discloses a GPU-based graph vector index construction method and device and a storage medium, and relates to the technical field of graph index construction. The method comprises the following steps: fragmenting an original data set based on a clustering algorithm to obtain a first preset number of fragmented data subsets; sending the ith fragmented data subset to the GPU, wherein i is a positive integer; constructing a local graph vector index of the ith fragmented data subset in the GPU; sending the (i + 1) th fragmented data subset to the GPU until all local graph vector indexes are obtained; and constructing a hierarchical global graph vector index according to all the local graph vector indexes. According to the method, the production line of the producer-consumer model is constructed, so that the data transmission time of the PCIe bus is completely covered by the calculation time of the GPU, the I / O bottleneck is eliminated, and the construction speed of the graph index is improved.

A GPU-based graph vector index construction method, apparatus, and storage medium are disclosed, relating to the field of graph index construction technology. The method includes: partitioning the original dataset using a clustering algorithm to obtain a first preset number of partitioned data subsets; sending the i-th partitioned data subset to the GPU, where i is a positive integer; constructing a local graph vector index for the i-th partitioned data subset on the GPU; sending the (i+1)-th partitioned data subset to the GPU, until all local graph vector indices are obtained; and constructing a hierarchical global graph vector index based on all local graph vector indices. This method, by constructing a producer-consumer model pipeline, completely masks the PCIe bus data transfer time with GPU computation time, eliminating I / O bottlenecks and improving the graph index construction speed.

The application belongs to the technical field of databases, and specifically discloses an efficient GPU near neighbor graph index construction method based on data locality, which comprises the following steps: dividing index original data into multiple subsets, and performing data processing on the multiple subsets to obtain rearranged data; constructing subgraphs corresponding to each subset in the rearranged data, and constructing a graph index based on the subgraphs; and refining the structure of the graph index to obtain a final graph index. The method can effectively alleviate the problems of insufficient memory resources and excessive calculation cost in the subgraph merging stage, significantly reduce the construction cost of the graph index, and effectively extend to large-scale data sets to improve scalability.

The invention belongs to the technical field of disk storage, and provides a monotonous path aware vector diagram index disk layout optimization method and system, and the method comprises the steps: preferentially distributing target nodes at the two ends of a front edge to a same target disk page through the sorting of target weights of edges corresponding to all nodes, according to the method, the data locality when each node in the vector diagram index is stored in the disk can be improved, so that the number of I / O units used when specified data in the disk is queried can be effectively reduced, and the query efficiency when the disk is queried is improved; and obtaining a total contribution value of each edge through the target disk layout optimization function and the target weight of the edge of each node, and sequentially distributing each out-of-neighbor node into the target disk page according to the total contribution value of the edge corresponding to the out-of-neighbor node of each target node in the target disk page to which the target node is distributed. And furthermore, the locality of the single target disk page data can be improved, so that the query efficiency when the target disk is queried can be further improved.

The invention discloses a heterogeneous time series data processing method and system, electronic equipment and a storage medium, and belongs to the technical field of industrial data processing. The method comprises the following steps: acquiring original working condition data of each device in a target area in real time, and converting the original working condition data into intermediate data in a unified format; performing standardization processing on the intermediate data in sequence, outputting standardized data, and performing quality verification on the standardized data; performing data fusion processing on the standardized data according to the quality verification result, and outputting and storing fused data; constructing a hierarchical bitmap index structure of fused data according to the basic index key; and obtaining and analyzing a query request, inputting an analysis result into the hierarchical bitmap index structure, performing accelerated query operation according to an index result, and outputting target data. According to the method, continuous and efficient data fusion and query processing in a large data volume state can be realized, and the expansibility is high.

Embodiments of the present application disclose a data query method, an electronic device and a computer readable storage medium. The electronic device stores a first data table including a first tuple and a first bitmap index corresponding to the first tuple. The first tuple includes one or more queryable attribute values. The first bitmap index includes a plurality of bits arranged in sequence. The method includes receiving a query request for the first data table. The query request includes a query item including an attribute value to be queried. A second bitmap index corresponding to the query item is generated based on the attribute value to be queried. The second bitmap index corresponding to the query item and values of the plurality of bits at the same positions in the first bitmap index corresponding to the first tuple are compared simultaneously to determine whether the first tuple satisfies the query item. Embodiments of the present application can improve the efficiency of data query.

The application discloses a GPU-based multi-modal index integration method. The method comprises the following steps: acquiring multi-modal original data and extracting a feature vector; mapping the feature vector to a unified common semantic space; constructing a double-layer hybrid index structure on a GPU, wherein a first layer adopts IVF-PQ index for coarse screening, and a second layer adopts graph index for fine screening; after receiving a query request, performing two-stage parallel search on the GPU, first screening a candidate set through IVF-PQ index, and then fine screening a final result through graph index. The application solves the cross-modal semantic alignment problem through the unified semantic space, realizes high-precision and low-delay retrieval of massive multi-modal data by using GPU parallel computing and a double-layer index structure, and effectively overcomes the technical bottleneck that precision and efficiency are difficult to be considered in the prior art.

The application discloses a communication product configuration method based on multi-modal analysis and vector matching, and particularly relates to the technical field of intelligent configuration of communication products, and is used for solving the problem that the efficiency of manually reading documents and manually selecting and configuring is low, errors are easy to occur, and it is difficult to trace back when the customer demand is proposed in the form of multi-source and multi-format documents, the field caliber is complex, and the rule constraints are numerous in the group and government and enterprise customer scenarios of operators. By taking the field dictionary and structured constraints as the basis, the multi-source and multi-format communication product demand document is sequentially converted into demand parameters, demand configuration graphs and multi-channel feature vectors, and vector matching of rule perception such as field mandatory, mutual exclusion of rights and interests, region, contract period and cost is performed on the hierarchical near neighbor graph index in combination with the template graph, so that the communication product configuration scheme meeting the business constraints is automatically output under the unified caliber, the workload of manually reading documents and selecting by experience is significantly reduced, and the accuracy and consistency of the configuration result in the complex scenario are improved.

In order to overcome the problems of precision and storage efficiency of an existing load searching method, the invention provides a network traffic load searching method, device and equipment, and the method comprises the following steps: generating a bitmap index structure by utilizing traffic layering and traffic load partitioning: layering a to-be-processed stream according to the size; differentially configuring index parameters for each level; partitioning the flow load by using the index parameter; filling a mapping record of the flow table, and constructing a load fragment addressing matrix; carrying out compression and disk falling to generate a bitmap index structure; carrying out load query based on the generated bitmap index structure; linearly loading each level of bitmap index structure and reading the bitmap index structure into a memory; reading an index parameter configured by the current hierarchy; performing noise reduction search in the hierarchy; performing accurate search in the hierarchy; and a parallel reduction algorithm is used, and accurate results obtained by each level are gathered and fed back to a user. Through the method, the load search accuracy and the storage efficiency are remarkably improved, and the method has a wide application prospect.

The invention relates to a K-nearest neighbor method based on quantum particle ball generation and quantum HNSW improvement, and belongs to the technical field of quantum calculation and machine learning. The method comprises the following steps: firstly, preparing classical data into a quantum state through a quantum random access memory and quantum angle coding, and efficiently generating high-purity pellets by utilizing a quantum Swap test and quantum minimum search so as to reduce the data scale; and then, dynamically adjusting the insertion hierarchy of the node in the hierarchical navigation small world according to the particle ball purity, and optimizing the graph structure balance. And finally, performing multi-layer iterative search on the constructed quantized graph index, and determining categories according to majority voting of neighbor nodes. According to the method, the parallel advantage of quantum computing is combined with an improved graph index structure, and the efficiency and expandability of a large-scale high-dimensional data classification task are remarkably improved.

The invention discloses an article recommendation method, and a training method, device and equipment of an article recommendation model, and belongs to the technical field of recommendation. The article recommendation method comprises the following steps: in response to a recommendation request of a user, determining a user vector, a historical behavior vector and article vectors of a plurality of articles; executing graph search in the graph index structure based on the user vector, and screening out a plurality of candidate items; generating a user interest vector based on an association relationship between each candidate item and the historical behavior vector; and based on the user vector, the user interest vector and the article vectors of the plurality of candidate articles, performing correlation scoring on each candidate article to obtain a scoring result, and based on the scoring result, determining a plurality of target articles to obtain an article recommendation result. According to the method, end-to-end training is realized by fusing the graph index recall and the deep attention mechanism and adopting the unified scoring model, so that the recommendation precision is improved while the recall efficiency and the expression ability are considered, and the method is suitable for construction of a large-scale online recommendation system.

The invention provides a method and a device for constructing a distributed approximate neighbor search graph index. The method comprises the following steps of: initializing each process, dividing batches for a read-only vector data set, and distributing a message buffer area; if the value of the counter is an even number, unprocessed data set batches are taken out, an adjacency list of each point is calculated in parallel, and a tuple is generated; if the counter value is an odd number, determining the affected points in the graph index updating process, and performing parallel pruning again and updating; the adjacency list length information is written into an adjacency list length array in parallel in the process; copying the information to a message sending buffer area in parallel by utilizing a parallel prefix and a calculation offset; broadcasting a sending message buffer area of each process; deserializing a tuple from the received message buffer area by each process, and updating a graph index in the process; generating tuples for the adjacency table of the updated points; printing related results by one process and outputting a graph index. According to the method, the index construction time is greatly shortened; and higher query accuracy can be provided in a query stage.