Data query method and related product thereof

Abstract

The present application discloses a data query method and a related product thereof. The method comprises: acquiring a scalar query range, a source object, B buckets, and an index map, wherein each bucket comprises a mixed object and corresponds to a scalar classification range, the mixed object is composed of a vector and a scalar, the mixed object indicates a query object of the same class as the source object, the index map comprises nodes of mixed objects and edges between the nodes, and the edges represent the vector similarities between the nodes; determining a target bucket on the basis of the scalar query range and scalar classification ranges of the B buckets; activating, in the index map, target nodes related to a mixed object in the target bucket and an edge between the target nodes to obtain an activated index map; and according to the scalar query range and on the basis of the activated index map, querying from the target bucket a target mixed object having vector similarity to the source object, and using a query object, indicated by the target mixed object, as a queried query object. By means of the present application, the query efficiency of a query object similar to the source object can be improved.

Description

Data query method and related products thereof

[0001] The present application claims priority from the Chinese patent application No. 2024112553786 entitled "Data query method and related products thereof" and filed with the China Patent Office on September 9, 2024, the content of which is incorporated herein in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of data query, in particular to a data query method and related products thereof. BACKGROUND

[0003] In the scenario of object query (such as picture query), it is usually required to query a plurality of data (such as a plurality of pictures) similar to a given source object (such as a picture) from a large number of objects to be queried (such as a large number of pictures).

[0004] In the existing application, the vector similarity between the given source object and each object to be queried can be calculated respectively, and the vector similarity can be used to reflect the object similarity between the source object and the corresponding object to be queried. Thus, the K objects to be queried with the vector similarity to the source object arranged in descending order in the front can be taken as the K objects similar to the source object.

[0005] However, the number of objects to be queried is often extremely large, such as reaching the order of magnitude of hundreds of millions, resulting in that the query efficiency is very low and the calculation overhead is also very large when the above-mentioned method is used to query the objects similar to the source object. SUMMARY

[0006] The present application provides a data query method and related products thereof, which can improve the query efficiency of the target mixed object with the vector similarity to the source object and reduce the query overhead of the target mixed object.

[0007] In one aspect, the present application provides a data query method, which comprises:

[0008] obtaining query data, the query data comprising a scalar query range and a source object;

[0009] B buckets and an index graph are acquired; each bucket corresponds to a scalar division range, each bucket contains at least one mixed object, each mixed object is composed of a vector and a scalar, each mixed object is used to indicate a respective query object, the source object and the query object are objects of the same category; the scalar of any mixed object contained in any bucket is within the scalar division range corresponding to the bucket; the index graph is constructed based on the vector similarity between the mixed objects in the B buckets; the index graph contains nodes and edges, the nodes are used to represent the mixed objects in the B buckets; the edges are used to represent the vector similarity between the nodes; B is a positive integer;

[0010] Based on the intersection between the scalar query range and the scalar division range corresponding to the B buckets, a target bucket to be queried is determined from the B buckets;

[0011] In the index graph, the target nodes related to the mixed objects in the target bucket and the edges between the target nodes are activated to obtain an activated index graph;

[0012] According to the scalar query range, a target mixed object with vector similarity with the source object is queried from the target bucket based on the activated index graph, and the query object indicated by the target mixed object is taken as the queried query object similar to the source object.

[0013] In one aspect, the present application provides a data query device, which comprises:

[0014] A first acquisition module is configured to acquire query data, the query data comprising a scalar query range and a source object;

[0015] A second acquisition module is configured to acquire B buckets and an index graph; each bucket corresponds to a scalar division range, each bucket contains at least one mixed object, each mixed object is composed of a vector and a scalar, each mixed object is used to indicate a respective query object, the source object and the query object are objects of the same category; the scalar of any mixed object contained in any bucket is within the scalar division range corresponding to the bucket; the index graph is constructed based on the vector similarity between the mixed objects in the B buckets; the index graph contains nodes and edges, the nodes are used to represent the mixed objects in the B buckets; the edges are used to represent the vector similarity between the nodes; B is a positive integer;

[0016] A determination module is configured to determine a target bucket to be queried from the B buckets based on the intersection between the scalar query range and the scalar division range corresponding to the B buckets;

[0017] An activation module is configured to activate, in the index graph, target nodes related to the mixed objects in the target bucket and edges between the target nodes to obtain an activated index graph;

[0018] The query module is configured to query, according to a scalar query range, a target mixed object having vector similarity with the source object from the target bucket based on the active index graph, and indicate a query object indicated by the target mixed object as a query object similar to the source object.

[0019] In an aspect, the present application provides a computer device, comprising a memory and a processor, the memory storing a computer program, and the computer program being executed by the processor to make the processor execute the method in the aspect.

[0020] In an aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program being executed by a processor to make the processor execute the method in the aspect.

[0021] According to an aspect of the present application, a computer program product is provided, which comprises a computer program stored in a computer readable storage medium. A processor of a computer device reads the computer program from the computer readable storage medium, and the processor executes the computer program to make the computer device execute the method provided in the aspect and various optional manners.

[0022] The present application can quickly locate the range (such as the range of the target bucket) where the object to be queried is located by the bucket strategy, without performing a large number of redundant queries on the objects that do not need to be queried, and can quickly index the target mixed object having vector similarity with the source object by the index graph strategy, and can indicate the query object indicated by the target mixed object as the final query object similar to the source object. By combining the bucket strategy and the index graph strategy, the query efficiency of the target mixed object can be greatly improved, thereby improving the query efficiency of the query object similar to the source object and reducing the query overhead of the query object similar to the source object. BRIEF DESCRIPTION OF DRAWINGS

[0023] FIG. 1 is a structural schematic diagram of a network architecture of a data query provided by an embodiment of the present application;

[0024] FIG. 2 is a schematic diagram of a scenario of a data query for a source object provided by an embodiment of the present application;

[0025] FIG. 3 is a flow schematic diagram of a data query method provided by an embodiment of the present application;

[0026] FIG. 4 is a structural schematic diagram of an index graph provided by an embodiment of the present application;

[0027] FIG. 5 is a framework schematic diagram of a data query provided by an embodiment of the present application;

[0028] FIG. 6 is a flowchart of a method for obtaining an activation index graph according to an embodiment of the present application;

[0029] FIG. 7 is a scenario diagram of obtaining an activation index graph according to an embodiment of the present application;

[0030] FIG. 8 is a flowchart of a method for querying a target hybrid object according to an embodiment of the present application;

[0031] FIG. 9 is a scenario diagram of indexing an index graph layer by layer according to an embodiment of the present application;

[0032] FIG. 10 is an effect diagram of an experimental result according to an embodiment of the present application;

[0033] FIG. 11 is a structural diagram of a data query device according to an embodiment of the present application;

[0034] FIG. 12 is a structural diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] First of all, it needs to be pointed out that all the data collected by the present application (such as query data, hybrid objects, query objects indicated by hybrid objects, and all related data) are collected with the consent and authorization of the object to which the data belongs (such as a user, an institution, or an enterprise), and the collection, use, and processing of related data need to comply with relevant laws, regulations, and standards in the relevant region.

[0036] Here, the related technical concepts involved in the present application are explained:

[0037] High-dimensional vector: a vector containing multiple coordinates (or components), which can be used to represent the characteristics of data.

[0038] Approximate k-nearest neighbor query: also known as vector similarity retrieval, is a method for quickly finding the k most similar points to a given query point in a large data set. For example, the HNSW algorithm (a graph structure-based indexing algorithm) is an algorithm for solving this approximate k-nearest neighbor query.

[0039] Vector index: a vector index is a data structure specifically used to store and retrieve high-dimensional vector data. It maps vector data into a compact and easy-to-search form, thereby accelerating operations such as vector similarity retrieval.

[0040] Hybrid object o: defined as a two-tuple (x, v) composed of a scalar attribute (which can be referred to as a scalar) x and a vector v, o.x represents the scalar attribute of the hybrid object, and o.v represents the vector of the hybrid object.

[0041] Scalar: a quantity with magnitude but no direction.

[0042] Please refer to FIG. 1, which is a structural schematic diagram of a network architecture for data query provided by an embodiment of the present application. As shown in FIG. 1, the network architecture can include a server 200 and a cluster of terminal devices, which can include one or more terminal devices, and the number of terminal devices will not be limited here. As shown in FIG. 1, the plurality of terminal devices can specifically include a terminal device 1, a terminal device 2, a terminal device 3, …, and a terminal device n; as shown in FIG. 1, the terminal device 1, the terminal device 2, the terminal device 3, …, and the terminal device n can all be network-connected with the server 200, so that each terminal device can perform data interaction with the server 200 through network connection.

[0043] The server 200 shown in FIG. 1 can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (content distribution network), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal device can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart television, a vehicle-mounted terminal, a smart home, and the like. The following will take the communication between the terminal device 1 and the server 200 as an example to specifically describe the embodiments of the present application.

[0044] Among them, the terminal device 1 can contain a query client, and the server 200 can be a background server of the query client. The query client can generate a query request in response to a user's query operation on data, and can send the query request to the server 200, so that the server 200 can perform data query through the query request.

[0045] Please refer to FIG. 2, which is a scenario diagram of data query for a source object provided by an embodiment of the present application. As shown in FIG. 2, the server 200 can parse a scalar query range and a source object (which can be an image in this case, and in actual application scenarios, the source object can also be other types of objects) from the query request sent by the query client. The scalar query range is the query range of the source object, and the server 200 needs to query a target mixed object having vector similarity with the source object within the scalar query range. Among them, a mixed object can be composed of a scalar and a vector, and the scalar and the vector of a mixed object are both used to indicate the same query object.

[0046] The server 200 can obtain the B buckets divided from the mixed objects, and an index graph constructed by the vector similarities between the mixed objects in the B buckets, the index graph can include nodes and edges, the nodes can be used to represent the mixed objects in the B buckets, and the edges between the nodes can be used to represent the vector similarities between the mixed objects. One bucket corresponds to one scalar division range, and the scalar of the mixed objects in one bucket can be in the scalar division range corresponding to the bucket.

[0047] Therefore, the server 200 can obtain the target bucket to be queried from the B buckets by the intersection between the scalar query range and the scalar division range corresponding to each bucket, and can activate the target nodes related to the mixed objects in the target bucket and the edges of the target nodes in the index graph to obtain an activated index graph. The server 200 can query the target mixed objects having the vector similarity with the source object from the target bucket through the activated index graph according to the above-mentioned scalar query range.

[0048] The server 200 can take the query object (which can also be an image, and the object type of the query object is the same as that of the source object) indicated by the target mixed object as the finally queried object similar to the source object. The server 200 can return the query object indicated by the target mixed object to the query client in the terminal device 1, so that the query client can output the received query object in the client interface for the user to view and use.

[0049] By using the method provided in the present application, the fast indexing and querying of the object similar to the source object can be realized by combining the bucket strategy and the index graph strategy, and the delay and overhead of querying the object similar to the source object can be reduced.

[0050] Please refer to FIG. 3, which is a flow diagram of a data query method provided by an embodiment of the present application. The execution subject in the embodiment of the present application can be a data query device (which can be referred to as a query device for short), which can be a computer device or a computer device cluster composed of multiple computer devices. The computer device can be a server, a terminal device, or other devices, and the present application does not limit this. As shown in FIG. 3, the method can include the following steps:

[0051] In step S101, query data is obtained, the query data including a scalar query range and a source object.

[0052] In an embodiment, the query device can obtain query data, which can include a scalar query range and a source object. As the name implies, the scalar query range is a query range of a scalar, and in the present application, the query device finally needs to query other objects similar to the source object in the scalar query range.

[0053] More, the query data can further include a query number K, which is the number of other objects similar to the source object that the application finally needs to query, in other words, the query device finally needs to query K other objects similar to the source object within the scalar query range. Optionally, the query number K can be set by the user according to actual needs, that is, the value of the query number K can be flexibly set and can not be fixed.

[0054] The above query data can be sent to the query device by the query client, for example, the process can include that the user can trigger data query in the query client and can provide (such as select or fill in the query client, etc.) the above query data such as the scalar query range, the source object, and the query number K used for data query to the query client, the query client can generate a query request through the query data, the query request can carry the query data, the query client can send the query request to the query device, and the query device can parse the query data from the received query request. In this case, the query device can be a background device of the query client.

[0055] Alternatively, the process of querying other objects similar to the source object described in the application can also be done by the query client itself, in which case the query client can query other objects similar to the source object by itself without sending a query request to the background device.

[0056] According to different application scenarios of the application, the source object and the similar objects queried from the source object in the application can also be different, and the following exemplary describes several application scenarios of the application, but is not limited to the following application scenarios. The application can be applied to any application scenario that needs to combine scalar and vector queries.

[0057] For example, in the application scenario of image query, the source object can be an image, and the scalar can be the upload time (or generation time, etc.) of the image. Therefore, the scalar query range can be a time range of the upload time of the image, such as 20100102-20100202 (i.e. the time range from January 2, 2010 to February 2, 2010), in which case the query device needs to query other images similar to the source object with upload time within the time range.

[0058] For example, in the application scenario of commodity query, the source object can be a commodity (which can be referred to by relevant description information (such as text information, video information, or image information, etc.) of the commodity), and the scalar can be the price of the commodity. Therefore, the scalar query range can be a price interval (i.e., a price range) of the commodity, such as 100-1000 (i.e., a price interval of 100 to 1000), in which case the query device needs to query other commodities similar to the source object and having a price within the price interval.

[0059] For example, in the application scenario of audio query (such as song query), the source object can be an audio, and the scalar can be the publication time (such as the time of publication) of the audio. Therefore, the scalar query range can be a time range of the publication time of the audio, such as 20120102-20120202 (i.e., a time range of January 2, 2012 to February 2, 2012), in which case the query device needs to query other audios similar to the source object and having a publication time within the time range.

[0060] Alternatively, in more application scenarios, the scalar can also be an object identifier (such as a number) of the query object, or any other type of scalar.

[0061] In step S102, B buckets and an index graph are obtained. Each bucket corresponds to a scalar division range, and each bucket contains at least one mixed object. Each mixed object is composed of a vector and a scalar, and each mixed object is used to indicate a respective query object. The source object and the query object are objects of the same category. The scalar of any mixed object contained in any bucket is within the scalar division range corresponding to the bucket. The index graph is constructed based on the vector similarity between the mixed objects in the B buckets. The index graph contains nodes and edges. The nodes are used to represent the mixed objects in the B buckets. The edges are used to represent the vector similarity between the nodes. B is a positive integer.

[0062] In an embodiment, the query device can obtain B buckets and an index graph, where B is a positive integer. The B buckets are divided from a large number of mixed objects. The large number of mixed objects can be mixed objects in a prepared database, or mixed objects sampled from a full database. In this case, since the order of magnitude of the data in the full database is very large, sampling a part of the mixed objects (in fact, a very large number of mixed objects are also sampled) can fully meet the query requirements of the user. The part of the mixed objects sampled can be used for the following bucket processing.

[0063] The vector of each mixed object can be a feature vector (which can be a high-dimensional vector) of the query object indicated by the mixed object, and the scalar of the mixed object can be a scalar associated with the query object. In other words, the vector and the scalar in a mixed object can both belong to the query object indicated by the mixed object, and the query object indicated by the mixed object is the query object to which the mixed object belongs.

[0064] The category of each query object indicated by a mixed object can be the same as the category of the source object, that is, the source object and the query object can be objects of the same category, that is, the other object similar to the source object that needs to be finally queried can be an object of the same category as the source object.

[0065] In this application, the vector of each mixed object can be generated by performing feature extraction processing on each query object indicated by the mixed object through the same feature extraction network (i.e., a feature extraction model, which can be an AI (artificial intelligence) model of deep learning). For example, if the query object is an image, the feature extraction network can be an image feature extraction network; if the query object is text, the feature extraction network can be a text feature extraction network; if the query object is a video, the feature extraction network can be a video feature extraction network; and if the query object is audio, the feature extraction network can be an audio feature extraction network.

[0066] For example, in the application scenario of image query, the vector and the scalar of a mixed object can both be used to indicate an image, the vector of the mixed object can be a feature vector of the image, and the scalar can be a scalar associated with the image, such as the upload time of the image.

[0067] The scalar of each mixed object in a bucket can be within the scalar division range corresponding to the bucket, or in other words, a bucket is constructed based on mixed objects whose scalars are within the scalar division range corresponding to the bucket.

[0068] The B buckets can be constructed in advance, and a process of constructing the B buckets is described as follows: the B scalar division ranges corresponding to the B buckets can be obtained by dividing a global scalar range. Optionally, the global scalar range can be a scalar range in which scalars of all mixed objects that need to be divided into buckets are located.

[0069] Optionally, the global scalar range can be determined by a maximum scalar possessed by all mixed objects that need to be divided into buckets, such as the global scalar range can be a range from a minimum value (such as 0) that the scalar can reach to a value of the maximum scalar. Alternatively, the global scalar range can also be determined by a minimum scalar possessed by all mixed objects that need to be divided into buckets and a maximum scalar possessed by all mixed objects that need to be divided into buckets, such as the global scalar range can be a range from a value of the minimum scalar to a value of the maximum scalar. The global scalar range is the overall and complete range that needs to be divided into scalar ranges, and the specific global scalar range can be determined according to an actual application scenario, as long as the global scalar range can contain scalars of all mixed objects that need to be divided into buckets, and the present application does not limit the global scalar range.

[0070] The query device can obtain the number B of buckets, which can be set according to an actual application scenario, such as 4 or 8, and the query device can generate B-1 quantile numbers of the global scalar range according to the number B of buckets, and the B-1 quantile numbers can be used to divide the global scalar range into B parts. For example, B is equal to 4, and the global scalar range is [0, 100], and the B-1 quantile numbers generated according to the global scalar range can be 25%*100, 50%*100, and 75%*100, that is, 25, 50, and 75.

[0071] Therefore, the query device can divide the global scalar range into B scalar division ranges according to the B-1 quantile numbers generated above, and the range sizes of the B scalar division ranges can be the same (or close). For example, for the case that B is equal to 4 and the global scalar range is [0, 100], the four scalar division ranges divided by the quantile numbers 25, 50, and 75 can include the scalar division range [0, 25], the scalar division range (25, 50], the scalar division range (50, 75], and the scalar division range (75, 100].

[0072] After obtaining the B scalar partition ranges of the scalar global range, the query device can construct the above-mentioned B sub-buckets through the B scalar partition ranges. One scalar partition range is a sub-interval of the scalar global range. For example, the query device can construct the above-mentioned B sub-buckets through the mixed objects whose scalars are in the B scalar partition ranges. In other words, the query device can construct one sub-bucket corresponding to one scalar partition range through the mixed objects whose scalars are in the scalar partition range.

[0073] In the above-mentioned index graph, each sub-bucket can be sequentially sorted according to the scalar values in the corresponding scalar partition range in ascending order, and the sorted serial number (such as 1, 2, 3…) can be used as the sub-bucket number of each sub-bucket. The sub-bucket number of each sub-bucket can be used to identify each sub-bucket.

[0074] In the present application, by uniformly dividing and constructing each sub-bucket, the number of mixed objects in each sub-bucket can be substantially uniform, so that the number of mixed objects in one sub-bucket does not become too large, which can cause the query efficiency to drop to the bottom. In other words, by uniformly dividing each sub-bucket, the present application can ensure high efficiency and high stability of data query.

[0075] The above-mentioned index graph can include nodes of each mixed object in the B sub-buckets and edges between the nodes (i.e., edges connecting the nodes). The nodes in the index graph can be used to represent the mixed objects in the B sub-buckets. One mixed object can have one or more nodes in the index graph to represent it. The index graph can be constructed through the vector similarity between the mixed objects in the B sub-buckets (i.e., the vector similarity between the vectors of the mixed objects). Therefore, the edges in the index graph can be used to represent the vector similarity between the nodes (i.e., the vector similarity between the mixed objects represented by the nodes). For example, the vector similarity between the nodes connected by the edges is relatively high, while the vector similarity between the nodes not connected by the edges is relatively low.

[0076] In the above-mentioned index graph, the S-layer index structure can be included, S is a positive integer, and the edges in the index graph include the edges constructed between the nodes of any mixed object in any sub-bucket and the nodes of the mixed objects in each sub-bucket. This also shows that the index graph is constructed through the vector similarity between the mixed objects in each sub-bucket.

[0077] The mixed object can have an index structure of a target layer number in the index graph, each layer index structure of the mixed object in the index graph smaller than and equal to the target layer number has a node, and the node of the mixed object in the layer index structure can construct less than or equal to M edges between the nodes of the mixed objects in a sub-bucket, M being a positive integer. In other words, the node of the mixed object can construct at most M edges between the nodes of the mixed objects in a sub-bucket in a layer index structure, and the value of M can be determined according to the actual application scenario, for example, M can be 16.

[0078] The node in the i-1th layer index structure includes the node in the i th layer index structure, that is, the node in the next layer index structure in the index graph can include the node in the previous layer index structure, and the nodes in the i-1th layer index structure and the i th layer index structure for representing the same mixed object can be connected by edges, i being an integer greater than 1 and less than or equal to S.

[0079] For example, if S is equal to 50, that is, the index graph can have 50 layer index structures, the target layer number of the mixed object in the index graph is the 10th layer index structure, each layer index structure from the 1st layer to the 10th layer in the index graph includes a node for representing the mixed object, and the nodes for representing the mixed object in different and adjacent layer index structures can be connected by edges.

[0080] In other words, the mixed object can have a node in the 1st layer index structure, a node in the 2nd layer index structure, a node in the 3rd layer index structure, …, and a node in the 10th layer index structure in the index graph, and the node of the mixed object in the 1st layer index structure can be connected by an edge to the node in the 2nd layer index structure, the node of the mixed object in the 2nd layer index structure can be connected by an edge to the node in the 3rd layer index structure, the node of the mixed object in the 3rd layer index structure can be connected by an edge to the node in the 4th layer index structure, and so on, and the node of the mixed object in the 9th layer index structure can be connected by an edge to the node in the 10th layer index structure.

[0081] In an embodiment, the flow of generating the index graph can include that the index graph is constructed by incremental insertion, i.e., the index graph is constructed by inserting each mixed object one by one. Therefore, when there is a newly added object to be queried (e.g., a newly added query object), the application can also easily insert the node of the mixed object corresponding to the newly added query object into the constructed index graph for indexing. The following description takes the process of inserting a mixed object (which can be any mixed object) into the index graph as an example for illustration.

[0082] The query device can generate the target layer number of any mixed object in a probability decreasing manner, i.e., the insertion probability of the node decreases layer by layer in the order of the layer number of the index graph from low to high. In other words, when the target layer number of the mixed object is generated in the probability decreasing manner, the probability of the target layer number being a lower layer number is higher, and the probability of the target layer number being a higher layer number is lower. For example, the probability decreasing manner can be an exponential decreasing manner, i.e., when the target layer number of the mixed object is generated, the values that the target layer number can take are exponentially decreasing in the order from low to high.

[0083] Therefore, it can be understood that when there is a mixed object to be inserted, the layer number of the index structure in the index graph is not fixed and can be increased, i.e., the layer number of the index structure in the index graph can be infinite (or a very large finite layer number can also be set), because the target layer number of the mixed object to be inserted is uncertain, and the target layer number can also be a large layer number.

[0084] Therefore, the query device can insert a node of the any mixed object into each insertion layer of the index structure corresponding to the any mixed object, and one insertion layer can insert one node of the any mixed object. The insertion layer of the any mixed object can include the index structure whose layer number is less than and equal to the target layer number of the any mixed object.

[0085] The query device can establish an edge between the node of the any mixed object and the node of the mixed object in each bucket in each layer of the index structure where the node of the any mixed object is located. Since the principle of establishing the edge between the node of the any mixed object and the node of the mixed object in each bucket in each layer of the index structure where the node of the any mixed object is located is the same, the following takes the process of establishing an edge between the node of the any mixed object and the node of the mixed object in any bucket (which can be referred to as a target bucket) in B buckets in any layer of the index structure (i.e., the index structure of the target layer, which can be referred to as the index structure of the target layer) as an example for specific description.

[0086] The query device can obtain M nodes in front of the nodes between the any mixed object and the nodes in the order from large to small according to the vector similarity between the any mixed object and the nodes in the node of the mixed object in the target layer of the index structure in the target bucket, and M is a positive integer. The query device can establish edges between the any mixed object and the M nodes in the index structure of the target layer.

[0087] In the above embodiment, the query device can also query the nodes in each bucket with the top M (i.e., the largest M) vector similarity with the any mixed object from each layer of the index structure in which the node of the any mixed object is located in the current constructed index graph, according to the same principle of querying the target mixed objects with the vector similarity with the source object in the active index graph, so as to establish the edges between the node of the any mixed object and the nodes of the mixed objects in each bucket in each layer of the index structure in which the node of the any mixed object is located.

[0088] In other words, the principle of querying the M nodes in the mixed object node in any bucket with the any mixed object node that needs to establish edges in any layer of the index structure in which the any mixed object is located in the current constructed index graph is the same as the principle of querying the K target mixed objects of the source object from the active index graph, which can be referred to the related description in the corresponding embodiment of FIG. 8 below. Therefore, it can be understood that the more recently inserted nodes can establish edges with more complete nodes (i.e., the nodes that have been inserted into the index graph at this time).

[0089] More, since each mixed object is incrementally inserted into the index graph, and a mixed object builds at most M edges between the nodes of the mixed objects in a layer index structure and a bucket, before establishing an edge between a mixed object (such as mixed object A) in a layer index structure and the nodes of the mixed objects in a bucket, it is possible that the nodes of other mixed objects in the bucket have already established edges with the mixed object A when they are inserted, so after obtaining the M nodes with the greatest vector similarity between the mixed object A and the nodes of the mixed objects in the bucket in the layer index structure, and establishing edges between the mixed object A and the M nodes, since the number of edges between the mixed object A and the nodes of the mixed objects in the bucket in the layer index structure is greater than M, it is necessary to prune the edges of the mixed object A, that is, to prune the number of edges between the mixed object A and the nodes of the mixed objects in the bucket in the layer index structure to M, for example, among the more than M edges between the mixed object A and the nodes of the mixed objects in the bucket in the layer index structure, the edges between the M nodes with the greatest vector similarity between the mixed object A and the nodes of the mixed objects in the bucket (the nodes of the mixed objects in the bucket) are retained, and the edges between the other nodes with the vector similarity not in the top M are removed.

[0090] Please refer to FIG. 4, which is a structural diagram of an index graph provided by an embodiment of the present application. As shown in FIG. 4, the index graph can have three layers of index structures, including a 0th layer of index structure, a 1st layer of index structure, and a 2nd layer of index structure, each of which can contain a plurality of nodes, the nodes in the next layer of index structure can include the nodes in the previous layer of index structure, and the nodes representing the same mixed object in adjacent index structures can be connected by edges (represented by dashed lines in FIG. 4).

[0091] Among them, the nodes with the same number in the index graph of FIG. 4 can be used to represent the same mixed object, the 2nd layer of index structure contains node ① and node ②, the 1st layer of index structure contains node ①, node ②, node ③, and node ④, and the 0th layer of index structure contains node ①, node ②, node ③, node ④, node ⑤, and node ⑥.

[0092] The node ① in the 2nd layer of index structure and the node ① in the 1st layer of index structure are used to represent the same mixed object, so the node ① in the 2nd layer of index structure and the node ① in the 1st layer of index structure can be connected by an edge; the node ② in the 2nd layer of index structure and the node ② in the 1st layer of index structure are used to represent the same mixed object, so the node ② in the 2nd layer of index structure and the node ② in the 1st layer of index structure can be connected by an edge.

[0093] And, the node 1 in the 1st layer index structure and the node 1 in the 0th layer index structure are used for representing the same mixed object, therefore, an edge can be connected between the node 1 in the 1st layer index structure and the node 1 in the 0th layer index structure; the node 2 in the 1st layer index structure and the node 2 in the 0th layer index structure are used for representing the same mixed object, therefore, an edge can be connected between the node 2 in the 1st layer index structure and the node 2 in the 0th layer index structure; the node 3 in the 1st layer index structure and the node 3 in the 0th layer index structure are used for representing the same mixed object, therefore, an edge can be connected between the node 3 in the 1st layer index structure and the node 3 in the 0th layer index structure; the node 4 in the 1st layer index structure and the node 4 in the 0th layer index structure are used for representing the same mixed object, therefore, an edge can be connected between the node 4 in the 1st layer index structure and the node 4 in the 0th layer index structure.

[0094] In step S103, the target bucket to be queried is determined from the B buckets based on the intersection between the scalar query range and the scalar division range corresponding to each of the B buckets.

[0095] Specifically, the query device can determine the target bucket to be queried from the B buckets based on the intersection between the scalar query range and the scalar division range corresponding to each of the B buckets. There can be one or more target buckets, which specify the range of other objects similar to the source object.

[0096] For example, the query device can select the bucket whose corresponding scalar division range intersects (i.e., overlaps, which can be partial overlap or complete overlap) with the scalar query range as the target bucket to be queried.

[0097] For example, the B buckets can include bucket 1, bucket 2, bucket 3, and bucket 4. The scalar division range corresponding to bucket 1 can be [0, 25], the scalar division range corresponding to bucket 2 can be (25, 50], the scalar division range corresponding to bucket 3 can be (50, 75], and the scalar division range corresponding to bucket 4 can be (75, 100]. The scalar query range is [31, 56], so the scalar division range corresponding to bucket 2 (25, 50] and the scalar division range corresponding to bucket 3 (50, 75] both intersect with the scalar query range [31, 56], and therefore, bucket 2 and bucket 3 can both be selected as the target bucket.

[0098] By selecting the target bucket, the query device does not need to query a large number of nodes of mixed objects that are not within the scalar query range specified by the user when indexing the nodes using the index graph, which greatly improves the efficiency of indexing the nodes using the index graph and greatly reduces the amount of calculation and index overhead for indexing the nodes.

[0099] Step S104, activating the target nodes related to the mixed objects in the target bucket and the edges between the target nodes in the index graph, to obtain an activated index graph.

[0100] In an embodiment, the query device can activate the nodes related to the mixed objects in the target bucket (which can be referred to as target nodes) and the edges between the target nodes in the index graph, and there can be multiple target nodes, so as to obtain an activated index graph. It can be understood that the activated index graph can be composed of the activated target nodes and the activated edges between the target nodes.

[0101] In the present application, when indexing the nodes through the index graph, the activated target nodes can be indexed through the activated edges based on the activated index graph, which improves the efficiency of indexing the nodes and reduces the overhead of indexing the nodes.

[0102] Step S105, querying the target mixed objects having vector similarity with the source object from the target bucket based on the activated index graph according to the scalar query range, and taking the query object indicated by the target mixed object as the query object similar to the source object.

[0103] In an embodiment, the query device can query the mixed objects having vector similarity with the source object from the target bucket based on the activated index graph obtained above according to the scalar query range described above, and the query mixed object can be referred to as a target mixed object, and there can be multiple target mixed objects, such as K target mixed objects, and the scalar of the target mixed object is within the scalar query range.

[0104] The specific process of querying the target mixed objects having vector similarity with the source object from the target bucket through the activated index graph can be referred to the related description in the corresponding embodiment of FIG. 8 below.

[0105] The vector similarity in the present application is the similarity between vectors, such as the cosine similarity between two vectors. The greater the vector similarity between two vectors, the more similar the two objects (such as two query objects or a query object and a source object) to which the two vectors belong. The vector similarity between the node and the source object described in the present application can refer to the vector similarity between the vector of the mixed object represented by the node and the vector of the source object. Similarly, the vector similarity between the mixed object and the source object described in the present application can refer to the vector similarity between the vector of the mixed object and the vector of the source object. The vector of the source object can also be a feature vector generated by performing feature extraction on the source object through the feature extraction network described above.

[0106] After the target mixed object is queried, the query device can take the query object indicated by the target mixed object as the final query object similar to the source object in the scalar query range. Therefore, the query device can return the query object indicated by the target mixed object (i.e., the query object similar to the source object) to the query client, so that the query client can display the query object returned by the query device in association with the source object for the user to view and use (such as download, save, forward, like, comment, etc.).

[0107] Please refer to FIG. 5, which is a schematic diagram of a data query framework provided by an embodiment of the present application. As shown in FIG. 5, the framework for data query in the present application can have two modules, one is an offline processing module and the other is an online processing module. In the offline processing module, relevant data required for subsequent data query (such as object query) can be prepared, such as the B buckets and the index graph described above. In the online processing module, it can be used to respond to the query request of the user and query other objects related to the source object (such as the K query objects indicated by the K target mixed objects in the query) for the user. The operation in the offline processing module can be performed in advance before the operation in the online processing module, which belongs to the preparation operation for data query.

[0108] In the offline processing module, the first operation "data preparation" can be performed, that is, the mixed object data set required for subsequent data query is prepared, and the mixed object data set can contain all mixed objects prepared for subsequent data query. The source of the mixed object can be determined according to the actual application scenario, or the source of the query object indicated by the mixed object can be determined according to the actual application scenario, such as being obtained by crawling from the network, or being obtained by sampling from another full database, or being generated by some means, etc., which is not limited in the present application.

[0109] After obtaining the mixed object data set, the second operation "constructing a mixed index graph" can be performed, which includes sub-operation 2.1 "data bucketing" and sub-operation 2.2 "graph connection construction". The sub-operation 2.1 "data bucketing" is an operation of performing bucketing on the mixed objects in the mixed object data set to obtain the B buckets described above. The sub-operation 2.2 "graph connection construction" is an operation of constructing an index graph by the B buckets obtained by bucketing. Therefore, by performing the second operation, a mixed index graph can be obtained, which is the index graph described above, mixed with the index of the scalar and the index of the vector.

[0110] The generated mixed index graph can be applied in the online processing module. In the online processing module, the third operation "query preparation" can be performed first, that is, the user can give a query range (that is, the above-mentioned scalar query range) and a source object through a query client. Thus, the query range and the source object can be given to a query device, and the query device can perform the fourth operation by indexing and searching K target mixed objects having vector similarity with the source object in the query range through the generated mixed index graph.

[0111] In the present application, the bucketing strategy and the index graph strategy are combined with each other, so that the target mixed object having vector similarity with the source object can be queried with low overhead, high efficiency and accuracy. The method of the present application can be applied to any scene requiring scalar and vector mixed query, and the scalar can belong to structured data, and the source object can belong to unstructured data. Therefore, the present application can also be applied to a scene requiring mixed query of structured data and unstructured data.

[0112] The present application can quickly locate the range (such as the range of the target bucket) of the object to be queried through the bucketing strategy, without performing a large amount of redundant query on the object that does not need to be queried. The target mixed object having vector similarity with the source object can be quickly indexed through the index graph strategy, and the query object indicated by the target mixed object can be used as the final query object similar to the source object. The present application combines the bucketing strategy and the index graph strategy, so that the query efficiency of the target mixed object can be greatly improved, the query efficiency of the query object similar to the source object can be improved, and the query overhead of the query object similar to the source object can be reduced.

[0113] Please refer to FIG. 6, which is a flowchart of a method for obtaining an active index graph according to an embodiment of the present application. As shown in FIG. 6, the method can include the following steps:

[0114] In step S201, an activation control parameter is obtained.

[0115] In an embodiment, the query device can obtain the activation control parameter. Since the edges connected between the target nodes are also many, the present application can control the number of activated edges between the target nodes through the activation control parameter (which can be denoted as al), so as to further improve the efficiency of node indexing and further reduce the overhead of node indexing.

[0116] The activation control parameter can be a positive integer, and the activation control parameter can be equal to M or slightly larger than M, such as 2 larger than M.

[0117] Step S202, calculate the ratio between the parameter value of the activation control parameter and the number of the target buckets, and obtain the activation connection number N.

[0118] In an embodiment, the query device can calculate the value obtained by dividing the parameter value of the activation control parameter (i.e. the value of the activation control parameter) by the number of the target buckets (i.e. the number of the target buckets), and can refer to the value as the activation connection number N. The activation connection number N can be less than or equal to M, and in general, the activation connection number N is less than M, so as to achieve the effect of accelerating the node index.

[0119] Subsequently, the edges between the target nodes can be activated by using the activation connection number N, as described below.

[0120] Step S203, based on the activation connection number N, activate the target nodes and the edges between the target nodes related to the mixed objects in the target buckets in the index graph, and obtain the activated index graph.

[0121] In an embodiment, any mixed object in the target buckets can be referred to as a target mixed object. Since the edges between the target nodes are activated according to the granularity of each mixed object in the target buckets, and the principle of activating the edges related to each mixed object is the same, the following is described by taking activating the edges related to the target mixed object as an example. It should be noted that for each mixed object in the target buckets, the following activation process of the related edges is performed.

[0122] The query device can take the nodes of the mixed objects in the target buckets in the index graph as target nodes, and can activate the target nodes, i.e. the query device can activate the nodes of each mixed object in the target buckets in each layer index structure of the index graph.

[0123] In addition, the query device can arrange the first N edges between the node of the target mixed object in each layer index structure of the index graph and the nodes of the mixed objects in each target bucket as target edges, i.e. the target mixed object can have N target edges between the node thereof and the nodes of the mixed objects in one target bucket in one layer index structure.

[0124] The edges between the node of the target mixed object and the nodes of the mixed objects in the target bucket in the index structure in which the node of the target mixed object is located can be arranged in descending order of the vector similarity between the target mixed object and the mixed objects in the target bucket. Therefore, the target edges between the node of the target mixed object and the nodes of the mixed objects in the target bucket in the index structure in which the node of the target mixed object is located can include edges between the node of the target mixed object and the nodes of the mixed objects in the target bucket in the index structure in which the node of the target mixed object is located and which have the topN (i.e., the largest N) vector similarity with the node of the target mixed object.

[0125] Therefore, the query device can activate N target edges between the node of the target mixed object and the nodes of the mixed objects in each target bucket in each index structure in which the node of the target mixed object is located in the index graph, i.e., N target edges between the node of the target mixed object and the nodes of the mixed objects in one target bucket in the index structure in which the node of the target mixed object is located. If there are F target buckets and the insertion layer of the target mixed object has C, a total of CxFxN edges can be activated for the target mixed object.

[0126] The query device can activate the target edges associated with each mixed object in each target bucket according to the above principle, and the activated target edges are the edges in the activated index graph. Of course, in addition to activating the target edges between the nodes of each mixed object in the target bucket and the nodes in each target bucket, the query device can also activate the edges between the nodes of the same mixed object in different and adjacent index structures, in other words, the activated index graph can also include the edges between the nodes of the same mixed object in different index structures in the target bucket.

[0127] Of course, it can be understood that since the nodes in the index graph are inserted in a probabilistic decreasing manner from low to high layers, for the nodes in the higher layer range in the index graph, which belong to the nodes of the mixed objects in which buckets are randomly determined based on the target layer number when each node is inserted, and therefore, it is possible that the nodes in the higher layer range in the index graph do not include the nodes of the mixed objects in the target bucket, of course, it is very likely that the nodes in the higher layer range in the index graph still include the nodes of the mixed objects in the target bucket.

[0128] In view of this, it can be understood that the activated index graph can include G index structures, G is a positive integer, and G is less than or equal to S, i.e., the number of index structures in the activated index graph can be less than or equal to the number of index structures in the original index graph. If the activated index graph has S index structures, i.e., G is equal to S, it indicates that the Gth index structure in the index graph also includes the nodes of the mixed objects in the target bucket.

[0129] Please refer to FIG. 7, which is a scene diagram of acquiring an activated index graph according to an embodiment of the present application. As shown in FIG. 7, it is assumed that the B buckets include four buckets, i.e., bucket 1, bucket 2, bucket 3 and bucket 4, and each of the buckets contains a plurality of mixed objects. The query device can divide the four buckets into target buckets, in which the corresponding scalar division ranges intersect with the given scalar query range. In this case, the target buckets can include bucket 2 and bucket 3.

[0130] The query device can activate the nodes of the mixed objects in the bucket 2 and the bucket 3 in the index graph, and activate the edges between the activated nodes (or a part of the edges between the activated nodes), so as to obtain an activated index graph.

[0131] According to the granularity of each target bucket and the granularity of each mixed object, the present application activates a proper number (e.g., N) of edges, so that the number of the activated edges is not particularly large, which is beneficial to the subsequent fast query of data. Of course, if the completeness of the query is to be ensured, the above-mentioned activation control parameter can be relatively large, so that N approaches M or is equal to M.

[0132] Please refer to FIG. 8, which is a flow diagram of a method for querying a target mixed object according to an embodiment of the present application. The present embodiment describes a process for querying a target mixed object from a target bucket by activating an index graph according to a scalar query range. As shown in FIG. 8, the process can include the following steps:

[0133] In step S301, based on the source object, the query device indexes from the Gth layer index structure of the activated index graph to the last layer index structure of the activated index graph layer by layer.

[0134] In an implementation, the query device can index from the Gth layer index structure of the activated index graph to the last layer index structure of the activated index graph layer by layer based on the source object. The last layer index structure is the last layer index structure of the activated index graph. If the number of layers of the index structure is counted from 0, the last layer index structure is the 0th layer index structure in the activated index graph.

[0135] For both the index graph and the activated index graph, the higher the layer of the index structure is, the larger the index granularity is, and the lower the layer of the index structure is, the smaller the index granularity is. Because the higher layer contains fewer nodes, and the lower layer contains more nodes.

[0136] Therefore, the query device can index from the higher layer index structure of the activated index graph to the lower layer index structure step by step. This process can be imagined as continuously narrowing and positioning the range of the similar object for querying the source object. Finally, the target mixed object can be specifically queried in the last layer index structure of the activated index graph, as described below.

[0137] Herein, the process of indexing from the jth layer index structure of the active index graph by the source object to the (j-1)th layer index structure is described, j is a number greater than 1 and less than or equal to G, thus, the jth layer index structure can be any layer index structure of the active index graph except the last layer index structure, and the process can include:

[0138] The querying device can obtain the cross-layer node in the jth layer index structure of the active index graph by the source object, which is the node used to index to the next layer index structure of the jth layer index structure, i.e., the (j-1)th layer index structure. Specifically, if j is equal to G, i.e., the jth layer index structure is the Gth layer index structure of the active index graph (the highest layer index structure of the active index graph), the querying device can obtain the target entry node of the active index graph in the Gth layer index structure as the cross-layer node in the jth layer index structure.

[0139] Herein, it needs to be noted that each of the B buckets described above can have an entry node in the index graph, and the entry node of a bucket in the index graph can be the last inserted node of the highest layer index structure in which the node of the mixed object in the bucket is located in the index graph, because the last inserted node is the most likely to establish an edge with the previously inserted node.

[0140] Therefore, it can be understood that the target entry node of the active index graph in the Gth layer index structure can be the entry node of the target bucket in which the entry node exists in the Gth layer index structure, if the Gth layer index structure (the Gth layer index structure described later is the Gth layer index structure of the active index graph) contains multiple entry nodes of multiple target buckets, then any one of the multiple entry nodes can be selected as the target entry node described above, or the last inserted one of the multiple entry nodes can be selected as the target entry node described above, which is not limited by the present application.

[0141] Subsequently, the present application can index from the Gth layer index structure to the last layer index structure of the active index graph based on the target entry node layer by layer in turn.

[0142] If j is not equal to G, i.e., the jth layer index structure is not the Gth layer index structure of the active index graph, the querying device can obtain the node with the highest vector similarity with the vector of the source object in the jth layer index structure as the cross-layer node in the jth layer index structure.

[0143] In the process of obtaining the node with the highest vector similarity between the vector of the source object in the jth layer index structure, the cross-layer node indexed to the jth layer index structure can be quickly indexed in the jth layer index structure by indexing, and the indexing principle is the same as the principle of querying and updating the result set in the last layer index structure by the target cross-layer node. In the process of obtaining the node with the highest vector similarity between the vector of the source object in the jth layer index structure, at most only one node can be recorded in the result set, and the node in the result set obtained after the final indexing is the node with the highest vector similarity between the vector of the source object in the jth layer index structure.

[0144] Therefore, the cross-layer node obtained in the jth layer index structure can be indexed to the j-1th layer index structure of the active index graph from the jth layer index structure. Indexing to the j-1th layer index structure of the active index graph from the jth layer index structure can refer to: indexing the cross-layer node in the j-1th layer index structure through the cross-layer node in the jth layer index structure, i.e., the node belonging to the same hybrid object as the cross-layer node, and further indexing the cross-layer node in the j-1th layer index structure. The indexing principle is the same as the principle of querying and updating the result set of the target hybrid object, so that the cross-layer node indexed in the j-1th layer index structure can be further indexed to the j-2th layer index structure of the active index graph, and so on.

[0145] The vector of the source object is the feature vector of the source object, which can also be generated by the query device by calling the feature extraction network to perform feature extraction on the source object.

[0146] Through the above process, it can be understood that in each layer index structure in the active index graph except the last layer index structure, a cross-layer node can be obtained for indexing to the next layer index structure, until the last layer index structure of the active index graph is indexed layer by layer.

[0147] In step S302, the target hybrid object is queried in the last layer index structure indexed to according to the scalar query range.

[0148] In an embodiment, the query device can query the target hybrid object with vector similarity between the source object (i.e., between the vector of the source object) in the last layer index structure indexed to according to the scalar query range, as described below.

[0149] The query device can obtain the vector of the source object, which can be generated by performing feature extraction on the source object through the feature extraction network.

[0150] The query device can perform several times of index processing in the last layer index structure through several index nodes to finally query the target mixed object. One index node can perform one index processing, and the process can include: the query device can obtain a result set and a candidate set obtained after the Zth index processing in the last layer index structure through the Zth index node. Z is a positive integer.

[0151] The result set (i.e., the result set after the Zth index processing) is contained in the nodes indexed after the Zth index processing, and the scalar (i.e., the scalar of the mixed object represented by the node) in the scalar query range and the top E nodes with the vector similarity between the vector of the source object arranged in descending order, in other words, the result set is used to update and record the top E nodes (i.e., the maximum E nodes) in the scalar query range and the vector similarity between the vector of the source object in all the latest indexed nodes when performing node indexing in the last layer index structure. E is a positive integer, and the value of E can be determined according to the actual application scenario. E can be greater than K or equal to K. The greater E is, the higher the accuracy and precision of node indexing will be.

[0152] The candidate set (i.e., the candidate set after the Zth index processing) can include the nodes added from the indexed nodes through the result set. When one indexed node is added to the candidate set, the vector similarity between the node and the source object (i.e., the vector similarity between the vector of the mixed object represented by the node and the vector of the source object) is greater than or equal to the minimum vector similarity between the nodes in the result set and the source object at that time. The capacity of the candidate set can be infinite, that is, the candidate set can continuously add new nodes without deleting (i.e., removing) the added nodes.

[0153] In particular, when the first index node is used for the first time, the first index node can be used to update the initial result set and the initial candidate set to obtain the result set and the candidate set after the first index processing. The initial result set and the initial candidate set can be initialized sets, and the initial result set and the initial candidate set can initially only include the first index node.

[0154] If Z is equal to 1, the Zth index node (i.e., the first index node) is a node of the mixed object represented by the target cross-layer node in the last layer index structure, i.e., a node in the last layer index structure having a connected edge with the target cross-layer node, and the node and the target cross-layer node represent the same mixed object.

[0155] If Z is greater than 1, the Zth index node (i.e., an index node after the first index node) is a node with the largest vector similarity between the vector of the source object from the candidate set after the (Z-1)th index processing (i.e., the last index processing). That is, the Zth index node is taken from the candidate set after the (Z-1)th index processing, and the subsequent Zth index processing is performed on the candidate set after the (Z-1)th index processing and the result set after the (Z-1)th index processing.

[0156] The query object can determine the target mixed object in the last layer index structure by using the result set and the candidate set obtained after the Zth index processing, as described below.

[0157] If the maximum vector similarity between the nodes in the candidate set after the Zth index processing and the vector of the source object (i.e., the maximum vector similarity among the vector similarities between each node in the candidate set and the source object) is less than the minimum vector similarity between the nodes in the result set after the Zth index processing and the vector of the source object (i.e., the minimum vector similarity among the vector similarities between each node in the result set and the source object), it indicates that the loop stop condition of the index processing is reached, and the subsequent index processing can be omitted, and the target mixed object can be directly selected from the result set after the Zth index processing, such as selecting K target mixed objects.

[0158] For example, the process of selecting the target mixed object from the result set obtained after the Zth indexing processing can include that the result set obtained after the Zth indexing processing can contain E nodes, the query data includes a query number K, and K is less than E. Therefore, the query device can sort the E nodes in the result set obtained after the Zth indexing processing in descending order of the vector similarity between the vectors of the source object and the nodes, to obtain the sorted E nodes.

[0159] Therefore, the query device can select the K mixed objects represented by the K nodes arranged in the front of the sorted E nodes as the target mixed object finally queried.

[0160] If the maximum vector similarity between the nodes in the candidate set obtained after the Zth indexing processing and the vector of the source object is greater than or equal to the minimum vector similarity between the nodes in the result set obtained after the Zth indexing processing and the vector of the source object, it indicates that the loop of the indexing processing has not reached the stop condition, and the next indexing processing can be continued to index new nodes and find the latest node with the top E vector similarity with the source object (i.e., the result set composed of the top E nodes).

[0161] For example, the (Z+1)th indexing node can be taken out from the candidate set obtained after the Zth indexing processing, and the node with the maximum vector similarity between the source object and the candidate set obtained after the Zth indexing processing can be taken out as the (Z+1)th indexing node. The (Z+1)th indexing processing can be performed in the last layer indexing structure through the (Z+1)th indexing node. The (Z+1)th indexing processing can be an updating processing performed on the candidate set obtained after the Zth indexing processing and taking out the (Z+1)th indexing node, and the result set obtained after the Zth indexing processing. Therefore, the query device can determine the target mixed object in the last layer indexing structure (i.e., the node in the last layer indexing structure representing the target mixed object) through the candidate set and the result set obtained after the (Z+1)th indexing processing, which is the same as the process of determining the target mixed object in the last layer indexing structure through the candidate set and the result set obtained after the Zth indexing processing. In this way, the indexing nodes are found one by one to update the candidate set and the result set, and when the loop stop condition is reached, the result set finally used to select the target mixed object is obtained.

[0162] Since the node is used to represent the mixed object, the result set and the candidate set containing the node can be understood as containing the mixed object represented by the node.

[0163] The following describes the process of performing the Z+1th indexing in the last-level index structure through the Z+1th index node. It can be known that the principle of performing any indexing in the last-level index structure through any index node is the same.

[0164] Each node connected with the Z+1th index node in the last-level index structure can be referred to as a neighbor node of the Z+1th index node. Therefore, the query device can sequentially calculate the vector similarity between the source object and each neighbor node of the Z+1th index node in descending order of the vector similarity between the Z+1th index node and each neighbor node.

[0165] Therefore, the query device can sequentially update the result set and the candidate set obtained after the Zth indexing by sequentially calculating the vector similarity between the source object and each neighbor node, that is, each time the vector similarity between a neighbor node and the source object is calculated, the result set and the candidate set at this time can be updated by the vector similarity, as described below. The neighbor node used to update the result set and the candidate set by calculating the vector similarity with the source object can belong to the indexed node in the last-level index structure.

[0166] Since the principle of updating the result set and the candidate set by the vector similarity between each calculated neighbor node and the source object is the same, the following describes the process of updating the current result set and the candidate set by the vector similarity between the neighbor node calculated in the S+1th time and the source object. S is a positive integer, and S is less than the number of neighbor nodes of the Z+1th index node, and S+1 can be equal to the number of neighbor nodes of the Z+1th index node.

[0167] It should be noted that the first vector similarity calculated in the first time can be directly updated to the result set and the candidate set obtained after the Zth indexing; the second vector similarity calculated in the second time can be updated to the result set and the candidate set obtained after the first indexing; the third vector similarity calculated in the third time can be updated to the result set and the candidate set obtained after the second indexing; and so on. The principle of updating the result set and the candidate set by the vector similarity calculated each time is the same.

[0168] Therefore, the query device can obtain the result set and the candidate set corresponding to the Sth calculation, which are obtained by iteratively updating the result set and the candidate set obtained after processing the Zth index in sequence, according to the S vector similarities calculated before the (S+1) th calculation. In other words, the result set and the candidate set corresponding to the Sth calculation are the result set and the candidate set updated by the Sth vector similarity.

[0169] If the number of nodes in the result set corresponding to the Sth calculation is less than E, the query device can add the neighbor node corresponding to the (S+1) th vector similarity calculated in the (S+1) th calculation (i.e., the vector similarity calculated in the (S+1) th calculation is the vector similarity between the neighbor node and the source object) to the result set and the candidate set corresponding to the Sth calculation, and obtain the result set and the candidate set corresponding to the (S+1) th calculation. That is, when the number of nodes in the result set is less than E, the indexed node can be directly added to the result set and the candidate set.

[0170] If the number of nodes in the result set corresponding to the Sth calculation is equal to E, and the vector similarity calculated in the (S+1) th calculation is greater than the minimum vector similarity between the nodes in the result set corresponding to the Sth calculation and the source object (i.e., the minimum vector similarity among the vector similarities between the E nodes in the result set and the source object), it indicates that a node with a greater vector similarity between the node and the source object than the minimum vector similarity is indexed. The neighbor node corresponding to the vector similarity calculated in the (S+1) th calculation can be added to the result set and the candidate set calculated in the Sth calculation, and the node corresponding to the minimum vector similarity can be removed from the result set, i.e., the result set and the candidate set corresponding to the (S+1) th calculation are obtained. The number of nodes in the result set corresponding to the (S+1) th calculation is still E, but a node with a greater vector similarity between the node and the source object than the minimum vector similarity is newly updated.

[0171] Further, if the number of nodes in the result set corresponding to the Sth calculation is equal to E, and the vector similarity calculated in the (S+1) th calculation is less than the minimum vector similarity between the nodes in the result set corresponding to the Sth calculation and the source object, it indicates that a node with a greater vector similarity between the node and the source object than the minimum vector similarity is not indexed. Therefore, the neighbor node corresponding to the vector similarity calculated in the (S+1) th calculation will not be added to the result set and the candidate set calculated in the Sth calculation. At this time, the result set and the candidate set corresponding to the Sth calculation can be directly used as the result set and the candidate set corresponding to the (S+1) th calculation.

[0172] And, if the number of nodes in the result set corresponding to the S+1th calculation is equal to E, and the vector similarity of the S+1th calculation is equal to the minimum vector similarity between the nodes in the result set corresponding to the Sth calculation and the source object, the neighbor node corresponding to the vector similarity of the S+1th calculation does not need to be added to the result set corresponding to the Sth calculation, and the result set corresponding to the Sth calculation can be directly used as the result set corresponding to the S+1th calculation. However, in this case, the neighbor node corresponding to the vector similarity of the S+1th calculation can be added to the candidate set corresponding to the Sth calculation to obtain the candidate set corresponding to the S+1th calculation.

[0173] The query device can iteratively update the result set and the candidate set corresponding to the Sth calculation according to the principles described above through the vector similarities calculated in sequence. After the result set and the candidate set are updated through the vector similarity of the last calculation, the candidate set and the result set after the Z+1th index processing can be obtained.

[0174] In the process of indexing and querying the nodes in the active index graph, the query device can also maintain a searched node set (which can also be referred to as a visited node set), which contains records of all indexed nodes. Subsequent indexing of the nodes in the searched node set is avoided, thereby avoiding repeated indexing and searching of the nodes.

[0175] Referring to FIG. 9, FIG. 9 is a schematic diagram of a scenario of layer-by-layer indexing of an index graph according to an embodiment of the present application. As shown in FIG. 9, the active index graph has G layers of index structures. The query device can start layer-by-layer indexing of the active index graph from a target entry node in the Gth layer of index structures of the active index graph. For example, the query device can index from the Gth layer of index structures of the active index graph to the G-1th layer of index structures of the active index graph, from the G-1th layer of index structures of the active index graph to the G-2th layer of index structures of the active index graph, and so on, until the last layer of index structures of the active index graph (i.e., the layer of index structures with the lowest number of layers).

[0176] Accordingly, the query device can specifically query and obtain the K target mixed objects having vector similarity with the source object from the last layer of index structures according to the principles described above.

[0177] Through the above process of the present application, the K target mixed objects having vector similarity with the source object are quickly and accurately queried from a large number of mixed objects through layer-by-layer indexing of the active index graph and multiple iterative updates of the result set and the candidate set in the last layer of index structures.

[0178] Please refer to FIG. 10, which is an effect diagram of an experimental result provided by an embodiment of the present application. In the actual verification process of the present application, the technical personnel compared the effect of the method provided by the present application with the effects of other methods. As shown in FIG. 10, the horizontal axis of the coordinate system can represent the recall rate of data query (i.e., the ratio of the correct data in the queried data to all the queried data), and the vertical axis can be used to represent the number of query processing per second, i.e., the number of query requests processed per second. Therefore, it can be understood that the values of the horizontal and vertical coordinates are both the higher the better.

[0179] As can be seen from FIG. 10, the curve obtained by using the method provided by the present application is at the uppermost position of the coordinate system, indicating that compared with other methods, the number of query requests processed per second using the method provided by the present application is the largest, and the recall rate is also the highest. The other methods used for comparative experiments of the method of the present application include the SeRF method (a query method based on privacy protection), the NHQ method (a composite query method), the ADBV method (a hybrid query method), the Milvus method (a vector retrieval engine method), the NGT method (a data query method in high-dimensional space), and the Vearch method (a deep learning vector-based retrieval method). Based on the above description, the method provided by the present application can achieve high-performance and excellent data query.

[0180] Please refer to FIG. 11, which is a structural diagram of a data query device provided by an embodiment of the present application. As shown in FIG. 11, the data query device 110 can include a first acquisition module 1101, a second acquisition module 1102, a determination module 1103, an activation module 1104, and a query module 1105.

[0181] The first acquisition module 1101 is configured to acquire query data, the query data including a scalar query range and a source object;

[0182] The second acquisition module 1102 is configured to acquire B buckets and an index graph; each bucket corresponds to a scalar division range, and each bucket contains at least one mixed object; each mixed object is composed of a vector and a scalar, and each mixed object is used to indicate a respective query object; the source object and the query object are objects of the same category; the scalar of any mixed object contained in any bucket is within the scalar division range corresponding to the bucket; the index graph is constructed based on the vector similarity between the mixed objects in the B buckets; the index graph contains nodes and edges; the nodes are used to represent the mixed objects in the B buckets; the edges are used to represent the vector similarity between the nodes; B is a positive integer;

[0183] The determining module 1103 is configured to determine a target sub-bucket to be queried from the B sub-buckets based on intersection between the scalar query range and scalar division ranges corresponding to the B sub-buckets respectively.

[0184] The activating module 1104 is configured to activate target nodes related to the mixed objects in the target sub-bucket and edges between the target nodes in the index graph, to obtain an activated index graph.

[0185] The querying module 1105 is configured to query target mixed objects having vector similarity with the source object from the target sub-bucket based on the activated index graph according to the scalar query range, and indicate the query object indicated by the target mixed object as the query object similar to the source object.

[0186] Optionally, the B scalar division ranges corresponding to the B sub-buckets are obtained by dividing a scalar global range, and the second obtaining module 1102 obtains the B sub-buckets in the following manner:

[0187] Obtain the number B of sub-buckets, and generate B-1 quantile numbers of the scalar global range based on the number B of sub-buckets;

[0188] Divide the scalar global range into B scalar division ranges based on the B-1 quantile numbers;

[0189] Based on the mixed objects respectively located in the B scalar division ranges, construct the B sub-buckets.

[0190] Optionally, the index graph comprises S-layer index structures, S is a positive integer, and edges in the index graph comprise edges constructed between a node of any mixed object in any sub-bucket and nodes of mixed objects in each sub-bucket.

[0191] The mixed object has an index structure of a target layer number in the index graph, the mixed object has a node in each index structure of a layer number less than and equal to the target layer number in the index graph, and the node of the mixed object has less than or equal to M edges between nodes of mixed objects in a sub-bucket in a layer index structure where the node is located, M is a positive integer.

[0192] The nodes in the i-1 layer index structure comprise the nodes in the i layer index structure, and the nodes for representing the same mixed object in the i-1 layer index structure and the i layer index structure are connected by edges, i is an integer greater than 1, and i is less than or equal to S.

[0193] Optionally, the activating module 1104 activates the target nodes related to the mixed objects in the target sub-bucket and the edges between the target nodes in the index graph in the following manner to obtain the activated index graph.

[0194] Obtain an activation control parameter.

[0195] calculate a ratio between the parameter value of the activation control parameter and the number of the target buckets, to obtain an activation connection number N, N being a positive integer;

[0196] based on the activation connection number N, activate target nodes and edges between the target nodes related to the mixed objects in the target buckets in the index graph, to obtain an activated index graph.

[0197] Optionally, any mixed object in the target buckets is a target mixed object; the activation module 1104 activates target nodes and edges between the target nodes related to the mixed objects in the target buckets in the index graph based on the activation connection number N, to obtain an activated index graph in the following manner:

[0198] the nodes of the mixed objects in the target buckets in the index graph are taken as the target nodes, and the target nodes are activated; and,

[0199] N edges between the node of the target mixed object and the nodes of the mixed objects in each target bucket in each index structure in which the node of the target mixed object is located are arranged in front as target edges; the node of the target mixed object has N target edges with the nodes of the mixed objects in one target bucket in one index structure in which the node of the target mixed object is located;

[0200] in each index structure in which the node of the target mixed object is located, N target edges between the node of the target mixed object and the nodes of the mixed objects in each target bucket are activated respectively;

[0201] wherein the edges between the node of the target mixed object and the nodes of the mixed objects in one target bucket in one index structure in which the node of the target mixed object is located are arranged in descending order of the vector similarity between the target mixed object and the mixed objects in the target bucket.

[0202] Optionally, the activated index graph contains G index structures, G being a positive integer and G being less than or equal to S;

[0203] the query module 1105 queries target mixed objects having vector similarity with the source object from the target buckets based on the activated index graph according to the scalar query range in the following manner:

[0204] based on the source object, index from the Gth index structure of the activated index graph to the last index structure of the activated index graph layer by layer in sequence;

[0205] according to the scalar query range, query the target mixed objects in the last index structure that is indexed to.

[0206] Optionally, the query module 1105 indexes from the Gth index structure of the activated index graph to the last index structure of the activated index graph layer by layer in sequence based on the source object in the following manner:

[0207] based on the source object, obtaining a cross-layer node in a jth layer index structure of the active index graph, j being an integer greater than 1 and less than or equal to G;

[0208] based on the cross-layer node in the jth layer index structure, indexing from the jth layer index structure to a (j-1)th layer index structure of the active index graph.

[0209] Optionally, the way in which the query module 1105 obtains the cross-layer node in the jth layer index structure of the active index graph based on the source object comprises:

[0210] if j is equal to G, obtaining a target entry node in a Gth layer index structure of the active index graph as the cross-layer node in the jth layer index structure; and,

[0211] if j is not equal to G, obtaining a node in the jth layer index structure having the highest vector similarity with the vector of the source object as the cross-layer node in the jth layer index structure.

[0212] Optionally, the way in which the query module 1105 queries the target hybrid object in the last layer index structure indexed to according to the scalar query range comprises:

[0213] obtaining a vector of the source object;

[0214] according to the scalar query range, querying the target hybrid object in the last layer index structure based on the target cross-layer node indexed to the last layer index structure, the vector of the source object and the vector of each node in the last layer index structure.

[0215] Optionally, the way in which the query module 1105 queries the target hybrid object in the last layer index structure according to the scalar query range based on the target cross-layer node indexed to the last layer index structure, the vector of the source object and the vector of each node in the last layer index structure comprises:

[0216] obtaining a result set and a candidate set obtained after Zth indexing processing in the last layer index structure based on a Zth index node; the result set contains at most E nodes having the scalar in the scalar query range and having the highest vector similarity with the vector of the source object in the order from high to low among the indexed nodes after the Zth indexing processing; the candidate set contains nodes added from the indexed nodes based on the result set; wherein, if Z is equal to 1, the Zth index node is a node of the hybrid object represented by the target cross-layer node in the last layer index structure, and if Z is greater than 1, the Zth index node is a node having the highest vector similarity with the vector of the source object taken from the candidate set obtained after the (Z-1)th indexing processing; Z and E are positive integers;

[0217] Determine the target mixed object in the last layer index structure based on the result set and the candidate set obtained after the Zth index processing.

[0218] Optionally, the query module 1105 determines the target mixed object in the last layer index structure based on the result set and the candidate set obtained after the Zth index processing in the following manner:

[0219] If the maximum vector similarity between the nodes in the candidate set obtained after the Zth index processing and the vector of the source object is less than the minimum vector similarity between the nodes in the result set obtained after the Zth index processing and the vector of the source object, the target mixed object is selected from the result set obtained after the Zth index processing.

[0220] If the maximum vector similarity between the nodes in the candidate set obtained after the Zth index processing and the vector of the source object is greater than or equal to the minimum vector similarity between the nodes in the result set obtained after the Zth index processing and the vector of the source object, the Z+1th index node is taken out from the candidate set obtained after the Zth index processing, and the Z+1th index processing is performed on the Z+1th index node in the last layer index structure.

[0221] Determine the target mixed object in the last layer index structure based on the result set and the candidate set obtained after the Z+1th index processing.

[0222] Optionally, the result set obtained after the Zth index processing contains E nodes, and the query data further includes a query number K, K is a positive integer, and K is less than E; the query module 1105 selects the target mixed object from the result set obtained after the Zth index processing in the following manner:

[0223] Sort the E nodes in the result set obtained after the Zth index processing in descending order of vector similarity between the nodes and the vector of the source object to obtain the sorted E nodes.

[0224] Determine the K mixed objects represented by the K nodes arranged in the front of the sorted E nodes as the target mixed objects.

[0225] Optionally, the query module 1105 performs the Z+1th index processing on the Z+1th index node in the last layer index structure in the following manner:

[0226] Calculate the vector similarity between the vector of the source object and each neighbor node of the Z+1th index node in descending order of vector similarity between the Z+1th index node and each neighbor node thereof;

[0227] The result set and the candidate set after the Zth index processing are iteratively updated based on the vector similarity between the vector of the source object and each neighbor node in sequence to obtain the result set and the candidate set after the Z+1th index processing.

[0228] The neighbor nodes of the Z+1th index node include nodes connected to the Z+1th index node by edges in the last layer index structure.

[0229] Optionally, the manner in which the query module 1105 iteratively updates the result set and the candidate set after the Zth index processing based on the vector similarity between the vector of the source object and each neighbor node in sequence includes:

[0230] The result set and the candidate set corresponding to the Sth calculation are obtained. The result set and the candidate set are iteratively updated based on the vector similarity calculated before the S+1th calculation on the result set and the candidate set after the Zth index processing. S is a positive integer, and S is less than the number of neighbor nodes of the Z+1th index node.

[0231] If the number of nodes in the result set corresponding to the Sth calculation is less than E, the neighbor node corresponding to the vector similarity of the S+1th calculation is added to the result set and the candidate set corresponding to the Sth calculation to obtain the result set and the candidate set corresponding to the S+1th calculation.

[0232] If the number of nodes in the result set corresponding to the Sth calculation is equal to E, and the vector similarity of the S+1th calculation is greater than the minimum vector similarity between the nodes in the result set corresponding to the Sth calculation and the vector of the source object, the neighbor node corresponding to the vector similarity of the S+1th calculation is added to the result set and the candidate set of the Sth calculation, and the node corresponding to the minimum vector similarity is removed to obtain the result set and the candidate set corresponding to the S+1th calculation.

[0233] If the number of nodes in the result set corresponding to the Sth calculation is equal to E, and the vector similarity of the S+1th calculation is less than the minimum vector similarity between the nodes in the result set corresponding to the Sth calculation and the vector of the source object, the result set and the candidate set corresponding to the Sth calculation are determined as the result set and the candidate set corresponding to the S+1th calculation.

[0234] If the number of nodes in the result set corresponding to the Sth calculation is equal to E, and the vector similarity of the (S+1)th calculation is equal to the minimum vector similarity between the nodes in the result set corresponding to the Sth calculation and the source object, the result set corresponding to the Sth calculation is determined as the result set corresponding to the (S+1)th calculation, and the neighbor node corresponding to the vector similarity of the (S+1)th calculation is added to the candidate set corresponding to the Sth calculation to obtain the candidate set corresponding to the (S+1)th calculation.

[0235] Optionally, the second obtaining module 1102 generates the index graph in the following procedure:

[0236] The target layer number of any mixed object is generated based on a probability decreasing manner, that is, the insertion probability of a node decreases layer by layer in a sequence from low to high according to the layer numbers of the index graph;

[0237] The nodes of any mixed object are inserted into the index structure of each insertion layer corresponding to the mixed object respectively, and the index structure of the insertion layer corresponding to any mixed object includes index structures with layer numbers less than and equal to the target layer number.

[0238] An edge between the node of any mixed object and the node of a mixed object in each bucket is established in each index structure in which the node of the mixed object is located.

[0239] Optionally, any bucket of the B buckets is a target bucket, and the index structure of any insertion layer of any mixed object is an index structure of a target layer.

[0240] The second obtaining module 1102 establishes an edge between the node of any mixed object and the node of a mixed object in each bucket in each index structure in which the node of the mixed object is located.

[0241] M nodes, where M is a positive integer, are obtained from the nodes of the mixed object in the target bucket in the index structure of the target layer, and the vector similarity between the node of any mixed object and the M nodes is arranged in descending order.

[0242] An edge between the node of any mixed object and the M nodes is established in the index structure of the target layer.

[0243] Optionally, the first obtaining module 1101 obtains the query data in the following manner:

[0244] The query request sent by the query client is obtained, and the query data is parsed from the query request.

[0245] The data query apparatus 110 further includes a sending module 1106 configured to:

[0246] The queried similar query objects to the source object are returned to the query client, so that the query client displays the returned query objects and the source object in association.

[0247] According to an embodiment of the present application, the steps involved in the data query method shown in FIG. 3 can be performed by various modules in the data query apparatus 110 shown in FIG. 11. For example, the step S101 shown in FIG. 3 can be performed by the first acquisition module 1101 in FIG. 11, the step S102 shown in FIG. 3 can be performed by the second acquisition module 1102 in FIG. 11; the step S103 shown in FIG. 3 can be performed by the determination module 1103 in FIG. 11, the step S104 shown in FIG. 3 can be performed by the activation module 1104 in FIG. 11, and the step S105 shown in FIG. 3 can be performed by the query module 1105 in FIG. 11.

[0248] The present application can quickly locate the range (such as the range of the target bucket) where the object to be queried is located through the bucketing strategy, without performing a large number of redundant queries on the objects that do not need to be queried, and can quickly index the target mixed object having the vector similarity with the source object through the index graph strategy, and can indicate the query object of the target mixed object as the finally queried similar query object to the source object. The present application combines the bucketing strategy and the index graph strategy, which can greatly improve the query efficiency of the target mixed object, thereby improving the query efficiency of the similar query object to the source object and reducing the query overhead of the similar query object to the source object.

[0249] According to an embodiment of the present application, the various modules in the data query apparatus 110 shown in FIG. 11 can be combined into one or several units respectively or all, or some of the units can be further split into a plurality of sub-units with smaller functions, which can achieve the same operation without affecting the implementation of the technical effects of the embodiments of the present application. The above modules are divided based on logical functions, and in actual application, the functions of one module can also be implemented by multiple units, or the functions of multiple modules can be implemented by one unit. In other embodiments of the present application, the data query apparatus 110 can also include other units, and in actual application, these functions can also be assisted by other units, and can be implemented by multiple units in cooperation.

[0250] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory) or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the functions of the module or unit.

[0251] According to an embodiment of the present application, a data query apparatus 110 as shown in FIG. 11 can be constructed on a general computer device (which can include a central processing unit (CPU), a random access memory (RAM), a read-only memory (ROM), and the like processing elements and storage elements) by running a computer program capable of performing each step involved in the corresponding method shown in each embodiment of the present application.

[0252] Referring to FIG. 12, FIG. 12 is a structural schematic diagram of a computer device according to an embodiment of the present application. As shown in FIG. 12, the computer device 1000 can include a processor 1001, a network interface 1004, and a memory 1005, and in some embodiments, the computer device 1000 can further include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can include a display, a keyboard, and optionally the user interface 1003 can further include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory or a non-volatile memory, for example, at least one disk memory. The memory 1005 can optionally be at least one storage device located away from the aforementioned processor 1001. As shown in FIG. 12, the memory 1005 as a computer storage medium can include an operating system, a network communication module, a user interface module, and a device control application program.

[0253] In the computer device 1000 shown in FIG. 12, the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application program stored in the memory 1005 to realize:

[0254] Obtaining query data, the query data comprising a scalar query range and a source object;

[0255] Obtaining B buckets and an index graph; each of the B buckets corresponds to a scalar partition range, each of the B buckets contains at least one mixed object, each of the mixed objects is composed of a vector and a scalar, each of the mixed objects is used to indicate a query object belonging to the mixed object, the source object and the query object are objects of the same category; the scalars of the mixed objects contained in any of the B buckets are within the scalar partition range corresponding to the bucket; the index graph is constructed based on vector similarities between the mixed objects in the B buckets; the index graph contains nodes and edges, the nodes are used to represent the mixed objects in the B buckets; the edges are used to represent the vector similarities between the nodes; B is a positive integer;

[0256] Determining a target bucket to be queried from the B buckets based on intersection between the scalar query range and the scalar partition ranges corresponding to the B buckets respectively;

[0257] Activating target nodes related to the mixed objects in the target bucket and edges between the target nodes in the index graph, to obtain an activated index graph;

[0258] Querying, according to the scalar query range, target mixed objects having vector similarities with the source object from the target bucket based on the activated index graph, and taking query objects indicated by the target mixed objects as query objects similar to the source object.

[0259] It should be understood that the computer device 1000 described in the embodiments of the present application can execute the description of the data query method in the embodiments of the present application, and can also execute the description of the data query device 110 in the embodiments corresponding to the foregoing FIG. 11, which will not be described here. In addition, the beneficial effects of using the same method will not be described here.

[0260] In addition, it should be pointed out here that the present application also provides a computer readable storage medium, and the computer readable storage medium stores a computer program, when the processor executes the computer program, the computer program can execute the description of the data query method in the embodiments of the present application, therefore, the description will not be repeated here. In addition, the beneficial effects of using the same method will not be described here. For technical details not disclosed in the computer storage medium embodiments of the present application, please refer to the description of the method embodiments of the present application.

[0261] As an example, the foregoing computer program can be deployed on a computer device for execution, or deployed on multiple computer devices located in one place for execution, or deployed on multiple computer devices distributed in multiple places and interconnected through a communication network for execution. The multiple computer devices distributed in multiple places and interconnected through a communication network can constitute a blockchain network.

[0262] The computer readable storage medium can be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. The computer readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the computer device. The computer readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0263] The present application provides a computer program product including a computer program stored in a computer readable storage medium. A processor of a computer device reads the computer program from the computer readable storage medium, and the processor executes the computer program to enable the computer device to perform the description of the data query method in the embodiments of the present application. Therefore, the description will not be repeated here. In addition, the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the embodiments of the computer readable storage medium, please refer to the description of the method embodiments of the present application.

[0264] The terms "first", "second", and the like in the description and claims of the present application and the accompanying drawings are used to distinguish different objects, and are not used to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product, or apparatus including a series of steps or units is not limited to the listed steps or modules, but can optionally include steps or modules not listed, or can optionally include other steps or units inherent to the process, method, device, product, or apparatus.

[0265] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software, or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0266] The above descriptions are only the preferred embodiment of the application, of course, cannot be used to limit the scope of the application, thus the equivalent variations made by the claims of the application, still belongs to the scope of the application covered.

Claims

1. A data query method, characterized by, The method comprises: acquiring query data, the query data comprising a scalar query range and a source object; acquiring B buckets and an index graph; each of the buckets corresponds to a scalar division range, each of the buckets contains at least one mixed object, each of the mixed objects is composed of a vector and a scalar, each of the mixed objects is used to indicate a respective query object, the source object and the query object are objects of the same category; the scalar of any mixed object contained in any bucket is within the scalar division range corresponding to the bucket; the index graph is constructed based on the vector similarity between the mixed objects in the B buckets; the index graph contains nodes and edges, the nodes are used to represent the mixed objects in the B buckets; the edges are used to represent the vector similarity between the nodes; B is a positive integer; determining a target bucket to be queried from the B buckets based on the intersection between the scalar query range and the scalar division ranges corresponding to the B buckets; activating target nodes related to the mixed objects in the target bucket and edges between the target nodes in the index graph to obtain an activated index graph; querying a target mixed object having vector similarity with the source object from the target bucket based on the activated index graph according to the scalar query range, and taking the query object indicated by the target mixed object as a queried query object similar to the source object.

2. The method of claim 1, wherein, The B scalar division ranges corresponding to the B buckets are obtained by dividing a global scalar range, and the process of obtaining the B buckets comprises: obtaining a bucket number B and generating B-1 quantile numbers of the global scalar range based on the bucket number B; uniformly dividing the global scalar range into the B scalar division ranges by using the B-1 quantile numbers; constructing the B buckets based on the mixed objects whose scalars are respectively within the B scalar division ranges.

3. The method of claim 1 or 2, wherein, The index graph comprises S-layer index structures, S is a positive integer, and the edges in the index graph comprise edges constructed between the node of any mixed object in any bucket and the nodes of the mixed objects in each bucket; wherein a mixed object has an index structure of a target layer in the index graph, the mixed object has a node in each index structure with a layer number less than and equal to the target layer number in the index graph, and the node of the mixed object has less than or equal to M edges between the nodes of the mixed objects in a bucket in the index structure in which the node is located, M is a positive integer; the nodes in an i-1 layer index structure comprise the nodes in an i layer index structure, the nodes used to represent the same mixed object in the i-1 layer index structure and the i layer index structure are connected by edges, i is an integer greater than 1, and i is less than or equal to S.

4. The method according to any one of claims 1 to 3, characterized in that, The process of activating the target nodes related to the mixed objects in the target bucket and the edges between the target nodes in the index graph to obtain an activated index graph comprises: obtaining an activation control parameter; Calculate a ratio between a parameter value of the activation control parameter and a number of the target buckets, to obtain an activation connection number N, N being a positive integer; Based on the activation connection number N, activate target nodes related to mixed objects in the target buckets and edges between the target nodes in the index graph, to obtain an activated index graph.

5. The method according to any one of claims 1 to 4, characterized in that, Any mixed object in the target buckets is a target mixed object; based on the activation connection number N, activate target nodes related to mixed objects in the target buckets and edges between the target nodes in the index graph, to obtain an activated index graph, including: Take nodes of the mixed objects in the target buckets in the index graph as the target nodes, and activate the target nodes; and, Arrange N edges in front of edges between nodes of the target mixed object and nodes of mixed objects in each of the target buckets in each layer index structure in which the nodes of the target mixed object are located, as target edges; the target mixed object has N target edges between the nodes of the target mixed object and nodes of mixed objects in one of the target buckets in a layer index structure in which the nodes of the target mixed object are located; In each layer index structure in which the nodes of the target mixed object are located, activate N target edges between the nodes of the target mixed object and nodes of mixed objects in each of the target buckets. The edges between the nodes of the target mixed object and nodes of mixed objects in one of the target buckets in a layer index structure in which the nodes of the target mixed object are located are arranged in descending order according to vector similarity between the target mixed object and the mixed objects in the target bucket.

6. The method according to any one of claims 1 to 5, wherein, The activated index graph contains G layer index structures, G being a positive integer and G being less than or equal to S; The target mixed object having vector similarity with the source object is queried from the target buckets based on the activated index graph according to the scalar query range, including: Based on the source object, index from a Gth layer index structure of the activated index graph to a last layer index structure of the activated index graph layer by layer in turn; Query the target mixed object in the last layer index structure in which the index is performed according to the scalar query range.

7. The method according to any one of claims 1 to 6, wherein The index from the Gth layer index structure of the activated index graph to the last layer index structure of the activated index graph layer by layer in turn based on the source object includes: Based on the source object, obtain a cross-layer node in a jth layer index structure of the activated index graph, j being an integer greater than 1 and j being less than or equal to G; Based on the cross-layer node in the jth layer index structure, index from the jth layer index structure to a (j-1)th layer index structure of the activated index graph.

8. The method according to any one of claims 1 to 7, wherein The obtaining of the cross-layer node in the jth layer index structure of the activated index graph based on the source object includes: If j is equal to G, obtain a target entry node of the activated index graph in the Gth layer index structure as the cross-layer node in the jth layer index structure; and If j is not equal to G, obtain a node having the highest vector similarity with a vector of the source object in the jth layer index structure as the cross-layer node in the jth layer index structure.

9. The method according to any one of claims 1 to 8, wherein, The querying the target mixed object in the last-layer index structure indexed to according to the scalar query range comprises: acquiring a vector of the source object; querying the target mixed object in the last-layer index structure indexed to according to the scalar query range, the vector of the source object and the vector of each node in the last-layer index structure.

10. The method of any one of claims 1-9, wherein, The querying the target mixed object in the last-layer index structure indexed to according to the scalar query range, the vector of the source object and the vector of each node in the last-layer index structure comprises: acquiring a result set and a candidate set obtained after the Zth index node performs Zth index processing in the last-layer index structure; the result set contains the maximum E nodes whose scalar is in the scalar query range and whose vector similarity with the vector of the source object is arranged in descending order in the nodes indexed after the Zth index processing; the candidate set contains the nodes added from the indexed nodes based on the result set; wherein, if Z is equal to 1, the Zth index node is the node of the mixed object represented by the target cross-layer node in the last-layer index structure, if Z is greater than 1, the Zth index node is the node whose vector similarity with the vector of the source object is the maximum in the candidate set obtained after the (Z-1)th index processing; Z and E are positive integers; determining the target mixed object in the last-layer index structure based on the result set and the candidate set obtained after the Zth index processing.

11. The method of any one of claims 1-10, wherein, The determining the target mixed object in the last-layer index structure based on the result set and the candidate set obtained after the Zth index processing comprises: if the maximum vector similarity between the node in the candidate set obtained after the Zth index processing and the vector of the source object is less than the minimum vector similarity between the node in the result set obtained after the Zth index processing and the vector of the source object, selecting the target mixed object from the result set obtained after the Zth index processing; if the maximum vector similarity between the node in the candidate set obtained after the Zth index processing and the vector of the source object is greater than or equal to the minimum vector similarity between the node in the result set obtained after the Zth index processing and the vector of the source object, taking out a (Z+1)th index node from the candidate set obtained after the Zth index processing and performing (Z+1)th index processing in the last-layer index structure based on the (Z+1)th index node; determining the target mixed object in the last-layer index structure based on the result set and the candidate set obtained after the (Z+1)th index processing.

12. The method of any one of claims 1-11, wherein, The result set obtained after the Zth index processing contains E nodes, and the query data further includes a query number K, K is a positive integer, and K is less than E; the target mixed object is selected from the result set obtained after the Zth index processing, including: The E nodes in the result set obtained after the Zth index processing are sorted according to the vector similarity between the vector of the source object and the vector of each neighbor node of the Z+1th index node from large to small, to obtain the sorted E nodes; The K mixed objects represented by the K nodes arranged in the front of the sorted E nodes are determined as the target mixed object.

13. The method of claim 11, wherein, The Z+1th index processing in the last layer index structure based on the Z+1th index node includes: The vector similarity between the vector of the source object and each neighbor node is calculated in turn according to the vector similarity between the Z+1th index node and each neighbor node from large to small; Based on the vector similarity between the vector of the source object and each neighbor node calculated in turn, the result set and the candidate set obtained after the Zth index processing are iteratively updated in turn to obtain the result set and the candidate set after the Z+1th index processing; Wherein, the neighbor nodes of the Z+1th index node include: the nodes connected with the Z+1th index node by edges in the last layer index structure.

14. The method of any one of claims 1-13, wherein, Based on the vector similarity between the vector of the source object and each neighbor node calculated in turn, the result set and the candidate set obtained after the Zth index processing are iteratively updated in turn, including: Obtain the result set and the candidate set corresponding to the Sth calculation; the result set and the candidate set are obtained by iteratively updating the result set and the candidate set obtained after the Zth index processing based on the vector similarity calculated before the S+1th time; S is a positive integer, and S is less than the number of neighbor nodes of the Z+1th index node; If the number of nodes in the result set corresponding to the Sth calculation is less than E, the neighbor node corresponding to the vector similarity of the S+1th calculation is added to the result set and the candidate set corresponding to the Sth calculation to obtain the result set and the candidate set corresponding to the S+1th calculation; If the number of nodes in the result set corresponding to the Sth calculation is equal to E, and the vector similarity of the S+1th calculation is greater than the minimum vector similarity between the nodes in the result set corresponding to the Sth calculation and the vector of the source object, the neighbor node corresponding to the vector similarity of the S+1th calculation is added to the result set and the candidate set of the Sth calculation, and the node corresponding to the minimum vector similarity is removed to obtain the result set and the candidate set corresponding to the S+1th calculation; If the number of nodes in the result set corresponding to the Sth calculation is equal to E, and the vector similarity of the S+1th calculation is less than the minimum vector similarity between the nodes in the result set corresponding to the Sth calculation and the vector of the source object, the result set and the candidate set corresponding to the Sth calculation are determined as the result set and the candidate set corresponding to the S+1th calculation; If the number of nodes in the result set corresponding to the Sth calculation is equal to E, and the vector similarity of the S+1th calculation is equal to the minimum vector similarity between the nodes in the result set corresponding to the Sth calculation and the vector of the source object, the result set corresponding to the Sth calculation is determined as the result set corresponding to the S+1th calculation, and the neighbor node corresponding to the vector similarity of the S+1th calculation is added to the candidate set corresponding to the Sth calculation to obtain the candidate set corresponding to the S+1th calculation.

15. The method of any one of claims 1-14, wherein, The process of generating the index graph comprises: generating the target layer number of any mixed object in a probability-decreasing manner; the probability-decreasing manner is a manner in which the insertion probability of a node decreases layer by layer in a low-to-high order of the layer numbers of the index graph; inserting the nodes of the any mixed object into the index structure of each insertion layer corresponding to the any mixed object respectively; the index structure of the insertion layer corresponding to the any mixed object includes index structures with layer numbers less than and equal to the target layer number; establishing an edge between the node of the any mixed object and the nodes of the mixed objects in each bucket in each index structure in which the node of the any mixed object is located.

16. The method of any one of claims 1-15, wherein, any insertion layer of the any mixed object is a target layer, and the index structure of the target layer is a target layer index structure; the step of establishing an edge between the node of the any mixed object and the nodes of the mixed objects in each bucket in each index structure in which the node of the any mixed object is located comprises: obtaining M nodes in which the vector similarity between the node of the any mixed object and the nodes of the mixed objects in the target bucket is arranged in a descending order in the nodes of the mixed objects in the target bucket in the index structure of the target layer, M being a positive integer; establishing an edge between the node of the any mixed object and the M nodes in the index structure of the target layer.

17. The method of any one of claims 1-16, wherein, the step of obtaining the query data comprises: obtaining a query request sent by a query client and parsing the query data from the query request; the method further comprises: returning the query objects similar to the source object to the query client, so that the query client displays the returned query objects and the source object in association.

18. A data query apparatus, characterized by the apparatus comprises: a first obtaining module configured to obtain query data, the query data comprising a scalar query range and a source object; The second acquisition module is configured to acquire B sub-buckets and an index graph; each of the sub-buckets corresponds to a scalar division range, each of the sub-buckets contains at least one mixed object, each of the mixed objects is composed of a vector and a scalar, each of the mixed objects is used to indicate a respective query object, the source object and the query object are objects of the same category; the scalar of any mixed object contained in a sub-bucket is within the scalar division range corresponding to the sub-bucket; the index graph is constructed based on vector similarities between mixed objects in the B sub-buckets; the index graph contains nodes and edges, the nodes are used to represent the mixed objects in the B sub-buckets, and the edges are used to represent the vector similarities between the nodes; B is a positive integer; The determination module is configured to determine a target sub-bucket to be queried from the B sub-buckets based on intersections between the scalar query range and the scalar division ranges corresponding to the B sub-buckets, respectively. The activation module is configured to activate target nodes related to the mixed objects in the target sub-bucket and edges between the target nodes in the index graph, to obtain an activated index graph. The query module is configured to query target mixed objects having vector similarities with the source object from the target sub-bucket based on the activated index graph according to the scalar query range, and take a query object indicated by the target mixed object as a query object similar to the source object.

19. A computer program product, characterised in that, A computer program is included, which, when executed by a processor, implements the steps of the method of any one of claims 1-17.

20. A computer device, comprising: A memory and a processor are included, the memory stores a computer program, and the computer program, when executed by the processor, causes the processor to perform the steps of the method of any one of claims 1-17.

Images