Visualization method, electronic device and medium for vector similarity search

Abstract

The present application relates to the field of visualization of unstructured data analysis and processing, and in particular to a method, electronic device and medium for visualization of vector similarity search. In the method for visualization of vector similarity search of the present application, a novel presentation method is provided for each stage of the indexing algorithm of the vector space, and a more concrete structure is objectively constructed for the vector space in a visualized manner. Through such a presentation method, the user can clearly understand what happens in the process of vector similarity search, and it is easier to understand the implementation process and mechanism of vector similarity search, satisfying the curiosity of the user and improving the user's human-computer interaction experience.

Description

Technical Field

[0001] The present application relates to the field of visualization of unstructured data analysis and processing, and in particular to a visualization method, electronic device and medium for vector similarity search. Background Art

[0002] In the era of big data, the demand for data analysis in production and operation activities is increasing. However, the data used as the analysis object comes from various channels and sources, and is not all standardized structured data. Therefore, the demand for unstructured data processing is the greatest. Therefore, people have developed a lot of computer processing algorithms for analyzing and processing unstructured data.

[0003] With the refinement of social division of labor, although many companies have a strong demand for unstructured data analysis algorithms, they do not invest in or research the algorithms themselves, but instead purchase more software as a service (SaaS) services. For service providers that provide analysis algorithms, when showing users the algorithms they have developed, it is often difficult to find a way to give users a good user experience and accurately describe how unstructured data processing technologies and algorithms work. In addition, for the data that is the object of analysis, these data are sometimes high-dimensional data (the dimension of description is greater than 3). For the processing of such high-dimensional data, it is even more difficult for users to understand unstructured data processing technologies and algorithms. Summary of the invention

[0004] Embodiments of the present application provide a method, device, and medium for visualizing vector similarity search.

[0005] In a first aspect, the present application provides a visualization method for vector similarity search, which is used in an electronic device. The method includes:

[0006] Determine a query vector and a target library for searching, and run a vector similarity search, wherein the target library includes multiple vectors, and the target library is presented as an N-layer hierarchical structure, each layer of the hierarchical structure includes at least one group, and each group includes at least one vector; each group in the N layers includes a representative vector; wherein N is an integer greater than or equal to 2;

[0007] Determine at least one group in a first layer of the N layers, and determine at least one first group within a preset range of the distance query vector based on representative vectors of the respective groups, and display at least a portion of the process of determining the first group in a first visualization manner;

[0008] In a second visualization manner, starting from each first group in at least one first group, presenting a process of entering the Mth layer layer by layer from the 1st layer, where M=2, 3, ..., N-1, N;

[0009] For the Mth layer, the candidate vector is determined by layer-by-layer search, where

[0010] In the case where M=2, 3, ..., N-1, in the Mth layer, among all the groups belonging to the group determined in the previous layer, a second group closest to the query vector is determined based on the representative vector of each group, and at least part of the process of determining the second group in each layer is displayed in a first visualization manner, and a process of entering the M+1th layer from the Mth layer is presented in a second visualization manner; and

[0011] In the case where M=N, a preset number of candidate vectors are determined from all vectors in the Mth layer belonging to the group determined in the previous layer based on the distance between each vector and the query vector; and at least part of the process of determining the candidate vectors is displayed in a third visualization manner, and

[0012] In a fourth visualization method, the candidate vectors are displayed.

[0013] It is understandable that common vector search algorithms only output the final search results to the user, and the specific search process is unknown to the user. However, through the visualization method of the present application, the vector search process can be presented to the user in a way that is easy for the user to understand, such as the specific visualization form shown in FIG. 7 below.

[0014] In a possible implementation of the first aspect, multiple groups are determined in a first layer of N layers, and based on representative vectors of each group, a first group closest to a query vector is determined, including:

[0015] Identify all groups in the first layer;

[0016] Determining a distance to a query vector based on a representative vector of each of all the groups; and

[0017] At least one first group of distance query vectors satisfying a preset condition is determined according to the distance.

[0018] In a possible implementation of the first aspect, in the case where M=2, 3, ..., N-1, among all the groups included in the Mth layer, based on the representative vectors of the respective groups, a second group closest to the query vector is determined, including

[0019] Determine all groups in the Mth layer;

[0020] Determining a distance to a query vector based on a representative vector of each of all the groups; and

[0021] According to the distance, at least one group whose distance query vector meets a preset condition is determined as a second group.

[0022] In a possible implementation of the first aspect, in the case of M=N, a preset number of candidate vectors are determined from all vectors included in the Mth layer based on the distance between each vector and the query vector, including

[0023] Determine all vectors in the Mth layer;

[0024] Based on the distances of all vectors to the query vector; and

[0025] According to the distance and preset conditions, a preset number of vectors are determined as candidate vectors.

[0026] In another possible implementation of the first aspect, a plurality of groups are determined in a first layer of N layers, and based on representative vectors of the respective groups, at least one first group whose distance query vector is within a preset range is determined, including:

[0027] Determine a starting vector in the first layer;

[0028] Starting from the starting vector, searching for the query vector step by step via at least one path, each path passing through at least a representative vector of a group in the first layer;

[0029] Determine at least one candidate path from at least one path, and determine, among the clusters passed by the candidate path, that the cluster closest to the query vector has a distance from the query vector within a preset range; wherein the distance between the cluster and the query vector is determined based on the distance between a representative vector of the cluster and the query vector;

[0030] For each candidate path of the at least one candidate path, a group closest to the query vector is determined as a first group, so as to obtain a corresponding first group corresponding to each candidate path.

[0031] In another possible implementation of the first aspect, searching the query vector step by step via at least one path includes:

[0032] Determine, according to the starting vector, at least one first representative vector associated with the starting vector, wherein the representative vector corresponds to the group;

[0033] Starting from at least one first representative vector, searching for a next representative vector to determine at least one path, wherein

[0034] When the distances of at least one subsequent representative vector associated with the first representative vector are greater than the distances between the first representative vector and the query vector, terminating the search for the query vector along the path and taking the first representative vector as the end point of the path;

[0035] When the distance of at least one representative vector associated with the first representative vector is less than the distance between the first representative vector and the query vector, the Lth representative vector closest to the query vector is used as the starting point of the next search, and the search for the L+1th representative vector is continued. When the distance of at least one representative vector associated with the searched L+1th representative vector is greater than the distance between the Lth representative vector and the query vector, the search for the query vector along the path is terminated, and the Lth vector is used as the end point of the path, where L is an integer greater than or equal to 2.

[0036] Through the above method, combined with the unstructured data involved in this application, the interpretability of the search path in the vector search process can be effectively improved. Specifically, the visualization method of this application presents the access records of the vectors involved in the vector search process to the user in combination with a specific visualization form (such as pictures, text, etc.), so that the user can understand how the query vector is gradually searched using the starting vector in the vector search process.

[0037] In another possible implementation of the first aspect, in the case where M=2, 3, ..., N-1, among all the groups included in the Mth layer, based on the representative vectors of the respective groups, a second group closest to the query vector is determined, including

[0038] Determine the starting vector of the Mth layer;

[0039] Starting from the starting vector, the query vector is searched step by step via at least one path, each path passing through at least a representative vector of a group in the Mth layer;

[0040] From the groups passed by the at least one path, a group having the smallest distance to the query vector is determined as the second group, wherein the distance is the distance between the representative vector of the group and the query vector.

[0041] In another possible implementation of the first aspect, in the case where M=N, a preset number of candidate vectors are determined from all vectors included in the Mth layer based on the distance between each vector and the query vector, including

[0042] Determine all vectors in the Mth layer;

[0043] Starting from the starting vector determined in the corresponding group of the Mth layer, the query vector is searched step by step via at least one path, each path passing through at least one vector;

[0044] From the groups passed by the at least one path, a group having the smallest distance to the query vector is determined as the second group, wherein the distance is the distance between the representative vector of the group and the query vector.

[0045] Determining the distance to the query vector based on the representative vector of each group in all vectors; and

[0046] According to the distance, a preset number of vectors are determined as candidate vectors in order of distance from small to large.

[0047] In a possible implementation of the first aspect, each group in the Nth layer and at least one group in the N-1th layer form a membership relationship in a non-overlapping manner.

[0048] In a possible implementation of the first aspect, the first visualization method includes a visualization method for displaying a connection from one vector to another vector.

[0049] In a possible implementation of the first aspect, for the first visualization manner, visualization presentation manners of connections between different vectors are different.

[0050] In a possible implementation of the first aspect above, the second visualization method is a visualization method for displaying a jump from one layer to another layer.

[0051] In a possible implementation of the first aspect, for the second visualization mode, different layers are adjusted in different visualization presentation modes.

[0052] In a possible implementation of the first aspect above, the vector similarity search is based on an inverted index algorithm.

[0053] In a possible implementation of the first aspect above, the vector similarity search is based on the HNSW algorithm.

[0054] In a possible implementation of the first aspect above, the preset number is set by receiving user input via a preset interface.

[0055] In a possible implementation of the first aspect, the starting vector is set by receiving user input via a preset interface.

[0056] In a possible implementation of the first aspect, at least one of the first visualization mode, the second visualization mode, and the third visualization mode is set by receiving user input via a preset interface.

[0057] In a second aspect, the present application provides a visualization device for vector similarity search, the device comprising:

[0058] An initialization module, wherein the initialization module determines a query vector and a target library for searching, and runs a vector similarity search, wherein the target library includes a plurality of vectors, and the target library is presented as an N-layer hierarchical structure, each layer of the hierarchical structure includes at least one group, and each group includes at least one vector; each group in the N layers includes a representative vector; wherein N is an integer greater than or equal to 2;

[0059] A first visualization processing module, wherein the first determination module determines at least one group in a first layer of the N layers, and determines at least one first group within a preset range of the distance query vector based on representative vectors of the respective groups, and displays at least part of the process of determining the first group in a first visualization manner;

[0060] A second visualization processing module, starting from each first group in at least one first group, presents a process of entering the Mth layer layer by layer from the 1st layer in a second visualization manner, where M=2, 3, ..., N-1, N;

[0061] The third visualization processing module determines the candidate vectors for the Mth layer by searching layer by layer, where

[0062] In the case where M=2, 3, ..., N-1, in the Mth layer, among all the groups belonging to the group determined in the previous layer, a second group closest to the query vector is determined based on the representative vector of each group, and at least part of the process of determining the second group in each layer is displayed in a first visualization manner, and a process of entering the M+1th layer from the Mth layer is presented in a second visualization manner; and

[0063] In the case where M=N, a preset number of candidate vectors are determined from all vectors in the Mth layer belonging to the group determined in the previous layer based on the distance between each vector and the query vector; and at least part of the process of determining the candidate vectors is displayed in a third visualization manner, and

[0064] The fourth visualization processing module displays the candidate vector in a fourth visualization manner.

[0065] In a third aspect, the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory,

[0066] The processor executes the computer program to implement the steps of the method of the first aspect of the present application.

[0067] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of the first aspect of the present application.

[0068] In a fifth aspect, the present application provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, the steps of the method of the first aspect of the present application are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0070] Figure 1 A schematic diagram showing an application scenario according to an embodiment of the present application.

[0071] Figure 2a An exemplary process of a visualization method for vector similarity search implemented by an electronic device according to the present application is shown.

[0072] Figure 2b An exemplary process of a visualization method for vector similarity search implemented by an electronic device according to the present application is shown.

[0073] Figure 3 A block diagram of an electronic device in an embodiment of the present application is shown.

[0074] Figure 4 A process depicting a visualization of the inverted index algorithm operating in a vector space is shown.

[0075] Figure 5a A phase diagram showing a visual representation of the inverted index algorithm.

[0076] Figure 5b A phase diagram showing a visual representation of the inverted index algorithm.

[0077] Figure 5c A phase diagram showing a visual representation of the inverted index algorithm.

[0078] Figure 6 The process of visualizing the HSNW algorithm running in vector space is shown.

[0079] Figure 7a A phase diagram showing a visual representation of the HSNW algorithm is shown.

[0080] Figure 7b A phase diagram showing a visual representation of the HSNW algorithm is shown.

[0081] Figure 7c A phase diagram showing a visual representation of the HSNW algorithm is shown.

[0082] Figure 7d A phase diagram showing a visual representation of the HSNW algorithm is shown.

[0083] Figure 7e A phase diagram showing a visualization of the HSNW algorithm is shown.

[0084] Figure 7f A phase diagram showing a visualization of the HSNW algorithm is shown.

[0085] Figure 7g A phase diagram showing a visualization of the HSNW algorithm is shown.

[0086] Figure 8 A flowchart of a step-by-step search process according to an embodiment of the present application is shown.

[0087] Fig. 9 A block diagram of a visualization device for vector similarity search is shown. DETAILED DESCRIPTION

[0088] The following describes embodiments of the present application with reference to the accompanying drawings.

[0089] Figure 1 A schematic diagram showing an application scenario according to an embodiment of the present application is shown. Figure 1 The application scenario shown includes a display terminal 100 for display, and a cloud device 200 as a data processing subject. The cloud device 200 runs an unstructured data analysis algorithm for unstructured data as a sample. The unstructured data as a sample can be data provided by the user, or sample data prepared by the algorithm developer himself. The process and results of the unstructured data analysis algorithm (hereinafter referred to as "analysis algorithm") running on the cloud device 200 are transmitted to the display terminal 100 for display to the user. The display terminal 100 can be a display screen of a desktop computer, a large-screen TV, a projection device, a mobile phone terminal, or a wearable device with a screen.

[0090] In another implementation, the analysis algorithm may not be executed in the cloud device 200, but may be executed on a terminal device 300 connected to the terminal 100 via a wired or wireless method.

[0091] The following is a brief introduction to vector processing in unstructured data analysis and processing. In unstructured data analysis and processing, vector embedding is an important method for unstructured data analysis and processing. Through machine learning, unstructured data (such as pictures, videos, text, DNA structure, etc.) is encoded into high-dimensional vectors. For example, a picture of a flower is encoded into a 512-dimensional floating-point vector. High-dimensional vectors have unified specifications and formats, which are conducive to computer processing.

[0092] There are many different applications for unstructured data analysis and processing, such as analyzing whether two data are identical or similar, classifying data according to certain rules, and recommending other data or data classification based on one data. In these applications, similarity search of data processed into vectors is the basis. As an example of an application, in the process of searching for images (objects) by images in many shopping applications, the image as the basis for the query is processed into vector a, and various images in the application database are also processed into vectors. Several vectors close to vector a are found in the database, such as vectors b, c, and d, and then restored to images based on vectors b, c, and d, and presented to the user as search results. Among them, the distance between vectors is used to identify the similarity between vectors. The distance can include Euclidean distance, cosine distance (or cosine similarity), etc. For Euclidean distance, the smaller the distance, the more similar or even the same the two vectors are, and the opposite is true for cosine distance.

[0093] The following describes a visualization method for vector similarity search provided by this application. Figure 2a An exemplary process of a visualization method for vector similarity search implemented by an electronic device according to the present application is shown.

[0094] In step S201, a query vector and a target library for searching are determined, and a vector similarity search is run. The target library includes multiple vectors, and the target library is presented as a hierarchical structure of N layers, where N is an integer greater than or equal to 2. Each layer of the hierarchical structure includes at least one group, each group includes at least one vector, and each group in the Nth layer and at least one group in the N-1th layer form a membership relationship in a non-overlapping manner. Forming a membership relationship in a non-overlapping manner here means that in the groups of the next layer, each group will only be attributed to a certain group in the previous layer, and there will be no situation where a group in the next layer is attributed to 2 or more groups in the previous layer. Each group in the N layers includes a representative vector. The representative vector here can be the center vector of the group. The center vector refers to the vector with the smallest distance fluctuation from the other vectors in the group. The vector with the smallest distance fluctuation refers to the vector whose distance difference with other vectors in the group meets certain conditions, such as the vector with the smallest average value of the distance difference with other vectors in the group.

[0095] However, the representative vector of a group may be any specified vector, not just the center vector.

[0096] In step S202, at least one group is determined in the first layer of the N layers, and based on the representative vectors of each group, at least one first group whose distance to the query vector is within a preset range is determined. The preset range of the distance can be predetermined in order to screen out multiple first groups.

[0097] In step S203, at least part of the process of determining the first group is displayed in a first visualization mode; the first visualization mode is a visualization mode for displaying the connection from one vector to another vector. For example, the representative vector of one group is connected to the representative vector of another group, so that the group search process can be presented in a visualization mode.

[0098] In step S204, a process of entering a next layer from a first layer is presented for each group in the at least one first group in a second visualization manner.

[0099] In step S205, for the Mth layer which is not the lowest layer, the candidate vector is determined by searching layer by layer, that is, for the case where M=2, 3, ..., N-1, in the Mth layer, among all the groups belonging to the group determined in the previous layer, based on the representative vector of each group, the second group closest to the query vector is determined, and in the switching process of each layer, for example, from the M-1th layer to the Mth layer, at least part of the process of determining the second group in each layer is displayed in a first visualization manner. In the second visualization manner, the process of entering the M+1th layer from the Mth layer is presented. In the second visualization manner, M can be an integer greater than 2, as long as it is less than or equal to N.

[0100] In step S206, for the bottom layer, when M=N, a preset number of candidate vectors are determined from all vectors in the Mth layer belonging to the group determined in the previous layer based on the distance between each vector and the query vector.

[0101] In step S207, at least part of the process of determining the candidate vector is displayed in a first visual manner, and

[0102] In step S208 , the candidate vectors are displayed in a third visualization manner to present the search result to the user in an emphasized manner.

[0103] According to the implementation of the present application, there may be a situation where there are only two layers of calculation. Figure 2b Another exemplary process of the visualization method of vector similarity search implemented by the electronic device according to the present application is shown. Figure 2b In the process shown in FIG. 1 , steps S211 to S214 are Figure 2a In the example shown, steps S201 to S204 are the same. The difference is that in step S205, the process enters the last layer, that is, the bottom layer. Specifically, in step S215, for the bottom layer, in the case of M=N, a preset number of candidate vectors are determined from all vectors in the Mth layer belonging to the group determined in the previous layer based on the distance between each vector and the query vector. Steps S216 and S217 are Figure 2aThe steps of the illustrated embodiment are consistent, showing at least part of the process of determining candidate vectors in a first visual manner, and showing the candidate vectors in a third visual manner, so as to present the search results to the user in an emphasized manner.

[0104] According to the embodiments of the present application, a novel presentation method is provided for the indexing algorithm of the vector space, and a more concrete structure is objectively constructed for the vector space in a visual manner. This novel presentation method provides a new perspective for users to understand the vector space and the vector search algorithm and calculation process running in the vector space, helping users to better understand the process of vector search.

[0105] Figure 3 A block diagram of an electronic device 300 in an embodiment of the present application is shown. In one embodiment, the electronic device 300 may include one or more processors 304, a system control logic 308 connected to at least one of the processors 304, a system memory 312 connected to the system control logic 308, a non-volatile memory (NVM) 316 connected to the system control logic 308, and a network interface 320 connected to the system control logic 308.

[0106] In some embodiments, the processor 304 may include one or more single-core or multi-core processors. In some embodiments, the processor 304 may include any combination of a general-purpose processor and a dedicated processor (e.g., a graphics processor, an application processor, a baseband processor, etc.). In an embodiment where the electronic device 300 uses an enhanced base station (Evolved Node B, eNB) 101 or a radio access network (Radio Access Network, RAN) controller 102, the processor 304 may be configured to execute various embodiments that comply with the present invention.

[0107] In some embodiments, system control logic 308 may include any suitable interface controller to provide any suitable interface to at least one of processors 304 and / or any suitable device or component in communication with system control logic 308 .

[0108] In some embodiments, the system control logic 308 may include one or more memory controllers to provide an interface to the system memory 312. The system memory 312 may be used to load and store data and / or instructions. In some embodiments, the memory 312 of the electronic device 300 may include any suitable volatile memory, such as a suitable dynamic random access memory (DRAM).

[0109] NVM / memory 316 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, NVM / memory 316 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as a hard disk drive (HDD), a compact disc (CD) drive, and a digital versatile disc (DVD) drive.

[0110] NVM / storage 316 may include a portion of storage resources on a device on which electronic device 300 is installed, or it may be accessible to the device but not necessarily a portion of the device. For example, NVM / storage 316 may be accessed over a network via network interface 320.

[0111] In particular, system memory 312 and NVM / storage 316 may include, respectively, a temporary copy and a permanent copy of instructions 324. Instructions 324 may include instructions that, when executed by at least one of processors 304, cause electronic device 300 to implement the above-described construction method. In some embodiments, instructions 324, hardware, firmware, and / or software components thereof may be additionally / alternatively placed in system control logic 308, network interface 320, and / or processor 304.

[0112] The network interface 320 may include a transceiver for providing a radio interface for the electronic device 300, thereby communicating with any other suitable device (such as a front-end module, an antenna, etc.) through one or more networks. In some embodiments, the network interface 320 may be integrated with other components of the electronic device 300. For example, the network interface 320 may be integrated with at least one of the processor 304, the system memory 312, the NVM / storage 316, and a firmware device (not shown) having instructions, and when at least one of the processors 304 executes the instructions, the electronic device 300 implements the above-mentioned construction method.

[0113] The network interface 320 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 320 may be a network adapter, a wireless network adapter, a telephone modem and / or a wireless modem.

[0114] In one embodiment, at least one of the processors 304 may be packaged together with logic for one or more controllers of the system control logic 308 to form a system in package (SiP). In one embodiment, at least one of the processors 304 may be integrated on the same die with logic for one or more controllers of the system control logic 308 to form a system on a chip (SoC).

[0115] The electronic device 300 may further include an input / output (I / O) device 332. The I / O device 332 may include a user interface to enable a user to interact with the electronic device 300; the design of the peripheral component interface enables the peripheral components to interact with the electronic device 300. In some embodiments, the electronic device 300 further includes a sensor for determining at least one of an environmental condition and location information related to the electronic device 300.

[0116] In some embodiments, the user interface may include, but is not limited to, a display (e.g., an LCD display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., an LED flash), and a keyboard.

[0117] In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.

[0118] The following is combined with Figure 4 Embodiments according to the present application are described. Figure 4 It is a process of visualizing the inverted index algorithm (hereinafter referred to as IVF algorithm) running in the vector space. Figure 4 In the example shown, based on the query vector, the 10 vectors with the strongest association with it in the vector space are searched. The distance between the vectors means the strength of the association, and the shorter the distance, the stronger the association. The number 10 is an example of a predetermined number.

[0119] Step S401, determine the query vector as the search target and the target library for the search query. The form of the query vector is determined by the vector space in which it is located. As an example of the query vector and the target library, for example, in the implementation of the "search by image" function of the online shopping platform, the query vector corresponds to the image of the item of interest entered by the user. The collection of vectors corresponding to a large number of images stored on the online shopping platform is the target library.

[0120] As a collection of vectors of the target library, it is organized into a hierarchical structure with multiple layers. For the IVF algorithm, the target library is usually divided into two layers.

[0121] The first layer (top or upper layer) includes multiple groups. Each group includes at least one vector, and the vectors contained in each group do not overlap. In other words, once a vector is attributed to a group, it will no longer be attributed to another group. Each group in the first layer has a representative vector. This representative vector can be the center vector of the group, or it can be a vector determined based on a preset rule to represent the group.

[0122] In the second layer (bottom layer or lower layer), multiple vectors are included. Among the multiple vectors in this layer, once a vector is attributed to one group, it will no longer belong to another group, which forms the so-called non-overlapping membership.

[0123] In order to understand the process of such a hierarchical structure more intuitively, we can use the figurative example of map addressing for an analogical explanation. For example, given a longitude and latitude coordinate, pre-query the city closest to the longitude and latitude coordinate in the map. The given longitude and latitude are the query vectors, the set of cities in the map is the target library, and the longitude and latitude of each city are the corresponding vectors in the target library. The hierarchical structure of each city in the map can be expressed as follows: the first layer organizes each "city" (vector) in units of "province" (group), and the second layer is the set of each city. The capital city of the province can be regarded as the representative vector of the group.

[0124] Step S402, determine all groups on the first layer, and determine representative vectors of all groups. Each group on the first layer has a representative vector. This representative vector can be the center vector of the group, or it can be a vector determined based on a preset rule to represent the group. Using the example of a map as an analogy, that is, determine the longitude and latitude of the capitals of all provinces on the map.

[0125] Step S403, based on the representative vectors of all groups, determine the distances between these representative vectors and the query vector. The distance here can be, for example, Euclidean distance, cosine distance, Hamming distance, etc. By analogy, confirm the distances between each provincial capital in the map and the given longitude and latitude coordinates.

[0126] Step S404, based on the distance between the representative vector of each group and the query vector, determine the selected group of the first layer (as an example of the first group). The selected group of the first layer can be one or more. If one selected group is determined, the group with the shortest distance is taken as the selected group. If multiple selected groups are determined, all distances are sorted, and a predetermined number of multiple groups are selected in order from small to large as the selected groups.

[0127] Step S405: presenting the determination of the group and the selected group in a visual manner. The visual presentation may be a whole or part of the process of carrying out the calculation results of the above steps.

[0128] For the determination of each group on the first layer, a metaphor-based space division can be used for visualization (as an example of the first visualization method). For example, different two-dimensional shapes are used to represent each group in the two-dimensional space, and a group containing more vectors is represented by a shape occupying a larger area. FIG5 shows a schematic diagram of the visualization process of the inverted index algorithm. Figure 5aAn example of using different two-dimensional shapes to represent groups is shown. In this embodiment, for ease of explanation, rectangles are used to represent each group. However, in other embodiments, other polygons can be used to represent groups. The figure includes multiple groups represented by rectangular shapes. The figure includes multiple two-dimensional shapes. Different polygons correspond to different vector distributions, such as the distance between each vector and the representative vector. The area enclosed by shape 501 is larger than that enclosed by shape 502, which means that the group corresponding to shape 501 contains more vectors than the group corresponding to shape 502. In some embodiments, the query vector can be displayed in the second layer at the same time, such as Figure 5a The vector in 5000.

[0129] In some embodiments, a group may be represented by a three-dimensional shape in a three-dimensional space, and a group containing more vectors may be represented by a three-dimensional shape with a larger volume. Furthermore, based on the shape, color may be used to represent the number of vectors contained in the group.

[0130] In some embodiments, the determination of each group on the first layer can also be visualized using dimensionality reduction projection. Dimensionality reduction helps convert data into low dimensions, especially two dimensions that are easier for users to understand and identify, so as to easily identify some important features and their attributes and avoid dimensionality problems when analyzing large data sets. The dimensionality reduction projection methods that can be used include UMAP, TSNE, MDS, PCA, etc.

[0131] In some embodiments, the determination of each group on the first layer may also be visualized using a node-link diagram, which may be a visualization type such as a tree diagram or a mechanics diagram.

[0132] For presentation of the selected group, at least part of the process of determining the candidate vector may be presented by connecting the query vector and the candidate vector determined as the query result (an example of the third visualization method). Figure 5c As shown, the process of determining the query result can be presented by connecting the query vector and the result vector. In addition to the static connection method, the process of connecting the query vector and the candidate vector can be presented by a dynamic connection method. The connection between the query vector and the candidate vector can be in the form of a linear arrow or in the form of an arrow of a certain two-dimensional shape.

[0133] As explained above, the first visualization method includes the presentation of groups and also includes the presentation of connections between vectors. In some embodiments, the presentation methods for connections between different vectors may also be different. For example, the presentation method for connections between vectors whose distances meet a threshold may be different from the presentation method for connections between vectors whose distances do not meet the threshold. In addition, when presenting groups and vectors, some specific numerical values ​​may be displayed.

[0134] According to the embodiments of the present application, a novel presentation method is provided for each stage of the indexing algorithm of the vector space, and a more concrete structure is objectively constructed for the vector space in a visual manner. Through such a presentation method, the user can clearly understand what happens during the operation and more easily understand the implementation process and mechanism of the operation.

[0135] Step S406, in a visual manner, corresponding to the selected group determined in step S404, presents the process from the first layer (top layer) to the second layer (bottom layer). The following is an example of a case where a selected group is determined in step S404. The second layer includes multiple vectors, and these vectors are all vectors belonging to the selected group. The vectors included in other unselected groups in the first layer will not be displayed in the second layer.

[0136] In some embodiments, the switching display from the top layer to the bottom layer can be displayed using a zooming method (as an example of a second visualization method). Figure 5a The group 501 shown in FIG. 5 is determined as the selected group. In step S406, the shape 501 is magnified and switched to the perspective of the two-dimensional plane of the second layer, and all vectors included in the group corresponding to the shape 501 in the second layer are displayed, as shown in FIG. Figure 5b shown. Figure 5b In the example, vectors such as vector 5011 and vector 5012 are displayed. In some embodiments, the query vector can also be displayed in the second layer, such as Figure 5b The vector in 5000.

[0137] In other embodiments, the process from the top layer to the bottom layer may be presented by switching perspectives or other animation effects.

[0138] According to the embodiments of the present application, a novel presentation method is provided for each stage of the indexing algorithm of the vector space, and a more concrete structure is objectively constructed for the vector space in a visual manner. Through such a presentation method, the user can clearly understand what happens during the operation and more easily understand the implementation process and mechanism of the operation.

[0139] Step S407, search into the bottom layer and determine all vectors included in the bottom layer. In other words, all vectors in the selected group determined in step S404 will be included in the subsequent calculation.

[0140] Step S408: Determine the distance between all underlying vectors and the query vector. The distance here is the same as described above, for example, the Euclidean distance between vectors.

[0141] Step S409: sort the vectors according to the distances between all the vectors and the query vector, and select the first 10 vectors with the shortest distances as the query result vectors.

[0142] Step S410, displaying the process of determining the result of the query vector in step S409 in a visual manner. Figure 5c As shown in , vectors can be represented by dots or other shapes. Figure 5c Vector 5011 shown. At least part of the process of determining the candidate vector can be presented by connecting the query vector and the candidate vector determined as the query result (an example of the third visualization method). Figure 5c As shown, the process of determining the query result can be presented by connecting the query vector and the result vector. In addition to the static connection method, the dynamic connection method can be used to present the connection between the query vector and the candidate vector, that is, the process from the query vector to the candidate vector. The connection between the query vector and the candidate vector can be in the form of a linear arrow, or in the form of an arrow of a certain two-dimensional shape.

[0143] Step S411, display the determined query result in a visual manner. In some embodiments, the vector determined as the query result may be displayed in an emphasized manner (as an example of a fourth visualization manner). Figure 5c As shown in FIG. 1 , a vector determined as a query result is enclosed by, for example, a circle. Other shapes may also be used as a means of emphasis, such as a square, a triangle, a polygon, etc. In addition, as a presentation method alone or in combination with the above-mentioned methods, the query result may be presented by changing the color, saturation, brightness, etc. of the dot or shape representing the vector.

[0144] It can be understood that the inverted index algorithm has the characteristics of high search efficiency and low introduction cost compared with other vector search algorithms, such as direct search algorithms such as the TOP-K algorithm, and can effectively improve the operating efficiency of the above-mentioned cloud device 200 or terminal device 300. The use of the visualization method of the present application to present the specific search process of the inverted index algorithm to the user is also conducive to the user's understanding of the difference in efficiency of vector search algorithms, and for R&D and testing users, it helps them choose a more suitable algorithm for related scientific research calculations.

[0145] In addition, it can be understood that, as mentioned above, the visualized vector search process helps users understand the "search" (or "index") process. For example, for the inverted index algorithm mentioned above, its essence is "clustering" (or "classification", "bucketing"), that is, grouping objects whose similarity meets certain conditions. For example, the inverted index algorithm divides the vector space into 128 buckets, some buckets are the set of "cars", some buckets are the set of "buildings", and some buckets are the set of "animals".

[0146] Based on the understanding of the essence of "indexing", users can adjust the parameters involved in the specific indexing algorithm through a visual interface. It is understandable that as the indexing algorithm parameters change, the visualization results presented to the user will also change accordingly, such as the number of clusters, the number of vector search results, the vector search path, the number of vectors accessed by the vector search, etc. Users can understand the meaning of the corresponding indexing parameters through the changes in the visualization results.

[0147] The following is combined with Figure 6 Embodiments according to the present application are described. Figure 6 It is a process of visualizing the HNSW algorithm (Hierarchical Navigable Small World) running in a vector space. Figure 4 The same embodiment shown in Figure 6 In the example shown, based on the query vector, the 10 vectors with the strongest association with it in the vector space are searched. The distance between the vectors means the strength of the association, and the shorter the distance, the stronger the association. The number 10 is an example of a predetermined number.

[0148] Step S601, determine the query vector as the search target and the target library for the search query. The form of the query vector is determined by the vector space in which it is located. As an example of the query vector and the target library, for example, in the implementation of the "search by image" function of the online shopping platform, the query vector corresponds to the image of the item of interest entered by the user. The collection of vectors corresponding to a large number of images stored on the online shopping platform is the target library.

[0149] As a collection of vectors of the target library, it is organized into a hierarchical structure with multiple layers. For the HNSW algorithm, the target library is usually divided into more than 3 layers.

[0150] The first layer (top layer or upper layer) includes multiple groups. Each group includes at least one vector, and the vectors contained in each group do not overlap. In other words, once a vector is attributed to a group, it will no longer be attributed to another group. Each group in the first layer has a representative vector. The following is explained using the center vector of the group as an example, but the present application is not limited to this, and the representative vector can also be determined based on other criteria.

[0151] The second layer also includes multiple groups. The groups included in the second layer are subgroups of the groups in the first layer, but the vectors included in each group do not overlap. A so-called non-overlapping affiliation is formed. In other words, once a group is a subgroup of a group in the upper layer, it will not become a subgroup of another group in the upper layer, nor will it have vector overlaps with other subgroups. However, in some other embodiments, in HNSW or some similar algorithms, it is not necessary to configure the groups and subgroups and even the vectors of each layer in the above-mentioned non-overlapping manner.

[0152] It can be understood that in this embodiment, the second layer is explained as an example of the middle layer. In the case where the hierarchical structure of the target library includes more than 4 layers, the middle layer includes more than one layer, which means that the groups corresponding to the first layer are continuously divided into multiple subgroups in each middle layer. For example, the group G1 in the top layer is presented as multiple subgroups G1-1, G1-2, and G1-3 in the second layer as the middle layer; in the third layer as the middle layer, the group G1-1 is divided into G1-1-1 and G1-1-2; the group G1-2 is divided into G1-2-1, G1-2-2, and G1-2-3; the group G1-3 is divided into G1-3-1 and G1-3-2. It can be understood that in each layer of the middle layer, the elements included are all in groups.

[0153] Back to the present embodiment, the third layer (bottom layer) includes multiple vectors. In the multiple vectors of this layer, once a vector is attributed to a group, it will no longer belong to another group, which forms a so-called non-overlapping affiliation.

[0154] In order to understand the process of such a hierarchical structure more intuitively, we will also use the figurative example of map addressing for an analogical explanation. For example, given a longitude and latitude coordinate, pre-query the city in the territory that is closest to the longitude and latitude coordinate. The given longitude and latitude are the query vector, the set of cities in the map is the target library, and the longitude and latitude of each city are the corresponding vectors in the target library. The hierarchical structure of each city in the map can be expressed as follows: the top layer organizes each "city" (subgroup) as a unit of "province" (group), and the middle layer is a collection of cities. The bottom layer is a collection of county towns, and the county town is an example of a vector. A representative county can be selected in the province as the representative vector of the group. A representative county can also be selected in each city as the representative vector of the subgroup.

[0155] Step S602, randomly determine a starting vector on the first layer. The starting vector belongs to a group in the first layer and can be any vector in the group. FIG7 shows a stage diagram of the HSNW algorithm visualization, which also shows a schematic diagram of the hierarchical search process. Figure 7a The following table shows the situation of each group in the first layer. Figure 7a As shown, multiple groups, such as group 71 to group 78, and their representative vectors 701 to 708 are included. In addition, the query vector 7000 and the initial vector 701s are also shown in the figure. For the sake of simplicity, the initial vector 701s is a vector of group 71. The present application is not limited to this, and it can also be any other vector in group 71, or it can be the representative vector of the group. Taking the example of a map as an analogy, the starting vector can be any county on the map.

[0156] Step S603, starting from the starting vector, determine a path connecting the representative vectors of each group to search the query vector step by step. The process of constructing a path for step-by-step search is explained below.

[0157] Figure 8 A flowchart of a step-by-step search process according to an embodiment of the present application is shown.

[0158] In step S801, at least one representative vector associated with the starting vector is first determined. Figure 7b As shown in , the representative vectors associated with the starting vector 701s include the representative vector 701 of the group 71 to which it belongs. In addition, in some cases, the representative vector 702 of the group 72 adjacent to the group 71 may also be included. In other embodiments, more representative vectors may be used as the starting point of the path. In this application, only two vectors are used as an exemplary illustration. When there are multiple vectors associated with the starting vector, the path can be extended by selecting a vector with a smaller distance (e.g., less than a certain threshold) based on the distance between the associated vector and the query vector.

[0159] In step S802, the next vector is searched from the vector associated with the initial vector to determine the search path. Figure 7b As shown, the next vector is searched starting from vectors 701 and 702 respectively.

[0160] For vector 701, it searches backwards for multiple representative vectors associated with it, and finds that among these multiple representative vectors, representative vector 706 is closest to the query vector, so vector 706 is used as the next starting point of the path. Similarly, representative vector 706 searches for vector 707 among the vectors associated with it, which is closest to the query vector, and then uses 707 as the next starting point of the path. In the search starting from vector 707, the distance between all vectors in the vectors associated with vector 707 and the query vector is greater than the distance between vector 707 and the query vector, and the search path terminates and continues to extend downward. Thus, the path 701-706-707 starting from the initial vector 701s and ending at vector 707 is determined.

[0161] Based on the same principle, for the search starting from 702, vector 703, vector 704 and vector 705 are determined in sequence from the associated vector of each node. Thus, the path 702-703-704-705 starting from the initial vector 701s and ending at vector 705 is determined.

[0162] Return to Figure 6 Explain the process of determining the target group in the first tier.

[0163] Step S604: Determine a candidate path from the determined paths. Determine, among the clusters that the candidate path passes through, the cluster closest to the query vector whose distance to the query vector is within a preset range. Figure 7b For the example shown, the distance between the query vector and the vector 707 closest to the query vector in the paths 701-706-707 is determined, and the distance between the query vector and the vector 705 closest to the query vector in the paths 702-703-704-705 is determined. In the case where only one path needs to be determined, only one vector with the closest distance, such as vector 705, needs to be determined. In the case where multiple paths need to be determined, only the vectors whose distances meet a certain threshold need to be determined.

[0164] Step S605: From the determined subsequent paths, select the group corresponding to the vector closest to the query vector as the top-level selected group. Figure 7b In the illustrated case, group 75 corresponding to vector 705 is selected as the top level group.

[0165] Step S606: present the determination of the groups in the top layer and the selected groups in a visual manner. The visual presentation may be a whole or part of the process of carrying out the calculation results of the above steps.

[0166] For the determination of each group on the first level, a metaphor-based space division can be used for visualization (as an example of the first visualization method). For example, different two-dimensional shapes are used to represent each group in the two-dimensional space, and a group containing more vectors is represented by a shape occupying a larger area, such as Figure 7a as shown in .

[0167] In some embodiments, a group may be represented by a three-dimensional shape in a three-dimensional space, and a group containing more vectors may be represented by a three-dimensional shape with a larger volume. Furthermore, based on the shape, color may be used to represent the number of vectors contained in the group.

[0168] In some embodiments, the determination of each group on the first layer can also be visualized using dimensionality reduction projection. Dimensionality reduction helps convert data into low dimensions, especially two dimensions that are easier for users to understand and identify, so as to easily identify some important features and their attributes and avoid dimensionality problems when analyzing large data sets. The dimensionality reduction projection methods that can be used include UMAP, TSNE, MDS, PCA, etc.

[0169] In some embodiments, the determination of each group on the first layer may also be visualized using a node-link diagram, which may be a visualization type such as a tree diagram or a mechanics diagram.

[0170] In some embodiments, for the connection between the starting vector and the representative vectors of each group in the search process, at least part of the process of determining the candidate vector can be presented by connecting the query vector and the candidate vector determined as the query result. Figure 7b As shown, the process of determining the query result can be presented by connecting the query vector and each representative vector. In addition to the static connection method, the process of connecting the query vector and the candidate vector can be presented by a dynamic connection method. The connection between the query vector and the candidate vector can be in the form of a linear arrow or in the form of an arrow of a certain two-dimensional shape.

[0171] Step S607, in a visual manner, corresponding to the selected group determined in step S605, presents the process from the first layer (top layer) to the second layer (middle layer). The second layer includes multiple groups, and these groups are vectors belonging to the selected group of the previous layer. Subgroups included in other unselected groups in the previous layer will not be displayed in the second layer.

[0172] In some embodiments, the switching display from the top layer to the middle layer can be displayed using a zooming method (as an example of the second visualization method). In some embodiments, the switching display from the top layer to the bottom layer can be displayed using a zooming method (as an example of the second visualization method). Figure 7c In step S607, the group 75 is magnified and switched to the perspective of the two-dimensional plane of the middle layer, and all subgroups included in the group corresponding to the group 75 in the second layer are displayed, as shown in FIG. Figure 7c shown. Figure 7c In the example, subgroups such as group 75a, group 75b, group 75c, group 75d, and group 75e are displayed. In some embodiments, the query vector can also be displayed in the second layer, such as Figure 7c The vector in 7000.

[0173] According to the embodiments of the present application, a novel presentation method is provided for each stage of the indexing algorithm of the vector space, and a more concrete structure is objectively constructed for the vector space in a visual manner. Through such a presentation method, the user can clearly understand what happens during the operation and more easily understand the implementation process and mechanism of the operation.

[0174] In some embodiments, when the intermediate layer includes multiple layers, different presentation modes may be used to present the switching of different layers. For example, when three intermediate layers are included, the presentation mode of switching from the first intermediate layer to the second intermediate layer may be the same as the presentation mode of switching from the second intermediate layer to the third intermediate layer.

[0175] Step S608: In the middle layer, determine the initial vector of the layer. The following description will take the representative vector in the layer group as an example of the initial vector. Figure 7d As shown, as an example, the representative vector of the middle layer is still vector 705. In the middle layer, vector 705 belongs to subgroup 75c. The initial vector may also be the representative vector of the subgroup, or may be a vector arbitrarily determined among the vectors included in the group. In other embodiments, the starting vector of the middle layer may also be other vectors, such as vectors in groups other than subgroup 75c.

[0176] Step S609, starting from the representative vector, determine at least one path to gradually search the query vector. The step-by-step search process implemented in the middle layer is substantially the same as that performed in the top layer. As shown in FIG7d, starting from the determined vector 705, determine at least one representative vector associated with the vector, and extend from the at least one representative vector to the representative vector of the next node associated with the representative vector to form at least one path. In the case where the distance between the representative vector of the previous node of the path and the representative vector of at least one next node associated with the representative vector is greater than the distance between the representative vector and the query vector, terminate the search for the query vector via the path, and use the representative vector of the previous node as the end point of the path.

[0177] like Figure 7d As shown, starting from vector 705, two paths, 705-7051-7052 path and 705-7053 path, are determined.

[0178] Step S610, determine a candidate path from the determined paths, and select a group corresponding to a vector closest to the query vector from the determined subsequent paths as a selected group of the second layer. Among the groups passed by the candidate path, determine that the group closest to the query vector is within a preset range of distance from the query vector. Figure 7c For the example shown, the distance between the query vector and the vector 7052 closest to the query vector in the path 705-7051-7052 is determined, and the distance between the query vector and the vector 7053 closest to the query vector in the path 705-7053 is determined. In the case where only one path needs to be determined, only one vector with the closest distance needs to be determined, such as vector 7052. In the case where multiple paths need to be determined, vectors whose distances meet a certain threshold can be determined.

[0179] In the candidate paths, the group closest to the query vector is selected as the selected group of the layer. For the 705-7051-7052 path, the group 75e corresponding to the vector 7052 is selected as the selected group of the layer.

[0180] In the above embodiment, only the case where the middle layer is one layer is described. In the case where the middle layer includes multiple layers, it is only necessary to repeat the process from step 607 to step 610 between each layer. Although step 607 describes the switching from the first layer to the second layer, in the case where the middle layer includes multiple layers, the switching between layers is substantially the same as step 607. In other words, it is the switching from the display mode of the group to the sub-groups included therein.

[0181] Step S611 presents the process from the middle layer to the bottom layer in a visual manner corresponding to the selected group determined in step S610.

[0182] In some embodiments, the switching display from the previous layer to the next layer can be displayed by zooming (as an example of the second visualization method). Figure 7d The group 75e shown in FIG. 1 is determined as the selected group. In step S611, the group 75e is magnified and switched to the perspective of the bottom two-dimensional plane, and all vectors included in the group 75e determined in the previous layer are displayed, as shown in FIG. Figure 7e shown. Figure 7e In some embodiments, the bottom layer may also display query vectors, such as vectors 7501-7506. Figure 7d The vector in 7000.

[0183] Step S612, starting from the initial vector determined in the bottom layer, gradually search for the vector closest to the query vector. The initial vector can be an arbitrarily determined vector in the group of this layer, or a representative vector in the group of this layer can be selected as in the above process. In this embodiment, the representative vector 7052 of the group of the bottom layer is still used as the initial vector in this layer. Starting from the initial vector, determine at least one path to gradually search for the query vector. The step-by-step search process implemented in the bottom layer is essentially the same as the process carried out in the top and middle layers. Although the target of the search in the top and middle layers is the group, it is still essentially a search performed in units of the representative vector of the group. Figure 7f As shown, starting from the determined vector 7052, at least one representative vector associated with the vector is determined, including vector 7502 and vector 7504, and extending from the at least one representative vector to the representative vector of the next node associated with the representative vector to form at least one path. In the case where the distance between the representative vector of the previous node of the path and the representative vector of at least one next node is greater than the distance between the representative vector and the query vector, the search for the query vector via the path is terminated, and the representative vector of the previous node is used as the end point of the path. Finally, two paths are determined starting from vector 7502, including the 7052-7501-7502-7503 path and the 7052-7501-7504-7505-7506 path.

[0184] Step S613, determine a candidate path from the determined paths, and select a vector closest to the query vector from the determined subsequent paths as a candidate vector. Among the clusters passed by the candidate path, determine that the cluster closest to the query vector is within a preset range of distance from the query vector. Figure 7fFor the example shown, the distance between the query vector and the vector 7503 closest to the query vector in the path 7501-7502-7503 is determined, and the distance between the query vector and the vector 7506 closest to the query vector in the path 7501-7504-7505-7506 is determined. In the case where only one path needs to be determined, only one vector with the closest distance needs to be determined, such as vector 7503. In the case where multiple paths need to be determined, the vectors whose distances meet a certain threshold can be determined, for example, two candidate vectors 7503 and 7506 are obtained.

[0185] Step S614, displaying the process of determining the result of the query vector in step S613 in a visual manner. Figure 7f As shown in , vectors can be represented by dots or other shapes. Figure 7f Vector 7503 is shown. At least part of the process of determining the candidate vector can be presented in a manner of connecting the query vector and the candidate vector determined as the query result (an example of the third visualization method). Figure 7f As shown, the process of determining the query result can be presented by connecting the query vector and the result vector. In addition to the static connection method, the process of connecting the query vector and the candidate vector can be presented by a dynamic connection method. The connection between the query vector and the candidate vector can be in the form of a linear arrow or in the form of an arrow of a certain two-dimensional shape.

[0186] Step S615: Display the determined query result in a visual manner. In some embodiments, the vector determined as the query result may be displayed in an emphasized manner (as an example of a fourth visualization manner). Figure 7f As shown in FIG. 1 , a vector determined as a query result is enclosed by, for example, a circle. Other shapes may also be used as a means of emphasis, such as a square, a triangle, a polygon, etc. In addition, as a presentation method alone or in combination with the above-mentioned methods, the query result may be presented by changing the color, saturation, brightness, etc. of the dot or shape representing the vector.

[0187] In the method described above, a process of searching for subsequent vectors for a query vector layer by layer is presented. Figure 7g The figure shows an exemplary whole process of such a downward search layer by layer. Starting from the first layer, based on the representative vectors of each group in the first layer, the group closest to the query vector 7000 is determined. In the middle layer, the subgroups included in the group determined in the previous layer are determined according to each representative vector. Until the bottom layer, the vector closest to the query vector 7000 is searched in the subgroup determined last time.

[0188] According to the embodiments of the present application, a novel presentation method is provided for each stage of the index algorithm of the vector space, and a more concrete structure is objectively constructed for the vector space in a visual way. Through such a presentation method, it helps users to successfully track the calculation process such as vector access records in the index search process, and fully displays the entire operation process to users, making the search process perceptible, improving the interpretability of the search path, increasing intuitive visibility, making the implementation process and mechanism of the operation easier to understand, optimizing the efficiency of users' understanding of index search, effectively enhancing the user's experience, increasing interest, and improving user interactivity.

[0189] It should be noted that, for the search process of HNSW explained in the above example, in order to reduce the redundancy of the calculation in the intermediate process, especially when the intermediate layer has multiple layers, only one path can be selected in the search of each intermediate layer for simple search, and multiple candidate paths can be determined only in the last layer, so as to obtain more accurate results by using such a relatively complex search. In some cases, the first layer can also select only one path for simple search. In such a case, the above steps S604 and 610 can be combined with step 613.

[0190] In the visualization method provided according to an embodiment of the present application, the user can be provided with the possibility of changing specific key parameters in the calculation process by providing an interface to the user. As an example, an interface can be provided for the user to modify the number of candidate vectors finally obtained. The user can intervene in the number of candidate vectors that can be obtained in the end by adjusting the preset conditions. The user can also affect the number of candidate vectors that can be obtained in the end by directly inputting numbers.

[0191] In such Figure 6 In the illustrated implementation and similar embodiments, the initial vector may be determined randomly or input by a user through a preset interface.

[0192] According to the embodiment of the present application, a preset interface is also provided for the user to input the determination of various visualization methods and modes. The user can choose from the preset visualization methods, or add the visualization method required by the user through self-programming.

[0193] For certain specific algorithms, the hierarchical structure of the target vector library and the affiliation between each group in the Nth layer and at least one group in the N-1th layer in a non-overlapping manner can also be set by receiving user input via a preset interface.

[0194] Through customized input methods and intuitive visual presentation, users can experience the significant differences in visual models and changes in actual search paths through different parameters and settings, further understand the implementation process of the algorithm and the construction of vector space, and further provide users with the possibility of exploring vector space.

[0195] According to an embodiment of the present application, a visualization device for vector similarity search is also provided. Fig. 9 As shown, the visualization device 9000 includes:

[0196] Initialization module 9001, the initialization module determines a query vector and a target library for searching, and runs a vector similarity search, wherein the target library includes multiple vectors, and the target library is presented as an N-layer hierarchical structure, each layer of the hierarchical structure includes at least one group, each group includes at least one vector, each group in the N-th layer and at least one group in the N-1-th layer form a membership relationship in a non-overlapping manner; each group in the N-th layer includes a representative vector; wherein N is an integer greater than or equal to 2;

[0197] A first visualization processing module 9002, a first determination module determines at least one group in a first layer of the N layers, and determines at least one first group whose distance to the query vector is within a preset range based on representative vectors of each group, and displays at least part of the process of determining the first group in a first visualization manner;

[0198] The second visualization processing module 9003 presents, in a second visualization manner, a process of entering the Mth layer layer by layer from the 1st layer starting from each first group of at least one first group, where M=2, 3, ..., N-1, N;

[0199] The third visualization processing module 9004 determines, for the Mth layer, candidate vectors in a layer-by-layer search manner, wherein, in the case where M=2, 3, ..., N-1, in the Mth layer, among all the groups belonging to the group determined in the previous layer, based on the representative vectors of each group, a second group closest to the query vector is determined, and at least part of the process of determining the second group in each layer is displayed in a first visualization manner, and a process of entering the M+1th layer from the Mth layer is presented in a second visualization manner; and, in the case where M=N, a preset number of candidate vectors are determined from all the vectors included in the group determined in the previous layer in the Mth layer based on the distance between each vector and the query vector; and at least part of the process of determining the candidate vectors is displayed in a third visualization manner, and

[0200] The fourth visualization processing module 9005 displays the candidate vector in a fourth visualization manner.

[0201] In some embodiments, the first visualization processing module 9002 implements the following processing, including: determining all groups in the first layer; determining the distance from the query vector based on the representative vector of each group in all the groups; and determining, based on the distance, at least one first group whose distance from the query vector meets a preset condition.

[0202] In the third visualization processing module 9004, the following processing is implemented: in the case of M=2, 3, ..., N-1, among all the groups included in the Mth layer, based on the representative vectors of each group, the second group closest to the query vector is determined, including: determining all the groups in the Mth layer; determining the distance to the query vector based on the representative vector of each group among all the groups; and, based on the distance, determining at least one group whose distance to the query vector meets a preset condition as the second group.

[0203] In the case of M=N, a preset number of candidate vectors are determined from all vectors included in the Mth layer based on the distance between each vector and the query vector, including determining all vectors in the Mth layer; based on the distance between all vectors and the query vector; and, according to the distance and preset conditions, determining a preset number of vectors as candidate vectors.

[0204] According to another embodiment, in the first visualization processing module 9002, the following processing is implemented, including: determining a starting vector in the first layer; starting from the starting vector, searching for a query vector step by step via at least one path, each path passing through at least a representative vector of a group in the first layer;

[0205] The query vector is searched step by step via at least one path, including

[0206] According to the starting vector, at least one first representative vector associated with the starting vector is determined, wherein the representative vector corresponds to the group; starting from the at least one first representative vector, a next representative vector is searched to determine at least one path, wherein, when the distance of at least one subsequent representative vector associated with the first representative vector is greater than the distance between the first representative vector and the query vector, the search for the query vector via the path is terminated, and the first representative vector is used as the end point of the path;

[0207] When the distance of at least one representative vector associated with the first representative vector is less than the distance between the first representative vector and the query vector, the Lth representative vector closest to the query vector is used as the starting point of the next search, and the search for the L+1th representative vector is continued. When the distance of at least one representative vector associated with the searched L+1th representative vector is greater than the distance between the Lth representative vector and the query vector, the search for the query vector along the path is terminated, and the Lth vector is used as the end point of the path, where L is an integer greater than or equal to 2.

[0208] Determine at least one candidate path from at least one path, and determine, among the clusters passed by the candidate path, that the cluster closest to the query vector has a distance from the query vector within a preset range; wherein the distance between the cluster and the query vector is determined based on the distance between a representative vector of the cluster and the query vector;

[0209] For each candidate path of the at least one candidate path, a group closest to the query vector is determined as a first group, so as to obtain a corresponding first group corresponding to each candidate path.

[0210] In the third visualization processing module 9004, the following processing is implemented: in the case of M=2, 3, ..., N-1, among all the groups included in the Mth layer, based on the representative vectors of each group, determine the second group closest to the query vector, including determining the starting vector of the Mth layer; starting from the starting vector, step by step searching for the query vector via at least one path, each path passing through the representative vector of at least one group in the Mth layer; from the groups passed by at least one path, determine the group with the smallest distance to the query vector as the second group, wherein the distance is the distance between the representative vector of the group and the query vector.

[0211] In the case of M=N, a preset number of candidate vectors are determined from all vectors included in the Mth layer based on the distance between each vector and the query vector, including: determining all vectors in the Mth layer; starting from the starting vector determined in the corresponding group of the Mth layer, searching the query vector step by step via at least one path, each path passing through at least one vector; from the groups passed by at least one path, determining the group with the smallest distance to the query vector as the second group, wherein the distance is the distance between the representative vector of the group and the query vector. Based on the representative vector of each group among all vectors, the distance to the query vector is determined; and according to the distance, a preset number of vectors are determined as candidate vectors in the order of distance from small to large.

[0212] The various embodiments disclosed in the present application may be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application may be implemented as a computer program or program code executed on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0213] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0214] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0215] In some cases, the disclosed embodiments can be implemented in hardware, firmware, software or any combination thereof. The disclosed embodiments can also be implemented as instructions carried or stored thereon by one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which can be read and executed by one or more processors. For example, instructions can be distributed over a network or by other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic cards or optical cards, flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signal digital signals, etc.) using the Internet with propagation signals in electrical, optical, acoustic or other forms. Therefore, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a machine (e.g., computer) readable form.

[0216] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0217] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0218] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0219] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present application.

Claims

1. A visualization method for vector similarity search, used in electronic equipment, characterized in that: The method comprises: Determine a query vector and a target library for searching, and run a vector similarity search, wherein the target library includes a plurality of vectors, and the target library is presented as an N-layer hierarchical structure, each layer of the hierarchical structure includes at least one group, and each group includes at least one vector; each group in the N layers includes a representative vector; wherein N is an integer greater than or equal to 2; Determine at least one group in a first layer of the N layers, and determine at least one first group within a preset range from the query vector based on representative vectors of each group, and display at least part of the process of determining the first group in a first visualization manner, wherein the first visualization manner includes a visualization manner for displaying connections from one vector to another vector; In a second visualization mode, starting from each of the at least one first group, presenting a process of entering the Mth layer layer by layer from the 1st layer, where M=2, 3, ..., N-1, N; wherein the second visualization mode is a visualization mode for displaying a jump from one layer to another layer; For the Mth layer, candidate vectors are determined by searching layer by layer, where: In the case where M=2, 3, ..., N-1, in the Mth layer, among all the groups belonging to the group determined in the previous layer, a second group closest to the query vector is determined based on the representative vector of each group, and at least part of the process of determining the second group in each layer is displayed in a first visualization manner, and a process of entering the M+1th layer from the Mth layer is presented in a second visualization manner; and In the case where M=N, a preset number of candidate vectors are determined from all vectors in the Mth layer belonging to the group determined in the previous layer based on the distance between each vector and the query vector; and at least part of the process of determining the candidate vectors is displayed in a third visualization manner, and Displaying the candidate vector in a fourth visualization manner; The third visualization method includes a visualization method for displaying a connection from the query vector to the candidate vector, and the fourth visualization method includes a visualization method for emphasizing the candidate vector.

2. The method according to claim 1, characterized in that Determining a plurality of groups in a first layer of the N layers, and determining a first group closest to the query vector based on representative vectors of the respective groups, comprises: determining all groups in the first layer; Determining a distance from the query vector based on a representative vector of each of the groups; and At least one first group whose distance from the query vector satisfies a preset condition is determined according to the distance.

3. The method according to claim 2, characterized in that In the case where M=2, 3, ..., N-1, among all the groups included in the Mth layer, based on the representative vectors of the respective groups, determining the second group closest to the query vector comprises: Determine all groups in the Mth layer; Determining a distance from the query vector based on a representative vector of each of the groups; and According to the distance, at least one group whose distance from the query vector meets a preset condition is determined as the second group.

4. The method according to claim 2, characterized in that In the case where M=N, a preset number of candidate vectors are determined from all vectors included in the Mth layer based on the distance between each vector and the query vector, including: Determine all vectors in the Mth layer; Based on the distances of all the vectors from the query vector; and According to the distance and a preset condition, a preset number of vectors are determined as the candidate vectors.

5. The method according to claim 1, characterized in that Determining a plurality of groups in a first layer of the N layers, and determining at least one first group within a preset range from the query vector based on representative vectors of the respective groups, comprises: Determining a starting vector in the first layer; Starting from the starting vector, searching the query vector step by step via at least one path, each of the paths passing through at least a representative vector of a group in the first layer; Determine at least one candidate path from the at least one path, and determine, among the clusters passed by the candidate path, that the cluster closest to the query vector has a distance from the query vector within a preset range; wherein the distance between the cluster and the query vector is determined according to the distance between a representative vector of the cluster and the query vector; For each candidate path of the at least one candidate path, a group closest to the query vector is determined as the first group, so as to obtain a corresponding first group corresponding to each candidate path.

6. The method according to claim 5, characterized in that Stepwise searching the query vector via at least one path comprises: Determine, according to the starting vector, at least one first representative vector associated with the starting vector, wherein the representative vector corresponds to a group; Starting from the at least one first representative vector, a next representative vector is searched to determine at least one path, wherein When the distances of at least one subsequent representative vector associated with the first representative vector are all greater than the distances between the first representative vector and the query vector, terminating the search for the query vector along the path and taking the first representative vector as the end point of the path; In a case where the distance of at least one representative vector associated with the first representative vector is less than the distance between the first representative vector and the query vector, the Lth representative vector closest to the query vector is used as the starting point of the next search, and the search for the L+1th representative vector is continued, and in a case where the distance of at least one representative vector associated with the searched L+1th representative vector is greater than the distance between the Lth representative vector and the query vector, the search for the query vector along the path is terminated, and the Lth vector is used as the end point of the path, wherein L is an integer greater than or equal to 2.

7. The method according to claim 5, characterized in that In the case where M=2, 3, ..., N-1, among all the groups included in the Mth layer, based on the representative vectors of the respective groups, determining the second group closest to the query vector comprises: Determine the starting vector of the Mth layer; Starting from the starting vector, searching the query vector step by step via at least one path, each of the paths passing through at least a representative vector of a group in the Mth layer; From the groups passed by the at least one path, a group having the smallest distance to the query vector is determined as the second group, wherein the distance is the distance between a representative vector of the group and the query vector.

8. The method according to claim 5, characterized in that In the case where M=N, a preset number of candidate vectors are determined from all vectors included in the Mth layer based on the distance between each vector and the query vector, including: Determine all vectors in the Mth layer; Starting from a starting vector determined in the corresponding group of the Mth layer, the query vector is searched step by step via at least one path, each of which passes through at least one vector; From the groups passed by the at least one path, determine a group having the smallest distance to the query vector as the second group, wherein the distance is the distance between a representative vector of the group and the query vector; Determining the distance from the query vector based on the representative vector of each group in all the vectors; and According to the distances, a preset number of vectors are determined as the candidate vectors in order of distance from small to large.

9. The method according to claim 1, characterized in that Each group in the Nth layer forms a membership relationship with at least one group in the N-1th layer in a non-overlapping manner.

10. The method according to claim 1, characterized in that For the first visualization method, the visualization presentation methods of the connections between different vectors are different.

11. The method according to claim 1, characterized in that For the second visualization method, different layers are adjusted in different visualization presentation methods.

12. The method according to claim 2, characterized in that The vector similarity search is based on an inverted index algorithm.

13. The method according to claim 2, characterized in that The vector similarity search is based on the HNSW algorithm.

14. The method according to claim 1, wherein: The preset number is set by receiving user input via a preset interface.

15. The method according to claim 5, characterized in that The initial vector is set by receiving user input via a preset interface.

16. The method according to claim 1, wherein: At least one of the first visualization mode, the second visualization mode and the third visualization mode is set by receiving user input via a preset interface.

17. A visualization device for vector similarity search, characterized in that: The device comprises: An initialization module, wherein the initialization module determines a query vector and a target library for searching, and runs a vector similarity search, wherein the target library includes a plurality of vectors, and the target library is presented as an N-layer hierarchical structure, each layer of the hierarchical structure includes at least one group, and each group includes at least one vector; each group in the N layers includes a representative vector; wherein N is an integer greater than or equal to 2; A first visualization processing module, wherein the first determination module determines at least one group in a first layer of the N layers, and determines at least one first group within a preset range from the query vector based on representative vectors of each group, and displays at least part of a process of determining the first group in a first visualization manner, wherein the first visualization manner includes a visualization manner for displaying a connection from one vector to another vector; A second visualization processing module, starting from each first group in the at least one first group, presents a process of entering the Mth layer layer by layer from the 1st layer in a second visualization manner, where M=2, 3, ..., N-1, N; wherein the second visualization manner is a visualization manner for displaying a jump from one layer to another layer; The third visualization processing module determines the candidate vector for the Mth layer by searching layer by layer, where In the case where M=2, 3, ..., N-1, in the Mth layer, among all the groups belonging to the group determined in the previous layer, a second group closest to the query vector is determined based on the representative vector of each group, and at least part of the process of determining the second group in each layer is displayed in a first visualization manner, and a process of entering the M+1th layer from the Mth layer is presented in a second visualization manner; and In the case where M=N, a preset number of candidate vectors are determined from all vectors in the Mth layer belonging to the group determined in the previous layer based on the distance between each vector and the query vector; and at least part of the process of determining the candidate vectors is displayed in a third visualization manner, and A fourth visualization processing module, which displays the candidate vector in a fourth visualization manner; The third visualization method includes a visualization method for displaying a connection from the query vector to the candidate vector, and the fourth visualization method includes a visualization method for emphasizing the candidate vector.

18. A computer device comprising a memory, a processor and a computer program stored in the memory, It is characterized in that The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 16.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

20. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

Images