Method and device for realizing index generation of vector database through computing power of intelligent computing center

Abstract

The invention provides a method and a device for realizing index generation of a vector database through computing power of an intelligent computing center. The method comprises the following steps: acquiring an index creation request; performing matching in a plurality of candidate algorithms included in the vector database according to the first configuration parameter, and determining a plurality of algorithms; screening the plurality of algorithms according to the first user performance preference information to determine a first target algorithm; and importing the first data column information and the first configuration parameter into the first target algorithm for index generation to obtain the first nearest neighbor index. According to the method, after the configuration parameters, the data column information and the user performance preferences set by the user are obtained, matching is performed in the candidate algorithms supported in the vector database, so that the optimal algorithm is determined, the nearest neighbor index is generated according to the algorithm, the efficiency of generating the nearest neighbor index is improved, and the user experience is improved. And the efficiency of querying the vector data through the nearest neighbor index is further improved.

Description

Technical Field

[0001] The present application relates to the technical field of computing power infrastructure, and in particular to a method and device for generating an index for a vector database through the computing power of an intelligent computing center. Background Art

[0002] With the development of artificial intelligence technology and computing power technology, the concept of intelligent computing center has emerged. "Intelligent computing center" refers to the use of large-scale heterogeneous computing power resources, including general computing power and intelligent computing power, mainly for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning and other scenarios) to provide the required computing power, data and algorithms. Intelligent computing center covers facilities, hardware, software, and can provide full-stack capabilities from bottom-level computing power to top-level application enablement.

[0003] Intelligent computing centers are currently widely used in many fields. How to use the computing power of intelligent computing centers to improve efficiency in certain fields has become a key issue. For example, in vector databases, it is generally necessary to perform similarity searches on vector data. In order to speed up this process, a nearest neighbor index can be established for the vector data. In existing vector databases, when establishing a nearest neighbor index for a vector data column, the user not only needs to specify the data column to be indexed, but also needs to specify the similarity function, the nearest neighbor index algorithm, and the parameters required by the algorithm. In addition, the user must also specify other non-vector data columns to be written in the nearest neighbor index. As data in the database is continuously inserted and deleted, the algorithm used by the nearest neighbor index remains unchanged when it is rewritten or updated, resulting in the problem of low efficiency in querying vector data through the nearest neighbor index. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device and storage medium for generating an index for a vector database through the computing power of an intelligent computing center, thereby solving the problem of low efficiency in querying vector data through nearest neighbor indexing in the prior art.

[0005] To solve the above problems, this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for generating an index of a vector database by using computing power of an intelligent computing center, the method comprising:

[0007] Step S1: obtaining an index creation request, where the index creation request is used to create a first nearest neighbor index in a vector database, and the index creation request includes a first configuration parameter, first data column information, and first user performance preference information for creating the first nearest neighbor index;

[0008] Step S2: According to the first configuration parameter, a plurality of candidate algorithms included in the vector database are matched to determine a plurality of algorithms, wherein the plurality of algorithms support the first configuration parameter, wherein the vector database includes vector data, and the candidate algorithms are used to query the vector data in the vector database;

[0009] Step S3: screening the multiple algorithms according to the first user performance preference information to determine a first target algorithm, where the first target algorithm is the algorithm with the highest matching degree with the first user performance preference information among the multiple algorithms;

[0010] Step S4: importing the first data column information and the first configuration parameter into the first target algorithm to generate an index, and obtaining the first nearest neighbor index.

[0011] Optionally, the step S3: screening the multiple algorithms according to the first user performance preference information to determine a first target algorithm includes:

[0012] Step S31: determining a first algorithm performance, a first empirical formula and a first data volume estimation value corresponding to each algorithm in the plurality of algorithms;

[0013] Step S32: Calculate a plurality of first performance indicators corresponding to each algorithm according to the first algorithm performance, the first empirical formula and the first data volume estimation value corresponding to each algorithm, wherein the first performance indicators are used to represent the computational efficiency of each algorithm;

[0014] Step S33: determining first weight information according to the first user performance preference information, wherein the first weight information includes weight values ​​corresponding to the plurality of first performance indicators one by one, wherein different user performance preference information corresponds to different weight information;

[0015] Step S34: Based on the first weight information, weighted sums are respectively performed on the multiple first performance indicators corresponding to each algorithm to obtain multiple first calculation scores, where the multiple first calculation scores correspond to the multiple algorithms one by one;

[0016] Step S35: determining a first target algorithm according to the multiple first calculation scores, wherein the first target algorithm is an algorithm having the highest first calculation score among the multiple algorithms.

[0017] Optionally, in step S4: importing the first data column information and the first configuration parameter into the first target algorithm to generate an index, and after obtaining the first nearest neighbor index, the method further includes:

[0018] Step S41: when the storage engine included in the vector database has first target data, acquiring the first target data, wherein the first target data is historical search data of the storage engine;

[0019] Step S42: Import the first target data into the first nearest neighbor index to obtain a second nearest neighbor index.

[0020] Optionally, in step S4: importing the first data column information and the first configuration parameter into the first target algorithm to generate an index, and after obtaining the first nearest neighbor index, the method further includes:

[0021] Step S51: receiving an index update request, the index update request being used to update the first nearest neighbor index, the index update request including second data column information;

[0022] Step S52: updating the first data amount estimation value corresponding to the first target algorithm according to the second data column information to obtain a second data amount estimation value;

[0023] Step S53: updating the plurality of first algorithm performances and the plurality of first empirical formulas corresponding to the plurality of algorithms according to the second data volume estimation value and the second data column information, to obtain a plurality of second algorithm performances and a plurality of second empirical formulas corresponding to the plurality of algorithms;

[0024] Step S54: updating the first nearest neighbor index based on the second data volume estimation value, the multiple second algorithm performances and the multiple second empirical formulas to obtain a third nearest neighbor index.

[0025] Optionally, after the step S51: receiving the index update request, the method further includes:

[0026] Step S61: obtaining the query count of the first nearest neighbor index, where the query count is the number of times the first data column information is queried through the first nearest neighbor index;

[0027] Step S62: updating the query times according to the index update request to obtain a target query times;

[0028] Step S63: storing the target query times in the vector database.

[0029] Optionally, in step S4: importing the first data column information and the first configuration parameter into the first target algorithm to generate an index, and after obtaining the first nearest neighbor index, the method further includes:

[0030] Step S71: receiving an index reconstruction request, wherein the index reconstruction request is used to delete the first nearest neighbor index and generate a fourth nearest neighbor index, and the index reconstruction request includes a second configuration parameter for creating the fourth nearest neighbor index, third data column information, and second user performance preference information;

[0031] Step S72: Calculate a plurality of second performance indicators corresponding to each algorithm according to the third algorithm performance, the third empirical formula and the third data volume estimation value corresponding to each algorithm, wherein the plurality of second performance indicators are used to represent the computational efficiency of each algorithm;

[0032] Step S73: determining second weight information according to the second user performance preference information, wherein the second weight information includes weight values ​​corresponding to the plurality of second performance indicators one by one, wherein different user performance preference information corresponds to different weight information;

[0033] Step S74: Based on the second weight information, weighted sums are respectively performed on the multiple second performance indicators corresponding to each algorithm to obtain multiple second calculation scores, where the multiple second calculation scores correspond to the multiple algorithms one by one;

[0034] Step S75: determining a second target algorithm according to the plurality of second calculation scores, wherein the second target algorithm is the algorithm having the highest corresponding second calculation score among the plurality of algorithms;

[0035] Step S76: importing the third data column information and the second configuration parameter into the second target algorithm to generate an index, and obtaining the fourth nearest neighbor index.

[0036] Optionally, in step S76: importing the third data column information and the second configuration parameter into the second target algorithm to generate an index, and after obtaining the fourth nearest neighbor index, the method further includes:

[0037] Step S81: if the storage engine included in the vector database has second target data, obtain the second target data, where the second target data is historical search data of the storage engine;

[0038] Step S82: importing the second target data and the third target data into the fourth nearest neighbor index to obtain a fifth nearest neighbor index, wherein the third target data is the data included in the first nearest neighbor index;

[0039] Step S83: Routing the user's vector database operation request to the fifth nearest neighbor index, and deleting the first nearest neighbor index.

[0040] In a second aspect, an embodiment of the present application further provides a device for generating an index of a vector database by using computing power of an intelligent computing center, the device comprising:

[0041] an acquisition module, configured to acquire an index creation request, wherein the index creation request is used to create a first nearest neighbor index in a vector database, and the index creation request includes a first configuration parameter, first data column information, and first user performance preference information for creating the first nearest neighbor index;

[0042] a matching module, configured to match a plurality of candidate algorithms included in the vector database according to the first configuration parameter, and determine a plurality of algorithms, wherein the plurality of algorithms support the first configuration parameter, wherein the vector database includes vector data, and the candidate algorithms are used to query the vector data in the vector database;

[0043] A screening module, configured to screen the plurality of algorithms according to the first user performance preference information to determine a first target algorithm, wherein the first target algorithm is the algorithm with the highest degree of matching with the first user performance preference information among the plurality of algorithms;

[0044] A generation module is used to import the first data column information and the first configuration parameter into the first target algorithm to generate an index, so as to obtain the first nearest neighbor index.

[0045] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps in the method described in the first aspect above.

[0046] In a fourth aspect, an embodiment of the present application further provides a readable storage medium for storing a program, which, when executed by a processor, implements the steps in the method described in the first aspect above.

[0047] In a fifth aspect, an embodiment of the present application further provides a computer program product, the computer program product is stored in a storage medium, and the computer program product is executed by at least one processor to implement the steps in the method described in the first aspect. The present application provides a method and device for generating an index for a vector database through the computing power of an intelligent computing center, the method comprising: obtaining an index creation request, the index creation request is used to create a first nearest neighbor index in a vector database, the index creation request includes a first configuration parameter, a first data column information, and a first user performance preference information for creating the first nearest neighbor index; matching a plurality of candidate algorithms included in the vector database according to the first configuration parameter, determining a plurality of algorithms, the plurality of algorithms supporting the first configuration parameter, wherein the vector database includes vector data, and the candidate algorithm is used to query the vector data in the vector database; screening the plurality of algorithms according to the first user performance preference information, determining a first target algorithm, the first target algorithm being the algorithm with the highest degree of matching with the first user performance preference information among the plurality of algorithms; importing the first data column information and the first configuration parameter into the first target algorithm for index generation, and obtaining the first nearest neighbor index. After obtaining the configuration parameters, data column information and user performance preferences set by the user, the present application matches the candidate algorithms supported in the vector database to determine an optimal algorithm, and generates a nearest neighbor index based on the algorithm, thereby improving the efficiency of generating the nearest neighbor index and further improving the efficiency of querying vector data through the nearest neighbor index. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of a method for generating an index for a vector database using the computing power of an intelligent computing center provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of the structure of a device for generating an index for a vector database using the computing power of an intelligent computing center provided in an embodiment of the present application;

[0050] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] The terms "first", "second" etc. in the embodiments of the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. In addition, the terms "include" and "have" and any variation thereof are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment. In addition, "and / or" is used in the present application to represent at least one of connected objects, such as A and / or B and / or C, representing the inclusion of separate A, separate B, separate C, and A and B all exist, B and C all exist, A and C all exist, and 7 situations that A, B and C all exist.

[0053] The "computing power" mentioned in this application is the ability of computer equipment or computing / data center to process information. It is the ability of computer hardware and software to work together to perform certain computing needs. It is the computing power to achieve target result output by processing information data. It is a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity. It mainly provides services to society through computing power infrastructure.

[0054] The "computing power (CP)" described in this application is the ability of a data center server to process data and output results. It is a comprehensive indicator to measure the computing power of a data center, including general computing power, supercomputing power and intelligent computing power. The commonly used unit of measurement is the number of floating point operations per second (Floating Point Operations Per Second, FLOPS, where 1EFLOPS=10^18FLOPS). The larger the value, the stronger the comprehensive computing power. According to calculations, 1EFLOPS is approximately the computing power output of 5 Tianhe-2A or 500,000 mainstream server CPUs (Central Processing Unit) or 2 million mainstream notebooks. The calculation formula is: CP=CP 通用 +CP 智能 +CP 超级 In the embodiment of the present application, the computing power adopts the half-precision floating point computing power (FP16) of the graphics card.

[0055] The "carrying capacity" (Network Power, NP) described in this application is a manifestation of the data transmission capability of computing facilities, including comprehensive capabilities such as network architecture, network bandwidth, transmission latency, intelligent management and scheduling. Carrying capacity involves network transmission within and between data centers, and is a comprehensive indicator for measuring network transmission scheduling capabilities. In the embodiment of this application, carrying capacity uses video memory bandwidth. In the embodiment of this application, storage capacity uses video memory bandwidth.

[0056] The "Storage Power" (SP) described in this application refers to the comprehensive capabilities of a data center in terms of data storage capacity, performance, safety and reliability, and green and low-carbon. It is a comprehensive indicator for measuring the data storage capacity of a data center, including external storage devices such as storage arrays and server built-in storage devices. The commonly used unit of measurement for storage capacity is exabyte (EB, 1EB = 2^60bytes), and the commonly used unit of measurement for performance is the number of reads and writes per second per unit capacity (IOPS / TB, Input / Output Operations Per Second / TB). The disaster recovery ratio is an important manifestation of safety and reliability. In the embodiment of this application, storage power uses video memory capacity.

[0057] The "computing power infrastructure" described in this application is a new type of information infrastructure that integrates information computing power, network carrying capacity, and data storage capacity. It can realize centralized computing, storage, transmission and application of information, and presents characteristics such as diversity and ubiquity, intelligence and agility, security and reliability, green and low-carbon. It is of great significance to promote industrial transformation and upgrading, enable my country's scientific and technological innovation, meet people's better life and achieve high-efficiency social governance.

[0058] The “computing power” mentioned in this application includes general computing power CP 通用 、Intelligent computing power CP 智能 And super computing power CP 超级 .

[0059] The "general computing power" mentioned in this application refers to the computing power provided by servers based on CPU chips, which is used to support basic general computing such as cloud computing and edge computing.

[0060] The "intelligent computing power" mentioned in this application refers to the large-scale deployment of intelligent computing centers based on special chips such as GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit) for various innovative artificial intelligence applications, such as natural language processing, machine vision, etc.

[0061] The "supercomputing power" mentioned in this application is mainly the computing power provided by high-performance computing clusters such as supercomputers. It utilizes the centralized computing resources of multiple computer systems working in parallel and uses a dedicated operating system to handle extremely complex or data-intensive problems. It is mainly used for calculations in cutting-edge scientific fields, such as planetary simulation, drug molecule design, and gene analysis.

[0062] The "intelligent computing center" mentioned in this application refers to a facility that provides the required computing power, data and algorithms for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning) by using large-scale heterogeneous computing resources, including general computing power (CPU) and intelligent computing power (GPU, FPGA, ASIC, etc.). The intelligent computing center covers facilities, hardware, and software, and can provide full-stack capabilities from bottom-level computing power to top-level application enablement.

[0063] The "computing resources" mentioned in this application refer to technologies and facilities with information computing, transmission, storage and application capabilities required for the development of a digital society, including but not limited to computing resources such as CPUs and GPUs, network resources such as switches and routers, storage resources such as storage arrays and distributed storage, security resources such as firewalls and intrusion detection systems, and supporting and guarantee resources such as wind, fire, water and electricity.

[0064] This application mainly uses the computing power of the intelligent computing center to realize the index generation of the vector database. Specifically, see Figure 1 , Figure 1 It is a flowchart of a method for generating an index for a vector database by using the computing power of an intelligent computing center provided in an embodiment of the present application.

[0065] Step S1: Obtain an index creation request, where the index creation request is used to create a first nearest neighbor index in a vector database, and the index creation request includes a first configuration parameter, first data column information, and first user performance preference information for creating the first nearest neighbor index.

[0066] In this embodiment, the execution subject of the present application can be a server storing a vector database, wherein the vector database is a special type of database specially designed to store and retrieve vector data. Vector data is usually high-dimensional and can represent feature vectors of images, text, audio or other complex data types. These feature vectors are usually generated by deep learning models or other machine learning algorithms to capture and represent key attributes of raw data. One of the core functions of a vector database is to support efficient similarity search, that is, to quickly find the most similar items based on the distance or similarity measure between vectors (such as Euclidean distance, cosine similarity, etc.). This search is usually called nearest neighbor search (NNS). The nearest neighbor index allows to quickly find the vector that is most similar (i.e., nearest neighbor) to a given query vector under a certain metric (such as Euclidean distance or cosine similarity). This data structure has applications in many fields, such as machine learning, pattern recognition, recommendation systems, and information retrieval.

[0067] When the user needs to create the first nearest neighbor index, an index creation request will be sent to the server. After receiving the index creation request, the server first saves the user's relevant settings in the first nearest neighbor index. Specifically, the user's relevant settings are obtained by analyzing the content included in the index creation request, which includes the first configuration parameters, first data column information and first user performance preference information for creating the first nearest neighbor index. It should be noted that the first configuration parameter may include the name, vector dimension, data type, etc. of the nearest neighbor index. The first data column information includes the names of other data columns. The first user performance preference information is the performance preference set by the user for the nearest neighbor index. It should be noted that the preference is, for example, "tends to run faster", "tends to use smaller memory", "tends to have more accurate query results" and "query results must be absolutely accurate". The performance preference selected by the user can make the generated first nearest neighbor index more in line with the needs of different users.

[0068] Step S2: Determine multiple algorithms by matching multiple candidate algorithms included in the vector database according to the first configuration parameter, and the multiple algorithms support the first configuration parameter, wherein the vector database includes vector data, and the candidate algorithms are used to query the vector data in the vector database.

[0069] In this embodiment, a plurality of candidate algorithms are pre-set in the vector database and a plurality of vector data are stored. The candidate algorithms are used to query the vector data in the vector database. The performance of the nearest neighbor index generated by different candidate algorithms is different. Therefore, a suitable algorithm needs to be selected according to user needs.

[0070] First, a preliminary match is performed with a plurality of candidate algorithms through a first configuration parameter in a vector database, and a plurality of algorithms supporting the first configuration parameter are selected.

[0071] Step S3: Screening the multiple algorithms according to the first user performance preference information to determine a first target algorithm, where the first target algorithm is the algorithm that has the highest degree of matching with the first user performance preference among the multiple algorithms.

[0072] In this embodiment, the determined multiple algorithms are further screened according to the determined first user performance preference information to determine the algorithm with the highest matching degree with the first user performance preference information, that is, the first target algorithm.

[0073] Exemplarily, similarity values ​​between the plurality of algorithms and the first user's performance preference information are calculated respectively, and the first target algorithm having the highest matching degree with the first user's performance preference information is determined according to the magnitude of the similarity values.

[0074] Step S4: importing the first data column information and the first configuration parameter into the first target algorithm to generate an index, and obtaining the first nearest neighbor index.

[0075] In this embodiment, other data columns that need to be written into the nearest neighbor index are determined based on the first data column information. This data column set should first include the data columns that must be written as required by the database system (such as the ID used to associate data with the main storage engine), and secondly include the names of other data columns set by the user.

[0076] The first configuration parameter set by the user and other data columns that need to be written into the nearest neighbor index are imported into the first target algorithm for index generation, and finally the first nearest neighbor index is generated. In the method of this embodiment, when the user creates the nearest neighbor index, he only needs to specify the vector data column to be indexed, and no longer needs to specify the algorithm and parameters to be used. By presetting the relevant configurations of various nearest neighbor index algorithms in the database, the appropriate algorithm can be automatically selected and the algorithm parameters can be determined, thereby reducing the settings that the user needs to make when building the nearest neighbor index and reducing the difficulty of use.

[0077] The present application provides a method for generating an index for a vector database by using the computing power of an intelligent computing center, the method comprising: obtaining an index creation request, the index creation request being used to create a first nearest neighbor index in a vector database, the index creation request comprising a first configuration parameter, a first data column information and a first user performance preference information for creating the first nearest neighbor index; matching multiple candidate algorithms included in the vector database according to the first configuration parameter, determining multiple algorithms, the multiple algorithms supporting the first configuration parameter, wherein the vector database comprises vector data, and the candidate algorithm is used to query the vector data in the vector database; screening the multiple algorithms according to the first user performance preference information, determining a first target algorithm, the first target algorithm being the algorithm with the highest degree of matching with the first user performance preference information among the multiple algorithms; importing the first data column information and the first configuration parameter into the first target algorithm for index generation, and obtaining the first nearest neighbor index. The present application obtains the configuration parameters, data column information and user performance preferences set by the user, matches the candidate algorithms supported in the vector database, thereby determining an optimal algorithm, and generating a nearest neighbor index according to the algorithm, thereby improving the efficiency of generating the nearest neighbor index, and further improving the efficiency of querying vector data through the nearest neighbor index.

[0078] In some feasible implementations, optionally, the step S3: screening the multiple algorithms according to the first user performance preference information to determine the first target algorithm includes:

[0079] Step S31: determining a first algorithm performance, a first empirical formula and a first data volume estimation value corresponding to each algorithm in the plurality of algorithms;

[0080] Step S32: Calculate a plurality of first performance indicators corresponding to each algorithm according to the first algorithm performance, the first empirical formula and the first data volume estimation value corresponding to each algorithm, wherein the first performance indicators are used to represent the computational efficiency of each algorithm;

[0081] Step S33: determining first weight information according to the first user performance preference information, wherein the first weight information includes weight values ​​corresponding to the plurality of first performance indicators one by one, wherein different user performance preference information corresponds to different weight information;

[0082] Step S34: Based on the first weight information, weighted sums are respectively performed on the multiple first performance indicators corresponding to each algorithm to obtain multiple first calculation scores, where the multiple first calculation scores correspond to the multiple algorithms one by one;

[0083] Step S35: determining a first target algorithm according to the multiple first calculation scores, wherein the first target algorithm is an algorithm having the highest first calculation score among the multiple algorithms.

[0084] In this embodiment, each algorithm corresponds to preset performance data and empirical formulas of various nearest neighbor index algorithms, specifically, the first algorithm performance, the first empirical formula and the first data volume estimation value corresponding to each algorithm. When constructing the nearest neighbor index, these performance data and empirical formulas are applied according to the vector dimension and the estimated data volume size, and an optimal algorithm is selected with reference to the user's setting preferences, and the algorithm parameters and other non-vector data columns that need to be written into the nearest neighbor index are determined.

[0085] Multiple first performance indicators are calculated for each algorithm through the first algorithm performance, the first empirical formula and the first data volume estimation value corresponding to each algorithm, wherein the first performance indicator is used to represent the computational efficiency of each algorithm, specifically, for example, including the response time of reading and writing, memory consumption, recall rate, etc. If the algorithm supports configurable parameters, several sets of typical parameters are selected for calculation respectively.

[0086] The first weight information is determined based on the first user performance preference information, wherein the first weight information represents the weight value corresponding to each first performance indicator. When the user performance preference information is different, the weight value corresponding to the same first performance indicator is different. Therefore, the user performance preference information can be used to screen multiple weight values.

[0087] After determining multiple weight values, the calculated indicators are weighted and summed to obtain a score that measures the quality of the algorithm (taking different parameters into account), that is, multiple first calculation scores. The algorithm with the highest score and the corresponding parameters are selected as the first target algorithm. The first target algorithm determined in this way has a higher accuracy, so that the generated nearest neighbor index is more in line with user needs.

[0088] Optionally, in step S4: importing the first data column information and the first configuration parameter into the first target algorithm to generate an index, and after obtaining the first nearest neighbor index, the method further includes:

[0089] Step S41: when the storage engine included in the vector database has first target data, acquiring the first target data, wherein the first target data is historical search data of the storage engine;

[0090] Step S42: Import the first target data into the first nearest neighbor index to obtain a second nearest neighbor index.

[0091] In this embodiment, after the first nearest neighbor index is created, if the main storage engine associated with the first nearest neighbor index has the first target data, the first target data is imported from the main storage engine into the nearest neighbor index. The first target data is historical retrieval data, etc. By importing the first target data into the created first nearest neighbor index, it can be ensured that the first nearest neighbor index accurately contains the historical data, thereby improving the retrieval accuracy of the first nearest neighbor index.

[0092] Optionally, in step S4: importing the first data column information and the first configuration parameter into the first target algorithm to generate an index, and after obtaining the first nearest neighbor index, the method further includes:

[0093] Step S51: receiving an index update request, the index update request being used to update the first nearest neighbor index, the index update request including second data column information;

[0094] Step S52: updating the first data amount estimation value corresponding to the first target algorithm according to the second data column information to obtain a second data amount estimation value;

[0095] Step S53: updating the plurality of first algorithm performances and the plurality of first empirical formulas corresponding to the plurality of algorithms according to the second data volume estimation value and the second data column information, to obtain a plurality of second algorithm performances and a plurality of second empirical formulas corresponding to the plurality of algorithms;

[0096] Step S54: updating the first nearest neighbor index based on the second data volume estimation value, the multiple second algorithm performances and the multiple second empirical formulas to obtain a third nearest neighbor index.

[0097] In this embodiment, after the first nearest neighbor index is created, the user can also initiate an index update request to update the first nearest neighbor index, wherein the server can obtain the second data column information, i.e., the data column involved in the operation, after parsing the index update request. It should be noted that if this operation does not involve the nearest neighbor index, the database operation is performed according to its predetermined process, and the process ends. If this operation involves the nearest neighbor index, the index update process continues.

[0098] Specifically, a database operation is performed on the vector database through an index update request. After the vector database operation is completed, if this operation is an insertion or deletion of data operation (affecting the amount of data in the nearest neighbor index), the first data amount estimate of the nearest neighbor index is updated to obtain the second data amount estimate.

[0099] Thus, all types of databases are operated, specifically, multiple first algorithm performances and multiple first empirical formulas corresponding to multiple algorithms are updated according to the second data volume estimation value and the second data column information, thereby obtaining multiple updated second algorithm performances and multiple second empirical formulas. In addition, the performance data (such as response time, memory consumption, etc.) of this operation is counted and the relevant performance indicators are updated.

[0100] Finally, the first nearest neighbor index is updated according to the obtained second data volume estimate, multiple second algorithm performances and multiple second empirical formulas to obtain a third nearest neighbor index. During the use of the database, it is necessary to continuously update the relevant performance indicators of the nearest neighbor index. By presetting the performance data and empirical formulas of various nearest neighbor index algorithms in the database and performing indicator statistics on the behavior of similarity queries, the algorithm implementation can be appropriately adjusted according to changes in data volume and query behavior, thereby realizing dynamic adjustment of the nearest neighbor index algorithm, algorithm parameters and other parameters during the continuous operation of the database to make it suitable for the current data status and usage behavior of the database and obtain the best performance.

[0101] Optionally, after the step S51: receiving the index update request, the method further includes:

[0102] Step S61: obtaining the query count of the first nearest neighbor index, where the query count is the number of times the first data column information is queried through the first nearest neighbor index;

[0103] Step S62: updating the query times according to the index update request to obtain a target query times;

[0104] Step S63: storing the target query times in the vector database.

[0105] In this embodiment, the query count of the first nearest neighbor index is the number of times the first nearest neighbor index is used to query vector data, for example, the frequency of use of other data columns that are jointly queried with the first nearest neighbor index. The query count of the first nearest neighbor index is stored in the server, and when the first nearest neighbor index is updated, the query count is updated according to the index update request to obtain a target query count, and the target query count is stored.

[0106] Specifically, during the use of the database, the similarity query behavior (human or program call) and performance are statistically analyzed and saved. The indicators include: the amount of data written to the nearest neighbor index (the number and size of data items), the distribution characteristics of vector data, the frequency of a non-vector data column participating in similarity query, the time consumed by the query operation, etc.

[0107] Optionally, in step S4: importing the first data column information and the first configuration parameter into the first target algorithm to generate an index, and after obtaining the first nearest neighbor index, the method further includes:

[0108] Step S71: receiving an index reconstruction request, wherein the index reconstruction request is used to delete the first nearest neighbor index and generate a fourth nearest neighbor index, and the index reconstruction request includes a second configuration parameter for creating the fourth nearest neighbor index, third data column information, and second user performance preference information;

[0109] Step S72: Calculate a plurality of second performance indicators corresponding to each algorithm according to the third algorithm performance, the third empirical formula and the third data volume estimation value corresponding to each algorithm, wherein the plurality of second performance indicators are used to represent the computational efficiency of each algorithm;

[0110] Step S73: determining second weight information according to the second user performance preference information, wherein the second weight information includes weight values ​​corresponding to the plurality of second performance indicators one by one, wherein different user performance preference information corresponds to different weight information;

[0111] Step S74: Based on the second weight information, weighted sums are respectively performed on the multiple second performance indicators corresponding to each algorithm to obtain multiple second calculation scores, where the multiple second calculation scores correspond to the multiple algorithms one by one;

[0112] Step S75: determining a second target algorithm according to the plurality of second calculation scores, wherein the second target algorithm is the algorithm having the highest corresponding second calculation score among the plurality of algorithms;

[0113] Step S76: importing the third data column information and the second configuration parameter into the second target algorithm to generate an index, and obtaining the fourth nearest neighbor index.

[0114] In this embodiment, after the first nearest neighbor index is used for a period of time, the database can trigger the reconstruction operation of the nearest neighbor index by regularly checking the performance data or according to the design requirements, or the reconstruction operation can be triggered by the user through the displayed maintenance instruction. After receiving the index reconstruction request of the user, the server will rebuild the first nearest neighbor index to generate a fourth nearest neighbor index. Specifically, the index reconstruction request includes the second configuration parameter for creating the fourth nearest neighbor index, the third data column information and the second user performance preference information.

[0115] It should be noted that the second configuration parameter may include the name of the nearest neighbor index, vector dimension, data type, etc. The second data column information includes the names of other data columns. The second user performance preference information is the performance preference for the nearest neighbor index set by the user. It should be noted that the preference may be, for example, "tends to run faster", "tends to use less memory", "tends to have more accurate query results", and "query results must be absolutely accurate". The performance preferences selected by the user can make the generated first nearest neighbor index more in line with the needs of different users.

[0116] Through the third algorithm performance, third empirical formula and third data volume estimation corresponding to various algorithms preset in the database system, various performance indicators (including read and write response time, memory consumption, recall rate, etc.) are calculated for each algorithm. If the algorithm supports configurable parameters, several sets of typical parameters are selected for calculation separately.

[0117] The second weight information is determined based on the second user performance preference information, wherein the second weight information represents the weight value corresponding to each second performance indicator. When the user performance preference information is different, the weight value corresponding to the same first performance indicator is different. Therefore, multiple weight values ​​can be screened through the user performance preference information.

[0118] After determining multiple weight values, a weighted sum is taken for each calculated indicator to obtain a score that measures the quality of the algorithm (taking different parameters into account), i.e., multiple second calculation scores. The algorithm with the highest score and the corresponding parameters are selected as the second target algorithm. The second target algorithm thus determined has a higher accuracy, thereby making the generated nearest neighbor index more in line with user needs.

[0119] The third data column information includes other data columns written into the nearest neighbor index. This data column set should first include the data columns that must be written as required by the database system (such as the ID used to associate data with the main storage engine), then include the names of other data columns set by the user, and then include the data columns that are most frequently used in conjunction with the nearest neighbor index (which can be selected using methods such as topN or greater than a certain threshold).

[0120] Finally, the fourth nearest neighbor index is generated by importing the second configuration parameters and the third data column information set by the user into the second target algorithm for index generation.

[0121] When the nearest neighbor index is rebuilt, the algorithm and algorithm parameters are re-determined based on the performance data and empirical formulas of various nearest neighbor index algorithms preset in the database, the user's preferences are referenced, and the existing similarity query behavior indicators are used to re-determine other non-vector data columns that need to be written into the nearest neighbor index. It should be noted that the timing of the rebuild can be: when the nearest neighbor index needs to be split, merged, migrated or compacted for the continuous operation of the database, or when the user actively initiates the nearest neighbor index rebuild operation.

[0122] Optionally, in step S76: importing the third data column information and the second configuration parameter into the second target algorithm to generate an index, and after obtaining the fourth nearest neighbor index, the method further includes:

[0123] Step S81: if the storage engine included in the vector database has second target data, obtain the second target data, where the second target data is historical search data of the storage engine;

[0124] Step S82: importing the second target data and the third target data into the fourth nearest neighbor index to obtain a fifth nearest neighbor index, wherein the third target data is the data included in the first nearest neighbor index;

[0125] Step S83: Routing the user's vector database operation request to the fifth nearest neighbor index, and deleting the first nearest neighbor index.

[0126] In this embodiment, after the fourth nearest neighbor index is generated, if the main storage engine associated with the fourth nearest neighbor index has second target data, the second target data is imported from the main storage engine into the nearest neighbor index. The second target data is historical retrieval data, etc. By importing the second target data into the created second nearest neighbor index, it can be ensured that the second nearest neighbor index accurately contains historical data, thereby improving the retrieval accuracy of the second nearest neighbor index.

[0127] After the second target data is imported, the user's database operation is rerouted to the fourth nearest neighbor index, thereby ensuring that the fourth nearest neighbor index can operate normally.

[0128] It is also necessary to delete the first nearest neighbor index to avoid affecting the operation of the fourth nearest neighbor index. For example, the user can create a data table and a nearest neighbor index using a SQL statement similar to the following:

[0129] CREATE TABLE CLOTHES(

[0130] ID BIGINT AUTO_INCREMENT NOT NULL,

[0131] AGE INT NOT NULL,

[0132] GENDER VARCHAR(12)NOT NULL,

[0133] STYLE FLOAT ARRAY

[512] NOT NULL,

[0134] INDEX STYLE_INDEX VECTOR(STYLE)FAST_QUERY_SPEED,

[0135] PRIMARY KEY(ID) )

[0137] In the above, the line STYLE FLOAT ARRAY

[512] NOT NULL defines a vector data column named STYLE, and the vector type is a 512-dimensional floating-point array. The line INDEX STYLE_INDEX VECTOR(STYLE)FAST_QUERY_SPEED defines a nearest neighbor index named STYLE_INDEX. The vector data column used is STYLE and the user wants a faster query speed.

[0138] After the data table is created and data is imported, users can query it using SQL statements similar to the following:

[0139] SELECT GENDER,DISTANCE($TARGET,STYLE)ASD ORDER BY DLIMIT 10

[0140] Since this SQL statement means to return the top 10 vectors closest to the target vector $TARGET, it meets the requirements for the nearest neighbor index, so the nearest neighbor index STYLE_INDEX will be used for the query. Since the data column GENDER is also included in the query, the frequency of use of the data column GENDER will increase. The more times the user uses the data column GENDER and the nearest neighbor index at the same time, the higher the possibility that the GENDER column will be included in the next nearest neighbor index reconstruction.

[0141] It should be noted that in this embodiment, rebuilding the nearest neighbor index includes two processes: creating a new nearest neighbor index and deleting the old nearest neighbor index. During the creation of the new nearest neighbor index and the import of data, the old nearest neighbor index still exists and can be used. When the new nearest neighbor index is fully ready, the user's database operation will be routed to the new nearest neighbor index, and then the old nearest neighbor index can be safely deleted. In this way, it is ensured that the database can provide services to the outside world uninterruptedly during the reconstruction of the nearest neighbor index.

[0142] After obtaining the configuration parameters, data column information and user performance preferences set by the user, the present application matches the candidate algorithms supported in the vector database to determine an optimal algorithm, and generates a nearest neighbor index based on the algorithm, thereby improving the efficiency of generating the nearest neighbor index and further improving the efficiency of querying vector data through the nearest neighbor index.

[0143] See also Figure 2 , Figure 2 1 is a structural diagram of a device for generating an index of a vector database by using the computing power of an intelligent computing center provided in an embodiment of the present application. Figure 2 As shown, the index generation device 200 for realizing a vector database through the computing power of an intelligent computing center includes:

[0144] An acquisition module 210 is used to acquire an index creation request, where the index creation request is used to create a first nearest neighbor index in a vector database, and the index creation request includes a first configuration parameter for creating the first nearest neighbor index, first data column information, and first user performance preference information;

[0145] A matching module 220, configured to match a plurality of candidate algorithms included in the vector database according to the first configuration parameter, and determine a plurality of algorithms, wherein the plurality of algorithms support the first configuration parameter, wherein the vector database includes vector data, and the candidate algorithms are used to query the vector data in the vector database;

[0146] A screening module 230 is used to screen the multiple algorithms according to the first user performance preference information to determine a first target algorithm, where the first target algorithm is the algorithm with the highest matching degree with the first user performance preference information among the multiple algorithms;

[0147] The generating module 240 is used to import the first data column information and the first configuration parameter into the first target algorithm to generate an index, so as to obtain the first nearest neighbor index.

[0148] Optionally, the screening module 230 includes:

[0149] A first determination submodule, used to determine a first algorithm performance, a first empirical formula, and a first data volume estimation value corresponding to each algorithm of the plurality of algorithms;

[0150] A calculation submodule, used to calculate a plurality of first performance indicators corresponding to each algorithm according to the first algorithm performance, the first empirical formula and the first data volume estimation value corresponding to each algorithm, wherein the first performance indicators are used to represent the calculation efficiency of each algorithm;

[0151] A second determining submodule is used to determine first weight information according to the first user performance preference information, wherein the first weight information includes weight values ​​corresponding to the plurality of first performance indicators one by one, wherein different user performance preference information corresponds to different weight information;

[0152] A weighting submodule, configured to perform weighted summation on the multiple first performance indicators corresponding to each algorithm based on the first weight information to obtain multiple first calculation scores, wherein the multiple first calculation scores correspond to the multiple algorithms one by one;

[0153] The third determination submodule is configured to determine the first target algorithm according to the multiple first calculation scores, where the first target algorithm is the algorithm with the highest first calculation score among the multiple algorithms.

[0154] Optionally, also include:

[0155] A first data acquisition module, configured to acquire first target data if there is first target data in the storage engine included in the vector database, wherein the first target data is historical search data of the storage engine;

[0156] The first data import module is used to import the first target data into the first nearest neighbor index to obtain a second nearest neighbor index.

[0157] Optionally, also include:

[0158] An update acquisition module, configured to receive an index update request, wherein the index update request is used to update the first nearest neighbor index, and the index update request includes second data column information;

[0159] A first updating module, configured to update the first data amount estimation value corresponding to the first target algorithm according to the second data column information to obtain a second data amount estimation value;

[0160] a second updating module, configured to update a plurality of the first algorithm performances and a plurality of the first empirical formulas corresponding to the plurality of algorithms according to the second data volume estimation value and the second data column information, so as to obtain a plurality of second algorithm performances and a plurality of second empirical formulas corresponding to the plurality of algorithms;

[0161] The third updating module is used to update the first nearest neighbor index based on the second data volume estimation value, the multiple second algorithm performances and the multiple second empirical formulas to obtain a third nearest neighbor index.

[0162] Optionally, also include:

[0163] A first acquisition module, configured to acquire the number of queries of the first nearest neighbor index, where the number of queries is the number of times the first data column information is queried through the first nearest neighbor index;

[0164] A fourth updating module, configured to update the query times according to the index update request to obtain a target query times;

[0165] The storage module is used to store the target query times in the vector database.

[0166] Optionally, also include:

[0167] A reconstruction acquisition module, configured to receive an index reconstruction request, wherein the index reconstruction request is used to delete the first nearest neighbor index and generate a fourth nearest neighbor index, wherein the index reconstruction request includes a second configuration parameter for creating the fourth nearest neighbor index, third data column information, and second user performance preference information;

[0168] A reconstruction calculation module, used to calculate a plurality of second performance indicators corresponding to each algorithm according to the third algorithm performance, the third empirical formula and the third data volume estimation value corresponding to each algorithm, wherein the plurality of second performance indicators are used to represent the calculation efficiency of each algorithm;

[0169] A first determining module, configured to determine second weight information according to the second user performance preference information, wherein the second weight information includes weight values ​​corresponding to the plurality of second performance indicators one by one, wherein different user performance preference information corresponds to different weight information;

[0170] A reconstruction weighting module, configured to perform weighted summation on the plurality of second performance indicators corresponding to each of the algorithms based on the second weight information to obtain a plurality of second calculation scores, wherein the plurality of second calculation scores correspond to the plurality of algorithms one by one;

[0171] A second determination module is used to determine a second target algorithm according to the multiple second calculation scores, where the second target algorithm is the algorithm with the highest second calculation score among the multiple algorithms;

[0172] A reconstruction generation module is used to import the third data column information and the second configuration parameter into the second target algorithm to generate an index to obtain the fourth nearest neighbor index.

[0173] Optionally, also include:

[0174] A second acquisition module, configured to acquire, when there is second target data in the storage engine included in the vector database, the second target data, where the second target data is historical search data of the storage engine;

[0175] a fifth updating module, configured to import the second target data and the third target data into the fourth nearest neighbor index to obtain a fifth nearest neighbor index, wherein the third target data is data included in the first nearest neighbor index;

[0176] The deleting module is used to route the vector database operation request of the user to the fifth nearest neighbor index and delete the first nearest neighbor index.

[0177] After obtaining the configuration parameters, data column information and user performance preferences set by the user, the present application matches the candidate algorithms supported in the vector database to determine an optimal algorithm, and generates a nearest neighbor index based on the algorithm, thereby improving the efficiency of generating the nearest neighbor index and further improving the efficiency of querying vector data through the nearest neighbor index.

[0178] The present application also provides an electronic device. Figure 3 , the electronic device may include a processor 301, a memory 302, and a program 3021 stored in the memory 302 and executable on the processor 301.

[0179] When the program 3021 is executed by the processor 301, it can achieve Figure 1 Any step in the corresponding method embodiment:

[0180] Obtaining an index creation request, where the index creation request is used to create a first nearest neighbor index in a vector database, where the index creation request includes a first configuration parameter, first data column information, and first user performance preference information for creating the first nearest neighbor index;

[0181] According to the first configuration parameter, a plurality of candidate algorithms included in the vector database are matched to determine a plurality of algorithms, wherein the plurality of algorithms support the first configuration parameter, wherein the vector database includes vector data, and the candidate algorithms are used to query the vector data in the vector database;

[0182] Screening the multiple algorithms according to the first user performance preference information to determine a first target algorithm, where the first target algorithm is the algorithm with the highest degree of matching with the first user performance preference information among the multiple algorithms;

[0183] The first data column information and the first configuration parameter are imported into the first target algorithm to generate an index, thereby obtaining the first nearest neighbor index.

[0184] Optionally, screening the multiple algorithms according to the first user performance preference information to determine a first target algorithm includes:

[0185] Determine a first algorithm performance, a first empirical formula, and a first data volume estimation value corresponding to each algorithm in the plurality of algorithms;

[0186] According to the first algorithm performance, the first empirical formula and the first data volume estimation value corresponding to each algorithm, a plurality of first performance indicators corresponding to each algorithm are respectively calculated, wherein the first performance indicators are used to represent the computational efficiency of each algorithm;

[0187] Determine first weight information according to the first user performance preference information, the first weight information including weight values ​​corresponding to the plurality of first performance indicators one by one, wherein different user performance preference information corresponds to different weight information;

[0188] Based on the first weight information, weighted sums are respectively performed on the multiple first performance indicators corresponding to each of the algorithms to obtain multiple first calculation scores, where the multiple first calculation scores correspond one-to-one to the multiple algorithms;

[0189] A first target algorithm is determined according to the multiple first calculation scores, where the first target algorithm is the algorithm with the highest first calculation score among the multiple algorithms.

[0190] Optionally, after importing the first data column information and the first configuration parameter into the first target algorithm to generate an index and obtaining the first nearest neighbor index, the method further includes:

[0191] In a case where the storage engine included in the vector database has first target data, acquiring the first target data, wherein the first target data is historical search data of the storage engine;

[0192] The first target data is imported into the first nearest neighbor index to obtain a second nearest neighbor index.

[0193] Optionally, after importing the first data column information and the first configuration parameter into the first target algorithm to generate an index and obtaining the first nearest neighbor index, the method further includes:

[0194] receiving an index update request, the index update request being used to update the first nearest neighbor index, the index update request including second data column information;

[0195] The first data amount estimation value corresponding to the first target algorithm is updated according to the second data column information to obtain a second data amount estimation value;

[0196] updating a plurality of the first algorithm performances and a plurality of the first empirical formulas corresponding to the plurality of algorithms according to the second data volume estimation value and the second data column information, to obtain a plurality of second algorithm performances and a plurality of second empirical formulas corresponding to the plurality of algorithms;

[0197] The first nearest neighbor index is updated based on the second data volume estimation value, the multiple second algorithm performances, and the multiple second empirical formulas to obtain a third nearest neighbor index.

[0198] Optionally, after receiving the index update request, the method further includes:

[0199] Obtaining the number of queries of the first nearest neighbor index, where the number of queries is the number of times the first data column information is queried through the first nearest neighbor index;

[0200] The query times are updated according to the index update request to obtain a target query times;

[0201] The target query times are stored in the vector database.

[0202] Optionally, after importing the first data column information and the first configuration parameter into the first target algorithm to generate an index and obtaining the first nearest neighbor index, the method further includes:

[0203] receiving an index reconstruction request, the index reconstruction request being used to delete the first nearest neighbor index and generate a fourth nearest neighbor index, the index reconstruction request comprising a second configuration parameter for creating the fourth nearest neighbor index, third data column information, and second user performance preference information;

[0204] According to the third algorithm performance, the third empirical formula and the third data volume estimation value corresponding to each algorithm, a plurality of second performance indicators corresponding to each algorithm are respectively calculated, wherein the plurality of second performance indicators are used to represent the computational efficiency of each algorithm;

[0205] Determine second weight information according to the second user performance preference information, the second weight information including weight values ​​corresponding to the plurality of second performance indicators one by one, wherein different user performance preference information corresponds to different weight information;

[0206] Based on the second weight information, weighted sums are respectively performed on the multiple second performance indicators corresponding to each algorithm to obtain multiple second calculation scores, where the multiple second calculation scores correspond to the multiple algorithms one by one;

[0207] Determine a second target algorithm according to the multiple second calculation scores, where the second target algorithm is the algorithm with the highest second calculation score among the multiple algorithms;

[0208] The third data column information and the second configuration parameter are imported into the second target algorithm to generate an index, so as to obtain the fourth nearest neighbor index.

[0209] Optionally, after importing the third data column information and the second configuration parameter into the second target algorithm to generate an index and obtaining the fourth nearest neighbor index, the method further includes:

[0210] In a case where the storage engine included in the vector database has second target data, acquiring the second target data, where the second target data is historical search data of the storage engine;

[0211] Importing the second target data and the third target data into the fourth nearest neighbor index to obtain a fifth nearest neighbor index, wherein the third target data is the data included in the first nearest neighbor index;

[0212] The vector database operation request of the user is routed to the fifth nearest neighbor index, and the first nearest neighbor index is deleted.

[0213] After obtaining the configuration parameters, data column information and user performance preferences set by the user, the present application matches the candidate algorithms supported in the vector database to determine an optimal algorithm, and generates a nearest neighbor index based on the algorithm, thereby improving the efficiency of generating the nearest neighbor index and further improving the efficiency of querying vector data through the nearest neighbor index.

[0214] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned method for generating an index of a vector database by the computing power of an intelligent computing center is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0215] An embodiment of the present application further provides a computer program product, which is stored in a storage medium. The computer program product is executed by at least one processor to implement the various processes of the above-mentioned multimodal large model image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0216] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0217] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0218] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for generating an index for a vector database by using the computing power of an intelligent computing center, characterized in that: The method comprises: Step S1: obtaining an index creation request, where the index creation request is used to create a first nearest neighbor index in a vector database, and the index creation request includes a first configuration parameter, first data column information, and first user performance preference information for creating the first nearest neighbor index; Step S2: According to the first configuration parameter, a plurality of candidate algorithms included in the vector database are matched to determine a plurality of algorithms, wherein the plurality of algorithms support the first configuration parameter, wherein the vector database includes vector data, and the candidate algorithms are used to query the vector data in the vector database; Step S3: screening the multiple algorithms according to the first user performance preference information to determine a first target algorithm, where the first target algorithm is the algorithm with the highest matching degree with the first user performance preference information among the multiple algorithms; Step S4: importing the first data column information and the first configuration parameter into the first target algorithm to generate an index, and obtaining the first nearest neighbor index.

2. The method according to claim 1, characterized in that The step S3: screening the multiple algorithms according to the first user performance preference information to determine a first target algorithm includes: Step S31: determining a first algorithm performance, a first empirical formula and a first data volume estimation value corresponding to each algorithm in the plurality of algorithms; Step S32: Calculate a plurality of first performance indicators corresponding to each algorithm according to the first algorithm performance, the first empirical formula and the first data volume estimation value corresponding to each algorithm, wherein the first performance indicators are used to represent the computational efficiency of each algorithm; Step S33: determining first weight information according to the first user performance preference information, wherein the first weight information includes weight values ​​corresponding to the plurality of first performance indicators one by one, wherein different user performance preference information corresponds to different weight information; Step S34: Based on the first weight information, weighted sums are respectively performed on the multiple first performance indicators corresponding to each algorithm to obtain multiple first calculation scores, where the multiple first calculation scores correspond to the multiple algorithms one by one; Step S35: determining the first target algorithm according to the multiple first calculation scores, wherein the first target algorithm is the algorithm having the highest first calculation score among the multiple algorithms.

3. The method according to claim 1, characterized in that The step S4: importing the first data column information and the first configuration parameter into the first target algorithm to generate an index, and after obtaining the first nearest neighbor index, the method further includes: Step S41: when the storage engine included in the vector database has first target data, acquiring the first target data, wherein the first target data is historical search data of the storage engine; Step S42: Import the first target data into the first nearest neighbor index to obtain a second nearest neighbor index.

4. The method according to claim 2, characterized in that: The step S4: importing the first data column information and the first configuration parameter into the first target algorithm to generate an index, and after obtaining the first nearest neighbor index, the method further includes: Step S51: receiving an index update request, the index update request being used to update the first nearest neighbor index, the index update request including second data column information; Step S52: updating the first data amount estimation value corresponding to the first target algorithm according to the second data column information to obtain a second data amount estimation value; Step S53: updating the plurality of first algorithm performances and the plurality of first empirical formulas corresponding to the plurality of algorithms according to the second data volume estimation value and the second data column information, to obtain a plurality of second algorithm performances and a plurality of second empirical formulas corresponding to the plurality of algorithms; Step S54: updating the first nearest neighbor index based on the second data volume estimation value, the multiple second algorithm performances and the multiple second empirical formulas to obtain a third nearest neighbor index.

5. The method according to claim 4, characterized in that After the step S51: receiving the index update request, the method further includes: Step S61: obtaining the query count of the first nearest neighbor index, where the query count is the number of times the first data column information is queried through the first nearest neighbor index; Step S62: updating the query times according to the index update request to obtain a target query times; Step S63: storing the target query times in the vector database.

6. The method according to claim 2, characterized in that The step S4: importing the first data column information and the first configuration parameter into the first target algorithm to generate an index, and after obtaining the first nearest neighbor index, the method further includes: Step S71: receiving an index reconstruction request, wherein the index reconstruction request is used to delete the first nearest neighbor index and generate a fourth nearest neighbor index, and the index reconstruction request includes a second configuration parameter for creating the fourth nearest neighbor index, third data column information, and second user performance preference information; Step S72: Calculate a plurality of second performance indicators corresponding to each algorithm according to the third algorithm performance, the third empirical formula and the third data volume estimation value corresponding to each algorithm, wherein the plurality of second performance indicators are used to represent the computational efficiency of each algorithm; Step S73: determining second weight information according to the second user performance preference information, wherein the second weight information includes weight values ​​corresponding to the plurality of second performance indicators one by one, wherein different user performance preference information corresponds to different weight information; Step S74: Based on the second weight information, weighted sums are respectively performed on the multiple second performance indicators corresponding to each algorithm to obtain multiple second calculation scores, where the multiple second calculation scores correspond to the multiple algorithms one by one; Step S75: determining a second target algorithm according to the plurality of second calculation scores, wherein the second target algorithm is the algorithm having the highest corresponding second calculation score among the plurality of algorithms; Step S76: importing the third data column information and the second configuration parameter into the second target algorithm to generate an index, and obtaining the fourth nearest neighbor index.

7. The method according to claim 6, characterized in that The step S76: importing the third data column information and the second configuration parameter into the second target algorithm to generate an index, and after obtaining the fourth nearest neighbor index, the method further includes: Step S81: if the storage engine included in the vector database has second target data, obtain the second target data, where the second target data is historical search data of the storage engine; Step S82: importing the second target data and the third target data into the fourth nearest neighbor index to obtain a fifth nearest neighbor index, wherein the third target data is the data included in the first nearest neighbor index; Step S83: Routing the user's vector database operation request to the fifth nearest neighbor index, and deleting the first nearest neighbor index.

8. A device for generating an index of a vector database by using the computing power of an intelligent computing center, characterized in that: The device comprises: an acquisition module, configured to acquire an index creation request, wherein the index creation request is used to create a first nearest neighbor index in a vector database, and the index creation request includes a first configuration parameter, first data column information, and first user performance preference information for creating the first nearest neighbor index; a matching module, configured to match a plurality of candidate algorithms included in the vector database according to the first configuration parameter, and determine a plurality of algorithms, wherein the plurality of algorithms support the first configuration parameter, wherein the vector database includes vector data, and the candidate algorithms are used to query the vector data in the vector database; A screening module, configured to screen the plurality of algorithms according to the first user performance preference information to determine a first target algorithm, wherein the first target algorithm is the algorithm with the highest degree of matching with the first user performance preference information among the plurality of algorithms; A generation module is used to import the first data column information and the first configuration parameter into the first target algorithm to generate an index, so as to obtain the first nearest neighbor index.

9. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is used to read the program in the memory to implement the steps in the method for generating an index for a vector database by using the computing power of an intelligent computing center as described in any one of claims 1 to 7.

10. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps in the method for generating an index for a vector database by using the computing power of an intelligent computing center as described in any one of claims 1 to 7 are implemented.

11. A computer program product, characterized in that The computer program product is stored in a storage medium, and the computer program product is executed by at least one processor to implement the steps in the method for generating an index for a vector database by using the computing power of an intelligent computing center as described in any one of claims 1 to 7.

Images