Vector data query method and device for computing power computing resources of intelligent computing center
By establishing the mapping relationship between the user vector ID and internal vector ID of vector data in the intelligent computing center, the problem of DiskANN service returning only internal vector ID is solved, realizing the user-friendliness of vector data query and improving user experience.
Patent Information
- Application Number
- CN202411216535.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-27
AI Technical Summary
In the intelligent computing center, DiskANN service can only return the internal vector ID of vector data, which makes it difficult for users to understand its meaning, limiting the user experience.
When the vector data is imported to the disk of the DiskANN service, a mapping relationship between the user vector ID and the internal vector ID is established and saved to the mapping table. After receiving the query request, the corresponding user vector ID is found and returned based on the internal vector ID of the target vector data. .
Through the mapping table of user vector ID and internal vector ID, the service diversity of the intelligent computing center is enriched, and users can understand query results more intuitively and improve user experience.
Smart Images

Figure CN119202073A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the technical field of computing power and computing resource management, and in particular, to a method and device for querying vector data of computing power and computing resources of an intelligent computing center. Background Art
[0002] With the development of artificial intelligence technology and computing power technology, the concept of intelligent computing center has emerged. "Intelligent computing center" refers to the use of large-scale heterogeneous computing power resources, including general computing power and intelligent computing power, mainly for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning and other scenarios) to provide the required computing power, data and algorithms. Intelligent computing center covers facilities, hardware, software, and can provide full-stack capabilities from bottom-level computing power to top-level application enablement.
[0003] In the Intelligent Computing Center, vector retrieval can be performed based on DiskANN. DiskANN is a vector retrieval engine based on distributed storage, which can store and retrieve billions of vector data on a single computer. Compared with traditional vector retrieval algorithms, DiskANN has higher storage efficiency and faster retrieval speed.
[0004] In the existing solution, when vector data is imported into the disk of the DiskANN service, the DiskANN service will generate an internal vector ID (identifier) for the vector data. When querying vector data, the DiskANN service can only return the internal vector ID of the queried vector data. It is difficult for users to understand the meaning of the internal vector ID, which brings certain limitations to the services of the intelligent computing center and is not conducive to user experience. Summary of the invention
[0005] An embodiment of the present invention provides a vector data query method and device for computing resources of an intelligent computing center, which is used to solve the problem that when performing vector data query, the DiskANN service can only return the internal vector ID of the queried vector data, and it is difficult for users to understand the meaning of the internal vector ID, which brings certain limitations to the services of the intelligent computing center.
[0006] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:
[0007] In a first aspect, an embodiment of the present invention provides a method for querying vector data of computing resources of an intelligent computing center, including:
[0008] Step S1: in the process of importing vector data into the disk of the DiskANN service, a mapping relationship between the user vector ID of the vector data and the internal vector ID of the DiskANN service is established, and the mapping relationship between the user vector ID and the internal vector ID is saved in a mapping table;
[0009] Step S2: receiving a query request, and querying target vector data similar to the vector data to be queried based on the query request;
[0010] Step S3: according to the target internal vector ID of the target vector data, searching the mapping table for a target user vector ID corresponding to the target vector ID;
[0011] Step S4: Return the target vector data and the target user vector ID.
[0012] Optionally, the data type of the user vector ID is int64, and the data type of the internal vector ID is uint32.
[0013] Optionally, the step S1 includes:
[0014] Step S11: during the process of importing the vector data into the disk of the DiskANN service, checking whether the user vector ID already exists in the mapping table;
[0015] Step S12: if the user vector ID does not exist in the mapping table, establish a mapping relationship between the user vector ID and the internal vector ID;
[0016] Step S13: If the user vector ID exists in the mapping table, an error message is generated and sent to the client.
[0017] Optionally, the mapping relationship includes:
[0018] A first mapping relationship between the internal vector ID and the user vector ID;
[0019] A second mapping relationship between the user vector ID and the internal vector ID.
[0020] Optionally, the data structure adopted by the first mapping relationship is that the internal vector ID is used as the data subscript and the user vector ID is used as the data content;
[0021] The data structure adopted by the second mapping relationship is that the user vector ID is used as a data subscript and the internal vector ID is used as data content.
[0022] Optionally, step S1 is performed in the Bulid process of vector data.
[0023] In a second aspect, an embodiment of the present invention provides a vector data query device for computing resources of an intelligent computing center, including:
[0024] An establishment module is used to establish a mapping relationship between a user vector ID of the vector data and an internal vector ID of the DiskANN service during the process of importing the vector data into the disk of the DiskANN service, and save the mapping relationship between the user vector ID and the internal vector ID into a mapping table;
[0025] A query module, configured to receive a query request and query target vector data similar to the vector data to be queried based on the query request;
[0026] A search module, configured to search the mapping table for a target user vector ID corresponding to the target vector ID according to the target internal vector ID of the target vector data;
[0027] The return module is used to return the target vector data and the target user vector ID.
[0028] In a third aspect, an embodiment of the present invention provides a server comprising: a processor, a memory, and a program stored on the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the vector data query method for computing power resources of an intelligent computing center as described in the first aspect above are implemented.
[0029] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the vector data query method for the computing power resources of the intelligent computing center as described in the first aspect above are implemented.
[0030] In a fifth aspect, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the steps of the vector data query method for computing resources of the intelligent computing center as described in the first aspect above.
[0031] In an embodiment of the present invention, during the process of importing vector data into the disk of the DiskANN service, a mapping relationship between the user vector ID of the vector data and the internal vector ID of the DiskANN service is established and saved in a mapping table. When a vector data query is performed, the user vector ID of the queried target vector data can be determined based on the mapping table of the user vector ID and the internal vector ID and returned to the client, thereby enriching the service diversity of the intelligent computing center. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0033] Figure 1 This is a schematic diagram of the overall process of vector data query based on DiskANN;
[0034] Figure 2 This is one of the flow charts of the vector data query method of the computing power computing resources of the intelligent computing center according to an embodiment of the present invention;
[0035] Figure 3 The second flowchart of the method for querying vector data of computing resources of the intelligent computing center computing power according to an embodiment of the present invention;
[0036] Figure 4 It is a structural schematic diagram of a vector data query device for computing resources of an intelligent computing center according to an embodiment of the present invention;
[0037] Figure 5 Schematic diagram of the structure of a server according to an embodiment of the present invention. DETAILED DESCRIPTION
[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0039] First, the technical terms involved in the present invention are briefly explained below.
[0040] The "computing power" mentioned in the present invention is the ability of computer equipment or computing / data center to process information. It is the ability of computer hardware and software to work together to execute certain computing requirements. It is the computing power to achieve target result output by processing information data. It is a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity. It mainly provides services to the society through computing power infrastructure.
[0041] The "computing power" (Computational Power, CP) described in the present invention is the ability of a data center server to process data and output results. It is a comprehensive indicator to measure the computing power of a data center, including general computing power, super computing power and intelligent computing power. The commonly used unit of measurement is the number of floating point operations performed per second (FLOPS: Floating Point Operations Per Second, 1EFLOPS = 10^18FLOPS). The larger the value, the stronger the comprehensive computing power. According to calculations, 1EFLOPS is approximately the computing power output of 5 Tianhe-2A or 500,000 mainstream server CPUs or 2 million mainstream notebooks. The calculation formula is: CP = CP 通用 +CP 智能 +CP 超级 .
[0042] The "Network Power" (NP) described in the present invention is a manifestation of the data transmission capability of computing facilities, including comprehensive capabilities such as network architecture, network bandwidth, transmission latency, intelligent management and scheduling, etc. The carrying capacity involves network transmission within and between data centers, and is a comprehensive indicator for measuring network transmission scheduling capabilities. In the embodiment of the present invention, the carrying capacity uses the memory bandwidth.
[0043] The "Storage Power" (SP) described in the present invention is the comprehensive ability of a data center in terms of data storage capacity, performance, safety and reliability, and green and low-carbon. It is a comprehensive indicator for measuring the data storage capacity of a data center, including external storage devices such as storage arrays and server built-in storage devices. The commonly used unit of measurement for storage capacity is exabyte (EB, 1EB = 2^60bytes), and the commonly used unit of measurement for performance is the number of reads and writes per second per unit capacity (IOPS / TB, Input / Output Operations Per Second / TB). The disaster recovery ratio is an important manifestation of safety and reliability.
[0044] The "computing power infrastructure" described in the present invention is a new type of information infrastructure that integrates information computing power, network carrying capacity, and data storage capacity. It can realize centralized computing, storage, transmission and application of information, and presents characteristics such as diversity and ubiquity, intelligence and agility, security and reliability, and green and low-carbon.
[0045] The “computing power” mentioned in the present invention includes general computing power, intelligent computing power and super computing power.
[0046] The “general computing power” mentioned in the present invention refers to the computing power provided by servers based on CPU (Central Processing Unit) chips, which is used to support basic general computing such as cloud computing and edge computing.
[0047] The "intelligent computing power" described in the present invention is a computing platform for large-scale deployment of special chips such as GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit) for various innovative applications of artificial intelligence, such as natural language processing and machine vision.
[0048] The "super computing power" mentioned in the present invention is mainly the computing power provided by high-performance computing clusters such as supercomputers. It utilizes the centralized computing resources of multiple computer systems working in parallel and uses a dedicated operating system to handle extremely complex or data-intensive problems. It is mainly used for calculations in cutting-edge scientific fields, such as planetary simulation, drug molecule design, and gene analysis.
[0049] The "intelligent computing center" described in the present invention refers to a facility that provides the required computing power, data and algorithms for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning scenarios) by using large-scale heterogeneous computing power resources, including general computing power (CPU: Central Processing Unit) and intelligent computing power (GPU: Graphics Processing Unit, FPGA: Field Programmable Gate Array, ASIC: Application Specific Integrated Circuit, etc.). The intelligent computing center covers facilities, hardware, and software, and can provide full-stack capabilities from bottom-level computing power to top-level application enablement.
[0050] The "computing resources" mentioned in the present invention refer to technologies and facilities with information calculation, transmission, storage and application capabilities required for the development of a digital society, including but not limited to computing resources such as CPU (Central Processing Unit) and GPU (Graphics Processing Unit), network resources such as switches and routers, storage resources such as storage arrays and distributed storage, security resources such as firewalls and intrusion detection systems, and supporting and guarantee resources such as wind, fire, water and electricity.
[0051] The “computing power resources” mentioned in the present invention refer to GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), etc.
[0052] The "DiskANN" described in the present invention is a vector search engine based on distributed storage, which can store and retrieve billions of vector data on a single computer. Compared with traditional vector search algorithms, DiskANN has higher storage efficiency and faster search speed.
[0053] In the embodiment of the present invention, the overall process of vector data query based on DiskANN can be found in the attached Figure 1 , the overall process mainly includes the following processes: Create, ImportData, Build, Load, Search, Close and Destroy. Among them, the Create process refers to the creation of the DiskANN index object. At this time, there is no data in the index object and no service can be provided to the outside. The ImportData process refers to the client sending the vector data to the Index service process, and the Index service process temporarily stores the vector data in the database of the Index service. After the data import process is completed, you can enter the Create process. The Create process refers to pushing the vector data temporarily stored in the database of the Index service to the DiskANN service process (i.e. Push Data). After the vector data is pushed, the DiskANN service process processes the vector data based on the computing power resources to obtain the graph index of the vector data, and saves the vector data and the graph index to the data file of the disk corresponding to the DiskANN service. The Load process refers to loading the graph index of the vector data (which may also include a small amount of vector data) into the memory. The search process refers to retrieving the vector data most similar to the vector data to be queried based on the graph index. The close process refers to closing the DiskANN index object. Destroy refers to destroying the DiskANN index object.
[0054] The above process includes three execution entities: the client, the Index service process and the DiskANN service process. The purpose of using the Index service process is to make DiskANN as lightweight as possible. The DiskANN service process only handles the core business of DiskANN, and other businesses are outsourced to the Index service process to ensure that the overall service is stable and reliable. Before executing the above process, the Index service process and the DiskANN service process need to make a handshake connection in advance to provide services for subsequent businesses. The above Index service process and DiskANN service process can be deployed on different physical machines or on the same physical machine, and the present invention does not limit this.
[0055] Please refer to Figure 2 , Figure 2 The flowchart of the method for querying vector data of computing resources of the intelligent computing center computing power according to an embodiment of the present invention is as follows:
[0056] Step S1: in the process of importing vector data into the disk of the DiskANN service, a mapping relationship between the user vector ID of the vector data and the internal vector ID of the DiskANN service is established, and the mapping relationship between the user vector ID and the internal vector ID is saved in a mapping table;
[0057] In the embodiment of the present invention, the user vector ID is an ID of vector data defined by a user.
[0058] The internal vector ID of the DiskANN service is the ID generated by the DiskANN service for the vector data imported into the DiskANN service. The ID can be generated for each imported vector data in sequence according to the order of the imported vector data. For example, the internal vector ID of the first imported vector data is 0, and the internal vector ID of the second imported vector data is 1.
[0059] Step S2: receiving a query request, and querying target vector data similar to the vector data to be queried based on the query request;
[0060] Step S3: according to the target internal vector ID of the target vector data, searching the mapping table for a target user vector ID corresponding to the target vector ID;
[0061] Step S4: Return the target vector data and the target user vector ID.
[0062] In the embodiment of the present invention, when the above steps are executed, the computing power and computing resources of the intelligent computing center are used to perform the above steps of importing and querying the vector data.
[0063] In an embodiment of the present invention, during the process of importing vector data into the disk of the DiskANN service, a mapping relationship between the user vector ID of the vector data and the internal vector ID of the DiskANN service is established and saved in a mapping table. When a vector data query is performed, the user vector ID of the queried target vector data can be determined based on the mapping table of the user vector ID and the internal vector ID and returned to the client, thereby enriching the service diversity of the intelligent computing center.
[0064] The above method in the embodiment of the present invention may be executed by the DiskANN service process or by the Index service process.
[0065] When the above method is executed by the DiskANN service process, the above step S1 includes: the DiskANN service process receives the vector data and the user vector ID corresponding to the vector data sent by the Index service process. The vector data and the user vector ID can be sent in the form of lists respectively. That is, the vector data is in one list, the user vector ID is in another list, and the data of each row in the two lists can correspond one to one. For example, the vector data of the first row of the vector data list corresponds to the user vector ID of the first row in the user vector ID list.
[0066] In the embodiment of the present invention, it is optional, please refer to Figure 3 , the above step S1 can be performed in the above creation (Bulid) process. Figure 3 In the flowchart shown, DiskANN instance refers to the instance that provides DiskANN service. Rockdb is the database of Index service process. Figure 3 It can be seen that the establishment of the mapping table is executed during the creation (Build) process.
[0067] In the embodiment of the present invention, optionally, the data type of the user vector ID is int64, and the data type of the internal vector ID is uint32. The data type of the user vector ID is int64, and the byte length is relatively larger, so it can meet more requirements.
[0068] The following is the code for establishing the mapping relationship between the user vector ID of the vector data and the internal vector ID of the DiskANN service in step S1 above:
[0069] message VectorPushDataRequest{
[0070] dingodb.pb.common.RequestInfo request_info=1;
[0071] / / vector index ID is equivalent to region_id
[0072] int64 vector_index_id = 2;
[0073] / / vector data vector data list
[0074] repeated dingodb.pb.common.Vector vectors=3;
[0075] / / vector id vector ID list
[0076] repeated int64 vector_ids=4;
[0077] }
[0078] Among them, "request_info" indicates the identifier of this import data request.
[0079] "vector_index_id" indicates the identifier of the index object.
[0080] "vectors" represents vector data.
[0081] “vector_ids” indicates user vector IDs.
[0082] In the embodiment of the present invention, optionally, step S1 includes:
[0083] Step S11: during the process of importing the vector data into the disk of the DiskANN service, checking whether the user vector ID already exists in the mapping table;
[0084] Step S12: if the user vector ID does not exist in the mapping table, establish a mapping relationship between the user vector ID and the internal vector ID;
[0085] Step S13: If the user vector ID exists in the mapping table, an error message is generated and sent to the client.
[0086] In an embodiment of the present invention, before establishing a mapping relationship between a user vector ID and the internal vector ID, it is necessary to determine whether the user vector ID already exists in the mapping table. If so, an error message is generated and sent to the client to remind the client to reset the user vector ID and re-import the data to avoid confusion in the mapping table.
[0087] In the embodiment of the present invention, optionally, the mapping relationship includes:
[0088] A first mapping relationship between the internal vector ID and the user vector ID;
[0089] A second mapping relationship between the user vector ID and the internal vector ID.
[0090] In the embodiment of the present invention, optionally, the data structure adopted by the first mapping relationship is that the internal vector ID is used as the data subscript and the user vector ID is used as the data content;
[0091] The data structure adopted by the second mapping relationship is that the user vector ID is used as a data subscript and the internal vector ID is used as data content.
[0092] The following is the code for the above two mapping relationships:
[0093] class DiskANNItem{
[0094] / / Internal vector ID to user vector ID mapping
[0095] std::vector<int64_t> diskann_to_vector_ids_;
[0096] / / Mapping relationship between user vector ID and internal vector ID
[0097] std::unordered_map<int64_t,uint32_t> vector_to_diskann_ids_;
[0098] Among them, std::vector<int64_t> diskann_to_vector_ids represents a first mapping relationship between an internal vector ID and the user vector ID;
[0099] std::unordered_map<int64_t,uint32_t> vector_to_diskann_ids represents a second mapping relationship from a user vector ID to the internal vector ID.
[0100] The structure of the mapping table in the embodiment of the present invention may be as shown in Table 1:
[0101]
[0102]
[0103] Table 1
[0104] Among them, A100, A101, A102... are user vector IDs, and 0, 1, 2... are internal vector IDs.
[0105] In an embodiment of the present invention, through the second mapping relationship in the mapping table, vector data can be queried based on the user vector ID, and the internal vector ID of the vector data can be queried. Alternatively, through the first mapping relationship in the mapping table, vector data can be queried based on the internal vector ID, and the user vector ID of the vector data can be queried.
[0106] In an embodiment of the present invention, if the DiskANN service process executes steps S1 to 4, the above-mentioned query request can be sent by the client to the Index service process, and then sent by the Index service process to the DiskANN service process. The DiskANN service process queries at least one target vector data similar to the vector data to be queried based on the vector data to be queried, and queries the mapping table based on the internal vector ID of the target vector data to obtain the user vector ID of the target vector data, and forwards the target vector data and the user vector ID to the client by the Index service process.
[0107] In an embodiment of the present invention, if the Index service process executes steps S1 to 4, the above-mentioned query request can be sent by the client to the Index service process, and forwarded by the Index service process to the DiskANN service process. The DiskANN service process queries at least one target vector data similar to the vector data to be queried based on the vector data to be queried and returns it to the Index service process. The Index service process queries the mapping table based on the internal vector ID of the target vector data, obtains the user vector ID of the target vector data, and returns the target vector data and the user vector ID to the client.
[0108] Please refer to Figure 4 The embodiment of the present invention further provides a vector data query device 10 for computing resources of an intelligent computing center, including:
[0109] Establishing module 11, used for establishing a mapping relationship between a user vector ID of the vector data and an internal vector ID of the DiskANN service during the process of importing the vector data into the disk of the DiskANN service, and saving the mapping relationship between the user vector ID and the internal vector ID into a mapping table;
[0110] A query module 12, configured to receive a query request and query target vector data similar to the vector data to be queried based on the query request;
[0111] A search module 13, configured to search the mapping table for a target user vector ID corresponding to the target vector ID according to the target internal vector ID of the target vector data;
[0112] The returning module 14 is used to return the target vector data and the target user vector ID.
[0113] In an embodiment of the present invention, during the process of importing vector data into the disk of the DiskANN service, a mapping relationship between the user vector ID of the vector data and the internal vector ID of the DiskANN service is established and saved in a mapping table. When a vector data query is performed, the user vector ID of the queried target vector data can be determined based on the mapping table of the user vector ID and the internal vector ID and returned to the client, thereby enriching the service diversity of the intelligent computing center.
[0114] In the embodiment of the present invention, optionally, the data type of the user vector ID is int64, and the data type of the internal vector ID is uint32.
[0115] In the embodiment of the present invention, optionally, the establishing module 11 includes:
[0116] A checking submodule, used for checking whether the user vector ID already exists in the mapping table during the process of importing the vector data into the disk of the DiskANN service;
[0117] An establishment submodule, used for establishing a mapping relationship between the user vector ID and the internal vector ID if the user vector ID does not exist in the mapping table;
[0118] The sending submodule is used to generate an error message and send it to the client if the user vector ID exists in the mapping table.
[0119] Optionally, the mapping relationship includes:
[0120] A first mapping relationship between the internal vector ID and the user vector ID;
[0121] A second mapping relationship between the user vector ID and the internal vector ID.
[0122] Optionally, the data structure adopted by the first mapping relationship is that the internal vector ID is used as the data subscript and the user vector ID is used as the data content;
[0123] The data structure adopted by the second mapping relationship is that the user vector ID is used as a data subscript and the internal vector ID is used as data content.
[0124] Please refer to Figure 5 The embodiment of the present invention further provides a server 20, including a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the computer program is executed by the processor 21, each process of the vector data query method embodiment of the computing power and computing resources of the above-mentioned intelligent computing center is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0125] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the above-mentioned vector data query method embodiment of the computing power and computing resources of the intelligent computing center is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0126] The present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above Figure 2 The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.
[0127] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0128] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.
[0129] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.
Claims
1. A vector data query method for computing resources of intelligent computing center, characterized in that: include: Step S1: in the process of importing vector data into the disk of the DiskANN service, a mapping relationship between the user vector ID of the vector data and the internal vector ID of the DiskANN service is established, and the mapping relationship between the user vector ID and the internal vector ID is saved in a mapping table; Step S2: receiving a query request, and querying target vector data similar to the vector data to be queried based on the query request; Step S3: according to the target internal vector ID of the target vector data, searching the mapping table for a target user vector ID corresponding to the target vector ID; Step S4: Return the target vector data and the target user vector ID.
2. The method according to claim 1, characterized in that The data type of the user vector ID is int64, and the data type of the internal vector ID is uint32.
3. The method according to claim 1, characterized in that The step S1 comprises: Step S11: during the process of importing the vector data into the disk of the DiskANN service, checking whether the user vector ID already exists in the mapping table; Step S12: if the user vector ID does not exist in the mapping table, establish a mapping relationship between the user vector ID and the internal vector ID; Step S13: If the user vector ID exists in the mapping table, an error message is generated and sent to the client.
4. The method according to claim 1, characterized in that The mapping relationship includes: A first mapping relationship between the internal vector ID and the user vector ID; A second mapping relationship between the user vector ID and the internal vector ID.
5. The method according to claim 4, characterized in that: The data structure adopted by the first mapping relationship is that the internal vector ID is used as a data subscript and the user vector ID is used as data content; The data structure adopted by the second mapping relationship is that the user vector ID is used as a data subscript and the internal vector ID is used as data content.
6. The method according to claim 1, characterized in that The step S1 is performed in the Bulid process of the vector data.
7. A vector data query device for computing resources of intelligent computing center, characterized in that: include: An establishment module is used to establish a mapping relationship between a user vector ID of the vector data and an internal vector ID of the DiskANN service during the process of importing the vector data into the disk of the DiskANN service, and save the mapping relationship between the user vector ID and the internal vector ID into a mapping table; A query module, configured to receive a query request and query target vector data similar to the vector data to be queried based on the query request; A search module, configured to search the mapping table for a target user vector ID corresponding to the target vector ID according to the target internal vector ID of the target vector data; The return module is used to return the target vector data and the target user vector ID.
8. A server, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the vector data query method for computing resources of an intelligent computing center as described in any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the vector data query method for computing resources of an intelligent computing center as described in any one of claims 1 to 6.
10. A computer program product, characterized in that It includes computer instructions, which, when executed by a processor, implement the steps of the vector data query method for the computing power and computing resources of an intelligent computing center as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Metering accessing of content in a content protection system or its analog
CN1690913A
Querying data and an associated ontology in a database management system
US20080172353A1
Method and system for implementing reverse directory lookup using hashed file metadata
US20180032540A1
Cited By
Method and device for unifying computing power resource vector distance of intelligent computing center
CN120429297A
A unified method and device for vector distance of computing power resources of intelligent computing center
CN120429297B