Method for optimizing computing power computing resources of intelligent computing center by deleting data structure

Abstract

The invention provides a method for optimizing computing power computing resources of an intelligent computing center by deleting a data structure, and the method comprises the steps: S1, obtaining a plurality of designated user IDs carried in a vector data deletion instruction when the vector data deletion instruction sent by a client is received; s2, deleting a plurality of appointed user IDs in the first mapping table to obtain an updated first mapping table, and recording upper boundary IDs and lower boundary IDs of a plurality of appointed internal IDs corresponding to the plurality of appointed user IDs before the plurality of appointed user IDs are deleted; s3, first operation is executed on a second mapping table, a second mapping relation from the user ID to the internal ID is stored in the second mapping table, and the first operation at least comprises the step of reconstructing the second mapping relation corresponding to the first user ID corresponding to the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before updating; and deleting the second mapping relationship corresponding to the plurality of specified user IDs in the second mapping table. The computing power computing resources of the intelligent computing center can be optimized.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of computing power infrastructure, and in particular, to a method for optimizing computing power resources of an intelligent computing center by deleting data structures. Background Art

[0002] With the development of artificial intelligence technology and computing power technology, the concept of intelligent computing center has emerged. "Intelligent computing center" refers to the use of large-scale heterogeneous computing power resources, including general computing power and intelligent computing power, mainly for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning and other scenarios) to provide the required computing power, data and algorithms. Intelligent computing center covers facilities, hardware, software, and can provide full-stack capabilities from bottom-level computing power to top-level application enablement.

[0003] The intelligent computing center can use Faiss as a vector database. Faiss is an efficient similarity search and clustering library. It can quickly process large-scale data and support similarity search in high-dimensional space.

[0004] In the related technology, during the Flat (brute force search) vector indexing process in Faiss, it is necessary to delete the data in the mapping table of the internal ID and user ID used to record the vector data to update the data structure. Currently, the process of deleting the ID in the mapping table is time-consuming and consumes a lot of computing power and computing resources of the intelligent computing center. Summary of the invention

[0005] An embodiment of the present invention provides a method for optimizing the computing power and computing resources of an intelligent computing center by deleting data structures, which is used to solve the problem that in the process of Flat vector indexing in Faiss, it is necessary to delete the data in the mapping table for recording the internal ID and user ID of vector data to update the data structure. Currently, the process of deleting the ID in the mapping table is time-consuming and consumes a lot of computing power and computing resources of the intelligent computing center.

[0006] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0007] In a first aspect, an embodiment of the present invention provides a method for optimizing computing resources of an intelligent computing center by deleting a data structure, including:

[0008] Step S1: when receiving a vector data deletion instruction sent by a client, obtaining multiple designated user IDs of multiple vector data to be deleted carried in the vector data deletion instruction;

[0009] Step S2: deleting the multiple specified user IDs in the first mapping table, obtaining an updated first mapping table, and recording the upper boundary IDs and lower boundary IDs of the multiple specified internal IDs corresponding to the multiple specified user IDs before being deleted; the first mapping table stores a first mapping relationship from the internal ID of the vector data in the vector database to the user ID;

[0010] Step S3: performing a first operation on a second mapping table, wherein the second mapping table stores a second mapping relationship between a user ID and an internal ID, and the first operation at least includes:

[0011] Reconstructing the second mapping relationship between the first user ID corresponding to the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before updating;

[0012] The second mapping relationships corresponding to the multiple specified user IDs in the second mapping table are deleted.

[0013] Optionally, the first operation further includes at least one of the following:

[0014] The second mapping relationship corresponding to the second user ID corresponding to the internal ID before the upper boundary ID in the first mapping table before the update remains unchanged;

[0015] The internal ID in the second mapping relationship corresponding to the third user ID corresponding to the internal ID after the lower boundary ID in the first mapping table before updating is subtracted by M, where M is the number of the multiple specified user IDs.

[0016] Optionally, the step S1 further includes:

[0017] Step S11: querying the second mapping table for a plurality of designated internal IDs corresponding to the plurality of designated user IDs;

[0018] Step S12: deleting the vector data corresponding to the multiple specified internal IDs in the vector database.

[0019] Optionally, the step S1 further includes:

[0020] Step S13: determining whether the number of vector data deleted from the vector database is zero;

[0021] Step S14: If the number of vector data deleted from the vector database is not zero, continue to execute the steps S2 and S3; if the number of vector data deleted from the vector database is zero, do not execute the steps S2 and S3.

[0022] Optionally, step S2 includes:

[0023] Step S21: allocating a first memory according to the number of specified user IDs in the vector data deletion instruction;

[0024] Step S22: deleting the designated user ID in the first mapping table to obtain an updated first mapping table, adding the deleted designated user ID to the first memory, and recording the upper boundary ID and the lower boundary ID of the multiple designated internal IDs corresponding to the multiple designated user IDs before being deleted;

[0025] Deleting the second mapping relationships corresponding to the multiple specified user IDs in the second mapping table in step S3 includes:

[0026] Step S31: according to the multiple designated user IDs stored in the first memory, deleting the second mapping relationships corresponding to the multiple designated user IDs in the second mapping table.

[0027] Optionally, the step S3 reconstructs the second mapping relationship between the first user ID and the internal ID corresponding to the upper boundary ID and the lower boundary ID in the first mapping table before the update, including: modifying the internal ID in the second mapping relationship between the first user ID and the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before the update according to the first mapping relationship in the first mapping table after the update.

[0028] In a second aspect, an embodiment of the present invention provides a device for optimizing computing resources of an intelligent computing center by deleting a data structure, including:

[0029] An acquisition module, configured to, when receiving a vector data deletion instruction sent by a client, acquire multiple specified user IDs of multiple vector data to be deleted carried in the vector data deletion instruction;

[0030] A first updating module is used to delete the multiple specified user IDs in the first mapping table, obtain an updated first mapping table, and record the upper boundary IDs and lower boundary IDs of the multiple specified internal IDs corresponding to the multiple specified user IDs before being deleted; the first mapping table stores a first mapping relationship from the internal ID of the vector data in the vector database to the user ID;

[0031] The second updating module is configured to perform a first operation on a second mapping table, wherein the second mapping table stores a second mapping relationship between a user ID and an internal ID, and the first operation at least includes:

[0032] Reconstructing the second mapping relationship between the first user ID corresponding to the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before updating;

[0033] The second mapping relationships corresponding to the multiple specified user IDs in the second mapping table are deleted.

[0034] In a third aspect, an embodiment of the present invention provides a server comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the method for optimizing the computing power resources of an intelligent computing center by deleting data structures as described in the first aspect above.

[0035] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for optimizing the computing power resources of an intelligent computing center by deleting data structures as described in the first aspect above are implemented.

[0036] In a fifth aspect, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the steps of the method for optimizing the computing power and computing resources of an intelligent computing center by deleting data structures as described in the first aspect above.

[0037] In an embodiment of the present invention, in the process of deleting data in a mapping table used to record an internal ID and a user ID for vector data to update the data structure of the mapping table, the upper boundary ID and the lower boundary ID of the multiple specified internal IDs corresponding to the first mapping table can be recorded before the multiple specified user IDs are deleted. For the second mapping table, only the second mapping relationship corresponding to the first user ID corresponding to the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before the update can be rebuilt, that is, there is no need to rebuild all the second mapping relationships in the second mapping table, and the present invention does not require new memory resources when rebuilding the second mapping table, thereby effectively reducing the processing time of the vector database and optimizing the computing power resources of the intelligent computing center. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0039] Figure 1 is a schematic diagram of a first mapping table in an embodiment of the present invention;

[0040] Figure 2 is a schematic diagram of a second mapping table in an embodiment of the present invention;

[0041] Figure 3This is a schematic diagram of one of the updating methods of the first mapping table in an embodiment of the present invention;

[0042] Figure 4 A flowchart of a method for optimizing computing power and computing resources of an intelligent computing center by deleting a data structure according to an embodiment of the present invention;

[0043] Figure 5 This is a second schematic diagram of an updating method of the first mapping table in an embodiment of the present invention;

[0044] Figure 6 is a schematic diagram of an updating method of the second mapping table in an embodiment of the present invention;

[0045] Figure 7 is a schematic diagram of an updated second mapping table in an embodiment of the present invention;

[0046] Figure 8 A schematic diagram of the structure of an apparatus for optimizing computing power and computing resources of an intelligent computing center by deleting data structures according to an embodiment of the present invention;

[0047] Fig. 9 Schematic diagram of the structure of a server according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] First, the technical terms involved in the present invention are briefly explained below.

[0050] The "computing power" mentioned in the present invention refers to the ability of computer equipment or computing / data centers to process information. It is the ability of computer hardware and software to work together to execute certain computing requirements. It is the computing power to achieve specified result output by processing information data. It is a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity. It mainly provides services to the society through computing power infrastructure.

[0051] The "computing power" (Computational Power, CP) described in the present invention is the ability of a data center server to process data and output results. It is a comprehensive indicator to measure the computing power of a data center, including general computing power, super computing power and intelligent computing power. The commonly used unit of measurement is the number of floating point operations performed per second (FLOPS: Floating Point Operations Per Second, 1EFLOPS = 10^18FLOPS). The larger the value, the stronger the comprehensive computing power. According to calculations, 1EFLOPS is approximately the computing power output of 5 Tianhe-2A or 500,000 mainstream server CPUs or 2 million mainstream notebooks. The calculation formula is: CP = CP 通用 +CP 智能 +CP 超级 .

[0052] The "Network Power" (NP) described in the present invention is a manifestation of the data transmission capability of computing facilities, including comprehensive capabilities such as network architecture, network bandwidth, transmission latency, intelligent management and scheduling, etc. The carrying capacity involves network transmission within and between data centers, and is a comprehensive indicator for measuring network transmission scheduling capabilities. In the embodiment of the present invention, the carrying capacity uses the memory bandwidth.

[0053] The "Storage Power" (SP) described in the present invention is the comprehensive ability of a data center in terms of data storage capacity, performance, safety and reliability, and green and low-carbon. It is a comprehensive indicator for measuring the data storage capacity of a data center, including external storage devices such as storage arrays and server built-in storage devices. The commonly used unit of measurement for storage capacity is exabyte (EB, 1EB = 2^60bytes), and the commonly used unit of measurement for performance is the number of reads and writes per second per unit capacity (IOPS / TB, Input / Output Operations Per Second / TB). The disaster recovery ratio is an important manifestation of safety and reliability.

[0054] The "computing power infrastructure" described in the present invention is a new type of information infrastructure that integrates information computing power, network carrying capacity, and data storage capacity. It can realize centralized computing, storage, transmission and application of information, and presents characteristics such as diversity and ubiquity, intelligence and agility, safety and reliability, green and low-carbon. It is of great significance to promote industrial transformation and upgrading, enable my country's scientific and technological innovation, meet people's better life and realize high-efficiency social governance.

[0055] The “computing power” mentioned in the present invention includes general computing power, intelligent computing power and super computing power.

[0056] The “general computing power” mentioned in the present invention refers to the computing power provided by servers based on CPU (Central Processing Unit) chips, which is used to support basic general computing such as cloud computing and edge computing.

[0057] The "intelligent computing power" described in the present invention is a computing platform for large-scale deployment of special chips such as GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit) for various innovative applications of artificial intelligence, such as natural language processing and machine vision.

[0058] The "super computing power" mentioned in the present invention is mainly the computing power provided by high-performance computing clusters such as supercomputers. It utilizes the centralized computing resources of multiple computer systems working in parallel and uses a dedicated operating system to handle extremely complex or data-intensive problems. It is mainly used for calculations in cutting-edge scientific fields, such as planetary simulation, drug molecule design, and gene analysis.

[0059] The "intelligent computing center" described in the present invention refers to a facility that provides the required computing power, data and algorithms for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning scenarios) by using large-scale heterogeneous computing power resources, including general computing power (CPU: Central Processing Unit) and intelligent computing power (GPU: Graphics Processing Unit, FPGA: Field Programmable Gate Array, ASIC: Application Specific Integrated Circuit, etc.). The intelligent computing center covers facilities, hardware, and software, and can provide full-stack capabilities from bottom-level computing power to top-level application enablement.

[0060] The "computing resources" mentioned in the present invention refer to the technologies and facilities with information calculation, transmission, storage and application capabilities required for the development of the digital society, including but not limited to computing resources such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), network resources such as switches and routers, storage resources such as storage arrays and distributed storage, security resources such as firewalls and intrusion detection systems, as well as supporting and guarantee resources such as wind, fire, water and electricity.

[0061] The “computing power resources” mentioned in the present invention refer to GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), etc.

[0062] The mapping table described in the present invention is used to store the mapping relationship between the user ID and the internal ID of the vector data in the vector database, and includes a first mapping table and a second mapping table. The first mapping table is used to store the first mapping relationship from the internal ID of the vector data in the vector database to the user ID, and the second mapping table is used to store the second mapping relationship from the user ID of the vector data in the vector database to the internal ID.

[0063] The user ID described in the present invention is the ID of the vector data defined by the user, and the internal ID refers to the ID of the vector data assigned by the vector database and used internally. Each vector data in the vector database has a unique user ID and an internal ID. The internal ID can be changed, but the user ID is unique.

[0064] Please refer to Figure 1 , Figure 1 This is a schematic diagram of a first mapping table of an embodiment of the present invention. The first mapping table is stored in an array, wherein the internal ID (such as 0, 1, 2...) is the subscript of the array, and the user ID (such as 10, 20, 30...) is the element of the array. Please refer to Figure 2 , Figure 2 This is a schematic diagram of a second mapping table according to an embodiment of the present invention. The second mapping table is stored in a key-value pair format, wherein the user ID is used as the key and the internal ID is used as the value.

[0065] When deleting vector data in the vector database, it is also necessary to delete the corresponding IDs in the first mapping table and the second mapping table. For example, when it is necessary to delete two vector data of user ID=30 and 50, the updated first mapping table is as follows: Figure 3As shown, it can be seen that the first mapping relationship between the user ID and the internal ID in the updated first mapping table has also changed. In the related art, since the first mapping relationship between the user ID and the internal ID has changed, the second mapping relationship in the second mapping table needs to be rebuilt. When rebuilding the second mapping relationship, all the second mapping relationships need to be deleted first and then rebuilt. During the reconstruction process, some memory is usually allocated to the second mapping table first, and the rebuilt second mapping relationship is inserted into the allocated memory. When the current memory size cannot meet the storage requirements of the second mapping relationship, the memory needs to be expanded. The current practice is to reallocate a larger memory for the second mapping table, transfer the second mapping relationship in the previously allocated memory to the newly allocated larger memory, and then release the previously allocated memory. This transfer of memory data may be performed multiple times, resulting in a long time consumption and a large consumption of computing power resources of the intelligent computing center.

[0066] To solve the above problems, please refer to Figure 4 , Figure 4 The present invention is a flowchart of a method for optimizing computing power and computing resources of an intelligent computing center by deleting a data structure according to an embodiment of the present invention. The method for optimizing computing power and computing resources of an intelligent computing center by deleting a data structure includes:

[0067] Step S1: when receiving a vector data deletion instruction sent by a client, obtaining multiple designated user IDs of multiple vector data to be deleted carried in the vector data deletion instruction;

[0068] Among them, one specified vector data corresponds to one specified user ID.

[0069] In the embodiment of the present invention, the multiple specified user IDs of the multiple vector data to be deleted can be listed one by one in a list manner. For example, the multiple specified user IDs of the multiple vector data to be deleted are: user ID = 30, user ID = 50, ... Of course, other lists can also be used, for example: user ID = 30, user ID = 50-100, ... The present invention is not limited to this.

[0070] Step S2: deleting the multiple specified user IDs in the first mapping table, obtaining an updated first mapping table, and recording the upper boundary IDs and lower boundary IDs of the multiple specified internal IDs corresponding to the multiple specified user IDs before being deleted; the first mapping table stores a first mapping relationship from the internal ID of the vector data in the vector database to the user ID;

[0071] The vector database described in the present invention may be Faiss, but other vector databases are certainly not excluded.

[0072] The upper boundary ID of the present invention refers to the minimum value among the multiple specified internal IDs, and the lower boundary ID refers to the maximum value among the multiple specified internal IDs. For example, if the multiple specified internal IDs are 2, 4, and 5, the upper boundary ID is 2 and the lower boundary ID is 5. The first mapping table before updating (before deleting the multiple specified user IDs) and the first mapping table after updating in the embodiment of the present invention can be referred to. Figure 3 It can be seen that the first mapping relationship between the user ID and the internal ID has changed in the updated first mapping table compared with the first mapping table before the update. For example, the user ID = 40, the corresponding internal ID in the first mapping table before the update is 3, and the corresponding internal ID in the first mapping table after the update becomes 2.

[0073] In an embodiment of the present invention, the method for deleting the multiple specified user IDs in the first mapping table can be: querying the multiple specified internal IDs corresponding to the multiple specified user IDs from the second mapping table, and deleting the multiple specified user IDs from the first mapping table based on the multiple specified IDs.

[0074] Step S3: performing a first operation on a second mapping table, wherein the second mapping table stores a second mapping relationship between a user ID and an internal ID, and the first operation at least includes:

[0075] Reconstructing the second mapping relationship between the first user ID corresponding to the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before updating;

[0076] The second mapping relationships corresponding to the multiple specified user IDs in the second mapping table are deleted.

[0077] In an embodiment of the present invention, in the process of deleting data in a mapping table used to record an internal ID and a user ID for vector data to update the data structure of the mapping table, the upper boundary ID and the lower boundary ID of the multiple specified internal IDs corresponding to the first mapping table can be recorded before the multiple specified user IDs are deleted. For the second mapping table, only the second mapping relationship corresponding to the first user ID corresponding to the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before the update can be rebuilt, that is, there is no need to rebuild all the second mapping relationships in the second mapping table, and the present invention does not require new memory resources when rebuilding the second mapping table, thereby effectively reducing the processing time of the vector database and optimizing the computing power resources of the intelligent computing center.

[0078] In some embodiments, optionally, the step S3 of reconstructing the second mapping relationship between the first user ID and the internal ID corresponding to the upper boundary ID and the lower boundary ID in the first mapping table before updating includes:

[0079] According to the first mapping relationship in the updated first mapping table, the internal ID in the second mapping relationship corresponding to the first user ID corresponding to the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before updating is modified.

[0080] It should be noted that since the first user ID corresponding to the internal ID between the upper boundary ID and the lower boundary ID also includes the user ID that has been deleted in the first mapping table, when modifying the internal ID in the second mapping relationship corresponding to the first user ID, there is no need to modify the value of the deleted user ID, thereby further saving efficiency.

[0081] In some embodiments, optionally, the first operation further includes at least one of the following:

[0082] The second mapping relationship corresponding to the second user ID corresponding to the internal ID before the upper boundary ID in the first mapping table before the update remains unchanged;

[0083] The internal ID in the second mapping relationship corresponding to the third user ID corresponding to the internal ID after the lower boundary ID in the first mapping table before updating is subtracted by M, where M is the number of the multiple specified user IDs.

[0084] Assuming that the upper boundary ID is the first internal ID in the updated first mapping table, the first operation does not include: maintaining the second mapping relationship corresponding to the second user ID corresponding to the internal ID before the upper boundary ID in the first mapping table before the update unchanged.

[0085] Assuming that the lower boundary ID is the last internal ID in the updated first mapping table, the first operation does not include: subtracting M from the internal ID in the second mapping relationship corresponding to the third user ID corresponding to the internal ID after the lower boundary ID in the first mapping table before update.

[0086] Assume that the upper boundary ID is the first internal ID in the updated first mapping table, and the lower boundary ID is the last internal ID in the updated first mapping table. That is to say, all second mapping relationships may need to be rebuilt. However, the reconstruction of the present invention is different from the reconstruction in the existing scheme. There is no need to reallocate memory, but only to modify the value in the second mapping table.

[0087] For example, see Figure 5, the second user ID before the upper boundary ID (also called the front segment internal ID) in the first mapping table before updating includes 10, 20. The first user ID between the upper boundary ID and the lower boundary ID (also called the middle segment internal ID) in the first mapping table before updating includes: 30, 40, 50. The third user ID after the lower boundary ID (also called the back segment internal ID) in the first mapping table before updating includes: 60, 70, 80. Different operations are performed for the second user ID corresponding to the front segment internal ID, the first user ID corresponding to the middle segment internal ID, and the third user ID corresponding to the back segment internal ID in the second mapping table. The second mapping relationship for the second user ID corresponding to the front segment internal ID remains unchanged, the second mapping relationship for the first user ID corresponding to the middle segment internal ID is rebuilt, and the internal ID in the second mapping relationship corresponding to the third user ID corresponding to the back segment internal ID is reduced by M. That is, only the second mapping relationship for the first user ID corresponding to the middle segment internal ID needs to be rebuilt, and all the second mapping relationships in the second mapping table do not need to be rebuilt. The reconstruction of the present invention is also different from the reconstruction in the existing solution. There is no need to reallocate memory, and only the value in the second mapping table needs to be modified. This can effectively reduce the processing time of the vector database and optimize the computing power resources of the intelligent computing center.

[0088] In some embodiments, optionally, the step S1 further includes:

[0089] Step S11: querying the second mapping table for a plurality of designated internal IDs corresponding to the plurality of designated user IDs;

[0090] Step S12: deleting the vector data corresponding to the multiple specified internal IDs in the vector database.

[0091] Since only internal IDs can be identified within the vector database, when deleting a vector in the vector database, it is necessary to query the second mapping table to obtain the designated internal ID corresponding to the designated user ID to be deleted, and delete the vector data based on the queried designated internal ID.

[0092] In some embodiments, optionally, the step S1 further includes:

[0093] Step S13: determining whether the number of vector data deleted from the vector database is zero;

[0094] Step S14: If the number of vector data deleted from the vector database is not zero, continue to execute the steps S2 and S3; if the number of vector data deleted from the vector database is zero, do not execute the steps S2 and S3.

[0095] In some cases, the vector data corresponding to the specified user ID to be deleted sent by the client may not exist in the vector database. Therefore, no vector data is deleted according to the vector data deletion instruction, that is, the number of deleted vector data is zero. At this time, there is no need to update the first mapping table and the second mapping table, that is, there is no need to execute steps S2 and S3, thereby avoiding invalid operations.

[0096] In some embodiments, optionally, step S2 includes:

[0097] Step S21: allocating a first memory according to the number of specified user IDs in the vector data deletion instruction;

[0098] Please refer to Figure 5 In the embodiment of the present invention, assuming that the number of specified user IDs in the vector data deletion instruction is 2, a first memory capable of storing 2 user IDs is allocated.

[0099] Step S22: deleting the designated user ID in the first mapping table to obtain an updated first mapping table, adding the deleted designated user ID to the first memory, and recording the upper boundary ID and the lower boundary ID of the multiple designated internal IDs corresponding to the multiple designated user IDs before being deleted;

[0100] Please refer to Figure 5 In the embodiment of the present invention, each time a specified user ID is deleted, it is added to the first memory, and the upper boundary ID and the lower boundary ID of the multiple specified internal IDs corresponding to the deleted multiple specified user IDs before being deleted are recorded. Please refer to Figure 5 , specify user ID = 30, the corresponding upper boundary ID before 50 is deleted is 2, and the lower boundary ID is 4.

[0101] Wherein, deleting the second mapping relationships corresponding to the multiple specified user IDs in the second mapping table in step S3 includes:

[0102] Step S31: according to the multiple designated user IDs stored in the first memory, deleting the second mapping relationships corresponding to the multiple designated user IDs in the second mapping table.

[0103] In an embodiment of the present invention, the second mapping relationship corresponding to the first user ID corresponding to the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before updating can be rebuilt first, and then the second mapping relationship corresponding to the multiple specified user IDs in the second mapping table can be deleted.

[0104] For example, see Figure 6, first obtain the first user ID (30, 40, 50) corresponding to the internal ID (i.e., the middle internal ID) between the upper boundary ID (e.g., 2) and the lower boundary ID (e.g., 4) in the first mapping table before the update, and then rebuild (i.e., modify the value) the second mapping relationship corresponding to the first user ID (30, 40, 50) in the second mapping table according to the first mapping relationship of the first user ID (30, 40, 50) in the updated first mapping table. Among them, user ID=30 and 50 are user IDs that have been deleted in the first mapping table. At this time, when rebuilding the second mapping relationship, there is no need to modify the value of user ID=30 and 50, thereby further improving efficiency. The internal ID corresponding to user ID=40 in the updated first mapping table is 2. At this time, the value corresponding to key=40 in the second mapping table is modified from 3 to 2. At the same time, it is also necessary to subtract M (e.g., subtract 2) from the internal ID in the second mapping relationship corresponding to the third user ID corresponding to the internal ID after the lower boundary ID in the first mapping table before the update. For example, the third user IDs corresponding to the internal IDs after the lower boundary ID include 60, 70, and 80, so it is necessary to reduce the value corresponding to key=60, 70, and 80 in the second mapping table by 2. Finally, according to the specified user ID stored in the first memory, delete the second mapping relationship corresponding to the specified user ID in the second mapping table, and delete Figure 6 The second mapping relationship corresponding to key=30,50 in . The second mapping table obtained after processing can be found in Figure 7 .

[0105] In some embodiments, optionally, the method further comprises:

[0106] Step S4: Check whether the designated user ID deleted from the first mapping table matches the size of the first memory. If not, generate an error message and return it to the client.

[0107] In some cases, there may be duplicate user IDs among the multiple designated user IDs sent by the client. Assume that the number of designated user IDs is 200, and there are 5 pairs of duplicate user IDs. When allocating the first memory, the first memory that can accommodate 200 user IDs is allocated, but in fact, the number of designated user IDs deleted from the first mapping table is 195. At this time, the designated user IDs deleted from the first mapping table do not match the size of the first memory, and an error message needs to be generated and fed back to the client.

[0108] Please refer to Figure 8 The embodiment of the present invention further provides a device 10 for optimizing computing resources of an intelligent computing center by deleting a data structure, comprising:

[0109] The acquisition module 11 is used for acquiring multiple designated user IDs of multiple vector data to be deleted carried in the vector data deletion instruction when receiving the vector data deletion instruction sent by the client;

[0110] The first updating module 12 is used to delete the multiple specified user IDs in the first mapping table, obtain an updated first mapping table, and record the upper boundary IDs and lower boundary IDs of the multiple specified internal IDs corresponding to the multiple specified user IDs before being deleted; the first mapping table stores a first mapping relationship from the internal ID of the vector data in the vector database to the user ID;

[0111] The second updating module 13 is configured to perform a first operation on a second mapping table, wherein the second mapping table stores a second mapping relationship between a user ID and an internal ID, and the first operation at least includes:

[0112] Reconstructing the second mapping relationship between the first user ID corresponding to the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before updating;

[0113] The second mapping relationships corresponding to the multiple specified user IDs in the second mapping table are deleted.

[0114] In an embodiment of the present invention, in the process of deleting data in a mapping table used to record an internal ID and a user ID for vector data to update the data structure of the mapping table, the upper boundary ID and the lower boundary ID of the multiple specified internal IDs corresponding to the first mapping table can be recorded before the multiple specified user IDs are deleted. For the second mapping table, only the second mapping relationship corresponding to the first user ID corresponding to the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before the update can be rebuilt, that is, there is no need to rebuild all the second mapping relationships in the second mapping table, and the present invention does not require new memory resources when rebuilding the second mapping table, thereby effectively reducing the processing time of the vector database and optimizing the computing power resources of the intelligent computing center.

[0115] In some embodiments, optionally, the first operation further includes at least one of the following:

[0116] The second mapping relationship corresponding to the second user ID corresponding to the internal ID before the upper boundary ID in the first mapping table before the update remains unchanged;

[0117] The internal ID in the second mapping relationship corresponding to the third user ID corresponding to the internal ID after the lower boundary ID in the first mapping table before updating is subtracted by M, where M is the number of the multiple specified user IDs.

[0118] In some embodiments, optionally, the acquisition module 11 includes:

[0119] A first query submodule, configured to query the second mapping table for a plurality of specified internal IDs corresponding to the plurality of specified user IDs;

[0120] The first deleting submodule is used to delete the vector data corresponding to the multiple specified internal IDs in the vector database.

[0121] In some embodiments, optionally, the acquisition module 11 further includes:

[0122] A judging submodule, used for judging whether the number of vector data deleted from the vector database is zero;

[0123] The execution submodule is used to trigger the first update module 12 and the second update module 13 to run if the number of vector data deleted from the vector database is not zero; if the number of vector data deleted from the vector database is zero, the first update module 12 and the second update module 13 are not triggered to run.

[0124] In some embodiments, optionally, the first updating module 12 includes:

[0125] An allocation submodule, used for allocating a first memory according to the number of specified user IDs in the vector data deletion instruction;

[0126] A second deletion submodule is used to delete the designated user ID in the first mapping table, obtain an updated first mapping table, add the deleted designated user ID to the first memory, and record the upper boundary ID and the lower boundary ID of the multiple designated internal IDs corresponding to the multiple designated user IDs before they are deleted;

[0127] The second updating module 13 includes:

[0128] The third deleting submodule is used to delete the second mapping relationships corresponding to the multiple designated user IDs in the second mapping table according to the multiple designated user IDs stored in the first memory.

[0129] In some embodiments, optionally, the second update module 13 is used to modify the internal ID in the second mapping relationship corresponding to the first user ID corresponding to the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before the update according to the first mapping relationship in the updated first mapping table.

[0130] In some embodiments, optionally, the device 10 for optimizing computing resources of the intelligent computing center by deleting data structures further includes:

[0131] The checking module is used to check whether the specified user ID deleted in the first mapping table matches the size of the first memory. If not, an error message is generated and returned to the client.

[0132] In some embodiments, optionally, the first mapping table is stored in an array format, wherein the internal ID is a subscript of the array and the user ID is an element of the array;

[0133] The second mapping table is stored in a key-value pair format, wherein the user ID is used as the key and the internal ID is used as the value.

[0134] Please refer to Fig. 9 The embodiment of the present invention further provides a server 20, including a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the computer program is executed by the processor 21, the various processes of the above-mentioned method embodiment for optimizing the computing power and computing resources of the intelligent computing center by deleting the data structure are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0135] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the above-mentioned method embodiment for optimizing the computing power and computing resources of an intelligent computing center by deleting a data structure can be implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0136] The present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above Figure 4 The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0137] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0138] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0139] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A method for optimizing computing power and computing resources of an intelligent computing center by deleting data structures, characterized in that: include: Step S1: when receiving a vector data deletion instruction sent by a client, obtaining multiple designated user IDs of multiple vector data to be deleted carried in the vector data deletion instruction; Step S2: deleting the multiple specified user IDs in the first mapping table, obtaining an updated first mapping table, and recording the upper boundary IDs and lower boundary IDs of the multiple specified internal IDs corresponding to the multiple specified user IDs before being deleted; the first mapping table stores a first mapping relationship from the internal ID of the vector data in the vector database to the user ID; Step S3: performing a first operation on a second mapping table, wherein the second mapping table stores a second mapping relationship between a user ID and an internal ID, and the first operation at least includes: Reconstructing the second mapping relationship between the first user ID and the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before updating; The second mapping relationships corresponding to the multiple specified user IDs in the second mapping table are deleted.

2. The method according to claim 1, characterized in that The first operation further includes at least one of the following: The second mapping relationship corresponding to the second user ID corresponding to the internal ID before the upper boundary ID in the first mapping table before the update remains unchanged; The internal ID in the second mapping relationship corresponding to the third user ID corresponding to the internal ID after the lower boundary ID in the first mapping table before updating is subtracted by M, where M is the number of the multiple specified user IDs.

3. The method according to claim 1, characterized in that The step S1 further comprises: Step S11: querying the second mapping table for a plurality of designated internal IDs corresponding to the plurality of designated user IDs; Step S12: deleting the vector data corresponding to the multiple specified internal IDs in the vector database.

4. The method according to claim 1, characterized in that The step S1 further comprises: Step S13: determining whether the number of vector data deleted from the vector database is zero; Step S14: If the number of vector data deleted from the vector database is not zero, continue to execute the steps S2 and S3; if the number of vector data deleted from the vector database is zero, do not execute the steps S2 and S3.

5. The method according to claim 1, characterized in that: The step S2 comprises: Step S21: allocating a first memory according to the number of specified user IDs in the vector data deletion instruction; Step S22: deleting the designated user ID in the first mapping table to obtain an updated first mapping table, adding the deleted designated user ID to the first memory, and recording the upper boundary ID and the lower boundary ID of the multiple designated internal IDs corresponding to the multiple designated user IDs before being deleted; Deleting the second mapping relationships corresponding to the multiple specified user IDs in the second mapping table in step S3 includes: Step S31: according to the multiple designated user IDs stored in the first memory, deleting the second mapping relationships corresponding to the multiple designated user IDs in the second mapping table.

6. The method according to claim 5, characterized in that The step S3 of reconstructing the second mapping relationship between the first user ID corresponding to the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before updating includes: According to the first mapping relationship in the updated first mapping table, the internal ID in the second mapping relationship corresponding to the first user ID corresponding to the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before updating is modified.

7. A device for optimizing computing power and computing resources of an intelligent computing center by deleting data structures, characterized in that: include: An acquisition module, configured to, when receiving a vector data deletion instruction sent by a client, acquire multiple specified user IDs of multiple vector data to be deleted carried in the vector data deletion instruction; A first updating module is used to delete the multiple specified user IDs in the first mapping table, obtain an updated first mapping table, and record the upper boundary IDs and lower boundary IDs of the multiple specified internal IDs corresponding to the multiple specified user IDs before being deleted; the first mapping table stores a first mapping relationship from the internal ID of the vector data in the vector database to the user ID; The second updating module is configured to perform a first operation on a second mapping table, wherein the second mapping table stores a second mapping relationship between a user ID and an internal ID, and the first operation at least includes: Reconstructing the second mapping relationship between the first user ID and the internal ID between the upper boundary ID and the lower boundary ID in the first mapping table before updating; The second mapping relationships corresponding to the multiple specified user IDs in the second mapping table are deleted.

8. A server, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of a method for optimizing computing power resources of an intelligent computing center by deleting data structures as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for optimizing the computing power and computing resources of an intelligent computing center by deleting data structures as described in any one of claims 1 to 6.

10. A computer program product, characterized in that It comprises computer instructions, which, when executed by a processor, implement the steps of the method for optimizing the computing power and computing resources of an intelligent computing center by deleting data structures as described in any one of claims 1 to 6.

Images