Method and device for supporting elastic expansion capability by computing power resource of intelligent computing center

Abstract

The invention provides a method and a device for supporting elastic scalability of computing power resources of an intelligent computing center, and the method comprises the following steps: S1, after a DiskANN service receives a task request, inserting the task request into a work queue corresponding to the type of the task request, configuring a plurality of work queues for the DiskANN service, different work queues correspond to different types of task requests; s2, the DiskANN service elastically adjusts the number of threads or the number of coroutines configured for the work queue according to the number of the task requests in the work queue; and S3, the DiskANN service obtains the task request from the work queue and processes the task request based on a thread or a coroutine configured for the work queue. The utilization rate of computing power resources of the intelligent computing center can be effectively improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of computing power infrastructure, and in particular, to a method and device for supporting elastic scaling capabilities of computing power resources in an intelligent computing center. Background Art

[0002] With the development of artificial intelligence technology and computing power technology, the concept of intelligent computing center has emerged. "Intelligent computing center" refers to the use of large-scale heterogeneous computing power resources, including general computing power and intelligent computing power, mainly for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning and other scenarios) to provide the required computing power, data and algorithms. Intelligent computing center covers facilities, hardware, software, and can provide full-stack capabilities from bottom-level computing power to top-level application enablement.

[0003] The Intelligent Computing Center supports vector retrieval based on DiskANN. DiskANN is a vector retrieval engine based on distributed storage, which can store and retrieve billions of vector data on a single computer. Compared with traditional vector retrieval algorithms, DiskANN has higher storage efficiency and faster retrieval speed.

[0004] The overall process of vector retrieval based on DiskANN mainly includes the following tasks: creating a DiskANN object (referred to as Create), importing vector data to the DiskANN object (referred to as ImportData), importing vector data into the DiskANN service (referred to as Push Data), building a graph index for the imported vector data (referred to as Build), loading the graph index into memory (referred to as Load), querying vector data based on the graph index (referred to as Search), closing the DiskANN object (referred to as Close), and destroying the DiskANN object (referred to as Destroy), etc. In the existing solution, the DiskANN service receives task requests (such as Push Data requests, Build requests, etc.) sent by the client, inserts all received task requests into a work queue, and processes the task requests stored in the work queue in sequence. Under this processing method, the computing resources of the intelligent computing center are difficult to be effectively utilized. Summary of the invention

[0005] The embodiment of the present invention provides a method and device for supporting elastic scaling capabilities of computing resources of an intelligent computing center, which is used to solve the problem that in the existing solution, the DiskANN service inserts all received task requests into a work queue and processes the task requests stored in the work queue in sequence. Under this processing method, the computing resources of the intelligent computing center are difficult to be effectively utilized.

[0006] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0007] In a first aspect, an embodiment of the present invention provides a method for supporting elastic scaling capabilities of computing resources in an intelligent computing center, including:

[0008] Step S1: After receiving a task request, the DiskANN service inserts the task request into a work queue corresponding to the type of the task request, wherein a plurality of work queues are configured for the DiskANN service, and different work queues correspond to different types of task requests;

[0009] Step S2: the DiskANN service flexibly adjusts the number of threads or coroutines configured for the work queue according to the number of task requests in the work queue;

[0010] Step S3: The DiskANN service obtains and processes the task request from the work queue based on the thread or coroutine configured for the work queue.

[0011] Optionally, step S2 includes:

[0012] Step S21: the DiskANN service determines whether the number of task requests in the work queue matches the number of threads or coroutines currently configured for the work queue;

[0013] Step S22: When the number of task requests in the work queue does not match the number of threads or coroutines currently configured for the work queue, the DiskANN service flexibly adjusts the number of threads or coroutines configured for the work queue.

[0014] Optionally, the mismatch between the number of task requests in the work queue and the number of threads or coroutines currently configured for the work queue includes: the number of task requests in the work queue is greater than or equal to a first preset threshold, and the number of threads or coroutines currently configured for the work queue is less than or equal to a second preset threshold; or, the number of task requests in the work queue is less than or equal to a third preset threshold, and the number of threads or coroutines currently configured for the work queue is greater than or equal to a fourth preset threshold;

[0015] The step S22 comprises:

[0016] Step S221: when the number of task requests in the work queue of the DiskANN service is greater than or equal to a first preset threshold, and the number of threads or coroutines currently configured for the work queue is less than or equal to a second preset threshold, increasing the number of threads or coroutines configured for the work queue;

[0017] Step S222: When the number of task requests in the work queue of the DiskANN service is less than or equal to a third preset threshold, and the number of threads or coroutines currently configured for the work queue is greater than or equal to a fourth preset threshold and continues for a preset time, the number of threads or coroutines configured for the work queue is reduced.

[0018] Optionally, before step S1, the step further includes:

[0019] Step S01: the DiskANN service obtains a configuration file, wherein the configuration file includes: different work queues configured for different types of task requests, and at least one of the following information configured for each work queue: a minimum number of threads, a maximum number of threads, a minimum number of coroutines, a maximum number of coroutines, and a maximum number of task requests for the work queue;

[0020] Step S02: the DiskANN service is initialized and configured according to the configuration file, and the initialization configuration includes: configuring different work queues for different types of task requests, and pre-configuring a preset number of threads or coroutines for different work queues.

[0021] Optionally, the step S02 includes:

[0022] Step S021: the DiskANN service determines whether to configure a thread or a coroutine for each work queue according to whether the task request corresponding to each work queue is an IO-intensive task or a CPU-intensive task.

[0023] Optionally, the task request includes at least one of the following:

[0024] Create a task request for the DiskANN object;

[0025] Task request to import vector data to DiskANN object;

[0026] Request the DiskANN object to perform a task request for the build operation;

[0027] Request the DiskANN object to perform a task request for a load operation;

[0028] When the DiskANN object has completed the build operation at the next startup, a task request is made to request the DiskANN object to perform the load operation;

[0029] Request the DiskANN object to perform a search operation task request;

[0030] Other task requests, the other task requests include at least one of the following: a task request requesting the DiskANN object to perform a reset operation to restore the state of the created object, a task request requesting the DiskANN object to perform a close operation, a task request requesting the DiskANN object to perform a destroy operation, a task request requesting the DiskANN object to obtain the current state, a task request requesting the DiskANN object to obtain the current number of vector data, a task request requesting the DiskANN object to be set to a data-free state, a task request requesting to export the internal parameters of a specified DiskANN object, and a task request requesting to export the internal parameters of all DiskANN objects.

[0031] In a second aspect, an embodiment of the present invention provides a device for supporting elastic scaling capabilities of computing resources in an intelligent computing center, including:

[0032] A storage module, configured to, after receiving a task request, insert the task request into a work queue corresponding to the type of the task request, wherein multiple work queues are configured for the DiskANN service, and different work queues correspond to different types of task requests;

[0033] An elastic adjustment module, used for elastically adjusting the number of threads or coroutines configured for the work queue according to the number of task requests in the work queue;

[0034] A processing module is used to obtain and process the task request from the work queue based on the thread or coroutine configured for the work queue.

[0035] In a third aspect, an embodiment of the present invention provides a server comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the method for supporting elastic scaling capabilities of computing resources of an intelligent computing center as described in the first aspect above are implemented.

[0036] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for supporting elastic scaling capabilities of computing resources of an intelligent computing center as described in the first aspect above are implemented.

[0037] In a fifth aspect, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the steps of the method for supporting elastic scaling capabilities of computing resources of an intelligent computing center as described in the first aspect above.

[0038] In an embodiment of the present invention, multiple work queues are configured for the DiskANN service, and different work queues correspond to different types of task requests. Therefore, after receiving a task request, the DiskANN service can insert the task request into the corresponding work queue according to the type of the task request. Putting the same type of task requests in one work queue is conducive to memory cache reuse and improves the running speed. At the same time, using different work queues for different task requests can globally control the use of computing resources and achieve the effect of current limiting. In addition, the number of threads or coroutines can be flexibly allocated to the work queue according to the number of task requests in the work queue, thereby effectively improving the utilization rate of computing resources of the intelligent computing center. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0040] Figure 1 This is a schematic diagram of the overall process of vector data query based on DiskANN;

[0041] Figure 2 A flowchart of a method for supporting elastic scaling capabilities of computing resources in an intelligent computing center according to an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of a work queue according to an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of a work queue and corresponding threads according to an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of a work queue and a corresponding coroutine according to an embodiment of the present invention;

[0045] Figure 6 A schematic diagram of the structure of a device for supporting elastic scaling capabilities of computing resources in an intelligent computing center according to an embodiment of the present invention;

[0046] Figure 7 Schematic diagram of the structure of a server according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] First, the technical terms involved in the present invention are briefly explained below.

[0049] The "computing power" mentioned in the present invention is the ability of computer equipment or computing / data center to process information. It is the ability of computer hardware and software to work together to execute certain computing requirements. It is the computing power to achieve target result output by processing information data. It is a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity. It mainly provides services to the society through computing power infrastructure.

[0050] The "computing power" (Computational Power, CP) described in the present invention is the ability of a data center server to process data and output results. It is a comprehensive indicator to measure the computing power of a data center, including general computing power, super computing power and intelligent computing power. The commonly used unit of measurement is the number of floating point operations performed per second (FLOPS: Floating Point Operations Per Second, 1EFLOPS = 10^18FLOPS). The larger the value, the stronger the comprehensive computing power. According to calculations, 1EFLOPS is approximately the computing power output of 5 Tianhe-2A or 500,000 mainstream server CPUs or 2 million mainstream notebooks. The calculation formula is: CP = CP 通用 +CP 智能 +CP 超级 .

[0051] The "Network Power" (NP) described in the present invention is a manifestation of the data transmission capability of computing facilities, including comprehensive capabilities such as network architecture, network bandwidth, transmission latency, intelligent management and scheduling, etc. The carrying capacity involves network transmission within and between data centers, and is a comprehensive indicator for measuring network transmission scheduling capabilities. In the embodiment of the present invention, the carrying capacity uses the memory bandwidth.

[0052] The "Storage Power" (SP) described in the present invention is the comprehensive ability of a data center in terms of data storage capacity, performance, safety and reliability, and green and low-carbon. It is a comprehensive indicator for measuring the data storage capacity of a data center, including external storage devices such as storage arrays and server built-in storage devices. The commonly used unit of measurement for storage capacity is exabyte (EB, 1EB = 2^60bytes), and the commonly used unit of measurement for performance is the number of reads and writes per second per unit capacity (IOPS / TB, Input / Output Operations Per Second / TB). The disaster recovery ratio is an important manifestation of safety and reliability.

[0053] The "computing power infrastructure" described in the present invention is a new type of information infrastructure that integrates information computing power, network carrying capacity, and data storage capacity. It can realize centralized computing, storage, transmission and application of information, and presents characteristics such as diversity and ubiquity, intelligence and agility, safety and reliability, green and low-carbon. It is of great significance to promote industrial transformation and upgrading, enable my country's scientific and technological innovation, meet people's better life and realize high-efficiency social governance.

[0054] The “computing power” mentioned in the present invention includes general computing power, intelligent computing power and super computing power.

[0055] The “general computing power” mentioned in the present invention refers to the computing power provided by servers based on CPU (Central Processing Unit) chips, which is used to support basic general computing such as cloud computing and edge computing.

[0056] The "intelligent computing power" described in the present invention is a computing platform for large-scale deployment of special chips such as GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit) for various innovative applications of artificial intelligence, such as natural language processing and machine vision.

[0057] The "super computing power" mentioned in the present invention is mainly the computing power provided by high-performance computing clusters such as supercomputers. It utilizes the centralized computing resources of multiple computer systems working in parallel and uses a dedicated operating system to handle extremely complex or data-intensive problems. It is mainly used for calculations in cutting-edge scientific fields, such as planetary simulation, drug molecule design, and gene analysis.

[0058] The "intelligent computing center" described in the present invention refers to a facility that provides the required computing power, data and algorithms for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning scenarios) by using large-scale heterogeneous computing power resources, including general computing power (CPU: Central Processing Unit) and intelligent computing power (GPU: Graphics Processing Unit, FPGA: Field Programmable Gate Array, ASIC: Application Specific Integrated Circuit, etc.). The intelligent computing center covers facilities, hardware, and software, and can provide full-stack capabilities from bottom-level computing power to top-level application enablement.

[0059] The "computing resources" mentioned in the present invention refer to the technologies and facilities with information calculation, transmission, storage and application capabilities required for the development of the digital society, including but not limited to computing resources such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), network resources such as switches and routers, storage resources such as storage arrays and distributed storage, security resources such as firewalls and intrusion detection systems, as well as supporting and guarantee resources such as wind, fire, water and electricity.

[0060] The “elastic scalability” described in the embodiments of the present invention may also be referred to as automatic expansion and contraction, that is, the ability to automatically update workload resources to meet demand.

[0061] The "DiskANN" described in the present invention is a vector search engine based on distributed storage, which can store and retrieve billions of vector data on a single computer. Compared with traditional vector search algorithms, DiskANN has higher storage efficiency and faster search speed.

[0062] In the embodiment of the present invention, the overall process of vector data query based on DiskANN can be found in the attached Figure 1 The overall process mainly includes the following tasks: creating a DiskANN object (referred to as Create), importing vector data to the DiskANN object (referred to as ImportData), importing vector data into the DiskANN service (referred to as Push Data), building a graph index for the imported vector data (referred to as Build), loading the graph index into memory (referred to as Load), querying vector data based on the graph index (referred to as Search), closing the DiskANN object (referred to as Close) and destroying the DiskANN object (referred to as Destroy).

[0063] Among them, creating a DiskANN object means that the DiskANN service creates a DiskANN object according to the Create request of the client. When the DiskANN object is created, there is no data in it. It is in a dataless state and cannot provide any services to the outside. The Create request of the client is sent to the DiskANN service through the Index service. After the DiskANN service completes the creation of the DiskANN object, it notifies the client through the Index service.

[0064] In the task of importing vector data for the DiskANN object, the client sends an ImportData request containing vector data to the Index service, and the Index service temporarily stores the vector data in the ImportData request in the database of the Index service. The Index service notifies the client after the data import is completed.

[0065] After the data import task, the client can send a build request to the Index service. According to the build request, the Index service executes the Push Data task, that is, the Index service imports the vector data in the database of the Index service to the DiskANN service. The DiskANN service notifies the Index service after the vector data is stored on the disk to form a file.

[0066] The build task refers to the process by which the DiskANN service processes the vector data of the imported DiskANN object based on the build request sent by the Index service, obtains the graph index of the vector data, and saves the vector data and the graph index to the data file on the disk corresponding to the DiskANN service. After the DiskANN service completes the build task, it notifies the client through the Index service.

[0067] The load task refers to the client sending a load request to the DiskANN service through the Index service. The DiskANN service loads the graph index of the vector data of the DiskANN object (which may also include a small amount of vector data) into the memory based on the load request. After the load is completed, the DiskANN service notifies the client through the Index service.

[0068] The search task refers to the client sending a vector data query request to the DiskANN service through the Index service. The DiskANN service retrieves the vector data most similar to the vector data to be queried based on the graph index, and returns the retrieved most similar vector data to the client through the Index service.

[0069] The Close task refers to the client sending a DiskANN object close request to the DiskANN service through the Index service, and the DiskANN service closes the DiskANN object based on the close request. After the close is completed, the DiskANN service notifies the client through the Index service.

[0070] The Destroy task means that the client sends a destruction request of the DiskANN object to the DiskANN service through the Index service, and the DiskANN service destroys the DiskANN object based on the destruction request. After the destruction is completed, the DiskANN service notifies the client through the Index service.

[0071] The above process includes three execution entities: the client, the Index service and the DiskANN service. The purpose of using the Index service is to make DiskANN as lightweight as possible. The DiskANN service only handles the core business of DiskANN, and other businesses are outsourced to the Index service to ensure that the overall service is stable and reliable. Before executing the above process, the Index service and the DiskANN service need to make a handshake connection in advance to provide services for subsequent businesses. The above Index service and DiskANN service can be deployed on different physical machines or on the same physical machine, and the present invention does not limit this.

[0072] The "process" described in the embodiments of the present invention is an instance of a program running in a computer. Each process has its own independent memory space and system resources, and can have multiple threads. Processes are independent of each other, they cannot share data directly, and must exchange data through inter-process communication (IPC).

[0073] The "thread" described in the embodiments of the present invention refers to an independent execution path executed within a process. A process can contain multiple threads, each of which runs independently and has its own execution order and state.

[0074] The "coroutine" described in the embodiments of the present invention is a more lightweight thread that can switch between different execution points instead of relying on the thread scheduling of the operating system. The switching of coroutines is explicitly controlled by the programmer and can be switched when needed to improve concurrency performance.

[0075] The "CPU-bound" described in the embodiments of the present invention is also called computation-intensive, which refers to programs that require a large amount of CPU computing power, usually involving a large amount of data processing and computing operations, such as scientific computing, image processing, encryption and decryption, etc. These applications usually require a high-speed CPU and a large amount of memory to complete the task.

[0076] The "IO-bound" mentioned in the embodiments of the present invention refers to programs that require a large number of input and output operations, such as network servers, database systems, and file transfers. These applications usually require high-speed disks, networks, and other input and output devices to complete tasks, while the CPU has relatively less computing power.

[0077] Please refer to Figure 2 , Figure 2 The flowchart of the method for supporting elastic scalability of computing resources in an intelligent computing center according to an embodiment of the present invention is as follows. The method for supporting elastic scalability of computing resources in an intelligent computing center includes:

[0078] Step S1: After receiving a task request, the DiskANN service inserts the task request into a work queue corresponding to the type of the task request, wherein a plurality of work queues are configured for the DiskANN service, and different work queues correspond to different types of task requests;

[0079] Please refer to Figure 3 , Figure 3 A schematic diagram of a work queue according to an embodiment of the present invention, Figure 3 As can be seen, multiple work queues are configured for the DiskANN service, and different work queues correspond to different types of task requests. For example, a work queue is configured for each of the PushData task, the Build task, and the Load task. When the DiskANN service receives a Push Data task request, it inserts the Push Data task request into the work queue corresponding to the Push Data task request. When it receives a Build task request, it inserts the Build task request into the work queue corresponding to the Build task request. At the same time, different task requests in a work queue can correspond to different DiskANN objects.

[0080] Under this configuration, different types of task requests can be diverted and inserted into different work queues. Task requests in different work queues do not interfere with each other and can be processed in parallel. At the same time, compared with the existing solution of using only one work queue, the number of task requests in each work queue in the present invention is greatly reduced, and it is not easy to be blocked, which can effectively improve the utilization rate of computing resources in the intelligent computing center and improve the processing efficiency of DiskANN service. At the same time, placing task requests of the same type in one work queue is also conducive to memory cache reuse and improves the running speed.

[0081] Step S2: the DiskANN service flexibly adjusts the number of threads or coroutines configured for the work queue according to the number of task requests in the work queue;

[0082] Step S3: The DiskANN service obtains and processes the task request from the work queue based on the thread or coroutine configured for the work queue.

[0083] In the embodiment of the present invention, a work queue and its corresponding thread may be referred to as a work group, and all threads in the work group share the work queue.

[0084] Please refer to Figure 4 , Figure 4 In the illustrated embodiment, the work queue corresponds to four threads, each thread can obtain a task request from the work queue and process it, and multiple threads can be executed concurrently to achieve simultaneous tasks, thereby improving processing efficiency and effectively improving the utilization of computing resources of the intelligent computing center.

[0085] In an embodiment of the present invention, when a work queue corresponds to multiple threads, each thread can obtain a task request from the work queue. A thread only processes one task request at a time. When the processed task request is completed, a new task request is obtained from the work queue. If there is no task request in the work queue, the thread waits.

[0086] In the embodiment of the present invention, a work queue and its corresponding coroutine may be referred to as a work group, and all coroutines in the work group share the work queue.

[0087] Please refer to Figure 5 , Figure 5 In the illustrated embodiment, the work queue corresponds to N coroutines, each coroutine can obtain a task request from the work queue and process it, and multiple coroutines can be executed concurrently to achieve simultaneous tasks, thereby improving processing efficiency and effectively improving the utilization of computing resources of the intelligent computing center.

[0088] In the embodiment of the present invention, multiple coroutines may correspond to one thread.

[0089] In an embodiment of the present invention, multiple work queues are configured for the DiskANN service, and different work queues correspond to different types of task requests. Therefore, after receiving a task request, the DiskANN service can insert the task request into the corresponding work queue according to the type of the task request. Putting the same type of task requests in one work queue is conducive to memory cache reuse and improves the running speed. At the same time, using different work queues for different task requests can globally control the use of computing resources and achieve the effect of current limiting. In addition, the number of threads or coroutines can be flexibly allocated to the work queue according to the number of task requests in the work queue, thereby effectively improving the utilization rate of computing resources of the intelligent computing center.

[0090] In some embodiments, optionally, step S2 includes:

[0091] Step S21: the DiskANN service determines whether the number of task requests in the work queue matches the number of threads or coroutines currently configured for the work queue;

[0092] Step S22: When the number of task requests in the work queue does not match the number of threads or coroutines currently configured for the work queue, the DiskANN service flexibly adjusts the number of threads or coroutines configured for the work queue.

[0093] In some embodiments, optionally, the mismatch between the number of task requests in the work queue and the number of threads or coroutines currently configured for the work queue includes: the number of task requests in the work queue is greater than or equal to a first preset threshold, and the number of threads or coroutines currently configured for the work queue is less than or equal to a second preset threshold; or, the number of task requests in the work queue is less than or equal to a third preset threshold, and the number of threads or coroutines currently configured for the work queue is greater than or equal to a fourth preset threshold;

[0094] The step S22 comprises:

[0095] Step S221: when the number of task requests in the work queue of the DiskANN service is greater than or equal to a first preset threshold, and the number of threads or coroutines currently configured for the work queue is less than or equal to a second preset threshold, increasing the number of threads or coroutines configured for the work queue;

[0096] Step S222: When the number of task requests in the work queue of the DiskANN service is less than or equal to a third preset threshold, and the number of threads or coroutines currently configured for the work queue is greater than or equal to a fourth preset threshold and continues for a preset time, the number of threads or coroutines configured for the work queue is reduced.

[0097] For example, assuming that the number of task requests in the work queue is greater than or equal to a first preset threshold (for example, 100), and the number of threads currently configured for the work queue is less than or equal to a second preset threshold (for example, 4), then it means that the number of task requests in the work queue does not match the number of threads or coroutines currently configured for the work queue. At this time, the number of threads is too small to process task requests in a timely manner, and it is necessary to flexibly adjust the number of threads configured for the work queue, for example, adjusting the number of threads configured for the work queue to a first specified value (for example, 10) or a first specified value range (such as 10-32). Optionally, the adjusted number of threads is not greater than the maximum number of threads configured for the work queue.

[0098] Assuming that the number of task requests in the work queue is less than or equal to the third preset threshold (for example, 50), and the number of threads currently configured for the work queue is greater than or equal to the fourth preset threshold (for example, 10), and this situation lasts for a preset time (for example, 2 minutes), it means that the number of task requests in the work queue does not match the number of threads or coroutines currently configured for the work queue. At this time, the number of threads is too large and there is a waste of computing power resources. It is necessary to flexibly adjust the number of threads configured for the work queue, for example, adjust the number of threads configured for the work queue to a second specified value (for example, 4) or a specified value range (such as 4-6). Optionally, the adjusted number of threads is not less than the minimum number of threads configured for the work queue.

[0099] That is, the elastic adjustment in the embodiment of the present invention may include increasing the number of threads or decreasing the number of threads to meet the real-time task request quantity requirement.

[0100] It should be noted that the first preset value, the second preset value, the third preset threshold or the fourth preset threshold in the embodiment of the present invention can be one value or multiple values. For example, the first preset threshold includes two values, and the second preset threshold also includes two values. Assume that the first first preset threshold is 1000, and the corresponding second preset threshold is 10, and the second first preset threshold is 2000, and the corresponding second preset threshold is 20, that is, when the number of task requests in the work queue is greater than or equal to 1000, and the number of threads currently configured for the work queue is less than or equal to 10, it means that the number of task requests in the work queue does not match the number of threads currently configured for the work queue; when the number of task requests in the work queue is greater than or equal to 2000, and the number of threads currently configured for the work queue is less than or equal to 20, it means that the number of task requests in the work queue does not match the number of threads currently configured for the work queue.

[0101] In some embodiments, optionally, before step S1, the step further includes:

[0102] Step S01: the DiskANN service obtains a configuration file, wherein the configuration file includes: different work queues configured for different types of task requests, and at least one of the following information configured for each work queue: a minimum number of threads, a maximum number of threads, a minimum number of coroutines, a maximum number of coroutines, and a maximum number of task requests for the work queue;

[0103] Step S02: the DiskANN service is initialized and configured according to the configuration file, and the initialization configuration includes: configuring different work queues for different types of task requests, and pre-configuring a preset number of threads or coroutines for different work queues.

[0104] The minimum number of threads refers to the minimum number of threads allocated to the work queue.

[0105] The maximum number of threads is the maximum number of threads that can be allocated to a work queue.

[0106] The minimum number of coroutines is the minimum number of coroutines allocated to a work queue.

[0107] The maximum number of coroutines is the maximum number of coroutines allocated to a work queue.

[0108] The maximum number of task requests in a work queue refers to the maximum number of task requests that can be stored in the work queue.

[0109] In the embodiment of the present invention, optionally, the step S1 includes: when inserting the task request into the corresponding work queue, if the number of task requests in the work queue is equal to the maximum number of task requests, executing a waiting operation, or returning an error message.

[0110] In some embodiments, the configuration file may optionally include at least one of the following: memory occupied by the task request, and timeout of the task request. For example, the memory configured for the build task is 10.0 Gb, the memory configured for the search task is 1.0 Gb, and the timeout configured for the import task is 30 seconds.

[0111] In the embodiment of the present invention, optionally, step S02 includes:

[0112] Step S021: the DiskANN service determines whether to configure a thread or a coroutine for each work queue according to whether the task request corresponding to each work queue is an IO-intensive task or a CPU-intensive task.

[0113] In some embodiments, optionally, the task request includes at least one of the following:

[0114] Create a task request for the DiskANN object;

[0115] Task request to import (PushData) vector data to the DiskANN object;

[0116] Request the DiskANN object to perform a task request for a build operation;

[0117] Request the DiskANN object to perform a load operation task request;

[0118] When the DiskANN object has completed the build operation at the next startup, a task request is made to request the DiskANN object to perform the load operation;

[0119] Request the DiskANN object to perform a search operation task request;

[0120] Other task requests, the other task requests include at least one of the following: a task request requesting the DiskANN object to perform a reset (Reset) operation to restore the state of the created object, a task request requesting the DiskANN object to perform a close (Close) operation, a task request requesting the DiskANN object to perform a destroy (Destroy) operation, a task request requesting the DiskANN object to obtain the current state, a task request requesting the DiskANN object to obtain the current number of vector data, a task request requesting the DiskANN object to be set to a data-free state, a task request requesting to export the internal parameters of a specified DiskANN object, and a task request requesting to export the internal parameters of all DiskANN objects.

[0121] Please refer to Figure 6 The embodiment of the present invention further provides a device 10 for supporting elastic scalability of computing resources in an intelligent computing center, including:

[0122] The storage module 11 is used for inserting the task request into a work queue corresponding to the type of the task request after receiving the task request, wherein a plurality of work queues are configured for the DiskANN service, and different work queues correspond to different types of task requests;

[0123] An elastic adjustment module 12, used for elastically adjusting the number of threads or coroutines configured for the work queue according to the number of task requests in the work queue;

[0124] The processing module 13 is used to obtain and process the task request from the work queue based on the thread or coroutine configured for the work queue.

[0125] In an embodiment of the present invention, multiple work queues are configured for the DiskANN service, and different work queues correspond to different types of task requests. Therefore, after receiving a task request, the DiskANN service can insert the task request into the corresponding work queue according to the type of the task request. Putting the same type of task requests in one work queue is conducive to memory cache reuse and improves the running speed. At the same time, using different work queues for different task requests can globally control the use of computing resources and achieve the effect of current limiting. In addition, the number of threads or coroutines can be flexibly allocated to the work queue according to the number of task requests in the work queue, thereby effectively improving the utilization rate of computing resources of the intelligent computing center.

[0126] In some embodiments, optionally, the elasticity adjustment module 12 includes:

[0127] A judging submodule, used to judge whether the number of task requests in the work queue matches the number of threads or coroutines currently configured for the work queue;

[0128] The adjustment submodule is used to flexibly adjust the number of threads or coroutines configured for the work queue when the number of task requests in the work queue does not match the number of threads or coroutines currently configured for the work queue.

[0129] In some embodiments, optionally, the mismatch between the number of task requests in the work queue and the number of threads or coroutines currently configured for the work queue includes: the number of task requests in the work queue is greater than or equal to a first preset threshold, and the number of threads or coroutines currently configured for the work queue is less than or equal to a second preset threshold; or, the number of task requests in the work queue is less than or equal to a third preset threshold, and the number of threads or coroutines currently configured for the work queue is greater than or equal to a fourth preset threshold;

[0130] The adjustment submodule is used to increase the number of threads or coroutines configured for the work queue when the number of task requests in the work queue is greater than or equal to a first preset threshold and the number of threads or coroutines currently configured for the work queue is less than or equal to a second preset threshold; or to reduce the number of threads or coroutines configured for the work queue when the number of task requests in the work queue is less than or equal to a third preset threshold, the number of threads or coroutines currently configured for the work queue is greater than or equal to a fourth preset threshold and continues for a preset time.

[0131] In some embodiments, optionally, the device 10 for supporting elastic scalability of computing resources in the intelligent computing center further includes:

[0132] An acquisition module is used to acquire a configuration file, wherein the configuration file includes: different work queues configured for different types of task requests, and at least one of the following information configured for each work queue: a minimum number of threads, a maximum number of threads, a minimum number of coroutines, a maximum number of coroutines, and a maximum number of task requests for the work queue;

[0133] An initialization module is used to perform initialization configuration according to the configuration file, wherein the initialization configuration includes: configuring different work queues for different types of task requests, and pre-configuring a preset number of threads or coroutines for different work queues.

[0134] In some embodiments, optionally, the initialization module is used to determine whether to configure a thread or a coroutine for each work queue according to whether the task request corresponding to each work queue is an IO-intensive task or a CPU-intensive task.

[0135] In some embodiments, the configuration file may optionally include at least one of the following: memory occupied by the task request, and timeout of the task request. For example, the memory configured for the build task is 10.0 Gb, the memory configured for the search task is 1.0 Gb, and the timeout configured for the import task is 30 seconds.

[0136] In some embodiments, optionally, the task request includes at least one of the following:

[0137] Create a task request for the DiskANN object;

[0138] Task request to import vector data to DiskANN object;

[0139] Request the DiskANN object to perform a task request for the build operation;

[0140] Request the DiskANN object to perform a task request for a load operation;

[0141] When the DiskANN object has completed the build operation at the next startup, a task request is made to request the DiskANN object to perform the load operation;

[0142] Request the DiskANN object to perform a search operation task request;

[0143] Other task requests, the other task requests include at least one of the following: a task request requesting the DiskANN object to perform a reset operation to restore the state of the created object, a task request requesting the DiskANN object to perform a close operation, a task request requesting the DiskANN object to perform a destroy operation, a task request requesting the DiskANN object to obtain the current state, a task request requesting the DiskANN object to obtain the current number of vector data, a task request requesting the DiskANN object to be set to a data-free state, a task request requesting to export the internal parameters of a specified DiskANN object, and a task request requesting to export the internal parameters of all DiskANN objects.

[0144] Please refer to Figure 7 The embodiment of the present invention further provides a server 20, including a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the computer program is executed by the processor 21, each process of the embodiment of the method for supporting elastic scaling capabilities of computing resources of the above-mentioned intelligent computing center is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0145] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the method embodiment of the above-mentioned intelligent computing center computing power resource supporting elastic scalability is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0146] The present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above Figure 2 The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0147] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0148] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0149] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A method for supporting elastic scaling of computing resources in an intelligent computing center, characterized in that: include: Step S1: After receiving a task request, the DiskANN service inserts the task request into a work queue corresponding to the type of the task request, wherein a plurality of work queues are configured for the DiskANN service, and different work queues correspond to different types of task requests; Step S2: the DiskANN service flexibly adjusts the number of threads or coroutines configured for the work queue according to the number of task requests in the work queue; Step S3: The DiskANN service obtains and processes the task request from the work queue based on the thread or coroutine configured for the work queue.

2. The method according to claim 1, characterized in that The step S2 comprises: Step S21: the DiskANN service determines whether the number of task requests in the work queue matches the number of threads or coroutines currently configured for the work queue; Step S22: When the number of task requests in the work queue does not match the number of threads or coroutines currently configured for the work queue, the DiskANN service flexibly adjusts the number of threads or coroutines configured for the work queue.

3. The method according to claim 2, characterized in that The mismatch between the number of task requests in the work queue and the number of threads or coroutines currently configured for the work queue includes: the number of task requests in the work queue is greater than or equal to a first preset threshold, and the number of threads or coroutines currently configured for the work queue is less than or equal to a second preset threshold; or the number of task requests in the work queue is less than or equal to a third preset threshold, and the number of threads or coroutines currently configured for the work queue is greater than or equal to a fourth preset threshold; The step S22 comprises: Step S221: when the number of task requests in the work queue of the DiskANN service is greater than or equal to a first preset threshold, and the number of threads or coroutines currently configured for the work queue is less than or equal to a second preset threshold, increasing the number of threads or coroutines configured for the work queue; Step S222: When the number of task requests in the work queue of the DiskANN service is less than or equal to a third preset threshold, and the number of threads or coroutines currently configured for the work queue is greater than or equal to a fourth preset threshold and continues for a preset time, the number of threads or coroutines configured for the work queue is reduced.

4. The method according to claim 1, characterized in that Before step S1, the following steps are also included: Step S01: the DiskANN service obtains a configuration file, wherein the configuration file includes: different work queues configured for different types of task requests, and at least one of the following information configured for each work queue: a minimum number of threads, a maximum number of threads, a minimum number of coroutines, a maximum number of coroutines, and a maximum number of task requests for the work queue; Step S02: the DiskANN service is initialized and configured according to the configuration file, and the initialization configuration includes: configuring different work queues for different types of task requests, and pre-configuring a preset number of threads or coroutines for different work queues.

5. The method according to claim 4, characterized in that The step S02 comprises: Step S021: The DiskANN service determines whether to configure a thread or a coroutine for each work queue according to whether the task request corresponding to each work queue is an IO-intensive task or a CPU-intensive task.

6. The method according to any one of claims 1 to 5, characterized in that The task request includes at least one of the following: Create a task request for the DiskANN object; Task request to import vector data to DiskANN object; Request the DiskANN object to perform a task request for the build operation; Request the DiskANN object to perform a task request for a load operation; When the DiskANN object has completed the build operation at the next startup, a task request is made to request the DiskANN object to perform the load operation; Request the DiskANN object to perform a search operation task request; Other task requests, the other task requests include at least one of the following: a task request requesting the DiskANN object to perform a reset operation to restore the state of the created object, a task request requesting the DiskANN object to perform a close operation, a task request requesting the DiskANN object to perform a destroy operation, a task request requesting the DiskANN object to obtain the current state, a task request requesting the DiskANN object to obtain the current number of vector data, a task request requesting the DiskANN object to be set to a data-free state, a task request requesting to export the internal parameters of a specified DiskANN object, and a task request requesting to export the internal parameters of all DiskANN objects.

7. A device for supporting elastic scalability of computing resources in an intelligent computing center, characterized in that: include: A storage module, configured to, after receiving a task request, insert the task request into a work queue corresponding to the type of the task request, wherein multiple work queues are configured for the DiskANN service, and different work queues correspond to different types of task requests; An elastic adjustment module, used for elastically adjusting the number of threads or coroutines configured for the work queue according to the number of task requests in the work queue; A processing module is used to obtain and process the task request from the work queue based on the thread or coroutine configured for the work queue.

8. A server, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the method for supporting elastic scaling capabilities of computing resources of an intelligent computing center as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for supporting elastic scaling capabilities of computing resources of an intelligent computing center as described in any one of claims 1 to 6.

10. A computer program product, characterized in that It includes computer instructions, which, when executed by a processor, implement the steps of the method for supporting elastic scaling capabilities of computing resources of an intelligent computing center as described in any one of claims 1 to 6.

Images