Interface creation methods, apparatus, devices, storage media, and computer program products

Abstract

This disclosure provides an interface creation method, apparatus, device, storage medium, and computer program product, relating to artificial intelligence technologies such as machine learning, forward computation, clustering, and accelerated computing. The method includes: determining the actual length and actual concurrency of each request received within a preset statistical period; clustering the actual length of each request and determining a target length corresponding to each length cluster center; clustering the actual concurrency of each request and determining a target concurrency corresponding to each concurrency cluster center; and creating multiple target interfaces with a length parameter of any target length and a concurrency parameter of any target concurrency. The target interfaces are used to record the core computation order and parameter pointers during the forward computation of the neural network model. Applying this method allows the created target interfaces to better meet actual needs, improving the utilization rate of the created target interfaces and further enhancing the efficiency of forward computation.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, specifically to the fields of artificial intelligence technology such as machine learning, forward computation, clustering processing, and accelerated computing, and particularly to an interface creation method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Cuda_graph is an interface provided by NVIDIA that can be used to record the computation order and parameter pointers of the kernel during the forward computation of a neural network model.

[0003] Compared to the common processing method of continuously sending instructions from the CPU (Central Processing Unit) to the GPU (Graphics Processing Unit), using Cuda_graph for forwarding can avoid the time spent on kernel startup, thereby effectively improving processing efficiency.

[0004] Therefore, how to use a more suitable Cuda graph to improve processing efficiency in the forward computation stage of a neural network model is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] This disclosure provides an interface creation method, apparatus, electronic device, computer-readable storage medium, and computer program product.

[0006] In a first aspect, embodiments of this disclosure propose an interface creation method, comprising: determining the actual length and actual concurrency of each request received within a preset statistical period; performing clustering processing on the actual length of each request and determining a target length corresponding to each length cluster center; performing clustering processing on the actual concurrency of each request and determining a target concurrency corresponding to each concurrency cluster center; creating multiple target interfaces with a length parameter of any target length and a concurrency parameter of any target concurrency; wherein the target interface is used to record the core computation order and parameter pointers during the forward computation of the neural network model.

[0007] Secondly, embodiments of this disclosure propose an interface creation apparatus, comprising: an actual parameter determination unit configured to determine the actual length and actual concurrency of each request received within a preset statistical period; a target length determination unit configured to perform clustering processing on the actual length of each request and determine a target length corresponding to each length cluster center; a target concurrency determination unit configured to perform clustering processing on the actual concurrency of each request and determine a target concurrency corresponding to each concurrency cluster center; and a target interface creation unit configured to create multiple target interfaces with a length parameter of any target length and a concurrency parameter of any target concurrency; wherein the target interface is used to record the core calculation order and parameter pointers during the forward computation of the neural network model.

[0008] Thirdly, embodiments of this disclosure provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the interface creation method described in any implementation of the first aspect.

[0009] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions that enable a computer to implement the interface creation method described in any implementation of the first aspect.

[0010] Fifthly, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, can implement the steps of the interface creation method as described in any implementation of the first aspect.

[0011] The interface creation scheme provided in this disclosure performs clustering operations on the length and concurrency of each request received within a statistical period. Based on the clustering results, multiple scientific and reasonable target lengths and multiple target concurrency are determined. This makes the multiple target interfaces created based on this more in line with actual needs, rather than being created blindly. This improves the utilization rate of the created target interfaces and further enhances the processing efficiency of the forward computation stage of the neural network model.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0014] Figure 1 This is an exemplary system architecture to which this disclosure can be applied;

[0015] Figure 2 A flowchart illustrating an interface creation method provided in this embodiment of the disclosure;

[0016] Figure 3 A flowchart illustrating another interface creation method provided in this disclosure embodiment;

[0017] Figure 4 A flowchart illustrating a method for removing occasional long requests provided in embodiments of this disclosure;

[0018] Figure 5 A flowchart illustrating a method for processing new requests using a target interface, as provided in this embodiment of the disclosure;

[0019] Figure 6 A structural block diagram of an interface creation apparatus provided in an embodiment of this disclosure;

[0020] Figure 7 This is a schematic diagram of the structure of an electronic device suitable for executing an interface creation method, provided as an embodiment of the present disclosure. Detailed Implementation

[0021] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding; these should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0022] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0023] Figure 1 An exemplary system architecture 100 is shown, which can be used to create methods, apparatuses, electronic devices and computer-readable storage media using the interfaces disclosed herein.

[0024] like Figure 1As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0025] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various applications for enabling information communication between the terminal devices 101, 102, and 103 and server 105 can be installed. These applications include interface creation applications, model training applications, and instant messaging applications.

[0026] Terminal devices 101, 102, and 103 and server 105 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with displays, including but not limited to smartphones, tablets, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices, and can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here. When server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When server 105 is software, it can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here.

[0027] Server 105 can provide various services through its built-in applications. Taking an interface creation application that can provide model acceleration training services as an example, when server 105 runs this interface creation application, it can achieve the following effects: First, it receives various requests from terminal devices 101, 102, and 103 through network 104; then, it determines the actual length and actual concurrency of each request received within a preset statistical period; next, it performs clustering processing on the actual length of each request and determines the target length corresponding to each length cluster center; next, it performs clustering processing on the actual concurrency of each request and determines the target concurrency corresponding to each concurrency cluster center; finally, it creates multiple target interfaces with a length parameter of any target length and a concurrency parameter of any target concurrency. These target interfaces are used to record the core calculation order and parameter pointers during the forward computation of the neural network model.

[0028] Furthermore, the server 105 can also use the multiple target interfaces created to accelerate the forward computation stage of the neural network model.

[0029] Because clustering operations require significant computing resources and power, the interface creation methods provided in the subsequent embodiments of this disclosure are generally executed by a server 105 with strong computing power and abundant computing resources. Correspondingly, the interface creation device is also generally located within the server 105. However, it should also be noted that when terminal devices 101, 102, and 103 also possess sufficient computing power and resources, they can also complete the aforementioned operations performed by the server 105 through their installed interface creation applications, thereby outputting the same results as the server 105. Especially when multiple terminal devices with different computing capabilities exist simultaneously, but the interface creation application determines that the terminal device has strong computing power and abundant remaining computing resources, it can allow the terminal device to perform the aforementioned operations, thereby appropriately reducing the computing pressure on the server 105. Accordingly, the interface creation device can also be located within the terminal devices 101, 102, and 103. In this case, the exemplary system architecture 100 may also exclude the server 105 and the network 104.

[0030] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0031] Please refer to Figure 2 , Figure 2 A flowchart of an interface creation method provided in this disclosure embodiment, wherein process 200 includes the following steps:

[0032] Step 201: Determine the actual length and actual concurrency of each request received within the preset statistical period;

[0033] This step aims to create the execution body of the method from the interface (e.g., Figure 1 The server 105 shown determines the actual length and actual concurrency of each request received within a preset statistical period.

[0034] The preset statistical period can be flexibly set according to the actual application scenario, such as 1 hour, 6 hours, 1 day or even 1 week, in order to collect sufficiently representative requests by setting a reasonable preset statistical period, so as to carry out subsequent processing according to their length parameters and concurrency parameters.

[0035] The received request is the request received by the neural network model during the forward computation stage, thus characterizing the request features of the neural network model during the forward computation stage.

[0036] Step 202: Cluster the actual lengths of each request and determine the target length corresponding to each length cluster center;

[0037] Building upon step 201, this step aims to have the aforementioned executing entity cluster the actual lengths of each request to obtain multiple length cluster centers, and further determine a target length that can represent the multiple actual lengths contained under each length cluster center. This target length can be the mean, median, mode, maximum, etc., of the multiple actual lengths contained under that cluster center, or the result of arithmetic operations on the mean, median, mode, maximum, and a certain correction coefficient, as long as it is sufficiently representative. The representativeness requirements vary in different application scenarios and can be set according to actual requirements. For example, if representativeness is required to cover all lengths under that cluster center, the maximum value can be selected as the target length.

[0038] Step 203: Cluster the actual concurrency of each request and determine the target concurrency corresponding to the cluster center of each concurrency;

[0039] Building upon step 201, this step aims to have the aforementioned executing entity cluster the actual concurrency of each request to obtain multiple concurrency cluster centers. Furthermore, it determines a target concurrency number that can represent the multiple actual concurrency numbers contained under each cluster center. This target concurrency number can be the mean, median, mode, or maximum value of the multiple actual concurrency numbers contained under that cluster center, or the result of arithmetic operations on the mean, median, mode, maximum value, and a certain correction coefficient, as long as it is sufficiently representative. The representativeness requirements vary in different application scenarios and can be set according to actual requirements. For example, if representativeness is required to cover all concurrency numbers under that cluster center, the maximum value can be chosen as the target concurrency number.

[0040] In addition, steps 202 and 203 are two parallel execution steps. There is no sequential, causal, or dependent relationship between them. They can be executed simultaneously or separately. The step sequence presented in process 200 is just a schematic diagram.

[0041] Step 204: Create multiple target interfaces with a length parameter of any target length and a concurrency parameter of any target concurrency.

[0042] Based on steps 202 and 203, this step aims to have the aforementioned executing entity create multiple target interfaces according to each target length and each target concurrency. That is, each target interface should have different length parameters and concurrency parameters. Therefore, the length parameter of each target interface should be any target length, and the concurrency parameter should be any target concurrency. In other words, the maximum number of target interfaces is the product of the number of target lengths and the number of target concurrency.

[0043] Since the request is a request received by the neural network model during the forward computation phase, the multiple target interfaces created in this disclosure based on the clustering results of request length and concurrency are the Cuda_graph interface used to record the core computation order and parameter pointers during the forward computation of the neural network model.

[0044] Compared to the common forward computation method where the CPU continuously sends instructions to the GPU, using Cuda_graph for forward computation of neural network models can avoid the time-consuming kernel startup, thereby effectively improving processing efficiency.

[0045] The interface creation method provided in this disclosure performs clustering operations on the length and concurrency of each request received within a statistical period, thereby determining multiple scientific and reasonable target lengths and multiple target concurrency based on the clustering results. This makes the multiple target interfaces created based on these results more in line with actual needs, rather than being created blindly, thereby improving the utilization rate of the created target interfaces and further improving the processing efficiency of the forward computation stage of the neural network model.

[0046] Please refer to Figure 3 , Figure 3 A flowchart of another interface creation method provided in this disclosure embodiment, wherein process 300 includes the following steps:

[0047] Step 301: Determine the actual length and actual concurrency of each request received within the preset statistical period;

[0048] Step 301 and as follows Figure 2 The steps shown in step 201 are the same. For the same parts, please refer to the corresponding parts of the previous embodiment. They will not be repeated here.

[0049] Step 302: Based on the preset first cluster center number, perform clustering processing on the actual length of each request to obtain multiple length cluster centers with the same number as the first cluster center number;

[0050] Step 303: Determine the maximum actual length under each length cluster center as the target length of the corresponding length cluster center;

[0051] Regarding step 202 in process 200, this embodiment provides a more specific implementation scheme through steps 302-303. Specifically, the first number of cluster centers is pre-defined so that, given the first number of cluster centers, the clustering algorithm ultimately clusters the actual lengths of each request into length cluster centers equal to the number of the first cluster centers. Considering that the target length of each length cluster center is ultimately used to guide the creation of the Cuda_graph interface, and that Cuda_graph interfaces with different target lengths as length parameters are used to handle which lengths they can process, this embodiment determines the maximum actual length under each length cluster center as the target length of the corresponding length cluster center, thereby ensuring that the subsequently created Cuda_graph interfaces function correctly.

[0052] Specifically, the clustering process can be implemented using the K-means clustering algorithm, or other clustering algorithms that can achieve the same effect can be used; no specific limitation is made here.

[0053] Step 304: Based on the preset second cluster center number, perform clustering processing on the actual concurrency of each request to obtain multiple concurrency cluster centers with the number of the second cluster center number;

[0054] Step 305: Determine the maximum actual concurrency under each concurrency cluster center as the target concurrency for the corresponding concurrency cluster center;

[0055] Regarding step 203 in process 200, this embodiment provides a more specific implementation scheme through steps 304-305. Specifically, the second number of cluster centers is pre-defined so that, given the given second number of cluster centers, the clustering algorithm ultimately clusters the actual concurrency of each request into cluster centers equal to the second number of cluster centers. Considering that the target concurrency of each concurrency cluster center is ultimately used to guide the creation of the Cuda_graph interface, and that Cuda_graph interfaces with different target concurrency as concurrency parameters are used to handle which subsequent concurrent requests they can process, this embodiment determines the maximum actual concurrency under each concurrency cluster center as the target concurrency of the corresponding concurrency cluster center, thereby ensuring that the subsequently created Cuda_graph interfaces function correctly.

[0056] The specific values ​​of the first and second cluster centers can be obtained based on historical experience or feature analysis of historical requests, in order to match the actual situation as closely as possible.

[0057] Step 306: Determine all parameter combinations formed by any target length and any target concurrency;

[0058] Step 307: Generate a corresponding target interface for each combination of parameters.

[0059] Regarding step 204 in process 200, this embodiment provides a more specific implementation scheme through steps 306-307. That is, firstly, all parameter combinations formed by any target length and any target concurrency are determined. That is, each target length can form a unique parameter combination for each target concurrency, and the number of all parameter combinations is the product of the number of target lengths and the number of target concurrency. That is, assuming there are 4 different target lengths and 8 different target concurrency, the maximum number of parameter combinations can be 4×8=32. In other words, this embodiment will generate a corresponding interface for each parameter combination to cover all parameter combinations without omission.

[0060] Unlike the embodiment shown in process 200, this embodiment provides a more specific implementation method for determining multiple target lengths based on a clustering algorithm through steps 302-303, and selects the maximum actual length under each length cluster center as the corresponding target length based on practical application; through steps 304-305, it provides a more specific implementation method for determining multiple target concurrency numbers based on an algorithm, and similarly selects the maximum actual concurrency number under each concurrency number cluster center as the corresponding target concurrency number based on practical application; through steps 306-307, it provides a target interface creation method that covers all parameter combinations of length parameters and concurrency parameters to avoid omitting all possible parameter combinations.

[0061] Additionally, it should be noted that there is no causal or dependency relationship between the lower-level implementation methods provided in steps 302-303, the lower-level implementation schemes provided in steps 304-305, and the lower-level implementation schemes provided in steps 306-307. Based on process 200, multiple different independent embodiments can be obtained by individually replacing the corresponding upper-level schemes. This embodiment exists only as a preferred embodiment that simultaneously includes corresponding lower-level schemes for all three upper-level schemes.

[0062] Based on any of the above embodiments, before performing clustering processing (i.e., clustering by length and by concurrency), occasional long requests received within a preset statistical period can be removed to avoid the impact of occasional long requests on the accuracy of subsequent clustering results. Occasional long requests are defined as a small number of requests with a length significantly greater than that of other requests.

[0063] One possible implementation method, including but not limited to, is shown in the following example. Figure 4 The process 400 shown includes the following steps:

[0064] Step 401: Sort the requests received within the preset statistical period in descending order of their actual length to obtain the sorting results;

[0065] Step 402: Remove requests whose length ranking is among the top preset proportions in the sorting results.

[0066] Steps 401-402 provide a method for removing some of the longer requests that rank higher in the length sorting results. Typically, this preset ratio can be set to 20%.

[0067] In addition to removing some long requests that rank high in the length sorting results, we can also determine whether there are abnormally large length differences by analyzing the length differences between adjacent requests in the length sorting results, and remove the long requests with abnormally large length differences as occasional long requests.

[0068] Based on any of the above embodiments, this embodiment further describes how to use multiple pre-created target interfaces to specifically accelerate the forward computation process of a neural network model. Figure 5 The flowchart shown provides a specific implementation method, and its process 500 includes the following steps:

[0069] Step 501: Use each target interface to process new requests whose maximum length does not exceed its own processing capacity;

[0070] For subsequent new requests, if the length of the new request does not exceed the target length of each target interface, then look at the current cumulative number of new requests N and the maximum length of all new requests L, and select a target interface with suitable processing capacity (i.e., a concurrency parameter exceeding N and a length parameter exceeding L) based on N and L.

[0071] If N is greater than the target concurrency of the target interface, then only the target concurrency is calculated at this time, and the remaining new requests continue to exist in the request pool. After the current target concurrency is calculated, the above steps are repeated to continue the calculation until all new requests in the request pool are calculated.

[0072] Step 502: For target requests that cannot be processed by all target interfaces, transfer them to the central processing unit for processing.

[0073] For target requests that cannot be handled by any of the target interfaces (i.e., excessively long requests), considering that the number of these target requests is relatively small and the cost of creating a suitable target interface for them is too high, they are directly handed over to the central processing unit for processing in the most conventional way, that is, the Cuda_graph interface is no longer used to speed up the processing.

[0074] Further reference Figure 6As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of an interface creation apparatus, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0075] like Figure 6 As shown, the interface creation device 600 of this embodiment may include: an actual parameter determination unit 601, a target length determination unit 602, a target concurrency determination unit 603, and a target interface creation unit 604. The actual parameter determination unit 601 is configured to determine the actual length and actual concurrency of each request received within a preset statistical period; the target length determination unit 602 is configured to perform clustering processing on the actual length of each request and determine the target length corresponding to each length cluster center; the target concurrency determination unit 603 is configured to perform clustering processing on the actual concurrency of each request and determine the target concurrency corresponding to each concurrency cluster center; the target interface creation unit 604 is configured to create multiple target interfaces with a length parameter of any target length and a concurrency parameter of any target concurrency; wherein the target interface is used to record the core calculation order and parameter pointers during the forward computation of the neural network model.

[0076] In this embodiment, the specific processing of the actual parameter determination unit 601, the target length determination unit 602, the target concurrency determination unit 603, and the target interface creation unit 604 in the interface creation device 600, and the resulting technical effects, can be found in the following references: Figure 2 The relevant descriptions of steps 201-204 in the corresponding embodiments will not be repeated here.

[0077] In some optional implementations of this embodiment, the target length determination unit 602 can be further configured as follows:

[0078] Based on the preset first cluster center number, the actual length of each request is clustered to obtain multiple length cluster centers with the same number as the first cluster center number;

[0079] The maximum actual length under each length cluster center is determined as the target length of the corresponding length cluster center.

[0080] In some optional implementations of this embodiment, the target concurrency determination unit 603 can be further configured to:

[0081] Based on the preset second cluster center number, the actual concurrency of each request is clustered to obtain multiple concurrency cluster centers with the number of the second cluster center number;

[0082] The maximum actual concurrency under each concurrency cluster center is determined as the target concurrency for the corresponding concurrency cluster center.

[0083] In some optional implementations of this embodiment, the target interface creation unit 601 can be further configured to:

[0084] Determine all parameter combinations formed by the length of any target and the concurrency of any target;

[0085] Generate a corresponding target interface for each combination of parameters.

[0086] In some optional implementations of this embodiment, the interface creation device 600 may further include:

[0087] The occasional long request removal unit is configured to remove occasional long requests received within a preset statistical period before performing clustering processing; wherein, occasional long requests are requests with a much larger actual length than other requests and a small number of them.

[0088] In some optional implementations of this embodiment, the occasional long request removal unit can be further configured as follows:

[0089] The requests received within the preset statistical period are sorted from largest to smallest according to their actual length to obtain the sorting result;

[0090] Remove requests whose length ranks first by a preset percentage in the sorting results.

[0091] In some optional implementations of this embodiment, the interface creation device 600 may further include:

[0092] The new request processing unit is configured to process new requests whose maximum length does not exceed its own processing capacity using each target interface.

[0093] The target request processing unit is configured to forward target requests that cannot be processed by any target interface to the central processing unit.

[0094] This embodiment exists as a device embodiment corresponding to the above method embodiment. The interface creation device provided in this embodiment performs clustering operations on the length and concurrency of each request received within a statistical period, thereby determining multiple scientific and reasonable target lengths and multiple target concurrency based on the clustering results. This makes the multiple target interfaces created based on this more in line with actual needs, rather than being created blindly, thereby improving the utilization rate of the created target interfaces and further improving the processing efficiency of the forward computation stage of the neural network model.

[0095] According to embodiments of this disclosure, this disclosure also provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the interface creation method described in any of the above embodiments when executed.

[0096] According to embodiments of this disclosure, this disclosure also provides a readable storage medium storing computer instructions that enable a computer to implement the interface creation method described in any of the above embodiments when executed.

[0097] According to embodiments of this disclosure, this disclosure also provides a computer program product that, when executed by a processor, can implement the interface creation method described in any of the above embodiments.

[0098] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0099] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0100] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0101] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the interface creation method. For example, in some embodiments, the interface creation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the interface creation method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the interface creation method by any other suitable means (e.g., by means of firmware).

[0102] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0103] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0104] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0105] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0106] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0107] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0108] According to the technical solution of this disclosure, by performing clustering operations on the length and concurrency of each request received within a statistical period, multiple scientific and reasonable target lengths and multiple target concurrency are determined based on the clustering results. This makes the multiple target interfaces created based on these results more in line with actual needs, rather than being created blindly, thereby improving the utilization rate of the created target interfaces and further improving the processing efficiency of the forward computation stage of the neural network model.

[0109] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0110] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for creating interfaces, comprising: determining actual lengths and actual concurrencies of requests received in a preset statistical period; clustering the actual lengths of the requests and determining target lengths corresponding to each length cluster center; clustering the actual concurrencies of the requests and determining target concurrencies corresponding to each concurrency cluster center; creating a plurality of target interfaces with a length parameter being any of the target lengths and a concurrency parameter being any of the target concurrencies, wherein the target interfaces are used to record a calculation order of cores and parameter pointers during forward calculation of a neural network model, and a maximum number of the target interfaces is a product of a number of the target lengths and a number of the target concurrencies; processing a new request with a maximum length not exceeding a processing capability of each of the target interfaces by using the target interfaces, including: in response to the length of the new request not exceeding the target lengths of the target interfaces, selecting a target interface according to a number of all new requests currently accumulated and a maximum length of all new requests.

2. The method of claim 1, wherein, The clustering the actual lengths of the requests and determining target lengths corresponding to each length cluster center, comprises: clustering the actual lengths of the requests according to a preset first cluster center number to obtain a plurality of length cluster centers with the first cluster center number; determining a maximum actual length under each of the length cluster centers as a target length of the corresponding length cluster center.

3. The method of claim 1, wherein, The clustering the actual concurrencies of the requests and determining target concurrencies corresponding to each concurrency cluster center, comprises: clustering the actual concurrencies of the requests according to a preset second cluster center number to obtain a plurality of concurrency cluster centers with the second cluster center number; determining a maximum actual concurrency under each of the concurrency cluster centers as a target concurrency of the corresponding concurrency cluster center.

4. The method of claim 1, wherein, The creating a plurality of target interfaces with a length parameter being any of the target lengths and a concurrency parameter being any of the target concurrencies, comprises: determining all parameter combinations formed by any of the target lengths and any of the target concurrencies; generating a corresponding target interface for each of the parameter combinations.

5. The method of claim 1, further comprising: before the clustering, removing sporadic long requests received in the preset statistical period, wherein the sporadic long requests are a small number of requests with actual lengths much larger than those of other requests.

6. The method of claim 5, wherein, The removing the sporadic long requests received in the preset statistical period, comprises: sorting the requests received in the preset statistical period according to the actual lengths from large to small to obtain a sorting result; removing requests with lengths ranked in a front preset proportion in the sorting result.

7. The method of any one of claims 1-6, further comprising: forwarding a target request that cannot be processed by all of the target interfaces to a central processor for processing.

8. An apparatus for creating interfaces, comprising: an actual parameter determination unit configured to determine actual lengths and actual concurrencies of requests received in a preset statistical period; a target length determination unit configured to cluster the actual lengths of the requests and determine a target length corresponding to each length cluster center; a target concurrency determination unit configured to cluster the actual concurrencies of the requests and determine a target concurrency corresponding to each concurrency cluster center; a target interface creation unit configured to create a plurality of target interfaces with a length parameter being any of the target lengths and a concurrency parameter being any of the target concurrencies; wherein the target interfaces are used to record the calculation order of cores and parameter pointers during forward calculation of a neural network model, and the maximum number of the target interfaces is the product of the number of the target lengths and the number of the target concurrencies; a new request processing unit configured to process new requests with a maximum length not exceeding the processing capacity of the target interfaces, including: in response to the length of the new request not exceeding the target length of the target interfaces, selecting a target interface according to the number of all new requests and the maximum length of all new requests currently accumulated.

9. The apparatus of claim 8, wherein, The target length determination unit is further configured to: cluster the actual lengths of the requests according to a preset first cluster center number to obtain a plurality of length cluster centers with the number of the first cluster center number; determine the maximum actual length under each length cluster center as the target length of the corresponding length cluster center.

10. The apparatus of claim 8, wherein, The target concurrency determination unit is further configured to: cluster the actual concurrencies of the requests according to a preset second cluster center number to obtain a plurality of concurrency cluster centers with the number of the second cluster center number; determine the maximum actual concurrency under each concurrency cluster center as the target concurrency of the corresponding concurrency cluster center.

11. The apparatus of claim 8, wherein, The target interface creation unit is further configured to: determine all parameter combinations formed by any of the target lengths and any of the target concurrencies; generate a corresponding target interface for each of the parameter combinations.

12. The apparatus of claim 8, further comprising: an accidental long request removal unit configured to remove accidental long requests received in the preset statistical period before the clustering processing; wherein the accidental long requests are a small number of requests with an actual length much larger than that of other requests.

13. The apparatus of claim 12, wherein, The accidental long request removal unit is further configured to: sort each request received in the preset statistical period according to the actual length from large to small to obtain a sorting result; remove requests with a length ranking in the top preset proportion in the sorting result.

14. The apparatus of any one of claims 8-13, further comprising: a target request processing unit configured to transfer target requests that cannot be processed by all the target interfaces to a central processor for processing.

15. An electronic device, comprising: at least one processor; and a memory connected in communication with the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the interface creation method of any one of claims 1-7.

16. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the interface creation method of any one of claims 1-7.

17. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the interface creation method according to any one of claims 1-7.

Images