Algorithm deployment method and device, electronic equipment and storage medium

Abstract

The invention provides an algorithm deployment method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining a target algorithm; adding a first function to the target algorithm according to a preset framework to generate an algorithm class; wherein the preset framework is used for determining a mode of adding a first function into a target algorithm, and the first function is used for loading, releasing and computational reasoning of the target algorithm; determining at least one service corresponding to the target algorithm; wherein the at least one service is at least one function using a target algorithm; determining a common part and a distinguishing part of at least one service, establishing an affiliation relationship between the common part and the distinguishing part, generating a service class, and establishing a corresponding relationship between an algorithm class and the service class; and in response to the obtained request for the service class, determining and deploying the corresponding algorithm class, and determining a data conversion mode of the request according to the corresponding common part and the distinguishing part.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an algorithm deployment method, apparatus, electronic device and storage medium. Background Technology

[0002] Currently, when the program system provides services to users, it provides corresponding interfaces and deploys corresponding algorithms based on the user's selection. Therefore, when a user uses the service multiple times or connects with multiple users, the program system needs to clear the previously deployed algorithms and redeploy the corresponding algorithms based on the new request each time it receives a service request.

[0003] It can be seen that when deploying services, it is necessary to encapsulate and deploy the algorithm corresponding to each service, which brings a huge workload to the development work. Furthermore, each service is isolated from the others, and different services independently occupy a part of the space for storing their respective algorithms, which increases costs and response time, and seriously affects efficiency and user experience. Summary of the Invention

[0004] In view of this, this application proposes an algorithm deployment method, apparatus, electronic device, and storage medium to solve or partially solve the above problems.

[0005] To achieve the above objectives, this application provides an algorithm deployment method, comprising:

[0006] Obtain the target algorithm;

[0007] A first function is added to the target algorithm according to a preset framework to generate an algorithm class; wherein, the preset framework is used to determine how the first function is added to the target algorithm, and the first function is used to load, release and perform computational inference on the target algorithm;

[0008] Identify at least one service corresponding to the target algorithm; wherein, the at least one service is at least one function that uses the target algorithm;

[0009] Determine the common and distinct parts of the at least one service, establish the subordinate relationship between the common and distinct parts, generate service classes, and establish the correspondence between the algorithm class and the service class;

[0010] In response to receiving a request for the service class, the corresponding algorithm class is determined and deployed, and the data conversion method of the request is determined based on the corresponding common part and the different part.

[0011] In some exemplary embodiments, the first function includes at least one of the following: a first sub-function for loading the algorithm model, reading the configuration file and initializing the algorithm; a second sub-function for releasing algorithm resources; and a third sub-function for algorithm computation and inference.

[0012] In some exemplary embodiments, determining the common and distinct portions of the at least one service includes:

[0013] The at least one service is parsed to determine the base class of the at least one service as the common part and the subclass of the at least one service as the distinguishing part; wherein the base class is the common part determined among the at least one service and the subclass is the different part determined among the at least one service.

[0014] In some exemplary embodiments, after establishing the correspondence between the algorithm class and the service class, the method further includes:

[0015] The algorithm class is stored in the algorithm storage unit; wherein, the algorithm storage unit is at least used to store algorithm classes formed by different target algorithms;

[0016] The service class is stored in the service storage unit; wherein the service storage unit is at least used to store the service classes formed corresponding to different target algorithms.

[0017] In some exemplary embodiments, after receiving a request for the service class, the method further includes:

[0018] The algorithm corresponding to the algorithm class is retrieved and loaded from the algorithm storage unit;

[0019] The service class is determined in the service storage unit, and resources are configured according to the common part and the different part corresponding to the service class.

[0020] In some exemplary embodiments, determining and deploying the corresponding algorithm class includes:

[0021] Generate a task identifier for the algorithm class and determine at least one process capable of executing the algorithm class;

[0022] The algorithm class and the task identifier are added to the shared waiting queue of the at least one process in the form of a task, and feedback information is returned.

[0023] In some exemplary embodiments, after adding the algorithm class to the shared wait queue of the at least one process, the method further includes:

[0024] In response to any process completing its current task, the next task of any process is obtained sequentially according to the shared waiting queue.

[0025] In some exemplary embodiments, after any process completes its current task, the method further includes:

[0026] Output the processing result of the current task to the shared result queue;

[0027] Based on the task identifier corresponding to the current task, establish the correspondence between the processing result and the corresponding algorithm class and service class.

[0028] In some exemplary embodiments, after determining at least one process capable of executing the algorithm class, the method further includes:

[0029] Determine the number of processes that need to be executed at the same time, and determine the number of processors capable of executing the at least one process;

[0030] Tasks are evenly distributed based on the number of processes and the number of processors.

[0031] Based on the same concept, this application also provides an algorithm deployment apparatus, including:

[0032] The acquisition module is used to acquire the target algorithm;

[0033] The first generation module is used to add a first function to the target algorithm according to a preset framework to generate an algorithm class; wherein, the preset framework is used to determine the way the first function is added to the target algorithm, and the first function is used to load, release and perform computational inference on the target algorithm;

[0034] A determining module is configured to determine at least one service corresponding to the target algorithm; wherein the at least one service is at least one function that uses the target algorithm;

[0035] The second generation module is used to determine the common parts and the distinct parts of the at least one service, establish the subordinate relationship between the common parts and the distinct parts, generate service classes, and establish the correspondence between the algorithm class and the service class;

[0036] The response module is used to respond to a request for the service class, determine and deploy the corresponding algorithm class, and determine the data conversion method of the request based on the corresponding common part and the different part.

[0037] Based on the same concept, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method as described in any of the preceding claims.

[0038] Based on the same concept, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing a computer to perform the method described in any of the preceding claims.

[0039] As can be seen from the above description, this application provides an algorithm deployment method, apparatus, electronic device, and storage medium. The method includes: obtaining a target algorithm; adding a first function to the target algorithm according to a preset framework to generate an algorithm class; wherein the preset framework is used to determine the way the first function is added to the target algorithm, and the first function is used to load, release, and perform computational inference on the target algorithm; determining at least one service corresponding to the target algorithm; wherein the at least one service is at least one function using the target algorithm; determining the common and distinct parts of the at least one service, establishing the subordinate relationship between the common and distinct parts, generating a service class, and establishing a correspondence between the algorithm class and the service class; in response to obtaining a request for the service class, determining and deploying the corresponding algorithm class, and determining the data conversion method of the request according to the corresponding common and distinct parts. This application utilizes a first function to encapsulate the algorithm with a unified interface. The algorithm can then be fixed in a corresponding dictionary. Subsequently, the inheritance mechanism of programming languages ​​such as Python can be used to parse different services, determine the common and distinct parts of similar services, and match them with the algorithm parts fixed in the dictionary. This allows for direct retrieval and loading of the corresponding algorithm from the fixed algorithm dictionary each time a service is used, avoiding the need to deploy the corresponding algorithm for each service each time. This reduces development workload, improves development efficiency and overall response time, and enhances user experience. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart illustrating an exemplary method provided in an embodiment of this application.

[0042] Figure 2This is a schematic diagram illustrating the multi-process asynchronous implementation of the exemplary method provided in the embodiments of this application.

[0043] Figure 3 This is a schematic diagram illustrating the exemplary method for uniformly distributing multiple processes according to embodiments of this application.

[0044] Figure 4 This is a schematic diagram of the structure of an algorithm deployment system constructed using an exemplary method provided in an embodiment of this application.

[0045] Figure 5 A schematic diagram of the structure of an exemplary device provided in an embodiment of this application.

[0046] Figure 6 This is a schematic diagram of the electronic device structure provided in an embodiment of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this specification clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0048] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar words used in the examples of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word covers the element or object listed after the word and its equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0049] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0050] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this application's technical solution, based on the prompt message.

[0051] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0052] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0053] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0054] As described in the background section, in some embodiments, different applications provide users with a wide variety of services. Furthermore, the same service may be further subdivided in different scenarios due to different interfaces involved. For example, an application might provide a font beautification service, which corresponds to a core algorithm. However, due to different interfaces—such as users generating service requests via website links or by inputting text or images—the application needs to retrieve and deploy the corresponding algorithm upon receiving the request. Consequently, algorithm or service development requires extensive algorithm encapsulation for various scenarios and services. Moreover, when providing a service, each new service requires clearing all algorithms associated with the previous service and redeploying all algorithms for the current service, increasing costs and response time, and severely impacting efficiency and user experience.

[0055] In light of the above-mentioned practical situation, this application provides an algorithm deployment method. This application utilizes a first function to encapsulate the algorithm with a unified interface, which then fixes the algorithm in a corresponding dictionary. Subsequently, the inheritance mechanism of programming languages ​​such as Python can be used to parse different services, determine the common and different parts of similar services, and correspond them with the algorithm parts fixed in the dictionary. This allows for direct retrieval and loading of the corresponding algorithm from the fixed algorithm dictionary each time a service is deployed, avoiding the need to deploy the corresponding algorithm for each service each time. This method reduces development workload, improves development efficiency and overall response time, and enhances user experience.

[0056] To make the objectives, technical solutions, and advantages of this specification clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0057] Figure 1 A flowchart illustrating an exemplary method provided in an embodiment of this application is shown.

[0058] like Figure 1 As shown in the embodiments of this application, the algorithm deployment method proposed by example includes the following steps.

[0059] Step 102: Obtain the target algorithm.

[0060] In this step, the target algorithm is the core algorithm applied to different services. For example, for a 3D reconstruction service, the target algorithm is a 3D reconstruction algorithm; for a face recognition service, the target algorithm is a face recognition algorithm. Of course, a specific service may correspond to more than one service, and the specific settings can be configured according to the specific application scenario.

[0061] Step 104: Add a first function to the target algorithm according to the preset framework to generate an algorithm class; wherein, the preset framework is used to determine the way the first function is added to the target algorithm, and the first function is used to load, release and perform computational inference on the target algorithm.

[0062] In this step, the target algorithm needs to be implemented in a fixed class manner, that is, the required algorithm interface needs to be standardized and unified. Taking the aforementioned 3D reconstruction algorithm and face recognition algorithm as examples, if these two algorithms are directly encapsulated or stored, due to the differences between the two algorithms themselves, their encapsulation or storage methods will inevitably differ, making it impossible for different algorithms to be standardized according to a unified format. Therefore, it is necessary to provide a unified "shell" for different algorithms. That is, the target algorithm can be packaged using a preset framework, and a first function can be added to the target function. This preset framework can be mainly formed by fixing the way the first function is added to the target algorithm to form a unified format. This unified format can be the preset framework. Among them, the first function is used to load, release, and perform calculation inference on the target algorithm. In some embodiments, the first function can be a function that includes all the above functions, or it can be composed of multiple functions, each function being used to implement one or more of the above specific functions.

[0063] In a more specific scenario, the first function can specifically include a first sub-function for loading the algorithm model, reading the configuration file, and initializing the algorithm; this first sub-function can specifically be the constructor `_init_()`. The first function can also include a second sub-function for releasing algorithm resources; this second sub-function can specifically be the algorithm's destructor. The first function can also include a third sub-function for algorithm computation and inference; this third sub-function can specifically be the algorithm's inference and computation function `forward`.

[0064] After adding a first function to the target algorithm, different target algorithms can achieve interface standardization through the "shell" provided by the first function, completing the implementation of a fixed class and generating corresponding algorithm classes. These algorithm classes can then be named or have corresponding identifiers set, and then stored in a specific storage space. For example, the decorator method of the Python programming language can be used to automatically save the algorithm class name and constructor to the corresponding algorithm dictionary, becoming an algorithm repository. That is, the algorithm classes can be stored in the corresponding algorithm storage unit, which is at least used to store algorithm classes formed by different target algorithms. Python is a high-level scripting language that combines interpretation, compilation, interactivity, and object-oriented features, and its source code follows the GPL (GNU General Public License) agreement.

[0065] Step 106: Determine at least one service corresponding to the target algorithm.

[0066] In this step, each target algorithm corresponds to at least one service. For example, a face recognition algorithm, primarily used for face recognition services, may correspond to only one service: face recognition. That is, a service is a specific function provided to the user, and its specific function can ultimately be achieved using the corresponding target algorithm. However, in different scenarios, the face recognition service may correspond to multiple specific services or interfaces. For instance, users can request face recognition services by entering a URL, a specific application, or clicking a specific link. While the core of these methods is face recognition, the specific service format (e.g., input / output methods) will differ depending on the scenario. Therefore, each core service may correspond to multiple specific services, and thus, one target algorithm may correspond to at least one service.

[0067] Step 108: Determine the common and distinct parts of the at least one service, establish the subordinate relationship between the common and distinct parts, generate a service class, and establish the correspondence between the algorithm class and the service class.

[0068] In this step, the generated service class is mainly used to manage the algorithm's requests and responses, which can be implemented primarily through a process-shared queue. For any given service, as described in step 106, a core service may correspond to multiple specific services. Therefore, these specific services must share a common core component. This allows us to determine the common and differentiating parts of these services. The common part is the shared core component of these specific services, while the differentiating part is the specific difference between different services, such as the data conversion methods for input and output. This also allows us to determine the hierarchical relationship between the common and differentiating parts. Subsequently, we can use a storage method such as a tree structure (where the common part can be the parent node of the tree structure, and the differentiating part is the child node of that parent node, which also reflects the hierarchical relationship) to fix the structure.

[0069] In some embodiments, the inheritance mechanism of the Python programming language can be used to parse at least one service. The base class implements request management, asynchronous / synchronous returns, and multi-process parallel processing, while the subclasses inherit the abstract interface of the base class to handle the intermediate conversion between web call parameters and algorithm parameters, as well as special functionalities required by the specific service, such as GPU resource allocation. That is, in some embodiments, determining the common and distinct parts of the at least one service includes: parsing the at least one service, determining the base class of the at least one service as the common part, and determining the subclasses of the at least one service as the distinct part; wherein the base class is the common part determined among the at least one service, and the subclasses are the distinct parts determined among the at least one service. In more specific scenarios, similar to the aforementioned algorithm classes, decorators in the Python programming language can be used to cleverly store the class name and class constructor of the service class in the corresponding dictionary using its inheritance mechanism. That is, the service class can be stored in the corresponding service storage unit, which is at least used to store the service classes corresponding to different target algorithms.

[0070] Next, it is necessary to establish a correspondence or mapping relationship between algorithm classes and service classes, so as to quickly determine the corresponding algorithm class after the service is determined.

[0071] In some embodiments, the Python decorator method used during registration completes the mapping from string to class. As shown in the following code example, after defining an algorithm, the registration class is used to decorate the algorithm class, saving the class name and constructor of the algorithm or service into the dictionary of the registration class.

[0072]

[0073]

[0074] In more specific scenarios, registering algorithms and services can be achieved through configuration files, enabling rapid algorithm loading and switching. In practical applications, developing a business algorithm and quickly deploying and encapsulating it into a callable service to handle complex request management requires significant development effort. Therefore, this embodiment pre-sets a framework, defines a general algorithm interface (for specific algorithm business logic) and a service interface (for algorithm parameter conversion, parsing, and validation), and then adds this class to the overall framework through configuration file adaptation, thus building the algorithm and corresponding services.

[0075] Regarding the Python decorator method used during registration, as mentioned above, each functional algorithm needs to implement two modules: an algorithm business class and an algorithm service class. This is achieved through decorators; the decorator class's registration function requires a class as an input parameter. The name and constructor of the class to be registered are then obtained through the class, and stored as key-value pairs in a dictionary. When the system starts, it reads the configuration file, retrieves the names of the two modules, and then obtains their class constructors, thereby loading the algorithm business and algorithm service classes. Specifically, in some embodiments, after receiving a request for the service class, the method further includes: retrieving and loading the algorithm corresponding to the algorithm class from the algorithm storage unit; determining the service class in the service storage unit, and configuring resources based on the common and distinct parts of the service class. The algorithm storage unit can be an algorithm dictionary or database corresponding to the algorithm class, and the service storage unit can be a service dictionary or database corresponding to the service class. Subsequently, as mentioned above, the differences between multiple services that share common components mainly lie in aspects such as data input and output formats. Therefore, it is necessary to convert the output data into the corresponding format according to the corresponding input format in order to complete the corresponding resource configuration.

[0076] Step 110: In response to receiving a request for the service class, determine and deploy the corresponding algorithm class, and determine the data conversion method of the request based on the corresponding common part and the different part.

[0077] In this step, once the algorithm and services have been fixed according to the preceding steps, the preparation for rapid algorithm deployment is complete. Using the fixed algorithm and services, the algorithm can be pre-loaded, and subsequent requests can directly retrieve the corresponding algorithm from the appropriate dictionary, without repeatedly loading the algorithm. Specifically, after a user generates a request using an application or webpage, this request generally corresponds to a specific service class. Based on the aforementioned correspondence, the corresponding algorithm class can be determined. Furthermore, based on the common and distinct parts of the corresponding service classes, the required data format for the request can be determined. Finally, after the algorithm class generates the corresponding output data, the output data is converted according to the appropriate data conversion method.

[0078] In some embodiments, certain functions of some applications may require support from multiple services. In other scenarios, different applications or functions may simultaneously generate demands for multiple services. Therefore, to enable multiple service invocation methods and concurrent request processing, in more specific scenarios, multi-process, thread, and coroutine combined with process-shared queues can be used to implement synchronous and asynchronous service invocation methods. In actual implementations, such as... Figure 2 As shown, a thread can be started in the asynoic module using the asynoic.to_thread method to push request data into the shared queue of the request process as a task. The synchronous service request processing function is not blocked; when another request is received, it can still be pushed into the request queue. Each request is identified with a unique ID. Multiple backend processes will then retrieve request data from the request queue, process it, and put the task ID into the result inference output queue. At this point, a coroutine can be started to retrieve data from the output queue and, according to the task ID, put it into a dictionary of asynchronous queues to generate a synchronous return, or directly use the POST callback interface for asynchronous return. Specifically, as... Figure 2 As shown, when requested data arrives, it can be enqueued via asynchronous threads. Multiple threads can be used to push the request into a shared input queue, while simultaneously generating corresponding feedback information. This feedback information notifies the requester that the request has been accepted but needs to wait in the queue for processing. Specifically, in some embodiments, determining and deploying the corresponding algorithm class includes: generating a task identifier for the algorithm class; determining at least one process capable of executing the algorithm class; adding the algorithm class and the task identifier as a task to the shared waiting queue of the at least one process; and returning feedback information. The task identifier is the corresponding task ID.

[0079] Subsequently, multiple processes share the input queue. After each process completes its own task, it can obtain a new task from the input queue. That is, in some embodiments, after adding the algorithm class to the shared waiting queue of the at least one process, the method further includes: in response to any process completing its current task, obtaining the next task of any process in sequence according to the shared waiting queue. When a process completes the processing of requested data, it generates output data, forming an output queue. The request corresponding to each piece of output data can be identified by the corresponding task identifier, thereby determining the ownership of each piece of output data. Figure 2 As shown, TaskID1 to TaskID3, etc., are the task IDs corresponding to each output data, and Result1 to Result3, etc., correspond to the specific content of the output data. Then, in the asynchronous result acquisition step, a correspondence between the specific output data and the corresponding request is established through the task IDs. Finally, the output data can be fed back to the corresponding requester based on this correspondence. Simultaneously, the data format of the output data can be converted using the corresponding service class. That is, in some embodiments, after any process completes its current task, the method further includes: outputting the processing result of the current task to a shared result queue; and establishing a correspondence between the processing result and the corresponding algorithm class and service class based on the task identifier corresponding to the current task.

[0080] In more specific scenarios, each business algorithm is integrated into the service architecture as a packaged module. In practice, multi-process technology is used to load each algorithm business module into different processes, achieving multi-instance computation. This ensures the independence of each instance's computation while also improving computer performance. For example, to deploy a large language model for meeting minutes, to fully utilize computer resources, we can start multiple processes, each loading the meeting minutes algorithm business. Figure 2 Processes 1 to N are shown.

[0081] Secondly, input and output queues that enable inter-process communication are created. The main process uses three coroutines to manage request enqueueing, monitor the output queue, and return results. To prevent coroutine enqueueing from blocking FastAPI request reception, the asynoc module is used, which involves starting threads through coroutines to enqueue requests. The monitoring coroutine continuously monitors the data in the output queue and then enters the returned task IDs into the asynchronous queue in the main thread. The scheduled coroutine then retrieves the data from the corresponding asynchronous queue, using it as feedback data for the corresponding request.

[0082] Finally, the return method can be varied according to the actual application scenario. For example, depending on the time taken by the business algorithm, for algorithms with long processing times, an asynchronous callback approach can be used. That is, after obtaining the result of the business algorithm in the business process, the result can be directly fed back via POST, thereby achieving a concurrent data feedback mode. When the business algorithm has a short processing time, an asynchronous acquisition and waiting approach can be used to achieve a synchronous return, thereby improving the overall framework's operating efficiency.

[0083] In other embodiments, each process needs to be assigned to a corresponding processor for processing, such as assigning different processes to individual GPUs or CPUs for data processing. This allocation method could be that each process is assigned to a specific processor, for example, process 1 is assigned to GPU-1, process 2 to GPU-2, process 3 to GPU-3, and so on. However, in practical applications, typically one algorithm instance corresponds to one process, such as algorithm instance 1 to process 1, algorithm instance 2 to process 2. If algorithm instance 1 needs to handle a large number of requests, while algorithm instance 2 needs to handle fewer requests, then according to the previous allocation method, GPU-1 would need to handle many processes, while other GPUs might be idle, which is inherently detrimental to the rational utilization of resources. Therefore, in this embodiment, in order to fully allocate current computing resources and achieve full utilization of hardware computing resources, the number of processes that need to run simultaneously and the number of processors capable of executing these processes can be used to evenly distribute these tasks by setting appropriate functions. That is, in some embodiments, after determining at least one process capable of executing the algorithm class, the method further includes: determining the number of processes that need to be executed at the same time, and determining the number of processors capable of executing the at least one process; and evenly distributing tasks according to the number of processes and the number of processors.

[0084] Specifically, such as Figure 3As shown, to ensure full utilization of the GPU resources provided by the algorithm, a transformation function is used in each service subclass to allocate the algorithm of the corresponding process instance to specific GPU resources. This ensures that the system automatically launches the maximum number of model instances based on the provided GPU resources, achieving more efficient processing performance. In practical applications, when adding a new algorithm service, firstly, the algorithm module is written according to the algorithm interface specified in the aforementioned embodiment; secondly, the algorithm service class is implemented, mainly by inheriting the base class and overriding the input transformation (converting the input of the front-end caller to the input of the algorithm and performing necessary form and content validation); finally, the algorithm output is converted to the output of the caller and its content is validated. Afterwards, it is necessary to determine how to orchestrate the relationship between computer resources (such as GPUs or CPUs), algorithm instances, and processes in this service. For example, in the local inference algorithm for the large language model of meeting minutes, this solution designs each process to load one algorithm instance and classifies each algorithm service instance into two corresponding GPU resources. The input parameter of the transformation function is the process ID, and the output is the GPU ID number, which allocates 1 or 2 GPU resources to each service algorithm instance based on the process ID.

[0085] For the aforementioned transformation function, specifically, taking the balanced allocation of N GPUs and M algorithm instances as an example, the following steps can be used to calculate the number of algorithm instances that each GPU should run. (1) Problem description: There are N GPUs in total, each GPU can run a maximum of K algorithm instances, and a total of M algorithm instances need to be allocated. (2) Objective: To allocate the M algorithm instances to the N GPUs as evenly as possible. (3) Allocation principles: 1. Basic allocation quantity per GPU: The minimum number of algorithm instances that each GPU must run; 2. Allocation of remaining algorithm instances: Allocate the remaining algorithm instances to a portion of the GPUs to make the load as balanced as possible. (4) Specific calculation formula: Basic allocation quantity per GPU: in, The operator represents rounding down; the remaining algorithm instances: Remainder = M % N; where the "%" operator represents modulo operation. (5) Allocation scheme: First Remainder block graphics cards: each graphics card runs Base+1 algorithm instances. The remaining N-Remainder block graphics cards: each graphics card runs Base algorithm instances. For example, suppose there are N=3 graphics cards, K=5 algorithm instances at maximum load, and M=8 algorithm instances to be allocated. Calculate the base allocation quantity: Calculate the remaining algorithm instances: Remainder = 8%3 = 2. Therefore, the allocation scheme is: the first two graphics cards run 2 + 1 = 3 algorithm instances. The remaining graphics card runs 2 algorithm instances. That is, the result is: Graphics card 1 runs 3 algorithm instances, graphics card 2 runs 3 algorithm instances, and graphics card 3 runs 2 algorithm instances.

[0086] Ultimately, the above approach employs asynchronous execution. When the algorithm service receives an algorithm request, it can use the `await` method to place the request into a request queue. Simultaneously, multiple processes are used to implement multiple instances of the algorithm, processing the requests in the queue. Synchronous returns are achieved through coroutine lookup, or asynchronous returns via POST callbacks. In this way, by utilizing multiple processes, threads, and coroutines, and leveraging asynchronous programming methods, concurrent synchronous and asynchronous returns from the algorithm service are achieved, thus enabling flexible algorithm invocation.

[0087] like Figure 4 The diagram shows the structure of the algorithm deployment system constructed by the method of this application. When the web configuration sends out the corresponding algorithm parameters, algorithm configuration is performed; examples include 3D reconstruction, 3D visual generation, and font enhancement. Through the aforementioned standardized algorithm structure, an algorithm registry is used to fix the algorithms, forming an algorithm repository. Then, each algorithm corresponds to a corresponding service, such as 3D reconstruction service, 3D display service, and font enhancement service. By extracting common and different parts as described above, a service registry is used to fix the services and create service instances. Afterwards, upon receiving request data, it is enqueued asynchronously / by thread, pushing the request into a shared queue. Preset configuration data (e.g., service type, queue length, number of processes, service name, algorithm name, etc.) is loaded to create algorithm instances and processes, generating processes 1 to N. Simultaneously, algorithm instances are created within each process; process 1 corresponds to instance 1, process N corresponds to instance N, and so on. Finally, a shared memory model (HTTP or GPPC model) is used for even resource allocation and inference requests. Ultimately, the corresponding results are output through asynchronous coroutines and asynchronous callbacks to achieve synchronous return of the corresponding output data (i.e., establishing a correspondence between the output data identifier and the corresponding request data, and then synchronously returning the output data). Of course, the system may also contain auxiliary structures such as a logging system, service performance testing tools, and some commonly used tools (download, upload, etc.).

[0088] As can be seen from the above embodiments, the algorithm deployment method provided in this application includes: obtaining a target algorithm; adding a first function to the target algorithm according to a preset framework to generate an algorithm class; wherein the first function is used to load, release, and perform computational inference on the target algorithm; determining at least one service corresponding to the target algorithm; determining the common and distinct parts of the at least one service, establishing a subordinate relationship between the common and distinct parts, generating a service class, and establishing a correspondence between the algorithm class and the service class; in response to obtaining a request for the service class, determining and deploying the corresponding algorithm class, and determining the data conversion method of the request according to the corresponding common and distinct parts. This application utilizes a first function to encapsulate the algorithm with a unified interface. The algorithm can then be fixed in a corresponding dictionary. Subsequently, the inheritance mechanism of programming languages ​​such as Python can be used to parse different services, determine the common and distinct parts of similar services, and match them with the algorithm parts fixed in the dictionary. This allows for direct retrieval and loading of the corresponding algorithm from the fixed algorithm dictionary each time a service is used, avoiding the need to deploy the corresponding algorithm for each service each time. This reduces development workload, improves development efficiency and overall response time, and enhances user experience.

[0089] It should be noted that the method in this application embodiment can be executed by a single device, such as a computer or server. The method in this application embodiment can also be applied in a distributed scenario, where multiple devices cooperate to complete the process. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this application embodiment, and the multiple devices will interact with each other to complete the method described.

[0090] It should be noted that the above description describes specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0091] Based on the same concept, corresponding to the methods of any of the above embodiments, this application also provides an algorithm deployment apparatus.

[0092] refer to Figure 5 The algorithm deployment device includes:

[0093] The acquisition module 510 is used to acquire the target algorithm.

[0094] The first generation module 520 is used to add a first function to the target algorithm according to a preset framework to generate an algorithm class; wherein, the preset framework is used to determine the way the first function is added to the target algorithm, and the first function is used to load, release and perform calculation inference on the target algorithm.

[0095] The determining module 530 is used to determine at least one service corresponding to the target algorithm; wherein the at least one service is at least one function that uses the target algorithm.

[0096] The second generation module 540 is used to determine the common parts and the distinct parts of the at least one service, establish the subordinate relationship between the common parts and the distinct parts, generate service classes, and establish the correspondence between the algorithm class and the service class.

[0097] The response module 550 is used to respond to a request for the service class, determine and deploy the corresponding algorithm class, and determine the data conversion method of the request based on the corresponding common part and the different part.

[0098] In some exemplary embodiments, the first function includes at least one of the following: a first sub-function for loading the algorithm model, reading the configuration file and initializing the algorithm; a second sub-function for releasing algorithm resources; and a third sub-function for algorithm computation and inference.

[0099] In some exemplary embodiments, the second generation module 540 is further configured to:

[0100] The at least one service is parsed to determine the base class of the at least one service as the common part and the subclass of the at least one service as the distinguishing part; wherein the base class is the common part determined among the at least one service and the subclass is the different part determined among the at least one service.

[0101] In some exemplary embodiments, the second generation module 540 is further configured to:

[0102] The algorithm class is stored in the algorithm storage unit; wherein, the algorithm storage unit is at least used to store algorithm classes formed by different target algorithms;

[0103] The service class is stored in the service storage unit; wherein the service storage unit is at least used to store the service classes formed corresponding to different target algorithms.

[0104] In some exemplary embodiments, the response module 550 is further configured to:

[0105] The algorithm corresponding to the algorithm class is retrieved and loaded from the algorithm storage unit;

[0106] The service class is determined in the service storage unit, and resources are configured according to the common part and the different part corresponding to the service class.

[0107] In some exemplary embodiments, the response module 550 is further configured to:

[0108] Generate a task identifier for the algorithm class and determine at least one process capable of executing the algorithm class;

[0109] The algorithm class and the task identifier are added to the shared waiting queue of the at least one process in the form of a task, and feedback information is returned.

[0110] In some exemplary embodiments, the response module 550 is further configured to:

[0111] In response to any process completing its current task, the next task of any process is obtained sequentially according to the shared waiting queue.

[0112] In some exemplary embodiments, the response module 550 is further configured to:

[0113] Output the processing result of the current task to the shared result queue;

[0114] Based on the task identifier corresponding to the current task, establish the correspondence between the processing result and the corresponding algorithm class and service class.

[0115] In some exemplary embodiments, the response module 550 is further configured to:

[0116] Determine the number of processes that need to be executed at the same time, and determine the number of processors capable of executing the at least one process;

[0117] Tasks are evenly distributed based on the number of processes and the number of processors.

[0118] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware.

[0119] The apparatus described above is used to implement the corresponding algorithm deployment method in the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0120] Based on the same concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the algorithm deployment method as described in any of the above embodiments.

[0121] Figure 6 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0122] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0123] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0124] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0125] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0126] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0127] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0128] The electronic devices described above are used to implement the corresponding algorithm deployment methods in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0129] Based on the same concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to execute the algorithm deployment method as described in any of the above embodiments.

[0130] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0131] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the algorithm deployment method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0132] Based on the same concept, corresponding to the methods of any of the above embodiments, this application also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processors to perform the algorithm deployment method. Corresponding to the execution entity for each step in each embodiment of the algorithm deployment method, the processor executing the corresponding step may belong to the corresponding execution entity.

[0133] The computer program products of the above embodiments are used to cause the computer and / or the processor to execute the algorithm deployment method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0134] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0135] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0136] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0137] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. An algorithm deployment method, characterized in that, include: Obtain the target algorithm; A first function is added to the target algorithm according to a preset framework to generate an algorithm class; wherein, the preset framework is used to determine how the first function is added to the target algorithm, and the first function is used to load, release and perform computational inference on the target algorithm; Identify at least one service corresponding to the target algorithm; wherein, the at least one service is at least one function that uses the target algorithm; Determine the common and distinct parts of the at least one service, establish the subordinate relationship between the common and distinct parts, generate service classes, and establish the correspondence between the algorithm class and the service class; In response to receiving a request for the service class, the corresponding algorithm class is determined and deployed, and the data conversion method of the request is determined based on the corresponding common part and the different part.

2. The method according to claim 1, characterized in that, The first function includes at least one of the following: a first sub-function for loading the algorithm model, reading the configuration file and initializing the algorithm; a second sub-function for releasing algorithm resources; and a third sub-function for algorithm computation and inference.

3. The method according to claim 1, characterized in that, Determining the common and distinct parts of the at least one service includes: The at least one service is parsed to determine the base class of the at least one service as the common part and the subclass of the at least one service as the distinguishing part; wherein the base class is the common part determined among the at least one service and the subclass is the different part determined among the at least one service.

4. The method according to claim 1, characterized in that, After establishing the correspondence between the algorithm class and the service class, the method further includes: The algorithm class is stored in the algorithm storage unit; wherein, the algorithm storage unit is at least used to store algorithm classes formed by different target algorithms; The service class is stored in the service storage unit; wherein the service storage unit is at least used to store the service classes formed corresponding to different target algorithms.

5. The method according to claim 4, characterized in that, After receiving a request for the service class, the method further includes: The algorithm corresponding to the algorithm class is retrieved and loaded from the algorithm storage unit; The service class is determined in the service storage unit, and resources are configured according to the common part and the different part corresponding to the service class.

6. The method according to claim 1, characterized in that, The process of determining and deploying the corresponding algorithm class includes: Generate a task identifier for the algorithm class and determine at least one process capable of executing the algorithm class; The algorithm class and the task identifier are added to the shared waiting queue of the at least one process in the form of a task, and feedback information is returned.

7. The method according to claim 6, characterized in that, After adding the algorithm class to the shared wait queue of the at least one process, the method further includes: In response to any process completing its current task, the next task of any process is obtained sequentially according to the shared waiting queue.

8. The method according to claim 7, characterized in that, The method further includes responding after any process completes its current task: Output the processing result of the current task to the shared result queue; Based on the task identifier corresponding to the current task, establish the correspondence between the processing result and the corresponding algorithm class and service class.

9. The method according to claim 6, characterized in that, After determining at least one process capable of executing the algorithm class, the method further includes: Determine the number of processes that need to be executed at the same time, and determine the number of processors capable of executing the at least one process; Tasks are evenly distributed based on the number of processes and the number of processors.

10. An algorithm deployment device, characterized in that, include: The acquisition module is used to acquire the target algorithm; The first generation module is used to add a first function to the target algorithm according to a preset framework to generate an algorithm class; wherein, the preset framework is used to determine the way the first function is added to the target algorithm, and the first function is used to load, release and perform computational inference on the target algorithm; A determining module is configured to determine at least one service corresponding to the target algorithm; wherein the at least one service is at least one function that uses the target algorithm; The second generation module is used to determine the common parts and the distinct parts of the at least one service, establish the subordinate relationship between the common parts and the distinct parts, generate service classes, and establish the correspondence between the algorithm class and the service class; The response module is used to respond to a request for the service class, determine and deploy the corresponding algorithm class, and determine the data conversion method of the request based on the corresponding common part and the different part.

11. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as claimed in any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing a computer to perform the method according to any one of claims 1 to 9.

Images