Multi-model operation resource optimization method and system based on dynamic queue

Abstract

The invention discloses a multi-model running resource optimization method based on a dynamic queue. The method comprises the steps that a user calls a model, a calling task of the model enters a task queue, a resource controller monitors residual resources of the dynamic queue in real time, and the calling task of the model is scheduled and controlled to enter the dynamic queue from the task queue to run; calling a corresponding model container to operate through a trigger according to the calling task entering the dynamic queue to operate, and entering a model operation queue to execute; when the resource controller monitors that the residual resources of the dynamic queue are lower than a threshold value in real time, the model container at the tail of the dynamic queue is removed according to the least recently used principle, the resources are released, and resource optimization during simultaneous operation of multiple models is achieved. According to the method and system, calling of more models can be achieved under the condition that resources are limited, and the resource utilization rate is increased.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer, in particular to a resource optimization strategy for multi-model containerized deployment and running. BACKGROUND

[0002] The current existing deployment and running mode of multiple algorithm models usually includes the following two schemes:

[0003] I. Traditional multi-model deployment and running mode: the traditional multi-model deployment usually pre-allocates fixed resources for each model, and its technical means mainly relies on the resource management function of the server operating system.

[0004] II. Multi-model deployment and running mode based on container technology: each model and its dependencies are packaged into an independent container to realize environment isolation. The principle is to utilize the lightweight and portability of the container, as well as the automatic scheduling capability of the container orchestration tool.

[0005] The traditional method of pre-allocating fixed resources for each model has obvious defects. When the workload of a certain model suddenly increases in a certain period of time, and the allocated resources cannot meet the demand, the model running efficiency will be greatly reduced. If other models are in an idle state, the resources allocated to them cannot be fully utilized, resulting in low overall resource utilization. Although using containers and container orchestration tools to deploy multiple models can improve the situation of low overall resource utilization, this method also has some shortcomings. The running of the container itself consumes additional resources, and if the number of containers running in the background is very large, it will increase a lot of resource overhead. And in the case of limited resources, the number of models that can be successfully deployed and run is also limited. However, if the model is called and run once and then completely releases the container resources, for some frequently accessed models, starting and killing containers is also a big expense, which will seriously affect the user experience.

[0006] In summary, there is an urgent need to propose a new method of introducing a resource monitoring controller and a dynamic queue to realize the simultaneous deployment and calling of multiple models. SUMMARY

[0007] In order to solve the problem of limited resources in the prior art that the number of model containers that can be run is limited, a method of introducing a resource monitoring controller and a dynamic queue is proposed to realize the simultaneous deployment and running of multiple models.

[0008] In a first aspect, the embodiments of the present application provide a multi-model running resource optimization method based on a dynamic queue, which comprises:

[0009] A dynamic queue is created to control the multiple models running simultaneously to occupy resources, a resource monitoring controller is created to control the maximum number of model containers running simultaneously, and a trigger is set in the model container;

[0010] A user calls a model, the calling task of the model enters a task queue, a resource controller monitors the remaining resources of the dynamic queue in real time, and the calling task of the control model is scheduled from the task queue to the dynamic queue to run;

[0011] The calling task running in the dynamic queue is awakened by the trigger to run the corresponding model container to enter the model running queue for execution.

[0012] When the resource controller monitors the remaining resources of the dynamic queue in real time and the remaining resources are lower than a threshold value, the model container at the tail of the dynamic queue is removed according to the least recently used principle, and the resources are released, so that the resources are optimized when multiple models run simultaneously.

[0013] In the embodiment of the application, the resource monitoring controller monitors the queue information in the dynamic queue and obtains the server resource information through the heartbeat mechanism to calculate the available resources of the server.

[0014] In the embodiment of the application, the dynamic queue is used to control the number of models running simultaneously; the dynamic queue is used for the resources occupied by all running models, the model container entering the dynamic queue is run, and the model container out of the queue releases the resources, so that multiple models run simultaneously.

[0015] In the embodiment of the application, the resource monitoring controller is used to control the maximum number of model containers running simultaneously; by monitoring the resource condition of the computing server, whether to increase the number of model containers running simultaneously is controlled in real time.

[0016] In the embodiment of the application, the trigger is added in the model container, so that when the model in the model container is accessed and triggered, the trigger updates the access time, and the model container is put at the head of the dynamic queue.

[0017] In the embodiment of the application, the task queue includes a task id, a container image of the model, a required storage capacity and a required memory capacity; the dynamic queue includes a container id, occupied storage and occupied memory; and the model running queue includes a container id, a model running start time and a model running end time.

[0018] In a second aspect, the embodiment of the application provides a multiple model running resource optimization system based on a dynamic queue, which adopts the multiple model running resource optimization method based on the dynamic queue, and the system includes:

[0019] Data preparation module: create dynamic queue to control the number of models running simultaneously, create resource monitoring controller to control the maximum number of model containers running simultaneously, and set trigger in model container;

[0020] Resource control module: user calls model, model calling task enters task queue, resource monitoring controller monitors the remaining resources of dynamic queue in real time, and schedules the calling task of the model to enter the dynamic queue from the task queue to run;

[0021] Model running module: the calling task running in the dynamic queue is awakened by the trigger to run the corresponding model container, and enters the model running queue for execution;

[0022] Queue adjustment module: when the resource controller monitors the remaining resources of the dynamic queue in real time and the remaining resources are lower than the threshold, the model container at the tail of the dynamic queue is removed according to the least recently used principle, and the resources are released, so as to realize the resource optimization of multiple models running simultaneously.

[0023] In the embodiment of the application, the resource monitoring controller monitors the queue information in the dynamic queue and obtains the server resource information through the heartbeat mechanism to calculate the available resources of the server.

[0024] In a third aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the steps of the multi-model running resource optimization method based on the dynamic queue.

[0025] In a fourth aspect, an embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the multi-model running resource optimization method based on the dynamic queue when executing the program.

[0026] Compared with the related prior art, the present application has the following outstanding beneficial effects:

[0027] 1) The method of the present application introduces a dynamic queue to control the number of models running simultaneously; the dynamic queue represents the resources occupied by all running models, and by making the model container entering the dynamic queue run and the model container out of the queue release resources, more models can be run.

[0028] 2) The method of the present application uses a resource monitoring controller to control the maximum number of model containers running simultaneously; by monitoring the resource condition of the server, it is determined whether to increase the number of model containers running simultaneously;

[0029] 3) The method of the present application introduces a trigger in the model container to realize LRU; when the model in the container is accessed, the trigger is triggered to update the access time, and the model container is put at the head of the dynamic queue. Attached Figure Description

[0030] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0031] Figure 1 This is a schematic diagram of the multi-model runtime resource optimization method based on dynamic queues according to the present invention;

[0032] Figure 2 This is a schematic diagram of a multi-model runtime resource optimization method based on dynamic queues according to an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram of the resource monitoring heartbeat mechanism method according to an embodiment of the present invention;

[0034] Figure 4 These are schematic diagrams of various queues in embodiments of the present invention;

[0035] Figure 5 This is a schematic diagram of the multi-model runtime resource optimization system based on dynamic queues according to the present invention;

[0036] Figure 6 This is a schematic diagram of the computer hardware of the present invention. Detailed Implementation

[0037] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0038] It should also be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0039] It should also be understood that, in various embodiments of the present invention, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0040] In several embodiments of the present application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely schematic, and the division of units is merely a logical function division. Actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices, or units, and can be electrical, mechanical, or other forms.

[0041] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0042] In addition, the functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present alone, or two or more units can be integrated into one unit.

[0043] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0044] In order to make the above features and effects of the present application more clear and easy to understand, the following embodiments are specifically described, and the detailed description is made in conjunction with the accompanying drawings. The present specification discloses one or more embodiments containing the features of the present application. The disclosed embodiments are only used for illustration. The protection scope of the present application is not limited to the disclosed embodiments, and the present application is defined by the appended claims.

[0045] The following is a system embodiment corresponding to the above method embodiment. The present embodiment can be implemented in cooperation with the above embodiments. The related technical details mentioned in the above embodiments are still valid in the present embodiment. In order to reduce repetition, they will not be described here. Correspondingly, the related technical details mentioned in the present embodiment can also be applied in the above embodiments.

[0046] In the process of researching the background architecture method of the agricultural algorithm model platform, it is found that the prior art cannot meet the simultaneous running of more models under limited computing resources. If the launched model reaches the upper limit of resources, new model algorithms cannot be started. However, resources are always limited, and there may be hundreds or thousands of model algorithms on the algorithm model platform. If the existing method is not improved and coordinated, not only many model containers cannot be started, but also unnecessary waste of resources is easily caused.

[0047] The present application has found through repeated practice and research that the defect can be solved by introducing a resource monitoring controller and a dynamic queue to realize the simultaneous deployment and calling of multiple models.

[0048] The present application aims to introduce a method of resource monitoring controller and dynamic queue to realize the simultaneous deployment and running of multiple models. The method of the present application not only enables more models to be called under limited resources, but also improves resource utilization.

[0049] The system of the present application will be described in detail below in combination with specific embodiments:

[0050] Embodiment one

[0051] As shown in Figure 1 , the present application proposes a multi-model running resource optimization method based on a dynamic queue. The method comprises:

[0052] Step 101, respectively creating a dynamic queue to control the resource occupied by multiple models running at the same time, creating a resource monitoring controller to control the maximum number of model containers running at the same time, and setting a trigger in the model container;

[0053] Step 102, the user calls the model, and the calling task of the model enters the task queue. The resource controller monitors the remaining resources of the dynamic queue in real time, and schedules the calling task of the model to enter the dynamic queue from the task queue for running;

[0054] Step 103, the calling task running in the dynamic queue calls the corresponding model container to run through the trigger, and enters the model running queue for execution;

[0055] Step 104, when the resource controller monitors the dynamic queue in real time and finds that the remaining resources are lower than the threshold value, the model container at the tail of the dynamic queue is removed according to the least recently used principle, and the resources are released, so as to realize the resource optimization of the multi-model simultaneous running time.

[0056] Specifically, as shown in the embodiment of the present application, Figure 2 As shown in the embodiment of the present application,

[0057] In the embodiment of the present application, the resource monitoring controller monitors the queue information in the dynamic queue and obtains the server resource information through the heartbeat mechanism to calculate the available resources of the server.

[0058] Specifically, as shown in the embodiment of the present application, Figure 3 As shown in the embodiment of the present application,

[0059] In the embodiment of the present application, the dynamic queue is used to control the number of simultaneously running models; the dynamic queue represents the resources occupied by all running models, and by making the model containers entering the dynamic queue run and the model containers out of the queue release resources, the simultaneous running of multiple models can be realized.

[0060] Specifically, in the embodiment of the present application, the dynamic queue is introduced to control the number of simultaneously running models; the dynamic queue represents the resources occupied by all running models, and by making the model containers entering the dynamic queue run and the model containers out of the queue release resources, the simultaneous running of multiple models can be realized.

[0061] In the embodiment of the present application, the resource monitoring controller controls the maximum number of simultaneously running model containers; by monitoring the resource condition of the computing server, whether to increase the number of simultaneously running model containers is controlled in real time.

[0062] Specifically, in the embodiments of the present application, a resource monitoring controller is used to control the maximum number of simultaneously running model containers; by monitoring the resource conditions of the server, it is determined whether to increase the number of simultaneously running model containers.

[0063] In the embodiments of the present application, the above-mentioned is achieved by adding a trigger to the model container, so that when the model in the model container is accessed and triggered, the trigger updates the access time, and the model container is placed at the head of the dynamic queue.

[0064] Specifically, in the embodiments of the present application, LRU is achieved by adding a trigger to the model container; when the model in the container is accessed and triggered by the access time update trigger, and the model container is placed at the head of the dynamic queue.

[0065] In the embodiments of the present application, the above-mentioned task queue includes: task id, container image of the model, required storage capacity and required memory capacity; the dynamic queue includes: container id, occupied storage and occupied memory; the model running queue includes: container id, model running start time and model running end time.

[0066] Specifically, in the embodiments of the present application, as shown in Figure 4 The task queue, dynamic queue and model running queue are shown in the following figures, respectively.

[0067] As described above, the system of the present application can be better implemented.

[0068] Compared with the prior art, the present application has the following outstanding advantages and beneficial effects:

[0069] The present application only needs to describe the things you want to do in natural language, and it will help you operate web pages, mobile applications and computer applications. The present application can make the UI interaction more convenient and efficient. For example, for the elderly, some people with low education level, or people with missing fingers, it is very difficult and complex to interact with UI by clicking. They prefer to use voice or handwritten natural language. The above-mentioned application will greatly reduce the complexity and difficulty of UI interaction.

[0070] Embodiment two

[0071] As shown in Figure 5 The embodiments of the present application provide a multi-model running resource optimization system based on a dynamic queue, which adopts the above-mentioned multi-model running resource optimization method based on a dynamic queue, and the system includes:

[0072] Data preparation module 201: create dynamic queue to control multiple models running at the same time to occupy resources, create resource monitoring controller to control the maximum number of model containers running at the same time, and set trigger in the model container;

[0073] Resource control module 202: user calls a model, the calling task of the model enters the task queue, the resource monitoring controller monitors the remaining resources of the dynamic queue in real time, and schedules the calling task of the control model to enter the dynamic queue from the task queue to run;

[0074] Model running module 203: the calling task running in the dynamic queue is awakened by the trigger to run the corresponding model container, and enters the model running queue for execution;

[0075] Queue adjustment module 204: when the resource controller monitors the remaining resources of the dynamic queue in real time and is lower than the threshold value, the model container at the tail of the dynamic queue is removed according to the least recently used principle, and the resources are released, so as to realize the resource optimization of multiple models running at the same time.

[0076] In the embodiment of the application, the resource monitoring controller monitors the queue information in the dynamic queue and obtains the server resource information through the heartbeat mechanism to calculate the available resources of the server.

[0077] Embodiment three

[0078] The embodiment of the application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the steps of the multi-model running resource optimization method based on the dynamic queue.

[0079] Embodiment four

[0080] The embodiment of the application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor realizes the steps of the multi-model running resource optimization method based on the dynamic queue when executing the program.

[0081] In addition, the multi-model running resource optimization method based on the dynamic queue of the embodiment of the application can be implemented by an electronic device, such as a computer device. Figure 1 The multi-model running resource optimization method based on the dynamic queue of the embodiment of the application described in the embodiment of the application can be implemented by an electronic device, such as a computer device. Figure 6 The hardware structure of the computer device according to the embodiment of the application is shown in the figure.

[0082] In some embodiments, the computer device can further include a communication interface 83 and a bus 80. As shown in the figure, Figure 6 The processor 81, the memory 82, the communication interface 83 are connected through the bus 80 and complete the communication between each other.

[0083] Specifically, the processor 81 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or can be configured to implement one or more integrated circuits that embody the embodiments of the present application.

[0084] The memory 82 can be used to store or buffer various data files that need to be processed and / or communicated, and possible computer program instructions executed by the processor 81.

[0085] The processor 81 realizes any one of the above-mentioned embodiments of the dynamic queue-based multi-model running resource optimization method by reading and executing the computer program instructions stored in the memory 82.

[0086] The technical features of the above-mentioned embodiments can be combined in any manner. In order to make the description simple, all possible combinations of the technical features in the above-mentioned embodiments are not described, however, as long as the combinations of the technical features do not contradict, they should be considered as the scope of the present application.

[0087] The above-mentioned embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. A dynamic queue-based multi-model running resource optimization method, characterized in that, The method includes: Dynamic queues are created to control the resource consumption of multiple models running simultaneously, and a resource monitoring controller is created to control the maximum number of model containers running simultaneously, and triggers are set in the model containers. When a user invokes a model, the invocation task of the model enters a task queue. The resource controller monitors the remaining resources of the dynamic queue in real time and schedules the invocation task of the model to enter the dynamic queue from the task queue for execution. The calling task that enters the dynamic queue for execution is triggered by the trigger to start the corresponding model container and enter the model execution queue for execution. When the resource controller monitors in real time that the remaining resources of the dynamic queue are below the threshold, it removes the tail model container of the dynamic queue according to the least recent use principle and releases the resources, thereby optimizing resources when multiple models are running simultaneously.

2. The method of claim 1, wherein, The resource monitoring controller monitors queue information in the dynamic queue and obtains server resource information through a heartbeat mechanism in order to calculate the available resources of the server.

3. The multi-model running resource optimization method based on dynamic queues according to claim 1 or 2, characterized in that, The dynamic queue is used to control the number of models running simultaneously. The dynamic queue is the resource occupied by all running models. By making the model containers that enter the dynamic queue run and the model containers that leave the queue release resources, multiple models can run simultaneously.

4. The multi-model running resource optimization method based on dynamic queues according to claim 1 or 2, characterized in that, The resource monitoring controller controls the maximum number of model containers running simultaneously; by monitoring the resource status of the computing server, it controls in real time whether to increase the number of model containers running simultaneously.

5. The multi-model running resource optimization method based on dynamic queues according to claim 1 or 2, characterized in that, By adding the trigger to the model container, when a model in the model container is accessed and triggered, the trigger updates the access time and puts the model container into the head of the dynamic queue.

6. The multi-model running resource optimization method based on dynamic queues according to claim 1 or 2, characterized in that, The task queue includes: task ID, model container image, required storage capacity, and required memory capacity; the dynamic queue includes: container ID, occupied storage, and occupied memory; the model running queue includes: container ID, model running start time, and model running end time.

7. A dynamic queue based multi-model run resource optimization system employing the dynamic queue based multi-model run resource optimization method according to any one of claims 1-6. The system includes: Data preparation module: Create dynamic queues to control the resource consumption of multiple models running simultaneously, create a resource monitoring controller to control the maximum number of model containers running simultaneously, and set triggers in the model containers; Resource control module: When a user calls a model, the calling task of the model enters the task queue. The resource monitoring controller monitors the remaining resources of the dynamic queue in real time and schedules the calling task of the model to enter the dynamic queue from the task queue for execution. Model execution module: The called task that enters the dynamic queue for execution is triggered by the trigger to start the corresponding model container and enter the model execution queue for execution; Queue Adjustment Module: When the resource controller monitors in real time that the remaining resources of the dynamic queue are lower than the threshold, it removes the tail model container of the dynamic queue according to the least recent use principle and releases the resources, thereby optimizing resources when multiple models are running simultaneously.

8. The multi-model running resource optimization system based on dynamic queues according to claim 7, characterized in that, The resource monitoring controller monitors queue information in the dynamic queue and obtains server resource information through a heartbeat mechanism in order to calculate the available resources of the server.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the multi-model runtime resource optimization method based on dynamic queues as described in any one of claims 1 to 6.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the multi-model runtime resource optimization method based on dynamic queues as described in any one of claims 1 to 6.

Images