Algorithm container management method and device, equipment and storage medium
By working together with the container scheduler and container agent, the scheduling and management of algorithm containers are decoupled, solving the problem of high maintenance costs in existing technologies and improving the system's flexibility and security.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-09
- Publication Date
- 2026-03-13
AI Technical Summary
In existing algorithm containerization solutions, the scheduling and lifecycle management of algorithm containers are highly correlated, resulting in high maintenance costs.
The container scheduler determines the target server node and sends container scheduling instructions to the container agent corresponding to the target server node, enabling the container agent to perform lifecycle management through container local sockets, thereby decoupling container scheduling and management.
It reduces system maintenance costs, improves system flexibility, maintainability and stability, and avoids security risks caused by exposed interfaces.
Smart Images
Figure CN121658153A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to algorithm container management methods, apparatus, devices and storage media. Background Technology
[0002] With the rapid development of deep learning and artificial intelligence technologies, image and video analysis algorithms are being used more and more widely. To efficiently deploy and manage these algorithms, containerization technology has become an important approach. Containerization technology encapsulates algorithms and their dependent environments into containers, enabling rapid deployment and flexible management of algorithms.
[0003] Currently, in existing algorithm containerization solutions, the scheduling and lifecycle management of algorithm containers are usually highly correlated. This means that if the lifecycle strategy or scheduling logic needs to be adjusted, the entire system often needs to be modified on a large scale, which increases maintenance costs. Summary of the Invention
[0004] The main purpose of this application is to provide an algorithm container management method, apparatus, device and storage medium, which aims to solve the technical problem that the scheduling and management of algorithm containers are highly correlated in the existing algorithm containerization schemes, resulting in high maintenance costs.
[0005] To achieve the above objectives, this application proposes an algorithm container management method. The method is applied to a container management platform, which includes an algorithm orchestration service, a container scheduler, and a container agent, comprising: In response to a task processing request, the target algorithm container required to execute the user task is determined through the algorithm orchestration service; The container scheduler determines the target server node to run the target algorithm container based on a preset container scheduling algorithm. The container scheduler sends container scheduling instructions to the container agent corresponding to the target server node, so that the container agent can manage the lifecycle of the target algorithm container through the container local socket.
[0006] In one embodiment, the step of determining the target server node for running the target algorithm container by the container scheduler based on a preset container scheduling algorithm includes: The container scheduler obtains a list of server information and checks whether each server node in the list meets the preset service conditions. Based on the inspection results, server nodes that meet the preset service conditions are identified as candidate server nodes; The container scheduler determines the target server node from the candidate server nodes to run the target algorithm container based on a preset container scheduling algorithm.
[0007] In one embodiment, the step of checking whether each server node in the server information list meets the preset service conditions includes: Obtain the network address of each server node in the server information list; Determine whether the network address is within a preset whitelist range; If so, check whether there are any idle graphics processing resources in each of the server nodes; If they exist, then determine whether each server node meets the task requirements of the user task based on the node label corresponding to each server node. If the conditions are met, then each server node is determined to meet the preset service conditions.
[0008] In one embodiment, the step of determining the target server node for running the target algorithm container from the candidate server nodes by the container scheduler based on a preset container scheduling algorithm includes: The container scheduler determines the memory resource utilization and graphics processing resource usage of the candidate server nodes based on the server information list. Based on the memory resource utilization rate and the graphics processing resource occupancy rate, the node score corresponding to the candidate server node is determined by a preset container scheduling algorithm. The target server node for running the target algorithm container is determined from the candidate server nodes based on the node score.
[0009] In one embodiment, the step of responding to a task processing request and determining the target algorithm container required to execute a user task via the algorithm orchestration service includes: In response to a task processing request, the algorithm orchestration service determines the task execution information corresponding to the user task based on the task processing request, and the task execution information includes an algorithm identifier. The target algorithm required to execute the user task is determined based on the algorithm identifier; Based on the target algorithm and the task requirements of the user task, determine the target algorithm container required to execute the user task.
[0010] In one embodiment, after the step of sending a container scheduling instruction to the container agent corresponding to the target server node through the container scheduler, so that the container agent can perform lifecycle management of the target algorithm container through the container local socket, the method further includes: The target algorithm is run through the target algorithm container to obtain the algorithm execution result; Determine the callback method for the algorithm's execution result; The algorithm execution result is sent back to the algorithm orchestration service using the callback method, so that the algorithm orchestration service can aggregate the algorithm execution result and generate the task processing result of the user task.
[0011] In one embodiment, the method further includes: In response to a video processing request, the algorithm orchestration service determines the storage location and frame extraction frequency of the video to be processed based on the video processing request. The video to be processed is obtained according to the storage location; Start the frame extraction thread, and extract frames from the video to be processed according to the frame extraction frequency to obtain a number of video frames. The video analysis algorithm container is invoked by the container scheduler to process each video frame and obtain the video processing result.
[0012] Furthermore, to achieve the above objectives, this application also proposes an algorithm container management device, wherein the device includes a container management platform, and the container management platform includes an algorithm orchestration service, a container scheduler, and a container agent, comprising: The request processing module is used to respond to task processing requests and determine the target algorithm container required to execute user tasks through the algorithm orchestration service. The scheduling decision module is used to determine the target server node for running the target algorithm container based on a preset container scheduling algorithm through the container scheduler; The container management module is used to send container scheduling instructions to the container agent corresponding to the target server node through the container scheduler, so that the container agent can perform lifecycle management of the target algorithm container through the container local socket.
[0013] In addition, to achieve the above objectives, this application also proposes an algorithm container management device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the algorithm container management method as described above.
[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the algorithm container management method described above.
[0015] This application provides an algorithm container management method. The method responds to a task processing request and determines the target algorithm container required to execute the user task through an algorithm orchestration service; determines the target server node to run the target algorithm container based on a preset container scheduling algorithm using a container scheduler; and sends container scheduling instructions to the container agent corresponding to the target server node through the container scheduler, enabling the container agent to manage the target algorithm container through container local sockets. Because this invention can determine the target server node to run the target algorithm container through the container scheduler and send container scheduling instructions to the container agent corresponding to the target server node, enabling the container agent to manage the container through container local sockets, it achieves decoupling of container scheduling and management. This solves the technical problem in existing algorithm containerization schemes where the scheduling and management of algorithm containers are highly correlated, leading to high maintenance costs. Attached Figure Description
[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a flowchart illustrating an embodiment of the algorithm container management method of this application. Figure 2 This is an architecture diagram of the container management platform in the algorithm container management method of this application; Figure 3 This is a flowchart illustrating the algorithm container startup process in the algorithm container management method of this application. Figure 4 This is a flowchart illustrating Embodiment 2 of the algorithm container management method of this application; Figure 5 This is a flowchart illustrating Embodiment 3 of the algorithm container management method of this application; Figure 6 This is a flowchart illustrating the user task processing in the algorithm container management method of this application. Figure 7 This is a schematic diagram of the module structure of the algorithm container management device according to an embodiment of this application; Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the algorithm container management method in this application embodiment.
[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.
[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.
[0022] The main solution of this application embodiment is: responding to a task processing request and determining the target algorithm container required to execute the user task through an algorithm orchestration service; determining the target server node to run the target algorithm container through a container scheduler based on a preset container scheduling algorithm; and sending a container scheduling instruction to the container agent corresponding to the target server node through the container scheduler, so that the container agent can manage the target algorithm container through the container local socket.
[0023] Because the scheduling and lifecycle management of algorithm containers in existing algorithm containerization solutions are usually highly correlated, once the lifecycle strategy or scheduling logic needs to be adjusted, it often requires large-scale modifications to the entire system, thereby increasing maintenance costs.
[0024] This application provides a solution that can determine the target server node for running the target algorithm container through a container scheduler and send container scheduling instructions to the container agent corresponding to the target server node, so that the container agent can manage the container through the container local socket. This decouples container scheduling and management, thereby solving the technical problem that the scheduling and management of algorithm containers are highly correlated in the existing algorithm containerization schemes, resulting in high maintenance costs.
[0025] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device, algorithm container management device, or container management platform capable of implementing the above functions. The following description uses a container management platform as an example to illustrate this embodiment and the subsequent embodiments.
[0026] Based on this, embodiments of this application provide an algorithm container management method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the algorithm container management method of this application.
[0027] In this embodiment, the algorithm container management method is applied to a container management platform, which includes an algorithm orchestration service, a container scheduler, and a container agent, and includes steps S10 to S30: Step S10: In response to the task processing request, determine the target algorithm container required to execute the user task through the algorithm orchestration service.
[0028] It should be noted that, referring to Figure 2 , Figure 2 This is an architecture diagram of the container management platform in the algorithm container management method of this application. Figure 2 As shown, the container management platform in this application includes an algorithm orchestration service, a container scheduler, and a container proxy. The algorithm orchestration service is a component responsible for receiving user requests, orchestrating tasks, coordinating downstream services, and ultimately returning a normalized result. It can seamlessly integrate with any third-party algorithm, adapting to varying input and output parameters. The container scheduler is a component responsible for resource-aware scheduling decisions. The container proxy is a proxy module deployed on each slave node, responsible for managing the lifecycle of the algorithm containers. In this embodiment, through... Figure 2 The layered architecture and modular design shown enable the algorithm orchestration service, container scheduler, and container agent to each assume different responsibilities, thereby decoupling container scheduling from container lifecycle management and improving the system's flexibility, maintainability, and stability.
[0029] It should be understood that the above-mentioned task processing request can be a request used to instruct the container management platform to process a task. The request may include the task that the user expects to execute and its corresponding details, such as the resource requirements of the algorithm container (e.g., CPU (Central Processing Unit), GPU (Graphics Processing Unit), memory, etc.), algorithm parameters (e.g., algorithm configuration parameters), input data format and storage location, etc. This embodiment does not limit these.
[0030] It is understandable that the user tasks mentioned above can be specific tasks that the user expects the container management platform to perform, such as object detection in a video or classification of an image.
[0031] It should be noted that the aforementioned target algorithm container can be a specific algorithm container used to execute user tasks, which can encapsulate a specific algorithm and its dependent environment, and be able to process user-submitted tasks. In this embodiment, the target algorithm container can be selected according to the needs of the user task, and it can be a combination of one or more algorithm containers. For example, if the user task is video object detection, the target algorithm container can be a Docker container that encapsulates an object detection algorithm.
[0032] Step S20: The container scheduler determines the target server node to run the target algorithm container based on a preset container scheduling algorithm.
[0033] It should be noted that the aforementioned preset container scheduling algorithm can be an algorithm used to determine the server node to run the target algorithm container. In practical applications, the preset container scheduling algorithm can be based on a multi-dimensional resource awareness strategy, which can consider various factors such as CPU, GPU, memory, network bandwidth, and user source IP, thereby selecting the optimal node to run the algorithm container. For example, the scheduling algorithm can prioritize nodes with low resource utilization, or select the node closest to the user based on the user's source IP.
[0034] It should be noted that the target server node mentioned above can be a physical or virtual server used to run the target algorithm container. In this embodiment, the target server node needs to have the resources required to run the algorithm container. For example, the target server node may be a server with a high-performance GPU, suitable for running algorithm containers that require a large amount of computing resources.
[0035] In practical applications, users can submit task processing requests through a client. These requests include detailed information about the user's task, such as the resource requirements of the algorithm container, algorithm parameters, and the storage location of the input data. Upon receiving the task processing request, the algorithm orchestration service preprocesses the input data (e.g., format conversion, frame extraction) and determines the target algorithm container needed to execute the user's task based on the user's requirements. Then, it sends the resource requirements of the target algorithm container to the container scheduler. This allows the scheduler to select the optimal server node as the target server node based on a pre-defined container scheduling algorithm, resource requirements, and a comprehensive consideration of the current system's resource status and the specific needs of the user's task (e.g., user source IP address, algorithm context requirements).
[0036] Step S30: Send a container scheduling instruction to the container agent corresponding to the target server node through the container scheduler, so that the container agent can perform lifecycle management of the target algorithm container through the container local socket.
[0037] It should be understood that the aforementioned container scheduling instructions can be instructions used to instruct the container agent to perform specific operations, such as starting, stopping, monitoring, or reclaiming algorithm containers. Specifically, the start instruction can be an instruction instructing the container agent to pull the image of the target algorithm container and start the container on the target server node; the stop instruction can be an instruction instructing the container agent to stop the target algorithm container that is currently running; the monitoring instruction can be an instruction instructing the container agent to monitor the running status of the target algorithm container and report status information periodically; and the reclamation instruction can be an instruction instructing the container agent to reclaim the target algorithm container that has completed its tasks and release system resources.
[0038] It should be noted that the aforementioned container local socket can be a Unix socket used for communication between the container agent and the local Docker daemon. It allows the container agent to interact directly with the Docker daemon without exposing Docker's remote interface over the network. In this embodiment, the container management platform can allow slave nodes to communicate only with the local Docker socket through the container agent, without exposing Docker's RESTful interface. This fundamentally eliminates the risk of malicious access due to interface exposure, and external scheduling requests can only be distributed to the container agent through the algorithm orchestration service, thus avoiding unauthorized operations. Compared to existing solutions that directly expose the Docker API, this significantly improves system security.
[0039] It should be understood that lifecycle management of the target algorithm container refers to managing the entire process of the target algorithm container from creation to destruction. This process can include operations such as container creation, startup, operation, monitoring, stopping, and recycling. In this embodiment, the container agent can communicate with the Docker daemon through a local socket to perform various container management operations. For example, the container agent can send a request to the Docker daemon through a local socket to pull the image of the target algorithm container and create a new container instance; the container agent can send a stop command to the Docker daemon through a local socket to stop the running container; the container agent can send a recycling command to the Docker daemon through a local socket to delete containers that have completed their tasks and release system resources.
[0040] In this embodiment, the container management platform can have a container scheduler and a container agent respectively responsible for container scheduling decisions and container lifecycle management. This avoids the drawback of coupling container scheduling logic and lifecycle management logic in the same component, allowing scheduling strategies to be optimized independently without affecting the stability of container management. At the same time, it can reduce system maintenance costs. When scheduling rules or lifecycle strategies need to be adjusted, there is no need to modify the entire system; only the corresponding modules need to be modified.
[0041] In the specific implementation, refer to Figure 3 , Figure 3 This is a flowchart illustrating the startup process of the algorithm container in the algorithm container management method of this application. Figure 3 As shown, users can submit a startup request for an algorithm container through the client. This request can include the resource requirements and algorithm parameters of the algorithm container. Upon receiving the user request, the algorithm orchestration service preprocesses the task data in the request and performs task orchestration before sending the task's resource requirements to the container scheduler. The container scheduler then selects the optimal target server node based on the resource requirements and the current system resource status, and sends container scheduling instructions to the container agent on that target server node. After receiving the container scheduling instructions, the container agent can communicate with the Docker daemon through a local socket to manage the algorithm container's lifecycle, such as performing operations like image pulling, container creation, startup, monitoring, and recycling. Finally, after completing its task, the algorithm container can send the results back to the algorithm orchestration service, which can then aggregate and standardize the results and return them to the user in a unified format.
[0042] In this embodiment, the container agent can register with any node in the container scheduler when it starts up. The nodes share data through a storage middleware (such as Redis) and can maintain a heartbeat with the container scheduler. The heartbeat interval can be set by the container agent when it registers. The heartbeat information can include various hardware indicators and instance details of the current server, so that the container scheduler can make scheduling decisions and provide the container instance details (such as IP) information to the outside world.
[0043] This embodiment provides an algorithm container management method. The method discloses a process that responds to a task processing request and determines the target algorithm container required to execute a user task through an algorithm orchestration service; determines the target server node for running the target algorithm container using a container scheduler based on a preset container scheduling algorithm; and sends container scheduling instructions to the container agent corresponding to the target server node through the container scheduler, enabling the container agent to manage the target algorithm container through a container local socket. Because this embodiment can determine the target server node for running the target algorithm container through the container scheduler and send container scheduling instructions to the container agent corresponding to the target server node, enabling the container agent to manage the container through a container local socket, it achieves decoupling of container scheduling and management. This solves the technical problem in existing algorithm containerization schemes where the scheduling and management of algorithm containers are highly correlated, leading to high maintenance costs.
[0044] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 , Figure 4 This is a flowchart illustrating Embodiment 2 of the algorithm container management method of this application.
[0045] In this embodiment, step S20 includes steps S201 to S203: Step S201: Obtain the server information list through the container scheduler and check whether each server node in the server information list meets the preset service conditions.
[0046] It should be noted that the above server information list can be a collection of detailed information on all available server nodes in the container management platform. This information may include, but is not limited to, server hardware resources (such as CPU, GPU, memory), network status, current load, number of deployed containers, etc.
[0047] It should be understood that the aforementioned preset service conditions can be conditions that the server nodes running the algorithm container need to meet. These conditions may include resource requirements, security policies, affinity / anti-affinity rules, etc. This embodiment does not impose any restrictions on these. Among them, resource requirements can be the CPU, GPU, memory, and other resources that the algorithm container needs to have; security policies can be the whitelist of user source IPs, security group rules, etc.; affinity / anti-affinity rules refer to the fact that some algorithm containers may need to be deployed on specific servers (affinity), or cannot be deployed on certain servers (anti-affinity).
[0048] Specifically, the step of checking whether each server node in the server information list meets the preset service conditions includes: obtaining the network address of each server node in the server information list; determining whether the network address is within the preset whitelist range; if so, checking whether there are idle graphics processing resources in each server node; if so, determining whether each server node meets the task requirements of the user task based on the node tag corresponding to each server node; if so, determining that each server node meets the preset service conditions.
[0049] It is understood that the aforementioned network address refers to a unique identifier of a server node in the network, used to identify the server node's location within the network, such as an IP address or hostname; the aforementioned preset whitelist range can be the set of network addresses that the container scheduler is allowed to access. In this embodiment, the addresses within the preset whitelist range can be pre-configured by the system administrator according to security policies to ensure that only trusted server nodes can participate in task scheduling. For example, the IP whitelist can be set to 192.168.1.0 / 24.
[0050] It should be noted that the aforementioned idle graphics processing resources can be unused GPU resources on server nodes. These GPU resources can be used to run tasks that require high-performance computing, such as image and video analysis tasks.
[0051] It should be noted that the above node labels can be key-value pairs attached to server nodes to identify specific attributes or uses of the nodes, and can be used to identify affinity / anti-affinity rules during task scheduling.
[0052] It should be understood that the aforementioned task requirements can be specific requirements of the user task for the server node, including resource requirements, security policies, affinity / anti-affinity rules, etc. For example, it may require 4 GPU cores, the user's source IP must be within a preset whitelist, or the task needs to be deployed on a server in a specific region. In this embodiment, user requirements can be specified by the user when submitting the task, or automatically generated by the algorithm orchestration service based on the task type.
[0053] In this embodiment, the container scheduler can traverse each server in the server information list and sequentially execute multiple check conditions using a check chain pattern to filter out candidate server nodes that meet the conditions. Each check condition in the check chain is pluggable, allowing for dynamic expansion of the check logic, and each check condition has a single responsibility, making it easy to maintain and reuse.
[0054] In practical applications, the container scheduler first checks whether the network address of each server node in the server information list is within the preset whitelist range. If the network address is not within the whitelist range, the server node will be excluded. If the network address is within the whitelist range, the scheduler checks the GPU resources of the server node to determine whether there are any idle graphics processing resources. If so, it continues to check the node label of the server node to determine whether it meets the task requirements of the user task. If the node label meets the task requirements, the scheduler determines that the server node meets the preset service conditions and identifies it as a candidate server node.
[0055] Step S202: Based on the inspection results, server nodes that meet the preset service conditions are identified as candidate server nodes.
[0056] It is understandable that the aforementioned candidate server nodes can be server nodes that meet the preset service conditions in the server information list. These nodes are potential target server nodes that can run the target algorithm container.
[0057] Step S203: The container scheduler determines the target server node to run the target algorithm container from the candidate server nodes based on a preset container scheduling algorithm.
[0058] In practical applications, the container scheduler can obtain a list of server information from storage middleware (such as Redis) or directly from each server node. Based on the resource requirements of the target algorithm container and preset service conditions, it checks each server node in the list. This includes checking whether the server node meets the CPU, GPU, and memory resource requirements of the target algorithm container; whether the server node conforms to the user's source IP whitelist and security group rules; whether the server node meets affinity / anti-affinity rules; and whether the server node's geographical location and region meet the requirements. Then, based on the check results, the container scheduler identifies server nodes that meet the preset service conditions as candidate server nodes and uses a preset container scheduling algorithm to select the optimal server node from the candidate server nodes as the target server node.
[0059] Further, step S203 includes: determining the memory resource utilization and graphics processing resource occupancy of the candidate server nodes by the container scheduler based on the server information list; determining the node score corresponding to the candidate server nodes by a preset container scheduling algorithm based on the memory resource utilization and graphics processing resource occupancy; and determining the target server node running the target algorithm container from the candidate server nodes based on the node score.
[0060] It should be understood that the above memory resource utilization rate can be the ratio of the memory resources used on the server node to the total memory resources, which can reflect the current memory resource usage of the server node; the above graphics processing resource utilization rate can be the ratio of the GPU resources used on the server node to the total GPU resources, which can reflect the current GPU resource usage of the server node.
[0061] It should be noted that the above node score can be a score reflecting the overall performance and resource availability of a server node. In this embodiment, in addition to memory resource utilization and graphics processing resource utilization, the node score of a server node can also be calculated by taking into account factors such as CPU utilization and network bandwidth.
[0062] In practical applications, the container scheduler can obtain the resource information of each candidate server node from the server information list, and calculate the memory resource utilization and graphics processing resource utilization of each candidate server node based on this information. Then, it scores each candidate server node based on factors such as memory resource utilization and graphics processing resource utilization using a preset container scheduling algorithm, obtains the node score corresponding to each candidate server node, and selects the candidate server node with the highest node score as the target server node.
[0063] In this embodiment, a method is disclosed to obtain a list of server information through a container scheduler and check whether each server node in the list meets preset service conditions; based on the check results, server nodes that meet the preset service conditions are determined as candidate server nodes; the container scheduler determines the target server node to run the target algorithm container from the candidate server nodes based on a preset container scheduling algorithm. Since this embodiment can filter candidate server nodes that meet the preset service conditions from all server nodes and determine the optimal server node as the target server node from the candidate server nodes according to the preset container scheduling algorithm, the target algorithm container can be allocated to the server node with the most abundant resources, thereby improving resource utilization and avoiding resource waste.
[0064] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the above embodiments can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 5 , Figure 5 This is a flowchart illustrating Embodiment 3 of the algorithm container management method of this application.
[0065] In this embodiment, step S10 includes steps S101 to S103: Step S101: In response to the task processing request, the algorithm orchestration service determines the task execution information corresponding to the user task based on the task processing request, wherein the task execution information includes an algorithm identifier.
[0066] It should be understood that the aforementioned algorithm identifier can be an identifier used to uniquely identify the algorithm. In this embodiment, the algorithm identifier of a user task can be specified by the user when submitting a task processing request, or it can be automatically generated by the algorithm orchestration service based on the task type.
[0067] Step S102: Determine the target algorithm required to execute the user task based on the algorithm identifier.
[0068] It is understandable that the target algorithm described above can be a specific algorithm used to perform the user task. For example, if the user task is image classification, then the target algorithm can be an image classification algorithm.
[0069] Step S103: Determine the target algorithm container required to execute the user task based on the target algorithm and the task requirements of the user task.
[0070] In this embodiment, through algorithm identification, the algorithm orchestration service can seamlessly integrate with any third-party algorithm container. This allows for adaptation regardless of the differences in input and output parameters, enabling the system to quickly adapt to new algorithms and business requirements. Furthermore, the algorithm orchestration service can select appropriate target algorithm containers based on the specific needs of user tasks, supporting diverse task requirements and thus improving system flexibility.
[0071] In practical applications, users can submit task processing requests through a client. Upon receiving the request, the algorithm orchestration service can parse it and extract detailed information about the user's task, including the storage location of the input data, algorithm identifier, and algorithm parameters. Then, the algorithm orchestration service can search for the corresponding target algorithm in the algorithm library based on the algorithm identifier and determine the target algorithm container required to execute the user's task based on the target algorithm and the task requirements of the user's task.
[0072] Furthermore, after step S30, the method further includes: running the target algorithm through the target algorithm container to obtain the algorithm running result; determining the callback method for the algorithm running result; and using the callback method to send the algorithm running result back to the algorithm orchestration service, so that the algorithm orchestration service aggregates the algorithm running result and generates the task processing result of the user task.
[0073] It should be understood that the above algorithm running results can be the output data generated by the target algorithm container after running the target algorithm. For example, for an image classification algorithm, the corresponding algorithm running results can be the classification results and confidence scores; for a video target detection algorithm, the corresponding algorithm running results can be the detected target and its location information. This embodiment does not impose any restrictions on this.
[0074] It should be noted that the above callback methods can be the way the target algorithm container sends the algorithm execution results back to the algorithm orchestration service, such as asynchronous callbacks, HTTP callbacks, etc. In the case of asynchronous callbacks, the target algorithm container sends the results to an intermediate storage (such as a message queue or object storage), and then the algorithm orchestration service can obtain the results by polling or listening for events; in the case of HTTP callbacks, the target algorithm container sends the results to a specific endpoint of the algorithm orchestration service through an HTTP request.
[0075] In the specific implementation, refer to Figure 6 , Figure 6 This is a flowchart illustrating the user task processing in the algorithm container management method of this application. Figure 6As shown, users can send the images or videos to be processed, along with their corresponding storage locations, algorithm identifiers, parameters, and other information, to the algorithm orchestration service via a client. Upon receiving the task processing request, the algorithm orchestration service can perform format identification and preprocessing on the input data (such as converting PDF documents to images or performing video frame extraction), transforming the task data into the format required by the corresponding algorithm. This allows the container scheduler to make scheduling decisions. During algorithm decision-making, multiple algorithms can be chained together or scheduled in parallel to achieve more complex business logic. Subsequently, the algorithm container can run the algorithm and send the algorithm's results (or intermediate results) back to the algorithm orchestration service via HTTP callbacks or message queues. The algorithm orchestration service can then aggregate and normalize the results from multiple containers or multiple calls, generating the task processing result, which is then returned to the user.
[0076] Furthermore, the method also includes: Step S100: In response to the video processing request, the storage location and frame skipping frequency of the video to be processed are determined based on the video processing request through the algorithm orchestration service.
[0077] It should be understood that the aforementioned video processing request can be a request used to instruct the algorithm orchestration service to process the input video. The request may include information such as the storage location of the video to be processed and the frame extraction frequency. The storage location can be the storage path or URL of the video file; the frame extraction frequency can be the number of frames extracted from the video per second, such as 1 or 2 frames per second.
[0078] Step S200: Obtain the video to be processed according to the storage location.
[0079] Step S300: Start the frame extraction thread, and extract frames from the video to be processed according to the frame extraction frequency to obtain a number of video frames.
[0080] It should be noted that the aforementioned frame extraction thread can be a thread used to extract frames from a video. In this embodiment, the frame extraction thread can extract frames from the video according to a specified frame extraction frequency and place the frames in a queue for subsequent processing.
[0081] Step S400: The video analysis algorithm container is invoked by the container scheduler to process each video frame and obtain the video processing result.
[0082] It should be noted that the aforementioned video analysis algorithm container can be a container that encapsulates a video analysis algorithm, which can analyze and process video frames.
[0083] In practical applications, users can submit video processing requests through a client. Upon receiving the request, the algorithm orchestration service can parse it, extracting information such as the video's storage location and frame extraction frequency, and then retrieve the video to be processed based on its storage location. After obtaining the video, a frame extraction thread can be started, extracting frames from the video according to the extraction frequency, and then placing the extracted frames into a queue. Subsequently, the container scheduler can call the video analysis algorithm container to process each video frame, aggregating the processing results to generate the final video processing result.
[0084] In this embodiment, a method is disclosed that responds to a task processing request and determines the task execution information corresponding to the user task based on the task processing request through an algorithm orchestration service. The task execution information includes an algorithm identifier. The method then determines the target algorithm required to execute the user task based on the algorithm identifier and the task requirements of the user task. Since the algorithm orchestration service in this embodiment can seamlessly access any third-party algorithm container through the algorithm identifier and can select a suitable target algorithm container according to the requirements of the user task, the scalability and flexibility of the system are improved.
[0085] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the algorithm container management method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.
[0086] This application also provides an algorithm container management device, please refer to... Figure 7 The algorithm container management device includes a container management platform, which comprises an algorithm orchestration service, a container scheduler, and a container agent, including: Request processing module 10 is used to respond to task processing requests and determine the target algorithm container required to execute user tasks through the algorithm orchestration service; The scheduling decision module 20 is used to determine the target server node for running the target algorithm container based on a preset container scheduling algorithm through the container scheduler; The container management module 30 is used to send container scheduling instructions to the container agent corresponding to the target server node through the container scheduler, so that the container agent can perform lifecycle management of the target algorithm container through the container local socket.
[0087] The algorithm container management device provided in this application, employing the algorithm container management method in the above embodiments, can solve the technical problem of high maintenance costs caused by the highly correlated scheduling and management of algorithm containers in existing algorithm containerization schemes. Compared with the prior art, the beneficial effects of the algorithm container management device provided in this application are the same as those of the algorithm container management method provided in the above embodiments, and other technical features in the algorithm container management device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.
[0088] This application provides an algorithm container management device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the algorithm container management method in Embodiment 1 above.
[0089] The following is for reference. Figure 8 This document illustrates a structural schematic diagram of an algorithm container management device suitable for implementing embodiments of this application. The algorithm container management device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 8 The algorithm container management device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0090] like Figure 8As shown, the algorithm container management device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in the read-only memory 1002 or a program loaded from the storage device 1003 into the random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the algorithm container management device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the algorithm container management device to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows algorithm container management devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.
[0091] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.
[0092] The algorithm container management device provided in this application, employing the algorithm container management method in the above embodiments, can solve the technical problems of algorithm container management. Compared with the prior art, the beneficial effects of the algorithm container management device provided in this application are the same as those of the algorithm container management method provided in the above embodiments, and other technical features in this algorithm container management device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.
[0093] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.
[0094] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0095] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the algorithm container management method in the above embodiments.
[0096] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.
[0097] The aforementioned computer-readable storage medium may be included in the algorithm container management device; or it may exist independently and not assembled into the algorithm container management device.
[0098] The aforementioned computer-readable storage medium carries one or more programs. When the one or more programs are executed by the algorithm container management device, the algorithm container management device: responds to a task processing request and determines the target algorithm container required to execute the user task through an algorithm orchestration service; determines the target server node to run the target algorithm container based on a preset container scheduling algorithm through a container scheduler; and sends a container scheduling instruction to the container agent corresponding to the target server node through the container scheduler, so that the container agent manages the target algorithm container through a container local socket.
[0099] Computer program code for performing the operations of this application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, and conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0100] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0101] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.
[0102] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described algorithm container management method. This solves the technical problem in existing algorithm containerization schemes where the scheduling and management of algorithm containers are highly correlated, leading to high maintenance costs. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the algorithm container management method provided in the above embodiments, and will not be repeated here.
[0103] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.
Claims
1. An algorithm container management method, characterized in that, The method described is applied to a container management platform, which includes an algorithm orchestration service, a container scheduler, and a container agent, comprising: In response to a task processing request, the target algorithm container required to execute the user task is determined through the algorithm orchestration service; The container scheduler determines the target server node to run the target algorithm container based on a preset container scheduling algorithm. The container scheduler sends container scheduling instructions to the container agent corresponding to the target server node, so that the container agent can manage the lifecycle of the target algorithm container through the container local socket.
2. The method as described in claim 1, characterized in that, The step of determining the target server node for running the target algorithm container through the container scheduler based on a preset container scheduling algorithm includes: The container scheduler obtains a list of server information and checks whether each server node in the list meets the preset service conditions. Based on the inspection results, server nodes that meet the preset service conditions are identified as candidate server nodes; The container scheduler determines the target server node from the candidate server nodes to run the target algorithm container based on a preset container scheduling algorithm.
3. The method as described in claim 2, characterized in that, The step of checking whether each server node in the server information list meets the preset service conditions includes: Obtain the network address of each server node in the server information list; Determine whether the network address is within a preset whitelist range; If so, check whether there are any idle graphics processing resources in each of the server nodes; If they exist, then determine whether each server node meets the task requirements of the user task based on the node label corresponding to each server node. If the conditions are met, then each server node is determined to meet the preset service conditions.
4. The method as described in claim 2, characterized in that, The step of determining the target server node for running the target algorithm container from the candidate server nodes by the container scheduler based on a preset container scheduling algorithm includes: The container scheduler determines the memory resource utilization and graphics processing resource usage of the candidate server nodes based on the server information list. Based on the memory resource utilization rate and the graphics processing resource occupancy rate, the node score corresponding to the candidate server node is determined by a preset container scheduling algorithm. The target server node for running the target algorithm container is determined from the candidate server nodes based on the node score.
5. The method according to any one of claims 1 to 4, characterized in that, The step of responding to a task processing request and determining the target algorithm container required to execute the user task through the algorithm orchestration service includes: In response to a task processing request, the algorithm orchestration service determines the task execution information corresponding to the user task based on the task processing request, and the task execution information includes an algorithm identifier. The target algorithm required to execute the user task is determined based on the algorithm identifier; Based on the target algorithm and the task requirements of the user task, determine the target algorithm container required to execute the user task.
6. The method as described in claim 5, characterized in that, After the step of sending container scheduling instructions to the container agent corresponding to the target server node through the container scheduler, so that the container agent can perform lifecycle management of the target algorithm container through the container local socket, the method further includes: The target algorithm is run through the target algorithm container to obtain the algorithm execution result; Determine the callback method for the algorithm's execution result; The algorithm execution result is sent back to the algorithm orchestration service using the callback method, so that the algorithm orchestration service can aggregate the algorithm execution result and generate the task processing result of the user task.
7. The method according to any one of claims 1 to 4, characterized in that, The method further includes: In response to a video processing request, the algorithm orchestration service determines the storage location and frame extraction frequency of the video to be processed based on the video processing request. The video to be processed is obtained according to the storage location; Start the frame extraction thread, and extract frames from the video to be processed according to the frame extraction frequency to obtain a number of video frames. The video analysis algorithm container is invoked by the container scheduler to process each video frame and obtain the video processing result.
8. An algorithm container management device, characterized in that, The device includes a container management platform, which comprises an algorithm orchestration service, a container scheduler, and a container agent, including: The request processing module is used to respond to task processing requests and determine the target algorithm container required to execute user tasks through the algorithm orchestration service. The scheduling decision module is used to determine the target server node for running the target algorithm container based on a preset container scheduling algorithm through the container scheduler; The container management module is used to send container scheduling instructions to the container agent corresponding to the target server node through the container scheduler, so that the container agent can perform lifecycle management of the target algorithm container through the container local socket.
9. An algorithm container management device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the algorithm container management method as described in any one of claims 1 to 7.
10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the algorithm container management method as described in any one of claims 1 to 7.