AI algorithm and micro-service scheduling method based on cloud-native technology and device thereof

Abstract

The application provides an AI algorithm and micro-service scheduling method and device based on cloud native technology, creates a target image through a preset image building engine, obtains an application configuration template and a parsing function, parses the application configuration template according to the parsing function to obtain application deployment parameters, containerizes and deploys the target image in a k8s cluster according to the application deployment parameters, obtains a target application, when the target application meets a preset container scheduling condition, obtains a resource change event of the target application, determines an event type of the resource change event, and containerizes and schedules the target application according to a corresponding event processing function according to the event type, which can provide different algorithm and micro-service running environments for different algorithms and micro-services, and there is no need to repeatedly build a running environment, and when the algorithm and micro-service run incorrectly, the algorithm and micro-service can be deployed on a suitable node through container scheduling to quickly recover the algorithm service and the micro-service.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to an AI algorithm and microservice scheduling method and apparatus based on cloud-native technology. Background Technology

[0002] In related technologies, in order to deploy an algorithm to a suitable operating environment, it is necessary to build a suitable environment for the algorithm to run on a physical machine. However, the operating environments of different algorithms are different, and during the operation of the algorithm, there may be program crashes or server downtime, which will cause the algorithm to fail. The differences in the algorithm operating environment, the repetitiveness of the building process, and the failure of the algorithm to run seriously slow down the application progress of the algorithm. Summary of the Invention

[0003] The main objective of this application is to propose an AI algorithm and microservice scheduling method and apparatus based on cloud-native technology, which can provide a stable algorithm running environment for different algorithms, quickly restore algorithm services when algorithm operation fails, and avoid repeatedly building the running environment, thus realizing the rapid application of algorithms.

[0004] To achieve the above objectives, a first aspect of this application proposes an AI algorithm and microservice scheduling method based on cloud-native technology, the method comprising:

[0005] Create the target image based on the preset image build engine;

[0006] Obtain the application configuration template and parsing function;

[0007] The application configuration template is parsed using the parsing function to obtain application deployment parameters;

[0008] The target image is containerized and deployed in a Kubernetes cluster according to the application deployment parameters to obtain the target application.

[0009] When the target application meets the preset container scheduling conditions, the resource change event of the target application is obtained;

[0010] Determine the event type of the resource change event, and call the corresponding event handling function to perform container scheduling for the target application based on the event type.

[0011] In some embodiments, the step of invoking the corresponding event handling function to perform container scheduling on the target application based on the event type includes:

[0012] If the event type is a processing event, the processing plugin is called according to the preset first plugin running order, and multiple initial nodes are obtained by filtering multiple nodes in the k8s cluster based on the processing plugin.

[0013] Get the preset scoring function;

[0014] The initial node is scored according to the scoring function to obtain the score value corresponding to the initial node;

[0015] The initial node arrangement order is obtained based on the score value;

[0016] A target node is determined from multiple initial nodes according to the orchestration order, and the target node is bound to the target application so that the target application runs on the target node.

[0017] In some embodiments, the step of invoking the corresponding event handling function to perform container scheduling on the target application based on the event type includes:

[0018] If the event type is a post-processing event, the post-processing plugin is called according to the preset second plugin execution order, and the nodes in the Kubernetes cluster are marked based on the post-processing plugin to obtain the target node;

[0019] The target node is bound to the target application so that the target application runs on the target node.

[0020] In some embodiments, obtaining the resource change event of the target application when the target application meets preset container scheduling conditions includes:

[0021] Add the target application to the scheduling queue;

[0022] If there are other applications in the scheduling queue, all applications in the scheduling queue are sorted according to a preset priority order to obtain the priority of the target application;

[0023] If the priority is a preset target priority, and the target application includes preset container scheduling information, then the resource change event of the target application is obtained.

[0024] In some embodiments, after adding the target application to the scheduling queue, the cloud-native AI algorithm and microservice scheduling method further includes:

[0025] If the target application exists only in the scheduling queue, and the target application includes preset container scheduling information, then the resource change event of the target application is obtained.

[0026] In some embodiments, after obtaining the resource change event of the target application, the cloud-native AI algorithm and microservice scheduling method further includes:

[0027] Add the resource change event to the work queue;

[0028] Retrieve the resource change event from the work queue.

[0029] In some embodiments, the cloud-native AI algorithm and microservice scheduling method further includes:

[0030] The nodes in the Kubernetes cluster are monitored using preset monitoring components to obtain monitoring data.

[0031] The monitoring data is stored in a preset database;

[0032] Receive query command;

[0033] The target monitoring data is retrieved from the database according to the query command.

[0034] To achieve the above objectives, a second aspect of this application proposes an AI algorithm and microservice scheduling device based on cloud-native technology, the device comprising:

[0035] The image build module is used to create target images based on a preset image build engine;

[0036] The first acquisition module is used to acquire application configuration templates and parsing functions;

[0037] The parsing module is used to parse the application configuration template according to the parsing function to obtain application deployment parameters;

[0038] The deployment module is used to containerize and deploy the target image in a Kubernetes cluster according to the application deployment parameters to obtain the target application.

[0039] The second acquisition module is used to acquire the resource change event of the target application when the target application meets the preset container scheduling conditions.

[0040] The container scheduling module is used to determine the event type of the resource change event, and call the corresponding event handling function to perform container scheduling for the target application according to the event type.

[0041] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, it implements the method described in the first aspect above.

[0042] To achieve the above objectives, a fourth aspect of the present application provides a storage medium, which is a computer-readable storage medium for computer-readable storage, wherein the storage medium stores one or more programs that can be executed by one or more processors to implement the method described in the first aspect.

[0043] This application proposes a cloud-native AI algorithm and microservice scheduling method, device, electronic device, and storage medium. It creates a target image through a preset image building engine, obtains an application configuration template and parsing function, parses the application configuration template using the parsing function to obtain application deployment parameters, and containerizes and deploys the target image in a Kubernetes cluster according to these parameters, resulting in a target application. When the target application meets preset container scheduling conditions, it obtains resource change events for the target application, determines the event type of the resource change event, and calls the corresponding event handling function to perform container scheduling on the target application. This application's embodiments build a Kubernetes cluster on physical machines and containerize and deploy the algorithm on nodes in the Kubernetes cluster according to the target image and application deployment parameters, allowing the algorithm to run on the corresponding node. Different target images are selected for different algorithms, and different containers are created based on different target images, providing different algorithm running environments for different algorithms without the need for repeated environment building. When an algorithm encounters an error, container scheduling can deploy the algorithm and microservices to suitable nodes to quickly restore the algorithm and microservices, achieving rapid application of algorithms and microservices. Attached Figure Description

[0044] Figure 1 This is a flowchart of the AI ​​algorithm and microservice scheduling method based on cloud-native technology provided in the embodiments of this application;

[0045] Figure 2 yes Figure 1 The first flowchart of step S150 in the process;

[0046] Figure 3 yes Figure 1 The second flowchart of step S150 in the process;

[0047] Figure 4 yes Figure 1 The first flowchart of step S160 in the process;

[0048] Figure 5 yes Figure 1 The second flowchart of step S160 in the process;

[0049] Figure 6 yes Figure 1 The flowchart of step S140 in the middle;

[0050] Figure 7 This is a schematic diagram of the structure of the AI ​​algorithm and microservice scheduling device based on cloud-native technology provided in the embodiments of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0054] In related technologies, in order to deploy an algorithm to a suitable operating environment, it is necessary to build a suitable environment for the algorithm to run on a physical machine. However, the operating environments of different algorithms are different, and during the operation of the algorithm, there may be program crashes or server downtime, which will cause the algorithm to fail. The differences in the algorithm operating environment, the repetitiveness of the building process, and the failure of the algorithm to run seriously slow down the application progress of the algorithm.

[0055] Based on this, embodiments of this application provide an AI algorithm and microservice scheduling method and apparatus, electronic device and storage medium based on cloud-native technology, which aims to provide a stable algorithm running environment for different algorithms and to quickly restore algorithm services when algorithm operation fails, so as to realize the rapid application of algorithms.

[0056] The AI ​​algorithm and microservice scheduling method and apparatus, electronic device and storage medium based on cloud-native technology provided in this application are specifically described through the following embodiments. First, the AI ​​algorithm and microservice scheduling method based on cloud-native technology in this application embodiment are described.

[0057] The AI ​​algorithm and microservice scheduling method based on cloud-native technology provided in this application relates to the field of internet technology. The AI ​​algorithm and microservice scheduling method based on cloud-native technology provided in this application can be applied to terminals, servers, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the AI ​​algorithm and microservice scheduling method based on cloud-native technology, but is not limited to the above forms.

[0058] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0059] Figure 1 This is an optional flowchart of the AI ​​algorithm and microservice scheduling method based on cloud-native technology provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S110 to S160.

[0060] Step S110: Create the target image based on the preset image building engine;

[0061] Step S120: Obtain the application configuration template and parsing function;

[0062] Step S130: Parse the application configuration template according to the parsing function to obtain the application deployment parameters;

[0063] Step S140: Containerize and deploy the target image in the Kubernetes cluster according to the application deployment parameters to obtain the target application;

[0064] Step S150: When the target application meets the preset container scheduling conditions, the resource change event of the target application is obtained.

[0065] Step S160: Determine the event type of the resource change event, and call the corresponding event handling function to perform container scheduling for the target application based on the event type.

[0066] In step S110 of some embodiments, the image building engine is Docker. Multiple physical machines are organized into a server cluster, and Docker is installed on each node of the server cluster. The application is packaged into a target image based on Docker to support container operation. The target image can be an algorithm image or a microservice image. It is understood that, to facilitate image reuse, the target image can be uploaded to an image repository. By organizing the computing power of the server cluster, a stable, out-of-the-box algorithm runtime environment can be provided, enabling the sharing of algorithm services among different AI applications. Ultimately, this achieves the containerization of large-scale distributed and numerous algorithm tasks and their scheduling within a container cloud platform.

[0067] This application embodiment builds a server cluster based on the AMD64 architecture and Linux operating system. Six CPU servers and one GPU server with AMD64 architecture are selected as physical machines, and Ubuntu 20.04 is installed as a unified server operating system environment. It should be noted that no other applications or dependencies are added after the server operating system is installed to ensure a single system environment. All servers are placed on the same local area network to ensure inter-cluster connectivity and that all servers can connect to the external network for convenient access to resources.

[0068] This application, based on cloud-native design principles, utilizes container technology and container orchestration to encapsulate the runtime environment of algorithm services, isolating the underlying physical machine environment from the algorithm's runtime environment. Container technology, through cgroups and namespaces, provides a unified running mode for processes running on Linux systems, allowing containers running on physical servers to be viewed as resource-constrained, view-isolated processes. Compared to traditional virtual machines, containers offer advantages such as faster startup and destruction speeds and lower resource consumption. A container runtime is software that executes containers and manages container images on nodes. Different container runtimes differ in performance; for example, Kata container runtime provides a virtual machine-like isolation environment, but container creation speed is affected. Containerd, based on cgroups and namespaces, offers weaker isolation but boasts very fast container creation speeds and is the most commonly used container runtime in the industry. Docker uses containerd as its underlying container runtime and provides encapsulation for image and container operations at higher layers, offering users a convenient entry point.

[0069] Given the need for rapid response in algorithm execution instances and microservice execution instances, this application embodiment selects containerd as the container runtime and uses docker as the interface for image and container management.

[0070] In steps S120 to S130 of some embodiments, the application configuration template includes metadata from the application deployment stage to decouple application implementation from application deployment. This eliminates the need to implement the deployment method during the application design phase; instead, it exposes an extensible interface. The Kubernetes cluster creates containers corresponding to the application based on this interface, where the application includes algorithms and microservices. The application configuration template must work in conjunction with the application implementation. Fields in the template should be consistent with the logic in the application implementation process. For example, during application implementation, the application may need to create several containers: a master container responsible for executing task distribution and data aggregation, and several worker containers implementing the tasks distributed by the master container and reporting the execution results. Therefore, the application configuration template should clearly specify the nodes that act as master containers, the nodes that act as worker containers, and the number of worker containers. The application configuration template allows for multiple deployment methods for an application, making it suitable for scenarios with high scalability requirements, such as scenarios where algorithms require specifying parallelism or the number of containers to be created before execution.

[0071] The application configuration template is specifically {"category": "mc", "export": 1", "kind": "Job", "parallelNum": 2", "targetPort": 10080", "webPath": " / result", "template": [{"name": "master", "serverPort": 10080", "masterPort": 10081}, {"name": "eval1", "masterPort": 10081", "Port": 10083}, {"name": "eval2", "masterPort": 10081", "Port": 10084}], "bestConfigInf o: {"cpu": "1 core 2 threads", "gpu": "GTX960", "memory": "2GB"}}, where category represents the application type, "export" indicates whether the application content is exposed via the web (1 for yes, 0 for no), kind represents the resource type on Kubernetes, parallelNum represents the number of parallel worker nodes (corresponding to the eval container in the template), targetPort represents the port exposed by the application (if "export" is 1, then "webPath" needs to be set, i.e., the web access path), template is the container template that needs to be created when the application is created, and bestConfigInfo is the system configuration information required by the application. The application configuration template hides the many details required when creating an application on Kubernetes from the user; the user only needs to care about what kind of application they need.

[0072] Each parsing function corresponds one-to-one with an application type. The parsing function defines the fields that can be in the template and their value ranges. Users can submit templates that meet the requirements within this range, which are then parsed by the parsing function to obtain application deployment parameters. These parameters are the values ​​of specific fields in the application configuration template, such as `export` and `parallelNum`, transforming the application configuration template into a resource that Kubernetes can parse and schedule and run in the cluster. This application uses Docker and Kubernetes to build a container cloud platform and implements an abstraction layer between the platform and the application configuration template. This abstraction allows application developers to write a parsing function only once to parse multiple templates, without having to write resource files conforming to Kubernetes specifications for each template. Furthermore, the application configuration template is much lighter than Kubernetes specification resource files.

[0073] In step S140 of some embodiments, the algorithm and microservices are split into multiple containers. By containerizing the algorithm and microservices and scheduling the containers to the cluster based on container orchestration technology, the distributed operation of the algorithm and microservices is achieved. Container orchestration technology is based on underlying containers and orchestrates and schedules the containers created in the system. Choosing a suitable container orchestration technology directly affects whether resources are scheduled in the most efficient way and whether the orchestration is organized in the most efficient way for the algorithm and microservices to run. This application uses Kubernetes technology as the container orchestration foundation, deploying all programs as Kubernetes-supported resources. Compared with other container orchestration technologies, it provides a more user-friendly and diverse extension method, allowing users to add custom plugins or extend the orchestration platform according to their actual needs, rather than being limited to its provided functions.

[0074] In step S150 of some embodiments, the preset container scheduling conditions can be creating a target application, deleting a target application, terminating a target application, etc. Based on the client-go client exposed by Kubernetes, it listens for resource change events generated by the apiserver in the master container due to applications being created, deleted, or terminated. By exposing the operations of creating, deleting, and terminating applications to users, users can start or stop applications at any time as needed. Specifically, based on the client-go client exposed by Kubernetes, it listens for resources of interest in the apiserver through a list-watch mechanism and caches the resources locally. To facilitate subsequent retrieval of resources locally, an indexer is used to create an index locally. Since there is a certain time interval between receiving resource changes and completing index creation, to solve the problem of speed mismatch between resource message generation and consumption, the resources are cached in a buffer implemented internally by client-go, which is a delta FIFO queue.

[0075] In step S160 of some embodiments, the apiserver uses Kafka as a microservice communication channel to send resource change events to the controller. The controller communicates with the underlying Kubernetes cluster and calls the corresponding event handling function based on the event type of the resource change event to perform container scheduling for the target application. This allows for monitoring and management of the target application. Container scheduling provides a stable operating environment for the algorithm service and enables rapid recovery and continued service after the algorithm service crashes. It should be noted that this application uses the Gin open-source framework to implement the web service. Both the apiserver and controller are microservices. The apiserver microservice uses Redis as a state cache and MongoDB for data persistence. The apiserver exposes operation interfaces as an HTTP service and uses a layered design to divide the entire service into three layers from top to bottom: Service layer, Dao layer, and Module layer. The Service layer handles HTTP requests, performing validation, authentication, parsing, storage, and forwarding, and calls the lower layer for specific logic processing based on business logic. The Dao layer encapsulates or processes the underlying data, decapsulates upper-layer requests, and calls the lower layer. The Module layer is used for interaction between the apiserver and the underlying data storage.

[0076] There are two communication methods between microservices: local communication and service communication based on Kubernetes. Local communication requires all microservices to be deployed on the same physical machine; in service communication, microservices communicate based on DNS domain names. By binding the IP address of a Pod to a domain name, access between microservices only requires attention to the domain name, not the specific IP address.

[0077] It should be noted that the two microservices, apiserver and controller, communicate locally. These two microservices communicate with other external systems using DNS domain names. Therefore, Kafka is used as the message middleware for communication to achieve asynchronous communication between the microservices and other systems.

[0078] Steps S110 to S160 of this embodiment involve creating a target image using a preset image building engine, obtaining an application configuration template and a parsing function, parsing the application configuration template using the parsing function to obtain application deployment parameters, containerizing and deploying the target image in a Kubernetes cluster according to the application deployment parameters to obtain the target application, obtaining resource change events of the target application, determining the event type of the resource change event, and calling the corresponding event handling function to perform container scheduling for the target application according to the event type. This embodiment builds a Kubernetes cluster on a physical machine and containerizes and deploys algorithms and microservices on nodes in the Kubernetes cluster according to the target image and application deployment parameters, so that the algorithms and microservices run on the corresponding nodes. Different target images are selected according to different algorithms and different microservices, and different containers are created based on different target images. This can provide different algorithm running environments and microservice running environments for different algorithms and microservices without repeatedly building running environments. When the algorithm and microservices encounter errors, container scheduling can deploy the algorithm and microservices to appropriate nodes to quickly restore the algorithm services and microservices, realizing the rapid application of algorithms and microservices.

[0079] Please see Figure 2 In some embodiments, step S150 may include, but is not limited to, steps S210 to S230:

[0080] Step S210: Add the target application to the scheduling queue;

[0081] Step S220: If there are other applications in the scheduling queue, sort all applications in the scheduling queue according to the preset priority order to obtain the priority of the target application.

[0082] Step S230: If the priority is the preset target priority and the target application includes preset container scheduling information, then obtain the resource change event of the target application.

[0083] After step S210 in some embodiments, if the scheduling queue contains only the target application and the target application includes preset container scheduling information, then the resource change event of the target application is obtained.

[0084] In steps S220 to S230 of some embodiments, if there are other applications in the scheduling queue, all applications in the scheduling queue are sorted according to a custom priority order to obtain the priority of the target application. If the target application has the highest priority and meets the conditions required for container scheduling, such as the target application needing to bind a PV that requires immediate binding, and the target application contains information that needs to be satisfied for Pod scheduling, then the resource change event of the target application is obtained, where Pod is the basic unit of Kubernetes scheduling.

[0085] Please see Figure 3 In some embodiments, steps S310 to S320 may be included after step S150:

[0086] Step S310: Add the resource change event to the work queue;

[0087] Step S320: Retrieve resource change events from the work queue.

[0088] In steps S310 to S320 of some embodiments, the work queue acts as a buffer for resource change events, reducing the mismatch between the speed of event notification and the speed of event consumption. Resource change events are consumed from the work queue, and event handling functions are invoked to perform business logic processing to handle the resource change events.

[0089] Please see Figure 4 In some embodiments, step S160 may include, but is not limited to, steps S410 to S450:

[0090] Step S410: If the event type is a processing event, the processing plugin is called according to the preset first plugin running order. Based on the processing plugin, multiple nodes in the k8s cluster are filtered to obtain multiple initial nodes.

[0091] Step S420: Obtain the preset scoring function;

[0092] Step S430: Score the initial node according to the scoring function to obtain the score value corresponding to the initial node;

[0093] Step S440: Obtain the initial node arrangement order based on the score values;

[0094] Step S450: Determine the target node from multiple initial nodes according to the arrangement order, and bind the target node to the target application so that the target application runs on the target node.

[0095] In steps S410 to S450 of some embodiments, if the event type is a processing event, i.e., the target application is in the scheduling filter phase, the filter plugins are called according to the execution order of the first plugins. The filter plugins check whether there are suitable nodes in the Kubernetes cluster to obtain multiple initial nodes. If a plugin fails during the sequential execution of the filter plugins, subsequent plugins will not be executed. The multiple initial nodes are scored according to the scoring function to obtain a score corresponding to each initial node. The scores are normalized to obtain the numerical score corresponding to the initial node. The initial nodes are sorted from high to low according to the numerical score to obtain the orchestration order. The initial node with the highest numerical score is determined as the target node according to the orchestration order. The target node is bound to the target application so that the target application runs on the target node in the most efficient way.

[0096] Please see Figure 5 In some embodiments, step S160 may also include, but is not limited to, steps S510 to S520:

[0097] Step S510: If the event type is a post-processing event, the post-processing plugin is called according to the preset second plugin running order, and the nodes in the k8s cluster are marked based on the post-processing plugin to obtain the target node.

[0098] Step S520: Bind the target node to the target application so that the target application runs on the target node.

[0099] In steps S510 to S520 of some embodiments, if the filter plugin fails to find a suitable node in the k8s cluster during the filter phase, the post-processing phase (postfilter phase) is entered. The event type is post-processing event. The postfilter plugin is called according to the second plugin execution order. The postfilter plugin adds a mark to a node in the k8s cluster to preempt the node, obtains the target node, and binds the target node to the target application so that the target application runs on the target node in the most efficient way.

[0100] It should be noted that if a post-processing plugin marks a node as schedulable, subsequent post-processing plugins will not run.

[0101] This application primarily focuses on two aspects of Kubernetes extensibility: custom controllers and custom scheduling plugins. The custom controller, based on the client-go client exposed by Kubernetes, monitors resources monitored by the apiserver using a list-watch mechanism. When a resource changes (e.g., added, updated, or deleted), it calls a handler function to consume the resource event and performs specific business logic processing. The custom scheduling plugin leverages the plugin extension capabilities of the Kubernetes scheduler, namely the scheduler framework. The scheduler's role is to select a suitable node for a Pod to run on. The node selection process consists of multiple stages. Each scheduling stage is extended based on the scheduler plugin and compiled into the native scheduler. It should be noted that a Pod can choose a specific scheduler during creation. The multiple stages of node selection are, in order: sort, prefilter, filter, postfilter, prescore, score, normalize score, reserve, permit, prebind, bind, and postbind. The sorting phase queues Pods according to a custom priority. The prefilter phase checks if Pods meet scheduling requirements. The filter phase runs filter plugins in the order they were executed to check for suitable nodes. The postfilter phase runs post-processing plugins in the order they were executed to preempt nodes if the filter phase did not find suitable ones. The prescore phase generates shared scheduling information for the score phase. The score phase scores the nodes obtained in the filter phase according to the scoring function. The normalize phase... The score normalization phase normalizes the score values. The reserve phase, after all the previous phases have passed, indicates that the Pod is schedulable and is used to deduct Pod resources from the cache. The permit phase prevents, allows, or delays Pod binding. The prebind phase prepares for the bind phase. If an error occurs in this phase, the unreserve phase is executed, and the Pod will re-enter the scheduling queue. The bind phase calls the binding plugins in a pre-defined third-party plugin execution order, and the binding plugin decides whether to process the current Pod. Once the Pod is processed, the remaining plugins are skipped. The postbind phase performs some cleanup work after the Pod is successfully bound.

[0102] Please see Figure 6In some embodiments, steps S610 to S640 may be included after step S140:

[0103] Step S610: Monitor the nodes in the k8s cluster according to the preset monitoring components and obtain monitoring data;

[0104] Step S620: Store the monitoring data in a preset database;

[0105] Step S630: Receive query command;

[0106] Step S640: Obtain target monitoring data from the database according to the query command.

[0107] In steps S610 to S640 of some embodiments, after the container cloud platform based on Docker and Kubernetes is built, it can provide PaaS platform services. However, the platform lacks observability, that is, it lacks the ability to observe the relevant clusters. Observability is an important indicator for measuring whether a system conforms to cloud-native specifications. Problems exposed by the system during runtime are sometimes temporary and need to be recorded and statistically analyzed through the observation system. That is, observability analysis is used to analyze the system's performance indicators at different times and locations. KubeSphere is an integrated operation and maintenance platform based on Kubernetes. Its functions include tenant management, platform performance observation, and container load observation and management. It integrates the performance observation capabilities provided by Prometheus and exposes a more convenient and easy-to-manage user interface. As a guarantee of the observable performance of Kubernetes clusters and physical machines, it allows users to collect and observe cluster indicators without having to enter the platform's backend for command-line operations. Users can simply perform operations on its provided dashboard interface to achieve this.

[0108] Create observation metrics such as the number of Pods and the number of calls. When a request is received or a Pod is created, the metric is automatically invoked to achieve the observation of a certain metric. At the same time, the port of the HTTP server is exposed to Promethean, and the metric is pulled periodically through the pull mechanism to realize the collection of cluster data and display the data to the outside world through Grafana.

[0109] Within Promethues, the Retrival component monitors nodes in the Kubernetes cluster to obtain monitoring data. This monitoring data is time-series data retrieved via HTTP pull. The monitoring data is stored in a Time Series Database (TSDB). An HTTP interface is provided through an HTTP server to obtain query commands. Based on the query commands, the target monitoring data is retrieved from the database and visualized using Grafana.

[0110] It's important to note that PaaS platform users fall into two categories: algorithm and microservice developers, and ordinary users who utilize algorithms and microservices. Developers create applications, package them into images, upload them to an image repository, and design application configuration templates to create and publish their applications on the application marketplace. Ordinary users cannot create applications themselves; instead, they add applications from the marketplace to their personal repositories to create and run application instances. The system exposes interfaces for application creation, management, and task management to application developers. Aside from writing the application in the backend, all other operations can be performed on the web page. Upon receiving a user request, the system frontend verifies its validity and forwards it to the backend. The backend executes the specific business logic and returns a response to the frontend based on the execution result.

[0111] It's important to further clarify that the PaaS platform includes a business system, which comprises an apiserver microservice and a controller microservice. This business system is developed in Go, using the third-party component Gin to implement an HTTP server that adheres to RESTful APIs. These two microservices are wrapped as StatefulSet resources, and corresponding service resources are configured to enable access both within and outside the cluster. NodePort is used to provide an access interface to the outside world, with the apiserver acting as the gateway for the business system. Application developers and ordinary users' actions on the page are sent to the backend via the frontend interface. The business system parses the parameters and executes the specific business logic. Operations involving data persistence are executed within the apiserver microservice; database data is persisted to MongoDB, and data related to the microservice's runtime status is stored in Redis. Since the business system includes both internal and external communication, the communication between the two microservices (apiserver and controller) is internal, while external communication refers to communication between these two microservices and other external systems. Therefore, Kafka is used as the message middleware to achieve asynchronous communication between the microservices and other systems.

[0112] Please see Figure 7 This application also provides an AI algorithm and microservice scheduling device based on cloud-native technology, which can implement the above-mentioned AI algorithm and microservice scheduling method based on cloud-native technology. The device includes:

[0113] Image building module 710 is used to create target images based on a preset image building engine;

[0114] The first acquisition module 720 is used to acquire application configuration templates and parsing functions;

[0115] The parsing module 730 is used to parse the application configuration template according to the parsing function to obtain the application deployment parameters;

[0116] Deployment module 740 is used to containerize and deploy the target image in the Kubernetes cluster according to the application deployment parameters to obtain the target application;

[0117] The second acquisition module 750 is used to acquire the resource change event of the target application when the target application meets the preset container scheduling conditions.

[0118] The container scheduling module 760 is used to determine the event type of resource change events and call the corresponding event handling function to schedule the target application into a container based on the event type.

[0119] The specific implementation of this cloud-native AI algorithm and microservice scheduling device is basically the same as the specific implementation of the cloud-native AI algorithm and microservice scheduling method described above, and will not be repeated here.

[0120] This application also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for communication between the processor and the memory. When the program is executed by the processor, it implements the aforementioned AI algorithm and microservice scheduling method based on cloud-native technology. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0121] This application also provides an electronic device, including:

[0122] The processor can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to achieve the technical solutions provided in the embodiments of this application.

[0123] The memory can be implemented in the form of read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory and called and executed by the processor using the cloud-native AI algorithm and microservice scheduling method of the embodiments of this application.

[0124] Input / output interfaces are used to implement information input and output;

[0125] The communication interface is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0126] A bus is used to transfer information between various components of a device, such as processors, memory, input / output interfaces, and communication interfaces.

[0127] The processor, memory, input / output interfaces, and communication interfaces communicate with each other within the device via a bus.

[0128] This application also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, which can be executed by one or more processors to implement the above-mentioned AI algorithm and microservice scheduling method based on cloud-native technology.

[0129] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0130] The AI ​​algorithm and microservice scheduling method, device, electronic device, and storage medium based on cloud-native technology provided in this application embodiment create a target image through a preset image building engine, obtain an application configuration template and parsing function, parse the application configuration template according to the parsing function to obtain application deployment parameters, and containerize and deploy the target image in a Kubernetes cluster according to the application deployment parameters to obtain the target application. When the target application meets the preset container scheduling conditions, the resource change event of the target application is obtained, the event type of the resource change event is determined, and the corresponding event handling function is called according to the event type to containerize the target application. In terms of scheduling, this embodiment of the application builds a Kubernetes cluster on physical machines and containersizes and deploys algorithms and microservices on nodes in the Kubernetes cluster according to the target image and application deployment parameters, so that the algorithms and microservices run on the corresponding nodes. Different target images are selected according to different algorithms and microservices, and different containers are created based on different target images. This can provide different algorithm running environments and microservice running environments for different algorithms and microservices, without the need to repeatedly build running environments. When the algorithm and microservices encounter errors, container scheduling can deploy the algorithm and microservices to appropriate nodes to quickly restore the algorithm services and microservices, realizing the rapid application of algorithms and microservices.

[0131] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0132] It will be understood by those skilled in the art that Figure 1-6 The technical solutions shown do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0133] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0134] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0135] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0136] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0137] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0138] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0139] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0140] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. An AI algorithm and microservice scheduling method based on cloud-native technology, characterized in that, The method includes: A target image is created based on a preset image building engine; wherein, the target image is an algorithm image or a microservice image; Obtain the application configuration template and parsing function; wherein, the application configuration template includes metadata from when the application is deployed; The application configuration template is parsed using the parsing function to obtain application deployment parameters; The target image is containerized and deployed in a Kubernetes cluster according to the application deployment parameters to obtain the target application. When the target application meets the preset container scheduling conditions, the resource change event of the target application is obtained; wherein, the container scheduling conditions include creating the target application, deleting the target application, or terminating the target application; Determine the event type of the resource change event, and call the corresponding event handling function to perform container scheduling for the target application based on the event type; The step of invoking the corresponding event handling function to perform container scheduling for the target application based on the event type includes: If the event type is a processing event, then the processing plugins are called according to the preset first plugin execution order. Based on the processing plugins, multiple nodes in the Kubernetes cluster are filtered to obtain multiple initial nodes. A preset scoring function is obtained. The initial nodes are scored according to the scoring function to obtain the score value corresponding to the initial nodes. The orchestration order of the initial nodes is obtained according to the score value. The target node is determined from the multiple initial nodes according to the orchestration order, and the target node is bound to the target application so that the target application runs on the target node. The step of invoking the corresponding event handling function to perform container scheduling for the target application based on the event type includes: If the event type is a post-processing event, the post-processing plugin is called according to the preset second plugin execution order. The nodes in the Kubernetes cluster are marked based on the post-processing plugin to obtain the target node. The target node is then bound to the target application so that the target application runs on the target node.

2. The AI ​​algorithm and microservice scheduling method based on cloud-native technology according to claim 1, characterized in that, The step of obtaining the resource change event of the target application when the target application meets the preset container scheduling conditions includes: Add the target application to the scheduling queue; If there are other applications in the scheduling queue, all applications in the scheduling queue are sorted according to a preset priority order to obtain the priority of the target application; If the priority is a preset target priority, and the target application includes preset container scheduling information, then the resource change event of the target application is obtained.

3. The AI ​​algorithm and microservice scheduling method based on cloud-native technology according to claim 2, characterized in that, After adding the target application to the scheduling queue, the cloud-native AI algorithm and microservice scheduling method further includes: If the target application exists only in the scheduling queue, and the target application includes preset container scheduling information, then the resource change event of the target application is obtained.

4. The AI ​​algorithm and microservice scheduling method based on cloud-native technology according to claim 1, characterized in that, After obtaining the resource change event of the target application, the AI ​​algorithm and microservice scheduling method based on cloud-native technology further includes: Add the resource change event to the work queue; Retrieve the resource change event from the work queue.

5. The AI ​​algorithm and microservice scheduling method based on cloud-native technology according to any one of claims 1 to 4, characterized in that, The AI ​​algorithm and microservice scheduling method based on cloud-native technology also include: The nodes in the Kubernetes cluster are monitored using preset monitoring components to obtain monitoring data. The monitoring data is stored in a preset database; Receive query command; The target monitoring data is retrieved from the database according to the query command.

6. An AI algorithm and microservice scheduling device based on cloud-native technology, characterized in that, The device includes: The image building module is used to create a target image based on a preset image building engine; wherein the target image is an algorithm image or a microservice image; The first acquisition module is used to acquire the application configuration template and parsing function; wherein, the application configuration template includes metadata during application deployment; The parsing module is used to parse the application configuration template according to the parsing function to obtain application deployment parameters; The deployment module is used to containerize and deploy the target image in a Kubernetes cluster according to the application deployment parameters to obtain the target application. The second acquisition module is used to acquire resource change events of the target application when the target application meets preset container scheduling conditions; wherein, the container scheduling conditions include creating the target application, deleting the target application, or terminating the target application; The container scheduling module is used to determine the event type of the resource change event, and call the corresponding event handling function to perform container scheduling for the target application according to the event type; The step of invoking the corresponding event handling function to perform container scheduling for the target application based on the event type includes: If the event type is a processing event, then the processing plugins are called according to the preset first plugin execution order. Based on the processing plugins, multiple nodes in the Kubernetes cluster are filtered to obtain multiple initial nodes. A preset scoring function is obtained. The initial nodes are scored according to the scoring function to obtain the score value corresponding to the initial nodes. The orchestration order of the initial nodes is obtained according to the score value. The target node is determined from the multiple initial nodes according to the orchestration order, and the target node is bound to the target application so that the target application runs on the target node. The step of invoking the corresponding event handling function to perform container scheduling for the target application based on the event type includes: If the event type is a post-processing event, the post-processing plugin is called according to the preset second plugin execution order. The nodes in the Kubernetes cluster are marked based on the post-processing plugin to obtain the target node. The target node is then bound to the target application so that the target application runs on the target node.

7. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory, wherein the program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 5.

8. A storage medium, said storage medium being a computer-readable storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the method according to any one of claims 1 to 5.

Images