Cluster image management method, device, computer equipment, readable storage medium and program product

Abstract

The present application relates to a cluster image management method, device, computer equipment, computer-readable storage medium and computer program product, and relates to the field of cluster operation and maintenance technology, which can improve the management efficiency of images in the cluster. The method includes: receiving an image management task sent by a central cluster to an edge cluster; screening nodes in the edge cluster to determine a target node related to the image management task; controlling the target node to execute the image management task, and generating image management task result information when the target node has completed the execution; sending the image management task result information to the central cluster to display the image management task result information through a visual interface of the central cluster.

Description

Technical Field

[0001] The present application relates to the field of cluster operation and maintenance technology, and in particular to a cluster image management method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of cloud computing, edge computing has gradually become an important technology for improving real-time processing capabilities and optimizing network resources.

[0003] However, the management of edge clusters, especially the efficient use of image resources, still faces many challenges. In scenarios such as artificial intelligence model training, image files are large and frequently updated, which can easily lead to dangling images and waste valuable storage space. Although Docker provides instructions for cleaning up dangling images, operation and maintenance personnel need to log in to the nodes one by one to perform operations. This traditional method is highly dependent on manual operations, which can easily increase the difficulty of management due to the expansion of node scale, making the cluster's management of dangling images inefficient. Summary of the invention

[0004] Based on this, it is necessary to provide a cluster mirror management method, apparatus, computer equipment, computer readable storage medium and computer program product to address the above technical issues.

[0005] In a first aspect, the present application provides a cluster image management method, comprising:

[0006] Receive image management tasks sent by the central cluster to the edge cluster;

[0007] Screening the nodes in the edge cluster to determine the target nodes related to the image management task;

[0008] Controlling the target node to execute the image management task, and generating image management task result information when the target node completes the execution;

[0009] The image management task result information is sent to the central cluster, so as to display the image management task result information through a visualization interface of the central cluster.

[0010] In one embodiment, the receiving image management task sent by the central cluster to the edge cluster includes:

[0011] Monitoring a task configuration file preset in the edge cluster;

[0012] If it is monitored that the preset task configuration file changes in response to the task issuing instruction of the central cluster, it is determined that the image management task issued by the central cluster to the edge cluster is received.

[0013] In one embodiment, controlling the target node to perform the image management task includes:

[0014] Obtaining scheduled task execution information for configuring the execution time of the image management task;

[0015] The target node is controlled to execute the image management task at a predetermined time specified by the scheduled task execution information.

[0016] In one embodiment, screening the nodes in the edge cluster to determine the target nodes related to the image management task includes:

[0017] Matching the node target task information of each of the nodes indicated by the task configuration file with the node current task information indicated by each of the nodes;

[0018] If the node target task information is consistent with the node current task information, the node target task information is continued to be matched with the node current task information indicated by the next node;

[0019] If there is a difference between the node target task information and the node current task information, the node corresponding to the node current task information is determined as the target node, and the target node task information is used to update the current node task information indicated by the target node.

[0020] In one embodiment, before controlling the target node to complete the corresponding type of image management task, the method further includes:

[0021] Binding the target node corresponding to the image management task with a task lock; the task lock is used to fix the target node thread so that the target node executes the serial image management task;

[0022] In the case of completing the image management task, after generating the image management task result information, the method further includes:

[0023] The task lock is unbound so that the target node receives and executes the image management task again.

[0024] In one of the embodiments, after determining the target node corresponding to the task configuration file, the method further includes:

[0025] When it is determined that the type of the image management task is to clean up expired images, obtaining image management task result information corresponding to the target node;

[0026] Calculating the task execution period of the image management task in the image management task result information;

[0027] If the task execution cycle indicates that the image management task executed by the target node is expired, the target node is controlled to delete the expired image management task in the image management task result information.

[0028] In a second aspect, the present application also provides a cluster image management device, including:

[0029] The image management task receiving module receives the image management tasks sent by the central cluster to the edge cluster;

[0030] A node screening module, screening the nodes in the edge cluster to determine the target nodes related to the image management task;

[0031] A task execution module controls the target node to execute the image management task, and generates image management task result information when the target node completes the execution;

[0032] The task result feedback module sends the image management task result information to the central cluster so as to display the image management task result information through a visual interface of the central cluster.

[0033] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0034] Receive image management tasks sent by the central cluster to the edge cluster;

[0035] Screening the nodes in the edge cluster to determine the target nodes related to the image management task;

[0036] Controlling the target node to execute the image management task, and generating image management task result information when the target node completes the execution;

[0037] The image management task result information is sent to the central cluster, so as to display the image management task result information through a visualization interface of the central cluster.

[0038] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0039] Receive image management tasks sent by the central cluster to the edge cluster;

[0040] Screening the nodes in the edge cluster to determine the target nodes related to the image management task;

[0041] Controlling the target node to execute the image management task, and generating image management task result information when the target node completes the execution;

[0042] The image management task result information is sent to the central cluster, so as to display the image management task result information through a visualization interface of the central cluster.

[0043] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0044] Receive image management tasks sent by the central cluster to the edge cluster;

[0045] Screening the nodes in the edge cluster to determine the target nodes related to the image management task;

[0046] Controlling the target node to execute the image management task, and generating image management task result information when the target node completes the execution;

[0047] The image management task result information is sent to the central cluster, so as to display the image management task result information through a visualization interface of the central cluster.

[0048] The above-mentioned cluster image management method, device, computer equipment, computer-readable storage medium and computer program product receive the image management task issued by the central cluster to the edge cluster; screen the nodes in the edge cluster to determine the target node related to the image management task; control the target node to execute the image management task, and generate the image management task result information when the target node is executed; send the image management task result information to the central cluster to display the image management task result information through the visual interface of the central cluster. In this application, the edge cluster receives the image management task from the central cluster, laying the foundation for the unified management of distributed tasks; by screening the nodes related to the image management task, accurately locating the task execution scope, and by controlling the target node to automatically execute the image management task, the decoupling between the central cluster and the edge node is achieved, and the resource utilization efficiency of the system is improved; the feedback task result information is reflected in the visual interface, forming a view of the distributed task execution status, reducing the operation and maintenance cost, thereby achieving the improvement of the efficiency of the entire cluster image management while ensuring the efficiency of edge computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A framework diagram of the interaction between an edge cluster and a central cluster in one embodiment;

[0051] Figure 2 A schematic diagram of a process flow of a cluster image management method in one embodiment;

[0052] Figure 3 A framework diagram of a node image management system in an edge cluster in an embodiment;

[0053] Figure 4 A schematic diagram of a flow chart of a method for manually executing an image management task in one embodiment;

[0054] Figure 5 A schematic diagram of a flow chart of a method for timing execution of image management tasks in one embodiment;

[0055] Figure 6 A structural block diagram of a cluster image management method and device in an embodiment;

[0056] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] In order to enable those skilled in the art to better understand the present application, the relevant technology is first introduced below.

[0059] Kubernetes (often referred to as K8s) is an open source container orchestration and management platform that aims to simplify the management of containerized applications and automate the deployment, expansion and operation processes. As an important part of the modern container ecosystem, K8s provides flexible and efficient management support for applications through resource objects and controller mechanisms. Among them, Configmap is a key K8s resource object used to store application configuration data. It can save various configuration information, such as environment variables, command line parameters, and configuration files, thereby decoupling applications from configurations, making configuration changes simpler and more flexible. DaemonSet is another important K8s resource object that is used to ensure that a replica (Pod) runs on each eligible node in the cluster, thereby achieving consistent task distribution and execution.

[0060] In the containerized technology stack, Docker plays a key role as the underlying container engine. Docker is an open source platform and tool that focuses on virtualization at the operating system level and supports containerized management of applications. Developers can use Docker to encapsulate applications and all their dependencies (such as libraries, tools, and configuration files) into an independent, portable container and run these containers in any environment that supports Docker. Docker Image is the core component of Docker technology. As a read-only template, it contains the file system and configuration required to run the container. The image is defined by the Dockerfile file and generated through building, and can then be shared and distributed through the Docker repository. However, in daily use, due to interruptions or improper operations (such as building, pushing, or deleting), Docker dangling images are often generated. These dangling images are not associated with any tags or containers, and are usually intermediate products or temporary images, which easily lead to waste of storage resources.

[0061] In recent years, with the vigorous development of the artificial intelligence industry, more and more universities and enterprises have actively participated in it, promoting the comprehensive deepening of technological innovation and application. In this context, in order to improve resource utilization and work efficiency, most institutions choose Kubernetes (K8s) as a unified orchestration and scheduling management platform to realize the automation and unified management of training resources. However, the high demand for computing and storage resources in the AI ​​model training process has brought new challenges, among which the image management problem is particularly prominent. The image volume that AI model training relies on is usually very large, which makes the image pulling process time-consuming and directly affects the execution efficiency of the task. At the same time, due to operational errors or interruptions of training tasks, a large number of dangling images are often generated. These images not only take up valuable disk space, but also cause great waste and management burden on system storage resources. Although Docker provides instructions for cleaning up dangling images, operation and maintenance personnel still need to manually log in to the edge nodes of the cluster one by one to perform operations. As the scale of cloud clusters continues to expand, the number of edge nodes and the amount of image management tasks have increased dramatically. This manual method is obviously difficult to meet the needs of efficient operation and maintenance.

[0062] In order to solve the above problems, many cloud companies have customized business operation management platforms (Ops) to provide operation and maintenance personnel with a centralized platform for monitoring and operating edge cluster resources. However, with the continuous expansion of the scale of K8s clusters and the increasing number of images they carry, this centralized management model is also facing new bottlenecks. Operation and maintenance personnel cannot directly view the image information or storage status of edge nodes through the central platform, but need to log in to the nodes one by one to obtain relevant data, which further reduces management efficiency. At the same time, although operations such as image cleanup are highly repetitive and can be executed on a scheduled basis, they still require manual intervention by operation and maintenance personnel. This approach not only increases the workload, but also significantly increases human resource costs, further exacerbating the complexity of system management.

[0063] The cluster image management method provided in the embodiment of the present application can be applied to Figure 1 The interactive framework of the central cluster and edge cluster shown in the figure can reduce the waste of storage resources, reduce the duplication of operation and maintenance personnel, and improve the overall resource utilization efficiency and management effect of the edge cluster. Among them, the central cluster and the edge cluster are deployed on the server. The central cluster is the core control unit in the distributed computing system. It is deployed in the data center and is responsible for task scheduling, status monitoring and resource allocation. Its role is to generate and distribute image management tasks to the edge cluster through a unified task management platform. The edge cluster is composed of nodes deployed close to the data source or user end, with the characteristics of rapid response, focusing on localized computing and resource management.

[0064] The central cluster has an operation management platform server, and the edge cluster connected to the central cluster can be one or more. Each edge cluster has a task configuration file in the node management namespace. Through the task configuration file, the edge cluster can interact with the business operation management platform of the central cluster. Each edge cluster has one or more node image management systems, and the node image management system is deployed on certain nodes depending on the configuration parameters of the Helm package installed. Specifically, the present invention is deployed through the Helm package, and the node image management system is screened by the NodeSelector (node ​​filter) of DaemonSet to deploy the node image management system. This system is mainly installed on nodes with heterogeneous chips such as GPU and NPU, which is convenient for managing large-volume image files. The node image management system listens to the task configuration file and executes related tasks. Each node management system will have a corresponding node image information and task execution result file, which is used to provide a query node image information entry for the operation management platform of the central cluster and save the manual task execution results. For installing the Helm package, the present invention has the following resource types, including ServiceAccount (identity identification), Role (authority role), RoleBinding (role binding), Configmap and Deamonset. Configmap is responsible for the orchestration of sending the task configuration file image-manager-task-cm. Deamonset is used to deploy the node management system. The configurable startup parameters include the time and interval of node image scheduled refresh, node image scheduled cleanup, and node image file scheduled cleanup, the expiration time of cleaning up dangling images, and whether to force deletion of the specified image. In particular, the docker-socket (container socket) needs to be specified in the container orchestration in Deamonset. Specifically, the path value is / var / run / docker.sock. ServiceAccount, Role, and RoleBinding are used to allow the node management system to operate the image information and task execution result files under the node namespace.

[0065] In an exemplary embodiment, Figure 2 As shown, a cluster image management method is provided, which is applied to Figure 1 The edge cluster in FIG. 1 is taken as an example to illustrate, including the following steps S202 to S208. Among them:

[0066] Step S202: receiving the image management task sent by the central cluster to the edge cluster.

[0067] Among them, the image management task is the operation instruction issued by the central cluster, involving operations such as pulling, updating, and cleaning up container images, aiming to ensure that the image status on the edge node is consistent with business requirements.

[0068] For example, the central cluster generates image management tasks according to the preset task rules, and distributes tasks to edge nodes through the native Configmap mechanism of K8s. The content of image management tasks includes synchronizing the latest image version, cleaning redundant images, or counting image status, etc. The task configuration data is stored in the Configmap in the form of key-value pairs. The image management system of the edge cluster will monitor the status changes of the Configmap in real time, and start task processing once an update is detected.

[0069] Step S204: Screen the nodes in the edge cluster to determine the target nodes related to the image management task.

[0070] Among them, the target node is controlled by the image management system, which triggers the specific task logic through the issuance and analysis of task instructions, completes the specified task, and generates corresponding task result information to feed back to the central cluster.

[0071] Exemplarily, before executing a task, the edge image management system identifies nodes that meet the task requirements through a screening mechanism, and comprehensively compares the task target information with the actual state of the node. In one embodiment, based on the comparison between the target information in the task configuration file and the node attributes, screening can be performed based on the difference in the comparison, or screening can be performed based on the node range issued by the unified judgment of the central cluster to screen out the target nodes in the edge cluster for executing related tasks.

[0072] Step S206, controlling the target node to execute the image management task, and generating image management task result information when the target node completes the execution.

[0073] Among them, the task result information is the execution record generated by the node after the task is completed, including but not limited to the task ID, execution status, timestamp, etc., which is used to feedback the task completion status and support subsequent management operations.

[0074] Exemplarily, the edge cluster triggers the node to perform the corresponding image management task according to the target node list. First, the content of the image management task is parsed, including the task type and execution parameters, and passed to the target node; then, the target node performs specific operations according to the task type. For example, for the image cleanup task, the node calls the local container engine interface to delete the specified image; for the image synchronization information task, the node updates its image record and synchronizes the status to the central cluster.

[0075] After the task is completed, the target node generates result information including the task ID, completion status and execution time, which is stored locally or directly returned to the central cluster for subsequent task auditing and monitoring.

[0076] Step S208: sending the image management task result information to the central cluster, so as to display the image management task result information through a visual interface of the central cluster.

[0077] Among them, the visualization interface is a tool deployed in the central cluster for operation and monitoring by operation and maintenance personnel. It presents the status of image management tasks and the image resource status of the node in an intuitive form, which facilitates rapid diagnosis of problems and optimization of resource allocation.

[0078] Exemplarily, after completing the image management task, the target node transmits the generated task result information to the central cluster through the network. The transmission method may include the target node directly updating the result information through the K8s Configmap resource; or using the application program interface to send the task result information in JSON format to the operation and maintenance management system of the central cluster. After receiving the task result, the central cluster stores it in the task result database or memory cache, and displays it to the operation and maintenance personnel through the visual interface of the operation management platform. The operation and maintenance personnel can view the task execution status, image cleanup details or synchronization progress of a specific node through the interface, and optimize the image management strategy based on the task result data.

[0079] In the above cluster image management method, the image management task sent by the central cluster to the edge cluster is received; the nodes in the edge cluster are screened to determine the target node related to the image management task; the target node is controlled to execute the image management task, and when the target node is executed, the image management task result information is generated; the image management task result information is sent to the central cluster to display the image management task result information through the visual interface of the central cluster. In this embodiment, the central cluster distributes the image management instructions to the edge cluster through a unified task configuration, laying the foundation for the unified management of distributed tasks; by screening the nodes related to the image management task, the task execution scope is accurately located, and by controlling the target node to automatically execute the image management task, the decoupling between the central cluster and the edge node is achieved, and the resource utilization efficiency of the system is improved; the feedback task result information is reflected in the visual interface, forming a view of the distributed task execution status, reducing the operation and maintenance costs, thereby achieving the improvement of the efficiency of the entire cluster image management while ensuring the efficiency of edge computing.

[0080] In an exemplary embodiment, receiving an image management task sent by a central cluster to an edge cluster includes: monitoring task configuration information pre-set in the edge cluster; if it is monitored that the pre-set task configuration information changes in response to a task issuance instruction of the central cluster, it is determined that the image management task sent by the central cluster to the edge cluster is received.

[0081] Exemplarily, the user inputs the task placement instruction through the visual interface. The central cluster parses the task placement instruction and filters out the completed task records in combination with the existing image management task result information. During the image management task generation process, the central cluster checks whether the adjusted task configuration information meets the storage size constraint. If it exceeds the limit, it further optimizes the task record or issues the task in batches.

[0082] In the image management system, the task receiving process of the edge node is implemented by monitoring the task configuration file. When the central cluster needs to send an image management task to the edge node, it will record the image management task by updating the task configuration file. When the image management system is running, it continuously monitors the status changes of the task configuration file. When an update event of the task configuration file is detected, the edge cluster will read the updated content and parse the task type, task parameters and target node information.

[0083] In a specific embodiment, the operator issues a task to a node of an edge cluster on the operation management platform of the central cluster, and uses the Configmap resource that comes with k8s as the carrier of the task configuration file image-manager-task-cm. The node management namespace is specifically image-manager-task-cm. The operation management platform itself has saved the kubeconfig of the edge cluster to manage the entire cluster. Therefore, the purpose of manually issuing tasks can be achieved by writing specific task information into the task configuration file. Specifically, the parameters of the structure of the issued task include the task ID, task action type, task issuance time and task parameters. Among them, the task type can specifically include refreshing node image information, cleaning up specified images, and cleaning up suspended images, etc. When the task action type is to clean up specified images, the task parameters are required parameters, which need to be passed in as a serialized string of slices composed of the name of the specified image and a combination of labels.

[0084] Write the node name and target task information into the task configuration file image-manager-task-cm under the node management namespace. Specifically, the central cluster operation management platform reads the fields entered by the operator and composes them into a new JSON-formatted task NewTask with task ID (TaskId), task action (Action), task delivery time (Time) and task execution parameters (Args); then. Obtain the cluster's permissions through the cluster's kubeconfig, and then obtain the partial content ContentT of the node of image-manager-task-cm and the image information of the node and the content ContentN of the task execution result file; then, deserialize ContentT and ContentN respectively to obtain the manual task slice TaskSlice and the result slice ResSlice. By comparison, remove the tasks that have been executed by TaskSlice and the tasks that have been issued for more than two days to obtain TaskSliceSlim, ensuring that the size of image-manager-task-cm does not exceed the limit of k8s file memory size; then, add the new task NewTask to the slice of TaskSliceSlim, serialize TaskSliceSlim, and obtain TaskString; finally, use the node name as the key key and TaskString as the value value, and save it to image-manager-task-cm. Specifically, the present invention uses k8s version 1.20, so the Configmap resource in this example has a size limit of 1MB. In the specific example of the present invention, the content in image-manager-task-cm is a collection of many node names and task serialization strings, such as [{nodeA:TaskStringA}, {nodeB:TaskStringB}, ...{nodeC:TaskStringC}].

[0085] In this embodiment, by monitoring the task configuration information, a rapid response to the task issuance instructions is achieved, thereby significantly improving the triggering efficiency of distributed tasks. Further, the communication frequency between the central cluster and the edge cluster is reduced, the resource utilization efficiency of the system is optimized, and basic support is provided for the dynamic scheduling of distributed image management tasks.

[0086] In an exemplary embodiment, the nodes in the edge cluster are screened to determine the target nodes related to the image management task, including: matching the node target task information of each node indicated by the task configuration information with the node current task information indicated by each node; if the node target task information and the node current task information are consistent, then continuing to match the node target task information with the node current task information indicated by the next node; if there is a difference between the node target task information and the node current task information, then determining the node corresponding to the node current task information as the target node, and updating the current node task information indicated by the target node with the target node task information.

[0087] Among them, the node target task information is the instruction issued by the central cluster through the task configuration information, which contains the specific content of the image management task that the target node needs to complete. The node current task information is the status record of the task currently being executed or completed by each edge node.

[0088] Specifically, after receiving the task configuration information of the central cluster, the image management system of the edge cluster parses the target task information of each node in the task configuration information. The image management system checks the current task information of each node one by one. If the target task information is completely consistent with the current task information, the node is skipped and the next node is checked. If the target task information of a node is found to be inconsistent with the current task information, the node is determined as the target node. The target task information is updated to the current task record of the node, and the node is marked as needing to execute the task. All nodes are traversed, and finally a target node list is generated for subsequent task distribution.

[0089] In this embodiment, through the node-by-node matching of task information, the target nodes that need to execute tasks can be accurately screened out to avoid repeated tasks or waste of resources on irrelevant nodes. At the same time, through differentiated judgment and status update, the system ensures the consistency of task instruction allocation and node status, thereby improving the accuracy and real-time performance of task scheduling.

[0090] In an exemplary embodiment, before controlling the target node to complete the corresponding type of image management task, it also includes: binding the target node corresponding to the image management task with a task lock; the task lock is used to fix the target node thread so that the target node executes a serial image management task; when the image management task is completed, after generating the image management task result information, it also includes: unbinding the task lock so that the target node receives the image management task again and executes it.

[0091] Among them, task lock is a control mechanism used to coordinate the execution order of multiple tasks in the target node to prevent concurrent task operations from causing resource competition or execution conflicts. Thread pinning is part of the task lock mechanism, which is used to lock the node's operation thread to the current task to ensure that the task is completed in order in single-threaded mode.

[0092] Exemplarily, after the target node receives the image management task, the image management system will bind the task lock to the target node and fix the execution thread of the current task. The node thread executes the target task under the protection of the task lock to avoid concurrent writing or data overwriting problems. After the task is executed, the target node generates task result information including the task ID, execution status and completion time, and stores it in the node task result record file. After the task result information is generated, the image management system automatically releases the task lock, releases the thread fixed state, and enables the node to receive new task instructions and continue execution.

[0093] In a specific embodiment, when the target task is to refresh the node image information task, the management operation of the node container image is implemented by calling the Docker-API (docker-client), and the locking mechanism is used to ensure that when executing the task, manual tasks and scheduled tasks are prevented from accessing the image information or task execution result files on the node at the same time to prevent data competition and inconsistent status. Call the ImageList method in docker-client to retrieve all existing image lists on the node and provide the meta information of the image. Update the retrieved image data to the task execution result file for query and management by the central cluster. Finally, release the task lock, end the task execution, and allow the node to continue processing new tasks.

[0094] In a specific embodiment, when the target task is to clean up the specified image task, docker-client is called to implement the management operation of the node container image, and a task lock is added to prevent manual tasks and scheduled tasks from accessing the node's task files and image data at the same time. Read the task parameters from the task configuration file, parse the image list that needs to be cleaned up, and these lists are recorded in the form of Registry (image registry), Tag (label) or ImageID (image file name). Process each ImageTasks (image task) in the image list item by item, and call the image deletion interface in docker-client to delete the corresponding image. Use the ImageList method of docker-client to obtain the node image list, record the latest image status in the task execution result file, and provide the latest data feedback for the central cluster. Finally, release the task lock, end the task execution, and allow the node to continue to process new tasks.

[0095] In a specific embodiment, when the target task is to clean up the dangling image task, docker-client is called to implement the management operation of the node container image, and a task lock is added to prevent manual tasks and scheduled tasks from accessing the node's task files and image data at the same time. The ImagePrune function of docker-client is used to clean up and trim the dangling images on the node. After the cleaning is completed, the ImageList interface is called to obtain the latest node image list, and the updated image information is saved in the task execution result file for the central cluster to query or analyze. Finally, the task lock is released to end the task execution so that the node can continue to process new tasks.

[0096] In this embodiment, the task lock mechanism prevents resource competition and state confusion caused by concurrent execution of multiple tasks on the same node by binding node threads, ensuring serialized task execution and improving the success rate of image management tasks. At the same time, the task lock mechanism enables the system to dynamically adapt to different task loads and priorities, further improving the flexibility of task scheduling and the scalability of the system.

[0097] In an exemplary embodiment, receiving an image management task sent by a central cluster to an edge cluster includes: obtaining scheduled task execution information for configuring the execution time of the image management task, and triggering the image management task at a predetermined time specified by the scheduled task execution information.

[0098] The scheduled task execution information is the parameter configuration used by the system to set the task execution time, including the task trigger time point, execution interval and trigger conditions.

[0099] Specifically, the image management system of the edge cluster parses the scheduled task execution information by listening to the task configuration file. The image management system configures the timer according to the scheduled task execution information. At the scheduled time specified by the scheduled task, the timer triggers the image management task.

[0100] In a specific embodiment, if the scheduled task is started for the first time, the node image information will be directly obtained and written into the node image information and task execution result file corresponding to the node. And if the task execution result file does not exist, it is necessary to create the task execution result file first. After completing the above operations, the timer of each scheduled task will be turned on. Specifically, scheduled tasks include regular refresh of node image information, regular cleaning of suspended images, and regular cleaning of expired tasks. Among them, the timer for regular refresh of node image information in the example of the present invention is set to the next hour of the current time, with an interval of 1 hour; the timers for regular cleaning of garbage images and regular cleaning of expired tasks are respectively set to 0:30 and 2:30 in the morning of the next day of the current time, with an interval of 24 hours, and the operation is chosen when the business traffic is small, in order to minimize the impact on the edge cluster.

[0101] In this embodiment, the image management task can be accurately triggered through the scheduled task execution information. The deployed edge node can automatically clean up the suspended images and refresh the node images, reducing the waste of storage resources and reducing the repetitive workload of back-end operation and maintenance personnel who need to manually enter the edge node side to perform operations, thereby reducing the operation and maintenance manpower costs and improving operational efficiency.

[0102] In an exemplary embodiment, after determining the target node corresponding to the task configuration information, it also includes: when it is determined that the type of image management task is to clean up expired images, obtaining the image management task result information corresponding to the target node; calculating the task execution cycle of the image management task in the image management task result information; if the task execution cycle indicates that the image management task executed by the target node is expired, controlling the target node to delete the expired image management task in the image management task result information.

[0103] Among them, the task execution cycle calculation is the process in which the system determines whether the task exceeds the execution time limit by comparing the timestamp in the task record with the current time.

[0104] Specifically, expired tasks are cleaned up regularly to ensure that the file size of the image information and task execution result files of the current node does not exceed the limit of k8s. Specifically, the node image management system will read the data ContentN of the task execution result part of the image information and task execution result file, take out the value of the task result key ResString, deserialize it to get ResSlice, traverse ResSlice, delete expired data, save NewResSlice after traversal, and serialize it to NewResString, and finally write the image information and task execution result file.

[0105] In this embodiment, the scheduled tasks and task record cleaning mechanism maintain the size and status of the file to ensure efficient use of node storage resources.

[0106] In order to enable those skilled in the art to better understand the above steps, the embodiment of the present application is illustrated below by using an example, but it should be understood that the embodiment of the present application is not limited to this.

[0107] In a specific embodiment, the image management system runs in an edge computing environment, where the edge cluster consists of multiple nodes, each of which has storage and computing capabilities to support the deployment and operation of containerized applications. Figure 3As shown in the figure, the edge cluster can implement manual task processing and periodic task execution. The central cluster is responsible for the generation, scheduling and management of global tasks. Image management tasks include operations such as cleaning images and synchronizing image information, aiming to optimize the image resource utilization of edge nodes. Image management tasks are stored and issued in the form of Configmap. The task execution process involves the screening of target nodes, task lock mechanism, scheduled task triggering, and the generation and cleaning of task results.

[0108] The steps for manually implementing image management are as follows: Figure 4 As shown, in step S402, the image management system of the edge cluster monitors the changes of Configmap in real time through the Informer mechanism.

[0109] In step S404, the central cluster responds to the image management task instructions input by the operation and maintenance personnel on the visual interface, and synchronizes the image management task instructions to the edge cluster by updating the task configuration file (Configmap). The edge cluster receives the image management task sent by the center. The visual interface is formed based on the node image information and task result information in the edge cluster.

[0110] Step S406, when the image management system detects a new task instruction, it parses the task content and determines whether it is related to the current node. The parsed content includes the task type, target image parameters, etc. The image management system compares the target task in the task configuration information with the node current task information. If the node target task is consistent with the node current task information, the image management system will skip the current node; if the target task is inconsistent with the current task information, the image management system marks the node as a target node and updates its current task record.

[0111] Step S408, if the current node is determined to be the target node in step S406, further determine whether the task name (TaskID) corresponding to the target node exists in the image and task execution file of the node. If it exists, it proves that it is a task that has been executed and will not be executed again; if it does not exist, it is considered to be an unprocessed manual task, and the node image management system executes the corresponding task processing program according to the type of image management task. In the process of executing the image management task, set a task lock for the target node to ensure the smooth execution of the current task. After the current image management task is executed, release the task lock to execute other tasks.

[0112] Step S410, after completing the image management task, the image information of the target node is refreshed and the task result information of the task to be executed is written into the image information and task result file of the node.

[0113] For the regular execution of mirroring tasks, such as Figure 5As shown, in step S502, the image management system of the edge cluster starts the timer and triggers the task at a predetermined time point according to the scheduled task execution information configured by the central cluster.

[0114] Step S504, when the timer is triggered, a specific image management task is triggered. The types of specific image management tasks include regularly refreshing node image information, regularly cleaning up junk images, and regularly cleaning up expired tasks.

[0115] Step S506, after completing the image management task, the image information of the target node is refreshed and the task result information of the task to be executed is written into the image information and task result file of the node.

[0116] In this embodiment, the Configmap resource of k8s is used to construct a message queue and realize effective interaction between the center and the edge without relying on other message queues. The node management system on the edge writes the node image information into the image information file of the node, and the center reads the image information file of the node to visualize the image information of the edge segment. At the same time, it supports operation and maintenance personnel to manually issue image management tasks on the business operation management platform on the center side, simplifying and improving the management process of edge node images to improve resource utilization efficiency and management effect.

[0117] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0118] Based on the same inventive concept, the embodiment of the present application also provides a cluster image management device for implementing the cluster image management method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more cluster image management device embodiments provided below can refer to the limitations of the cluster image management method above, and will not be repeated here.

[0119] In an exemplary embodiment, Figure 6As shown, a cluster image management device is provided, including: an image management task receiving module 610, a node screening module 620, a task execution module 630 and a task result feedback module 640, wherein:

[0120] An image management task receiving module 610 receives an image management task sent by a central cluster to an edge cluster;

[0121] A node screening module 620 screens the nodes in the edge cluster to determine a target node related to the image management task;

[0122] The task execution module 630 controls the target node to execute the image management task, and generates image management task result information when the target node completes the execution;

[0123] The task result feedback module 640 sends the image management task result information to the central cluster so as to display the image management task result information through the visual interface of the central cluster.

[0124] In one embodiment, the image management task receiving module 610 is also used for receiving the image management task sent by the central cluster to the edge cluster, including: monitoring the pre-set task configuration file in the edge cluster; if it is monitored that the pre-set task configuration file changes in response to the task issuance instruction of the central cluster, it is determined that the image management task sent by the central cluster to the edge cluster is received.

[0125] In one embodiment, the image management task receiving module 610 is also used to control the target node to execute the image management task, including: obtaining scheduled task execution information for configuring the execution time of the image management task; and controlling the target node to execute the image management task at the predetermined time specified by the scheduled task execution information.

[0126] In one embodiment, the node screening module 620 is also used to screen the nodes in the edge cluster and determine the target nodes related to the image management task, including: matching the node target task information of each node indicated by the task configuration file with the node current task information indicated by each node; if the node target task information is consistent with the node current task information, then continuing to match the node target task information with the node current task information indicated by the next node; if there is a difference between the node target task information and the node current task information, then determining the node corresponding to the node current task information as the target node, and using the target node task information to update the current node task information indicated by the target node.

[0127] In one embodiment, the task execution module 630 is also used for controlling the target node to complete the corresponding type of image management task before, and also includes: binding the target node corresponding to the image management task with a task lock; the task lock is used to fix the target node thread so that the target node executes a serial image management task; when the image management task is completed, after generating the image management task result information, it also includes: unbinding the task lock so that the target node receives the image management task again and executes it.

[0128] In one embodiment, the task execution module 630 is also used for determining the target node corresponding to the task configuration file, and also includes: when it is determined that the type of the image management task is to clean up expired images, obtaining the image management task result information corresponding to the target node; performing a task execution cycle on the image management task in the image management task result information; calculating if the task execution cycle indicates that the image management task executed by the target node is expired, controlling the target node to delete the expired image management task in the image management task result information.

[0129] Each module in the above cluster image management device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0130] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store node mirror information and task result information data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a cluster mirror management method is implemented.

[0131] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0132] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0133] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0134] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0135] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0136] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0137] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0138] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A cluster image management method, characterized in that: The method comprises: Receive image management tasks sent by the central cluster to the edge cluster; Screening the nodes in the edge cluster to determine the target nodes related to the image management task; Controlling the target node to execute the image management task, and generating image management task result information when the target node completes the execution; The image management task result information is sent to the central cluster, so as to display the image management task result information through a visualization interface of the central cluster.

2. The method according to claim 1, characterized in that The image management task sent by the receiving center cluster to the edge cluster includes: Monitoring a task configuration file preset in the edge cluster; If it is monitored that the preset task configuration file changes in response to the task issuing instruction of the central cluster, it is determined that the image management task issued by the central cluster to the edge cluster is received.

3. The method according to claim 1, characterized in that The controlling the target node to perform the image management task includes: Obtaining scheduled task execution information for configuring the execution time of the image management task; The target node is controlled to execute the image management task at a predetermined time specified by the scheduled task execution information.

4. The method according to claim 1, characterized in that: The screening of the nodes in the edge cluster to determine the target nodes related to the image management task includes: Matching the node target task information of each of the nodes indicated by the task configuration file with the node current task information indicated by each of the nodes; If the node target task information is consistent with the node current task information, the node target task information is continued to be matched with the node current task information indicated by the next node; If there is a difference between the node target task information and the node current task information, the node corresponding to the node current task information is determined as the target node, and the target node task information is used to update the current node task information indicated by the target node.

5. The method according to claim 1, characterized in that Before controlling the target node to complete the corresponding type of image management task, the method further includes: Binding the target node corresponding to the image management task with a task lock; the task lock is used to fix the target node thread so that the target node executes the serial image management task; In the case of completing the image management task, after generating the image management task result information, the method further includes: The task lock is unbound so that the target node receives and executes the image management task again.

6. The method according to any one of claims 1 to 5, characterized in that: After determining the target node related to the image management task, the method further includes: When it is determined that the type of the image management task is to clean up expired images, obtaining image management task result information corresponding to the target node; Calculating the task execution period of the image management task in the image management task result information; If the task execution cycle indicates that the image management task executed by the target node is expired, the target node is controlled to delete the expired image management task in the image management task result information.

7. A cluster image management device, characterized in that: The device comprises: The image management task receiving module receives the image management tasks sent by the central cluster to the edge cluster; A node screening module, screening the nodes in the edge cluster to determine the target nodes related to the image management task; A task execution module controls the target node to execute the image management task, and generates image management task result information when the target node completes the execution; The task result feedback module sends the image management task result information to the central cluster so as to display the image management task result information through a visual interface of the central cluster.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images