A mirror data processing method and device

Abstract

The application relates to the technical field of data processing, in particular to an image data processing method and device. The method is applied to each node of a cluster storing image data. The method comprises the following steps: deploying a Harbor instance and a Registry instance and establishing a synchronization mechanism between the Harbor instance and the Registry instance based on a deployment instruction; receiving image data uploaded by a user client, wherein the image data comprises data of each image package after deduplication processing and metadata of each image package; loading the image data to a file directory of the Registry instance; and synchronizing the image data from the Registry instance to the Harbor instance.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and apparatus for processing mirror data. Background Technology

[0002] Harbor is an open-source solution for building enterprise-grade private Docker image repositories. It adds essential features for enterprise users, such as access control, image signing, security vulnerability scanning, and remote synchronization, to DockerRegistry. It also provides a graphical management interface and Chinese language support for domestic users, making it a mainstream container image repository for cloud-native users in China.

[0003] The current Harbor image synchronization mechanism involves creating projects, repositories, and synchronization rules on each deployed Harbor node. On a specific Harbor instance node, the image package is first loaded into the Docker Daemon using the `Docker load` command, and then pushed to the Harbor repository using the `Docker push` command. Harbor then uses synchronization rules to synchronize the image to other Harbor instances. This approach has several drawbacks. First, image transfer is significantly affected by network and disk I / O, severely impacting the consistency of image synchronization results across Harbor instances. Second, loading images using the `Docker load` command is heavily reliant on disk I / O, and using Harbor's synchronization rules for image synchronization requires cross-node communication, making it highly dependent on network connectivity. Summary of the Invention

[0004] This application provides a method and apparatus for processing mirror data.

[0005] In a first aspect, this application provides a method for processing mirrored data, applied to nodes in a cluster that stores mirrored data, the method comprising:

[0006] Based on the deployment instructions, deploy a Harbor instance and a Registry instance, and establish a synchronization mechanism between the Harbor instance and the Registry instance;

[0007] Receive image data uploaded by user clients, wherein the image data includes the data of each image package after deduplication and the metadata of each image package;

[0008] The image data is loaded into the file directory of the Registry instance, and the image data is synchronized from the Registry instance to the Harbor instance.

[0009] Optionally, the method further includes:

[0010] After confirming that the aforementioned mirror number synchronization is complete, delete the Registry instance.

[0011] Optionally, the method further includes:

[0012] Receive a target Pod creation request sent by the client, parse the image information in the request parameters, the image information including the name of the target image required to create the target Pod;

[0013] Based on the image information, determine whether there is a target project corresponding to the target image;

[0014] If the target project does not exist, then create the target project corresponding to the target image; if the target project exists, then determine whether there is a target repository corresponding to the target project.

[0015] If the target repository does not exist, then create the target repository corresponding to the target project; if the target repository exists, then determine whether there is a target synchronization rule corresponding to the target project.

[0016] If the target synchronization rule does not exist, then create the target synchronization rule corresponding to the target project; if the target synchronization rule exists, then end the judgment process.

[0017] Optionally, the image information further includes the pull strategy for the target image and the key required to pull the target image; the method further includes:

[0018] Based on the image information and the metadata information of the target image, the image data of the target image is pulled from the image data, and the target Pod is created based on the pulled image data of the target image.

[0019] Optionally, project information corresponding to each image and repository information corresponding to each project are preset, and the method includes:

[0020] Parse the name of the target image, determine the name of the target project corresponding to the target image, and perform the step of determining whether the target project corresponding to the target image exists;

[0021] Based on the target project name, determine the target repository corresponding to the target project, and execute the step of determining whether a target repository corresponding to the target project exists;

[0022] The steps for creating a target synchronization rule for the target project after determining that the target synchronization rule does not exist include:

[0023] Based on the information of each node in the cluster, a target synchronization rule is created to synchronize the updated mirror data of this node to other nodes.

[0024] Secondly, this application provides a mirror data processing apparatus, applied to each node of a cluster storing mirror data, the apparatus comprising:

[0025] The deployment unit is used to deploy Harbor instances and Registry instances based on deployment instructions, and to establish a synchronization mechanism between the Harbor instances and the Registry instances.

[0026] The receiving unit is used to receive image data uploaded by the user client, wherein the image data includes the data of each image package after deduplication and the metadata of each image package;

[0027] The processing unit is used to load the image data into the file directory of the Registry instance and synchronize the image data from the Registry instance to the Harbor instance.

[0028] Optionally, the device further includes:

[0029] The deletion unit is used to delete the Registry instance after determining that the mirror number synchronization is complete.

[0030] Optionally, the apparatus further includes a parsing unit, a judging unit, and a creating unit:

[0031] The receiving unit is also used to receive a target Pod creation request sent by the client;

[0032] The parsing unit is used to parse the image information in the request parameters, the image information including the name of the target image required to create the target Pod;

[0033] The determination unit is used to determine, based on the image information, whether there is a target project corresponding to the target image;

[0034] If the target project does not exist, the creation unit is used to create the target project corresponding to the target image; if the target project exists, the judgment unit is also used to determine whether a target repository corresponding to the target project exists.

[0035] If the target repository does not exist, the creation unit is used to create a target repository corresponding to the target project; if the target repository exists, the judgment unit is also used to determine whether a target synchronization rule corresponding to the target project exists.

[0036] If the target synchronization rule does not exist, the creation unit is used to create the target synchronization rule corresponding to the target project; if the target synchronization rule exists, the judgment process ends.

[0037] Optionally, the image information further includes the pull strategy for the target image and the key required to pull the target image; the creation unit is also used for:

[0038] Based on the image information and the metadata information of the target image, the image data of the target image is pulled from the image data, and the target Pod is created based on the pulled image data of the target image.

[0039] Optionally, the device includes a predetermined project information corresponding to each image and a predetermined repository information corresponding to each project.

[0040] After the determining unit parses the name of the target image and determines the name of the target project corresponding to the target image, the judging unit performs the step of judging whether there is a target project corresponding to the target image;

[0041] After the determining unit determines the target warehouse corresponding to the target project based on the target project name, the judging unit performs the step of judging whether the target warehouse corresponding to the target project exists.

[0042] When the judgment unit determines that the target synchronization rule does not exist and creates a target synchronization rule corresponding to the target project, the creation unit is specifically used for:

[0043] Based on the information of each node in the cluster, a target synchronization rule is created to synchronize the updated mirror data of this node to other nodes.

[0044] Thirdly, embodiments of this application provide a mirror data processing apparatus, which includes:

[0045] Memory, used to store program instructions;

[0046] A processor is configured to invoke program instructions stored in the memory and execute the steps of the method as described in any one of the first aspects above, according to the obtained program instructions.

[0047] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the steps of the method as described in any of the first aspects above.

[0048] In summary, the image data processing method provided in this application is applied to each node of the cluster storing image data. The method includes: deploying a Harbor instance and a Registry instance based on deployment instructions, and establishing a synchronization mechanism between the Harbor instance and the Registry instance; receiving image data uploaded by a user client, wherein the image data includes data of each image package after deduplication and metadata of each image package; loading the image data into the file directory of the Registry instance, and synchronizing the image data from the Registry instance to the Harbor instance.

[0049] The image data processing method provided in this application effectively reduces the size of image data files by adjusting the delivery format of the image data. Based on the same-node image synchronization mechanism, image synchronization is achieved without relying on cross-node network I / O, effectively reducing network and disk I / O usage during image synchronization and the complexity of processing large batches of image data.

[0050] Furthermore, since Harbor projects, repositories, and synchronization rules are all triggered and automatically created by Pod creation events, the entire image synchronization operation can achieve the beneficial effects of low disk I / O, low network I / O, and low operational complexity. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings of the embodiments of this application.

[0052] Figure 1 A detailed flowchart of a mirror data processing method provided in an embodiment of this application;

[0053] Figure 2 This is a schematic diagram of a mirror data processing process provided in an embodiment of this application;

[0054] Figure 3 This is a schematic diagram of the structure of a mirror data processing device provided in an embodiment of this application;

[0055] Figure 4 This is a schematic diagram of the hardware architecture of a mirror data processing device provided in an embodiment of this application. Detailed Implementation

[0056] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “the,” and “the” as used in this application and claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any and all possible combinations comprising one or more of the associated listed items.

[0057] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" may also be interpreted as "when," "when," or "in response to a determination."

[0058] Currently, Harbor officially offers a Kubernetes-based deployment solution. This solution requires users to provide shared storage and ensure high availability of the database, such as deploying pgSQL and Redis components on each node of the shared storage cluster. The image synchronization process based on this solution is as follows:

[0059] 1. Deploy Harbor instances on each node of the Kubernetes cluster and create projects, repositories, and synchronization rules. For example, if the Kubernetes cluster consists of 3 nodes, deploy Harbor instances on each of the 3 nodes and create projects, repositories, and synchronization rules on each node.

[0060] 2. Use the Docker load command on a node where a Harbor instance resides (e.g., node 1) to load the image package (e.g., image package 1) into the Docker Daemon.

[0061] 3. Use the Docker push command on the node where the Harbor instance resides to push the image package to the Harbor repository. This pushes image package 1 onto node 1.

[0062] 4. Harbor uses synchronization rules and the interfaces provided by the Registry to automatically complete the process of synchronizing images to other Harbor instances. For example, image package 1 can be synchronized from node 1 to nodes 2 and 3.

[0063] 5. If multiple image packages are involved, repeat steps 2 to 4 for each image package.

[0064] However, projects, repositories, and synchronization rules all need to be created manually, and uploading image packages to Harbor instances (nodes) also requires manual operation. Furthermore, loading images using the Docker `load` command is heavily reliant on disk I / O, and using Harbor's synchronization rules to synchronize images (synchronizing image packages loaded on one node to other nodes) requires cross-node communication, making it highly dependent on network connectivity. Therefore, if the network and disk I / O in the Harbor deployment environment are poor, image synchronization failures can lead to image inconsistencies. Moreover, in scenarios involving large-scale image synchronization, creating projects, repositories, and synchronization rules remains overly cumbersome, and manual operation is not highly feasible.

[0065] To address the technical shortcomings of existing technologies, namely the heavy reliance on network and disk I / O for Harbor instance mirror synchronization and the complexity of creating projects, repositories, and synchronization rules, this application proposes a method to reduce network and disk I / O usage. Furthermore, using the novel mirror synchronization method proposed in this application, Harbor projects, repositories, and synchronization rules can be automatically created. Especially in scenarios involving large-scale mirror synchronization, this automation mechanism can effectively reduce operational complexity.

[0066] For example, see Figure 1 The diagram shown is a detailed flowchart of a mirror data processing method provided in an embodiment of this application. The method includes the following steps:

[0067] Step 100: Based on the deployment instructions, deploy the Harbor instance and the Registry instance, and establish a synchronization mechanism between the Harbor instance and the Registry instance.

[0068] Specifically, after the Kubernetes cluster environment is prepared, each node can trigger (either user-triggered or based on a preset script) the steps of deploying Harbor instances and Registry instances. That is, a Docker Registry service is created on each node where the Harbor instance is located, and synchronization rules between the Harbor instance and the local node are created.

[0069] Step 110: Receive image data uploaded by the user client, wherein the image data includes the data of each image package after deduplication and the metadata of each image package.

[0070] In this embodiment of the application, the image data uploaded by the user is the data of several image packages (image data after removing duplicate layers), and the metadata information of each image package.

[0071] Specifically, each image package to be uploaded is parsed to obtain the data of each layer of each image package. Then, the data of each layer of several image packages is deduplicated to finally obtain the image data of several image packages.

[0072] For example, if image package 1 includes image layers 1, 2, and 3, image package 2 includes image layers 1, 3, and 4, and image package 3 includes image layers 2, 3, and 4, then the image data corresponding to image packages 1, 2, 3, and 3 after deduplication are: image layers 1, 2, 3, and 4.

[0073] Step 120: Load the image data into the file directory of the Registry instance, and synchronize the image data from the Registry instance to the Harbor instance.

[0074] Specifically, the image-based Registry data source is loaded into the Docker Registry service data directory. Finally, the synchronization rules automatically complete the process of synchronizing the image to the Harbor instance on this node.

[0075] In this embodiment of the application, after the mirror number synchronization is completed, the Registry instance is deleted.

[0076] In other words, once the image data of a node has been uploaded to the Harbor instance, the Registry instance used to temporarily store the image data, as well as the image data temporarily stored in the file directory of the Registry instance, can be deleted.

[0077] At this point, each node stores image data. This image data includes the data of all image packages, as well as the metadata of each image package. In practical applications, the image data used to build that image package can be obtained from the image data based on the metadata of an image package. In this way, there is no need to synchronize image packages between nodes, and it no longer relies on the network between nodes.

[0078] Furthermore, in this embodiment of the application, Harbor projects, repositories, and synchronization rules are automatically created through condition triggering. In scenarios involving large-scale image synchronization, this automation mechanism can effectively reduce operational complexity.

[0079] Specifically, the system receives a target Pod creation request sent by the client, parses the image information in the request parameters, and the image information includes the name of the target image required to create the target Pod;

[0080] Based on the image information, determine whether there is a target project corresponding to the target image;

[0081] If the target project does not exist, then create the target project corresponding to the target image; if the target project exists, then determine whether there is a target repository corresponding to the target project.

[0082] If the target repository does not exist, then create the target repository corresponding to the target project; if the target repository exists, then determine whether there is a target synchronization rule corresponding to the target project.

[0083] If the target synchronization rule does not exist, then create the target synchronization rule corresponding to the target project; if the target synchronization rule exists, then end the judgment process.

[0084] In this embodiment of the application, a preferred implementation is as follows: Given preset project information corresponding to each image and repository information corresponding to each project, the name of the target image is parsed to determine the name of the target project corresponding to the target image, and a step of determining whether a target project corresponding to the target image exists is performed; next, based on the target project name, the target repository corresponding to the target project is determined, and a step of determining whether a target repository corresponding to the target project exists is performed; finally, when it is determined that the target synchronization rule does not exist, a preferred implementation is as follows:

[0085] Based on the information of each node in the cluster, a target synchronization rule is created to synchronize the updated mirror data of this node to other nodes.

[0086] Furthermore, in this embodiment of the application, the image information also includes the pull strategy for the target image and the key required to pull the target image.

[0087] Therefore, the above method may also include the following steps:

[0088] Based on the image information and the metadata information of the target image, the image data of the target image is pulled from the image data, and the target Pod is created based on the pulled image data of the target image.

[0089] The image data processing process provided in this application embodiment will be described in detail below with reference to specific application scenarios. For example, see [link to relevant documentation]. Figure 2 The diagram shown is a schematic representation of a mirror data processing procedure provided in an embodiment of this application. The procedure is as follows:

[0090] 1. Prepare the Kubernetes cluster environment.

[0091] 2. Prepare mirrored data in Registry mode.

[0092] Specifically, the data resides on the user's client at this point. The user uploads the Registry-mode image data to the storage node (data upload requires a Docker Registry instance to be created). In this embodiment, the Registry-mode image data refers to the deduplicated image data of each image package, plus the image package's metadata. The image data can be compressed and uploaded to each Harbor instance (storage node), then decompressed. Alternatively, the image data can be uploaded directly without compression.

[0093] 3. Deploy Harbor instances on the Kubernetes Master node respectively.

[0094] Specifically, a Kubernetes Master node refers to the node used to form a Kubernetes cluster. Deploying a Harbor instance can be understood as deploying a Harbor environment.

[0095] 4. Temporarily deploy Docker Registry instances on the Kubernetes Master node. These are used to transfer Registry-mode image data uploaded by clients to the Harbor instance.

[0096] 5. Deploy Kubernetes Admission Webhook instances on the Kubernetes Master node. Deploy a Webhook instance on each node.

[0097] 6. Establish a connection between the Harbor instance and the Docker Registry instance on the same node.

[0098] 7. Trigger image data synchronization from the Docker Registry instance to the Harbor instance.

[0099] 8. After the image synchronization is complete, clean up the temporarily deployed Docker Registry instance.

[0100] At this point, the image data has been uploaded to the Harbor instance on every node of the cluster. This means that each node's Harbor instance stores the image data and metadata for each image package. There is no need to perform an image package synchronization process between nodes.

[0101] 9. Trigger a request to create a Kubernetes Pod.

[0102] 10. The Admission Webhook instances on each Kubernetes node intercept requests to create Pods and automatically complete the creation of Harbor projects, repositories, and synchronization rules.

[0103] Specifically, the purpose of the synchronization rules is to trigger subsequent synchronization between nodes based on the configured synchronization rules for subsequent business upgrades (image data updates).

[0104] 11. After receiving a Pod creation request, the Kubernetes Kubelet component parses the image information in the request parameters, such as the image name, image pull strategy, and image pull key.

[0105] 12. The Kubernetes Kubelet component calls the Docker component to pull the image into the DockerDaemon based on the image information.

[0106] 13. The request to create a Kubernetes Pod has ended.

[0107] Further, see Figure 2 As shown, the automated creation process of the Harbor projects, repositories, and synchronization rules is described in detail below.

[0108] 1. Issue a request to create a Kubernetes Pod via the web page or CLI.

[0109] Specifically, users can issue a request to create a target Pod through a web page or a command-line page.

[0110] 2. The request first enters the Kube-apiserver component of Kubernetes, where it is then directed to a pre-created Admission Webhook for processing.

[0111] 3. Admission Webhook initialization: First, initialize the client for accessing the Kubernetes interface based on the Kubernetes service name and port number. Then, use the client to call the Kubernetes interface to obtain necessary dependency data, such as Kubernetes node information.

[0112] 4. After receiving the request to create a Kubernetes Pod, the Admission Webhook parses the image information in the request parameters, such as the image name, image pull strategy, and image pull key.

[0113] 5. Parse the image name to obtain the project name.

[0114] 6. Parse the mirror pull key to obtain the authentication information for requesting the Harbor service.

[0115] 7. If the project to be created already exists in the list of created projects, ignore the creation process; otherwise, based on the node information, project name, and Harbor service authentication information, call the Harbor interface to complete the creation operation of the Harbor project on a single node.

[0116] 8. If the repository to be created already exists in the repository list, ignore the creation process; otherwise, based on the node information and the authentication information of the Harbor service, call the Harbor interface to complete the creation operation of the Harbor repository on all nodes.

[0117] 9. If the synchronization rule to be created already exists in the synchronization rule list, ignore the creation process; otherwise, based on the node information, repository information, and Harbor service authentication information, call the Harbor interface to complete the creation of Harbor synchronization rules on all nodes.

[0118] For example, see Figure 3 The diagram shown is a structural schematic of a mirror data processing device provided in an embodiment of this application. This device is applied to each node of a cluster storing mirror data and includes:

[0119] Deployment unit 30 is used to deploy a Harbor instance and a Registry instance based on deployment instructions, and to establish a synchronization mechanism between the Harbor instance and the Registry instance;

[0120] The receiving unit 31 is used to receive image data uploaded by the user client, wherein the image data includes the data of each image package after deduplication and the metadata of each image package;

[0121] Processing unit 32 is used to load the image data into the file directory of the Registry instance and synchronize the image data from the Registry instance to the Harbor instance.

[0122] Optionally, the device further includes:

[0123] The deletion unit is used to delete the Registry instance after determining that the mirror number synchronization is complete.

[0124] Optionally, the apparatus further includes a parsing unit, a judging unit, and a creating unit:

[0125] The receiving unit 31 is further configured to receive a target Pod creation request sent by the client;

[0126] The parsing unit is used to parse the image information in the request parameters, the image information including the name of the target image required to create the target Pod;

[0127] The determination unit is used to determine, based on the image information, whether there is a target project corresponding to the target image;

[0128] If the target project does not exist, the creation unit is used to create the target project corresponding to the target image; if the target project exists, the judgment unit is also used to determine whether a target repository corresponding to the target project exists.

[0129] If the target repository does not exist, the creation unit is used to create a target repository corresponding to the target project; if the target repository exists, the judgment unit is also used to determine whether a target synchronization rule corresponding to the target project exists.

[0130] If the target synchronization rule does not exist, the creation unit is used to create the target synchronization rule corresponding to the target project; if the target synchronization rule exists, the judgment process ends.

[0131] Optionally, the image information further includes the pull strategy for the target image and the key required to pull the target image; the creation unit is also used for:

[0132] Based on the image information and the metadata information of the target image, the image data of the target image is pulled from the image data, and the target Pod is created based on the pulled image data of the target image.

[0133] Optionally, the device includes a predetermined project information corresponding to each image and a predetermined repository information corresponding to each project.

[0134] After the determining unit parses the name of the target image and determines the name of the target project corresponding to the target image, the judging unit performs the step of judging whether there is a target project corresponding to the target image;

[0135] After the determining unit determines the target warehouse corresponding to the target project based on the target project name, the judging unit performs the step of judging whether the target warehouse corresponding to the target project exists.

[0136] When the judgment unit determines that the target synchronization rule does not exist and creates a target synchronization rule corresponding to the target project, the creation unit is specifically used for:

[0137] Based on the information of each node in the cluster, a target synchronization rule is created to synchronize the updated mirror data of this node to other nodes.

[0138] These units can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more digital signal processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when one of these units is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these units can be integrated together to form a system-on-a-chip (SOC).

[0139] Furthermore, regarding the mirror data processing apparatus provided in this application embodiment, from a hardware perspective, the hardware architecture schematic diagram of the mirror data processing apparatus can be found in [reference needed]. Figure 4 As shown, the mirror data processing device may include: a memory 40 and a processor 41.

[0140] The memory 40 is used to store program instructions; the processor 41 calls the program instructions stored in the memory 40 and executes the above method embodiment according to the obtained program instructions. The specific implementation method and technical effect are similar, and will not be described again here.

[0141] Optionally, this application also provides a mirror data processing device, including at least one processing element (or chip) for performing the above method embodiments.

[0142] Optionally, this application also provides a program product, such as a computer-readable storage medium storing computer-executable instructions for causing the computer to perform the above-described method embodiments.

[0143] Here, a machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, a machine-readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0144] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0145] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0146] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0147] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0148] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0149] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0150] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for processing mirrored data, characterized in that, The method, applied to each node of a cluster storing image data, includes: Based on the deployment instructions, deploy a Harbor instance and a Registry instance, and establish a synchronization mechanism between the Harbor instance and the Registry instance; Receive image data uploaded by user clients, wherein the image data includes the data of each image package after deduplication and the metadata of each image package; The image data is loaded into the file directory of the Registry instance, and the image data is synchronized from the Registry instance to the Harbor instance.

2. The method as described in claim 1, characterized in that, The method further includes: After confirming that the mirror data synchronization is complete, delete the Registry instance.

3. The method as described in claim 1, characterized in that, The method further includes: Receive a target Pod creation request sent by the client, parse the image information in the request parameters, the image information including the name of the target image required to create the target Pod; Based on the image information, determine whether there is a target project corresponding to the target image; If the target project does not exist, then create the target project corresponding to the target image; if the target project exists, then determine whether there is a target repository corresponding to the target project. If the target repository does not exist, then create the target repository corresponding to the target project; if the target repository exists, then determine whether there is a target synchronization rule corresponding to the target project. If the target synchronization rule does not exist, then create the target synchronization rule corresponding to the target project; if the target synchronization rule exists, then end the judgment process.

4. The method as described in claim 3, characterized in that, The image information also includes the pull strategy for the target image and the key required to pull the target image; the method further includes: Based on the image information and the metadata information of the target image, the image data of the target image is pulled from the image data, and the target Pod is created based on the pulled image data of the target image.

5. The method as described in claim 4, characterized in that, The method includes: (The provided text contains pre-defined project information for each image and repository information for each project.) Parse the name of the target image, determine the name of the target project corresponding to the target image, and perform the step of determining whether the target project corresponding to the target image exists; Based on the target project name, determine the target repository corresponding to the target project, and execute the step of determining whether a target repository corresponding to the target project exists; The steps for creating a target synchronization rule for the target project after determining that the target synchronization rule does not exist include: Based on the information of each node in the cluster, a target synchronization rule is created to synchronize the updated mirror data of this node to other nodes.

6. A mirror data processing apparatus, characterized in that, The device, used on each node of a cluster for storing image data, includes: The deployment unit is used to deploy Harbor instances and Registry instances based on deployment instructions, and to establish a synchronization mechanism between the Harbor instances and the Registry instances. The receiving unit is used to receive image data uploaded by the user client, wherein the image data includes the data of each image package after deduplication and the metadata of each image package; The processing unit is used to load the image data into the file directory of the Registry instance and synchronize the image data from the Registry instance to the Harbor instance.

7. The apparatus as claimed in claim 6, characterized in that, The device further includes a parsing unit, a judging unit, and a creation unit: The receiving unit is also used to receive a target Pod creation request sent by the client; The parsing unit is used to parse the image information in the request parameters, the image information including the name of the target image required to create the target Pod; The determination unit is used to determine, based on the image information, whether there is a target project corresponding to the target image; If the target project does not exist, the creation unit is used to create the target project corresponding to the target image; if the target project exists, the judgment unit is also used to determine whether a target repository corresponding to the target project exists. If the target repository does not exist, the creation unit is used to create a target repository corresponding to the target project; if the target repository exists, the judgment unit is also used to determine whether a target synchronization rule corresponding to the target project exists. If the target synchronization rule does not exist, the creation unit is used to create the target synchronization rule corresponding to the target project; if the target synchronization rule exists, the judgment process ends.

8. The apparatus as claimed in claim 7, characterized in that, The image information also includes the pull strategy for the target image and the key required to pull the target image; the creation unit is further configured to: Based on the image information and the metadata information of the target image, the image data of the target image is pulled from the image data, and the target Pod is created based on the pulled image data of the target image.

9. The apparatus as claimed in claim 8, characterized in that, The device includes a determination unit, which has pre-set project information corresponding to each image and repository information corresponding to each project. After the determining unit parses the name of the target image and determines the name of the target project corresponding to the target image, the judging unit performs the step of judging whether there is a target project corresponding to the target image; After the determining unit determines the target warehouse corresponding to the target project based on the target project name, the judging unit performs the step of judging whether the target warehouse corresponding to the target project exists. When the judgment unit determines that the target synchronization rule does not exist and creates a target synchronization rule corresponding to the target project, the creation unit is specifically used for: Based on the information of each node in the cluster, a target synchronization rule is created to synchronize the updated mirror data of this node to other nodes.

10. A mirror data processing apparatus, characterized in that, The mirror data processing device includes: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the steps of the method as described in any one of claims 1-5 according to the obtained program instructions.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing the computer to perform the steps of the method as described in any one of claims 1-5.

Images