A container image data processing method, device, equipment and medium

Abstract

The present application discloses a container image data processing method, device, equipment, and medium, which are applied to a container image data processing device that interfaces with a container image repository. The method includes: obtaining event information sent by the container image repository through a preset interactive interface to indicate the successful upload of an image file, and determining a target image file that meets preset file processing conditions based on the event information; creating a first index file corresponding to the target image file; processing the file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file; and sending the second index file to the container image repository through a preset interactive interface. The present invention, driven by events, utilizes a container image data processing device to create an index file containing multiple CPU architectures, reducing the tedious operations when users upload images, and has significant effects in the production and push of multi-architecture data for large-scale images.

Description

Technical Field

[0001] The present invention relates to the field of container image technology, and in particular to a container image data processing method, apparatus, device, and medium. Background Art

[0002] Container images depend on the platform they run on. Machines with different operating systems (OSs) and CPUs (Central Processing Units) will run on different CPU architectures. In the container image registry, an index file describes the multiple CPU architectures of the image. This allows downloading images from machines with different OSs and CPU architectures to find the image for the corresponding architecture using the image index file.

[0003] In the field of cloud computing applications, more and more cloud infrastructure is built on cloud-native architectures such as K8s (aka Kubernetes). With the popularization of domestic chips with different CPU architectures such as Feiteng and Loongson, clouds that support hybrid architectures are becoming increasingly common. This requires that the same image version must support multiple CPU architectures.

[0004] In the traditional way, users need to develop their own programs and then write multi-CPU architecture index files for each image version. They need to manually push these files to the container image repository for storage using tools such as docker manifest or manifest-tool. This method obviously increases the developer's workload and learning cost, as well as the management cost of the image multi-architecture index files.

[0005] As can be seen from the above, in the process of uploading container images to the container image repository, how to avoid the cumbersome image uploading process on the user side and the high management cost of the image due to the traditional uploading method is a problem to be solved in this field. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a container image data processing method, device, equipment and medium, which can realize an event-driven multi-CPU architecture container image automatic push process based on the container image data processing device described in this solution, so that users do not need to worry about the multi-architecture data production and push of the image, nor do they need to consider the management of the multi-CPU architecture data index file of the image, but can directly push the constructed image to the container image repository. This method has significant effects in the scenario of multi-architecture data production and push of large-scale images. The specific scheme is as follows:

[0007] In a first aspect, the present application discloses a container image data processing method, which is applied to a container image data processing device that interfaces with a container image repository. The method includes:

[0008] Obtaining event information sent by the container image repository through a preset interactive interface to indicate successful image file upload, and determining a target image file that meets preset file processing conditions based on the event information;

[0009] Creating a first index file corresponding to the target image file;

[0010] Processing the file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file;

[0011] The second index file is sent to the container image repository through a preset interactive interface.

[0012] Optionally, before obtaining event information sent by the container image repository through a preset interactive interface to indicate successful image file upload, and determining a target image file that meets preset file processing conditions based on the event information, the process further includes:

[0013] A configuration file containing multiple configuration items is created, and configuration parameters for configuring the configuration items are obtained through a preset configuration parameter acquisition interface, so as to configure the container image data processing device using the configuration file; wherein the configuration items include: a configuration item for docking with a container image repository, a configuration item for setting the image CPU architecture supported by the device for processing, a configuration item for setting a check timeout for a manifest file, a configuration item for setting a scanning period for abnormal images, a configuration item for setting an output path for abnormal images, and a configuration item for setting a maximum number of threads in a thread pool, any one or more combinations thereof.

[0014] Optionally, determining the target image file that meets a preset file processing condition based on the event information includes:

[0015] Determine whether the event information is a preset image push event;

[0016] If the event information is a preset image push event, obtaining the image file corresponding to the event information, and determining whether the metadata type of the image file is a preset metadata type;

[0017] If the metadata type of the image file is a preset metadata type, obtaining an image name of the image file corresponding to the event information, and determining a first architecture type of the image file based on the image name; the image name is a name set by the client and includes the first architecture type of the image file;

[0018] Identifying the image architecture type of the image file using a preset type identification method to obtain a second architecture type of the image file;

[0019] Determining whether the first architecture type is consistent with the second architecture type;

[0020] If the first architecture type is consistent with the second architecture type, determining a target architecture type of the image file, and determining whether the target architecture type is a preset architecture type;

[0021] If the target architecture type is a preset architecture type, checking the manifest file corresponding to the image file in the container image repository according to a preset check and retry mechanism;

[0022] If the check fails, the image file is determined to be an abnormal image file;

[0023] If the check passes, the image file is determined as the target image file.

[0024] Optionally, the creating a first index file corresponding to the target image file includes:

[0025] Creating a first index file corresponding to the target image file and including all image CPU architectures supported and processed by the device set in the configuration item;

[0026] Accordingly, the processing of the file information in the first index file based on the preset inspection and processing method to obtain a second index file corresponding to the target image file includes:

[0027] Determine whether all the mirrored CPU architectures in the first index file are available;

[0028] If all the mirrored CPU architectures in the first index file are available, determining the first index file as the second index file;

[0029] If not all of the mirrored CPU architectures in the first index file are available, an unavailable CPU architecture is determined from all of the mirrored CPU architectures, and the unavailable architecture information corresponding to the unavailable CPU architecture in the first index file is deleted. Then, the first index file after the unavailable architecture information is deleted is determined as the second index file.

[0030] Optionally, determining whether all the mirrored CPU architectures in the first index file are available includes:

[0031] Determine the current target CPU architecture from all the mirrored CPU architectures in the first index file;

[0032] Determine whether there is a manifest file corresponding to the current target CPU architecture in the container image repository;

[0033] If the manifest file corresponding to the current target CPU architecture does not exist in the container image repository, the current target CPU architecture is marked as an unavailable CPU architecture;

[0034] If a manifest file corresponding to the current target CPU architecture exists in the container image repository, the current target CPU architecture is marked as an available CPU architecture, and the current target CPU architecture is re-determined based on the remaining image CPU architectures other than the determined unavailable CPU architecture and available CPU architecture in the first index file, and a step of determining whether a manifest file corresponding to the current target CPU architecture exists in the container image repository is triggered;

[0035] It is determined whether all the mirrored CPU architectures in the first index file are available according to the marking conditions of the unavailable CPU architectures and the available CPU architectures.

[0036] Optionally, the container image data processing method further includes:

[0037] If this is the first docking operation, the historical image files in the container image repository are obtained, and the preset recursive rules are used to extract the historical image files in the container image repository into the preset image list;

[0038] Determine the mapping relationship between each historical image file and each preset thread based on the preset image list and the preset calculation rule;

[0039] Determine a target image file from the historical image files corresponding to the current preset thread based on the mapping relationship using the preset thread;

[0040] Accordingly, the step of creating a first index file corresponding to the target image file, processing file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file, and sending the second index file to the container image repository through a preset interactive interface includes:

[0041] Creating a first index file corresponding to the target image file using the preset thread, and processing file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file;

[0042] The second index file is sent to the container image repository by using the preset thread through a preset interactive interface.

[0043] Optionally, the container image data processing method further includes:

[0044] Determine the image file corresponding to the second index file that is not successfully sent to the container image repository as an abnormal image file;

[0045] The image information corresponding to the abnormal image file is saved in a preset abnormal file; the image information includes the image name, image version number and image hash value;

[0046] Parsing all the image information in the preset abnormal file according to the abnormal image scanning period set in the configuration item, and then triggering the process of creating the first index file and the second index file;

[0047] If the second index file corresponding to the abnormal image file is created successfully, the second index file is sent to the container image repository through a preset interactive interface, and the image information corresponding to the current abnormal image file in the preset abnormal file is deleted.

[0048] In a second aspect, the present application discloses a container image data processing device, comprising:

[0049] An image file determination module is configured to obtain event information sent by the container image repository through a preset interactive interface to indicate successful image file upload, and determine a target image file that meets preset file processing conditions based on the event information;

[0050] A first index file creation module, configured to create a first index file corresponding to the target image file;

[0051] a second index file creation module, configured to process the file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file;

[0052] The file sending module is used to send the second index file to the container image repository through a preset interactive interface.

[0053] In a third aspect, the present application discloses an electronic device, comprising:

[0054] Memory, used to store computer programs;

[0055] A processor is configured to execute the computer program to implement the aforementioned container image data processing method.

[0056] In a fourth aspect, the present application discloses a computer storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the aforementioned disclosed container image data processing method are implemented.

[0057] This application first obtains event information sent by the container image repository through a preset interactive interface to indicate the successful upload of the image file, and determines the target image file that meets the preset file processing conditions based on the event information, then creates a first index file corresponding to the target image file, processes the file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file, and finally sends the second index file to the container image repository through the preset interactive interface. In this way, for the user side, the user does not need to worry about the production and push of related data of the image, nor does it need to consider the management of the image data index file. The user still needs to develop his own program and push the constructed image to the container image repository in the previous way. The subsequent index file creation process is processed by the container image data processing device proposed in this application that is connected to the container image repository and automatically pushed to the container image repository. This method improves the rationality of image file upload and reduces the tedious process of pushing images on the user side. It has a significant effect in the scenario of producing and pushing multi-architecture data of large-scale images. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0059] Figure 1 A flow chart of a container image data processing method provided in this application;

[0060] Figure 2 This is a data processing flow chart of a container image data processing device proposed in this application;

[0061] Figure 3 A specific container image data processing flow chart provided for this application;

[0062] Figure 4 A schematic diagram of the structure of a container image data processing device provided by this application;

[0063] Figure 5 This is a structural diagram of an electronic device provided in this application. DETAILED DESCRIPTION

[0064] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0065] In the existing technology, users need to develop their own programs and then compile multi-CPU architecture index files for each image version. These files must be manually pushed to the container image repository for storage using tools such as docker manifest or manifest-tool. This approach significantly increases the developer's workload and learning costs, as well as the management costs of the image's multi-architecture index files. However, this application proposes a container image data processing device that interfaces with a container image repository. This device can automatically complete the index file creation process in an event-driven manner, achieving significant results in scenarios where large-scale image multi-architecture data is produced and pushed.

[0066] The embodiment of the present invention discloses a container image data processing method, which is applied to a container image data processing device that interfaces with a container image repository. Figure 1 Said method comprises:

[0067] Step S11: obtaining event information sent by the container image repository through a preset interactive interface to indicate successful image file upload, and determining a target image file that meets preset file processing conditions based on the event information.

[0068] It is understandable that after the user develops his own program, the user pushes the built image file to the container image repository through the client (for example: docker push, devops pipeline, etc.). Accordingly, after receiving the image file uploaded by the user, the container image repository will send a corresponding event notification to the container image data processing device proposed in this solution, which is pre-connected to the above-mentioned container image repository. After receiving the event notification sent by the container image repository, the device will determine and search for the corresponding uploaded image file, determine the image file as the target image file, and then perform subsequent data processing operations. It should be pointed out that the container image data processing device proposed in the present invention can be connected to a commonly used image repository registry or an enterprise-level image repository harbor.

[0069] In this embodiment, before obtaining event information sent by the container image repository through a preset interactive interface to indicate the successful upload of the image file, and determining the target image file that meets the preset file processing conditions based on the event information, it can also include: creating a configuration file containing multiple configuration items, and obtaining configuration parameters for configuring the configuration items through a preset configuration parameter acquisition interface, so as to use the configuration file to configure the container image data processing device; wherein, the configuration items include: any one or more combinations of configuration items for docking with the container image repository, configuration items for setting the image CPU architecture supported by the device for processing, configuration items for setting the check timeout time for the manifest file, configuration items for setting the scanning period for abnormal images, configuration items for setting the output path for abnormal images, and configuration items for setting the maximum number of threads in the thread pool.

[0070] It is understandable that before the device is used, some configuration settings must be made for the device. The requirements of each hybrid architecture cloud are different. For example, some hybrid architectures support amd64 (X86 machines) and arm64, while some hybrid architectures support amd64, arm64, mips64le, etc. This device first creates a configuration file and extracts these frequently changing items into the configuration file as configuration items. Finally, the configuration file is used to configure the container image data processing device. In a specific embodiment, the configuration items in the configuration file may include: registryServer: the access address of the container image repository; registryUser: the account of the container image repository; registryPassword: the account and password of the container image repository; archList: the image CPU architecture supported by this device (this device can support almost all operating systems and CPU architectures such as linux / arm64, linux / mips64el, linux / amd64, linux / mips64, linux / mips32le, linux / arm); checkManifestTimeout: the check timeout of the image manifest (the default is 5s); errorImageJobPeriod: the scanning period of abnormal image files (the default is 60s); imageErrorPath: the output path of abnormal image files; syncMaxPool, the maximum number of threads in the thread pool. It should be noted that the configuration items that can be set in the configuration file include but are not limited to the configuration items mentioned above.

[0071] In this embodiment, the determining of the target image file that meets the preset file processing conditions based on the event information may include: determining whether the event information is a preset image push event; if the event information is a preset image push event, obtaining the image file corresponding to the event information, and determining whether the metadata type of the image file is a preset metadata type; if the metadata type of the image file is a preset metadata type, obtaining the image name of the image file corresponding to the event information, and determining the first architecture type of the image file based on the image name; the image name is a name set by the client and contains the first architecture type of the image file; using the preset type identification The method identifies the image architecture type of the image file to obtain a second architecture type of the image file; determines whether the first architecture type is consistent with the second architecture type; if the first architecture type is consistent with the second architecture type, determines the target architecture type of the image file, and determines whether the target architecture type is a preset architecture type; if the target architecture type is the preset architecture type, checks the manifest file corresponding to the image file in the container image repository according to a preset check and retry mechanism; if the check fails, determines the image file as an abnormal image file; if the check passes, determines the image file as the target image file.

[0072] It is understandable that the container image repository may send multiple notifications (i.e., the above-mentioned "event notifications") to the container image data processing device, and this device will only perform subsequent data processing on the image file after receiving the notification of the preset image push event. In a specific embodiment, the preset image push event can be an event generated by an image push, that is, after receiving the notification from the container image repository, this device checks whether the event corresponding to the notification is an event generated by an image push. If so, the corresponding image file will be processed; if not, the image file will not be processed.

[0073] If the notification event corresponds to an image push, the image file's metadata type is checked. In one embodiment, the regular expression `^application / vnd.docker.distribution.manifest.v\d\+(json|prettyjws)` can be used to match the image's metadata type. In one embodiment, if the default metadata type is manifest.v\d\, the device only processes metadata of the manifest.v\d\ type, ignoring all other types and filtering out the remaining types.

[0074] Next, the process proceeds to the next step. If the metadata type of the image file is a preset metadata type, the image name of the image file corresponding to the event information is obtained, and the first architecture type of the image file is determined based on the image name. The image architecture type of the image file is then identified using a preset type identification method to obtain the second architecture type of the image file. Finally, a determination is made as to whether the first architecture type and the second architecture type are consistent. This step is intended to check whether the architecture type in the image name is consistent with the architecture type of the image file itself. If they are consistent, it indicates that the current image file is a correctly named file.

[0075] It should be noted that this method requires the user to name the image file in a fixed way when uploading it. In the specific implementation process, the naming method of xxx-architecture can be used, for example: nginx (default is amd64), nginx-arm64, nginx-mips64el, nginx-amd64, etc.

[0076] The next step is to determine the target architecture type of the image file and whether it is a preset architecture type. The preset architecture type can be the image CPU architecture supported by the device as defined in the archList configuration item. In other words, the architecture type of the file must be an architecture type that the device can process before the next step can be entered.

[0077] Then enter the next processing process. If the target architecture type is a preset architecture type, the manifest file corresponding to the image file in the container image repository is checked according to the preset check and retry mechanism. It can be understood that when the image file is uploaded to the container image repository, the last step is to upload the manifest file of the image, and the storage of the image data in the container image repository is delayed. This delay is related to the storage medium used by the container image repository. That is, when the container image repository sends a notification of successful image upload, the image data is not necessarily stored in the container image repository. In order to avoid the impact of this delay on the processing of multi-architecture data of the image, this method adds a check and retry mechanism for the manifest file, and uses the timeout time set in the configuration item checkManifestTimeout to constrain the time and number of retries. If the set timeout time is exceeded, it is considered that the check has failed, the processing fails, and the image file is determined to be an abnormal image file; if the manifest file of the image is ready within the specified time, it means that the check has passed, and the image file is determined to be the target image file.

[0078] Step S12: creating a first index file corresponding to the target image file.

[0079] It is understandable that there is only one copy of the multi-CPU architecture data index file for each image version. For example, if you upload an nginx image, the multi-architecture data of this nginx image in the container image repository includes two CPU architectures, amd64 and arm64. If the currently uploaded architecture is mips64el, then when creating the index file of the multi-CPU architecture data of the current image, you cannot only include the current architecture. Otherwise, when uploading to the container image repository, the amd64 and arm64 architecture data of the current image version may be lost. To avoid this problem, this method requires that the created image multi-CPU architecture index file must include all CPU architectures supported by the configuration item archList.

[0080] Step S13: processing the file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file.

[0081] In this embodiment, the purpose of this step is to check whether the manifest file pointed to by each CPU architecture in the index file exists in the container image repository. If not, the non-existent CPU architecture data will be deleted, and finally an index file of the mirrored multiple CPU architectures will be formed, and the RestApi interface of the container image repository will be called to push the data to the container image repository. If the push fails, the image file will be confirmed as an abnormal image file.

[0082] In this embodiment, the container image data processing method may further include: determining the image file corresponding to the second index file that is not successfully sent to the container image repository as an abnormal image file; saving the image information corresponding to the abnormal image file to a preset abnormal file; the image information includes the image name, image version number and image hash value; parsing all the image information in the preset abnormal file according to the abnormal image scanning cycle set in the configuration item, and then triggering the process of creating the first index file and the second index file; if the second index file corresponding to the abnormal image file is successfully created, sending the second index file to the container image repository through a preset interactive interface, and deleting the image information corresponding to the current abnormal image file in the preset abnormal file.

[0083] It is understandable that the present invention identifies the image files corresponding to the second index file that was not successfully sent to the container image repository and the image files that were not checked by the manifest file described in step 11 as abnormal image files. In a specific embodiment, the image information of these abnormal image files will be saved in an "error-images" file. The module will periodically scan the "error-images" file and parse the information of each image in the file according to the errorImageJobPeriod parameter set in the configuration file, and then repeat the steps of creating the first index file and the second index file for them. If the second index file corresponding to a certain abnormal image file is successfully created, the image is cleared from the "error-images" file, otherwise the processing continues until success. It should be pointed out that in the process of processing mirrored multi-CPU architecture data, it is inevitable to encounter images that fail to be processed for other reasons, resulting in the loss of multi-architecture data of these images in the container image repository. In order to avoid this situation as much as possible, this solution reduces human intervention by saving the image information of abnormal image files and scanning them regularly, thereby achieving the purpose of automatically re-uploading images.

[0084] Step S14: Send the second index file to the container image repository through a preset interactive interface.

[0085] The preset interactive interface described in this embodiment includes but is not limited to the RestApi interface.

[0086] In this embodiment, the container image data processing method may further include: if the current docking operation is the first docking operation, obtaining historical image files in the container image repository, and extracting the historical image files in the container image repository into a preset image list using a preset recursive rule; determining a mapping relationship between each of the historical image files and each preset thread based on the preset image list and a preset calculation rule; and determining a target image file from the historical image files corresponding to the current preset thread using the preset thread based on the mapping relationship.

[0087] Correspondingly, the creating of a first index file corresponding to the target image file, processing the file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file, and sending the second index file to the container image repository through a preset interactive interface may include: using the preset thread to create a first index file corresponding to the target image file, and processing the file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file; and using the preset thread to send the second index file to the container image repository through a preset interactive interface.

[0088] It should be pointed out that when the container image repository is first connected to this device, a large amount of image data already exists in the container image repository, so this method proposes to convert all existing images in the container image repository into the multi-CPU architecture image data proposed in this solution and re-upload them to the container image repository. In a specific embodiment, all image version files in the container image repository can be obtained recursively, recursively taking 100 data each time until all image version files are taken out and placed in a mirror list, and then according to the total length of the mirror list and the configuration parameter syncMaxPool, the size of the thread pool and the number of image version files processed in each thread are calculated. Finally, each thread is responsible for converting the processed image into mirror multi-CPU architecture data and pushing it to the container image repository. After waiting for all threads to complete processing, the end mark of the end of the stock data processing is output.

[0089] The thread pool size calculation process is as follows:

[0090] When I n <syncMaxPool,

[0091] When I n When ≥syncMaxPool, Pool n =syncMaxPool;

[0092] Among them, Pooln is the size of the thread pool, I n is the length of the mirror list;

[0093] The number of image version files processed by each thread is as follows:

[0094] T n =I n / Pool n , T n Indicates the total number of image versions that each thread needs to process;

[0095] After the image version files in the image list are evenly distributed to each thread, the number of remaining image version files is calculated as follows:

[0096] T left =I n -T n *Pool n , T left Indicates the total number of remaining image versions after the image versions in the image list are evenly distributed to each thread;

[0097] The starting position of each thread processing image version in the image list is calculated as follows:

[0098] T start =Pool i *T n , T start Indicates the starting position of each thread processing the image version in the image list, Pool i Represents the i-th thread in the thread pool;

[0099] The end position of each thread processing image version in the image list is calculated as follows:

[0100] T end =Pool i *T n +T n , T end Indicates the end position of the image version processed by each thread in the image list;

[0101] When Pool i =Pool n When -1, the calculation formula is as follows:

[0102] T end =T end +T left .

[0103] The container image data processing device described in the present invention can also be divided into a configuration module, a multi-architecture data preprocessing module, a multi-architecture data generation module, a multi-architecture data automatic push module, an image missed scan module and an inventory data processing module according to the functional division. Among them, the configuration module is a module for configuring the container image data processing device through a configuration file containing multiple configuration items after the configuration file is created; the multi-architecture data preprocessing module is a module for receiving event information sent by a container image repository and determining a target image file; the multi-architecture data generation module is a module for creating a first index file corresponding to the target image file; the multi-architecture data automatic push module is a module for creating a second index file based on the first index file, and sending the second index file to the container image repository through a preset interactive interface; the image missed scan module is a module for processing abnormal image files; and the inventory data processing module is a module for processing historical image files. As Figure 2 The figure shows a data processing flow chart of a container image data processing device proposed in the present application. In the figure, the user uploads the image file to the image warehouse (i.e., the above-mentioned "container image warehouse") through a preset client. After the container image data processing device receives the event information sent by the image warehouse, it pre-processes the image file. After the pre-processing is successful, the corresponding image file is sent to the multi-architecture data generation module. Finally, the generated index file is pushed to the image warehouse through the multi-architecture data automatic push module. When the image warehouse is first docked, the stock data in the image warehouse is placed in the stock data processing module. After the stock data is processed by multiple threads, an index file is finally generated and pushed to the image warehouse. The error image (i.e., the above-mentioned "abnormal image file") generated by the multi-architecture data pre-processing module, the stock data processing module or the multi-architecture automatic push module will enter the image missed scanning module, and the error image will be placed in the multi-architecture data generation module for execution according to the preset scanning cycle. The configuration management module on the right is used to configure the device using a configuration file containing multiple configuration items.

[0104] This embodiment first obtains event information sent by the container image repository through a preset interactive interface to indicate the successful upload of the image file, and determines the target image file that meets the preset file processing conditions based on the event information. Then, a first index file corresponding to the target image file is created, and the file information in the first index file is processed based on a preset inspection and processing method to obtain a second index file corresponding to the target image file. Finally, the second index file is sent to the container image repository through the preset interactive interface. In this way, this embodiment uses the container image data processing device in combination with the application of various configuration items in the configuration file to achieve the creation of index files for multiple CPU architectures based on event-driven and automatically push them to the container image repository. For the user side, the user does not need to worry about the production and push of related image data, nor does it need to consider the management of image data index files. The user still needs to develop their own program and push the constructed image to the container image repository in the previous way. This embodiment also proposes a method for determining and processing abnormal image files, which reduces manual intervention in actual operations, improves the rationality of image file uploads as a whole, and ultimately reduces the tedious process of pushing images on the user side. It has a significant effect in the scenario of producing and pushing multi-architecture data of large-scale images.

[0105] Figure 3 A specific container image data processing flow chart provided in this application embodiment. Figure 3 As shown, the method includes:

[0106] Step S21: obtaining event information sent by the container image repository through a preset interactive interface to indicate successful image file upload, and determining a target image file that meets preset file processing conditions based on the event information.

[0107] Step S22: creating a first index file corresponding to the target image file and including all image CPU architectures supported and processed by the device set in the configuration item.

[0108] Step S23: Determine whether all the mirrored CPU architectures in the first index file are available.

[0109] In this embodiment, the determination of whether all the mirrored CPU architectures in the first index file are available may include: determining the current target CPU architecture from all the mirrored CPU architectures in the first index file; determining whether there is a manifest file corresponding to the current target CPU architecture in the container image repository; if there is no manifest file corresponding to the current target CPU architecture in the container image repository, marking the current target CPU architecture as an unavailable CPU architecture; if there is a manifest file corresponding to the current target CPU architecture in the container image repository, marking the current target CPU architecture as an available CPU architecture, and re-determining the current target CPU architecture based on the remaining mirrored CPU architectures other than the determined unavailable CPU architecture and available CPU architecture in the first index file, and triggering the step of determining whether there is a manifest file corresponding to the current target CPU architecture in the container image repository; determining whether all the mirrored CPU architectures in the first index file are available based on the marking of the unavailable CPU architecture and the available CPU architecture.

[0110] It is understandable that this step checks the created first index file. This method requires that the first index file for multiple CPU architectures must contain all CPU architectures defined by the configuration item archList. However, since the manifest files corresponding to each CPU architecture may not all exist, if the manifest file corresponding to a certain CPU architecture exists, it indicates that this architecture is available. If the manifest file corresponding to a certain CPU architecture does not exist, it indicates that this architecture is unavailable. Therefore, this step checks whether all the mirrored CPU architectures in the first index file are available.

[0111] Step S24: If all the mirrored CPU architectures in the first index file are available, the first index file is determined as the second index file.

[0112] It is understandable that if all the mirrored CPU architectures in the first index file are available, the first index file can be directly determined as the second index file.

[0113] Step S25: If not all of the mirrored CPU architectures in the first index file are available, an unavailable CPU architecture is determined from all of the mirrored CPU architectures, and the unavailable architecture information corresponding to the unavailable CPU architecture in the first index file is deleted. The first index file after the unavailable architecture information is deleted is then determined as the second index file.

[0114] It is understandable that if not all of the mirrored CPU architectures in the first index file are available, the architecture information corresponding to the architecture marked as unavailable is deleted.

[0115] Step S26: Send the second index file to the container image repository through a preset interactive interface.

[0116] This embodiment first obtains event information sent by the container image repository through a preset interactive interface to indicate that the image file has been successfully uploaded, and determines a target image file that meets the preset file processing conditions based on the event information, creates a first index file corresponding to the target image file and containing all image CPU architectures supported and processed by this device as set in the configuration item, and determines whether all the image CPU architectures in the first index file are available. If all the image CPU architectures in the first index file are available, the first index file is determined as the second index file. If not all the image CPU architectures in the first index file are available, unavailable CPU architectures are determined from all the image CPU architectures, and unavailable architecture information corresponding to the unavailable CPU architecture in the first index file is deleted. The first index file after the unavailable architecture information is deleted is then determined as the second index file. In this way, this embodiment elaborates on the process of creating the first index file and the second index file using the container image data processing device in this solution, that is, checking the manifest files corresponding to each CPU architecture in the first index file, and finally forming the second index file. Ultimately, the present invention can reduce the learning cost and time cost of developers or operation and maintenance personnel, reduce the management cost of multi-CPU architecture image data, and make the management of multi-architecture data of container images in hybrid architecture cloud computing scenarios very simple.

[0117] See also Figure 4 As shown, the embodiment of the present application discloses a container image data processing device, which may specifically include:

[0118] An image file determination module 11 is configured to obtain event information indicating successful image file upload sent by the container image repository through a preset interactive interface, and determine a target image file that meets preset file processing conditions based on the event information;

[0119] A first index file creation module 12, configured to create a first index file corresponding to the target image file;

[0120] A second index file creation module 13 is configured to process the file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file;

[0121] The file sending module 14 is configured to send the second index file to the container image repository through a preset interactive interface.

[0122] This method first obtains event information sent by the container image repository through a preset interactive interface to indicate the successful upload of the image file, and determines the target image file that meets the preset file processing conditions based on the event information. It then creates a first index file corresponding to the target image file, processes the file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file, and finally sends the second index file to the container image repository through the preset interactive interface. In this way, this embodiment uses a container image data processing device, combined with the application of various configuration items in the configuration file, to implement the creation process of index files for multiple CPU architectures based on event-driven, and automatically pushes them to the container image repository. For the user side, the user does not need to worry about the production and push of related image data, nor does it need to consider the management of image data index files. The user still needs to develop their own program and push the constructed image to the container image repository in the previous way. This embodiment also proposes a method for determining and processing abnormal image files, which reduces manual intervention in actual operations, improves the rationality of image file uploads as a whole, and ultimately reduces the tedious process of pushing images on the user side, which has a significant effect in the scenario of producing and pushing multi-architecture data of large-scale images.

[0123] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram should not be considered as any limitation to the scope of application of the present application.

[0124] Figure 5 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a display 24, an input / output interface 25, a communication interface 26, and a communication bus 27. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the container image data processing method disclosed in any of the aforementioned embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0125] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 26 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0126] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0127] The operating system 221 is used to manage and control the hardware devices on the electronic device 20 and the computer program 222. The operating system 221 can be Windows Server, NetWare, Unix, Linux, etc. In addition to including computer programs capable of implementing the container image data processing method executed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs capable of performing other specific tasks.

[0128] Furthermore, the present application also discloses a computer-readable storage medium, where the computer-readable storage medium includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, magnetic disks, or optical disks, or any other form of storage medium known in the art. When the computer program is executed by a processor, the aforementioned container image data processing method is implemented. For the specific steps of this method, reference can be made to the corresponding content disclosed in the aforementioned embodiments, and no further description will be given here.

[0129] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. Professionals can also further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0130] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0131] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0132] The container image data processing method, apparatus, device, and storage medium provided by the present invention have been described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is intended only to facilitate understanding of the method and core concept of the present invention. Furthermore, those skilled in the art will appreciate that variations in the specific implementation methods and scope of application may occur based on the concepts of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A method for processing container image data, characterized in that: A container image data processing device for docking with a container image repository includes: Obtaining event information sent by the container image repository through a preset interactive interface to indicate successful image file upload, and determining a target image file that meets preset file processing conditions based on the event information; Creating a first index file corresponding to the target image file; Processing the file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file; Sending the second index file to the container image repository through a preset interactive interface; Wherein, the method further includes: If the docking operation is the first docking operation, historical image files in the container image repository are obtained, and the historical image files in the container image repository are extracted into a preset image list using a preset recursive rule; a mapping relationship between each historical image file and each preset thread is determined based on the preset image list and a preset calculation rule; and a target image file is determined from the historical image files corresponding to the current preset thread based on the mapping relationship using the preset thread; The determining the mapping relationship between each historical image file and each preset thread based on the preset image list and the preset calculation rule includes: Calculate the size of the preset thread pool and the number of historical image version files processed by each preset thread based on the total length of the preset image list and the configuration parameter syncMaxPool; The calculation formula for the preset thread pool size is: when hour, ; when hour, ; in, To preset the thread pool size, is the total length of the preset mirror list, It is the configuration parameter syncMaxPool; The formula for the number of image version files processed by each thread is: , Indicates the total number of image versions that each thread needs to process.

2. The container image data processing method according to claim 1, characterized in that: Before obtaining event information sent by the container image repository through a preset interactive interface to indicate successful image file upload, and determining a target image file that meets preset file processing conditions based on the event information, the method further includes: A configuration file containing multiple configuration items is created, and configuration parameters for configuring the configuration items are obtained through a preset configuration parameter acquisition interface, so as to configure the container image data processing device using the configuration file; wherein the configuration items include: a configuration item for docking with a container image repository, a configuration item for setting the image CPU architecture supported by the device for processing, a configuration item for setting a check timeout for a manifest file, a configuration item for setting a scanning period for abnormal images, a configuration item for setting an output path for abnormal images, and a configuration item for setting a maximum number of threads in a thread pool, any one or more combinations thereof.

3. The container image data processing method according to claim 1, characterized in that: The determining, based on the event information, a target image file that meets a preset file processing condition includes: Determine whether the event information is a preset image push event; If the event information is a preset image push event, obtaining the image file corresponding to the event information, and determining whether the metadata type of the image file is a preset metadata type; If the metadata type of the image file is a preset metadata type, obtaining an image name of the image file corresponding to the event information, and determining a first architecture type of the image file based on the image name; the image name is a name set by the client and includes the first architecture type of the image file; Identifying the image architecture type of the image file using a preset type identification method to obtain a second architecture type of the image file; Determining whether the first architecture type is consistent with the second architecture type; If the first architecture type is consistent with the second architecture type, determining a target architecture type of the image file, and determining whether the target architecture type is a preset architecture type; If the target architecture type is a preset architecture type, checking the manifest file corresponding to the image file in the container image repository according to a preset check and retry mechanism; If the check fails, the image file is determined to be an abnormal image file; If the check passes, the image file is determined as the target image file.

4. The container image data processing method according to claim 1, characterized in that: The step of creating a first index file corresponding to the target image file includes: Creating a first index file corresponding to the target image file and including all image CPU architectures supported and processed by the device set in the configuration item; Accordingly, the processing of the file information in the first index file based on the preset inspection and processing method to obtain a second index file corresponding to the target image file includes: Determine whether all the mirrored CPU architectures in the first index file are available; If all the mirrored CPU architectures in the first index file are available, determining the first index file as the second index file; If not all of the mirrored CPU architectures in the first index file are available, an unavailable CPU architecture is determined from all of the mirrored CPU architectures, and the unavailable architecture information corresponding to the unavailable CPU architecture in the first index file is deleted. Then, the first index file after the unavailable architecture information is deleted is determined as the second index file.

5. The container image data processing method according to claim 4, characterized in that: The determining whether all the mirrored CPU architectures in the first index file are available includes: Determine the current target CPU architecture from all the mirrored CPU architectures in the first index file; Determine whether there is a manifest file corresponding to the current target CPU architecture in the container image repository; If the manifest file corresponding to the current target CPU architecture does not exist in the container image repository, the current target CPU architecture is marked as an unavailable CPU architecture; If a manifest file corresponding to the current target CPU architecture exists in the container image repository, the current target CPU architecture is marked as an available CPU architecture, and the current target CPU architecture is re-determined based on the remaining image CPU architectures other than the determined unavailable CPU architecture and available CPU architecture in the first index file, and a step of determining whether a manifest file corresponding to the current target CPU architecture exists in the container image repository is triggered; It is determined whether all the mirrored CPU architectures in the first index file are available according to the marking conditions of the unavailable CPU architectures and the available CPU architectures.

6. The container image data processing method according to claim 1, characterized in that: The creating a first index file corresponding to the target image file, processing file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file, and sending the second index file to the container image repository through a preset interactive interface includes: Creating a first index file corresponding to the target image file using the preset thread, and processing file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file; The second index file is sent to the container image repository by using the preset thread through a preset interactive interface.

7. The container image data processing method according to claim 2 or 4, characterized in that: Also includes: Determine the image file corresponding to the second index file that is not successfully sent to the container image repository as an abnormal image file; Saving the image information corresponding to the abnormal image file into a preset abnormal file; The image information includes the image name, image version number and image hash value; Parsing all the image information in the preset abnormal file according to the abnormal image scanning period set in the configuration item, and then triggering the process of creating the first index file and the second index file; If the second index file corresponding to the abnormal image file is created successfully, the second index file is sent to the container image repository through a preset interactive interface, and the image information corresponding to the current abnormal image file in the preset abnormal file is deleted.

8. A container image data processing device, characterized in that: include: An image file determination module is configured to obtain event information sent by the container image repository through a preset interactive interface to indicate successful image file upload, and determine a target image file that meets preset file processing conditions based on the event information; A first index file creation module, configured to create a first index file corresponding to the target image file; a second index file creation module, configured to process the file information in the first index file based on a preset inspection and processing method to obtain a second index file corresponding to the target image file; A file sending module, configured to send the second index file to the container image repository through a preset interactive interface; The container image data processing device is specifically used to: If the docking operation is the first docking operation, historical image files in the container image repository are obtained, and the historical image files in the container image repository are extracted into a preset image list using a preset recursive rule; a mapping relationship between each historical image file and each preset thread is determined based on the preset image list and a preset calculation rule; and a target image file is determined from the historical image files corresponding to the current preset thread based on the mapping relationship using the preset thread; The container image data processing device is specifically used to: Calculate the size of the preset thread pool and the number of historical image version files processed by each preset thread based on the total length of the preset image list and the configuration parameter syncMaxPool; The calculation formula for the preset thread pool size is: when hour, ; when hour, ; in, To preset the thread pool size, is the total length of the preset mirror list, It is the configuration parameter syncMaxPool; The formula for the number of image version files processed by each thread is: , Indicates the total number of image versions that each thread needs to process.

9. An electronic device, characterized in that: The method comprises a processor and a memory; wherein, when the processor executes the computer program stored in the memory, the container image data processing method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that Used to store a computer program; wherein, when the computer program is executed by a processor, the container image data processing method according to any one of claims 1 to 7 is implemented.

Images