Image distribution method, electronic device and storage medium
The image distribution method optimizes the process by identifying and distributing image file layers directly between repositories, enhancing efficiency and stability by avoiding local data transfer and ensuring data accuracy.
Patent Information
- Application Number
- JP2022581580
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-06-30
- Filing Date
- 2021-03-18
- Publication Date
- 2024-11-11
- Estimated Expiration
- 2041-03-18
AI Technical Summary
Existing image distribution methods in virtualization technology consume high network bandwidth and are prone to file conflicts due to the need to pull entire image files locally, reducing efficiency and stability.
An image distribution method that identifies the storage location of each layer data in a source image repository and distributes it directly to a target repository, avoiding local pulling and pushing, and includes verification steps to ensure data accuracy and avoid conflicts.
Improves efficiency and stability by reducing bandwidth consumption and preventing file conflicts during image distribution, especially in multi-task scenarios.
Smart Images

Figure 0007580501000001 
Figure 0007580501000002 
Figure 0007580501000003
Abstract
Description
[Technical field]
[0001] FIELD OF THE DISCLOSURE The embodiments of the present application relate to the field of communications, and in particular to an image distribution method, a server and a storage medium. [Background technology]
[0002] With the rapid development of IT technology, virtualization technology is being applied more and more widely in the computer field due to its features such as convenience, simplicity, low cost, and high security. Among them, image repository is an important network element in virtualization technology, and is used to store various image files (Images) of users. Images are stored in layers in the image repository, and different images are identified by different reference relationships (links). In other words, images are stored in the image repository not as a single file, but as a series of layers and links. This is a special file structure that shares each layer data and reduces storage capacity.
[0003] In the process of using an image, users often need to distribute images among multiple image repositories. Image distribution is usually based on the Docker system, an open source application container engine. That is, first pull an image file from the source image repository, tag the image, and then push the image file to the target image repository corresponding to the new tag. Image distribution in this way requires pulling the entire image file locally, which consumes a lot of network bandwidth and reduces distribution efficiency. In addition, storing an image file locally often requires a certain amount of time, so if multiple distribution tasks correspond to the same image file, it is easy to cause image file conflicts. Summary of the Invention [Problem to be solved by the invention]
[0004] The objective of the embodiments of the present application is to provide an image distribution method, a server and a storage medium, and a new image distribution method improves the efficiency and stability of the image distribution process. [Means for solving the problem]
[0005] In order to solve the above-mentioned technical problems, an embodiment of the present application provides an image distribution method executed in an image distribution device, the image distribution method including: obtaining a distribution request for an image file; identifying a storage location of the image file from a source image repository of the image file; and distributing the image file to a target image repository based on the storage location of the image file; after distributing each layer data of the image file from the source image repository to the target image repository according to the first storage location, identifying a third storage location in the target image repository for each layer data of the distributed image file, and if it is verified that each layer data stored in the third storage location does not match each layer data stored in the first storage location, again executing the step of distributing each layer data of the image file from the source image repository to the target image repository according to the first storage location; Includes.
[0006] An embodiment of the present application further provides a server, the server including at least one processor and a memory communicatively connected to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to perform the image distribution method described above.
[0007] An embodiment of the present application further provides a computer-readable storage medium, the computer-readable storage medium storing a computer program, which, when executed by a processor, realizes the image distribution method described above. [Brief description of the drawings]
[0008] One or more embodiments are illustrated by way of example only with corresponding images in the accompanying drawings, and these illustrative illustrations do not constitute limitations on the embodiments. [Figure 1] 1 is a flowchart of an image distribution method in a first embodiment of the present application. [Diagram 2] 11 is a flowchart of an image distribution method in a second embodiment of the present application. [Diagram 3] 13 is a flowchart of an image distribution method according to a third embodiment of the present application. [Figure 4] FIG. 13 is a schematic diagram illustrating the configuration of an image distribution device according to a fourth embodiment of the present application. [Diagram 5] FIG. 13 is a schematic diagram illustrating the configuration of an image distribution device according to a fourth embodiment of the present application. [Figure 6] FIG. 13 is a schematic diagram illustrating the configuration of a server according to a fifth embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the following describes in detail the embodiments of the present application in combination with the accompanying drawings. However, those skilled in the art can understand that many technical details are presented in the embodiments of the present application to allow the reader to better understand the present application. However, the technical solution sought to be protected by the present application can be realized without these technical details and various changes and modifications based on the following embodiments. The division of the following embodiments is made for the convenience of explanation and should not be construed as any limitation on the specific embodiments of the present application, and the embodiments can be combined or referenced with each other as long as they are not inconsistent.
[0010] A first embodiment of the present invention relates to an image distribution method, according to which a technical means of obtaining a distribution request for an image file, identifying a first storage location in a source image repository of each layer data of the image file, and distributing each layer data of the image file from the source image repository to a target image repository according to the first storage location, when image distribution is required, by identifying the storage location of each layer data of the image file before performing image distribution, it is not necessary to pull the image locally and then push it to the target image repository, which significantly improves the efficiency of image distribution and avoids file conflicts that may occur when pulling the image file locally, thereby improving the stability of the image distribution process.
[0011] The following describes the details of the implementation of the image distribution method of this embodiment, but the following content is merely implementation details provided for ease of understanding and is not a necessary condition for implementing this invention.
[0012] As shown in FIG. 1, the specific flow of this embodiment specifically includes the following steps.
[0013] In step 101, a request to distribute an image file is received.
[0014] Specifically, the image file in this embodiment is an image file requested in a distribution request, that is, an image file to be distributed. When a user needs to distribute an image between multiple image repositories, a distribution request for an image file is sent to an IDServer (Image Distribute Server, image distribution system server side) by a DClient (Image Distribute Client, image distribution system client side). Available IDClients include Web (web page client), App (application for mobile terminals), command line binary programs, command line scripts, etc. The execution subject in this embodiment is the IDServer, which is deployed on the same virtual node as the IDClient or on the same virtual machine node and can perform the function of managing and controlling the image repository. Here, the image distribution request holds at least an image list including the file name of the image file to be distributed or the names of multiple image files to be distributed, the address of the source image repository, and the address of the target image repository.
[0015] In step 102, a first storage location within the source image repository for each layer data of the image file is identified.
[0016] Specifically, after receiving a distribution request for an image file from the IDServer, image identification is performed in the source image repository, i.e., the image file name and the address of the source image repository in the distribution request are analyzed, and the storage location, i.e., the first storage location, of each layer data of the image file to be distributed in the source image repository is identified from the image repository in association with a special storage structure of the image in the image repository. Among them, each layer data of the image file, i.e., the underlying data, is determined by a special storage structure of the image file in the image repository, and the underlying data includes, but is not limited to, repositories, blobs, layers, manifests, etc. Among them, repositories are used to store basic information of the image, including tenants, tags, reference relationships, etc. Blobs are used to store the actual data of each layer of the image. Layers are used to store the reference relationship of an image to the actual data layer. Manifests are used to store the tags of an image and the reference relationship of each tag to the actual data layer. Once the underlying data for the image file in the source image repository is identified, the underlying data for the image file is locked and the underlying data is preserved.
[0017] In one example, the storage of the underlying data may be temporary, for example, the underlying data may be written to the random access memory RAM of the server, i.e., volatile memory, and the memory space occupied by the underlying data may be released after the execution of the current image distribution operation is completed. In addition, the underlying data may be persistent, for example, the underlying data may be written to a non-volatile memory such as a database, a local magnetic disk of the server, or a cloud-side storage, so that the image file does not need to be located again when a verification process or other distribution operation is performed on the image file later, thereby improving the efficiency of the image file distribution.
[0018] In step 103, each layer data of the image file is distributed from the source image repository to the target image repository according to the first storage location.
[0019] Specifically, when distributing an image file, the file data distributed to the target image repository still conforms to the specific structure for storing the image, i.e., the image file distributed to the target image repository is consistent with the storage structure of the image file in the source image repository, and the underlying data is also consistent.
[0020] Therefore, the first embodiment of the present application provides an image distribution method, according to which a technical means is provided for obtaining a distribution request for an image file, identifying a first storage location in a source image repository of each layer data of the image file, and distributing each layer data of the image file from the source image repository to a target image repository according to the first storage location. When image distribution is required, by identifying the storage location of each layer data of the image file before performing image distribution, it is not necessary to pull the image locally and then push it to the target image repository. This significantly improves the efficiency of processing distribution requests on the image distribution system server side when tasks of multiple image distribution requests are progressing simultaneously, and avoids file conflicts that may occur when pulling image files locally, thereby improving the stability of the image distribution process.
[0021] The second embodiment of the present invention relates to an image distribution method, and the second embodiment is almost the same as the first embodiment. In this embodiment, before distributing each layer data of an image file from a source image repository to a target image repository, it is determined whether an image file identical to the image file to be distributed exists in the target image repository, and if the determination result is YES, each layer data of the image file is distributed from the source image repository to the target image repository, otherwise, the image file distribution operation is not performed. By inspecting the image file in the target image repository, unnecessary data transmission operations during image distribution can be avoided.
[0022] A specific flowchart of the image distribution method in this embodiment is shown in FIG. 2, which specifically includes the following steps:
[0023] In step 201, a request to distribute an image file is obtained.
[0024] In step 202, a first storage location within the source image repository for each layer data of the image file is identified.
[0025] Steps 201 and 202 in this embodiment are the same as steps 101 and 102 in the first embodiment of the present application, and the relevant implementation details have already been specifically described in the first embodiment, so they will not be repeated here.
[0026] In step 203, determine whether the same image file as the image file exists in the target image repository. If the same image file as the image file does not exist in the target image repository, execute step 204, and distribute each layer data of the image file from the source image repository to the target image repository according to the first storage location; otherwise, end the image distribution process.
[0027] Specifically, when an image file identical to the image file to be distributed exists in the target image repository, it indicates that the image file requested by the user is already present in the image repository, eliminating the need for an image distribution operation, thereby saving transmission resources in a virtual machine environment, reducing the occupied bandwidth, and further increasing the speed of image distribution, and when multiple image distribution request tasks are progressing simultaneously, greatly improving the processing efficiency of distribution requests on the server side of the image distribution system.
[0028] In one example, a method for determining that an image file identical to the image file does not exist in the target image repository is as follows.
[0029] First, search for the image name of the image file to be distributed in the target image repository, that is, determine whether an image file with the same name as the image file to be distributed exists in the target image repository. If an image file with the same name as the image file does not exist in the target image repository, determine that an image file with the same name as the image file to be distributed does not exist in the target image repository. In this case, distribute each layer data of the image file from the source image repository to the target image repository according to the first storage location.
[0030] If an image file with the same name as the image file exists in the target image repository, it is necessary to further identify a storage location in the target image repository of each layer data of the image file with the same name as the image file, i.e., a second storage location. Then, the second storage location is compared with each layer data stored in the first storage location, and if each layer data stored in the second storage location does not match each layer data stored in the first storage location, it is determined that an image file identical to the image file to be distributed does not exist in the target image repository. In this case, each layer data of the image file is distributed from the source image repository to the target image repository according to the first storage location. Otherwise, it is determined that an image file identical to the image file to be distributed exists in the target image repository, and the image distribution process is terminated.
[0031] In this embodiment, the file characteristics of the image file are obtained from each layer data of the image file, and if the file characteristics of the image file in the target image repository with the same name as the image file to be distributed match the file characteristics of the image file to be distributed, it is determined that the two are the same image file. The file characteristics compared here include, but are not limited to, the number of data, size, md5 code, sha256 code, etc. of the image file. If there is a difference in any one file characteristic, it is determined that the two are different image files.
[0032] In one example, a specific operation method for distributing each layer data of an image file from a source image repository to a target image repository according to a first storage location can be performed by copying each layer data of the image file to be distributed from the first storage location, and overwriting and storing each layer data of the copied image file in the second storage location. When distributing images in this manner, the data distributed to the target image repository still automatically conforms to the specific structure of storing images.
[0033] As a result, this embodiment provides an image distribution method, according to which, before distributing an image, an image file in a target image repository with the same name as the image file to be distributed is identified, and if an exactly the same image file exists, an image distribution operation is not performed, thereby reducing the bandwidth occupied by data transmission, thereby significantly improving processing efficiency when multiple distribution requests occur simultaneously, and also avoiding the problem of file contention for the same image file occurring in the target image repository after distribution is completed.
[0034] Those skilled in the art will understand that the division of the steps of the above various methods is merely for the purpose of clarity, and that when implemented, they may be integrated into one step or some steps may be subdivided into multiple steps, and as long as they contain the same logical relationship, they will all fall within the scope of protection of the present application; and as long as insignificant modifications are made to the algorithms and processes, or insignificant design is introduced into the algorithms or processes without changing their core design, they will all fall within the scope of protection of the present application.
[0035] The third embodiment of the present invention relates to an image distribution method, and the third embodiment is almost the same as the first embodiment. In this embodiment, after distributing each layer data of the image file from the source image repository to the target image repository, the third storage location of each layer data of the image file after distribution and the first storage location are verified. If they match, the image distribution process is terminated to indicate that the image distribution is successful; if they do not match, the step of distributing each layer data of the image file from the source image repository to the target image repository according to the first storage location is performed again to indicate that an error occurs in the image file during distribution. In this embodiment, verifying the file after distribution ensures the accuracy of the image file distribution and further improves the stability of the image distribution process.
[0036] A specific flowchart of the image distribution method in this embodiment is shown in FIG. 3, which specifically includes the following steps:
[0037] In step 301, a request to distribute an image file is obtained.
[0038] In step 302, a first storage location within the source image repository for each layer data of the image file is identified.
[0039] In step 303, determine whether the same image file as the image file exists in the target image repository. If the same image file as the image file does not exist in the target image repository, execute step 304, and distribute each layer data of the image file from the source image repository to the target image repository according to the first storage location; otherwise, end the image distribution process.
[0040] Steps 301 to 304 in this embodiment are almost the same as steps 301 to 304 in the second embodiment, and the relevant implementation details have already been specifically described in the second embodiment, so they will not be repeated here.
[0041] In step 305, verify whether each layer data stored in the third storage location of the image file after distribution is consistent with each layer data stored in the first storage location. If it is verified that each layer data stored in the third storage location is not consistent with each layer data stored in the first storage location, terminate the distribution process. Otherwise, execute step 304 again to distribute each layer data of the image file from the source image repository to the target image repository according to the first storage location.
[0042] Specifically, after the distribution of the image file is completed, a third storage location in the target image repository for each layer data of the distributed image file is identified. Then, it is verified whether each layer data stored in the third storage location matches each layer data stored in the first storage location. If it is verified that each layer data stored in the third storage location does not match each layer data stored in the first storage location, step 304 is executed again. By verifying the stored data of the distributed image file, it is guaranteed that there are no missing or incorrect data in the distributed image file, and the stability of the image file in the image distribution process is further improved.
[0043] In this embodiment, when verifying each layer data of the image file in this embodiment, similar to the second embodiment, the file characteristics of the two image files, such as the number of data files, size, md5 code, sha256 code, etc. are compared. If there is a difference in any one of the file characteristics between the post-distribution image file in the target image repository and the image file in the source image repository, it indicates that the stored data of both do not match.
[0044] In addition, the user can set an upper limit on the number of image redistributions on the server side of the image distribution system. If an error occurs multiple times in image distribution, it is assumed that there may be a problem in the system, and an error message is pushed to the administrator, who is then prompted to check whether other errors have occurred in the system, thus preventing the image distribution process from falling into a circular error and wasting system resources.
[0045] In one example, when storage address information of an image file needs to be permanently stored, after identifying a first storage location in the source image repository for each layer data of the image file, and after identifying a second storage location in the target image repository for each layer data of an image file having the same name as the image file, the first storage location and the second storage location may each be written to a non-volatile memory such as a database, a local magnetic disk of a server, or cloud-side storage.
[0046] In this embodiment, when it is determined after step 305 to execute step 304 again, first obtain the saved first storage location from the non-volatile memory. Besides directly identifying the third storage location of the distributed image file in the target image repository, the previously saved second storage location can also be obtained from the non-volatile memory. Then, complete the verification process of the distributed image file by verifying whether each layer data stored in the first storage location is consistent with each layer data stored in the second storage location or the third storage location. If each layer data stored in the first storage location is consistent with each layer data stored in the second storage location or the third storage location, end the data distribution process; otherwise, execute step 304 again.
[0047] Therefore, the present embodiment provides an image distribution method, which verifies whether each layer data of the distributed image file in the target image repository is consistent with each layer data of the image file in the source image repository after the image distribution is completed, thereby ensuring the accuracy of the image file distribution and further improving the stability of the image distribution process.
[0048] The fourth embodiment of the present application relates to an image distribution device, as shown in FIG.
[0049] The receiving module 401 is configured to obtain a distribution request for an image file, resolve the image name, and then send the image name to the control module.
[0050] The control module 402 is configured to identify a first storage location within the source image repository for each layer data of an image file to be distributed.
[0051] In one example, the method is further configured to identify a second storage location within the target image repository for each layer data of an image file having the same name as the image file to be distributed.
[0052] In one example, the method is further configured to identify a third storage location within the target image repository for each layer data of the image file after distribution.
[0053] In one example, the method is further configured to, after identifying a first storage location in the source image repository for each layer data of the image file, and after identifying a second storage location in the target image repository for each layer data of an image file having the same name as the image file, write the first storage location and the second storage location to a non-volatile memory such as a database, a local magnetic disk of the server, or cloud-side storage, respectively.
[0054] The verification module 403 is configured to determine that an image file identical to the image file does not exist in the target image repository according to the first storage location and the second storage location identified by the control module 402, and to send the verification result to the distribution module 404.
[0055] In one example, the device is further configured to retrieve the first storage location and / or the second storage location from the non-volatile memory.
[0056] In one example, it is further configured to verify whether the third storage location of each layer data of the image file after distribution is consistent with the first storage location, and send the verification result to the distribution module 404.
[0057] The distribution module 404 is configured to distribute each layer data of the image file from the source image repository to the target image repository according to the verification result sent from the verification module and the first storage location.
[0058] In a specific application, the configuration and interaction of the image device in this embodiment is shown in Figure 5. Here, a receiving module 501 is connected to a client side and configured to receive an image distribution request issued by a user through the client side.
[0059] The control module 502, the validation module 503, and the distribution module 504 are all communicatively coupled to the source image repository and the target image repository simultaneously.
[0060] This embodiment is an apparatus embodiment corresponding to the first, second or third embodiment, and it is easy to understand that this embodiment can be implemented in combination with the first or second embodiment. The relevant technical details described in the first, second or third embodiment are also valid in this embodiment, so they will not be described here to reduce duplication. Therefore, the relevant technical details described in this embodiment are also applicable to the first, second or third embodiment.
[0061] In addition, each module in this embodiment is a logical module, and in actual application, one logical unit may be one physical unit, may be a part of one physical unit, and may be realized by a combination of multiple physical units. In addition, in order to highlight the creative part of this application, means that are not related to solving the technical problem raised in this application are not introduced in this embodiment, but this does not mean that there are no other means in this embodiment.
[0062] The fifth embodiment of the present invention relates to a server. As shown in Fig. 6, the server includes at least one processor 601 and a memory 602 communicably connected to the at least one processor 601, and the memory 602 stores instructions executable by the at least one processor 601. The instructions are executed by the at least one processor 601, so that the at least one processor 601 can execute the image distribution method described above.
[0063] Here, the memory and the processor are connected in a bus manner, which may include any number of interconnected buses and bridges, and which connects together various circuits of one or more processors and memories. The bus may also connect together various other circuits, such as peripherals, voltage stabilizers, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, and provides a means for communicating with various other devices over a transmission medium. The data processed by the processor is transmitted over a wireless medium via an antenna, which in this embodiment also receives data and transmits data to the processor.
[0064] A processor may provide a variety of functions including bus management and general processing, as well as timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory, on the other hand, may be used to store data used by a processor in performing operations.
[0065] A fifth embodiment of the present application relates to a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method embodiments described above.
[0066] That is, those skilled in the art will understand that implementing all or part of the steps in the methods of the above embodiments can be realized by instructing related hardware by a program. The program is stored in one storage medium and includes several instructions for causing one device (which may be a one-chip computer, chip, etc.) or a processor to execute all or part of the steps of the methods according to the embodiments of the present application. Meanwhile, the storage medium includes various media capable of storing program code, such as USB memory, removable hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.
[0067] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing the present application, and that in actual application, various changes in form and details may be made without departing from the spirit and scope of the present application.
Claims
1. An image distribution method executed on an image distribution device, comprising: Obtaining a distribution request for an image file; identifying a first storage location within a source image repository for each layer data of the image file; Distributing each layer data of the image file from the source image repository to a target image repository according to the first storage location; after distributing each layer data of the image file from the source image repository to a target image repository according to the first storage location; identifying a third storage location within the target image repository for each layer data of the image file after distribution; if it is verified that the layer data stored in the third storage location does not match the layer data stored in the first storage location, again performing the step of distributing the layer data of the image file from the source image repository to a target image repository according to the first storage location; Image distribution methods including:
2. Distributing each layer data of the image file from the source image repository to a target image repository in response to the first storage location includes: copying each layer data of the image file from the first storage location and storing the copied each layer data of the image file in the target image repository.
2. The image distribution method of claim 1.
3. and determining, in response to the first storage location, that an image file identical to the image file does not exist in the target image repository prior to distributing each layer data of the image file from the source image repository to the target image repository.
2. The image distribution method of claim 1.
4. Determining that an image file that is the same as the image file does not exist in the target image repository includes: determining whether an image file with the same name as the image file exists in the target image repository; determining that an image file having the same name as the image file does not exist in the target image repository if an image file having the same name as the image file does not exist in the target image repository.
4. The image distribution method according to claim 3.
5. Determining that an image file that is the same as the image file does not exist in the target image repository includes: If an image file having the same name as the image file exists in the target image repository, identifying a second storage location in the target image repository for each layer data of the image file having the same name as the image file; determining that an image file identical to the image file does not exist in the target image repository when the layer data stored in the second storage location does not match the layer data stored in the first storage location; The image distribution method of claim 4 further comprising:
6. after identifying a first storage location within a source image repository for each layer data of the image file; writing the first storage location to a non-volatile memory; and again performing the step of distributing each layer data of the image file from the source image repository to a target image repository in response to the first storage location, the first storage location being obtained from the non-volatile memory.
2. The image distribution method of claim 1.
7. Distributing each layer data of the image file from the source image repository to a target image repository in response to the first storage location includes: copying each layer data of the image file from the first storage location, and overwriting and storing the copied each layer data of the image file in the second storage location; 6. The image distribution method of claim 5, comprising:
8. after identifying a second storage location in the target image repository for each layer data of an image file having the same name as the image file; writing the second storage location to a non-volatile memory; after distributing each layer data of the image file from the source image repository to a target image repository according to the first storage location; obtaining the second storage location from the non-volatile memory; if it is verified that the layer data stored in the second storage location does not match the layer data stored in the first storage location, again performing the step of distributing the layer data of the image file from the source image repository to a target image repository according to the first storage location; The image distribution method of claim 7 further comprising:
9. A server, At least one processor; a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to execute the image distribution method of any one of claims 1 to 8. server.
10. 1. A computer-readable storage medium, comprising: A computer program is stored, and when the computer program is executed by a processor, the image distribution method according to any one of claims 1 to 8 is realized. A computer-readable storage medium.
Citation Information
Patent Citations
Distributed storage based Docker image downloading method
CN106506587A
Method, system and computer program for managing remote deployment of virtual machine in network environment
JP2011123891A
Container providing support system and container providing support method
JP2020095547A
Application server provisioning system and method based on disk image profile
US20080235266A1
Conflict checking using configuration images
US6564369B1
