Mirror image container volume optimization method and device, equipment and storage medium
By finely building and optimizing Docker images, selecting appropriate basic images, decomposing applications, building and deleting redundant data layer by layer, the problem of container image volume expansion is solved, and storage and transmission efficiency is improved and container startup speed is achieved. improve.
Patent Information
- Application Number
- CN202411015104.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-12-03
AI Technical Summary
The expansion of container image volume leads to problems such as large storage space, low transmission efficiency and long startup time, especially when servers with limited resources or poor cross-regional network conditions.
By building Docker images in a refined way, selecting the appropriate basic images, decomposing the application into multiple components, building the images layer by layer, using the deduplication algorithm to delete redundant data, and compressing the images to obtain the optimized image.
Significantly reduce the image volume, reduce storage and transmission costs, improve container startup speed, simplify the image structure, facilitate update and maintenance, and reduce management costs.
Smart Images

Figure CN119065782A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, equipment and storage medium for optimizing the volume of a mirror container. Background Art
[0002] Container technology is a process-level virtualization technology. The container image packages the application and its dependent library files, operating environment, etc. Relying on container technology, lightweight release and iteration of applications can be achieved through the deployment and instantiation of container images. With the rapid development of cloud technology, typical container products such as Docker and Rocket have been widely used in the application practices of enterprises, governments, banks and other institutions. At the same time, driven by edge computing and Internet of Things technologies, container technology has gradually moved from the cloud native ecosystem to the edge. Due to its lightweight and platform-independent nature, containers are highly adaptable to the application needs of massive heterogeneous edge devices. However, as the complexity of applications increases, the size of container images also expands, which brings several significant problems:
[0003] Large storage space usage: Large images require more storage space, which can be a problem for servers or cloud instances with limited resources.
[0004] Inefficient transmission: Large images take longer to transmit, which is especially noticeable across regions or when network conditions are poor, increasing deployment delays and costs.
[0005] Long startup time: When the container starts, it needs to download or decompress the image from the image repository first. Larger images will naturally increase the startup time.
[0006] Therefore, the existing technology still needs to be improved and developed. Summary of the invention
[0007] The present invention provides a method, device, equipment and storage medium for optimizing the volume of an image container, which are used to significantly reduce the image volume by finely constructing a Docker image, deleting redundant data and performing compression processing.
[0008] A first aspect of the present invention provides a method for optimizing the volume of an image container, the method comprising: selecting a base image from a preset image library according to the characteristics and requirements of an application to be optimized; decomposing the application to be optimized to obtain a decomposition list including multiple components; building a Docker image layer by layer according to the decomposition list and the base image; using a deduplication algorithm to scan files of each image layer of the Docker image to obtain a cleaned Docker image; and compressing the cleaned Docker image to obtain an optimized Docker image.
[0009] Optionally, in a first implementation manner of the first aspect of the present invention, selecting a base image from a preset image library according to the characteristics and requirements of the application to be optimized includes: determining a search keyword according to the characteristics and requirements of the application to be optimized; searching for multiple candidate images from the preset image library based on the search keyword; evaluating the volumes of the multiple candidate images, and selecting the candidate image with the smallest volume as the base image.
[0010] Optionally, in a second implementation of the first aspect of the present invention, the application to be optimized is decomposed to obtain a decomposition list including multiple components, including: decomposing the application to be optimized according to functions to obtain multiple components; determining the dependencies and calling order between the multiple components, and dividing the multiple components into multiple levels according to the dependencies and calling order of each component; generating a decomposition list according to the components included in each level and the dependencies corresponding to the components.
[0011] Optionally, in a third implementation of the first aspect of the present invention, building the Docker image layer by layer according to the decomposition list and the base image includes: creating a core file of the image, the core file including a series of instructions and configurations required to build the Docker image; creating a new image layer for each instruction of the core file, thereby obtaining multiple image layers; according to the decomposition list, copying the multiple components and the dependencies corresponding to the components to corresponding positions of the multiple image layers, so as to build the Docker image layer by layer.
[0012] Optionally, in a fourth implementation of the first aspect of the present invention, a deduplication algorithm is used to scan files of each image layer of a Docker image to obtain a cleaned Docker image, including: based on a hash value comparison algorithm, traversing files of each layer of the Docker image, and calculating a hash value for each file; for multiple files with the same hash value, deleting files of other layers while retaining only files of the base layer; and establishing hard links or soft links pointing to the retained files at corresponding positions of other layers.
[0013] Optionally, in a fifth implementation of the first aspect of the present invention, the cleaned Docker image is compressed to obtain an optimized Docker image, including: exporting the cleaned Docker image as a tar file; compressing the exported tar file; evaluating the compression ratio of the compressed tar file, and if the compression ratio is greater than a preset value, using the compressed tar file as the optimized Docker image.
[0014] Optionally, in a sixth implementation method of the first aspect of the present invention, the cleaned Docker image is compressed to obtain an optimized Docker image, and then further includes: when the Docker image needs to be updated, obtaining update content; comparing the update content with the existing Docker image to identify the image layer to be updated; transmitting and updating the image layer to be updated according to the update content to obtain an updated Docker image.
[0015] The second aspect of the present invention provides a device for optimizing the volume of an image container, comprising: a selection module, used to select a basic image from a preset image library according to the characteristics and requirements of an application to be optimized; a decomposition module, used to decompose the application to be optimized to obtain a decomposition list including multiple components; a construction module, used to construct a Docker image layer by layer according to the decomposition list and the basic image; a deletion module, used to scan files of each image layer of the Docker image using a deduplication algorithm to obtain a cleaned Docker image; and an optimization module, used to compress the cleaned Docker image to obtain an optimized Docker image.
[0016] Optionally, in a first implementation of the second aspect of the present invention, the selection module includes: a determination unit, used to determine a search keyword according to the characteristics and requirements of the application to be optimized; a search unit, used to search for multiple candidate images from a preset image library based on the search keyword; and a selection unit, used to evaluate the volumes of the multiple candidate images and select the candidate image with the smallest volume as the base image.
[0017] Optionally, in a second implementation of the second aspect of the present invention, the decomposition module includes: a decomposition unit, used to decompose the application to be optimized according to functions to obtain multiple components; a division unit, used to determine the dependencies and calling order between the multiple components, and divide the multiple components into multiple levels according to the dependencies and calling order of each component; a generation unit, used to generate a decomposition list according to the components included in each level and the dependencies corresponding to the components.
[0018] Optionally, in a third implementation of the second aspect of the present invention, the building module includes: a first creation unit, used to create a core file of the image, the core file including a series of instructions and configurations required to build the Docker image; a second creation unit, used to create a new image layer for each instruction of the core file, thereby obtaining multiple image layers; a building unit, used to copy the multiple components and the dependencies corresponding to the components to the corresponding positions of the multiple image layers according to the decomposition list, so as to build the Docker image layer by layer.
[0019] Optionally, in a fourth implementation of the second aspect of the present invention, the deletion module includes: a calculation unit, which is used to traverse the files of each layer of the Docker image based on a hash value comparison algorithm and calculate the hash value of each file; a deletion unit, which is used to delete files of other layers on the basis of retaining only files of the base layer for multiple files with the same hash value; and a link unit, which is used to establish hard links or soft links pointing to the retained files at corresponding positions of other layers.
[0020] Optionally, in a fifth implementation of the second aspect of the present invention, the optimization module includes: an export unit, used to export the cleaned Docker image as a tar file; a compression unit, used to compress the exported tar file; and an optimization unit, used to evaluate the compression ratio of the compressed tar file. If the compression ratio is greater than a preset value, the compressed tar file is used as the optimized Docker image.
[0021] A third aspect of the present invention provides a mirror container volume optimization device, comprising: a memory and at least one processor, wherein the memory stores computer-readable instructions, and the memory and the at least one processor are interconnected through lines; the at least one processor calls the computer-readable instructions in the memory so that the mirror container volume optimization device performs the various steps of the mirror container volume optimization method as described above.
[0022] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, which, when executed on a computer, enable the computer to execute the various steps of the image container volume optimization method described above.
[0023] In the technical solution provided by the present invention, according to the characteristics and requirements of the application to be optimized, a basic image is selected from a preset image library, the application to be optimized is decomposed to obtain a decomposition list including multiple components, and a Docker image is constructed layer by layer according to the decomposition list and the basic image, and a deduplication algorithm is used to scan the files of each image layer of the Docker image to obtain a cleaned Docker image, and finally the cleaned Docker image is compressed to obtain an optimized Docker image. By selecting a suitable basic image, decomposing the application, and constructing and cleaning the image layer by layer, the size of the image can be significantly reduced, which helps to reduce storage and transmission costs and improve the startup speed of the container. Moreover, the layered Docker image makes each layer clearer and independent, which is convenient for subsequent updates and maintenance. At the same time, by compressing and optimizing the image structure, management costs can also be reduced and operation and maintenance efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 A first flow chart of a method for optimizing the volume of an image container provided by an embodiment of the present invention;
[0025] Figure 2 A second flow chart of the image container volume optimization method provided by an embodiment of the present invention;
[0026] Figure 3 A third flow chart of the image container volume optimization method provided by an embodiment of the present invention;
[0027] Figure 4 A fourth flow chart of the image container volume optimization method provided by an embodiment of the present invention;
[0028] Figure 5 A fifth flow chart of the image container volume optimization method provided by an embodiment of the present invention;
[0029] Figure 6 A sixth flow chart of the image container volume optimization method provided by an embodiment of the present invention;
[0030] Figure 7 A schematic diagram of the structure of a mirror container volume optimization device provided by an embodiment of the present invention;
[0031] Figure 8 A schematic diagram of the structure of a mirror container volume optimization device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0032] The embodiment of the present invention provides a method, device, equipment and storage medium for optimizing the volume of an image container. The method is used to significantly reduce the volume of an image by finely constructing a Docker image, deleting redundant data and compressing it. The method includes: selecting a basic image from a preset image library according to the characteristics and requirements of the application to be optimized; decomposing the application to be optimized to obtain a decomposition list including multiple components; constructing a Docker image layer by layer according to the decomposition list and the basic image; using a deduplication algorithm to scan the files of each image layer of the Docker image to obtain a cleaned Docker image; compressing the cleaned Docker image to obtain an optimized Docker image.
[0033] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0034] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 A first embodiment of a method for optimizing the volume of an image container in an embodiment of the present invention includes:
[0035] S101. Select a base image from a preset image library according to the characteristics and requirements of the application to be optimized.
[0036] It is understandable that the execution subject of the present invention may be a mirror container volume optimization device, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.
[0037] In this embodiment, the characteristics of the application to be optimized include: the programming language (such as Java, Python, Node.js, etc.) and framework (such as Spring Boot, Django, React, etc.) used by the application, the external libraries, services and tools that the program depends on, and the operating environment of the application.
[0038] These dependencies may require specific environment configuration or pre-installed packages,
[0039] The operating environment of an application refers to whether it runs in a server environment (such as a web server, database server, etc.) or in a client environment (such as a browser, mobile device, etc.).
[0040] The requirements of the application to be optimized include the application's demand for CPU, memory, disk, and network bandwidth.
[0041] In this embodiment, the preset image library includes an operating system image, a programming language image, and an application image.
[0042] In this embodiment, when there are multiple base images that meet the requirements, the base image with the smallest volume is selected.
[0043] S102: Decompose the application program to be optimized to obtain a decomposition list including multiple components.
[0044] In this embodiment, the architecture of the application is analyzed, different functional modules or components are identified, and the dependencies, configuration files, execution scripts, etc. of each component are listed, and then a breakdown list is created to clarify the build order and required resources of each component.
[0045] S103, building a Docker image layer by layer according to the decomposition list and the base image.
[0046] In this embodiment, when building a Docker image layer by layer according to the decomposition list and the base image, first write a Dockerfile, use the FROM instruction to specify the base image, then use the COPY, ADD and other instructions to copy the files and dependencies of each component to the image, and then use the RUN instruction to perform necessary installation, compilation, configuration and other operations.
[0047] In this embodiment, when building the Docker image layer by layer, multiple related commands are merged into one RUN instruction as much as possible to reduce the number of image layers.
[0048] In this embodiment, when building the Docker image layer by layer, a multi-stage build technology is used to decompose the building process into multiple stages, and only necessary files and dependencies are retained in each stage.
[0049] S104: Use a deduplication algorithm to scan files of each image layer of the Docker image to obtain a cleaned Docker image.
[0050] In this embodiment, a deduplication algorithm is used to scan files of each image layer of the Docker image, identify and delete redundant files, so as to reduce the image size.
[0051] In this embodiment, tools such as DockerSlim or DockerSlim-Slim can be used to scan the files of each image layer of the Docker image and identify duplicate files and data. These tools can delete duplicate files and replace them with hard links or symbolic links in the image, thereby reducing storage space.
[0052] S105: compress the cleaned Docker image to obtain an optimized Docker image.
[0053] In this embodiment, the cleaned Docker image is compressed to further reduce its size for easy storage and transmission.
[0054] In this embodiment, the image can be saved as a gzip compressed tar file using the Docker built-in docker save command. You can also use a compression tool (such as gzip) to compress the image. The compressed image will be automatically decompressed when deployed, so it will not affect the operation of the application.
[0055] The present embodiment provides a method for optimizing the volume of an image container, which selects a base image from a preset image library according to the characteristics and requirements of an application to be optimized, decomposes the application to be optimized, obtains a decomposition list including multiple components, and builds a Docker image layer by layer according to the decomposition list and the base image, uses a deduplication algorithm to scan the files of each image layer of the Docker image to obtain a cleaned Docker image, and finally compresses the cleaned Docker image to obtain an optimized Docker image. By selecting a suitable base image, decomposing the application, building and cleaning the image layer by layer, etc., the size of the image can be significantly reduced, which helps to reduce storage and transmission costs and improve the startup speed of the container. Moreover, the layered Docker image makes each layer clearer and independent, which is convenient for subsequent updates and maintenance. At the same time, by compressing and optimizing the image structure, management costs can also be reduced and operation and maintenance efficiency can be improved.
[0056] See also Figure 2 The second embodiment of the image container volume optimization method in the embodiment of the present invention includes:
[0057] S201. Determine search keywords according to the characteristics and requirements of the application to be optimized.
[0058] In this embodiment, the type of application to be optimized (such as Web service, database, backend service, etc.), the programming language or framework used (such as Node.js, Python, Django, etc.), and any specific dependencies or libraries are determined according to the characteristics and requirements of the application to be optimized. Based on this information, search keywords can be constructed, which will be used to find suitable candidate images in the image library.
[0059] In this embodiment, the keywords are exemplified as follows:
[0060] Keywords related to application type are:
[0061] Web services: nginx, apache, httpd, tomcat;
[0062] Database: mysql, postgresql, mongodb, redis; back-end service: nodejs, python, java, go, flask, django.
[0063] Keywords related to programming languages or frameworks include:
[0064] Node.js: node, nodejs, express, koa;
[0065] Python: python, django, flask, gunicorn;
[0066] Java: java, openjdk, tomcat, spring-boot;
[0067] Go: go, golang, echo, gin.
[0068] Keywords related to specific dependencies or libraries are:
[0069] Specific libraries: tensorflow (for machine learning), react (for front-end frameworks);
[0070] Operating system environment: alpine (lightweight Linux distribution), ubuntu, centos, debian.
[0071] Keywords related to optimization goals include:
[0072] Minimized size: slim, alpine (indicates lightweight or optimized image versions);
[0073] Security hardening: secure, hardened, official (official security hardening version).
[0074] For a web service built with Node.js and Express framework, the keywords are "nodejs, express"; for a back-end application that requires MySQL database support, the keywords are "mysql, official"; for a Python Flask application that pursues minimal size, the keywords are "python, flask, alpine".
[0075] S202: Search and obtain multiple candidate images from a preset image library based on the search keyword.
[0076] In this embodiment, each candidate image is preset with a corresponding keyword. When searching from a preset image library (such as Docker Hub, Google Container Registry, etc.) based on the search keyword, there may be multiple candidate images that match the keyword.
[0077] S203: Evaluate the volumes of multiple candidate images, and select the candidate image with the smallest volume as the base image.
[0078] In this embodiment, for multiple candidate images, the candidate image with the smallest size is selected as the base image. Using smaller images can reduce the total storage requirements, thereby reducing costs. Moreover, smaller images require less time to load into memory and start containers, which helps to improve the responsiveness and availability of applications.
[0079] In other embodiments, the adaptability of multiple candidate images may be evaluated, and the candidate image with the highest applicability may be selected as the base image, that is, the best candidate image may be selected based on factors such as function, dependency, configuration, size, and community evaluation.
[0080] In this embodiment, the search keywords are determined according to the characteristics and requirements of the application to be optimized, and multiple candidate images are searched from a preset image library based on the search keywords. By determining the search keywords according to the characteristics and requirements of the application, a suitable basic image can be accurately located, thereby improving the accuracy and efficiency of the selection. Moreover, by evaluating the volumes of multiple candidate images and selecting the candidate image with the smallest volume as the basic image, the total storage requirement can be reduced, thereby reducing costs.
[0081] See also Figure 3 A third embodiment of a method for optimizing the volume of an image container in an embodiment of the present invention includes:
[0082] S301, decomposing the application program to be optimized according to its functions to obtain multiple components.
[0083] In this embodiment, the application to be optimized is divided into several functional modules, and then the application to be optimized is decomposed according to the function to obtain multiple components, each component (i.e., functional module) is responsible for a group of related functions. For example, an e-commerce application may be divided into components such as user management, product management, order processing, and payment system.
[0084] S302: Determine the dependency relationship and calling sequence among the multiple components, and divide the multiple components into multiple levels according to the dependency relationship and calling sequence of each component.
[0085] In this embodiment, a component diagram or dependency diagram in UML (Unified Modeling Language) is used to visualize the dependency relationship between components. Each component is represented by a rectangle, and the dependency relationship is represented by a line with an arrow, and the arrow points to the dependent component.
[0086] In this embodiment, the calling sequence between components is determined according to the execution flow of the application program.
[0087] In this embodiment, components are divided into different layers according to the dependency relationship and calling sequence of components, for example, data access layer, business logic layer, presentation layer, etc.
[0088] In this embodiment, it is ensured that each layer only interacts with its directly adjacent layer and cross-layer calls are avoided to keep the code clear and maintainable.
[0089] S303: Generate a decomposition list according to the components included in each level and the dependencies corresponding to the components.
[0090] In this embodiment, for each component, other components or external resources (such as databases, files, other services, etc.) that it depends on are listed.
[0091] In this embodiment, it is possible to analyze whether the dependencies are reasonable and whether there are unnecessary dependencies or circular dependencies.
[0092] In this embodiment, the above analysis results are organized into a document, including the components included in each layer and the dependencies corresponding to each component, so as to generate a decomposition list.
[0093] See also Figure 4 A fourth embodiment of a method for optimizing the volume of an image container in an embodiment of the present invention includes:
[0094] S401. Create the core file of the image. The core file includes a series of instructions and configurations required to build the Docker image.
[0095] S402: Create a new image layer for each instruction of the core file, thereby obtaining multiple image layers.
[0096] S403: According to the decomposition list, multiple components and dependencies corresponding to the components are copied to corresponding positions of multiple image layers, and the Docker image is built layer by layer.
[0097] In this embodiment, illustratively, the decomposition list includes component A (dependency library X, Y), component B (dependency library Z) and component C (no additional dependencies).
[0098] For example, the core file of the image is written according to the decomposition list and the dependency relationship of the components. Each instruction in the core file of the image creates a new image layer.
[0099] The following is an example of some of the contents of the image's core file:
[0100] #Use base image
[0101] FROM ubuntu:latest
[0102] #Install component A's dependency library
[0103] RUN apt-get update&&apt-get install-y
[0104] library-x
[0105] library-y
[0106] #Copy the code of component A to the image
[0107] COPY. / componentA / app / componentA
[0108] #(Optional) Set the working directory of component A
[0109] WORKDIR / app / componentA
[0110] #(Optional) Execute the installation or build script of component A
[0111] RUN . / install.sh
[0112] #Next, repeat the above steps for component B
[0113] RUN apt-get install-y library-z
[0114] COPY. / componentB / app / componentB
[0115] WORKDIR / app / componentB
[0116] RUN . / install.sh
[0117] #...Repeat similar instructions for other components.
[0118] In this embodiment, when building a Docker image, unnecessary layers are minimized.
[0119] In this embodiment, by building the Docker image layer by layer, it is possible to avoid including duplicate files in the image, thereby reducing the size of the image. At the same time, by utilizing the Docker cache mechanism, the image building process can be accelerated, especially when multiple image layers can be shared.
[0120] See also Figure 5 A fifth embodiment of a method for optimizing the volume of an image container in an embodiment of the present invention includes:
[0121] S501, based on the hash value comparison algorithm, traverse the files of each layer of the Docker image and calculate the hash value of each file.
[0122] In this embodiment, a hash value-based comparison algorithm (such as SHA-256) is used to traverse all files in each image layer, and then a hash value is calculated for each file, and the file path and hash value are stored in a data structure (such as a hash table) for subsequent search.
[0123] S502: For multiple files with the same hash value, delete files at other levels while retaining only files at the base level.
[0124] In this embodiment, the hash values of files in different layers are compared to find duplicate files with the same hash value, and the same files in other layers are deleted based on the files in the base layer.
[0125] S503: Establish a hard link or a soft link pointing to the reserved file at a corresponding position of another layer.
[0126] In this embodiment, at the level of deleted files, hard links pointing to retained files are created, and soft links (symbolic links) can be created if cross-file system references are required.
[0127] Furthermore, the changes to remove redundant files and create links are submitted as a new image layer.
[0128] In this embodiment, based on the hash value comparison algorithm, the files of each layer of the Docker image are traversed, and the hash value of each file is calculated. For multiple files with the same hash value, the files of other layers are deleted while only the files of the base layer are retained, and hard links or soft links pointing to the retained files are established at the corresponding positions of the other layers. By deleting duplicate files and sharing file contents, redundant data in the image can be reduced, the optimized image size is smaller, storage space and bandwidth can be saved, and the complexity of the image is reduced, simplifying maintenance work.
[0129] See also Figure 6 A sixth embodiment of a method for optimizing the volume of an image container in an embodiment of the present invention includes:
[0130] S601. Export the cleaned Docker image as a tar file.
[0131] In this embodiment, the Docker image can be exported as a tar file using the docker save command. The file contains all layers and metadata of the image.
[0132] S602: compress the exported tar file.
[0133] In this embodiment, a compression tool such as gzip, bzip2, xz, etc. is selected to compress the tar file.
[0134] S603: Evaluate the compression ratio of the compressed tar file. If the compression ratio is greater than a preset value, use the compressed tar file as the optimized Docker image.
[0135] In this embodiment, the compression ratio is the ratio of the compressed file size to the original file size (usually expressed as a percentage). For example, if the original tar file is 100MB and the compressed file is 50MB, the compression ratio is 50%.
[0136] In this embodiment, the calculated compression ratio is compared with a preset threshold value, and if the compression ratio is greater than the preset value, the compression is successful.
[0137] Furthermore, if the compression ratio is lower than a preset value, compression may continue until the compression ratio of the compressed tar file is greater than the preset value.
[0138] In this embodiment, the file of the cleaned Docker image is compressed to obtain an optimized Docker image, and then the following includes: when the Docker image needs to be updated, the update content is obtained; the update content is compared with the existing Docker image to identify the image layer to be updated; the image layer to be updated is transmitted and updated according to the update content to obtain an updated Docker image.
[0139] In this embodiment, when comparing the updated content with the existing Docker image, the image layer structure of the existing Docker image is analyzed and which layers have changed are identified. You can use Docker's official tools (such as dockerhistory) or third-party tools (such as dive, docker-slim, etc.) to view the layer information of the image and help identify the differences.
[0140] In this embodiment, the update content includes newly added files, modified files, deleted files, or changed configurations.
[0141] In this embodiment, only the update content is transmitted to the location of the image layer to be updated, which can be achieved through the Docker API, the Docker Registry API or other file transfer tools.
[0142] In this embodiment, by compressing the Docker image, the space it occupies on the disk can be significantly reduced. At the same time, the smaller file size means that less network bandwidth and shorter time are required when pushing or pulling the image. The compressed image is easier to back up and restore because the backup file is smaller and the recovery process is faster. In addition, by setting a preset compression ratio, the compression strategy can be adjusted according to different needs and environments.
[0143] The above describes the method for optimizing the volume of the mirror container in the embodiment of the present invention. The following describes the device in the embodiment of the present invention. Figure 7 , the implementation of the image container volume optimization device in the embodiment of the present invention includes:
[0144] A selection module 701 is used to select a base image from a preset image library according to the characteristics and requirements of the application to be optimized;
[0145] A decomposition module 702, used to decompose the application to be optimized to obtain a decomposition list including multiple components;
[0146] A construction module 703 is used to construct a Docker image layer by layer according to the decomposition list and the base image;
[0147] A deletion module 704 is used to scan the files of each image layer of the Docker image using a deduplication algorithm to obtain a cleaned Docker image;
[0148] The optimization module 705 is used to compress the cleaned Docker image to obtain an optimized Docker image.
[0149] In this embodiment, the selection module 701 includes: a determination unit 7011, which is used to determine a search keyword according to the characteristics and requirements of the application to be optimized; a search unit 7012, which is used to search for multiple candidate images from a preset image library based on the search keyword; and a selection unit 7013, which is used to evaluate the volumes of the multiple candidate images and select the candidate image with the smallest volume as the base image.
[0150] In this embodiment, the decomposition module 702 includes: a decomposition unit 7021, which is used to decompose the application to be optimized according to functions to obtain multiple components; a division unit 7022, which is used to determine the dependencies and calling order between the multiple components, and divide the multiple components into multiple levels according to the dependencies and calling order of each component; a generation unit 7021, which is used to generate a decomposition list according to the components included in each level and the dependencies corresponding to the components.
[0151] In this embodiment, the construction module 703 includes: a first creation unit 7031, which is used to create a core file of the image, and the core file includes a series of instructions and configurations required to build a Docker image; a second creation unit 7032, which is used to create a new image layer for each instruction of the core file, thereby obtaining multiple image layers; a construction unit 7033, which is used to copy the multiple components and the dependencies corresponding to the components to the corresponding positions of the multiple image layers according to the decomposition list, so as to build the Docker image layer by layer.
[0152] In this embodiment, the deletion module 704 includes: a calculation unit 7041, which is used to traverse the files of each layer of the Docker image based on a hash value comparison algorithm and calculate the hash value of each file; a deletion unit 7042, which is used to delete the files of other layers on the basis of retaining only the files of the base layer for multiple files with the same hash value; a link unit 7043, which is used to establish a hard link or soft link pointing to the retained file at the corresponding position of the other layer.
[0153] In this embodiment, the optimization module 705 includes: an export unit 7051, which is used to export the cleaned Docker image as a tar file; a compression unit 7052, which is used to compress the exported tar file; an optimization unit 7053, which is used to evaluate the compression ratio of the compressed tar file. If the compression ratio is greater than a preset value, the compressed tar file is used as the optimized Docker image.
[0154] In this embodiment, according to the characteristics and requirements of the application to be optimized, a basic image is selected from a preset image library, the application to be optimized is decomposed to obtain a decomposition list including multiple components, and a Docker image is constructed layer by layer according to the decomposition list and the basic image, and a deduplication algorithm is used to scan the files of each image layer of the Docker image to obtain a cleaned Docker image, and finally the cleaned Docker image is compressed to obtain an optimized Docker image. By selecting a suitable basic image, decomposing the application, and constructing and cleaning the image layer by layer, the size of the image can be significantly reduced, which helps to reduce storage and transmission costs and improve the startup speed of the container. Moreover, the layered Docker image makes each layer clearer and independent, which is convenient for subsequent updates and maintenance. At the same time, by compressing and optimizing the image structure, management costs can also be reduced and operation and maintenance efficiency can be improved.
[0155] above Figure 7 The image container volume optimization device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The image container volume optimization device in the embodiment of the present invention is described in detail from the perspective of hardware processing.
[0156] Figure 8 800 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 810 (for example, one or more processors) and memory 820, and one or more storage media 830 (for example, one or more mass storage devices) storing application programs 833 or data 832. Among them, the memory 820 and the storage medium 830 may be short-term storage or persistent storage. The program stored in the storage medium 830 may include one or more modules (not shown), and each module may include a series of instruction operations in the device 800. Furthermore, the processor 810 may be configured to communicate with the storage medium 830 to execute a series of instruction operations in the storage medium on the device 800.
[0157] The device 800 may also include one or more power supplies 840, one or more wired or wireless network interfaces 850, one or more input and output interfaces 860, and / or one or more operating systems 831, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc.
[0158] An embodiment of the present invention further provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the image container volume optimization method.
[0159] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.
[0160] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.
[0161] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for optimizing the volume of an image container, characterized in that: The image container volume optimization method comprises: Select a base image from the preset image library based on the characteristics and requirements of the application to be optimized; Decompose the application to be optimized to obtain a decomposition list including multiple components; Building a Docker image layer by layer according to the decomposition list and the base image; Use a deduplication algorithm to scan the files of each image layer of the Docker image to obtain the cleaned Docker image; The cleaned Docker image is compressed to obtain an optimized Docker image.
2. The image container volume optimization method according to claim 1, characterized in that: The step of selecting a base image from a preset image library according to the characteristics and requirements of the application to be optimized includes: Determine search keywords based on the characteristics and requirements of the application to be optimized; Searching for multiple candidate images from a preset image library based on the search keyword; The volumes of the multiple candidate images are evaluated, and the candidate image with the smallest volume is selected as the base image.
3. The image container volume optimization method according to claim 1, characterized in that: The application to be optimized is decomposed to obtain a decomposition list including multiple components, including: Decompose the application to be optimized according to its functions to obtain multiple components; Determine the dependencies and calling order between multiple components, and divide the multiple components into multiple levels according to the dependencies and calling order of each component; Generates an exploded list based on the components included in each layer and the dependencies corresponding to the components.
4. The image container volume optimization method according to claim 1, characterized in that: The step of building the Docker image layer by layer according to the decomposition list and the base image includes: Create the core file of the image, which includes a series of instructions and configurations required to build the Docker image; Create a new image layer for each instruction of the core file, thus obtaining multiple image layers; According to the decomposition list, the multiple components and the dependencies corresponding to the components are copied to corresponding positions of the multiple image layers to build the Docker image layer by layer.
5. The image container volume optimization method according to claim 1, characterized in that: Use the deduplication algorithm to scan the files of each image layer of the Docker image to obtain the cleaned Docker image, including: Based on the hash value comparison algorithm, traverse the files of each layer of Docker image and calculate the hash value of each file; For multiple files with the same hash value, delete the files at other levels while retaining only the files at the base level; A hard link or soft link pointing to the retained file is established at the corresponding position of other layers to obtain the cleaned Docker image.
6. The image container volume optimization method according to claim 1, characterized in that: The step of compressing the cleaned Docker image to obtain an optimized Docker image includes: Export the cleaned Docker image as a tar file; Compress the exported tar file; Evaluate the compression ratio of the compressed tar file. If the compression ratio is greater than a preset value, use the compressed tar file as the optimized Docker image.
7. The image container volume optimization method according to claim 1, characterized in that: The cleaned Docker image is compressed to obtain an optimized Docker image, and then the following steps are further included: When the Docker image needs to be updated, obtain the updated content; Compare the updated content with the existing Docker image to identify the image layer to be updated; The image layer to be updated is transmitted and updated according to the update content to obtain an updated Docker image.
8. A mirror container volume optimization device, characterized in that: include: A selection module is used to select a base image from a preset image library according to the characteristics and requirements of the application to be optimized; A decomposition module, used for decomposing the application to be optimized to obtain a decomposition list including multiple components; A construction module, used to build a Docker image layer by layer according to the decomposition list and the base image; The deletion module is used to scan the files of each image layer of the Docker image using a deduplication algorithm to obtain the cleaned Docker image; The optimization module is used to compress the cleaned Docker image to obtain an optimized Docker image.
9. A mirror container volume optimization device, characterized in that: comprising a memory and at least one processor, wherein the memory has computer-readable instructions stored therein; The at least one processor calls the computer-readable instructions in the memory to execute the various steps of the image container volume optimization method according to any one of claims 1 to 7.
10. A computer-readable storage medium having computer-readable instructions stored thereon, characterized in that: When the computer-readable instructions are executed by a processor, the steps of the method for optimizing the volume of a mirrored container as described in any one of claims 1 to 7 are implemented.
Citation Information
Cited By
File distribution method and device based on layered architecture and medium
CN121029717A
