Container recovery method and device, equipment and storage medium

Abstract

The invention relates to the technical field of computers, and discloses a container recovery method and device, equipment and a storage medium, and the method comprises the steps: analyzing a writable layer of a target container, and obtaining and storing incremental data information of the target container; creating a same-mirror-image new container of the target container based on the original mirror image of the target container; and reading incremental data information, and executing incremental operation on the same-mirror-image new container according to the read content. According to the method and the device, the incremental behavior of the container is analyzed through the writable layer, the corresponding incremental data information is stored, the incremental behavior is reperformed according to the incremental data information in the new container with the same mirror image, the container is recovered, and the incremental content and the development environment of the original container are restored and reserved. According to the method, the container does not need to be solidified into a mirror image, disordered expansion of a mirror image file cannot be caused, disk resources of a host cannot be occupied, container scheduling cannot be affected, and scenes such as continuous development and development debugging of developers and deletion, pause, re-recovery and migration of the container are supported.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a container recovery method, apparatus, device, and storage medium. Background Art

[0002] To effectively utilize high-performance machines, many companies are moving away from providing high-spec development machines to individuals. Instead, they are adopting a cloud development model. Individuals only need a simple development machine, request a container that meets the required configuration from a cluster, and begin development. The development environment starts with a user-built image, allowing users to temporarily change the environment during debugging. This cloud development model is very convenient, but it also has certain drawbacks.

[0003] If the user stops the container and then restarts it from the image, some temporary environments and operation instruction records changed during the previous development process will be lost, affecting continuous development. A common practice in existing technologies is to directly save the container as an image, so that the temporary environment can be completely preserved. However, this will solidify some temporary files that are not needed later, causing the image to become increasingly bloated, and the host disk resources are limited, which in turn affects container scheduling. In addition, in a cloud development environment, if there is a downtime and inability to start, the temporary environment or incremental data in the user's container may be lost, which will also affect the user's development and debugging work. Summary of the Invention

[0004] The main purpose of this application is to provide a container recovery method, device, equipment and storage medium, aiming to solve the technical problem of how to protect temporary changes in the environment or incremental data in cloud development containers.

[0005] A first aspect of the present application provides a container recovery method, the container recovery method comprising:

[0006] Parse the writable layer of the target container to obtain and store the incremental data information of the target container;

[0007] Create a new container with the same image as the target container based on the original image used by the target container;

[0008] Read the incremental data information and perform corresponding incremental operations on the new container with the same image based on the read content.

[0009] The present application also provides a container recovery device, the container recovery device comprising:

[0010] The parsing and backup module is used to parse the writable layer of the target container, obtain and store the incremental data information of the target container;

[0011] Create a new container module, which is used to create a new container with the same image as the target container based on the original image used by the target container;

[0012] The container change module is used to read incremental data information and perform corresponding incremental operations on the new container with the same image based on the read content.

[0013] A third aspect of the present application provides a computer device, comprising: a memory and at least one processor, wherein instructions are stored in the memory; and the at least one processor calls the instructions in the memory so that the computer device executes the above-mentioned container recovery method.

[0014] A fourth aspect of the present application provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned container recovery method.

[0015] This application analyzes the incremental behavior of the container through a writable layer, and stores the incremental data information corresponding to the incremental behavior. In the new container with the same image, the incremental behavior is redone according to the incremental data information to achieve the recovery of the container, thereby realizing the restoration and preservation of the incremental content and development environment of the original container. This application does not need to solidify the container into an image, will not cause disordered expansion of the image file, will not occupy the host disk resources, and will not affect the container scheduling. It can optimize the preservation of the temporary environment and incremental content of the original container, etc., and support developers' continuous development and development debugging. In addition, this application can be applied to scenarios such as container deletion and restoration, suspension and restoration, and container migration. It can assist developers in restoring containers when resources are released by deleting containers, reducing resource waste, and can completely restore the user's development environment in the event of a downtime and inability to start, achieving disaster recovery, and assisting in container migration. This embodiment is aimed at completely protecting the temporarily changed environment in the cloud development container, and can flexibly schedule containers while ensuring the continuity of user development. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flowchart of a first embodiment of a method for recovering a container in an embodiment of the present application;

[0017] Figure 2 This is a schematic diagram of the hierarchical structure of the container in the embodiment of the present application;

[0018] Figure 3 This is a schematic diagram of multiple containers sharing an image layer in an embodiment of the present application;

[0019] Figure 4 This is a schematic diagram of solidifying a container layer into a mirror layer in the prior art;

[0020] Figure 5 This is a schematic diagram of the layering of containers and OverlayFS in an embodiment of the present application;

[0021] Figure 6 This is a schematic diagram of container recovery in an embodiment of the present application;

[0022] Figure 7 This is a functional module diagram of an embodiment of a container recovery device in an embodiment of the present application;

[0023] Figure 8 This is a schematic diagram of an embodiment of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The terms "first," "second," "third," "fourth," and the like (if any) in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] Currently, many companies' businesses involve a large number of complex tasks, such as training deep learning models, which often require a large number of high-performance machines. Code debugging also requires high-performance machines. In this situation, equipping each developer with a high-performance development machine can result in high costs for the company. Therefore, to effectively utilize these high-performance machines, many companies are moving away from providing high-performance personal development machines to individuals and adopting a cloud development model. Individuals only need to set up a simple development machine, request a Docker container with the required configuration within a Kubernetes cluster, and then conduct development work by launching a Jupyter Notebook page or connecting remotely to VSCode. The development environment starts with a user-built image, allowing users to temporarily change the environment during debugging. This cloud development model is very convenient, but it also comes with its own challenges.

[0026] The first is the problem of resource waste. If a user applies for a high-configuration machine but does not use it for a period of time, the resources will be occupied. When resources are tight, other users may not be able to apply for the required resources, resulting in low development efficiency. If the user stops the container and then restarts it from the image, some temporary environments and operation instruction records changed during the previous development process will be lost, affecting continuous development. In existing technologies, a common practice is to directly save the container as an image, so that the temporary environment can be completely preserved. However, this will solidify some temporary files that are not needed later, causing the image to become increasingly bloated. Due to the limited disk resources of the host, this will affect container scheduling.

[0027] Secondly, in a cloud development environment, if a system crashes and fails to start, the temporary environment in the user's container will be lost, affecting the user's development and debugging work. Therefore, a disaster recovery solution is needed to fully restore the user's development environment in the event of a crash.

[0028] Based on this, a strategy is needed to effectively pause and resume containers, or to modify containers to low resource configuration when high-configuration debugging is not needed temporarily.

[0029] refer to Figure 1 In one embodiment of the present application, a method for restoring a container is provided. The method for restoring a container includes:

[0030] S100: Parse the writable layer of the target container to obtain and store incremental data information of the target container.

[0031] Specifically, a container is a process that runs an application and contains its dependencies (bins / libs). It is the delivery entity for container services. A container is a running instance created from an image. It can be created, started, stopped, and deleted. Each container is an isolated and secure platform. The top layer of the container is readable and writable.

[0032] Images: Templates used to create containers (read-only). Different containers can be created based on images with different configurations.

[0033] When an existing image file in an image is modified, Docker uses the storage driver to perform a copy-on-write (COW) operation. The read-only image file is first copied to the upper read-write layer, where the modification is then performed. This container storage structure is implemented by its storage driver. Currently, mainstream storage drivers include union file systems such as OverlayFS.

[0034] The target container (Container) is an old container, which can be one of a container to be paused, a container to be deleted, a container to be migrated, a container to be copied, and the like.

[0035] The target container includes a container mount, a container layer, and an image layer. The image layer may include the first sublayer to the nth sublayer. The container mount, the container layer, the first sublayer to the nth sublayer are located in the container in sequence.

[0036] For example, in Docker, images (or image layers) are hierarchical, with each layer representing a difference from the previous one. These layers, stacked together, form the container's starting file system. When creating a new container using the same image, a new writable layer, called the container layer, is added to the image layer. All changes made to the running container, such as writing new files, modifying existing files, and deleting files, are recorded in the writable container layer.

[0037] When an image is used by multiple containers, each container shares the image layer and has its own independent container layer.

[0038] Figure 2 This is a schematic diagram of the hierarchical structure of the container in the embodiment of the present application; Figure 2 The container includes image layers (R / O) and container layers (Container layer), the image layer is a read-only layer, and the container layer is a thin writable layer (ie, Thin R / W layer).

[0039] Figure 3 This is a schematic diagram of multiple containers sharing an image layer in an embodiment of the present application; Figure 3 Multiple containers share the same image layer, which contains multiple sublayers. Image layers are generally immutable. Each container contains an independent container layer (i.e., a thin writable layer or Thin R / W layer).

[0040] The target container is the container that the developer has already created. During the development process, the developer can modify the target container as needed to create a corresponding development environment. For example, writing new files, modifying existing files, and deleting files will all be recorded in the writable container layer of the target container.

[0041] In some cases, for example, developers do not use the target container temporarily and need to delete or destroy the container to release the resources occupied by the target container. Alternatively, developers want to stop or pause the target container. Alternatively, developers need to migrate the target container. In a cloud development environment, if the target container fails to start due to a downtime, the temporary environment or incremental data in the target container will be lost.

[0042] If the developer subsequently restarts the creation of a container from the image, some temporary environment and operation instruction records of the previous target container development process changes will be lost, which will affect continuous development or the developer's development and debugging work.

[0043] In the prior art, to prevent the loss of a temporary environment or incremental data of a container, the container layer is saved or solidified into an image layer or the container is committed into a new image. Figure 4 This is a schematic diagram of solidifying a container layer into a mirror layer in the prior art; Figure 4 Before the container is saved, the image layer contains 4 sublayers (i.e., image layer1-image layer4). After the container layer is saved as an image layer, the image layer changes from 4 sublayers to 5 sublayers (i.e., including image layer1-image layer5).

[0044] This shows that this method of saving temporary incremental content through images and frequently recycling can save container incremental content, and subsequent new containers created with new images can have the same content as the old containers. However, if this method is used to save temporary incremental content during multiple pauses, resumes, or migrations, the image will become larger and larger, causing the image size to expand rapidly, affecting container scheduling and startup speed.

[0045] To solve the above problems and prevent the loss of temporary environments or incremental data in the target container, this embodiment protects temporarily changed environments or incremental data in the cloud development container.

[0046] The writable layer is readable and writable by OverlayFS, but not directly writable by users. Therefore, the contents of the writable layer cannot be directly backed up or replicated. For example, deletions and replacements in the upperdir cannot be directly replicated and require special handling. Based on this, this embodiment parses the writable layer of the target container to analyze incremental behavior of the target container and obtain and store the corresponding incremental data information. This incremental data information includes changes to the target container, such as file deletions, additions, modifications, and replacements.

[0047] Incremental data information can be stored in shared storage.

[0048] S200: Creating a new container with the same image as the target container based on the original image used by the target container.

[0049] Specifically, the target container consists of an image layer and a container layer. The image layer contains the original image or original image file. Based on the original image, a new container with the same image as the target container can be created. That is, the original image is used as the image layer, and a container layer is created on top of the image layer.

[0050] The new container with the same image can be newly created in any of the following situations: when the target container needs to be restored after being deleted, when the target container needs to be copied, when the target container crashes and cannot be started during cloud development, when the target container needs to be restored after being stopped, when the target container needs to be migrated, etc. This application does not impose any restrictions on this.

[0051] S300: Read incremental data information, and perform corresponding incremental operations on the new container with the same image according to the read content.

[0052] Specifically, in order to continue to enable the temporary environment or incremental content of the target container, this embodiment needs to restore the target container. Specifically, this embodiment uses the incremental data information as a guide to implement changes to the new container of the same image. Read the incremental data information corresponding to the target container, and use the read content to re-execute the same incremental operation or change operation or file operation on the files and / or folders of the new container of the same image. For example, if the target container is found to have deleted files through upperdir analysis, the same file deletion behavior is performed in the new container. If the target container is found to have replaced files through upperdir analysis, the same file replacement behavior is performed in the new container, that is, the old files of the new container are deleted and the new files are copied, so that the merged layer displays the replaced new files and shields the old files. In fact, the old files in the image file are not deleted. If the target container is found to have added files through upperdir analysis, the same file addition behavior is performed in the new container.

[0053] After the incremental operation is completed, the new container with the same image is equivalent to a copy of the target container, that is, the temporary development environment reconstruction and recovery of the target container is realized.

[0054] Taking the target container using OverlayFS (overlay file system) technology as an example, OverlayFS is a file system used to implement joint mounts for Linux file systems. Its core concept is stacking, stacking multiple folders in order. The characteristics of OverlayFS are upper-layer file priority, zero-copy mount merging, and read-only underlying file systems. When merging multiple folders, the folders are stacked in upper-lower order, that is, the read-only file directory and the read-write file directory under the file system are merged, allowing users to make modifications based on the read-write file directory without affecting the read-only file directory itself. What users ultimately see is mergeddir. OverlayFS technology can create joint mounts between multiple image layers, providing efficient file system storage and performance. Use the OverlayFS joint file system to merge directories in different underlying file systems within the container.

[0055] Figure 5 This is a schematic diagram of the layering of containers and OverlayFS in the embodiment of this application; refer to Figure 5 The OverlayFS structure includes Merged, Upperdir, and Lowerdir. The container structure includes container mount, container layer, and image layer.

[0056] lowerdir: This is the read-only underlying directory or lower directory that contains the file system contents of the underlying image and is immutable. That is, the read-only lowerdir corresponds to the image layer.

[0057] upperdir: This is the writable top-level directory or upper directory that contains the container's top-level file system and reflects any runtime modifications. It is used to store changes to the container's file system, including changes to files and directories caused by writes, modifications, or deletions. In other words, the writable upperdir corresponds to the container layer, or the upperdir contains the contents of the container layer.

[0058] The contents of the upper directory will overwrite the contents of the lower directory.

[0059] merged: This is a joint mount point or joint mount directory, which contains a merged view of lowerdir and upperdir, showing the final contents of the union file system. It is a unified view presented to the outside world, also known as the view layer. Changes in upperdir will overwrite the corresponding content in lowerdir.

[0060] When a file needs to be modified, it is copied from the read-only Lowerdir to the writable Upperdir using COW, and the modification result is also saved in the Upperdir layer. In the container, the lower read-only layer is the image layer, and the writable layer is the container layer.

[0061] Containers leverage this OverlayFS feature, setting image files to read-only and merging them with the container's read-write layer. This provides container users with a holistic view of the virtual operating system, where the image files reside in the lower layer by default, while the container's read-write layer resides in the upper layer. Whenever a user needs to access the image files in the lower layer, a COPY UP operation in OverlayFS is triggered.

[0062] Container design employs a key storage strategy, copy on write (COW), which treats image files as read-only. When a container that relies on this image file needs to modify the image file, it must first copy it to the container's upper read-write layer before accessing it. This strategy offers the advantage of allowing the same image file to be used by different container instances, significantly saving image file storage space.

[0063] Figure 6 This is a schematic diagram of container recovery in an embodiment of the present application; Figure 6 Parsing the writable layer of the target container is equivalent to parsing the upperdir layer. User changes to the container through the merged layer are recorded in the upperdir layer. The merged layer supports user read and write, that is, it supports user readable and writable. The upperdir layer supports merged layer write.

[0064] By parsing the upperdir layer, the acquired incremental data can be stored in shared storage. A new container with the same image as the target container is then created. Based on the content read from the shared storage, redo operations can be performed on the new container (i.e., corresponding incremental operations, such as adding files, deleting files, and replacing files, among other operations). This allows the incremental content of the target container to be saved in the new container, allowing it to inherit the target container's operating environment and incremental content.

[0065] This embodiment analyzes the incremental behavior of the container through the writable layer, and stores the incremental data information corresponding to the incremental behavior. In the new container with the same image, the incremental behavior is redone according to the incremental data information to achieve the recovery of the container, thereby realizing the restoration and preservation of the incremental content and development environment of the original container. This embodiment does not need to solidify the container into an image, will not cause disordered expansion of the image file, will not occupy the host disk resources, and will not affect the container scheduling. It can optimize the preservation of the temporary environment and incremental content of the original container, etc., and support developers to continuously develop and debug development. In addition, this embodiment can be applied to scenarios such as container deletion and restoration, suspension and restoration, and container migration. It can assist developers in restoring containers when resources are released by deleting containers, reducing resource waste, and can completely restore the user's development environment in the event of a downtime and inability to start, achieving disaster recovery, and assisting in container migration. This embodiment is aimed at completely protecting the temporarily changed environment in the cloud development container, and can flexibly schedule containers while ensuring the continuity of user development.

[0066] In one embodiment, a new container different from the target container image can be created based on another new image, and the incremental data information can be read. The corresponding incremental operation is performed on the new container based on the read content. The resource configuration of the new container and the target container can be the same or different.

[0067] In one embodiment, step S100 parses the writable layer of the target container to obtain and store incremental data information of the target container, including:

[0068] Parse the writable layer of the target container;

[0069] If a new file is found during parsing, the new file will be stored;

[0070] If a deleted file is parsed, the first path corresponding to the deleted file is obtained and stored;

[0071] If a modified file is parsed, the second path and the replacement file corresponding to the modified file are obtained and stored.

[0072] Specifically, the writable layer of the target container can truly record all changes made to the running target container. For example, writing new files, modifying existing files, and deleting files will all be recorded in the writable layer.

[0073] Therefore, by parsing the writable layer of the target container, the temporarily modified content of the target container can be accurately parsed, and the incremental data information can be obtained based on the temporarily modified content.

[0074] In this embodiment, if a newly added file, ie, a regular file, is found when parsing the writable layer of the target container, the newly added file is stored or backed up in the shared storage.

[0075] If the writable layer of the target container is parsed to contain a deleted file, such as a character device file, the path of the deleted file itself is obtained as the first path, and the first path is stored or recorded. Specifically, the first path can be recorded into a file.

[0076] It should be noted that the deletion file is used to indicate which old file in the container needs to be deleted. The first path of the deletion file is the path of the old file to be deleted in the container.

[0077] If the writable layer of the target container is parsed to contain a modified file, such as an Opaque folder, the path of the modified file is obtained as the second path, stored or recorded, and the replacement content in the modified file is copied and stored as a replacement file, where the replacement content is the folder content in the modified file. Specifically, the second path can be recorded in a file.

[0078] Modifying a file is essentially replacing or shielding an old file in the container. The second path is equivalent to the path of the old file to be replaced or modified in the container.

[0079] Among them, deleted files and modified files both have certain characteristics, based on which deleted files and modified files can be easily distinguished. Newly added files are files other than deleted files and modified files.

[0080] Newly added files, deleted files, and modified files are all differential files, which are the changed data of the image file.

[0081] In a specific embodiment, the newly added files and / or replaced files may be stored according to the original path in the target container, so that the newly added files and replaced files may be added to the new container according to the original path.

[0082] This embodiment parses the writable layer of the container to accurately obtain the changes in the original container compared to the image, identify incremental data such as deleted information, added information, and replaced information, and facilitate subsequent container recovery.

[0083] In one embodiment, in step S300, incremental data information is read, and corresponding incremental operations are performed on the new container of the same image according to the read content, including:

[0084] If the read content is a file path, delete the old file with the file path in the container file of the new container with the same image, where the file path is the first path corresponding to the deleted file or the second path corresponding to the modified file;

[0085] If the read content is file content, the file content is added to the container file of the new container of the same image, where the file content is a new file or a replacement file.

[0086] Specifically, the incremental data information contains the first path corresponding to the deleted file and the second path corresponding to the modified file. The deleted file can indicate which old file in the new container of the same image needs to be deleted, and the modified file actually replaces the corresponding old file in the new container of the same image with the folder content in the modified file.

[0087] Therefore, if the first or second path is read, the old file in the merged container of the same image will be deleted. The container file of the new container of the same image is a non-image file, which can be a file in the merged layer, or a file in the merged layer and the upperdir layer.

[0088] In this embodiment, deleting old files only deletes related content in the container file of the upper layer of the container and does not affect the image files of the lower image layers.

[0089] When performing file operations in a container, if you are reading a file, then:

[0090] If this file exists in the container layer, read the file in the container layer directly.

[0091] If this file does not exist in the container layer, it will be read from the image layer.

[0092] If this file exists in both the container layer and the image layer, the file in the container layer will be read directly. The file in the container layer will overwrite the file in the image layer.

[0093] If you are adding a new file (writing) or deleting a file, then

[0094] When writing a file, if the file exists in the container layer, the file is modified directly in the container layer.

[0095] When writing a file, if the file does not exist in the container layer but exists in the image layer, the file in the image layer will be completely copied to the container layer. When modifying the file later, the modification is done directly in the copy in the container layer.

[0096] When you delete a file, folder, or directory, a whiteout file is created at the container layer to "block" or "hide" the file, folder, or directory at the image layer. The file at the image layer still exists.

[0097] When you delete a directory or folder, an opaque folder or directory is created at the container layer to "block" the folder at the image layer. The folder at the image layer still exists.

[0098] When renaming a folder, it will succeed only if both the source and target paths are at the container level.

[0099] Based on this, deleting the old file in the first or second path in the merged container of the same image will not delete the old file corresponding to the first or second path in the image file of the new container of the same image. Instead, a whiteout file will be created. The whiteout file is created when the user deletes the file and is used to shield the file with the same name in the bottom layer (lowerdir layer). At the same time, the whiteout file is invisible in the merged layer, so that the user cannot see the deleted (shielded) files or directories. This is equivalent to using the whiteout file to shield the old files corresponding to the bottom layer that do not need to be seen by the user, rather than actually deleting the old files in the image layer. The old files are hidden and cannot be seen by the user, thereby achieving an effect similar to that of file deletion.

[0100] Incremental data includes both new and replaced files. New files are essentially copied to a new container within the same image. Replaced files replace old files in the second path, effectively also copying them to the new container within the same image. Therefore, if a new or replaced file is read, it is copied to the container file within the new container within the same image.

[0101] Copying new and replaced files to a new container with the same image actually writes to the writable layer, which does not change the image file of the image layer. Since the content of the writable layer is displayed first, modifying the file can block or replace the image file with the same name in the image layer (lowerdir layer), so that users in the merged layer can only see the replaced file but not the replaced old file.

[0102] Among them, according to the original path of the new file and the replacement file in the target container, the new file and the replacement file can be copied to the corresponding path of the new container with the same image, so that the new file or the replacement file can be written into the new container.

[0103] This embodiment can delete old files to be deleted or replaced in the new container of the same image based on the file path, and can also copy or write new files or replacement files to the new container of the same image, thereby implementing various incremental operations on the new container of the same image.

[0104] In one embodiment, parsing the writable layer of the target container includes:

[0105] Determine the parsed character device file as a deleted file;

[0106] Determine the parsed Opaque file attribute folder as a modified file;

[0107] Other parsed files are determined to be newly added files, where other files are regular files except character device files and Opaque file attribute folders.

[0108] Specifically, if the writable layer of the container includes a character device file, it indicates that the underlying file corresponding to the character device file in the container is to be deleted.

[0109] If the container's writable layer contains a folder with the opaque file attribute, it means that the corresponding old file in the container will be replaced.

[0110] When a file needs to be modified, the system copies the underlying file and makes modifications to the copy rather than directly modifying the original file. This avoids modifying the original file, ensuring data consistency and security. When the upper layer needs to delete an existing file in the lower layer, the container uses Whiteout technology to mark the deleted file. Marked files are not deleted or overwritten. Instead, a special file called whiteout is created to indicate the file's deletion or modification. During subsequent file access and operations, the system determines the file's visibility and status based on the presence of the whiteout file. However, the file still exists in the underlying image.

[0111] Character device files include hidden files, namely whiteout files. Whiteout files are created when users delete files and are used to shield the underlying files with the same name. At the same time, the whiteout files are invisible in the merge layer, so users cannot see the deleted files or directories. Whiteout files are not ordinary files, but character devices with both major and minor device numbers 0 (for example, they can be created by "mknod <name>c 0 0" command to create a whiteout file with the same name as the old file to be deleted). When the user checks the directory entry of the parent directory through the ls command in the merged layer, overlayfs will automatically filter out the whiteout file itself and the lower layer files and directories with the same name as the whiteout file, achieving the purpose of hiding the file and making the user think that the file has been deleted.

[0112] Finding character device files is achieved, for example, through find{local_diff_path}-type c.

[0113] Opaque is achieved by setting the "trusted.overlay.opaque" extended attribute value to "y" on the directory corresponding to the upper layer (the file system where the upper layer is located needs to support the xattr extended attribute). When overlayfs reads directory entries with directories of the same name in the upper and lower layers, if the directory of the upper layer has the opaque attribute set, it will ignore the directory entries in all directories with the same name in the layers below it.

[0114] Opaque file attribute folder, for example, through getfattr -R -n 'trusted.overlay.opaque'

[0115] {local_diff_path} implementation.

[0116] The opaque directory or opaque folder created at the container level has a similar effect to the whiteout file.

[0117] For example, a normal mergeddir is a merged view of all layers. If there is a folder with the Opaque attribute in the upper layer, the contents of the folders in the same directory in the lower layer will be shielded, and only the contents of the folders in the upper layer will be displayed.

[0118] All files in the writable layer of the target container, except character device files and folders with opaque file attributes, are regular files.

[0119] This embodiment can accurately distinguish behaviors such as file deletion, file replacement, and file addition in a container by using the characteristics of character device files and Opaque file attribute folders.

[0120] In one embodiment, deleting the old file at the file path in the container file of the new container of the same image includes:

[0121] If the file path is the path of the Opaque file attribute folder, delete the old file corresponding to the path of the Opaque file attribute folder in the view layer of the new container of the same image;

[0122] If the file path is the path of a character device file, delete the old file corresponding to the path of the character device file in the view layer of the new container of the same image.

[0123] Specifically, in this embodiment, deleting old files only deletes related content in the upper layer of the container and does not affect the lower image layer.

[0124] Read the Opaque list and delete the old file corresponding to the path of the Opaque file attribute folder in the mergeddir, making the old file corresponding to the file path invisible in the merged layer (view layer), while the old file in the image layer or lowerdir layer is not actually deleted. The old file is the folder in the underlying image file.

[0125] In a specific embodiment, the second path corresponding to the Opaque file attribute folder in merged may also be deleted.

[0126] Read the character file list and delete the old files corresponding to the character device file path in mergeddir, making the old files corresponding to the file path invisible in the merged layer (view layer), while the old files in the image layer or lowerdir layer are not actually deleted. The old files may include files and / or folders in the underlying image file.

[0127] In a specific embodiment, the first path corresponding to the character device file in merged may also be deleted.

[0128] It should be noted that the modified files and replaced files in this application are folders, and the deleted files can be either files or folders.

[0129] Deleting old files in the container file of a new container with the same image does not delete the corresponding old files in the image file of the new container with the same image. Instead, a whiteout file is created. The whiteout file is created when the user deletes a file and is used to shield the underlying file with the same name. At the same time, the whiteout file is invisible in the merged layer, making the user unable to see the deleted (shielded) files or directories. This is equivalent to using the whiteout file to shield the corresponding old files that do not need to be seen by the user. It does not actually delete the old files in the image layer. The old files are hidden and cannot be seen by the user, thereby achieving an effect similar to that of deleting the file.

[0130] This embodiment achieves the purpose of file deletion or replacement by deleting the old files merged in the view layer in the container.

[0131] In a specific embodiment, the following is a storage example:

[0132] -rw-r--r--1root root 0Oct 21 21:02c_files.txt

[0133] drwxr-xr-x 9root root 7Oct 21 20:55diff

[0134] -rw-r--r--1root root 202Oct 21 21:02opaque_files.txt

[0135] The first path can be stored in the character file list, c_files.txt, and the second path can be stored in the opaque list, opaque_files.txt. New and replaced files can be stored in the diff file according to the path within the target container. c_files.txt, opaque_files.txt, and diff all store incremental data.

[0136] When reading incremental data information and performing corresponding incremental operations, the second path in opaque_files.txt can be read first, and the old file corresponding to the second path in merged of the new container of the same image can be deleted. The old file is a folder.

[0137] Then read the first path in c_files.txt and delete the old file corresponding to the first path in merged of the new container of the same image. The old file is a file or folder.

[0138] Then read the new and / or replaced files in the diff and copy them to the merged. The new and replaced files in the diff are stored according to the original path in the target container, so they can be directly copied to the merged according to the original path.

[0139] For example, in the upperdir (writable layer) is / workspace / new_file, and in the shared storage it is stored as / workspace / new_file according to the original path (without changing the path).

[0140] The following example creates a shared storage:

[0141] / root / .cache / pip / http-v2 / a

[0142] / root / .cache / pip / http-v2 / 0 / 2

[0143] / root / .cache / pip / http-v2 / 9

[0144] / root / .cache / pip / http-v2 / 7 / 7

[0145] / root / .cache / pip / wheels

[0146] The following is an example of permission settings:

[0147] drwxr-xr-x 4root root 2Aug 5 10:26data

[0148] drwxr-xr-x 4root root 5Oct 21 20:55etc

[0149] drwx-------8root root 7Oct 21 20:59root

[0150] drwxr-xr-x 2root root 1Aug 5 10:26run

[0151] drwxr-xr-x 3root root 1Dec 16 2022usr

[0152] drwxr-xr-x 3root root 1Nov 30 2022var

[0153] drwxr-xr-x 3root root 2Oct 21 21:00worksplace

[0154] In one embodiment, the resource configuration of the new container with the same image may be different from the resource configuration of the target container.

[0155] Specifically, when creating a container, corresponding resources are allocated to meet different development requirements. In this embodiment, the newly created container with the same image and the target container share the same image. The configuration of the new container with the same image can be the same as or different from that of the target container, depending on the actual scenario requirements. This application does not impose any restrictions on this.

[0156] For example, if a user requests a highly configured machine for a container but doesn't use it for a period of time, resources will be occupied and wasted. When resources are tight, other users may not be able to request the resources they need, reducing development efficiency. Alternatively, the resources configured for the current container may be too low to meet current development needs, necessitating an improvement in the container configuration. Alternatively, a machine may crash and become unbootable during development, resulting in data loss and requiring a disaster recovery solution.

[0157] Based on the above requirements, the new container with the same image can have a lower configuration than the target container. Alternatively, the new container with the same image can have a higher configuration than the target container. Alternatively, the new container with the same image can have the same configuration as the target container.

[0158] For example, if a container originally required GPU resources for algorithm debugging, it can be deleted when idle and a new container requiring only two CPU cores can be created. The temporary environment or incremental data of the new container with the same image can then be restored based on the incremental data. The freed-up GPU resources can be applied to other scenarios, ensuring full resource utilization and avoiding waste.

[0159] For example, when processing data, a container with a 90-core CPU is required. When idle, the 90-core CPU container can be converted to a 2-core CPU container to release 88-core CPU resources, avoiding resource waste.

[0160] This embodiment can change the resource configuration of the new container with the same image as the demand when creating a new container to meet new development needs, and realizes the protection and recovery of the incremental content of the container without wasting resources and ensuring full utilization of resources.

[0161] In one embodiment, step S100 parses the writable layer of the target container to obtain and store incremental data information of the target container, including:

[0162] Regularly parse the writable layer of the target container to obtain and store the incremental data information of the target container;

[0163] and / or,

[0164] Before the target container is processed or if a backup instruction is received, the writable layer of the target container is parsed to obtain and store the incremental data information of the target container.

[0165] Specifically, in order to prevent unexpected accidents such as downtime and inability to start, which may result in the loss of the temporary environment or incremental data of the original container, this embodiment will regularly or periodically parse the writable layer of the target container, obtain and store the incremental data information of the target container, and each regular analysis is equivalent to updating the stored incremental data information to keep the incremental data information updated and synchronized. For example, the writable layer of the target container is parsed once every 1 or 2 hours, and the corresponding incremental data information is updated. In addition, regular backup storage can not only prevent the risk of downtime, because most files have been backed up regularly, it can also speed up the backup when pausing, deleting or migrating containers, shortening the backup time.

[0166] Of course, this application does not impose any limitation on the regular time interval.

[0167] This embodiment can also parse the writable layer of the target container before the target container is paused, deleted, migrated, or processed, or parse the writable layer of the target container under the user's backup instruction to obtain and store the latest incremental data information before the target container is processed, thereby ensuring the synchronous update of the incremental data information.

[0168] This embodiment can update incremental data information in a timely manner by regularly and promptly parsing the writable layer under specific needs, ensuring synchronization of incremental data information with the incremental data of the original container, and thus ensuring complete replication, restoration and retention of incremental content or temporary environments when restoring the container.

[0169] refer to Figure 7 In one embodiment of the present application, a container recovery device is further provided, the container recovery device comprising:

[0170] The parsing and backup module 100 is used to parse the writable layer of the target container, obtain and store the incremental data information of the target container;

[0171] A new container module 200 is used to create a new container with the same image as the target container based on the original image used by the target container;

[0172] The container modification module 300 is used to read the incremental data information and perform corresponding incremental operations on the new container of the same image according to the read content.

[0173] In one embodiment, the parsing and backup module 100 includes:

[0174] Parsing module, used to parse the writable layer of the target container;

[0175] A first storage module is used to store the newly added file if a new file is parsed;

[0176] The second storage module is configured to obtain and store the first path corresponding to the deleted file if a deleted file is parsed;

[0177] The third storage module is used to obtain and store the second path and replacement file corresponding to the modified file if a modified file is parsed.

[0178] In one embodiment, the container modification module 300 includes:

[0179] A deletion module, configured to delete an old file with the file path in a container file of a new container of the same image if the read content is a file path, wherein the file path is a first path corresponding to the deleted file or a second path corresponding to the modified file;

[0180] The copy and add module is used to add the file content to the container file of the new container of the same image if the read content is file content, wherein the file content is a new file or a replacement file.

[0181] In one embodiment, the parsing module is specifically configured to:

[0182] Determine the parsed character device file as a deleted file;

[0183] Determine the parsed Opaque file attribute folder as a modified file;

[0184] Other parsed files are determined to be newly added files, where other files are regular files except character device files and Opaque file attribute folders.

[0185] In one embodiment, the deletion module includes:

[0186] The first deletion module is used to delete the old file corresponding to the path of the Opaque file attribute folder in the view layer of the new container of the same image if the file path is the path of the Opaque file attribute folder;

[0187] The second deletion module is configured to delete the old file corresponding to the path of the character device file in the view layer of the new container of the same image if the file path is the path of the character device file.

[0188] In one embodiment, the resource configuration of the new container with the same image may be different from the resource configuration of the target container.

[0189] In one embodiment, the parsing and backup module 100 includes:

[0190] The first parsing and backup module is used to regularly parse the writable layer of the target container to obtain and store incremental data information of the target container;

[0191] and / or,

[0192] The second parsing and backup module is configured to parse the writable layer of the target container before the target container is processed or if a backup instruction is received, and obtain and store incremental data information of the target container.

[0193] This embodiment fully protects the temporary changes in the cloud development container environment, ensuring the continuity of user development while enabling flexible container scheduling. It supports pausing, deleting, resuming, and migrating containers.

[0194] Figure 8 7 is a structural diagram of a computer device provided in an embodiment of the present application. The computer device 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 710 (for example, one or more processors) and a memory 720, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 733 or data 732. Among them, the memory 720 and the storage medium 730 can be temporary storage or permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the computer device 700. Furthermore, the processor 710 can be configured to communicate with the storage medium 730 to execute a series of instruction operations in the storage medium 730 on the computer device 700.

[0195] The computer device 700 may further include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input and output interfaces 760, and / or one or more operating systems 731, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 8 The illustrated computer device structure does not limit the computer device and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0196] The present application also provides a computer device, which includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of the container recovery method in the above embodiments.

[0197] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to execute the steps of the container recovery method.

[0198] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0199] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store program code.

[0200] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.< / name>

Claims

1. A method for recovering a container, characterized in that: The container recovery method includes: Parsing the writable layer of the target container to obtain and store incremental data information of the target container; Creating a new container with the same image as the target container based on the original image used by the target container; The incremental data information is read, and corresponding incremental operations are performed on the new container with the same image according to the read content.

2. The container recovery method according to claim 1, characterized in that: Parsing the writable layer of the target container to obtain and store incremental data information of the target container includes: Parse the writable layer of the target container; If a new file is found during parsing, the new file is stored; If a deleted file is parsed, obtaining and storing the first path corresponding to the deleted file; If a modified file is parsed, the second path and the replacement file corresponding to the modified file are obtained and stored.

3. The container recovery method according to claim 2, characterized in that: The step of reading the incremental data information and performing corresponding incremental operations on the new container with the same image according to the read content includes: If the read content is a file path, deleting the old file of the file path in the container file of the new container with the same image, wherein the file path is the first path corresponding to the deleted file or the second path corresponding to the modified file; If the read content is file content, the file content is added to the container file of the new container with the same image, wherein the file content is a newly added file or a replaced file.

4. The container recovery method according to claim 2, characterized in that: Parsing the writable layer of the target container includes: Determine the parsed character device file as a deleted file; Determine the parsed Opaque file attribute folder as a modified file; The other files parsed are determined to be newly added files, wherein the other files are regular files except the character device files and the Opaque file attribute folder.

5. The container recovery method according to claim 3, characterized in that: Deleting the old file at the file path in the container file of the new container with the same image includes: If the file path is the path of the Opaque file attribute folder, then delete the old file corresponding to the path of the Opaque file attribute folder in the view layer of the new container of the same image; If the file path is a path of a character device file, then the old file corresponding to the path of the character device file in the view layer of the new container of the same image is deleted.

6. The container recovery method according to claim 1, characterized in that: The resource configuration of the new container with the same image may be different from the resource configuration of the target container.

7. The container recovery method according to any one of claims 1 to 6, characterized in that: Parsing the writable layer of the target container to obtain and store incremental data information of the target container includes: Regularly parse the writable layer of the target container to obtain and store incremental data information of the target container; and / or, Before the target container is processed or if a backup instruction is received, the writable layer of the target container is parsed to obtain and store incremental data information of the target container.

8. A container recovery device, characterized in that: The recovery device of the container comprises: A parsing and backup module is used to parse the writable layer of the target container, obtain and store the incremental data information of the target container; A new container module is used to create a new container with the same image as the target container based on the original image used by the target container; The container modification module is used to read the incremental data information and perform corresponding incremental operations on the new container with the same image according to the read content.

9. A computer device, characterized in that: The computer device includes: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the computer device to execute the container recovery method according to any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the container recovery method according to any one of claims 1 to 7 is implemented.

Images