Multi-container stacking method based on gateway

Abstract

The invention relates to a container design technology of embedded gateway equipment, discloses a gateway-based multi-container stacking method, and solves the problems of high repeated space occupation and long container creation time when multiple containers are created by adopting a traditional LXC technology in gateway equipment. According to the scheme, the method comprises the steps that S1, a base container file system mirror image is manufactured, specifically, a minimum LXC container suitable for gateway equipment is created, a rootfs catalogue of the container is extracted and packaged, and a squashfs read-only file system format is adopted to construct the base container mirror image; s2, integration and mounting of a base container file system: splitting a framework partition from an existing partition of gateway equipment firmware, and writing a base container mirror image into a specified offset address of the partition; after the gateway system is started, mounting the framework partition to a preset directory, so that the rootfs directory of the base container can be accessed; and S3, creating an application container based on a stacking technology: starting an OverlayFS function of a Linux kernel of the gateway, rewriting container creation logic of the LXC, and configuring a hierarchical storage path of the OverlayFS when the application container is created.

Description

TECHNICAL FIELD

[0001] The present application relates to a container design technology of an embedded gateway device, and in particular to a gateway-based multi-container stacking method. BACKGROUND

[0002] With the rapid development of the Internet of Things (IoT) and the smart home industry, gateway devices (such as PON gateways, smart routers, etc.) as the key hub connecting terminal devices and external networks, the edge computing tasks they carry are increasingly heavy. From smart device control, local data storage and analysis, to security monitoring, scene linkage, and other diversified smart services, a large number of edge applications need to be deployed on gateway devices to achieve low-latency, high-reliability local service response.

[0003] Traditional gateway embedded systems mostly use static compilation or firmware customization development mode, resulting in rigidity of application deployment and complexity of maintenance. When new functions or updated applications are needed, the entire device firmware needs to be recompiled and upgraded, which is not only tedious, time-consuming and laborious, but also may cause device failure due to firmware compatibility problems.

[0004] To solve the rigidity problem of traditional development mode, the industry has gradually introduced container technology into embedded gateway devices, among which LXC (Linux Containers) has become the mainstream choice due to its lightweight and high isolation advantages. LXC directly reuses the cgroups (control groups) and namespaces (namespaces) technologies of the Linux kernel, without additional virtualization layers, and can achieve application isolation running with lower resource consumption, effectively reducing the coupling degree of application deployment. However, in resource-constrained gateway devices (limited memory and storage capacity), LXC technology still faces some problems: (1) High repeated space occupation: When creating a new container, traditional LXC needs to allocate a complete root file system (rootfs) for each container. When multiple edge applications need to be deployed on a gateway, the root file systems of multiple containers will repeatedly occupy storage resources, leading to rapid depletion of gateway storage, limiting the number of deployable applications, and reducing the utilization efficiency of storage resources.

[0005] (2) Long time-consuming for container creation: When creating a container, traditional LXC needs to copy the complete root file system, configure the network and resource limits, and a series of processes, which usually takes 6-7 seconds. In scenarios where multiple applications need to be deployed quickly, the long container creation time will cause service online delay, which cannot meet the real-time deployment requirements.

[0006] In addition, the independent file systems of multiple containers not only increase the storage burden, but also cause the complexity of application updating and maintenance to increase, the base dependency of each container needs to be updated separately, and version inconsistency problems are prone to occur; meanwhile, the writable feature of the container file system can cause malicious programs to tamper with the base file, causing system failure or security risks, and the traditional scheme lacks an effective base environment protection mechanism.

[0007] Therefore, developing a container technical solution that adapts to the resource constraints of gateway devices, effectively reduces the container space occupation, improves the creation efficiency, and at the same time considers the maintenance convenience and security, has become a key requirement for promoting the large-scale deployment of Internet of Things gateway devices and multi-edge applications. SUMMARY

[0008] The technical problem to be solved by the present application is to provide a multi-container stacking method based on a gateway, which solves the problems of high repeated space occupation and long container creation time when creating multiple containers in a gateway device using traditional LXC technology.

[0009] The technical solution adopted by the present application to solve the above technical problem is: A multi-container stacking method based on a gateway, comprising the following steps: S1. Making a base container file system image: creating a minimized LXC container suitable for a gateway device, extracting the rootfs directory of the container and packaging it, and using the squashfs read-only file system format to build a base container image; S2. Integration and mounting of the base container file system: splitting a framework partition from the existing partition of the gateway device firmware, writing the base container image to the specified offset address of the partition; after the gateway system starts, mounting the framework partition to a preset directory, so that the rootfs directory of the base container can be accessed; S3. Creating an application container based on stacking technology: enabling the OverlayFS function of the Linux kernel of the gateway, rewriting the container creation logic of LXC, and configuring the layered storage path of OverlayFS when creating an application container, wherein the lower directory points to the rootfs directory of the base container, and the upper directory is an application container exclusive writable directory.

[0010] Further, in step S1, the minimized LXC container is created by the lxc-create command in combination with the lxc-busybox template, and the packaging format uses tar.gz.

[0011] Further, in step S2, the size of the framework partition is set according to the size of the base container image and the subsequent expansion requirements, and the partition configuration is completed through the dts file of the gateway system.

[0012] Further, in step S3, the path configuration of the OverlayFS is achieved by modifying the lxc.rootfs.path field in the LXC container configuration file.

[0013] Further, in step S3, the rewritten container creation logic automatically creates the lower directory and the upper directory required by the OverlayFS when executed, and generates the corresponding container configuration file, without the need to repeatedly copy the base file system.

[0014] Further, in step S1, the constructed base container image has a read-only attribute and cannot be modified by external forces during the running life cycle of the gateway system, and can only be reset by restoring the factory settings.

[0015] The beneficial effects of the present application are: (1) Reduce storage occupation and improve gateway resource utilization: The present application breaks through the limitation of traditional LXC container "each container independently occupies a complete base file system", and reuses the base file system (rootfs) through a unified read-only base container. All application containers only need to store their own application incremental data, without the need to repeatedly occupy the base file system space. In the traditional scheme, each container needs to occupy about 1MB of base file storage, while in the present scheme, the storage increment of multiple container deployment only comes from application exclusive data, greatly reducing the storage pressure of the gateway, enabling resource-constrained gateway devices to deploy more edge applications, and improving the utilization efficiency of storage resources.

[0016] (2) Shorten container creation time and improve deployment efficiency: Traditional LXC container creation requires complete copying of the base file system, configuring resource limits, etc., and the time consumption is usually 6-7 seconds, which is difficult to meet the rapid deployment requirements. The present application rewrites the LXC creation logic and uses the OverlayFS layering storage feature. When creating an application container, only the path of the lower read-only base container and the upper writable directory need to be configured, without the need to copy the base file system, so that the container creation time is shortened to less than 1 second, and the deployment efficiency is greatly improved.

[0017] (3) Strengthen system security and stability, and reduce maintenance cost The base container uses the squashfs read-only file system format and cannot be modified by external forces during the entire life cycle of the gateway, which avoids the risk of malicious tampering or accidental damage to the base file system. Even if the application container runs abnormally, only the application container needs to be reset or the factory settings need to be restored to restore the original state of the base container, without the need to recompile the firmware, which greatly reduces the system maintenance difficulty and fault recovery cost, and ensures the business stability of the gateway device. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1A multi-container stack system framework diagram in an embodiment of the present application.

[0019] Figure 2 A gateway-based multi-container stack method flow chart in an embodiment of the present application.

[0020] Figure 3 A base container making and mounting flow chart in an embodiment of the present application.

[0021] Figure 4 A container creation and starting flow chart in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The present application aims to provide a gateway-based multi-container stack method to solve the problems of high repeated space occupation and long container creation time when using traditional LXC technology to create multiple containers in a gateway device. The core idea is: through the innovative architecture of read-only base container reuse + OverlayFS layered stacking, the core pain points of high space occupation and slow creation speed in the multi-container deployment of the gateway device are solved, and the lightweight and efficient deployment of the container is realized.

[0023] Specifically, the technical means adopted by the present application based on the above core idea include: (1) Constructing a read-only base container, extracting the minimum base file system required by the gateway device, and encapsulating it into an image in squashfs read-only format to ensure that the base file cannot be tampered with and can be reused, avoiding repeated occupation of base file storage resources by multiple containers.

[0024] (2) Integrating the base container image into the gateway firmware exclusive partition, and automatically mounting after system startup, so that all application containers can directly access the shared base file system without the need for separate copying.

[0025] (3) Rewriting the container creation logic based on OverlayFS layered storage technology, and the application container only needs to configure the lower layer read-only base container path and the upper layer exclusive writable directory, without the need to copy the base file during creation, only storing the application incremental data, which greatly shortens the creation time, and at the same time guarantees the system security and stability through the read-only base container.

[0026] The scheme of the present application will be further described below in combination with the drawings and embodiments.

[0027] The embodiment provides a gateway-based multi-container stacking method. In terms of a hardware basis, the embodiment is implemented based on an embedded Linux gateway device, without adding special hardware, and only needs to optimize container creation logic, a file system and partition configuration at a software level. The embedded Linux gateway device needs to support OverlayFS function, support partition management and file system mounting, and support LXC container tool chain compilation, firmware image building and dts file configuration on a development environment.

[0028] Figure 1 The system architecture of the multi-container stacking in the embodiment is shown in FIG. 1. Figure 1 The system architecture includes, from top to bottom, a container instance layer, an application and configuration layer, a stacking layer and a basic support layer. The container instance layer includes a plurality of independent application container instances created by the embodiment. The containers are functional carriers for actual business scenarios, can respectively carry different edge applications such as intelligent control, data processing and security monitoring, are isolated and independently operated, and are the final application output layer of the architecture.

[0029] In the application and configuration layer, each combination of configuration and application corresponds to one container instance in the upper layer. In the application and configuration layer, The "configuration" specifically refers to the individualized configuration of each application container; and the "application" specifically refers to a specific business plug-in or application program carried by each container, is the functional core of the container instance, and is the only source of the increment of the container space.

[0030] The stacking layer is the core technical support layer of the architecture. The layer realizes the sharing and isolation of the basic file system of the multi-container through the mechanism of the lower layer read-only + upper layer writable of OverlayFS. The core role is to connect the upper application container and the bottom base container, take the read-only file system of the base container as the lower path, and take the writable space of each container as the upper path, so that all modifications of the upper container are only written into the upper path, the basic resources of the lower path are reused, and the space is avoided from being repeatedly occupied.

[0031] The basic support layer is the basic file system support of the entire architecture. The layer is derived from the minimized rootfs directory extracted in the base container production process, contains the minimized basic file system resources required for the operation of the gateway device, and cannot be modified by external forces in the whole life cycle of the gateway due to the adoption of the squashfs read-only file system. All upper container instances reuse the rootfs resources of the layer through OverlayFS, without creating independent basic file systems.

[0032] The gateway-based multi-container stacking method provided in the embodiment is shown in FIG. 2. Figure 2As shown, the implementation process is as follows: S1. Base container file system image making: In this step, the base container is a minimized LXC container suitable for the gateway device, which only contains the basic files required for running the embedded Linux system (such as kernel dependent libraries, basic commands, file system directory structure), and does not contain any application layer functions, ensuring the lightweight and universality of the base container.

[0033] In a specific implementation, according to the operation mode of the LXC container, the lxc-create -n Base -fdefault.conf -t lxc-busybox command is used to create a container named Base on the gateway device. After successful creation, the container is packaged into Base.tar.gz using the tar command.

[0034] Subsequently, the base container image is made: the base container should not be modified by external forces in all life cycles of the gateway system, so when considering the file system format, the squashfs read-only file system is used to image build the aforementioned packaged file. Finally, a small image file integrating the base container minimal file system is generated, named framework.bin.

[0035] It should be noted that only the rootfs of the base container file system is needed in the image file, so the rootfs directory contained in Base.tar.gz needs to be extracted for image building. The rootfs directory is extracted instead of the complete container package because the rootfs already contains all the basic files required for running the container, and the redundant contents such as container configuration files and logs are removed, further reducing the image size.

[0036] And the read-only image format is used to ensure the integrity and security of the base container, to avoid the base files being tampered with, causing all application containers dependent on the base container to run abnormally, and to simplify the fault recovery process. That is, resetting the base container can be achieved by restoring the factory settings.

[0037] S2. Base container file system integration and mounting: Since the gateway device firmware is an integrated firmware with multiple partitions, such as bootloader, config, kernel, rootfs, other, etc., these contents need to be burned into the flash, so that the partition contents can be read from the fixed offset and run after the system starts. Therefore, in the existing partition, a partition is split out to provide the base container, named framework partition, and the partition size is temporarily set to 10MB according to the size of the base container and the possible content expansion. The partition is configured in the dts file in the linux system.

[0038] When building the gateway firmware image, the base container image framework.bin is written to the specified offset address of the framework partition to generate a complete system firmware containing the base container; after the gateway device is burned with the firmware and started, the system automatically mounts the framework partition to the preset local directory (such as / mnt / framework), so that the rootfs directory of the base container can be accessed by the gateway system and the subsequent application container.

[0039] The base container making and mounting process is described in Figure 3 .

[0040] S3. Create an application container based on the stacking technology: In this step, first confirm that the Linux kernel of the gateway device has enabled the OverlayFS function (enabled through the kernel configuration item), which provides bottom-layer support for layered storage; then rewrite the container creation logic of LXC (can be implemented by customizing the function interface or replacing the lxc-create command of the LXC open source library), and add the OverlayFS path configuration function.

[0041] When an application container needs to be created, the rewritten lxc-create command will automatically perform the following operations: Create a writable upper directory (upper path) dedicated to the application container, which is used to store application incremental data (such as application programs, configuration files, and running logs); Set the base container rootfs directory as a read-only lower directory (lower path); Modify the configuration file of the application container, and configure the lxc.rootfs.path section as "overlayfs:[lower path]:[upper path]" to realize OverlayFS layered mounting; after the configuration is completed, the application container can be started directly without copying the base container basic file system.

[0042] The container creation and startup process is described in Figure 4 .

[0043] In the present embodiment, the container creation logic of LXC is rewritten instead of developing a brand new container tool, in order to be compatible with the core functions of the existing LXC container, such as isolation and resource limitation, to reduce the development cost, and to realize seamless integration of the stacking function.

[0044] The application container only creates a writable upper directory, and reuses the read-only lower directory of the base container, which can completely solve the problem of repeated space occupation caused by the traditional LXC container independently occupying a complete base file system for each container, so that the storage increment of the application container only comes from the application itself, and the storage resource utilization of the gateway is greatly improved.

[0045] All modifications of the application container are stored in a dedicated writable directory, which is isolated from the base container and other application containers, to ensure that each application container runs independently and does not interfere with each other; at the same time, if the application container runs abnormally, it only needs to delete its dedicated writable directory to reset, without affecting the base container and other application containers, simplifying the maintenance process.

[0046] Although the embodiments of the present application have been described, it can be understood by those skilled in the art that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and spirits of the present application, and all of them are within the protection scope of the present application.

Claims

1. A gateway-based multi-container stacking method, characterized in that, Includes the following steps: S1. Create a base container file system image: Create a minimal LXC container suitable for the gateway device, extract the rootfs directory of the container and package it, and build the base container image using the squashfs read-only file system format; S2. Integration and mounting of the base container file system: Split a framework partition from the existing partitions of the gateway device firmware, and write the base container image to the specified offset address of the partition; After the gateway system starts, mount the framework partition to the preset directory so that the rootfs directory of the base container can be accessed. S3. Create application containers based on stacking technology: Enable the OverlayFS function of the gateway's Linux kernel, rewrite the LXC container creation logic, and configure the hierarchical storage path of OverlayFS when creating application containers. The lower-level directory points to the rootfs directory of the base container, and the upper-level directory is a writable directory dedicated to the application container.

2. The gateway-based multi-container stacking method as described in claim 1, characterized in that, In step S1, the minimized LXC container is created using the lxc-create command in conjunction with the lxc-busybox template, and the packaging format is tar.gz.

3. The gateway-based multi-container stacking method as described in claim 1, characterized in that, In step S2, the size of the framework partition is set according to the base container image size and subsequent expansion requirements, and the partition configuration is completed through the gateway system's DTS file.

4. The gateway-based multi-container stacking method as described in claim 1, characterized in that, In step S3, the path configuration of the OverlayFS is achieved by modifying the lxc.rootfs.path field in the LXC container configuration file.

5. The gateway-based multi-container stacking method as described in claim 1, characterized in that, In step S3, the rewritten container creation logic automatically creates the lower and upper directories required by OverlayFS and generates the corresponding container configuration files when executed, without having to repeatedly copy the basic file system.

6. A gateway-based multi-container stacking method as described in any one of claims 1 to 5, characterized in that, In step S1, the constructed base container image is read-only and cannot be modified by external forces during the gateway system's lifecycle. It can only be reset by restoring factory settings.

Images