Multi-tenant Data Persistence Method, Apparatus, Storage Medium and Computer Device
By mounting the S3 bucket on the compute node of the K8S cluster and creating an isolated user directory for the tenants of the Zeppelin platform, the problem of data persistence in the Zeppelin service is solved, and data security, reliability and isolation are achieved.
Patent Information
- Application Number
- CN202111432747.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-29
- Publication Date
- 2025-06-24
- Estimated Expiration
- 2041-11-29
AI Technical Summary
The Zeppelin service deployed in K8S does not have data persistence by default, resulting in the user-written notebook code and uploaded data being at risk of loss when the container exits abnormally or the service is stopped.
Mount the S3 bucket in each compute node of the K8S cluster, and create a separate user directory for the tenant when using the Zeppelin platform, located in the mounted S3 bucket, ensuring that the user directories of different tenants are isolated from each other.
The persistence of user data on Zeppelin platform is achieved, data loss is avoided, and data security and reliability are ensured by isolating data directories of different tenants.
Smart Images

Figure CN114168156B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technologies. Specifically, this application relates to a multi-tenant data persistence method, a multi-tenant data persistence device, a computer-readable storage medium, and a computer device. Background Art
[0002] Most Internet enterprises provide products similar to Notebook. Such products use an interactive method for data analysis, data modeling, and data visualization. Zeppelin Notebook is one of the widely used big data analysis, modeling, and visualization products.
[0003] Based on Zeppelin, a multi-tenant notebook application deployed in K8S and connected to a big data platform can be custom-built. However, the services deployed in K8S do not have the data persistence function by default. When the Zeppelin container exits abnormally or the service stops, the notebook code written by users and the uploaded data will face the risk of loss.
[0004] Currently, there are several ways to solve such problems. For example, data persistence can be achieved based on the method of local disk mounting. However, this method requires programmatic solutions to the problems of multi-user data volume creation and mounting, and there are also problems such as code and data chaos caused by multiple people editing the same data when mounting the same data volume directory in a multi-tenant scenario. In addition, just mounting the host machine disk directory will cause problems with inconsistent mounted files when the container restarts and stops and is scheduled to different host machines. For another example, the distributed file system can also be manually mounted on all computing nodes in the cluster. However, in this method, all computing nodes need to manually install mounting software and set configuration files on the host machine, and also need to deal with operation and maintenance problems such as machine restart and cluster expansion. Summary of the Invention
[0005] To at least solve one of the above technical defects, this application provides a multi-tenant data persistence method, a multi-tenant data persistence device, a computer-readable storage medium, and a computer device with the following technical solutions.
[0006] An embodiment of the present application provides a multi-tenant data persistence method for the Zeppelin platform, where the Zeppelin platform is deployed in the host of the K8S cluster. The multi-tenant data persistence method includes: mounting an S3 bucket in each computing node of the K8S cluster; when the tenant uses the Zeppelin platform, creating a separate user directory for the tenant in the computing node corresponding to the tenant, where the user directories of multiple tenants corresponding to the same computing node are located in the S3 bucket mounted by the computing node, and the user directories of different tenants in the same S3 bucket are isolated from each other.
[0007] In some embodiments, mounting the S3 bucket in each computing node of the K8S cluster includes: imaging software related to the mounting of the S3 bucket; and mounting the S3 bucket into each computing node by the DaemonSet in the K8S cluster based on the imaged related software.
[0008] In some embodiments, the software related to the mounting of the S3 bucket includes a startup script and a dockerfile.
[0009] In some embodiments, mounting the S3 bucket into each computing node by the DaemonSet in the K8S cluster based on the imaged related software includes: creating a ConfigMap object that contains the basic configuration information for mounting the S3 bucket; creating a DaemonSet object that mounts the S3 bucket into the target directory of each computing node based on the startup script, the dockerfile, and the ConfigMap object.
[0010] In some embodiments, creating a separate user directory for the tenant in the computing node corresponding to the tenant when the tenant uses the Zeppelin platform includes: when any tenant uses the Zeppelin platform, determining whether the tenant wants to save data; if so, creating a separate user directory for the tenant in the computing node corresponding to the tenant.
[0011] In some embodiments, creating a separate user directory for a tenant in the computing node corresponding to the tenant includes: specifying the mounting type as Host; determining the target directory in the target computing node to be mounted according to the Host; creating an initialization container, and mounting the target directory in the target computing node to be mounted in the initialization container; creating the user directory for the tenant in the target directory; creating a Zeppelin service container, and mounting the user directory corresponding to the tenant in the Zeppelin service container.
[0012] In some embodiments, creating a separate user directory for a tenant in the computing node corresponding to the tenant further includes: after creating the user directory for the tenant in the target directory, setting operation permissions for the user directory.
[0013] An embodiment of the present application further provides a multi-tenant data persistence device for a Zeppelin platform, where the Zeppelin platform is deployed in a host machine of a K8S cluster. The multi-tenant data persistence device includes a mounting module and an operation module. The mounting module is used to mount an S3 bucket in each computing node of the K8S cluster. The persistence operation module is used to create a separate user directory for the tenant in the computing node corresponding to the tenant when the tenant uses the Zeppelin platform, where user directories of multiple tenants corresponding to the same computing node are located in the S3 bucket mounted by the computing node, and user directories of different tenants in the same S3 bucket are isolated from each other.
[0014] An embodiment of the present application further provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the multi-tenant data persistence method according to any of the above embodiments.
[0015] An embodiment of the present application further provides a computer device. The computer includes one or more processors; a memory; one or more computer programs, where the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and the one or more computer programs are configured to: execute the multi-tenant data persistence method according to any of the above embodiments.
[0016] Compared with the prior art, the present application has the following beneficial effects:
[0017] The multi-tenant data persistence method, multi-tenant data persistence device, computer-readable storage medium, and computer device of the embodiments of the present application use the Linux Fuse technology in combination with Docker and k8s DaemonSet to implement the mounting of the S3 file directory by the computing nodes in the K8S cluster. The Zeppelin services created by multiple tenants achieve the persistence of the code written by users and the uploaded data through the file directory mounted on the computing nodes.
[0018] Additional aspects and advantages of the present application will be given in part in the following description, and these will become obvious from the following description or be understood through the practice of the present application. Brief Description of the Drawings
[0019] The above and / or additional aspects and advantages of the present application will become obvious and easy to understand from the following description of the embodiments in conjunction with the drawings, where:
[0020] Figure 1 It is a flowchart of the data persistence method for some embodiments of the present application;
[0021] Figure 2 It is a schematic structural diagram of the data persistence device for some embodiments of the present application;
[0022] Figure 3 It is a flowchart of the data persistence method for some embodiments of the present application;
[0023] Figure 4 It is a flowchart of the data persistence method for some embodiments of the present application;
[0024] Figure 5 It is a schematic diagram of the scenario of the data persistence method for some embodiments of the present application;
[0025] Figure 6 It is a flowchart of the data persistence method for some embodiments of the present application;
[0026] Figure 7 It is a flowchart of the data persistence method for some embodiments of the present application;
[0027] Figure 8 It is a schematic diagram of the communication between the computer-readable storage medium and the processor for some embodiments of the present application;
[0028] Figure 9 It is a schematic diagram of the computer device for some embodiments of the present application. Detailed Embodiments
[0029] Embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where like or similar reference numerals denote like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present application, and should not be construed as a limitation to the present application.
[0030] Those skilled in the art of the present technology can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the", and "said" used herein may also include the plural forms. It should be further understood that the term "including" used in the specification of the present application means the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or their groups. The phrase "and / or" used herein includes all or any unit and all combinations of one or more related listed items.
[0031] Those skilled in the art of the present technology can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those of ordinary skill in the art to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as here.
[0032] Please refer to Figure 1 , an embodiment of the present application provides a multi-tenant data persistence method. This method is used for the Zeppelin platform, and the Zeppelin platform is deployed in the host of the K8S cluster. The multi-tenant data persistence method includes:
[0033] 01: Mount the S3 bucket in each computing node of the K8S cluster;
[0034] 02: When a tenant uses the Zeppelin platform, create a separate user directory for the tenant in the computing node corresponding to the tenant. Among them, the user directories of multiple tenants corresponding to the same computing node are located in the S3 bucket mounted by the computing node, and the user directories of different tenants in the same S3 bucket are isolated from each other.
[0035] Please refer to Figure 2 , an embodiment of the present application also provides a multi-tenant data persistence device 10. The multi-tenant data persistence method of the embodiment of the present application can be implemented by the multi-tenant data persistence device 10 of the embodiment of the present application. Among them, the multi-tenant data persistence device 10 includes a mounting module 11 and an operation module 12. Step 01 can be implemented by the mounting module 11. Step 02 can be implemented by the operation module 12.
[0036] That is to say, the mounting module 11 can be used to mount the S3 bucket in each computing node of the K8S cluster. The operation module 12 can be used to create a separate user directory for the tenant in the computing node corresponding to the tenant when the tenant uses the Zeppelin platform. Among them, the user directories of multiple tenants corresponding to the same computing node are located in the S3 bucket mounted by the computing node, and the user directories of different tenants in the same S3 bucket are isolated from each other.
[0037] Among them, K8S (i.e., Kubernetes) is an open-source container cluster management system developed by Google. Based on Docker technology, k8s provides a series of complete functions such as deployment and operation, resource scheduling, service discovery, and dynamic scaling for containerized applications, which can improve the convenience of large-scale container cluster management. The Zeppelin platform is an application deployed in the K8S cluster, and the functions of this application are: querying big data in a visual way through a Web interface, performing big data calculations, and providing rich graphical displays such as bar charts and pie charts for the operation results, supporting SQL statement query of data, supporting the writing of code scripts in multiple programming languages (Java, Python, Scala, Shell, etc.), and supporting multiple data processing engines to process data. Thus, it is convenient for users to create interactive documents and charts using multiple programming languages. As an example, the Zeppelin platform can be, for example, Zeppelin Notebook.
[0038] Currently, the Zeppelin service deployed in K8S does not have the data persistence function by default. When the Zeppelin service container exits abnormally or the service stops, the relevant operation data of the tenant (which can also be understood as the user or project group) on the Zeppelin platform, such as the written notebook code and uploaded data, will face the risk of loss.
[0039] In the multi-tenant data persistence method and the multi-tenant data persistence device 10 of the embodiment of the present application, first, the S3 bucket is mounted in each computing node of the K8S cluster. Subsequently, when the tenant uses the Zeppelin platform, a separate user directory can be created for the tenant in the S3 bucket of the corresponding computing node, and the user directories of different tenants can be isolated from each other. Thus, the persistent storage and recording of the data related to the user's operations are realized, and the data of different tenants will not affect each other.
[0040] Please refer to Figure 3 , in some embodiments, step 01 of mounting the S3 bucket in each computing node of the K8S cluster includes:
[0041] 011: Software related to mirror creation and mounting of S3 buckets;
[0042] 012: Software related to mirror creation by DaemonSet in the K8S cluster, which mounts the S3 bucket to each computing node.
[0043] Please refer to Figure 2 , in some embodiments, both step 011 and step 012 can be implemented by the mounting module 11. That is to say, the mounting module 11 can be further used for software related to mirror creation and mounting of S3 buckets, and the software related to mirror creation by DaemonSet in the K8S cluster mounts the S3 bucket to each computing node.
[0044] In some embodiments, the software related to the mounting of the S3 bucket includes a startup script and a dockerfile.
[0045] Specifically, the startup script is mainly used to obtain the authentication parameters and configuration parameters related to the s3 bucket. Among them, the authentication parameters include, for example, the access key, that is, the secret key Secret key and the key Key. The authentication parameters can be used to ensure the security of accessing the service, and the S3 storage service uses the above access key to identify which tenant is accessing which S3 bucket. The configuration parameters include, for example, the number of retries when a network problem occurs during mounting, whether to allow other tenants to operate on files, etc. The code executed by the startup script is, for example: / usr / bin / s3fs${BUCKET}${MNT_POINT}-d-d-f-o.endpoint=allow_other,retries=5.
[0046] A dockerfile is a script interpreted by the Docker program and is a text file used to build an image. Its text content contains instructions and descriptions for building the image one by one. In addition to specifying the above startup script, the dockerfile also includes the minimum running alpine image layer, system dependent packages, and the s3fs (i.e., S3 fuse) program for performing the S3 bucket file mounting operation. The finally produced image will be pushed to the image repository.
[0047] In the multi-tenant data persistence method and the multi-tenant data persistence device 10 of the embodiments of the present application, the software and dependencies required for S3Fuse mounting are encapsulated by docker images, avoiding modification of the host environment and reducing the installation and operation and maintenance costs at the same time.
[0048] Please refer to Figure 4, in some embodiments, step 012 is to mount the S3 bucket into each computing node by the software related to image making by the DaemonSet in the K8S cluster, including:
[0049] 0121: Create a ConfigMap object, which contains the basic configuration information for mounting the S3 bucket;
[0050] 0122: Create a DaemonSet object, which mounts the S3 bucket into the target directory of each computing node based on the startup script, dockerfile, and ConfigMap object.
[0051] Please refer to Figure 2 , in some embodiments, both step 0121 and step 0122 can be implemented by the mounting module 11. That is to say, the mounting module 11 can further be used to create a ConfigMap object, which contains the basic configuration information for mounting the S3 bucket. The mounting module 11 can further be used to create a DaemonSet object, which mounts the S3 bucket into the target directory of each computing node based on the startup script, dockerfile, and ConfigMap object.
[0052] Specifically, first create a ConfigMap object for the K8S cluster. The ConfigMap object is an API object in the K8S cluster, equivalent to a collection of many configuration parameters or environmental parameters required for running programs, mainly used to decouple the static code of program operation from dynamically changing parameters and facilitate the modification of application configuration parameters. The ConfigMap object contains the basic configuration information for mounting the S3 bucket, such as the S3 bucket name, the aforementioned secret key and secret, etc. After creating the ConfigMap object, create a DaemonSet object for the K8S cluster to execute the mounting operation process. The DaemonSet object contains the software related to S3 bucket mounting in the aforementioned image making. The DaemonSet object can mount the ConfigMap object to obtain the parameters for program operation and mount the S3 bucket into the target directory of the computing node. By way of example, please refer to Figure 5 , Figure 5Both Node1 and Node2 in it are computing nodes. Each computing node corresponds to a host machine, and each host machine has a target directory: / mnt / s3fs directory. The S3 bucket encapsulates the software and dependencies required for S3 fuse mounting through the image of the dockerfile in docker, and mounts the S3 bucket to the / mnt / s3fs directory with the help of the DaemonSet object, so as to realize the mounting and use of distributed object storage under each computing node.
[0053] In the multi-tenant data persistence method and the multi-tenant data persistence device 10 of the embodiment of the present application, the software and dependencies required for S3 Fuse mounting are encapsulated by the docker image, avoiding modifying the host machine environment, and at the same time docker reduces the cost of installation and operation and maintenance.
[0054] Please refer to Figure 6 , in some embodiments, in step 02 when the tenant uses the Zeppelin platform, a separate user directory is created for the tenant in the computing node corresponding to the tenant, including:
[0055] 021: When any tenant uses the Zeppelin platform, determine whether the tenant wants to save data;
[0056] 022: If so, create a separate user directory for the tenant in the computing node corresponding to the tenant.
[0057] Please refer to again Figure 2 , in some embodiments, both step 021 and step 022 can be implemented by the operation module 12. That is to say, the operation module 12 can be used to determine whether the tenant wants to save data when any tenant uses the Zeppelin platform, and create a separate user directory for the tenant in the computing node corresponding to the tenant when the tenant wants to save data.
[0058] Specifically, during the process of the tenant using the Zeppelin platform, if the tenant needs to save the written code and uploaded data to realize data persistence, it is determined that a separate user directory needs to be created for the tenant in the computing node corresponding to the tenant. In this way, the user directory is only created for the tenants with requirements, and there is no need to create user directories for all tenants, avoiding performing unnecessary operations.
[0059] Please refer to Figure 7 , in some embodiments, step 022 creates a separate user directory for the tenant in the computing node corresponding to the tenant, including:
[0060] 0221: Specify the mounting type as Host;
[0061] 0222: Determine the target directory in the target computing node to be mounted according to the Host;
[0062] 0223: Create an initialization container and mount the target directory in the target computing node to be mounted in the initialization container;
[0063] 0224: Create a user directory for the tenant in the target directory;
[0064] 0225: Create a Zeppelin service container and mount the user directory corresponding to the tenant in the Zeppelin service container.
[0065] Please refer to Figure 2 , in some implementations, steps 0221 to 0225 can all be implemented by the operation module 12. That is to say, the operation module 12 can be used to specify the mounting type as Host and determine the target directory in the target computing node to be mounted according to the Host. The operation module 12 can also be used to create an initialization container, mount the target directory in the target computing node to be mounted in the initialization container, and create a user directory for the tenant in the target directory. The operation module 12 can also be used to create a Zeppelin service container and mount the user directory corresponding to the tenant in the Zeppelin service container.
[0066] Specifically, please refer to Figure 5 , when it is determined that the tenant needs to save data, the mounting type will be determined as Host. The container in the computing node can judge which target directory in which computing node needs to be mounted through this parameter Host. In other words, it can be judged which target directory on which host needs to be mounted through the parameter Host. Subsequently, create an initialization container and mount the target directory on the target host determined in step 0222 in this initialization container. Subsequently, create a separate user directory for the user under this target directory, and this separate user directory can be used to store the data of the current tenant. Subsequently, create a Zeppelin service container. When the Zeppelin service container mounts the target directory in the host, it will, according to different tenants, specify to mount the user directory corresponding to the current tenant under this target directory. Thus, different tenants mount the same S3 storage bucket and mount different user directories in this storage bucket to achieve isolation between the data of different tenants.
[0067] Similarly, Hive pods, Spark pods, Shell pods started through the Zeppelin service (such as Figure 5When data needs to be saved during runtime as shown, it can also be achieved by mounting the user directories of the tenants with the same S3 bucket targets mentioned above, which will not be elaborated here.
[0068] Please refer to Figure 7 , in some embodiments, step 022 creating a separate user directory for the tenant in the computing node corresponding to the tenant further includes:
[0069] 0226: After creating the user directory for the tenant in the target directory, set the operation permissions for the user directory.
[0070] Please refer to Figure 2 , in some embodiments, step 0226 can be implemented by the operation module 12. That is to say, the operation module 12 can also be used to set the operation permissions for the user directory after creating the user directory for the tenant in the target directory.
[0071] Among them, the operation permissions can include read, write and other operation permissions. By setting the operation permissions, the container can perform operations such as reading or writing data under the user directory during runtime.
[0072] In summary, the multi-tenant data persistence method and the multi-tenant data persistence device 10 of the embodiments of the present application use the linux Fuse technology combined with Docker and k8s DaemonSet to realize the mounting of the S3 file directory by the computing nodes in the K8S cluster. The Zeppelin Notebook services created by multiple tenants achieve the persistence of the code written by the tenants and the uploaded data by mounting the file directories on the computing nodes. In addition, the mounting logic of the S3 bucket is decoupled from the business logic of Zeppelin to avoid mutual influence. Furthermore, different user directories are mounted according to different tenants to achieve secure isolation of code and data. In addition, the software and dependencies required for S3 Fuse mounting are encapsulated in a docker image to avoid modifying the host environment and reduce the cost of installation and operation and maintenance. In addition, after the K8S cluster is expanded using the K8S DaemonSet object, the distributed S3 file system of the newly added nodes is automatically mounted, reducing the intervention operation of manual operation and maintenance.
[0073] The solution of the present application aims to combine the cloud-native K8S DaemonSet, containerization and introduce the distributed data storage technology S3 to solve the problems existing in the production of the Zeppelin platform, so that when users use this platform for data analysis and visualization, they do not need to pay attention to the underlying storage resources, worry about code and data loss and data privacy issues, and at the same time, system development and operation and maintenance personnel do not need to pay attention to the maintenance issues of distributed data storage when containers go wrong and the cluster scales in and out.
[0074] The content of the method embodiments of this application is applicable to the device embodiments of this application. The functions specifically implemented by the device embodiments of this application are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above methods. For specific details, please refer to the descriptions in the method embodiments and will not be elaborated here.
[0075] In addition, please refer to Figure 8 , embodiments of this application provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the multi-tenant data persistence method described in any of the above embodiments. Among them, the computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. That is, the storage device includes any medium that can store or transmit information in a readable form by a device (such as a computer, mobile phone), and can be a read-only memory, a magnetic disk, or an optical disk, etc.
[0076] The content of the method embodiments of this application is applicable to the storage medium embodiments of this application. The functions specifically implemented by the storage medium embodiments of this application are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above methods. For specific details, please refer to the descriptions in the method embodiments and will not be elaborated here.
[0077] In addition, please refer to Figure 9 , embodiments of this application also provide a computer device. The computer device described in this embodiment can be a server, a personal computer, a network device, and other devices. The computer device includes: one or more processors, a memory, and one or more computer programs, where the one or more computer programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more computer programs are configured to execute the multi-tenant data persistence method described in any of the above embodiments.
[0078] The content of the method embodiments of this application is applicable to the computer device embodiments of this application. The functions specifically implemented by the computer device embodiments of this application are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above methods. For specific details, please refer to the descriptions in the method embodiments and will not be elaborated here.
[0079] In addition, each functional unit in various embodiments of the present application may be integrated into one processing module, may exist separately as individual physical units, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0080] The above are only some embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.
Claims
1. A multi-tenant data persistence method for the Zeppelin platform, where the Zeppelin platform is deployed in the host machine of the K8S cluster, characterized in that, including: mounting the S3 bucket in each computing node of the K8S cluster; when the tenant uses the Zeppelin platform, creating a separate user directory for the tenant in the computing node corresponding to the tenant, wherein the user directories of multiple tenants corresponding to the same computing node are located in the S3 bucket mounted by the computing node, and the user directories of different tenants in the same S3 bucket are isolated from each other; when the tenant uses the Zeppelin platform, creating a separate user directory for the tenant in the computing node corresponding to the tenant includes: when any tenant uses the Zeppelin platform, determining whether the tenant wants to save data; if so, creating a separate user directory for the tenant in the computing node corresponding to the tenant, specifically including: specifying the mount type as Host; determining the target directory in the target computing node to be mounted according to the Host; creating an initialization container and mounting the target directory in the target computing node to be mounted in the initialization container; creating the user directory for the tenant in the target directory; creating a Zeppelin service container and mounting the user directory corresponding to the tenant in the Zeppelin service container.
2. The multi-tenant data persistence method according to claim 1, wherein mounting the S3 bucket in each computing node of the K8S cluster includes: making an image of the software related to the mounting of the S3 bucket; mounting the S3 bucket into each computing node by the DaemonSet in the K8S cluster based on the software made from the image.
3. The multi-tenant data persistence method according to claim 2, wherein The software related to the mounting of the S3 bucket includes a startup script and a dockerfile.
4. The multi-tenant data persistence method according to claim 3, wherein mounting the S3 bucket into each computing node by the DaemonSet in the K8S cluster based on the software made from the image includes: creating a ConfigMap object, which contains the basic configuration information for mounting the S3 bucket; creating a DaemonSet object, and mounting the S3 bucket into the target directory of each computing node based on the startup script, the dockerfile and the ConfigMap object.
5. The multi-tenant data persistence method according to claim 1, characterized in that when creating a separate user directory for the tenant in the computing node corresponding to the tenant, it further includes: after creating the user directory for the tenant in the target directory, setting the operation permissions for the user directory.
6. A multi-tenant data persistence device for the Zeppelin platform, where the Zeppelin platform is deployed in the host machine of the K8S cluster, characterized in that, including: a mounting module for mounting the S3 bucket in each computing node of the K8S cluster; An operation module, configured to create a separate user directory for a tenant in a computing node corresponding to the tenant when the tenant uses the Zeppelin platform, wherein user directories of multiple tenants corresponding to the same computing node are located in an S3 storage bucket mounted by the computing node, and user directories of different tenants in the same S3 storage bucket are isolated from each other; The step of creating a separate user directory for the tenant in the computing node corresponding to the tenant when the tenant uses the Zeppelin platform includes: When any tenant uses the Zeppelin platform, determining whether the tenant needs to save data; If so, creating a separate user directory for the tenant in the computing node corresponding to the tenant, specifically including: Specifying the mounting type as Host; Determining a target directory in a target computing node to be mounted according to the Host; Creating an initialization container and mounting the target directory in the target computing node to be mounted in the initialization container; Creating the user directory for the tenant in the target directory; Creating a Zeppelin service container and mounting the user directory corresponding to the tenant in the Zeppelin service container.
7. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the multi-tenant data persistence method according to any one of claims 1 to 5 is implemented.
8. A computer device, characterized in that, It includes: One or more processors; A memory; One or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and the one or more computer programs are configured to: execute the multi-tenant data persistence method according to any one of claims 1 to 5.
Citation Information
Patent Citations
Backup data recovery method and device, electronic equipment and storage medium
CN112214351A
k8s multi-tenant-based Juypter Notebook code far-end storage method and device
CN112631527A