File system multi-lease method, system, engine, device, medium and program product

Abstract

The embodiment of the invention provides a file system multi-lease method, system, engine, device, medium and program product, and the method comprises the steps: for a multi-lease distributed file system, a target user renting the system can use a service instance to generate an obtaining request for a target file in the system; and then, verifying the obtaining permission of the target user to the target file according to the obtaining request. When the verification result reflects that the target user has the permission to obtain the target file, it is indicated that the target file is related to the requirement of the target user, and the service instance can obtain the target file in the distributed file system. In the method, for a multi-lease distributed file system, files related to respective demands of different users in the distributed file system can be isolated through verification of the user on the file acquisition permission, and the target user acquires the file related to the own demand and cannot acquire the file related to the demands of other users, so that the user experience is improved, and the user experience is improved. And therefore, the security among the files is ensured.

Description

Technical Field

[0001] The present invention relates to the field of storage technologies, and in particular, to a multi-tenancy method, system, engine, device, medium, and program product for a file system. Background Art

[0002] With the generation of massive amounts of data, when using traditional local file systems, there are often problems such as inconvenient data access, and inconvenient data management and maintenance. To improve the management ability of massive data, a distributed file system (DFS) has emerged as the times require. And with the development of cloud computing technology, this distributed file system can also be provided to users as a cloud service, that is, users can rent DFS to read files related to their own needs from it.

[0003] In practice, for a DFS storing files related to the respective needs of different users, it can also support multi-tenancy, that is, multiple users rent the same DFS. After any user rents this DFS, the user can read all the data in the DFS, that is, the user can not only read files related to their own needs from the DFS, but also read files that are not related to their own needs, that is, related to the needs of other users. At this time, there will be security issues between files related to the respective needs of different users.

[0004] Based on the above description, in the scenario of DFS multi-tenancy, how to improve the security between files related to the respective needs of different users has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a multi-tenancy method, system, engine, device, medium, and program product for a file system to ensure the security between files related to the respective needs of different users in a multi-tenancy scenario.

[0006] In a first aspect, an embodiment of the present invention provides a multi-tenancy method for a file system, including:

[0007] Obtaining an acquisition request generated by a service instance used by a target user for a target file in a distributed file system, where different users have acquisition permissions for different files in the distributed file system;

[0008] Verifying the acquisition permission of the target user for the target file according to the acquisition request;

[0009] Sending a verification result indicating that the target user has acquisition permission for the target file to the service instance, so that the service instance acquires the target file.

[0010] Second aspect, an embodiment of the present invention provides a distributed file system, including: a file database, a metadata engine, and a service instance used by a target user; different users have access rights to different files in the file database;

[0011] The metadata engine is configured to obtain an acquisition request generated by the service instance for a target file in the file database; verify the access right of the target user to the target file according to the acquisition request; and send a verification result indicating that the target user has the access right to the target file to the service instance;

[0012] The service instance is configured to obtain the target file according to the verification result.

[0013] Third aspect, an embodiment of the present invention provides a metadata engine, including: a proxy component, configured to obtain an acquisition request generated by a service instance used by a target user for the target file; verify the access right of the target user to the target file according to the acquisition request; and send a verification result indicating that the target user has the access right to the target file, so that the service instance can obtain the target file.

[0014] Fourth aspect, an embodiment of the present invention provides an electronic device, including: the memory is configured to store one or more computer instructions, wherein when the one or more computer instructions are executed by the processor, the file system multi-tenancy method described in the first aspect above is implemented. The electronic device may further include a communication interface for communicating with other devices or communication systems.

[0015] Fifth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the file system multi-tenancy method described in the first aspect.

[0016] Sixth aspect, an embodiment of the present invention provides a computer program product. The computer program product includes a computer program or instruction, and when the computer program or instruction is executed by a processor, the processor can implement the file system multi-tenancy method described in the first aspect.

[0017] The multi-tenancy method for a file system provided by an embodiment of the present invention is such that different users have access rights to different files in a distributed file system, that is, the distributed file system is multi-tenant for users. Based on this, a target user renting the system can use a service instance to generate an access request for a target file in the system. Subsequently, the access rights of the target user to the target file can be further verified according to this access request, and the verification result can be fed back to the service instance used by the target user. When the verification result indicates that the target user has access rights to the target file, the service instance can obtain the target file in the distributed file system.

[0018] In the above method, for a target user renting a distributed file system, when it is verified that the user has access rights to a target file, it indicates that this target file is a file related to the needs of the target user, rather than a file related to the needs of other users. Then, the target user can normally obtain the target file.

[0019] For a multi-tenant distributed file system, any user renting the system has access rights to files related to their own needs, but has no access rights to files unrelated to their own needs and related to the needs of other users. Therefore, through access verification, this user can be prevented from obtaining files related to the needs of other users, that is, the files in the distributed file system are isolated according to user needs, thereby ensuring the security between files related to the needs of different users in the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for describing the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0021] Figure 1 It is a schematic structural diagram of a distributed file system provided by an embodiment of the present invention;

[0022] Figure 2 It is a schematic structural diagram of another distributed file system provided by an embodiment of the present invention;

[0023] Figure 3 It is a schematic structural diagram of yet another distributed file system provided by an embodiment of the present invention;

[0024] Figure 4 It is a structural diagram of a metadata engine provided by an embodiment of the present invention;

[0025] Figure 5Flowchart of a method for multi-tenancy of a file system provided by an embodiment of the present invention;

[0026] Figure 6 Flowchart of another method for multi-tenancy of a file system provided by an embodiment of the present invention;

[0027] Figure 7 Schematic structural diagram of a device for multi-tenancy of a file system provided by an embodiment of the present invention;

[0028] Figure 8 Schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0029] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0030] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. "Plural" generally includes at least two, but does not exclude the case of including at least one.

[0031] It should be understood that the term " / and" used herein is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the associated objects before and after.

[0032] Depending on the context, the words "if", "when" as used herein may be interpreted as "when...", "when...", "in response to determining" or "in response to identifying". Similarly, depending on the context, the phrase "if determined" or "if identifying (stated condition or event)" may be interpreted as "when determined", "in response to determining", "when identifying (stated condition or event)", or "in response to identifying (stated condition or event)".

[0033] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data that have been authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or reject.

[0034] It should also be noted that the term "including", "comprising", or any other variant thereof is intended to cover non-exclusive inclusion, such that a commodity or system including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such commodity or system. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the commodity or system including said element.

[0035] The following will describe in detail some embodiments of the present invention with reference to the accompanying drawings. Without conflict between the embodiments, the following embodiments and the features in the embodiments can be combined with each other. Additionally, the step timings in the following method embodiments are only examples and are not strictly limited.

[0036] Figure 1 It is a schematic structural diagram of a distributed file system provided by an embodiment of the present invention. As Figure 1 shown, the system may include: a file database, a metadata engine, and service instances used by target users.

[0037] Among them, the file database is used to store files. And the files can be stored as objects in the file database. Optionally, the storage of files can also be provided to users as cloud services such as Object Storage Service (OSS).

[0038] The files stored in the file database of the distributed file system provided by each embodiment of the present invention can be files related to the respective different needs of multiple users, that is, the distributed file system provided by each embodiment of the present invention is multi-tenant. The relationship between files and user needs can also be understood as that users have access rights to the files related to their own needs. Therefore, for a multi-tenant distributed file system, different users have access rights to different files in the file database.

[0039] And user needs can also be related to scenarios. Optionally, for example, in an artificial intelligence scenario, the user's need can be model training, and then the file database can store files related to this model training need, such as training samples.

[0040] For another example, in the Internet of Things (IoT) scenario, a user's requirement may be to analyze data collected by IoT devices. Among them, data analysis can be achieved through Function Compute. Then, the file database can store files related to this data processing requirement, and the files can store the data to be analyzed collected by the devices.

[0041] For another example, a user's requirement may be the processing of multimedia data, such as image processing or audio-to-text conversion, etc. Then, the file database can store files related to this data processing requirement, such as the multimedia data to be processed. The processing of multimedia data can also be achieved through Function Compute.

[0042] Among them, a user who rents a distributed file system can access the system by means of a service instance. Optionally, the service instance can be deployed in the client used by the user, and the service instance can be regarded as a functional module that enables the client to have the ability to access the distributed file system. Optionally, the client can be a virtual device represented as a pod, and the service instance can be a container deployed in the pod. Both the pod and the containers in it run in an isolated environment.

[0043] Based on the above description, the working process of the system provided in this embodiment can be as follows:

[0044] For any user who rents a distributed file system, which can be called the target user, they can use the service instance to access the system. A common access process can be that the target user obtains the target file in the file database. At this time, the target user can trigger an operation on the service instance to make the service instance generate a request to obtain the target file. The service instance can further send the obtain request to the metadata engine. Optionally, the same user can generate obtain requests from service instances running on different clients. Optionally, the obtain request can be transmitted between the service instance and the metadata engine through Remote Procedure Call (RPC for short).

[0045] After that, the metadata engine can verify the target user's access permission to the target file according to the obtain request to obtain a verification result. Among them, the specific authentication process of the metadata engine can refer to the relevant description in the following embodiments. The metadata engine can also feedback the verification result to the service instance, so that the service instance can determine whether to obtain the target file according to the verification result.

[0046] Specifically, if the verification result indicates that the target user has the access right to the target file, it means that the target file is related to the needs of the target user, and the service instance can obtain the target file from the file database. If the verification result indicates that the target user does not have the access right to the target file, it means that the target file is not related to the needs of the target user, and the service instance cannot obtain the target file.

[0047] Optionally, the metadata engine can be implemented as a database, such as Remote Dictionary Server (referred to as redis), KiTV as a distributed transactional key-value database, MariaDB, etc.

[0048] In this embodiment, for a multi-tenant distributed file system, the target user who leases the system can use the service instance to generate an access request for the target file in the file database. After receiving the access request, the metadata engine can verify the access right of the target user to the target file and feedback the verification result to the service instance used by the target user. When the verification result indicates that the target user has the access right to the target file, it means that this target file is a file related to the needs of the target user, rather than a file related to the needs of other users, and the target user can obtain this target file.

[0049] For a multi-tenant distributed file system, any user who leases the system has the access right to the files related to their own needs, but does not have the access right to the files related to the needs of other users. Therefore, by virtue of the access verification function of the metadata engine, this user cannot obtain the files related to the needs of other users, that is, it realizes the partitioning and isolation of the files in the file database according to user needs, thus ensuring the security between the files related to the needs of different users in the system.

[0050] According to the description in the above embodiment, an important role of the metadata engine can be to verify the access rights of the target user. On this basis, in order for the service instance used by the target user to obtain the target file, optionally, another important role of the metadata engine can be to feedback the target metadata of the target file to the service instance used by the target user.

[0051] In one case, when the verification result indicates that the target user has the access right to the target file, the above-mentioned target metadata can be included in the verification result and sent to the service instance used by the target user. After obtaining the target metadata, the service instance can find the target file according to the target metadata and obtain the file. For the file database storing a large number of files, the use of metadata can improve the search speed of the target file. Among them, the target metadata can be used to describe the storage location, file name, size, modification time and other attributes of the target file.

[0052] In another case, when the verification result reflects that the target user does not have the permission to obtain the target file, the target metadata will not be fed back to the service instance, so that the service instance cannot obtain the target file, that is, the isolation of files in the file database is achieved, ensuring the security between files related to the needs of different users.

[0053] In this embodiment, the target metadata required to obtain the target file can be directly fed back to the service instance by the metadata engine, and the service instance will not directly access the metadata engine and read the metadata from it. Therefore, the security of the data in the metadata engine can be improved.

[0054] For the two important functions of the metadata engine mentioned above, they can be implemented by different components in the metadata engine. Then Figure 2 FIG. is a schematic structural diagram of another distributed file system provided by an embodiment of the present invention. In Figure 1 Based on the illustrated embodiment, the metadata engine in the system may include: a proxy component cluster and a storage component.

[0055] Among them, optionally, each proxy component in the proxy component cluster may form a consistent hashing ring. Any proxy component in the proxy cluster has the ability to verify permissions. And the setting of the proxy component cluster is to improve the efficiency and availability of permission verification. The storage component may store the file identifiers of different files in the file database. Optionally, the file identifier may be represented as metadata, or a key-value pair. The key-value pair consists of a primary key and its corresponding metadata.

[0056] When a service instance used by a target user generates an acquisition request, the target proxy component in the proxy component cluster may verify the permissions of the target user. Optionally, the service instance used by the target user may determine the target proxy component from the proxy component cluster according to the identity identifiers of the proxy components in the proxy component cluster and the identity identifier of the target user. Specifically, the service instance may perform a hashing calculation on the identity identifiers of different proxy components in the proxy component cluster to obtain a first hash value; at the same time, it may also perform a hashing calculation on the identity identifier of the target user to obtain a second hash value. Then, the proxy component corresponding to the target hash value that is closest to the second hash value among the first hash values is determined as the target proxy component.

[0057] Among them, the identity identifiers of the proxy components may be assigned when the cloud service provider creates a distributed file system. Optionally, the identity identifier of the user may specifically include a user ID and / or a user token. Different types of identity identifiers may be assigned by relevant components in the distributed file system during the user registration phase.

[0058] The following details the process of the target proxy component performing permission verification on the target user:

[0059] After the target proxy component receives the acquisition request generated by the target user for the target file, optionally, the target proxy component may verify the acquisition permission of the target user for the target file according to the identity identifier of the target user and the file identifier of the target file in the acquisition request.

[0060] Specifically, the target proxy component may determine the set of file identifiers corresponding to the target user according to the identity identifier of the target user. If the file identifier of the target file in the acquisition request is included in this set of file identifiers, the target proxy component may determine that the target user has the acquisition permission for the target file. Among them, the set of file identifiers corresponding to the target user may include the identifiers of different files for which the target user has acquisition permission in the file database. This set of file identifiers can be regarded as a directory, and the target user has acquisition permission for the files under this directory.

[0061] For the determination of the set of file identifiers corresponding to the target user, when the identity identifier includes the user ID and the user token, the target proxy component may first determine the user token of the target user according to the user ID of the target user, and then further determine the set of file identifiers corresponding to this user token. When the identity identifier includes the user ID or the user token, the target proxy component may directly determine the set of file identifiers according to the user ID or the user token. Among them, the corresponding relationship between the user ID or the user token and the set of file identifiers may be set during the user registration phase, and the specific process may refer to the description in the following related embodiments.

[0062] For the file identifiers in the set, in an optional case, they may directly be presented as the metadata of the file, so the set of file identifiers is essentially a set of metadata. And during the user registration phase, a proxy component in the metadata engine may create a set of metadata according to the permission information in the registration request. Among them, both the set of metadata and the permission information are used to describe which files in the file database the user has acquisition permission for, only with different presentation forms.

[0063] Among them, for any registered user, any proxy component or a specific proxy component in the cluster may create a set of metadata for it. Optionally, the basis for selecting the specific proxy component may be: if the hash value corresponding to the identity identifier of any user is closest to the hash value corresponding to the identity identifier of a proxy component in the proxy component cluster, then select this proxy component to create the set of metadata corresponding to this any user. Optionally, the above selection process may be executed by the registration module component in the distributed file system.

[0064] Based on this, when the target proxy component receives a request to obtain a target file, it can directly obtain the metadata set associated with the identity identifier of the target user from the storage component according to the identity identifier of the target user in the obtain request. If the metadata of the target file in the obtain request is included in the metadata set corresponding to the target user, the target proxy component can determine that the target user has the permission to obtain the target file.

[0065] After completing the permission verification of the target user in the above manner, the service instance used by the target user can directly obtain the target file according to the target metadata of the target file in the obtain request.

[0066] In this embodiment, the target proxy component can implement the permission verification of the target user by using the identity identifier in the obtain request and the file identifier presented as metadata. After passing the verification, the service instance used by the target user can also directly obtain the target file by using the metadata of the target file.

[0067] The data in the storage component can also be stored in the form of key-value pairs. A key-value pair is composed of the metadata of the file and the primary key corresponding to the metadata. For the file identifiers in the set, in another optional case, the file identifier can be presented as a key-value pair, and the file identifier set is essentially a key-value pair set. And during the user registration phase, the proxy component in the metadata engine can create a key-value pair set according to the permission information in the registration request. Among them, for any registered user, the registration component can also select a specific proxy component in the proxy component cluster through hash calculation to create the key-value pair set corresponding to the any user.

[0068] Based on this, when the target proxy component receives a request to obtain a target file, it can determine the key-value pair set corresponding to the target user according to the identity identifier of the target user in the obtain request. The key-value pair set includes the primary key of the file that the target user has the permission to obtain and its metadata. If the target primary key presented as the file identifier in the obtain request is included in the key-value pair set corresponding to the target user, it is determined that the target user has the permission to obtain the target file.

[0069] After completing the permission verification of the target user in the above manner, the service instance used by the target user can obtain the target file according to the target metadata corresponding to the target primary key.

[0070] In this embodiment, the target proxy component can implement the permission verification of the target user by using the identity identifier in the obtain request and the file identifier presented as the primary key. After passing the verification, the service instance used by the target user can obtain the target file by means of the target primary key. And with the help of the key-value pair set, the target proxy component can implement the permission verification of the target user more quickly. Further, it can also improve the speed at which the target user reads the file.

[0071] In the above description, the set of file identifiers can be created by the proxy component during the user registration phase. Optionally, as Figure 2 shown, the distributed file system may also include a management and control component and a registration component. The user registration can be completed collaboratively by the management and control component, the registration component, and the proxy component. The registration process can be described in detail as follows:

[0072] When the identity identifiers used for permission verification include the user ID and the user token, the management and control component can respond to the registration request of the target user to generate the user ID of the target user. The management and control component can send this user ID to the registration component, so that the registration component can further create the user token of this target user.

[0073] When the identity identifiers used for permission verification include the user ID, the management and control component can respond to the registration request of the target user to generate the user ID of the target user.

[0074] When the identity identifiers used for permission verification include the user token, the management and control component can respond to the registration request of the target user and send a registration request to the registration component, so that the registration component can further create the user token of this target user.

[0075] After the management and control component and / or the registration component create the identity identifier for the target user, the identity identifier can also be sent to the target proxy component in the proxy cluster, and this target proxy component can also generate the set of file identifiers corresponding to the target user according to the permission information in the registration request. The reason why the identity identifier is sent to the target proxy component instead of other proxy components is that the target hash value corresponding to the identity identifier of the target proxy component is closest to the second hash value corresponding to the identity identifier of the target user.

[0076] Furthermore, the identity identifier and the set of file identifiers generated by the target proxy component can also be fed back to the management and control component together. Finally, the management and control component can create a client in the form of a container group according to the identity identifier and the set of file identifiers. The container group contains service instances for the target service. At this time, that is, the registration of the target user is completed. Since the set of file identifiers can be regarded as a directory, after the user registration is completed, it can also be considered that the directory is mounted to the service instance.

[0077] In this embodiment, the creation of the container group can be achieved through the collaborative work of the management and control component, the registration component, and the proxy component, and the user can use the service instances in the container group to access the file database.

[0078] According to Figure 2 the description in the shown embodiment, the set of file identifiers can be regarded as a directory, and the user has the permission to obtain the files under this directory; according to Figure 1As can be seen from the description of the embodiments shown, the service instance used by the user can be a container running in an isolated environment. And in practice, there may also be cases of container escape, that is, the container runs outside the isolated environment.

[0079] Then when the container used by the target user escapes, the container can send a permission update request to the proxy component cluster, that is, request to mount a new directory from the target proxy component. The permission update request may include an identity identifier and updated permission information. Then the container can send the permission update request to the target proxy component through hash calculation for the proxy component to process. Among them, the updated permission information corresponds to the set of file identifiers of the new files that the target user wants to obtain.

[0080] Furthermore, the target proxy component can verify the validity of the permission update request. Specifically, the target proxy component can determine the reference permission information of the target user according to the identity identifier of the target user in the permission update request. The reference permission information corresponds to the set of file identifiers created for the target user by the target proxy component during the registration phase. If the updated permission information in the permission update request is different from this reference permission information, it indicates that the target user wants to obtain files related to the needs of other users, but the target user obviously does not have the permission to obtain these files. Then the target proxy component can determine that this permission update request is invalid. The target proxy component will not modify the target user's access permission to the files in the file database, that is, will not modify the set of file identifiers corresponding to the target user.

[0081] And in practice, most of the updated permission information is different from the reference permission information. Therefore, most permission update requests will be determined to be invalid.

[0082] In this embodiment, when the container acting as the service instance escapes and requests to update the permission information, the target proxy component can, by virtue of its own permission verification function, determine whether the target can update the permission information. When the target user cannot update the permission information, the target proxy component can prevent the modification of the permission information, thus ensuring that the target user cannot obtain files related to the needs of other users, and thus ensuring the security of the files in the file database.

[0083] In the above embodiments, after the target user passes the permission verification, the service instance used by the target user can obtain the target file. To ensure the security of the files in the file database, optionally, a distributed cache can be used to limit the service instance from directly obtaining the target file from the file database. Then the process of obtaining the target file from the cache is combined with Figure 3 the description in the embodiments shown for understanding.

[0084] Figure 3This is a schematic structural diagram of another distributed file system provided by an embodiment of the present invention. In Figure 2 Based on the illustrated embodiment, the system may further include: a cache node cluster.

[0085] Optionally, the cache node cluster may form a consistent hashing ring.

[0086] For a target file that a target user wants to obtain, if the target file is stored in a target cache node in the cache node cluster, it indicates that the target file has been used by the target user or a user with the same requirements as the target user. Then, the service instance used by the target user can directly obtain the target file from the target cache node.

[0087] If the target file is not stored in the target cache node, it indicates that the target file has not been used by the target user or a user with the same requirements as the target user. Then, the target cache node can, according to the file identifier of the target file in the acquisition request, first read the target file from the file database to the local, and then the service instance used by the target user can obtain the target file from the target cache node.

[0088] For the process of the target cache node reading the target file from the file database, optionally, the target cache node can, according to the file identifier of the target file in the acquisition request, find the target file in the file database and read the file to the target cache node. When the file identifier is in the form of a key-value pair, the target cache node can, according to the target metadata corresponding to the target primary key in the key-value pair, read the target file to the target cache node.

[0089] Optionally, the determination method of the target cache node may be: the service instance used by the target user can determine according to the identity identifiers of each cache node in the cluster and the identity identifier of the target user. Specifically, a hash calculation may be performed on the identity identifier of the cache node to obtain a third hash value, and a hash calculation may be performed on the identity identifier of the target user to obtain a second hash value. The cache node corresponding to the target hash value that is closest to the second hash value among the third hash values is the target cache node.

[0090] In this embodiment, regardless of whether the target file is stored in the cache node, the service instance will not directly access the file database during the process of obtaining the target file, thus ensuring the security of the files in the file database.

[0091] In addition, for the distributed file system provided in the above embodiments, when it is multi-tenant for users, the working process of the file system can also be understood in combination with the following content.

[0092] Suppose a distributed file system is leased to users A and B. User A uses the files in the file system for model training. User B uses the files in the file system for audio-to-text conversion. Files 1 to 200 can be stored in the file database of the distributed file system. Among them, files 1 to 100 are the training samples required by user A. Files 101 to 200 are the audio required by user B.

[0093] Based on the above assumption, user A can use service instance 1 to send a request to obtain file 10 to proxy component 1 in the proxy component cluster. Then, proxy component 1 in the proxy component cluster can verify whether user A has the permission to obtain file 10 according to the identity identifier of user A and the file identifier of file 10 in this request.

[0094] After verification, proxy component 1 can feedback the verification result indicating that user A has the permission to obtain file 10, which contains the metadata of file 10, to service instance 1. Finally, service instance 1 can read file 10 from cache node 1 in the cache node cluster based on this metadata. Among them, if other users with the same requirements as user A, or user A has used file 10 before, then file 10 can be stored in cache node 1. If no user has used file 10, cache node 1 needs to read file 10 from the file database into cache node 1.

[0095] The verification process of proxy component 1, and the determination processes of proxy component 1 and cache node 1 can refer to the descriptions in the above related embodiments.

[0096] When user B also uses service instance 2 to send a request to obtain file 10 to proxy component 1, proxy component 1 can also verify the permission of user B. Since file 10 does not match user B's audio-to-text conversion requirements, proxy component 1 determines that user B does not have the permission to obtain file 10. Then, proxy component 1 can feedback the verification result indicating that user B does not have the permission to obtain file 10 to user B. Finally, service instance 2 used by user B cannot obtain file 10 due to the lack of metadata.

[0097] According to the above scenario embodiments, by using the permission verification function of the proxy component in the metadata engine, files in the file system suitable for different user requirements can be isolated, so that users with one requirement cannot obtain files corresponding to other requirements, thereby ensuring the security between files in the file system.

[0098] In addition, the content not described in detail in this embodiment can also refer to the relevant descriptions in the above embodiments, and will not be elaborated here.

[0099] Based on the distributed file system provided by the above embodiments,Figure 4 The figure is a schematic structural diagram of a metadata engine provided by an embodiment of the present invention. As Figure 4 shown, the engine may include a proxy component.

[0100] The proxy component may first obtain a fetch request generated by a service instance used by a target user for a target file, and then verify the fetch permission of the target user for the target file according to the fetch request. When the proxy component determines that the target user has the fetch permission for the target file, it may send a verification result reflecting that the target user has the fetch permission to the service instance used by the target user. Finally, the service instance may obtain the target file.

[0101] In this embodiment, by virtue of the permission verification function of the metadata engine, the user is prevented from obtaining files related to the needs of other users, that is, the files in the file database are partitioned and isolated according to user needs, thereby ensuring the security between files related to different user needs in the system.

[0102] In addition, for the content not described in detail in this embodiment, reference may also be made to the relevant descriptions in the above embodiments, which will not be elaborated herein.

[0103] Optionally, in order to improve the efficiency of permission verification and ensure the high availability of permission verification, a proxy component cluster may also be set in the metadata engine. Then Figure 4 in the shown embodiment, the proxy component for determining whether the target user has the fetch permission for the target file may be the target proxy component in the proxy component cluster. The determination process of the target proxy component may be implemented by the service instance through a hash calculation method. For the specific process, reference may be made to Figure 2 the relevant descriptions in the shown embodiment, which will not be elaborated herein.

[0104] Optionally, as Figure 4 shown, the metadata engine may include a storage component for storing the file identifiers of different files in the distributed file system. Optionally, the file identifier may be represented as metadata or a key-value pair. The key-value pair consists of a primary key and its corresponding metadata. After the target proxy component determines that the target user has the fetch permission for the target file, it may include the file identifier of the target file in the verification result and send it to the service instance used by the target user, so that the service instance may obtain the target file based on this file identifier.

[0105] Based on the above embodiments, the working process of the proxy component in the metadata engine may also be described from the perspective of the process when implementing multi-tenancy of the file system. Then Figure 5 The figure is a flowchart of a method for multi-tenancy of a file system provided by an embodiment of the present invention. The method provided by the embodiment of the present invention may be executed by the target proxy component in the metadata engine of the distributed file system. As Figure 1As shown in the figure, the method may include the following steps:

[0106] S101, obtaining an acquisition request generated by a service instance used by a target user for a target file in a distributed file system, where different users have acquisition permissions for different files in the distributed file system.

[0107] S102, verifying the acquisition permission of the target user for the target file according to the acquisition request.

[0108] S103, sending a verification result indicating that the target user has acquisition permission for the target file to the service instance, so that the service instance can obtain the target file.

[0109] In this embodiment, the target user may first use the service instance to generate an acquisition request for the target file in the distributed file system, and the acquisition request may be sent to the target proxy component. Then, the target proxy component can verify the permissions of the target user and feedback the verification result to the service instance used by the target user. If the verification result indicates that the target user has permission to acquire the target file, the service instance can obtain the target file.

[0110] For the specific implementation manners of the steps in this embodiment and the technical effects that can be achieved, reference may be made to the relevant descriptions in the above embodiments, which will not be elaborated here.

[0111] Figure 6 It is a flowchart of another multi-tenancy method for a file system provided by an embodiment of the present invention. As Figure 6 shown, the method may include the following steps:

[0112] S201, obtaining an acquisition request generated by a service instance used by a target user for a target file in a distributed file system, where different users have acquisition permissions for different files in the distributed file system.

[0113] For the specific implementation process of the above step S201, reference may be made to Figure 5 the specific descriptions of the relevant steps in the shown embodiment, which will not be elaborated here.

[0114] S202, determining a file identifier set corresponding to the target user according to the identity identifier of the target user in the acquisition request, where the file identifier set includes the identifiers of the files for which the target user has acquisition permissions.

[0115] S203, verifying the acquisition permission of the target user for the target file according to the file identifier set and the file identifier of the target file.

[0116] S204, sending a verification result indicating that the target user has acquisition permission for the target file to the service instance, so that the service instance can obtain the target file.

[0117] The acquisition request received by the target proxy component may include the identity identifier of the target user and the file identifier of the target file. The target proxy component may first determine the set of file identifiers corresponding to the target user according to the identity identifier of the target user, and then verify the acquisition permission of the target user for the target file according to the set of file identifiers and the file identifier of the target file in the acquisition request. In an optional manner, if the file identifier of the target file is included in the set of file identifiers corresponding to the target user, it is determined that the target user has the acquisition permission for the target file. The target proxy component may generate a verification result indicating that the target user has the acquisition permission to the service instance used by the target user, so that the service instance can acquire the target file.

[0118] For the content included in the identity identifier of the user and the file identifier respectively, and the allocation method of the identifier, reference can be made to the description in the embodiments shown in Figure 2 and will not be elaborated here.

[0119] For the registration process of the user and the specific verification process of the acquisition permission by the proxy component, reference can also be made to the description in the embodiments shown in Figure 2 and will not be elaborated here.

[0120] In addition, for the content not described in detail in this embodiment, reference can also be made to the relevant descriptions in the above embodiments and will not be elaborated here.

[0121] Optionally, the service instance used by the target user may be specifically represented as a container. In this case, there may be a situation of container escape. After the escape occurs, the container may also send a permission update request to the proxy component, that is, request the proxy component to update its own acquisition permission. The proxy component may use its own permission verification function to verify whether the container can update its own acquisition permission, so as to reject the illegal permission update of the container and ensure the security of the files in the file system.

[0122] The following will detail the file system multi-tenancy device of one or more embodiments of the present invention. Those skilled in the art can understand that these file system multi-tenancy devices can all be configured by using commercially available hardware components through the steps taught by this solution.

[0123] Figure 8 For a schematic structural diagram of a file system multi-tenancy device provided by an embodiment of the present invention, as shown in Figure 8 the device may include:

[0124] An acquisition module 11, configured to acquire an acquisition request generated by a service instance used by a target user for a target file in a distributed file system, where different users have acquisition permissions for different files in the distributed file system.

[0125] The verification module 12 is used to verify the access right of the target user to the target file according to the acquisition request.

[0126] The sending module 13 is used to send a verification result indicating that the target user has the access right to the target file to the service instance, so that the service instance can acquire the target file.

[0127] Optionally, the verification module 12 is used to verify the access right of the target user to the target file according to the identity identifier of the target user and the file identifier of the target file in the acquisition request.

[0128] Optionally, the verification module 12 is used to determine a set of file identifiers corresponding to the target user according to the identity identifier, and the set of file identifiers includes the identifiers of the files to which the target user has the access right; if the file identifier in the acquisition request is included in the set of file identifiers, it is determined that the target user has the access right to the target file.

[0129] The set of file identifiers includes a set of key-value pairs, and the set of key-value pairs includes the metadata of the files to which the target user has the access right and the primary keys corresponding to the metadata.

[0130] Optionally, the verification module 12 is used to determine that the target user has the access right to the target file if the target primary key in the file identifier is included in the set of key-value pairs.

[0131] Optionally, the verification module 12 is used to determine the target metadata corresponding to the target primary key in the set of key-value pairs; send the verification result including the target metadata to the service instance, so that the service instance can acquire the target file according to the target metadata.

[0132] Optionally, the device further includes: a registration module 14, which is used to respond to the registration request of the target user to generate the identity identifier of the target user; create a set of file identifiers corresponding to the target user according to the permission information in the registration request, and the permission information describes the access rights of the target user to different files in the distributed file system.

[0133] Optionally, the service instance is manifested as a container running in an isolated environment.

[0134] The acquisition module 11 is used to receive a permission update request generated when the container runs outside the isolated environment.

[0135] The verification module 12 is configured to determine that the permission update request is invalid according to the identity identifier of the target user and the updated permission information in the permission update request, and does not modify the access permission of the target user to the files in the distributed file system.

[0136] Figure 7 The device shown can execute Figures 5 - 6 the method of the embodiment shown. For parts not described in detail in this embodiment, reference can be made to the relevant descriptions of Figures 5 - 6 the embodiment shown. For the execution process and technical effects of this technical solution, refer to the descriptions in Figures 5 - 6 the embodiment shown, which will not be elaborated here.

[0137] In a possible design, the multi-tenancy method of the file system provided in the above embodiments can be applied to an electronic device, such as Figure 8 shown. The electronic device may include: a processor 31 and a memory 32. Among them, the memory 32 is used to store a program that supports the electronic device to execute the multi-tenancy method of the file system provided in the above Figures 5 - 6 shown embodiment, and the processor 31 is configured to execute the program stored in the memory 32.

[0138] The program includes one or more computer instructions. When one or more computer instructions are executed by the first processor 31, the following steps can be implemented:

[0139] Obtain an access request generated by a service instance used by a target user for a target file in a distributed file system, where different users have access permissions to different files in the distributed file system;

[0140] Verify the access permission of the target user to the target file according to the access request;

[0141] Send a verification result indicating that the target user has access permission to the target file to the service instance, so that the service instance can obtain the target file.

[0142] Optionally, the processor 31 is further configured to execute all or part of the steps in the foregoing Figures 5 - 6 shown embodiment.

[0143] Among them, the structure of the electronic device may further include a communication interface 33, which is used for the electronic device to communicate with other devices or communication systems.

[0144] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the above electronic device, which includes a program for executing the Figures 5 - 6 multi-tenancy method of the file system shown above.

[0145] In addition, an embodiment of the present invention provides a computer program product. The computer program product includes a computer program or instructions. When the computer program or instructions are executed by a processor, the processor is enabled to implement the above-mentioned Figures 5 - 6 steps or functions of the file system multi-tenancy method shown.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-tenancy method for a file system, characterized in that Including: Obtain a fetch request generated by a service instance used by a target user for a target file in a distributed file system, where different users have fetch permissions for different files in the distributed file system; Verify the fetch permission of the target user for the target file according to the fetch request; Send a verification result indicating that the target user has the fetch permission for the target file to the service instance, so that the service instance can obtain the target file.

2. The method according to claim 1, characterized in that, The verifying the fetch permission of the target user for the target file according to the fetch request includes: Verify the fetch permission of the target user for the target file according to the identity identifier of the target user in the fetch request and the file identifier of the target file.

3. The method according to claim 2, wherein The verifying the fetch permission of the target user for the target file according to the identity identifier of the target user and the file identifier of the target file in the file fetch request includes: Determine a set of file identifiers corresponding to the target user according to the identity identifier, where the set of file identifiers includes the identifiers of the files for which the target user has fetch permissions; If the file identifier in the fetch request is included in the set of file identifiers, determine that the target user has the fetch permission for the target file.

4. The method according to claim 3, wherein The set of file identifiers includes a set of key-value pairs, and the set of key-value pairs includes the metadata of the files for which the target user has fetch permissions and the primary keys corresponding to the metadata; The determining that the target user has the fetch permission for the target file if the file identifier in the fetch request is included in the set of file identifiers includes: If the target primary key in the file identifier is included in the set of key-value pairs, determine that the target user has the fetch permission for the target file; The sending a verification result indicating that the target user has the fetch permission for the target file to the service instance so that the service instance can obtain the target file includes: In the set of key-value pairs, determine the target metadata corresponding to the target primary key; Send the verification result including the target metadata to the service instance, so that the service instance can obtain the target file according to the target metadata.

5. The method according to claim 3, wherein The method further includes: Respond to the registration request of the target user to generate the identity identifier of the target user; Create a set of file identifiers corresponding to the target user according to the permission information in the registration request, where the permission information describes the fetch permissions of the target user for different files in the distributed file system.

6. The method according to claim 5, characterized in that The service instance is manifested as a container running in an isolated environment; The method further includes: Receive a permission update request generated when the container is running outside the isolated environment; Determine that the permission update request is invalid according to the identity identifier of the target user and the updated permission information in the permission update request; Do not modify the fetch permissions of the target user for the files in the distributed file system.

7. A distributed file system, characterized in that, Including: A file database, a metadata engine, and a service instance used by a target user; Different users have fetch permissions for different files in the file database; The metadata engine is used to obtain the acquisition request generated by the service instance for the target file in the file database; Verify the acquisition permission of the target user for the target file according to the acquisition request; Send the verification result indicating that the target user has the acquisition permission for the target file to the service instance; The service instance is used to obtain the target file according to the verification result.

8. The system according to claim 7, wherein The metadata engine includes a proxy component cluster; The target proxy component in the proxy component cluster is used to verify the acquisition permission of the target user for the target file according to the identity identifier of the target user and the file identifier of the target file in the acquisition request.

9. The system according to claim 8, wherein The target proxy component is used to determine the key-value pair set corresponding to the target user according to the identity identifier of the target user in the acquisition request, where the key-value pair set includes the metadata of the files that the target user has the acquisition permission for and the primary keys corresponding to the metadata; If the target primary key in the file identifier is included in the key-value pair set, it is determined that the target user has the acquisition permission for the target file; read the target metadata corresponding to the target primary key from the key-value pair set; The service instance is used to obtain the target file according to the target metadata.

10. The system according to claim 7, wherein The service instance used by the target user is used to determine the target cache node according to the identity identifiers of the cache nodes in the cache node cluster and the identity identifier of the target user; Read the target file from the target cache node included in the cache node cluster.

11. The system according to claim 10, wherein The target cache node is further used to, if the target file is not included in the target cache node, read the target file from the file database to the target cache node according to the file identifier of the target file in the acquisition request.

12. A metadata engine, characterized in that, Includes: A proxy component, which is used to obtain the acquisition request generated by the service instance used by the target user for the target file; Verify the acquisition permission of the target user for the target file according to the acquisition request; Send the verification result indicating that the target user has the acquisition permission for the target file, so that the service instance obtains the target file.

13. The engine according to claim 12, characterized in that, The engine further includes a proxy component cluster, and the proxy component in the proxy component cluster corresponds to the target user; The engine further includes: a storage component, which is used to store the file identifiers of different files in the distributed file system, where the file identifier includes metadata or key-value pairs, and the key-value pair is composed of a primary key and the metadata corresponding to the primary key; the proxy component is used to send the verification result, so that the service instance obtains the target file according to the file identifier of the target file in the verification result.

14. An electronic device, characterized in that, Includes: A memory and a processor; wherein, executable code is stored on the memory, and when the executable code is executed by the processor, the processor executes the file system multi-tenancy method according to any one of claims 1 to 6.

15. A non-transitory machine-readable storage medium, characterized in that, The non-transitory machine-readable storage medium stores executable code that, when executed by a processor of an electronic device, causes the processor to execute the file system multi-tenancy method according to any one of claims 1 to 6.

16. A computer program product, characterized in that, It includes a computer program or instruction that, when executed by a processor, causes the processor to be able to implement the steps in the file system multi-tenancy method according to any one of claims 1 to 6.

Images