Distributed file access method, device and system

Abstract

The distributed file access method is applied to a client, a server is an owner of a source file, and the client accesses the source file of the server through a network; the method comprises the following steps: determining information of a source file requested to be accessed, wherein the information of the source file requested to be accessed comprises information of the source file requested to be accessed at a server; sending a request message to a server, wherein the request message comprises source file information requested to be accessed; receiving a reply message of the server, wherein the reply message comprises metadata of the source file, extension attribute information and a copy of the source file; wherein the extended attribute information comprises a first encryption strategy of the server, and the metadata of the source file is data describing a storage mode and an organization management structure of the source file of the server; determining a second encryption strategy according to the extended attribute information; the second encryption strategy is consistent with the first encryption strategy; performing local management on the copy of the source file according to the metadata of the source file; and encrypting and / or decrypting the copy of the source file according to the second encryption strategy.

Description

Technical Field

[0001] The present application relates to the field of terminals, and in particular to a method, device and system for accessing distributed files. Background Art

[0002] Distributed file system is a core technology that provides data inter-access capabilities between multiple devices. It can effectively solve the problem of limited storage space on a single device and data management between multiple devices. In the early stage, distributed file system was mainly used in server scenarios to manage massive data across network device nodes. With the development of mobile devices, the data storage capacity of mobile devices is increasing, and the scenario of a user using multiple devices for collaborative data processing is becoming more and more frequent. Distributed file system has also begun to be used in the terminal field. Since a large amount of data on mobile devices belongs to the user's private data, the personal data of the user on a single device is protected by file encryption schemes for data security. However, the existing distributed file system scheme cannot provide the demand for distributed file protection according to security levels. In this way, the user's security is reduced after the user's data is transmitted through distributed transmission. Due to the complex network environment of mobile devices and unstable network bandwidth, and file metadata is the hot data to be accessed, the existing distributed technology has low data transmission performance and cannot efficiently utilize bandwidth, affecting the access experience of distributed data. Summary of the invention

[0003] The embodiments of the present application provide a distributed file access method, device and system, which can achieve high transmission performance of distributed metadata, efficiently utilize bandwidth, and provide globally consistent security level protection for source files.

[0004] In the first aspect, an embodiment of the present application provides a distributed file access method, which can be applied to a client, where the client device establishes an end-to-end link with a server, the server is the owner of the source file, and the client device accesses the source file of the server through a network; the distributed file access method includes: determining the source file information requested for access, the source file information requested for access includes the information of the source file requested for access on the server; sending a request message to the server, the request message includes the source file information requested for access; receiving a reply message from the server, the reply message includes the metadata of the source file, extended attribute information and a copy of the source file; wherein the extended attribute information includes a first encryption policy of the server, and the metadata of the source file is data that describes the storage method and organizational management structure of the source file of the server; determining a second encryption policy based on the extended attribute information; the second encryption policy is consistent with the first encryption policy; locally managing the copy of the source file based on the metadata of the source file; encrypting and / or decrypting the copy of the source file based on the second encryption policy.

[0005] Therefore, the distributed file access method provided in the embodiment of the present application transmits the encryption policy of the original file through metadata and extended attributes, establishes an encryption policy and organizational management structure on the client that are globally consistent with the server, facilitates indexing and query of distributed file access, realizes global security protection of distributed file data, improves data security, and improves the accessibility of distributed files.

[0006] In one possible implementation, determining the source file information requested for access includes: determining an access operation to the source file requested for access; the access operation includes one or a combination of the following: open, read, copy, add, delete, modify, and check; determining a processing scenario based on the access operation, the processing scenario includes delayed transmission and merged transmission.

[0007] Therefore, the distributed file access method provided in the embodiment of the present application classifies and identifies the processing scenarios of access operations, transmits on demand, reduces the number of interactions with the server, improves the metadata access performance of the distributed file system, and improves the user experience.

[0008] In a possible implementation, determining a processing scenario according to an access operation includes: determining the processing scenario as a merged transmission according to the following situation: there are one or more associated requests, and the one or more associated requests include one or more delayed transmission request messages cached locally.

[0009] Therefore, the method provided in the embodiment of the present application can optimize file access operations, merge and optimize access operations in combination with file processing scenarios, and can carry multiple access operations that can be merged in a single network request in the access of a distributed file system, thereby improving the access performance of distributed files and network bandwidth utilization, and improving user experience.

[0010] In a possible implementation, the first encryption policy includes an encryption algorithm for the file and an encryption level for the device, and determining the second encryption policy according to the extended attribute information includes: creating a first master key according to the encryption algorithm and the encryption level of the device configured in the second encryption policy file.

[0011] Therefore, the method provided in the embodiment of the present application obtains the encryption policy information of the server through the extended attribute information and establishes a protection mechanism consistent with the server.

[0012] In a possible implementation, the extended attribute information also includes a key ID, and the key ID is generated by the server through a Hash algorithm according to the encryption algorithm of the file and the encryption level of the device.

[0013] Therefore, the method provided in the embodiment of the present application obtains the key ID of the server through extended attributes, queries and indexes the local master key according to the key ID, and realizes the generation and management of globally consistent distributed keys.

[0014] In a possible implementation, before encrypting and / or decrypting the copy of the source file according to the second encryption policy, the method includes: determining that the first master key has been loaded according to the key ID, and associating the first master key.

[0015] Therefore, the method provided in the embodiment of the present application verifies the encryption policy according to the key ID, thereby achieving globally consistent management and control of distributed keys.

[0016] In a possible implementation, encrypting and / or decrypting the received source file according to the second encryption strategy includes: calling the first master key to generate the first data key; and decrypting the copy of the source file according to the first data key.

[0017] Therefore, in the method provided in the embodiment of the present application, the accessible state of the source file copy is consistent with the source file of the server device, and the protection granularity of the distributed file data is refined by the first data key matching the encryption policy, thereby realizing hierarchical protection of the distributed file data and improving the security of the data on different devices.

[0018] In a possible implementation, the copy of the source file is managed according to the metadata of the source file, including: encrypting and saving the copy of the source file according to the first data key, and organizing and managing the copy of the source file according to the metadata information of the source file.

[0019] Therefore, the method provided in the embodiment of the present application adopts a globally consistent encryption strategy and metadata format to achieve hierarchical protection of distributed file data and improve the security and accessibility of data on different devices.

[0020] In a possible implementation, the method provided by the embodiment of the present application is also applied to the server, and the method includes: receiving a request message to obtain source file information that the client requests to access; obtaining the metadata of the source file and a copy of the source file based on the source file information; determining the extended attribute information of the source file based on the source file information, the extended attribute information including a first encryption policy; sending a reply message, the reply message including the metadata of the source file, the extended attribute information and a copy of the source file.

[0021] Therefore, the method provided in the embodiment of the present application transmits the security information of the source file across devices through the file extension attributes on the server side, and transmits the encryption policy of the file as the extended attribute information of the file to the client device, so as to establish an encryption policy and organizational management structure on the client side that is globally consistent with the server side, facilitate the indexing and query of distributed files, realize global security protection of distributed file data, improve data security, and improve the accessibility of distributed files.

[0022] In one possible implementation, the request message is a merged request message, and the request message is parsed to obtain source file information requested for access, including: parsing the merged request message to obtain multiple source file information requested for access, wherein the multiple source file information requested for access includes multiple associated requests for the same source file.

[0023] Therefore, the method provided in the embodiment of the present application obtains multiple associated requests carried in a network request on the server side. The multiple associated requests are associated access operations for the same source file and can be processed in batches, thereby improving the access performance of file metadata and network bandwidth utilization, and improving user experience.

[0024] In a possible implementation, metadata of the source file and a copy of the source file are obtained according to the source file information, including: obtaining metadata of the source file according to the source file information requested for access, and performing unified metadata adaptation on the format of the metadata of the source file.

[0025] Therefore, the method provided in the embodiment of the present application can shield the different implementation details of heterogeneous metadata in the underlying file system through a unified meta-adapter, so that the metadata of the source file in the underlying file system has the same data structure expression at the distributed file system layer. Metadata with the same data structure can conveniently carry the extended attributes of different platforms or file systems, and establish a unified distributed file organization and management architecture.

[0026] In a possible implementation, the extended attribute information of the source file is determined based on the source file information, including: the extended attribute information also includes a key ID, in which the file encryption algorithm, the device encryption level, and the key ID are all identified by globally unique characters or strings.

[0027] Therefore, the method provided in the embodiment of the present application can transmit the security information of the source file across devices through the file extension attributes on the server side, and pass the file's encryption policy as the file's extended attribute information to the client device, so as to establish an encryption policy on the client side that is globally consistent with the server side.

[0028] In a possible implementation, determining the extended attribute information of the source file also includes: uniformly adapting the encryption algorithm of the file, the encryption level of the device, and the key ID in the extended attribute information to obtain the extended attributes expressed in the same data structure.

[0029] Therefore, the method provided in the embodiment of the present application can shield the different implementation details of heterogeneous metadata in the extended attribute information through a unified meta-adapter, so that the extended attribute information has the same data structure expression at the distributed file system layer, and the extended attribute information with the same data structure can be conveniently carried by the metadata of different platforms or file systems, thereby establishing a unified distributed metadata organization and management architecture.

[0030] In a possible implementation, the request message is a merged request message, and the server-side method also includes: batch processing the merged request messages, and assembling the results of the batch processing to obtain a reply message; the batch processing includes performing continuous or associated operations on multiple associated requests for the same source file; the results of the batch processing include metadata information of the source file, extended attribute information of the source file, and a copy of the source file.

[0031] Therefore, the method provided in the embodiment of the present application can perform batch processing on the merged request messages to perform centralized and continuous access operations or related operations on the same access object, eliminate redundant query, indexing, encryption, decryption and verification operations in the process of secure access to source files, reduce the number of interactions with the underlying file system and distributed key management components, improve access performance and efficiency, and enhance user experience.

[0032] In a second aspect, an embodiment of the present application provides a distributed file system, the system is applied to a terminal device, the terminal device includes a client and a server; the client and the server establish an end-to-end link, the client and the server are compatible in encryption strategy, the server is the owner of the source file, and the client device accesses the source file of the server through the network; wherein, when the distributed file system runs on the terminal device, it executes a method as any one of the first aspects.

[0033] In a third aspect, an embodiment of the present application provides a terminal device, comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method provided in the first aspect.

[0034] In a fourth aspect, an embodiment of the present application provides a distributed file access device, characterized in that the device runs computer program instructions to execute the method provided in the first aspect. Exemplarily, the device may be a chip or a processor.

[0035] In a fifth aspect, an embodiment of the present application provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method provided in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the multiple embodiments disclosed in this specification, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only the multiple embodiments disclosed in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] The following is a brief introduction to the drawings required for describing the embodiments or prior art.

[0038] Figure 1 A schematic diagram of the hardware system configuration provided in the embodiment of the present application;

[0039] Figure 2 A diagram of the software application system architecture provided for the embodiments of the present application;

[0040] Figure 3 A schematic diagram of an application scenario of a distributed file system provided in an embodiment of the present application;

[0041] Figure 4 A flowchart of a distributed file access method proposed in an embodiment of the present application;

[0042] Figure 5 A schematic diagram of the encryption strategy configuration in the method provided in Example 1 of the present application;

[0043] Figure 6 A schematic diagram of file access request and data transmission in the method provided in Example 1 of the present application;

[0044] Figure 7 A schematic diagram of client distributed key management and access control in the method provided in Example 1 of the present application;

[0045] Figure 8 A schematic diagram of distributed key management and access control on the server side in the method provided in Example 1 of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0047] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0048] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refers to two or more systems, and multiple terminals refers to two or more terminals.

[0049] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0050] In the description of the embodiments of the present application, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0051] In the description of the embodiments of the present application, the terms "first\second\third, etc." or module A, module B, module C, etc. are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that the specific order or sequence can be interchanged where permitted so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0052] Relevant terms involved in the embodiments of this application:

[0053] The file system is the system software responsible for managing and storing files. The operating system accesses the source file data through the access interface provided by the file system, and the user accesses the source file on the disk through the operating system. The access interface can be an I / O interface or a read / write interface.

[0054] Distributed file system can be understood as: one computer cannot store a large number of files, so several computers are organized through the network to store a large number of files and receive requests from a large number of users. These organized computers communicate through the network. Distributed file system is software that provides data interoperability between multiple devices, which can effectively solve the problem of limited storage space on a single device and data management between multiple devices.

[0055] Distributed files are data that are exchanged between multiple devices, including source file data, source file metadata, and copies of source files distributed on other devices.

[0056] The unified meta adapter is used to translate heterogeneous raw metadata into a unified format and description.

[0057] In the description of the embodiments of the present application, the numbers representing the steps, such as S110, S120, etc., do not necessarily mean that the steps will be executed in this manner. If permitted, the order of the previous and next steps can be interchanged, or they can be executed simultaneously.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0059] A related solution provides a network file system NFS. The network file system NFS establishes a corresponding communication protocol by adapting the standard file system access interface. Users can access files on other devices just like accessing local files. However, after the source file data on the server is transmitted to the client device through the network, the encryption policy of the original file becomes invalid on the client device, and the server device cannot control the access policy of the file data across the end; and the data cache of the client device cannot be locally protected according to the encryption policy of the original file on the server when it is stored on the disk locally; at the same time, the data interaction between the client device and the server device is generally simply forwarded through the network according to the file system interface. Due to the small amount of data, the network bandwidth cannot be effectively utilized, and the access performance is low.

[0060] The embodiments of the present application provide a distributed file access method, device and system, wherein the distributed file access method transmits the security access level of the file through a unified extended attribute, so that the encryption policy of the client device for file caching is consistent with the provider of the source file, thereby realizing a consistent security encryption policy across devices; by classifying the access operations, the processing scenario can be identified, and the intelligent transmission of distributed files and on-demand caching can be realized according to the processing scenario, thereby improving the access performance of distributed files.

[0061] Figure 1 This is a schematic diagram of the hardware system configuration provided in the embodiment of the present application. Figure 1As shown, the hardware system configuration of the embodiment of the present application includes a client device 1, a server device 2 and a server 3. The server device 2 is the owner of the source file, and the client device 1 caches and accesses the source file of the server device 2 through the network. The server 3 performs user authentication and network access control on the client device 1 and the server device 2 respectively. Among them, the client device 1 is a terminal device with both computing and storage capabilities, including a tablet, a mobile phone, a desktop computer, etc. The server device 2 is a terminal device with both computing and storage capabilities, including a tablet, a mobile phone, a desktop computer, etc.

[0062] In the embodiment of the present application, a network connection is established between the client device 1 and the server device 2 through the server 3, and when the connection is established, it is checked whether the encryption policies of the files between the devices match. The network connection between the client device 1 and the server device 2 includes WIFI, mobile Internet, etc., and the embodiment of the present application does not limit the specific access method of the network.

[0063] It is understandable that a terminal device can be a client or a server. When the terminal device acts as a provider of source files, the terminal device is a server; when the terminal device requests to access source files, the terminal device is a client.

[0064] Figure 2 This is a diagram of the software application system architecture provided in the embodiment of this application. Figure 2 As shown, the software application system of the terminal device adopts a stacked architecture, including an application layer APPS20, which is used to provide user-oriented application services and an I / O interface for access requests to a distributed file system 21; a distributed file system 21, which is used to transmit data and provide a globally unified distributed key service according to the processing scenario of the access request; in order to be compatible with different file systems and different operating system platforms, the distributed file system 21 performs unified metadata adaptation on the metadata of the source files of the underlying file system 22, and converts the heterogeneous original metadata into a string with a unified format and description.

[0065] The metadata is data describing the storage mode and organizational management structure of the source files of the underlying file system 22. In the embodiment of the present application, the metadata includes the following information:

[0066] (1) Directory entry information: includes the name and ID of the directory entry contained in each directory in the directory tree; a directory entry includes a subdirectory or a file, and all files in the directory tree are identified by a globally unique file ID; illustratively, the file ID of the root directory is fixed at 2.

[0067] (2) Attribute information: including the creation time of each directory, the creation time of files, the modification time of directories or files, and the number and size of file copies.

[0068] (3) Location information: including the local hard disk where the file is stored.

[0069] Each of the above information is identified by a different globally unique character or string.

[0070] Since different file systems or device platforms have inconsistent metadata structures for file caches, which makes access management inconvenient, in the embodiments of the present application, the metadata of files in the underlying file system needs to be in a unified format.

[0071] In the embodiment of the present application, the distributed file system completes the unified meta-adaptation of the underlying file metadata through the data provider-server. The unified meta-adapter shields the different implementation details of the heterogeneous metadata in the underlying file system, so that the metadata of the source file in the underlying file system has the same data structure expression at the distributed file system layer.

[0072] After unified meta-adaptation, the character strings carrying directory entry information, attribute information, location information and other information are arranged according to a unified data structure to facilitate the normalized management of distributed files across platforms and systems.

[0073] Exemplarily, the metadata of the unified meta-adapted file is arranged as follows: directory entry information+attribute information+location information.

[0074] The underlying file system 22 is system software responsible for managing and storing distributed files.

[0075] Distributed files are data exchanged between multiple devices, including source file data, source file metadata, and source file copies. The source file data and / or source file copies in distributed files need to be protected by encryption policies on terminal devices for data security.

[0076] Exemplarily, on the underlying file system 22, the protection of source file A, source file B, and source file C requires different security levels. Strategy 1, Strategy 2, and Strategy 3 are three encryption strategies with different security levels. Source file A, source file B, and source file C are protected by the keys generated by Strategy 1, Strategy 2, and Strategy 3, respectively.

[0077] Figure 3 A schematic diagram of an application scenario of a distributed file system provided in an embodiment of the present application. The application scenario is based on Figure 1 The hardware system configuration shown and Figure 2 The software system architecture in the terminal device is shown, and the relevant introduction of the concepts, units, etc. involved in the following content can be found above.

[0078] like Figure 3As shown in FIG. 1 , the distributed file system includes an intelligent data transmission component and a distributed key component. In the following content, the client is referred to as the client, and the server is referred to as the server.

[0079] Among them, on the client side, the intelligent data transmission component 311 includes a request determination unit, a scene recognition unit, and an intelligent transmission unit.

[0080] The request determination unit determines source file information and an access operation requested for access.

[0081] The source file information requested for access includes the ID of the source file requested for access, the device information where the source file is located, the logical block address (LBA) or physical address stored in the hard disk of the device, and the like.

[0082] Access operations include opening, reading, copying, adding, deleting, modifying, and checking the files requested for access.

[0083] The scenario recognition unit determines a processing scenario according to the access operation, where the processing scenario includes delayed transmission or combined transmission.

[0084] In order to reduce the number of interactions with the server, the client may cache one or more request messages according to the processing scenario, and delay or merge the transmission of one or more request messages.

[0085] In some practicable implementations, the processing scenario may be determined to be delayed transmission according to one of the following situations: the access operation to the file is an access operation that does not carry data, or there is a subsequent association request.

[0086] Exemplarily, if the access operation is to open a file, and opening the file only determines the attribute information of the file without reading the source file data, it is not necessary to send network request information, cache the request information, and delay the transmission.

[0087] Exemplarily, if the current access operation is to open a file, and the subsequent associated request is to read the file content, and the purpose of opening the file is to subsequently read the file content, the access request for opening the current file may be delayed for transmission.

[0088] In some practicable implementations, the processing scenario is determined to be combined transmission according to the following situation: there are one or more associated requests, and the one or more associated requests are one or more previously cached request messages for delayed transmission.

[0089] Exemplarily, if the current access operation is to open a file, and the subsequent access operation is to read the file, and the objects to be opened and read are the same source file, the access request to open the file can be cached and the transmission can be delayed. Subsequently, the access request to read the file and the request to open the file can be merged into one access request, thereby sending one network request to process multiple access operations.

[0090] To reduce the number of interactions between the client and the server, the underlying file system can cache one or more request messages.

[0091] In some feasible implementations, the file processing scenario may also be local processing, and there is no need to send a request message.

[0092] Exemplarily, in a scenario where the attribute information of the source file is valid in the local cache and the "open" operation is only for obtaining the attribute information of the source file, the cache of the attribute information of the source file can be read locally without sending access request information.

[0093] In some feasible implementations, when the copy of the source file is valid in the local cache, the access operation processing scenario for the file can also be synchronous processing, that is, when the client device modifies the copy content of the source file, the server modifies the source file content.

[0094] The intelligent transmission unit sends a request message to the server according to the processing scenario, and the request message includes the source file information requested to be accessed.

[0095] If the processing scenario is delayed transmission, the request message is cached locally to delay transmission, and the number of request messages cached locally to delay transmission may be one or more.

[0096] If the processing scenario is combined transmission, one or more delayed transmission request messages are combined with the current request message, and the combined request message is sent. It can be understood that the one or more delayed transmission request messages combined with the current request message are mutually associated association requests.

[0097] On the service side, the intelligent data transmission component 321 includes a request parsing unit, a unified meta adapter, a strategy determination unit, and a batch processing unit.

[0098] The request parsing unit receives the request message from the client and obtains the source file information requested for access.

[0099] In a scenario where the request message is a merged request message, the merged request message is parsed to obtain information of multiple source files requested to be accessed.

[0100] The unified metadata adapter obtains metadata of the source file according to the source file information requested for access, and performs unified metadata adaptation on the format of the metadata of the source file.

[0101] In the embodiment of the present application, the unified meta adapter shields the different implementation details of the heterogeneous metadata in the underlying file system so that the metadata of the source file in the underlying file system has the same data structure expression at the distributed file system layer.

[0102] Metadata with the same data structure can easily carry extended attributes of different platforms or file systems and establish a unified encryption strategy for distributed files.

[0103] The policy determination unit obtains the extended attribute information of the source file and performs unified meta-adaptation on the extended attribute information; the extended attribute information includes a first encryption policy, and the first encryption policy includes an encryption algorithm of the file and an encryption level of the device. The description information of the encryption level of the device and the encryption algorithm of the file is stored in the distributed key service component 322.

[0104] In some feasible implementations, the extended attribute information further includes a key ID, and the key ID is used to index the master key.

[0105] The server of the distributed file system can generate a unique key ID through a hash algorithm and output it to the client. The client verifies the encryption policy based on the key ID.

[0106] The file encryption algorithm, device encryption level, key ID, etc. are all identified by globally unique characters or strings.

[0107] In some feasible implementations, the unified meta-adapter performs unified meta-adaptation on the extended attributes of different source files so that the extended attributes of the source files have the same data structure expression, which facilitates the transmission and indexing of globally consistent encryption policies in different platforms or file systems.

[0108] Exemplarily, the data structure of the extended attribute of the source file is arranged as follows: encryption algorithm+encryption level+key ID.

[0109] The batch processing unit performs batch processing on the merged request messages, and the batch processing includes obtaining metadata of multiple source files; performing unified meta-adaptation on the formats of the metadata of multiple source files to obtain metadata information of multiple source files in a unified format; obtaining extended attribute information of multiple source files; for multiple different source files, performing unified meta-adaptation on different extended attribute information to obtain multiple extended attributes of the same data structure, and the extended attributes include encryption policies of the source files; obtaining copies of multiple source files, and the copies of each source file are protected by the encryption policy of the original source file for data security.

[0110] In some possible implementations, batch processing includes performing consecutive or associated operations on multiple associated requests for the same source file.

[0111] By adopting the method provided in the embodiment of the present application, batch processing of merged request messages can perform centralized and continuous access operations on the same access object, eliminate redundant indexing, encryption and decryption operations in the process of data access to the source file, reduce the number of interactions with the client, improve access performance and efficiency, and enhance user experience.

[0112] The batch processing unit is also used to send a reply message to the client; the reply message includes metadata information of the source file, extended attribute information and a copy of the source file.

[0113] Exemplarily, the data structure of the reply message may be: a copy of the source file+metadata information of the source file+extended attribute information.

[0114] In some feasible implementations, the results of batch processing can be assembled to obtain a reply message; the results of batch processing include metadata information of multiple source files, extended attribute information of multiple source files and multiple source file copies, and the multiple source file copies are protected by their own encryption strategies for data security.

[0115] Exemplarily, the data structure of the reply message may be: a copy of source file 1 + metadata information of source file 1 + extended attribute information of source file 1; ...; a copy of source file n + metadata information of source file n + extended attribute information of source file n.

[0116] By adopting the method provided in the embodiment of the present application, the server carries the extended attribute information describing the source file encryption policy when sending the reply information, so as to establish a globally consistent encryption policy on the client, realize hierarchical protection of distributed file data, refine the protection granularity of distributed files, and improve the security of data on different devices.

[0117] On the client side, the intelligent data transmission component 311 further includes a receiving unit and a policy encryption unit.

[0118] The receiving unit receives a reply message from the server, where the reply message includes metadata information of the source file, extended attribute information, and a copy of the source file.

[0119] The extended attribute information includes an encryption policy, according to which the encryption algorithm of the file and the encryption level of the device are determined, and the extended attribute information also includes a key ID, according to which the master key is indexed.

[0120] In some feasible implementations, the client's underlying file system can cache the source file's metadata information and copies of the source file, and organize and manage the source file's copies according to the received source file's metadata information, thereby establishing a directory tree consistent with the server.

[0121] Exemplarily, the reply message includes a copy of source file A, a copy of source file B, and a copy of source file C. The client's underlying file system saves them in the form of cache A, cache B, and cache C, and organizes and manages them according to the metadata information of the received source files.

[0122] In order to free up data space in memory, the client will also cache the received file copies and store the source file copies on the local hard disk.

[0123] The policy encryption unit encrypts and saves the source file copy according to the encryption policy carried in the extended attribute information. The encryption policy includes the encryption algorithm of the file consistent with the server and the encryption level of the device.

[0124] Exemplarily, in the underlying file system, cache A, cache B and cache C are encrypted using strategy 1, strategy 2 and strategy 3 respectively, and strategy 1, strategy 2 and strategy 3 are encryption strategies carried in the extended attribute information that are consistent with the server.

[0125] The intelligent data transmission component 311 also performs unified metadata adaptation on the metadata of the underlying file system to obtain metadata in the same format.

[0126] On the client side, the distributed key service component 312 includes a distributed key management unit and a policy setting unit.

[0127] The distributed key management unit (Key Management Service) can create its own master key on demand and use the master key to generate, encrypt and decrypt data keys.

[0128] The master key is an encryption key of a key, mainly used to encrypt and protect the data key. A user master key can encrypt multiple data keys. The master key of client 1 can be recorded as the first master key.

[0129] The policy setting unit loads an encryption policy consistent with the server according to the encryption policy information carried in the extended attribute information; and indexes the first master key according to the key ID information carried in the extended attribute information. In this way, the present application can provide a globally unified data key for the source file copy for different terminal devices.

[0130] In some feasible implementations, the client's access operation to the cached source file copy of the underlying file system or the local source file copy written to the disk can first associate the key ID, load the first master key, generate a data key, decrypt the source file copy according to the data key, and then open, read, copy, add, delete, modify, check, etc. the file requested to be accessed.

[0131] The method provided in the embodiment of the present application transmits the encryption policy of the original file through extended attributes, establishes an encryption policy on the client that is consistent with the server, realizes hierarchical protection of distributed file data, can refine the protection granularity of distributed files, and improves data security.

[0132] On the service side, the distributed key component 322 also includes a distributed key management unit and a policy setting unit.

[0133] The distributed key management unit can create a master key of the server as needed, and use the master key to generate, encrypt and decrypt data keys; the master key of the server 2 can be recorded as the second master key.

[0134] The distributed key management unit can also generate a unique key ID through a hash algorithm. The key ID is used to verify the encryption policy and can also be used to index the master key.

[0135] The policy setting unit configures and manages encryption policies, which include the encryption algorithm for files and the encryption level for devices.

[0136] The distributed file system provided in the embodiment of the present application, under a stacked architecture, delays transmission and merges transmission for access to distributed file data in combination with processing scenarios, thereby achieving high-performance data communication; normalization of metadata formats of different file systems is achieved through a unified meta-adapter, thereby optimizing the data structure of file metadata to adapt to hot spot operations for file access; encryption policies for original files are transmitted through metadata and extended attributes, and encryption policies that are globally consistent with those on the server are established on the client, thereby achieving global security protection of distributed file data and improving data security.

[0137] The above is the introduction to the architecture and application scenarios of the distributed file system provided by the embodiment of the present application. Next, based on the above content, the access method of the distributed file provided by the embodiment of the present application is introduced. Among them, the introduction to the concepts, units, etc. involved in the following content can be found above.

[0138] The distributed file access method proposed in the embodiment of the present application is applied to the client, and the client establishes an end-to-end link with the server. The server is the owner of the source file, and the client accesses the source file of the server through the network; the method includes: determining the source file information requested to be accessed; sending a request message to the server, the request message includes the source file information requested to be accessed; receiving a reply message from the server, the reply message includes the metadata of the source file, extended attribute information and a copy of the source file; the extended attribute information includes the first encryption policy of the server; determining a second encryption policy based on the extended attribute information; the second encryption policy is consistent with the first encryption policy; locally managing the copy of the source file based on the metadata of the source file; encrypting and / or decrypting the locally managed copy of the source file based on the second encryption policy.

[0139] The distributed file access method proposed in the embodiment of the present application is also applied to the server, including: parsing the request message to obtain the source file information requested for access; obtaining a copy of the source file and the metadata of the source file based on the source file information; determining extended attribute information based on the source file information, the extended attribute information including a first encryption policy; sending a reply message to the client, the reply message including the metadata of the source file, the extended attribute information and a copy of the source file.

[0140] The distributed file access method proposed in the embodiment of the present application is described below with reference to the flowchart.

[0141] Figure 4 This is a flow chart of the distributed file access method proposed in the embodiment of the present application. It can be understood that the method can be executed by any terminal or platform with computing and processing capabilities. The distributed file access method is applied to the client, and the client establishes an end-to-end link with the server. The server is the owner of the source file, and the client accesses the source file of the server through the network; the method includes the following steps:

[0142] S11, determining the source file information requested for access.

[0143] The source file information requested for access includes information of the source file requested for access on the server, a logical block address (LBA) or a physical address stored in a hard disk on the server, and the like.

[0144] In some practicable implementations, determining the source file information requested for access also includes determining an access operation on the source file requested for access, including one or a combination of operations such as opening, reading, copying, adding, deleting, modifying, and checking the source file.

[0145] In some practicable implementations, the source file information and access operation requested for access may be transmitted to the distributed file system through the data I / O interface of the system.

[0146] Exemplarily, the user inputs the source file information and access operation requested for access through the application layer APPS20 of the client, and transmits the source file information and access operation requested for access to the distributed file system 21 through the data I / O interface.

[0147] S12, determining a processing scenario according to the access operation, where the processing scenarios include delayed transmission and combined transmission.

[0148] In some feasible implementations, the processing scenario is classified and identified according to the file access operation and the input parameters to determine whether the processing scenario is to be delayed or merged.

[0149] In some practicable implementations, the processing scenario may be determined to be delayed transmission according to one of the following situations: the I / O interface of the access operation does not read the source file content data, or there is a subsequent associated request.

[0150] Exemplarily, if the access operation is to open a source file, it is not necessary to send network request information, and the request information can be cached locally to delay transmission.

[0151] Exemplarily, if the current access operation is to open a source file, and the subsequent associated request is to read the source file content, the access request for the current open file may be transmitted with delay.

[0152] In some practicable implementations, the processing scenario is determined to be combined transmission according to the following situation: there are one or more associated requests, and the one or more associated requests include one or more delayed transmission request messages cached locally.

[0153] For example, if the current access operation is to open a file, and the operation goal is to read the contents of the source file on the server, and the subsequent access operation is to read the source file, which will also be executed quickly, the access request to open the file can be delayed for transmission. At the same time, the access request to read the file and the request to open the file can be merged into one access request, so as to send one network request to handle multiple access operations in a distributed scenario.

[0154] In some feasible implementations, the file access operation processing scenario may also be processed locally, and there is no need to send a request message.

[0155] For example, in a scenario where a file is opened only to obtain attribute information of the file, and the attribute information of the file is valid in the local cache, it is not necessary to send access request information.

[0156] In some achievable implementations, the file access operation processing scenario may also be synchronous processing, that is, when the client device modifies the copy content of the source file, the server modifies the source file content. .

[0157] S13, sending a request message to the server according to the processing scenario, where the request message includes source file information requested to be accessed.

[0158] If the processing scenario is delayed transmission, the request message is cached locally to delay transmission, and the number of request messages cached locally to delay transmission may be one or more.

[0159] If the processing scenario is merged transmission, one or more delayed transmission request messages are merged with the current request message, and the merged request message is sent.

[0160] It can be understood that one or more delayed transmission request messages and the current request message are mutually associated association requests.

[0161] In some possible implementations, the request message includes access operation information for the source file requested to be accessed.

[0162] S14, receiving a reply message from the server, the reply message including metadata information of the source file of the server, extended attribute information and a copy of the source file, wherein the extended attribute information includes the first encryption policy of the server.

[0163] The metadata of the source file is data describing the storage method and organizational management structure of the source file on the server side, and the metadata includes directory item information, attribute information, location information, etc. The above information is identified by different globally unique characters or strings.

[0164] In some feasible implementations, the first encryption strategy of the server includes an encryption algorithm of the file and an encryption level of the device.

[0165] In some feasible implementations, the extended attribute information also includes a key ID, and the key ID is generated by the server through a hash algorithm based on the encryption algorithm of the file and the encryption level of the device.

[0166] S15, determining a second encryption policy according to the extended attribute; the second encryption policy is consistent with the first encryption policy.

[0167] The client configures the encryption algorithm of the file and the encryption level of the device according to the second encryption policy.

[0168] S16, locally managing the copy of the source file according to the metadata of the source file.

[0169] In some feasible implementations, the client's underlying file system can cache the metadata information of the source file and the copy of the source file, and organize and manage the copy of the source file based on the received metadata information of the source file, thereby establishing a directory tree consistent with the server, facilitating indexing and query of distributed file access.

[0170] Exemplarily, the reply message includes a copy of source file A, a copy of source file B, and a copy of source file C. The client's underlying file system saves them in the form of cache A, cache B, and cache C, and organizes and manages them according to the metadata information of the received source files.

[0171] In order to free up data space in memory, the client will also cache the received file copies and store the source file copies on the local hard disk.

[0172] S17, encrypt and / or decrypt the copy of the source file according to the second encryption strategy.

[0173] In some practicable implementations, the client encrypts and stores the source file copy according to a second encryption policy, wherein the second encryption policy includes an encryption algorithm for the file and an encryption level for the device consistent with those of the server.

[0174] Exemplarily, the client uses strategy 1, strategy 2 and strategy 3 to encrypt cache A, cache B and cache C respectively in the underlying file system. Strategy 1, strategy 2 and strategy 3 are encryption strategies consistent with the server determined according to the extended attribute information.

[0175] Before encrypting and / or decrypting the copy of the source file, the method of the embodiment of the present application also includes: creating a first master key according to the second encryption strategy, and using the first master key to generate, encrypt and decrypt the first data key. In some achievable implementations, the client can index the first master key according to the key ID information carried in the extended attribute information. The client's access operation to the cached copy of the source file of the underlying file system or the local copy of the source file can first associate the first master key, call the first master key, generate the first data key, decrypt the copy of the source file according to the first data key, and then open, read, copy, add, delete, modify, check, etc. the file requested for access.

[0176] The distributed file access method provided in the embodiment of the present application is also applied to the server, such as Figure 4 As shown, the access to the source file on the server is performed according to the following steps:

[0177] S21, receiving a request message, obtaining source file information requested by the client, the source file information including information such as a logical block address (LBA) or a physical address stored in a hard disk of the device.

[0178] In some feasible implementations, the request message also includes an access operation to the source file, and the access operation includes one or a combination of operations such as opening, reading, copying, adding, deleting, modifying, and checking the source file.

[0179] In a scenario where the request message is a merged request message, multiple associated requests may be obtained by parsing the merged request message, and the multiple associated requests include multiple pieces of source file information requested to be accessed.

[0180] In some practicable implementations, the multiple source file information requested for access include multiple consecutive or associated access operations for the same source file.

[0181] S22, obtaining metadata of the source file and a copy of the source file according to the source file information requested for access. The metadata of the source file is data describing the storage method and organizational management structure of the source file on the server side, and the metadata includes directory item information, attribute information, and location information. Each of the above information is identified by different globally uniquely defined characters or strings.

[0182] In some practicable implementations, the format of the metadata of the source file may be uniformly adapted.

[0183] For different devices, a unified meta adapter can be used to shield different implementation details of heterogeneous metadata in the underlying file system, so that the metadata of the source file in the underlying file system has the same data structure expression at the distributed file system layer.

[0184] Metadata with the same data structure can easily carry extended attributes of different platforms or file systems and establish a unified encryption strategy for distributed files.

[0185] S23, determining the extended attribute information of the source file according to the source file information, the extended attribute information includes a first encryption strategy, the first encryption strategy includes the encryption algorithm of the file and the encryption level of the device. The description information of the encryption level of the device and the encryption algorithm of the file is stored in the distributed key service component 322.

[0186] In some feasible implementations, the encryption attribute of the source file may be read, and the extended attribute information of the source file may be determined according to the encryption attribute of the source file.

[0187] In some feasible implementations, the extended attribute information further includes a key ID; the key ID is generated by the server through a hash algorithm and output to the client; the client performs verification and indexing of the encryption strategy according to the key ID.

[0188] In some feasible implementations, the key ID may also be generated by the client through a hash algorithm and output to the server, and the server performs verification and indexing of the encryption strategy according to the key ID.

[0189] In the embodiment of the present application, the encryption algorithm of the source file, the encryption level of the device, the key ID, etc. are all identified by globally unique characters or strings.

[0190] In some feasible implementations, the unified meta-adapter performs unified meta-adaptation on the extended attributes of different source files so that the extended attributes of the source files have the same data structure expression, which facilitates the transmission and indexing of globally consistent encryption policies in different platforms or file systems.

[0191] Exemplarily, the data structure of the extended attribute of the source file is arranged as follows: the encryption algorithm of the file+the encryption level of the device+the key ID.

[0192] In some feasible implementations, the merged request messages can be processed in batches, and the batch processing includes obtaining metadata of multiple source files; performing unified meta-adaptation on the formats of the metadata of multiple source files to obtain metadata information of multiple source files of the same data structure; obtaining extended attribute information of multiple source files; for multiple different source files, performing unified meta-adaptation on different extended attribute information to obtain multiple extended attributes of the same data structure; obtaining copies of multiple source files, and each copy of the source file is protected by the original first encryption strategy for data security.

[0193] Batch processing of merged request messages also includes centralized and continuous access operations for multiple access operations on the same source file, so as to eliminate redundant verification, caching, indexing, encryption and decryption operations in the process of data access to the same source file, which is conducive to reducing memory consumption, reducing the number of interactions with the underlying file system and distributed key service components, improving access performance and efficiency, and enhancing user experience.

[0194] S24, sending a reply message to the client, the reply message including the metadata of the source file, extended attribute information and a copy of the source file.

[0195] In some feasible implementations, the metadata of multiple source files in a unified format and the extended attributes of each source file may be combined and spliced ​​to obtain a reply message.

[0196] Exemplarily, the data structure of the reply message may be: a copy of the source file+metadata information of the source file+extended attribute information.

[0197] In some feasible implementations, the results of batch processing can be assembled to obtain a reply message; the results of batch processing include metadata information of multiple source files, extended attribute information of multiple source files, and multiple source file copies, and the multiple source file copies are protected by their respective first encryption strategies for data security.

[0198] Exemplarily, the data structure of the reply message may be: a copy of source file 1 + metadata information of source file 1 + extended attribute information of source file 1; ...; a copy of source file n + metadata information of source file n + extended attribute information of source file n.

[0199] Before receiving the request message, the method of the embodiment of the present application also includes: the server configures and verifies a first encryption policy, the first encryption policy includes an encryption algorithm for the file and an encryption level for the device; creates a second master key according to the first encryption policy, and the second master key is used to generate, encrypt and decrypt a second data key.

[0200] Example 1

[0201] The following describes the distributed file access method proposed in Example 1 of the present application in conjunction with a specific implementation method. For the introduction of the concepts, units, etc. involved in the following content, please refer to the above.

[0202] In the security configuration stage, the distributed file access method proposed in Example 1 of the present application adopts a globally unified encryption strategy for different platforms or file systems to implement the configuration of cross-device encryption strategies.

[0203] Figure 5 This is a schematic diagram of the encryption strategy configuration in the method provided in Example 1 of this application. Figure 5 As shown in the figure, the configuration of the cross-device encryption policy is described from the client and server aspects respectively.

[0204] On the client side:

[0205] S511, complete user security authentication.

[0206] Exemplarily, the transport layer security protocol can be used to perform server identity authentication and encrypt server communication. Exemplarily, the client can be one of the devices such as a tablet, a mobile phone or a computer, and the client logs in to a unified cloud account to complete user security authentication.

[0207] S512, mount the distributed file system provided in the embodiment of the present application.

[0208] S513, determine the server information, connect to server 3, establish a network connection with the server, and data can be transmitted between the client and the server. The server is the owner of the source file.

[0209] For example, the network connection between the client and the server can be WIFI or mobile Internet, and the protection of the transport layer can use SSL (TLS 1.0). In RDP5.2 and higher versions, the security level of the transmission includes Federal Information Processing Standard (FIPS), high, client compatible, low and other security levels. This application does not limit the specific access method of the network and the protection security level of the transport layer.

[0210] S514, the client obtains information about a first encryption policy from the server through a network, where the information about the first encryption policy includes information about an encryption algorithm for the file and information about an encryption level of the device.

[0211] File encryption algorithms include DES encryption algorithm, AES encryption algorithm, RSA encryption algorithm, Base64 encryption algorithm, MD5 encryption algorithm, SHA1 encryption algorithm, etc.

[0212] The encryption level of the device is a customized protection level. The server can set the encryption level from low to high according to the usage scenario, including accessible at boot (not logged in), accessible at first login, lock screen cannot be written or read; lock screen can be written but not read, etc.

[0213] S515, verify the compatibility of the encryption policy and create a first master key.

[0214] In some achievable implementations, the client generates a key ID through a hash algorithm, and sends the key ID as verification information to the server to verify the encryption policy. The client determines whether it supports the encryption policy of the server based on the returned verification result. If the verification succeeds, it means that the client supports the encryption policy of the server, and the client creates a first master key that matches the first encryption policy of the server; if the verification fails, it means that the encryption policies of the two devices are incompatible, and the link is disconnected.

[0215] Exemplarily, if the client does not support the encryption algorithm of the server, the connection is disconnected; or if the client does not support the encryption level of the server, the connection is disconnected.

[0216] On the server side:

[0217] S521, complete user security authentication.

[0218] In some practicable implementations, the transport layer security protocol can be used to perform server identity authentication and encrypt server communications. For example, the server can be one of the devices such as a tablet, a mobile phone or a computer. The server logs in to a unified cloud account, mounts a distributed file system, and completes user security authentication.

[0219] S522, mount the distributed file system provided in the embodiment of the present application.

[0220] S523, linking to the server, the server establishes a network connection with the client and monitors the request message sent by the client.

[0221] When the server uses SSL (TLS1.0) to protect the communication between the client and the server during a native Remote Desktop Protocol (RDP) connection, a certificate is required to authenticate the server. You can select a certificate that is already installed on the terminal server, or you can use the default self-signed certificate.

[0222] For example, the network connection between the client and the server can be WIFI or mobile Internet, and the protection of the transport layer can use SSL (TLS 1.0). In RDP5.2 and higher versions, the security level of the transmission includes Federal Information Processing Standard (FIPS), high, client compatible, low and other security levels. This application does not limit the specific access method of the network and the protection security level of the transport layer.

[0223] S524, configuring a local first encryption strategy, including determining an encryption algorithm for the file and an encryption level for the device.

[0224] In some practicable implementations, the server receives the key ID of the client, verifies whether the local encryption policy is compatible with the encryption policy of the client based on the key ID of the client, and sends the first encryption policy to the client if the verification result is successful;

[0225] If the verification result fails, a verification failure message is sent to the client.

[0226] In some achievable implementations, the key ID may be generated by a distributed key management unit of the server, and the server sends the key ID as verification information to the client, and the client verifies the encryption policy. The client verifies whether the local encryption policy is compatible with the encryption policy of the server based on the key ID. If the verification result is successful, a connection is established; if the verification result is failed, a verification failure message is sent to the server and the connection is disconnected.

[0227] S525, the server creates a second master key according to the first encryption strategy.

[0228] At this point, the client and server have implemented a global unified security configuration across devices. The encryption policy is set by the server that provides the data. The client needs to control file access and data transmission through the distributed file system and the loading status of the distributed key. Next, we will introduce file access requests and data transmission in detail.

[0229] Figure 6 Schematic diagram of file access request and data transmission in the method of the embodiment of the present application. Figure 6 As shown, the two aspects of sending request information from the client and processing the message from the server are described respectively.

[0230] The process of the client sending a request message includes the following steps:

[0231] S516, determining the source file information requested for access, where the source file information includes the server information where the source file exists.

[0232] The user inputs the source file information and access operation requested for access in the application layer APPS20 of the client. The source file information requested for access includes the server ID information of the source file requested for access, the logical address (logical block address, LBA) or physical address stored in the device hard disk, the file name, the basic attributes of the file, the file extension attributes, etc.; the server information where the source file exists can be determined based on the source file information requested for access.

[0233] In one practicable implementation, determining the source file information requested for access includes determining an access operation on the source file requested for access, such as one or a combination of operations such as opening, reading, copying, adding, deleting, modifying, and checking the source file.

[0234] S517, determining a processing scenario according to the access operation, where the processing scenarios include delayed transmission and combined transmission.

[0235] The identification of processing scenarios includes the following situations:

[0236] a. If the access operation is to open a file, and opening the file only determines the attribute information of the file without reading the source file data, it is not necessary to send the network request information, cache the request information, and delay the transmission.

[0237] b. If the current access operation is to open a file, and the subsequent associated request is to read the file content, and the purpose of opening the file is to subsequently read the file content, the access request for the current open file can be cached and transmitted later.

[0238] c. If the current access operation is to open a file, and the subsequent access operation is to read the file, and the objects to be opened and read are the same source file, the access request to open the file can be cached and the transmission can be delayed. The access request to read the file and the request to open the file can be subsequently merged into one access request, so that one network request can be sent to process multiple access operations. This processing scenario is merged transmission.

[0239] d. If there are one or more associated requests, and the one or more associated requests are one or more delayed transmission request messages cached locally, the processing scenario is combined transmission.

[0240] Processing scenarios also include local processing or synchronous processing.

[0241] e. If the attribute information of the source file is valid in the local cache, and the source file is opened only to obtain the attribute information of the source file, it can be processed locally. This processing scenario is local processing.

[0242] f. If the local cache of the copy of the source file is valid, the server modifies the content of the source file while the client device modifies the copy of the source file, and the access operation to the file is synchronous processing. This processing scenario is synchronous processing.

[0243] S518: Send a request message to the server according to the processing scenario, where the request message includes source file information requested to be accessed.

[0244] If the processing scenario is delayed transmission, the request message is cached locally to delay transmission, and the number of request messages cached locally to delay transmission may be one or more.

[0245] If the processing scenario is merge transmission, one or more delayed request messages are merged with the current request message and the merged request message is sent. The message processing process on the server includes the following steps:

[0246] S526, after receiving the request message, the server parses the request message and obtains the source file information requested for access. The source file information includes the server information where the source file requested for access is located, the logical block address (LBA) or physical address stored in the device hard disk, the file name, the basic attributes of the file, the extended attributes of the file, etc.

[0247] If the request message is a merged request message, multiple associated requests can be obtained by parsing the request message, and each associated request includes a source file information.

[0248] S527, obtaining metadata of the source file according to the source file information requested for access, and performing unified metadata adaptation on the format of the metadata of the source file.

[0249] The unified meta adapter shields different implementation details of heterogeneous metadata in the underlying file system, so that the metadata of the source files in the underlying file system have the same data structure expression at the distributed file system layer.

[0250] Metadata with the same data structure can easily carry extended attributes of different platforms or file systems and establish a unified distributed metadata encryption strategy.

[0251] In some feasible implementations, it is determined whether the request message carries an association request message. If so, metadata information required by the association request message is read and unified metadata adaptation is performed.

[0252] S528, determining the extended attribute information of the source file according to the source file information requested for access, the extended attribute information including the local encryption policy of the server, and the encryption policy including the encryption algorithm of the file and the encryption level of the device.

[0253] In some achievable implementations, the encryption attribute of the source file may be determined based on the source file information; the extended attribute information of the source file may be determined based on the encryption attribute of the source file, and the extended attribute information may include the encryption algorithm of the file and the encryption level of the device.

[0254] In some achievable implementations, the extended attribute information also includes a key ID, which is used to describe the name and index information of the master key, and is also used to verify the encryption policy. The master key includes a first master key and a second master key.

[0255] In the extended attribute information, the encryption algorithm of the file, the encryption level of the device, and the key ID are all identified by globally unique characters or strings.

[0256] In the embodiment of the present application, the encryption level of the device and the description information of the encryption algorithm of the file are stored in the distributed key service component 322, and the distributed file system can generate a key ID through a hash algorithm.

[0257] In some feasible implementations, the encryption algorithm of the file, the encryption level of the device, the key ID, etc. in the extended attribute information of multiple different source files may be uniformly adapted to obtain the extended attributes expressed in the same data structure.

[0258] If the request message is a merged request message, the merged request messages are batch processed, and the batch processing includes obtaining metadata of multiple source files, where the multiple source file information requested for access includes multiple continuous or associated access operations for the same source file; performing unified meta-adaptation on the formats of the metadata of the multiple source files to obtain metadata information of the multiple source files in a unified format; obtaining extended attribute information of the multiple source files; for multiple different source files, uniformly converting different extended attribute information; obtaining copies of the multiple source files, and the copies of each source file are protected by the original source file encryption policy for data security.

[0259] In some possible implementations, batch processing includes performing consecutive or associated operations on multiple associated requests for the same source file.

[0260] By adopting the method provided in the embodiment of the present application, batch processing of merged request messages can perform centralized and continuous access operations or related operations on the same access object, eliminating redundant query, indexing, encryption, decryption, verification and other operations in the process of secure access to source files, reducing the number of interactions with the client, improving access performance and efficiency, and enhancing user experience.

[0261] S529, sending a reply message; the reply message includes metadata information of the source file, extended attribute information and a copy of the source file.

[0262] In some feasible implementations, the results of batch processing can be assembled to obtain a reply message; the results of batch processing include metadata information of multiple source files in a unified format, extended attribute information of multiple source files in a unified format, and multiple source file copies, and the multiple source file copies are protected by the original encryption strategy for data security.

[0263] When sending a reply message, the server carries the extended attribute information describing the encryption policy of the source file, so as to establish a globally consistent encryption policy on the client, realize hierarchical protection of distributed file data, refine the protection granularity of distributed files, and improve the security of data on different devices. And combined with the classification processing of access operations, efficient data transmission and access are achieved.

[0264] After the reply message is transmitted to the client and cached, the encryption policy of the copy of the source file is determined by the extended attribute information, so that the encryption policy of the copy of the source file on the client can be consistent with that on the server. The client generates a first data key based on the encryption policy, and the client manages the first data key based on the metadata information of the source file. The client needs to control file access through the distributed file system and the loading status of the first master key.

[0265] Figure 7The present invention provides a schematic diagram of distributed key management and access control in the method of the embodiment of the present application. Figure 7 As shown, the following steps of the client are used to implement the distributed key management and access control mechanism.

[0266] S711, receiving a reply message from the server, where the reply message includes metadata information of the source file, extended attribute information, and a copy of the source file.

[0267] The extended attribute information includes a first encryption policy of the server, and the first encryption policy of the server includes an encryption algorithm of the file and an encryption level of the device.

[0268] S712, the client configures a local second encryption policy according to the extended attribute information; the second encryption policy is consistent with the first encryption policy.

[0269] S713, the underlying file system of the client caches the metadata information of the source file and the copy of the source file, thereby reducing the number of interactions between the client and the metadata server.

[0270] Exemplarily, the reply message includes a copy of source file A, a copy of source file B, and a copy of source file C, which exist in the underlying file system in the form of cache A, cache B, and cache C, and are organized and managed according to the metadata information of the received source files.

[0271] In order to free up data space in the memory, the client will also cache the received file copies and store them on the local hard disk.

[0272] S714: Query the first master key according to the key ID, where the key ID is an ID generated by the client through a hash algorithm for the first encryption policy.

[0273] In some feasible implementations, the key ID is the key ID carried in the extended attribute information, and is an ID generated by the server through a hash algorithm for the first encryption strategy.

[0274] S715, determining whether the first master key has been loaded into the distributed file system, if the first master key has been loaded, executing S76; if the first master key has not been loaded, terminating the connection.

[0275] S716, associating with the first master key, and encrypting the source file copy according to the first master key.

[0276] In some achievable implementations, the first master key may be called through an API to generate a first data key; and the source file copy may be encrypted according to the first data key.

[0277] S717, Access a copy of the source file. The file can be accessed based on the following conditions:

[0278] If the source file copy is cached locally, or the source file copy has been saved locally and encrypted, then step S718 is executed. If the source file copy has no data locally, then a request to access the file is sent to the server. Steps S516-S518 are executed.

[0279] S718, sending an access request with the first data key to the local storage device, decrypting the cached source file copy using the first data key, and returning the decrypted data to the local user APPS interface.

[0280] In some feasible implementations, an access request with a first data key may be sent to a local storage device, a copy of the source file stored in the local storage device may be decrypted using the first data key, and the decrypted data may be returned to a local user APPS interface.

[0281] Similarly, if Figure 8 As shown, the server as a source device can implement a distributed key management and access control mechanism through the following steps.

[0282] S721, determine the first encryption strategy, associate it with the corresponding second master key, and generate a second data key.

[0283] In some feasible implementations, the second data key may be generated by calling the second master key through an API.

[0284] S722: Query the second master key according to the key ID, where the key ID is an ID generated by the client through a hash algorithm for the first encryption strategy.

[0285] In some feasible implementations, the key ID may be an ID generated by the server using a hash algorithm for the first encryption strategy.

[0286] S723, associate with the second master key, generate a second data key to encrypt the source file.

[0287] In some achievable implementations, the second master key may be called through an API to generate a second data key; and the source file copy may be encrypted according to the second data key.

[0288] S724, sending an access request with the second data key to the local storage device, decrypting the source file using the second data key, and returning the decrypted data to the local user APPS interface.

[0289] An embodiment of the present application provides a terminal device, including: at least one memory for storing programs; at least one processor for executing the programs stored in the memory, when the program stored in the memory is executed, the processor is used to execute the method provided in any of the above embodiments.

[0290] The embodiment of the present application provides a distributed file access device, which runs computer program instructions to execute the method provided in any of the above embodiments. Exemplarily, the device may be a chip or a processor.

[0291] An embodiment of the present application provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method provided in any of the above-mentioned embodiments.

[0292] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0293] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0294] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0295] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

Claims

1. A distributed file access method, characterized in that: Applied to a client, the client establishes an end-to-end link with a server, the server is the owner of the source file, and the client accesses the source file of the server through a network; the method includes: Determine source file information requested for access, wherein the source file information requested for access includes information on the server side of the source file requested for access; Sending a request message to the server, wherein the request message includes information about the source file requested to be accessed; Receive a reply message from the server, the reply message including metadata of the source file, extended attribute information and a copy of the source file; wherein the extended attribute information includes a first encryption policy of the server, and the metadata of the source file is data describing a storage method and an organizational management structure of the source file of the server; Determine a second encryption strategy according to the extended attribute information; the second encryption strategy is consistent with the first encryption strategy; locally managing a copy of the source file according to the metadata of the source file; The copy of the source file is encrypted and / or decrypted according to the second encryption policy.

2. The method according to claim 1, characterized in that: The determining of the source file information requested for access includes: Determine an access operation to the source file requested for access; the access operation includes one or a combination of the following: open, read, copy, add, delete, modify, check; A processing scenario is determined according to the access operation, where the processing scenario includes delayed transmission and merged transmission.

3. The method according to claim 2, characterized in that The determining of a processing scenario according to the access operation includes: The processing scenario is determined to be combined transmission according to the following situation: there are one or more associated requests, and the one or more associated requests include one or more delayed transmission request messages cached locally.

4. The method according to any one of claims 1 to 3, characterized in that: The first encryption strategy includes an encryption algorithm of the file and an encryption level of the device, and determining the second encryption strategy according to the extended attribute information includes: The encryption algorithm of the file and the encryption level of the device are configured according to the second encryption policy, and a first master key is created.

5. The method according to any one of claims 1 to 4, characterized in that: The extended attribute information also includes a key ID, and the key ID is generated by the server through a Hash algorithm according to the encryption algorithm of the file and the encryption level of the device.

6. The method according to any one of claims 1 to 5, characterized in that: Before encrypting, packaging and / or decrypting the copy of the source file according to the second encryption policy, the method includes: It is determined according to the key ID that the first master key has been loaded, and the first master key is associated.

7. The method according to any one of claims 1 to 6, characterized in that: The encrypting and / or decrypting the received source file according to the second encryption strategy includes: The first master key is called to generate a first data key; and the source file copy is decrypted according to the first data key.

8. The method according to any one of claims 1 to 7, characterized in that: The managing the copy of the source file according to the metadata of the source file includes: The copy of the source file is encrypted and saved according to the first data key, and the copy of the source file is organized and managed according to the metadata information of the source file.

9. The method according to claim 1, characterized in that: Also applied to the server, the method includes: Receive the request message and obtain the source file information requested by the client; Acquire metadata of the source file and a copy of the source file according to the source file information; Determine extended attribute information of the source file according to the source file information, wherein the extended attribute information includes the first encryption strategy; A reply message is sent, wherein the reply message includes the metadata of the source file, the extended attribute information, and a copy of the source file.

10. The method according to claim 9, characterized in that The request message is a combined request message, and the parsing of the request message to obtain the source file information requested to be accessed includes: The merged request message is parsed to obtain information of multiple source files requested to be accessed, wherein the information of multiple source files requested to be accessed includes multiple associated requests for the same source file.

11. The method according to claim 9 or 10, characterized in that: The acquiring the metadata of the source file and the copy of the source file according to the source file information includes: The metadata of the source file is obtained according to the source file information requested for access, and the format of the metadata of the source file is uniformly adapted.

12. The method according to any one of claims 9 to 11, characterized in that: The step of determining the extended attribute information of the source file according to the source file information includes: The extended attribute information also includes a key ID. In the extended attribute information, the encryption algorithm of the file, the encryption level of the device, and the key ID are all identified by globally unique characters or strings.

13. The method according to any one of claims 9 to 12, characterized in that: The determining of the extended attribute information of the source file also includes: uniformly adapting the encryption algorithm of the file, the encryption level of the device, and the key ID in the extended attribute information to obtain the extended attributes expressed in the same data structure.

14. The method according to any one of claims 9 to 13, characterized in that: The request message is a combined request message, and the method on the server side further includes: The merged request messages are batch processed, and the results of the batch processing are assembled to obtain the reply message; the batch processing includes continuous or associated operations on multiple associated requests for the same source file; the results of the batch processing include metadata information of multiple source files, extended attribute information of multiple source files, and multiple copies of the source files.

15. A distributed file system, characterized in that: The system is applied to a terminal device, which includes a client and a server. The client establishes an end-to-end link with the server. The server is the owner of the source file, and the client accesses the source file of the server through the network. When the terminal device acts as the client, the system executes the method described in any one of claims 1 to 8 when it is running.

16. The system according to claim 15, characterized in that When the terminal device serves as the server, the system executes the method described in any one of claims 9 to 14 when running.

17. A terminal device, characterized in that: include: at least one memory for storing a program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 14.

18. A computer storage medium, wherein instructions are stored in the computer storage medium, and when the instructions are executed on a computer, the computer executes the method according to any one of claims 1 to 14.

Images