Method, device, system and storage medium for accessing a file

Abstract

The application discloses a method, device and system for accessing files and a storage medium, and belongs to the computer field. The method comprises the following steps: receiving a data access request, wherein the data access request comprises access requirement information, the access requirement information is used for indicating content in a first file that needs to be accessed by a first user, and the first file is stored in an object file storage system; when it is determined that the granularity of accessing the content in the first file is a first granularity based on the access requirement information, accessing the first file based on account information of the first user and the access requirement information; and when it is determined that the granularity of accessing the content in the first file is a second granularity based on the access requirement information, accessing the first file based on specified administrator account information and the access requirement information, wherein the second granularity is smaller than the first granularity. The application can enrich the access services provided to users.

Description

[0001] This application claims priority to Chinese Patent Application No. 202210264898.8, filed on March 17, 2022, entitled "An Efficient Fine-Grained Data Lake Access Control Scheme", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of computers, and in particular to a method, apparatus, system, and storage medium for accessing files. Background Technology

[0003] For a data lake with separate storage and compute, the data lake includes a structured query language (SQL) engine and an object file storage system. The object file storage system includes at least one file, each used to store data. The SQL engine receives SQL statements from users, which instruct the user to access the file they wish to access. The SQL engine then accesses that file in the object file storage system based on the SQL statement.

[0004] File systems in an object file storage system typically use structured data files, storing data in list format. For example, an object file storage system might contain a file storing employee data, with four columns: employee name, employee address, employee department, and employee job title. Each row in this file stores the employee's name, address, department, and job title.

[0005] Currently, users can access entire files located in the object file storage system using the SQL engine. This means the SQL engine provides file-level access, which is too simplistic. Summary of the Invention

[0006] This application provides a method, apparatus, system, and storage medium for accessing files, thereby enriching the access services provided to users. The technical solution is as follows:

[0007] In a first aspect, embodiments of this application provide a method for accessing a file. In this method, a data access request is received, the data access request including access request information. This access request information indicates that a first user needs to access content in a first file, the first file being stored in an object file storage system. When the granularity of accessing the content in the first file is determined to be a first granularity based on the access request information, the first file is accessed based on the first user's account information and the access request information. When the granularity of accessing the content in the first file is determined to be a second granularity based on the access request information, the first file is accessed based on specified administrator account information and the access request information, the second granularity being smaller than the first granularity.

[0008] Specifically, the granularity of accessing the content in the first file is determined based on the access request information. When the determined granularity is the first granularity, the first file is accessed based on the first user's account information and the access request information. When the determined granularity is the second granularity, the first file is accessed based on the specified administrator account information and the access request information. This provides users with both first-granularity and second-granularity access services, enriching the access services offered to users.

[0009] Since the first granularity is greater than the second granularity, when the determined granularity is the first granularity, the first file is accessed based on the first user's account information and the access request information. This eliminates the need to borrow the administrator's account information to access the first file, thereby improving the efficiency of accessing the first file and the performance of reading and writing the first file.

[0010] If the first user is configured with permissions to access content at the second granularity, then the first user will not only be able to access that content in the first file, but also all other content in the first file. This automatically expands the first user's permissions to include access to any content in the first file, resulting in excessively broad permissions and hindering access management. However, in this application, when the granularity is determined to be the second granularity, access to the first file is based on the specified administrator account information and the access request information. This uses the administrator account information instead of the first user's account information to access the first file. Therefore, it is unnecessary to configure permissions for the first user to access the second granularity, thus avoiding the expansion of the first user's access permissions and facilitating access management.

[0011] In one possible implementation, the access request information includes the identification information of a first file, with the first granularity being the file granularity; or, the access request information includes the identification information of the first file and the identification information of partitions within the first file, with the first granularity being the partition granularity. The first granularity can be either file-level or partition-level, thus providing users with file-level access services or partition-level access services, enriching the provided access services. Furthermore, the partition granularity is a newly defined granularity, meaning that this application can also provide partition access services.

[0012] In one possible implementation, file granularity refers to needing to access the entire contents of the first file. Partition granularity refers to needing to access the entire contents of a partition within the first file.

[0013] In another possible implementation, an authentication request is sent to the file path authentication module. This authentication request includes authentication information indicating the first user, the file path of the first file, and the first user's access operation to that file path. This authentication information is obtained based on the access request information and the first user's account information. The authentication request triggers the file path authentication module to authenticate the first user's permission to access the file path based on first permission information and the authentication information. The file path indicates the storage location of the first file, and the first permission information indicates the user's identity and access operation. An authentication response is received from the file path authentication module after successful authentication. This authentication response includes a temporary credential. The temporary credential, the file path, and the operation type of the access operation are stored correspondingly in the object file storage system. Based on the temporary credential, the access request information, and the file path, the first file is accessed.

[0014] Since a temporary credential is received from the file path authentication module after the permission is authenticated, the temporary credential is used to access the first file in the object file storage system, thereby improving the security of accessing the first file.

[0015] In another possible implementation, the first file is a structured data file that stores data in a list format. The access request information includes the identification information of the first file and first information, which indicates at least one column and / or at least one row of the first file. The second granularity is row-column granularity. Alternatively, the first file is a semi-structured data file that includes at least one data segment. The data segment stores data with the same business attributes. The access request information includes the identification information of the first file and the identification information of one or more data segments within the first file. The second granularity is data segment granularity. The second granularity can be either row-column granularity or data segment granularity, thus providing users with row-column granularity access services or data segment granularity access services, enriching the provided access services.

[0016] In another possible implementation, an access instruction is sent to the data filtering engine, which includes the access request information. The data filtering engine includes administrator account information. The access instruction is used to trigger the data filtering engine to access the first file based on the administrator account information and the access request information.

[0017] Since the data filtering engine includes administrator account information, sending an access command to the data filtering engine allows it to access the first file based on the specified administrator account information and the access request. This allows the administrator account information to replace the first user's account information and access the first file using the administrator account information, eliminating the need to configure second-level permissions for the first user. This avoids expanding the first user's access permissions and simplifies access control.

[0018] In another possible implementation, the first user's access to the content is authenticated based on the second permission information, the first user's account information, and the access request information. The second permission information indicates the user's identity and access operation. After the first user's access permission to the content is authenticated, the granularity of accessing the content in the first file is determined based on the access request information.

[0019] After the first user's access permission to the content is authenticated, the granularity of accessing the content in the first file is determined, and then different methods are used to access the first file based on different granularities, thereby improving the security of accessing the first file.

[0020] In another possible implementation, first permission information is generated based on second permission information. The first permission information indicates the user's identity and access operation for accessing the file path of the first file, and the file path indicates the storage location of the first file. This allows for the automatic generation of first permission information, improving the efficiency and reducing the cost of obtaining it.

[0021] Secondly, this application provides an access system comprising: a computing engine and an object file storage system.

[0022] The computing engine receives data access requests, which include access request information indicating the contents of a first file that a first user needs to access. The first file is stored in an object file storage system.

[0023] The computing engine is also used to access the first file based on the first user's account information and the access request information when determining the granularity of accessing the content in the first file as the first granularity based on the access request information.

[0024] The calculation engine is also used to access the first file based on the specified administrator account information and the access request information when the granularity of accessing the content in the first file is determined to be the second granularity based on the access request information. The second granularity is smaller than the first granularity.

[0025] Specifically, the computing engine determines the granularity of accessing the content in the first file based on the access request information. When the determined granularity is the first granularity, the first file is accessed based on the first user's account information and the access request information. When the determined granularity is the second granularity, the first file is accessed based on the specified administrator account information and the access request information. This provides users with both first-granularity and second-granularity access services, enriching the access services offered to users.

[0026] Since the first granularity is greater than the second granularity, when the granularity is determined to be the first granularity, the computing engine accesses the first file based on the first user's account information and the access request information. This way, it does not need to borrow the administrator's account information to access the first file, improving the efficiency of accessing the first file and the performance of reading and writing the first file.

[0027] If the first user is configured with permissions to access content at the second granularity, then the first user will not only be able to access that content in the first file, but also all other content in the first file. This automatically expands the first user's permissions to include access to any content in the first file, leading to excessively broad permissions and hindering access management. However, in this application, when the granularity is determined to be the second granularity, the computing engine accesses the first file based on the specified administrator account information and the access request information. This uses the administrator account information instead of the first user's account information to access the first file. Therefore, it is unnecessary to configure permissions for the first user to access the second granularity, thus avoiding the expansion of the first user's access permissions and facilitating access management.

[0028] In one possible implementation, the access request information includes the identification information of a first file, with the first granularity being the file granularity; or, the access request information includes the identification information of the first file and the identification information of partitions within the first file, with the first granularity being the partition granularity. The first granularity can be either file-level or partition-level, thus providing users with file-level access services or partition-level access services, enriching the provided access services. Furthermore, the partition granularity is a newly defined granularity, meaning that this application can also provide partition access services.

[0029] In one possible implementation, file granularity refers to needing to access the entire contents of the first file. Partition granularity refers to needing to access the entire contents of a partition within the first file.

[0030] In another possible implementation, the system also includes a file path authentication module.

[0031] The computing engine is used to send an authentication request to the file path authentication module. The authentication request includes authentication information, which is used to indicate the first user, the file path of the first file, and the first user's access operation to the file path. The authentication information is obtained based on the access request information and the account information of the first user. The file path is used to indicate the storage location of the first file.

[0032] The file path authentication module is used to authenticate the first user's permission to access the file path by means of the access operation based on the first permission information and the authentication information. The first permission information is used to indicate the user's identity and access operation that can access the file path. After the permission is authenticated, an authentication response is sent to the computing engine. The authentication response includes a temporary credential.

[0033] An object file storage system is used to store the temporary credential, the file path, and the operation type of the access operation.

[0034] The computing engine is also used to access the first file based on the temporary credential, the access request information, and the file path.

[0035] Since the computing engine receives a temporary credential sent by the file path authentication module after the permission is authenticated, it uses the temporary credential to access the first file in the object file storage system, thereby improving the security of accessing the first file.

[0036] In another possible implementation, the first file is a structured data file that stores data in a list format. The access request information includes the identification information of the first file and first information, which indicates at least one column and / or at least one row of the first file. The second granularity is row-column granularity. Alternatively, the first file is a semi-structured data file that includes at least one data segment. The data segment stores data with the same business attributes. The access request information includes the identification information of the first file and the identification information of one or more data segments within the first file. The second granularity is data segment granularity. The second granularity can be either row-column granularity or data segment granularity, thus providing users with row-column granularity access services or data segment granularity access services, enriching the provided access services.

[0037] In another possible implementation, the system also includes a data filtering engine that includes administrator account information.

[0038] The computing engine sends access instructions to the data filtering engine. These instructions include the file path of the first file and the access request information, where the file path indicates the storage location of the first file.

[0039] The data filtering engine is used to access the first file based on the administrator account information, the file path, and the access request information.

[0040] Since the data filtering engine includes administrator account information, the computing engine sends an access command to the data filtering engine. The data filtering engine then accesses the first file based on the specified administrator account information and the access request. In this way, the administrator account information can be used instead of the first user's account information to access the first file. This eliminates the need to configure second-level permissions for the first user, thus avoiding expanding the first user's access rights and facilitating access control.

[0041] In another possible implementation, the computing engine is further used to authenticate the first user's access to the content based on the second permission information, the first user's account information, and the access request information. The second permission information indicates the user's identity and access operation. After the first user's access permission to the content is authenticated, the granularity of accessing the content in the first file is determined based on the access request information. Because the computing engine determines the granularity of accessing the content in the first file after the first user's access permission is authenticated, and then uses different methods to access the first file based on different granularities, the security of accessing the first file is improved.

[0042] In another possible implementation, the system also includes a linked permission module.

[0043] The linked permission module is used to generate first permission information based on second permission information. The first permission information indicates the user's identity and access operation for accessing the file path of the first file, and the file path indicates the storage location of the first file. This allows for the automatic generation of first permission information, improving the efficiency and reducing the cost of obtaining it.

[0044] Thirdly, this application provides an apparatus for accessing files, used to perform the method in the first aspect or any possible implementation thereof. Specifically, the apparatus includes units for performing the method in the first aspect or any possible implementation thereof.

[0045] Fourthly, this application provides an apparatus for accessing files, the apparatus including a processor and a memory. The processor and the memory are interconnected via an internal connection. The memory stores a program, and the processor executes the program in the memory, causing the apparatus to perform the method of the first aspect or any possible implementation thereof.

[0046] Fifthly, this application provides a computer program product comprising a computer program stored in a computer-readable storage medium, wherein the computer program is loaded by a processor to implement the method described in the first aspect or any possible implementation thereof.

[0047] In a sixth aspect, this application provides a computer-readable storage medium for storing a computer program, which is loaded by a processor to execute the method described in the first aspect or any possible implementation thereof.

[0048] In a seventh aspect, this application provides a chip including a memory and a processor, the memory for storing computer instructions, and the processor for calling and executing the computer instructions from the memory to perform the method of the first aspect or any possible implementation of the first aspect. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the structure of an access system provided in an embodiment of this application;

[0050] Figure 2 This is a schematic diagram of a document provided in an embodiment of this application;

[0051] Figure 3 This is a schematic diagram of another access system provided in an embodiment of this application;

[0052] Figure 4This is a flowchart of a method for accessing files provided in an embodiment of this application;

[0053] Figure 5 This is a schematic diagram of another access system according to an embodiment of this application;

[0054] Figure 6 This is a schematic diagram of another access system according to an embodiment of this application;

[0055] Figure 7 This is a flowchart of a method for obtaining first permission information provided in an embodiment of this application;

[0056] Figure 8 This is a schematic diagram of a device structure for accessing files provided in an embodiment of this application;

[0057] Figure 9 This is a schematic diagram of another device structure for accessing files provided in an embodiment of this application. Detailed Implementation

[0058] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0059] See Figure 1 This application provides an access system 100, which includes a computing engine 101 and an object file storage system 102, and the computing engine 101 and the object file storage system 102 communicate with each other.

[0060] In some embodiments, the access system 100 is a storage-compute separation database system, wherein the object file storage system 102 is responsible for data storage, and the computing engine 101 is responsible for data computation.

[0061] In some embodiments, the access system 100 is applied to scenarios such as data lakes with separate storage and computing, and to the field of big data processing.

[0062] The object file storage system 102 is used to store at least one file. For any file stored in the object file storage system 102, the file is used to store data.

[0063] In some embodiments, this file may be a structured data file, which stores data in list format; therefore, a structured data file is a data table. Each column in this file stores data with the same business attributes.

[0064] For structured data files, which are essentially data tables, the file's identification information is the identification information of that data table. For example, the file's identification information is its filename, which means the file's identification information is the table name.

[0065] For example, see Table 1 below. The file is a structured data file. The file is a data table with five columns of data. The data table is used to store company information. The table name is "Company information". The file name is also "Company information". The file is a company information table with the same table name and file name.

[0066] Referring to Table 1 below, each column of this file stores data with the same business attribute. As shown in Table 1, the first column stores row numbers, meaning each data entry in the first column has the business attribute of being a row number. The second column stores company names, meaning each data entry in the second column has the business attribute of being a company name. The third column stores industry names, meaning each data entry in the third column has the business attribute of being an industry name. The fourth column stores cities, meaning each data entry in the fourth column has the business attribute of being a city name. The fifth column stores countries, meaning each data entry in the fifth column has the business attribute of being a country name.

[0067] Table 1: Company Information

[0068]

[0069] In some embodiments, the file is a semi-structured data file that includes at least one data fragment, wherein any data fragment in the file is used to store data with the same business attributes.

[0070] For example, see Figure 2The semi-structured data file shown comprises four data segments: the first segment, the second segment, the third segment, and the fourth segment. The first segment stores company names, such as "Company 1," "Company 2," "Company 3," "Company 4," "Company 5," and "Company 6," meaning each data point in the first segment has the business attribute of a company name. The second segment stores industry names, such as "Internet," "Internet," "Telecommunications," "Logistics," "Telecommunications," and "Logistics," meaning each data point in the second segment has the business attribute of an industry name. The third segment stores cities, such as "City 1," "City 1," "City 2," "City 2," "City 1," and "City 3," meaning each data point in the third segment has the business attribute of a city name. The fourth segment stores countries, such as "Country 1," "Country 2," "Country 1," "Country 1," "Country 1," and "Country 3," meaning each data point in the fourth segment has the business attribute of a country name.

[0071] In some embodiments, the semi-structured data file is an Extensible Markup Language (XML) file, etc., in which the tag blocks are data fragments.

[0072] In some embodiments, the file may also include at least one partition.

[0073] In some embodiments, the file is stored in the object file storage system 102, and the file path of the file is used to indicate the storage location of the file in the object file storage system 102. For example, suppose the file path of the file shown in Table 1 is "C:\windows\system32\Company information", and this file path is used to indicate the storage location of the file shown in Table 1 in the object file storage system 102.

[0074] See Figure 1 The first user has a need to access the contents of a file stored in the object file storage system 102. The granularity at which the first user accesses the contents of this file may be either a first granularity or a second granularity, where the second granularity is smaller than the first granularity.

[0075] In some embodiments, the first granularity may be file granularity, meaning the first user needs to access the entire contents of the file; or, the first granularity may be partition granularity, meaning the first user needs to access a partition of the file. File granularity refers to needing to access the entire contents of the first file. Partition granularity refers to needing to access the entire contents of a partition within the first file.

[0076] In some embodiments, the file is a structured data file, and the second granularity is row-column granularity, meaning the first user needs to access at least one row and / or at least one column of the file. Alternatively, the file is a semi-structured data file, and the second granularity is data fragment granularity, meaning the first user needs to access data fragments within the file.

[0077] Therefore, it can provide users with access services at the file level, or at the partition level, or at the row and column level, or at the data fragment level, thus enriching the access services provided to users.

[0078] The first user is the user who has data access needs, also known as the business user. Optionally, the first user may be an application, etc.

[0079] When a first user needs to access data, the first user sends a data access request to the computing engine 101. The data access request includes access request information, which indicates the content of a first file that the first user needs to access. The first file is a file stored in the object file storage system 102.

[0080] The computing engine 101 receives the data access request and determines the granularity of accessing the content in the first file based on the access request information. When the determined granularity is the first granularity, the first file is accessed based on the first user's account information and the access request information. When the determined granularity is the second granularity, the first file is accessed based on the specified administrator account information and the access request information, where the second granularity is smaller than the first granularity.

[0081] In some embodiments, the computing engine 101 includes an interface through which a first user can invoke the data access request to the computing engine 101. Optionally, the interface may include a Java Database Connectivity (JDBC) interface or an Open Database Connectivity (ODBC) interface, etc.

[0082] In some embodiments, the first file is a structured data file, and the access requirement information includes identification information of the first file, but does not include identification information of partitions in the first file or first information, whereby the first information is used to indicate at least one column and / or at least one row in the first file. The granularity determined by the computing engine 101 based on the access requirement information is the file granularity.

[0083] In some embodiments, the first file is a semi-structured data file, and the access requirement information includes the identification information of the first file, but does not include the identification information of partitions in the first file or the identification information of data segments in the first file. The granularity determined by the computing engine 101 based on the access requirement information is the file granularity.

[0084] In some embodiments, the first file is a structured data file, and the access requirement information includes the identification information of the first file and the identification information of the partitions in the first file, but the access requirement information does not include the first information. The granularity determined by the computing engine 101 based on the access requirement information is the partition granularity.

[0085] In some embodiments, the first file is a semi-structured data file, and the access request information includes the identification information of the first file and the identification information of the partitions in the first file, but does not include the identification information of the data segments in the first file. The granularity determined by the computing engine 101 based on the access request information is the partition granularity.

[0086] In some embodiments, the first file is a structured data file, and the access requirement information includes identification information and first information of the first file, wherein the first information is used to indicate at least one column and / or at least one row in the first file. The granularity determined by the calculation engine 101 based on the access requirement information is row and column granularity.

[0087] In some embodiments, the first file is a semi-structured data file, and the access request information includes identification information of the first file and identification information of data segments within the first file. The granularity determined by the computing engine 101 based on the access request information is the data segment granularity.

[0088] In some embodiments, see Figure 1 and Figure 3 The computing engine 101 includes a computing module 1011 and a routing module 1012. The computing module 1011 in the computing engine 101 receives the data access request, and the routing module 1012 in the computing engine 101 determines the granularity of accessing the content in the first file based on the access request information.

[0089] In some embodiments, the access request information further includes a first operation type, which indicates a first access operation performed by the first user to access the first file. Optionally, the first access operation includes querying, updating, inserting, or deleting.

[0090] In some embodiments, the operation of the computing engine 101 accessing the first file may be: querying the content of the first file and returning the queried content to the first user. Alternatively, if the access request information includes content to be updated, the operation of the computing engine 101 accessing the first file may be: updating all or part of the content in the first file to the content to be updated. Alternatively, if the access request information includes content to be inserted, the operation of the computing engine 101 accessing the first file may be: inserting the content to be inserted into the first file. Alternatively, the operation of the computing engine 101 accessing the first file may be: deleting all or part of the content in the first file, etc.

[0091] In some embodiments, the data access request may also include the account information of the first user. Alternatively, the data access request may not include the account information of the first user. The communication connection between the computing engine 101 and the first user is bound to the account information of the first user, and the computing engine 101 obtains the account information of the first user bound to the communication connection. Optionally, the communication connection is a session between the first user and the computing engine 101.

[0092] In some embodiments, the access system 100 includes one or more computing engines 101. Optionally, the computing engine 101 is a Hive engine or a Spark engine, that is, the access system 100 includes one or more Hive engines, and / or one or more Spark engines, etc.

[0093] In some embodiments, the Hive engine is a data warehouse tool based on Hadoop (a distributed system infrastructure) that can map structured data files to a table and provide query functionality.

[0094] In some embodiments, the Spark engine is a fast and general-purpose computing engine designed specifically for large-scale data processing.

[0095] In some embodiments, see Figure 3 The access system 100 also includes a file path authentication module 103, which communicates with the computing engine 101 and the object file storage system 102.

[0096] The calculation engine 101 is used to send an authentication request to the file path authentication module 103 at the determined granularity of the first granularity. The authentication request includes authentication information, which is used to instruct the first user, the file path of the first file, and the second access operation of the first user to access the file path. The authentication information is obtained based on the access request information and the account information of the first user.

[0097] The file path authentication module 103 is used to receive the authentication request, authenticate the first user's permission to access the file path using the second access operation based on the first permission information and the authentication information. The first permission information is used to indicate the user identity who can access the file path and the third access operation that can access the file path. After the permission authentication is successful, the module sends an authentication response to the computing engine 101. The authentication response includes a temporary credential and sends storage information to the object file storage system 102. The storage information includes the temporary credential, the file path and the second operation type. The second operation type is the operation type of the second access operation.

[0098] The object file storage system 102 is used to receive the storage information and save the temporary credential, the file path, and the second operation type accordingly.

[0099] The computing engine 101 is also used to access the first file based on the temporary credential, the access request information, and the file path.

[0100] The second access operation is a mapping of the first access operation, and it is an operation that can access the object file storage system 102. Typically, the second access operation includes read and / or write operations.

[0101] For example, the first access operation is a query, and the second access operation mapped from the query operation is a read operation. Suppose we need to query the content of the first file, then we read the first file from the object file storage system 102 and retrieve the content to be queried from the read first file.

[0102] For example, the first access operation is an update, and the second access operation mapped from the update operation includes read and write operations. Suppose that a portion of the content in the first file needs to be updated to the content to be updated, then the first file is read from the object file storage system 102, the portion of the content in the first file is updated to the content to be updated, and the updated first file is written to the object file storage system 102 to overwrite the first file already saved in the object file storage system 102.

[0103] In some embodiments, the authentication information includes the user identity of the first user, the file path of the first file, and the second operation type. The user identity of the first user is obtained by the computing engine 101 based on the user's account information, the second operation type is obtained by mapping the first operation type, and the file path of the first file is obtained by the computing engine 101 based on the identification information of the first file.

[0104] In some embodiments, the authentication information includes the account information of the first user, the identification information of the first file, and the first operation type.

[0105] In some embodiments, the user identity of the first user includes the user group to which the first user belongs and / or the role of the first user, etc.

[0106] In some embodiments, the first permission information includes the file path, the identity of the user who can access the file path, and a third operation type, wherein the third operation type is the type of a third access operation that allows access to the file path.

[0107] In some embodiments, see Figure 3 The access system 100 also includes a linkage permission module 104, which communicates with the computing engine 101 and the file path authentication module 103. The linkage permission module 104 stores the aforementioned first permission information.

[0108] After receiving the authentication request, the file path authentication module 103 obtains the file path of the first file, the user identity of the first user, and the second operation type of the second access operation based on the authentication information included in the authentication request. It then retrieves first permission information, including the file path, from the linked permission module 104. If the user identity of the first user is the same as the user identity included in the first permission information, and the second operation type of the second access operation is the same as the third operation type of the third access operation included in the first permission information, then the permission authentication is successful, indicating that the first user has permission to access the file path using the second access operation.

[0109] In some embodiments, the linkage permission module 104 includes a first read / write interface. The file path authentication module 103 calls the first read / write interface of the linkage permission module 104 to obtain first permission information including the file path from the linkage permission module 104 through the first read / write interface.

[0110] In some embodiments, the authentication information includes the user identity of the first user, the file path of the first file, and the second operation type. The file path authentication module 103 directly obtains the file path of the first file, the user identity of the first user, and the second operation type of the second access operation from the authentication information.

[0111] In some embodiments, the authentication information includes the account information of the first user, the identification information of the first file, and the first operation type. The file path authentication module 103 obtains the user identity of the first user based on the account information of the first user, maps the first operation type to obtain the second operation type, and obtains the file path of the first file based on the identification information of the first file.

[0112] In some embodiments, see Figure 3 The access system 100 also includes a data filtering engine 105, which includes specified administrator account information; the data filtering engine 105 communicates with the computing engine 101 and the object file storage system 102 respectively.

[0113] The computing engine 101 is used to send an access instruction to the data filtering engine 105 at the determined granularity of the second granularity. The access instruction includes the file path of the first file and the access request information.

[0114] Data filtering engine 105 is used to access the first file based on the administrator account information, the file path, and the access request information.

[0115] Optionally, the data filtering engine 105 also communicates with the linkage permission module 104.

[0116] See Figure 3 In some embodiments, after receiving the data access request, the computing module 1011 of the computing engine 101 authenticates the first user's access rights to the content based on the second permission information, the first user's account information, and the access request information. The second permission information is used to indicate the user identity and the fourth access operation that can access the content.

[0117] After successful authentication, if the granularity determined by the routing module 1012 of the computing engine 101 is the first granularity, the routing module 1012 of the computing engine 101 sends an authentication request to the file path authentication module 103. If the granularity determined by the routing module 1012 of the computing engine 101 is the second granularity, the routing module 1012 of the computing engine 101 sends an access instruction to the data filtering engine 105.

[0118] In some embodiments, the second permission information includes content identification information of the content, the identity of the user who can access the content, and the fourth operation type of the fourth access operation that can access the content.

[0119] In some embodiments, the linkage permission module 104 stores second permission information and includes a second read / write interface. After receiving the data access request, the computing module 1011 of the computing engine 101 obtains the content identifier information of the content based on the access request information and obtains the user identity of the first user based on the account information of the first user. It then calls the second read / write interface in the linkage permission module 104 to obtain the second permission information, including the content identifier information, from the linkage permission module 104. If the user identity of the first user is the same as the user identity included in the second permission information, and the first operation type of the first access operation is the same as the fourth operation type of the fourth access operation included in the second permission information, then the authentication of the first user's access to the content is successful, indicating that the first user has permission to access the content.

[0120] The content identification information is part of the access request information.

[0121] In some embodiments, when the content is the entire content of a first file, the content identification information of the content includes the identification information of the first file. Alternatively, when the content is a partition of the first file, the content identification information of the content includes the identification information of the first file and the identification information of the partition. Alternatively, when the content is at least one column or at least one row in the first file, the content identification information of the content includes the identification information of the first file and the column identifier of the at least one column; or, the content identification information of the content includes the identification information of the first file and the row number of the at least one row. Alternatively, when the content is at least one data segment in the first file, the content identification information of the content includes the identification information of the first file and the identification information of each data segment in the at least one data segment.

[0122] In some embodiments, for the authentication operation of authorizing the first user's access to the content, and for the determination operation of determining the granularity of accessing the content in the first file, the computing module 1011 of the computing engine 101 may first perform the authentication operation after receiving the data access request, and then the routing module 102 of the computing engine 101 may perform the determination operation. That is, the computing module 1011 of the computing engine 101 may first authenticate the first user's access to the content. After successful authentication, the routing module 1012 of the computing engine 101 determines the granularity of accessing the content in the first file based on the access request information. Alternatively,

[0123] After receiving the data access request, the computing module 1011 of the computing engine 101 may first perform the determination operation, and then the computing module 1011 of the computing engine 101 may perform the authentication operation. That is, the routing module 1012 of the computing engine 101 may first determine the granularity of accessing the content in the first file based on the access request information, and then the computing module 1011 of the computing engine 101 authenticates the first user's permission to access the content. Alternatively,

[0124] After receiving the data access request, the computing module 1011 of the computing engine 101 performs the authentication operation, and at the same time, the routing module 1012 of the computing engine 101 performs the determination operation. That is, the authentication operation and the determination operation are performed simultaneously.

[0125] In some embodiments, see Figure 3 The access system 100 also includes an identity authentication center 106, which is used to store the correspondence between user account information and user identity.

[0126] In some embodiments, the operation of the computing module 1011 of the computing engine 101 to obtain the user identity of the first user is as follows: the computing module 1011 of the computing engine 101 queries the user identity of the first user from the identity authentication center 106 based on the account information of the first user.

[0127] In some embodiments, the file path authentication module 103 obtains the user identity of the first user by querying the user identity of the first user from the identity authentication center 106 based on the account information of the first user.

[0128] In some embodiments, see Figure 3 The access system 100 also includes a metadata center 107, which receives and stores metadata of the first file input by the second user. This metadata includes the identification information of the first file, the type of operation to be performed on the first file, and the file path of the first file. Optionally, the operation type may be creating the first file, deleting the first file, querying the first file, or modifying the first file, etc.

[0129] In some embodiments, the first file is a structured data file, and the metadata of the first file includes one or more of the following: column identifier of each column in the first file, column type of each column in the first file, row separator of the first file, or column separator of the first file, etc.

[0130] In some embodiments, the first file is a semi-structured data file, and the metadata of the first file includes one or more of the following: identification information of each data segment in the first file, the type of each data segment in the first file, or, the line separator of the first file, which is used to distinguish each line of data in any data segment in the first file.

[0131] In some embodiments, the operation of the computing engine 101 to obtain the file path of the first file is as follows: the computing engine 101 obtains metadata including the identification information of the first file from the metadata center 107, and obtains the file path of the first file from the metadata.

[0132] In some embodiments, the file path authentication module 103 obtains the file path of the first file by: obtaining metadata including the identification information of the first file from the metadata center 107, and obtaining the file path of the first file from the metadata.

[0133] In some embodiments, the metadata center 107 displays a first interface to a second user, who can input metadata of a first file on the first interface, and the first interface receives the metadata of the first file input by the second user. Optionally, the first interface includes a website user interface (Web UI), etc.

[0134] In some embodiments, when receiving the metadata of the first file, the metadata center 107 also obtains the account information of the second user and verifies the metadata of the first file based on the account information of the second user. In implementation,

[0135] Metadata Center 107 verifies the legitimacy of the second user based on the second user's account information. During the verification process, the second user's identity is obtained, and the types of operations the second user can perform are determined based on that identity. If the metadata includes operation types that need to be performed on the first file that the second user can perform, the metadata verification of the first file passes, and the metadata of the first file is then saved.

[0136] In some embodiments, the metadata center 107 verifies the legitimacy of the second user and obtains the user identity of the second user as follows:

[0137] The identity authentication center 106 stores the correspondence between account information and user identities. The metadata center 107 queries the identity authentication center 106 to see if it stores the second user's account information. If the identity authentication center 106 stores the second user's account information, it verifies that the second user is a legitimate user. Based on the second user's account information, it queries the identity authentication center 106 to retrieve the second user's identity.

[0138] In some embodiments, the metadata center 10 obtains the operation type that the second user can operate as follows:

[0139] The metadata center 107 stores the correspondence between user identity and operation type. Based on the user identity of the second user, the metadata center 107 obtains the corresponding operation type from the correspondence between user identity and operation type as the operation type that the second user can operate.

[0140] In some embodiments, the metadata center 107 saves the metadata of the first file as follows: the metadata center 107 queries whether metadata including the identification information of the first file has been saved. If the metadata has been saved, the saved metadata is updated to the metadata of the first file. If the metadata has not been saved, the metadata of the first file is saved directly.

[0141] The metadata center 107 includes specified administrator account information. After the metadata verification of the first file passes, if the operation type included in the metadata is to create the first file, the metadata center 107, based on the specified administrator account information, creates the file path of the first file in the object file storage system 102, and the storage location corresponding to that file path is used to save the first file. If the operation type included in the metadata is to delete the first file, the metadata center 107, based on the specified administrator account information and the file path of the first file, determines the first file in the object file storage system 102 and deletes the determined first file. If the operation type included in the metadata is to query the first file, the metadata center 107, based on the specified administrator account information and the file path of the first file, determines the first file in the object file storage system 102, obtains the description information and / or attribute information of the first file, and returns the obtained content to the second user. If the operation type included in the metadata is to modify the first file, the metadata center 107, based on the specified administrator account information and the file path of the first file, determines the first file in the object file storage system 102 and modifies the description information and / or attribute information of the first file.

[0142] See Figure 3 The linkage permission module 104 is also used to receive second permission information configured by the permission administrator. This second permission information indicates the user identity and fourth access operation that can access the content of the first file. Based on the second permission information, first permission information is generated. This first permission information indicates the user identity and third access operation that can access the file path of the first file. The second permission information and the first permission information are then saved.

[0143] In some embodiments, the linkage permission module 104 is further configured to obtain the metadata of the first file from the metadata center 107, obtain at least one user identity from the identity authentication center 106, and display a second interface to the permission administrator. The second interface includes the metadata of the first file and the at least one user identity.

[0144] In this way, the access administrator selects the content identifier information of the content in the first file from the metadata of the first file, selects the user identity that can access the content from the at least one user identity, and inputs the fourth operation type of the fourth access operation that allows access to the content into the second interface, thus obtaining the second access information. The second access information includes the content identifier information, the selected user identity, and the input fourth operation type. Optionally, the second interface includes a Web UI, etc.

[0145] In some embodiments, the first file is a structured data file, and the metadata of the first file includes the file identifier of the first file and the column identifier of each column of the first file. The content identification information of the content includes the file identifier of the first file, or the content identification information of the content includes the file identifier of the first file and the column identifier of at least one column in the first file, or the content identification information of the content includes the file identifier of the first file and the row number of at least one row in the first file.

[0146] In some embodiments, the first file is a semi-structured data file, and the metadata of the first file includes the file identifier of the first file and the identification information of each data segment of the first file. The content identification information includes the file identifier of the first file, or the content identification information includes the file identifier of the first file and the identification information of at least one data segment in the first file.

[0147] In some embodiments, the operation of the linkage permission module 104 in generating the first permission information is as follows:

[0148] (1): The linkage permission module 104 obtains the file path of the first file based on the content identifier information of the content in the second permission information.

[0149] In some embodiments, the content identification information of the content includes the identification information of the first file. Metadata including the identification information of the first file is obtained from the metadata center 107. This metadata is the metadata of the first file. The file path of the first file is obtained from the metadata of the first file.

[0150] (2): The linkage permission module 104 maps the fourth operation type included in the second permission information to obtain the third operation type.

[0151] (3): The linkage permission module 104 reads the user identity from the second permission information and combines the file path of the first file, the user identity and the third operation type to form the first permission information.

[0152] In this embodiment, the computing engine receives a data access request and determines the granularity of accessing the content in the first file based on the access request information. When the determined granularity is a first granularity, the first file is accessed based on the first user's account information and the access request information. When the determined granularity is a second granularity, the first file is accessed based on the specified administrator account information and the access request information. This provides the user with both first-granularity and second-granularity access services, enriching the access services offered to the user. Since the first granularity is greater than the second granularity, when the determined granularity is the first granularity, the computing engine accesses the first file based on the first user's account information and the access request information, thus avoiding the need to borrow administrator account information to access the first file, improving the efficiency of accessing the first file and the performance of reading and writing the first file.

[0153] Furthermore, if the first user is configured to access content at the second granularity, the first user will not only be able to access that content in the first file, but also all other content in the first file. This automatically expands the first user's permissions to include any content in the first file, leading to excessively broadened permissions and hindering access management. However, in this application, when the defined granularity is the second granularity, the computing engine accesses the first file based on the specified administrator account information and the access request information. This uses the administrator account information instead of the first user's account information to access the first file. Therefore, it is unnecessary to configure the first user with permissions to access the second granularity, thus avoiding the expansion of the first user's access permissions and facilitating access management.

[0154] See Figure 4 This application provides a method 400 for accessing files, the method 400 being applied to... Figure 1 or Figure 3 The access system 100 shown includes the following steps 401 to 410 in the method 400.

[0155] Step 401: The computing engine receives a data access request, which includes access requirement information indicating the contents of a first file that the first user needs to access. The first file is stored in an object file storage system.

[0156] In some embodiments, the first user is a business user performing data access services, and the first user sends a data access request to the computing engine.

[0157] In some embodiments, the access request information is an access statement for accessing the database, such as an SQL statement.

[0158] In some embodiments, the access request information includes content identification information of the content and a first operation type, wherein the first operation type is used to indicate a first access operation for accessing the content in the first file. Optionally, the first access operation is, for example, querying the first file, updating the first file, or deleting the first file.

[0159] In some embodiments, when the first access operation indicated by the first operation type is to update the first file, the access request information also includes the content to be updated.

[0160] In some embodiments, the data access request may also include the account information of the first user.

[0161] In some embodiments, the access request information may include the following types of information, which will be described in detail below.

[0162] Type 1, the access request information includes the identification information of the first file and the first operation type.

[0163] The access request information does not include the partition identification information of the first file. When the first file is a structured data file, the access request information does not include the first information, which indicates at least one column and / or at least one row in the first file. When the first file is a semi-structured data file, the access request information does not include the partition identification information of the first file.

[0164] In this case, the content identification information is the identification information of the first file.

[0165] For access request information of type 1, the content is the entire content of the first file, indicating that the first user needs to access the entire content of the first file, and the granularity of the first user's access to this content in the first file is the file granularity.

[0166] For example, the access request information is: Select * From Company information. This access request information includes the identification information of the first file "Company information" as shown in Table 1 and the first operation type "Select", which means querying the first file.

[0167] Type 2, the access request information includes the identification information of the first file, the identification information of the partition in the first file, and the first operation type.

[0168] When the first file is a structured data file, the access request information does not include first information, which indicates at least one column and / or at least one row in the first file. When the first file is a semi-structured data file, the access request information does not include partition identification information in the first file.

[0169] In this case, the content identification information includes the identification information of the first file and the identification information of the partition in the first file.

[0170] For type 2 access request information, the content is the partition of the first file, indicating that the first user needs to access the partition of the first file, and the granularity of the first user accessing the content in the first file is the partition granularity.

[0171] Type 3, the first file is a structured data file, and the access request information includes the identification information of the first file, the first information and the first operation type, the first information being used to indicate at least one column and / or at least one row in the first file.

[0172] For type 3 access request information, the content is at least one column or at least one row of the first file, indicating that the first user needs to access at least one column or at least one row of the first file, and the granularity of the first user accessing the content in the first file is row and column granularity.

[0173] In some embodiments, the first information includes a column identifier of the at least one column in the first file, and the content is the at least one column in the first file, indicating that the first user needs to access the at least one column of the first file. In this case, the content identifier information of the content includes the identifier information of the first file and the column identifier of the at least one column in the first file.

[0174] For example, the access request information is: Select Name, City From Company information. This access request information includes the identification information of the first file "Company information" as shown in Table 1, the column identifier of the second column of the first file "Name", the column identifier of the fourth column of the first file "City", and the first operation type "Select", which means querying the first file.

[0175] In some embodiments, the first information includes a column identifier of at least one column in the first file and row filtering information corresponding to each column in the at least one column. The content is at least one line in the first file, indicating that the first user needs to access at least one line of the first file. In this case, the content identification information includes the identification information of the first file and the column identifier of the at least one column in the first file.

[0176] For any one of the at least one columns, the content of that column can be located from the first file as one or more lines of the filter information for that row. This content is the content of the located one or more lines.

[0177] For example, referring to Table 1 above, assuming the first information includes the column identifier "City" in the fourth column of Table 1 and the corresponding row filter information "City 1", the first, second, and fifth rows in the fourth column where City is "City 1" can be located from the first file shown in Table 1 based on the first information. The access request information is: Select *From Companyinformation Where City=City1, where the first operation type "Select" is querying the first file.

[0178] In some embodiments, the first information includes the line number of at least one line in the first file, and the content is the at least one line in the first file, indicating that the first user needs to access the at least one line in the first file. In this case, the content identification information of the content includes the identification information of the first file and the line number of the at least one line in the first file.

[0179] Type 4, the first file is a semi-structured data file, and the access request information includes the identification information of the first file, the identification information of at least one data segment in the first file, and the first operation type.

[0180] In this case, the content identification information includes the identification information of the first file and the identification information of the at least one data segment in the first file.

[0181] For type 4 access request information, the content is at least one data fragment of the first file, indicating that the first user needs to access at least one data fragment of the first file, and the granularity of the first user accessing the content in the first file is the data fragment granularity.

[0182] For type 4 access request information, the data access request is a remote procedure call (RPC) request, which includes type 4 access request information.

[0183] For example, the data access information in this RPC request includes, for example, Figure 2 The first file's identifier information "Company information", the first data segment's identifier information "Name", the third data segment's identifier information "Country", and the first operation type "Query first file" are shown.

[0184] In summary, the access request information includes at least the identification information of the first file and the first operation type, and may also include first information, the identification information of the partition in the first file, or the identification information of the data segment in the first file, etc.

[0185] Step 402: The computing engine authenticates the first user's access rights to the content based on the second permission information, the first user's account information, and the access request information. After the first user's access rights to the content are authenticated, step 403 is executed.

[0186] The second permission information is used to indicate the identity of the user who can access the content and the fourth access operation that allows access to the content.

[0187] See Figure 5 In some embodiments, the computing engine is the Hive engine, which authenticates the first user's access to the content through operations 4021 to 4023 as follows.

[0188] 4021: For the content identifier information of the content in the access request information, the Hive engine determines whether the content exists in the object file storage system based on the content identifier information. If the content exists in the object file storage system, the following 4022 operation is performed.

[0189] In 4021, the content identification information includes the identification information of the first file. Based on this identification information, the Hive engine retrieves the metadata of the first file from the metadata center. The metadata of the first file includes the identification information. If the operation type in the first file's metadata is "delete first file," it means the metadata center has deleted the first file from the object file storage system, and the Hive engine determines that the object file storage system does not contain this content based on the first file's metadata. If the operation type in the first file's metadata is "create first file," "modify first file," or "query first file," it means the object file storage system stores the first file. The Hive engine determines whether the object file storage system contains this content based on the first file's metadata and the content identification information.

[0190] In some embodiments, the Hive engine retrieves the metadata of the first file as follows:

[0191] The Hive engine sends a first retrieval command to the metadata center, which includes the identification information of the first file. The metadata center receives the first retrieval command, retrieves the metadata including the identification information of the first file from the stored metadata (the retrieved metadata is the metadata of the first file), and sends a first retrieval response to the Hive engine, which includes the metadata of the first file. Alternatively,

[0192] The Hive engine sends a first retrieval command to the metadata center. The metadata center receives the first retrieval command, retrieves each saved metadata element, and sends a first retrieval response to the Hive engine, which includes each metadata element. The Hive engine receives the first retrieval response and retrieves metadata, including the identifier information of the first file, from each metadata element; the retrieved metadata is the metadata of the first file.

[0193] In some embodiments, when the metadata of the first file includes operations of creating, modifying, or querying the first file, the Hive engine determines whether the object file storage system has the content based on the metadata of the first file and the content identifier information as follows:

[0194] If the content identification information includes the identification information of the first file and the identification information of the partition in the first file, and the metadata of the first file also includes the identification information of the partition in the first file, then it is determined that the content exists in the object file storage system. If the metadata of the first file does not include the identification information of the partition in the first file, then it is determined that the content does not exist in the object file storage system.

[0195] If the content identification information includes the identification information of the first file and the column identifier of at least one column in the first file, and the metadata of the first file also includes the column identifier of the at least one column, then it is determined that the object file storage system contains the content. If the metadata of the first file does not include the column identifier of the at least one column, then it is determined that the object file storage system does not contain the content.

[0196] If the content identification information includes the identification information of the first file and the identification information of a data segment within the first file, and the metadata of the first file also includes the identification information of the data segment, then it is determined that the object file storage system contains the content. If the metadata of the first file does not include the identification information of the data segment, then it is determined that the object file storage system does not contain the content.

[0197] If the content identification information includes the identification information of the first file and the line number of at least one line in the first file, then the object file storage system can be considered to have the content when it is determined that the first file is stored in the object file storage system.

[0198] The metadata of the first file includes the file path of the first file, so the computing engine reads the file path of the first file from the metadata of the first file.

[0199] Operation 4021 is optional. This means you can skip operation 4021 and directly execute operation 4022. Alternatively, you can execute operation 4021 first, and then execute operation 4022.

[0200] 4022: The Hive engine obtains second permission information from the linkage permission module, which includes the content identifier information of the content, the user identity that can access the content, and the fourth operation type of the fourth access operation that can access the content.

[0201] In some embodiments, the linkage permission module includes a second read / write interface. The Hive engine reads each second permission information stored in the linkage permission module through the second read / write interface, and obtains the second permission information corresponding to the content based on the content identifier information of the content.

[0202] In some embodiments, the Hive engine sends a second retrieval command to the linkage permission module, the second retrieval command including the content identifier information of the content. The linkage permission module receives the second retrieval command, retrieves the second permission information corresponding to the content from each saved second permission information, based on the content identifier information, and sends a second retrieval response to the Hive engine, the second retrieval response including the retrieved second permission information.

[0203] If the content identification information includes the identification information of the first file and the content identification information does not include other information, then the obtained second permission information is the second permission information that includes the identification information of the first file.

[0204] If the content identification information includes the identification information of the first file and the line number of at least one line in the first file, and the content identification information does not include other information, then the obtained second permission information is the second permission information that includes the identification information of the first file.

[0205] If the content identification information includes the identification information of the first file and the column identifier of at least one column in the first file, and the content identification information does not include other information, the obtained second permission information is the second permission information that includes the identification information of the first file and the column identifier of the at least one column.

[0206] If the content identification information includes the identification information of the first file and the identification information of the partition in the first file, and the content identification information does not include other information, then the obtained second permission information is the second permission information that includes the identification information of the first file and the identification information of the partition.

[0207] If the content identification information includes the identification information of the first file and the identification information of the data fragment in the first file, and the content identification information does not include other information, then the obtained second permission information includes the identification information of the first file and the second permission information of the data fragment.

[0208] 4023: The Hive engine authenticates the first user's access rights to the content based on the second permission information, the first user's account information, and the first operation type.

[0209] In 4023, the Hive engine determines the user's identity based on the user's account information, compares the user's identity with the user identity included in the second permission information, and compares the first operation type with the fourth operation type included in the second permission information. If the comparison shows that the user's identity matches the user identity included in the second permission information, and that the first operation type matches the fourth operation type included in the second permission information, then the authentication for the first user to access the content is successful.

[0210] In some embodiments, the identity authentication center stores the correspondence between user account information and user identity. The Hive engine queries the identity authentication center for the first user's identity based on the first user's account information.

[0211] See Figure 6 In some embodiments, the computing engine is the Spark engine, which authenticates the first user's access to the content through operations 4121 to 4125 as follows.

[0212] 4121: The Spark engine sends the access request information to the metadata center.

[0213] In some embodiments, the Spark engine also sends the first user's account information to the metadata center.

[0214] 4122: For the content identifier information of the content in the access request information, the metadata center determines whether the content exists in the object file storage system based on the content identifier information. If the content exists in the object file storage system, the following operation 4123 is performed.

[0215] For details on how the metadata center determines whether the object file storage system contains the content, please refer to section 4021 above for the detailed process by which the Hive engine determines whether the object file storage system contains the content. These details will not be elaborated upon here.

[0216] If the Spark engine also sends the first user's account information to the metadata center, the metadata center, upon determining that the content exists in the object file storage system, retrieves the second permission information, including the content identifier information, from the linked permission module. Based on this second permission information, the first user's account information, and the first operation type, it authenticates the first user's permission to access the content. After the first user's permission to access the content is authenticated, the following operations 4123 are executed.

[0217] The process by which the metadata center obtains the second permission information and authenticates the first user's access to the content is described in sections 4022 and 4023 above. It will not be explained in detail here.

[0218] 4123: The metadata center sends a confirmation message to the Spark engine.

[0219] Specifically, if the content does not exist in the object file storage system, a denial message is sent to the Spark engine. Alternatively, if authentication for the first user to access the content fails, a denial message is sent to the Spark engine.

[0220] 4124: The Spark engine receives the confirmation information and obtains the second permission information, which includes the content identifier information of the content, from the linkage permission module. The second permission information includes the content identifier information of the content, the identity of the user who can access the content, and the fourth operation type of the fourth access operation that can access the content.

[0221] The process of Spark engine obtaining the second permission information is described in section 4022 above, and will not be explained in detail here.

[0222] If the Spark engine receives this denial message, the operation ends.

[0223] 4125: The Spark engine authenticates the first user's access rights to the content based on the second permission information, the first user's account information, and the first operation type.

[0224] The Spark engine's authentication process for the first user's access to the content is similar to the Hive engine's authentication process for the first user's access to the content described in section 4023 above, and will not be explained in detail here.

[0225] Among them, the operations 4121-4123 above are optional operations, that is, the operations 4121-4123 can be omitted, and Spark will directly execute the operations 4124-4125. That is, the Spark engine obtains the second permission information, including the content identifier information of the content, from the linkage permission module, and authenticates the first user's permission to access the content based on the second permission information, the first user's account information and the first operation type.

[0226] Step 403: The calculation engine determines the granularity of accessing the content of the first file based on the access requirement information. If the determined granularity is the first granularity, proceed to step 404. If the determined granularity is the second granularity, proceed to step 409.

[0227] In step 403, the calculation engine determines the granularity of accessing the content of the first file based on the content identification information of the content included in the access request information.

[0228] The above describes four types of access request information. Next, we will explain the process of determining the granularity for each type of access request information.

[0229] For the access request information of type 1 mentioned above, this access request information includes the identification information of the first file. However, this access request information does not include the identification information of the partitions of the first file. Furthermore, when the first file is a structured data file, this access request information does not include the first information. When the first file is a semi-structured data file, this access request information does not include the identification information of the data segments in the first file. In this case, the content identification information is the identification information of the first file, and the calculation engine determines the granularity of accessing the content of the first file to be at the file granularity.

[0230] For the access request information of type 2 above, the access request information includes the identifier information of the first file and the identifier information of the partition in the first file. Furthermore, when the first file is a structured data file, the access request information does not include the first information. When the first file is a semi-structured data file, the access request information does not include the identifier information of the partition in the first file. In this case, the content identifier information is the identifier information of the first file and the identifier information of the partition in the first file, and the calculation engine determines the granularity of accessing the content of the first file to be the partition granularity.

[0231] The first granularity is either file granularity or partition granularity. Therefore, if the determined granularity is the first granularity, the content identification information of the content includes the identification information of the first file, or the content identification information of the content includes the identification information of the first file and the identification information of the partition in the first file.

[0232] For the access request information of type 3 above, the first file is a structured data file. The access request information includes the identification information of the first file and first information, whereby the first information indicates at least one row and / or at least one column in the first file. In this case, the content identification information is the identification information of the first file and the column identifier of the at least one column in the first file, or the content identification information is the identification information of the first file and the row number of the at least one row in the first file. The calculation engine determines the granularity of accessing the content of the first file to be row-column granularity.

[0233] For the access request information of type 4 above, the first file is a semi-structured data file. This access request information includes the identification information of the first file and the identification information of the data segments in the first file. At this time, the content identification information is the identification information of the first file and the identification information of the data segment in the first file. The calculation engine determines the granularity of accessing the content of the first file to be the data segment granularity.

[0234] The second granularity can be either row / column granularity or data segment granularity. Therefore, if the determined granularity is the second granularity, the content identification information includes the identification information of the first file and the column identifier of at least one column in the first file. Optionally, the access request information may also include row filtering information corresponding to the at least one column. Alternatively, the content identification information includes the identification information of the first file and the row number of at least one row in the first file. Or, the content identification information includes the identification information of the first file and the identification information of the data segment in the first file.

[0235] The computing engine includes a computing module and a routing module. Step 402 is executed by the computing module, and step 403 is executed by the routing module. The computing module may execute step 402 first, followed by the routing module executing step 403. Alternatively, the routing module may execute step 403 first, followed by the computing module executing step 402. Or, the computing module may execute step 402 simultaneously with the routing module. After the computing engine successfully authenticates the first user's access to the content, and if the granularity determined by the routing module is the first granularity, it executes step 404; if the granularity determined by the routing module is the second granularity, it executes step 409.

[0236] Step 404: The computing engine sends an authentication request to the file path authentication module. The authentication request includes authentication information, which is used to instruct the first user, the file path of the first file, and the second access operation of the first user to access the file path.

[0237] In some embodiments, the authentication information includes the file path of the first file, the user identity of the first user, and the second operation type. The second operation type is the operation type that can access the object file storage system corresponding to the first operation type. The computing engine can obtain the second operation type by mapping the first operation type. The second operation type includes read operations and / or write operations.

[0238] In some embodiments, the authentication information includes the identifier information of the first file, the account information of the first user, and the first operation type. The authentication information may also include other information, which will not be listed here.

[0239] See Figure 5 or Figure 6 In step 404, the routing module of the computing engine (Hive engine or Spark engine) sends an authentication request to the file path authentication module.

[0240] Step 405: The file path authentication module receives the authentication request and authenticates the first user's permission to access the file path using the second access operation based on the first permission information and the authentication information. The second access operation is the access operation corresponding to the second operation type.

[0241] The linkage permission module stores the first permission information corresponding to at least one file. For any file, the first permission information corresponding to the file includes the file path of the file, the identity of the user who can access the file path, and the third operation type of the third access operation that can access the file path.

[0242] In some embodiments, the authentication information includes the file path of the first file, the user identity of the first user, and the second operation type. In step 405, the file path authentication module reads first permission information, including the file path, from the linked permission module based on the file path. The read first permission information corresponds to the first file. The user identity of the first user is compared with the user identity included in the read first permission information, and the second operation type is compared with the third operation type included in the read first permission information. If the comparison shows that the user identity of the first user and the user identity included in the read first permission information are the same, and the comparison shows that the second operation type and the third operation type included in the read first permission information are the same, then the authentication of the first user's access to the file path using the second access operation is successful.

[0243] In some embodiments, the authentication information includes the identification information of a first file, the account information of a first user, and a first operation type. The file path authentication module first obtains the file path of the first file based on the identification information of the first file, obtains the user identity of the first user based on the account information of the first user, and maps the first operation type to obtain a second operation type. Then, it authenticates the first user's permission to access the file path using the second access operation.

[0244] In some embodiments, the file path authentication module obtains metadata including the identification information of the first file from the metadata center. This metadata is the metadata of the first file, and the file path of the first file is read from the metadata of the first file. Additionally, the file path authentication module obtains the user identity of the first user from the identity authentication center based on the account information of the first user.

[0245] Step 406: When the file path authentication module successfully authenticates the first user's permission to access the file path using the second access operation, it sends storage information to the object file storage system and an authentication response to the computing engine. The storage information includes a temporary credential, the file path, and the second operation type. The authentication response includes the temporary credential.

[0246] In step 406, when the file path authentication module successfully authenticates the first user's permission to access the file path using the second access operation, it assigns a temporary credential.

[0247] Step 407: The object file storage system receives the storage information and saves the corresponding relationship between the temporary credential, the file path, and the second operation type.

[0248] The object file storage system stores the correspondence between temporary credentials, file paths, and operation types. In step 407, the object file storage system receives this storage information and saves the temporary credential, the file path, and the second operation type in the correspondence between temporary credentials, file paths, and operation types.

[0249] In the mapping relationship between temporary credentials, file paths, and operation types, if the temporary credential is stored for a specified duration, the object file storage system deletes the record containing the temporary credential from the mapping relationship between temporary credentials, file paths, and operation types.

[0250] Step 408: The computing engine receives the authentication response and, based on the temporary credential, the access request information, and the file path, accesses the first file in the object file storage system.

[0251] The access request information includes the content identification information of the content and the first operation type of the first access operation. The content identification information includes the identification information of the first file, or the content identification information includes the identification information of the first file and the identification information of the partition in the first file.

[0252] Assuming the first access operation is to query the first file, the second access operation mapped from the first access operation includes a read operation. In step 408, the first file is accessed according to the following process.

[0253] 1-1: The computing engine sends a read request to the object file storage system, which includes the temporary credential and the file path of the first file.

[0254] For example, consider the access request information listed above: Select * From Company information. In this example, the first operation type "Select" queries the first file, and the content identifier information is "Company information". The second access operation mapped by the computing engine to "Select" includes a read operation.

[0255] Assume the temporary credential assigned by the file path authentication module is "P1", and the object file storage system stores the correspondence between temporary credentials, file paths, and operation types as shown in Table 2 below. The first record in this correspondence includes temporary credential 1, the file path of the first file as shown in Table 1, "C:\windows\system32\Companyinformation", and the second operation type "read operation" for the second access operation to access that file path.

[0256] Table 2

[0257]

[0258] In 1-1, the computing engine sends a read request to the object file storage system, which includes the temporary credential "P1" and the file path of the first file "C:\windows\system32\Company information".

[0259] 1-2: The object file storage system receives the read request and, based on the temporary credentials included in the read request, obtains the corresponding file path and second operation type from the correspondence between the temporary credentials, file path, and operation type.

[0260] For example, the object file storage system receives the read request, which includes a temporary credential "P1" and the file path of the first file, "C:\windows\system32\Company information". Based on the temporary credential "P1", the corresponding file path "C:\windows\system32\Company information" and the second operation type "read operation" are obtained from the correspondence between temporary credentials, file paths and operation types shown in Table 2.

[0261] 1-3: If the file path included in the read request is the same as the file path obtained and the second access operation corresponding to the second operation type includes a read operation, the object file storage system reads the first file based on the file path of the first file and returns the first file to the computing engine.

[0262] The file path "C:\windows\system32\Company information" included in the read request is the same as the file path "C:\windows\system32\Company information" obtained. Furthermore, the second access operation corresponding to the second operation type (read operation) obtained includes a read operation. Therefore, the object file storage system reads the first file as shown in Table 1 based on the file path "C:\windows\system32\Company information" of the first file and returns the first file as shown in Table 1 to the computing engine.

[0263] 1-4: The computing engine receives the first file, and returns the first file to the first user, where the content identification information includes the identification information of the first file; where the content identification information includes the identification information of the first file and the identification information of the partition in the first file, the engine obtains the content of the partition from the first file and returns the content of the partition to the first user.

[0264] The content identification information includes the identification information of the first file, indicating that the first user needs to query the entire content of the first file. The content identification information also includes the identification information of the first file and the identification information of the partition within the first file, indicating that the first user needs to query the content of that partition within the first file.

[0265] For example, the computing engine receives a first file as shown in Table 1, and the content identification information includes the file identification information "Company information" as shown in Table 1, and returns the first file as shown in Table 1 to the first user.

[0266] Assuming the first access operation is to update the first file, the second access operation mapped from the first access operation includes read and write operations, and the access request information includes the content to be updated. In step 408, the first file is accessed according to the following process.

[0267] 2-1: The computing engine sends a read request to the object file storage system, which includes the temporary credential and the file path.

[0268] 2-2: The object file storage system receives the read request and, based on the temporary credentials included in the read request, obtains the corresponding file path and the second operation type from the correspondence between the temporary credentials, file path, and operation type.

[0269] 2-3: If the file path included in the read request is the same as the file path obtained and the second access operation corresponding to the second operation type includes a read operation, the object file storage system reads the first file based on the file path and returns the first file to the computing engine.

[0270] 2-4: The computing engine receives the first file, and updates the content in the first file to the content to be updated, where the content identification information includes the identification information of the first file and the identification information of the partition in the first file; and updates the content of the partition in the first file to the content to be updated, where the content identification information includes the identification information of the first file and the identification information of the partition in the first file.

[0271] 2-5: The computing engine sends a write request to the object file storage system, which includes the temporary credential, the first file, and the file path.

[0272] 2-6: The object file storage system receives the write request and, based on the temporary credentials included in the write request, obtains the corresponding file path and second operation type from the correspondence between the temporary credentials, file path, and operation type.

[0273] 2-7: If the file path included in the write request is the same as the file path obtained and the second access operation corresponding to the second operation type includes a write operation, the object file storage system replaces the first file saved at that file path with the first file included in the write request.

[0274] The first access operation could also be other operations, such as deleting the first file, etc., which will not be listed here.

[0275] Step 409: The computing engine sends an access instruction to the data filtering engine, which includes the file path of the first file and the access request information.

[0276] See Figure 5 or Figure 6 In step 409, the routing module of the computing engine (Hive engine or Spark engine) sends an access command to the data filtering engine.

[0277] Step 410: The data filtering engine receives the access instruction and accesses the first file based on the administrator account information, the file path, and the access request information.

[0278] In some embodiments, the data filtering engine authenticates the first user's access to the content based on the second permission information, the first user's account information, and the access request information. After the first user's access to the content is authenticated, the first file is accessed based on the administrator account information, the file path, and the access request information. The second permission information is used to indicate the identity of the user who can access the content and the fourth access operation that allows access to the content.

[0279] The detailed process of the data filtering engine authenticating the first user's access to the content is described in step 402 above, and will not be elaborated further here.

[0280] The first file is a structured data file. The access request information includes the identification information of the first file, first information, and a first operation type for the first access operation. The first information includes a column identifier of at least one column of the first file, or the first information includes a column identifier of at least one column of the first file and row filtering information corresponding to the at least one column, or the first information includes the row number of the at least one row in the first file. Alternatively, the first file is a semi-structured data file, and the content identification information includes the identification information of the first file and the identification information of data segments in the first file.

[0281] Assuming the first access operation is to query the first file, the second access operation mapped from the first access operation includes a read operation. In step 410, the first file is accessed according to the following process.

[0282] 3-1: The data filtering engine sends a read request to the object file storage system. The read request includes the administrator account information and the file path of the first file.

[0283] For example, consider the access request information listed above: Select *From Company informationWhere City=City1. In this example, the first operation type "Select" queries the first file. The content identifier information includes "Company information" and the column identifier "City" in the fourth column. The row filter information corresponding to the fourth column is "City1". The second access operation mapped by the calculation engine to "Select" includes a read operation.

[0284] The data filtering engine sends a read request to the object file storage system. The read request includes the administrator account information "administrators" and the file path of the first file "C:\windows\system32\Companyinformation".

[0285] 3-2: When the object file storage system receives the read request and determines that the account information included in the read request is administrator account information, it reads the first file based on the file path of the first file and returns the first file to the data filtering engine.

[0286] Because the administrator has higher privileges, the object file storage system can directly read the first file based on its file path when it determines that the account information included in the read request is that of the administrator account.

[0287] For example, the object file storage system receives the read request, reads the first file as shown in Table 1 based on the file path "C:\windows\system32\Company information", and returns the first file as shown in Table 1 to the data filtering engine.

[0288] 3-3: The data filtering engine receives the first file, and in the access request information, it includes the first information. Based on the first information, it obtains the content in the first file. In the content identification information, it includes the identification information of the data fragment in the first file. It then obtains the content in the first file, which is the content of the data fragment.

[0289] In some embodiments, the first information includes a column identifier of at least one column in the first file, and the data filtering engine obtains the content of the at least one column from the first file based on the first information and returns the content of the at least one column to the calculation engine.

[0290] In some embodiments, the first information includes a column identifier and row filtering information of at least one column in the first file. The data filtering engine obtains the content of the at least one column from the first file based on the first information as one or more rows of the row filtering information, and returns the one or more rows of the content to the calculation engine.

[0291] For example, the data filtering engine receives the first file as shown in Table 1, and based on the column identifier "City" in the fourth column and the row filtering information "City 1" corresponding to the fourth column, it obtains three rows from the first file as shown in Table 1 whose fourth column contains "City 1". These three rows are the first, second and fifth rows in Table 1. The engine returns the contents of these three rows to the calculation engine, and the calculation engine returns the contents of these three rows to the first user.

[0292] In some embodiments, the first information includes the line number of at least one line in the first file, and the data filtering engine obtains the content of the at least one line from the first file based on the first information and returns the content of the at least one line to the calculation engine.

[0293] In some embodiments, the first information includes identification information of a data segment in the first file. The data filtering engine obtains the content of the data segment from the first file based on the first information and returns the content of the data segment to the computing engine.

[0294] 3-4: The data filtering engine returns the contents of the first file to the calculation engine.

[0295] 3-5: The computing engine receives the contents of the first file and returns the contents of the first file to the first user.

[0296] Assuming the first access operation is to update the first file, the second access operation mapped from the first access operation includes read and write operations, and the access request information includes the content to be updated. In step 410, the first file is accessed according to the following process.

[0297] 4-1: The data filtering engine sends a read request to the object file storage system. The read request includes the administrator account information and the file path of the first file.

[0298] 4-2: When the object file storage system receives the read request and determines that the account information included in the read request is administrator account information, it reads the first file based on the file path of the first file and returns the first file to the data filtering engine.

[0299] 4-3: The data filtering engine receives the first file, and in the access request information including the first information, updates at least one column or at least one row indicated by the first information in the first file to the content to be updated; in the access request information including the identification information of the data segment in the first file, updates the content in the data segment in the first file to the content to be updated.

[0300] In some embodiments, the first information includes a column identifier of at least one column in the first file, and the data filtering engine updates the at least one column in the first file with the content to be updated.

[0301] In some embodiments, the first information includes a column identifier and row filtering information of at least one column in the first file. The data filtering engine determines from the first file that the content of the at least one column is the content of one or more rows of the row filtering information, and updates the content of the one or more rows to the content to be updated.

[0302] In some embodiments, the first information includes the line number of at least one line in the first file, and the data filtering engine updates the content of the at least one line in the first file to the content to be updated based on the first information.

[0303] In some embodiments, the first information includes identification information of a data segment in the first file, and the data filtering engine updates the content of the data segment in the first file to the content to be updated based on the identification information of the data segment.

[0304] 4-4: The data filtering engine sends a write request to the object file storage system. The write request includes the administrator account information, the file path of the first file, and the updated first file.

[0305] 4-6: When the object file storage system receives the write request and determines that the account information included in the write request is administrator account information, it replaces the first file saved at the file path with the first file included in the write request.

[0306] The first access operation could also be other operations, such as deleting the first file, etc., which will not be listed here.

[0307] In some embodiments, for the first permission information and the second permission information in the linkage permission module, the linkage permission module receives the second permission information configured by the permission administrator and generates the first permission information based on the second permission information.

[0308] In this way, the permissions administrator authorizes access to file content by configuring the second permission information, and the linkage permission module generates the first permission information based on the second permission information. The computing engine in the access system uses the second permission information for authentication, while the file path authentication module uses the first permission information for authentication. This way, two-dimensional authentication is achieved with only one authorization from the permissions administrator.

[0309] In some embodiments, the first user accesses the first file in an open-source manner. Specifically, the first user sends authentication information to the file path authentication module via a client. This authentication information includes the file path of the first file, the first user's identity, and the second operation type of the second access operation. The file path authentication module authenticates the first user's permission to access the file path using the second access operation based on this authentication information. Upon successful authentication, it sends an authentication response to the client. This response includes a temporary credential and storage information sent to the object file storage system. This storage information includes the temporary credential, the file path, and the second operation type. The client receives the authentication response and, based on the temporary credential, the file path, and the access request information, accesses the first file in the object file storage system. This achieves transparent file access.

[0310] In this embodiment, since the computing engine determines the granularity of accessing the content in the first file based on the access request information, when the determined granularity is the first granularity, the computing engine requests the file path authentication module to authenticate the first user's permission to access the file path of the first file. After successful authentication, it obtains a temporary credential allocated by the file path authentication module. Based on this temporary credential, the access request information and the file path are used to access the object file storage system. Because the computing engine directly accesses the object file storage system, the file read and write performance is improved. When the determined granularity is the second granularity, the computing engine requests the data filtering engine to access the object file storage system. The data filtering engine includes specified administrator account information, which allows it to read the first file from the object file storage system. Based on the second granularity, it segments the data that the first user needs to access from the first file, thereby providing the user with access services that are smaller in granularity than the file itself, enriching the access services provided to the user.

[0311] See Figure 7This application provides a method 700 for obtaining first permission information. Wherein, the above... Figure 1 or Figure 3 The first permission information in the illustrated embodiment, or, as described above. Figure 4 The first permission information in the illustrated embodiment is obtained through method 700. Method 700 includes the following steps:

[0312] Step 701: The linkage permission module receives the second permission information, which is used to indicate the user identity and the fourth access operation that can access the content of the first file.

[0313] The linkage permission module retrieves the metadata of the first file from the metadata center. The metadata of the first file is any metadata stored in the metadata center. It retrieves at least one user identity from the identity authentication center and displays a second interface to the permission administrator. The second interface includes the metadata of the first file and the at least one user identity.

[0314] In this way, the access administrator selects the content identifier information of the content in the first file from the metadata of the first file, selects the user identity that can access the content from the at least one user identity, and inputs the fourth operation type of the fourth access operation that allows access to the content into the second interface, thus obtaining the second access information. The second access information includes the content identifier information, the selected user identity, and the input fourth operation type. The linked access module reads the second access information from the second interface.

[0315] In some embodiments, the first file is a structured data file, and the metadata of the first file includes the file identifier of the first file and the column identifier of each column of the first file. Optionally, the content identifier information of the content selected by the permission administrator includes the file identifier of the first file, or the content identifier information of the content selected by the permission administrator includes the file identifier of the first file and the column identifier of at least one column in the first file, or the content identifier information of the content selected by the permission administrator includes the file identifier of the first file and the row number of at least one row in the first file.

[0316] In some embodiments, the first file is a semi-structured data file, and the metadata of the first file includes the file identifier of the first file and the identification information of each data segment of the first file. Optionally, the content identification information of the content selected by the permission administrator includes the file identifier of the first file, or the content identification information of the content selected by the permission administrator includes the file identifier of the first file and the identification information of at least one data segment in the first file.

[0317] Step 702: The linkage permission module generates the first permission information based on the second permission information.

[0318] In step 702, the linkage permission module generates the first permission information through the following operations 7021-7024, which are as follows:

[0319] 7021: The linkage permission module obtains the file path of the first file based on the content identifier information of the content in the second permission information.

[0320] In some embodiments, the content identification information of the content includes the identification information of the first file. The linkage permission module obtains metadata including the identification information of the first file from the metadata center. This metadata is the metadata of the first file, and the file path of the first file is obtained from the metadata of the first file.

[0321] 7022: The linkage permission module maps the fourth operation type included in the second permission information to obtain the third operation type.

[0322] The fourth operation type corresponds to the access operation configured by the administrator to access the contents of the first file. This fourth access operation can include querying, updating, or deleting the first file. The third operation type corresponds to the fourth operation type and allows access to the object file storage system. The third operation type includes read and / or write operations.

[0323] 7023: The linkage permission module reads the user identity from the second permission information and combines the file path of the first file, the user identity, and the third operation type to form the second permission information.

[0324] Step 703: The linkage permission module saves the first permission information and the second permission information.

[0325] The above steps 701-703 can be repeated to generate a large amount of first and second permission information from the linkage permission module.

[0326] In this embodiment, the linked permission module receives the second permission information configured by the permission administrator and generates first permission information based on the second permission information. The first permission information is used to indicate the user's identity and access operation that can access the file path of the first file. This allows for automatic generation of the first permission information, improving the efficiency and reducing the cost of obtaining it. Since the permission administrator only needs to configure the second permission information, the linked permission module automatically generates the first permission information based on it. The second permission information is used to authenticate the user's permission to access the content of the first file, and the first permission information is used to authenticate the user's permission to access the file path of the first file. Thus, the permission administrator only needs to authorize once (configuring the second permission information), and the system uses both the second and first permission information for two-dimensional authentication.

[0327] See Figure 8 This application provides a file access device 800, which can be deployed... Figure 1 or Figure 3 The computing engine in the system shown, or deployed on Figure 4 , Figure 5 or Figure 6 On the computing engine in the illustrated embodiment. The device 800 includes:

[0328] The communication unit 801 is used to receive a data access request, which includes access request information, which indicates the contents of a first file that the first user needs to access, and the first file is stored in an object file storage system.

[0329] Processing unit 802 is configured to access the first file based on the account information of the first user and the access request information when the granularity of accessing the content in the first file is determined to be the first granularity based on the access request information.

[0330] The processing unit 802 is further configured to access the first file based on the specified administrator account information and the access request information when the granularity of accessing the content in the first file is determined to be the second granularity based on the access request information, wherein the second granularity is smaller than the first granularity.

[0331] Optionally, for details of the implementation process of the communication unit 801 receiving data access requests, please refer to [link to relevant documentation]. Figure 4 The details of step 401 in the illustrated embodiment will not be described in detail here.

[0332] Optionally, for the detailed implementation process of the processing unit 802 accessing the first file based on the first user's account information and the access request information, see [link to documentation]. Figure 4 The details of steps 405-408 in the illustrated embodiment will not be described in detail here.

[0333] Optionally, the processing unit 802 accesses the first file based on the specified administrator account information and the access request information. For a detailed implementation process, please refer to [link to documentation]. Figure 4 The details of steps 409-410 in the illustrated embodiment will not be described in detail here.

[0334] Optionally, the access request information includes the identification information of the first file, where the first granularity is the file granularity; or,

[0335] Optionally, the access request information includes the identification information of the first file and the identification information of the partitions in the first file, with the first granularity being the partition granularity.

[0336] Optionally, the communication unit 801 is further configured to send an authentication request to the file path authentication module. The authentication request includes authentication information, which is used to indicate the first user, the file path of the first file, and the first user's access operation to the file path. The authentication information is obtained based on the access request information and the account information of the first user. The authentication request is used to trigger the file path authentication module to authenticate the first user's permission to access the file path using the access operation based on the first permission information and the authentication information. The file path is used to indicate the storage location of the first file, and the first permission information is used to indicate the identity of the user who can access the file path and the access operation.

[0337] The communication unit 801 is also used to receive the authentication response sent by the file path authentication module after the permission is authenticated. The authentication response includes a temporary credential, and the temporary credential, the file path and the operation type of the access operation are stored in the object file storage system.

[0338] Processing unit 802 is used to access the first file based on the temporary credential, the access request information, and the file path.

[0339] Optionally, for details on how communication unit 801 sends an authentication request to the file path authentication module, see [link to documentation]. Figure 4 The details of step 404 in the illustrated embodiment will not be described in detail here.

[0340] Optionally, for the detailed implementation process of the communication unit 801 receiving the authentication response, please refer to [link / reference]. Figure 4 The details of step 408 in the illustrated embodiment will not be described in detail here.

[0341] Optionally, the processing unit 802 accesses the first file based on the temporary credential, the access request information, and the file path. For a detailed implementation process, see [link to relevant documentation]. Figure 4 The details of step 408 in the illustrated embodiment will not be described in detail here.

[0342] Optionally, the first file is a structured data file that stores data in list format. The access request information includes the identification information of the first file and first information, whereby the first information indicates at least one column and / or at least one row of the first file, and the second granularity is row and column granularity; or,

[0343] The first file is a semi-structured data file, which includes at least one data segment. The data segment is used to store data with the same business attributes. The access requirement information includes the identification information of the first file and the identification information of one or more data segments in the first file. The second granularity is the data segment granularity.

[0344] Optionally, the communication unit 801 is further configured to send an access instruction to the data filtering engine, the access instruction including the access request information, the data filtering engine including administrator account information, and the access instruction being used to trigger the data filtering engine to access the first file based on the administrator account information and the access request information.

[0345] Optionally, for details on how the communication unit 801 sends access commands to the data filtering engine, see [link to documentation]. Figure 4 The details of step 409 in the illustrated embodiment will not be described in detail here.

[0346] Optionally, the processing unit 802 is also used for:

[0347] Based on the second permission information, the first user's account information, and the access request information, the first user's access permission to the content is authenticated. The second permission information is used to indicate the identity of the user who can access the content and the access operation.

[0348] After the first user's access permission to the content is authenticated, the granularity of accessing the content in the first file is determined based on the access request information.

[0349] Optionally, the processing unit 802 is also used for:

[0350] First permission information is generated based on the second permission information. The first permission information is used to indicate the user identity and access operation that can access the file path of the first file. The file path is used to indicate the storage location of the first file.

[0351] Optionally, for a detailed implementation process of the processing unit 802 generating the first permission information, see [link to documentation]. Figure 7 The details of step 702 in the illustrated embodiment will not be described in detail here.

[0352] In this embodiment, since the first granularity is larger than the second granularity, when the determined granularity is the first granularity, the processing unit accesses the first file based on the first user's account information and the access request information. This avoids using the administrator's account information to access the first file, improving the efficiency of accessing the first file and the performance of reading and writing it. When the determined granularity is the second granularity, the processing unit accesses the first file based on the specified administrator account information and the access request information. This uses the administrator's account information instead of the first user's account information to access the first file. Therefore, it is not necessary to configure permissions for the first user to access the second granularity, thus avoiding expanding the first user's access permissions and facilitating permission management.

[0353] See Figure 9This application provides a schematic diagram of a file access device 900. This device 900 can be a computing engine as described in any of the above embodiments, for example, it can be... Figure 1 , Figure 3 , Figure 4 , Figure 5 or Figure 6 The illustrated embodiment provides a computing engine. The device 900 includes at least one processor 901, internal connections 902, memory 903, and at least one transceiver 904.

[0354] The device 900 is a hardware-structured device that can be used to implement... Figure 8 The functional modules in the device 800. For example, those skilled in the art will conceive of them. Figure 8 The processing unit 802 in the illustrated device 800 can be implemented by the at least one processor 901 calling code in the memory 903. Figure 8 The communication unit 801 in the device 800 shown can be implemented by the transceiver 904.

[0355] Optionally, the device 900 can also be used to implement the functions of the computing engine in any of the above embodiments.

[0356] Optionally, the processor 901 described above may be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.

[0357] The aforementioned internal connection 902 may include a pathway for transmitting information between the aforementioned components. Optionally, the internal connection 902 may be a single board or a bus, etc.

[0358] The transceiver 904 described above is used for communicating with other devices or communication networks.

[0359] The aforementioned memory 903 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0360] The memory 903 stores the application code that executes the solution of this application, and its execution is controlled by the processor 901. The processor 901 executes the application code stored in the memory 903, and cooperates with at least one transceiver 904, thereby enabling the device 900 to realize the functions of the method of this patent.

[0361] In a specific implementation, as one example, the processor 901 may include one or more CPUs, for example... Figure 9 CPU0 and CPU1 in the CPU.

[0362] In a specific implementation, as one example, the device 900 may include multiple processors, for example... Figure 9 Processors 901 and 907 are mentioned. Each of these processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor here can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0363] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0364] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method of accessing a file, characterized by, The method comprises: receiving a data access request, the data access request comprising access requirement information, the access requirement information being used to indicate content in a first file that a first user needs to access, the first file being stored in an object file storage system; when it is determined based on the access requirement information that a granularity of accessing the content in the first file is a first granularity, accessing the first file based on account information of the first user and the access requirement information; when it is determined based on the access requirement information that the granularity of accessing the content in the first file is a second granularity, accessing the first file based on specified administrator account information and the access requirement information, the second granularity being smaller than the first granularity; wherein the first file is a structured data file, the first file stores data in a list form, the access requirement information comprises identification information of the first file and first information, the first information being used to indicate at least one row of the first file and / or at least one column of the first file, and the second granularity is a row-column granularity; or the first file is a semi-structured data file, the first file comprises at least one data segment, the data segment being used to save data with the same business attribute, the access requirement information comprises identification information of the first file and identification information of one or more data segments in the first file, and the second granularity is a data segment granularity.

2. The method of claim 1, wherein, The access requirement information comprises identification information of the first file, and the first granularity is a file granularity; or the access requirement information comprises identification information of the first file and identification information of a partition in the first file, and the first granularity is a partition granularity.

3. The method of claim 2, wherein, The accessing the first file based on the account information of the first user and the access requirement information comprises: sending an authentication request to a file path authentication module, the authentication request comprising authentication information, the authentication information being used to indicate the first user, a file path of the first file and an access operation of the first user on the file path, the authentication information being obtained based on the access requirement information and the account information of the first user, the authentication request being used to trigger the file path authentication module to authenticate, based on first permission information and the authentication information, a permission of the first user to access the file path with the access operation, the file path being used to indicate a storage location of the first file, and the first permission information being used to indicate a user identity and an access operation that can access the file path; receiving an authentication response sent by the file path authentication module after the permission authentication is passed, the authentication response comprising a temporary credential, the temporary credential, the file path and an operation type of the access operation being stored in the object file storage system in correspondence; accessing the first file based on the temporary credential, the access requirement information and the file path.

4. The method of claim 1, wherein, The accessing the first file based on the specified administrator account information and the access requirement information comprises: The access instruction is sent to a data filtering engine, the access instruction including the access requirement information, the data filtering engine including the administrator account information, and the access instruction being used to trigger the data filtering engine to access the first file based on the administrator account information and the access requirement information.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: authenticating the first user's access right to the content based on second right information, the first user's account information, and the access requirement information, the second right information being used to indicate a user identity and an access operation that can access the content; after the authentication of the first user's access right to the content is passed, determining a granularity of accessing content in the first file based on the access requirement information.

6. The method of claim 5, wherein, The method further includes: generating first right information based on the second right information, the first right information being used to indicate a user identity and an access operation that can access a file path of the first file, the file path being used to indicate a storage location of the first file.

7. An access system, characterized in that The system includes a computing engine and an object file storage system. The computing engine is configured to receive a data access request, the data access request including access requirement information, the access requirement information being used to indicate content in a first file that a first user needs to access, the first file being stored in the object file storage system. The computing engine is further configured to, when a granularity of accessing content in the first file is determined to be a first granularity based on the access requirement information, access the first file based on the first user's account information and the access requirement information. The computing engine is further configured to, when the granularity of accessing content in the first file is determined to be a second granularity based on the access requirement information, access the first file based on specified administrator account information and the access requirement information, the second granularity being smaller than the first granularity. The first file is a structured data file, the first file storing data in a list form, the access requirement information including identification information of the first file and first information, the first information being used to indicate at least one row of the first file and / or at least one column of the first file, and the second granularity being a row-column granularity; or The first file is a semi-structured data file, the first file including at least one data segment, the data segment being used to save data with the same business attribute, the access requirement information including identification information of the first file and identification information of one or more data segments in the first file, and the second granularity being a data segment granularity.

8. The system of claim 7, wherein, The access requirement information includes identification information of the first file, and the first granularity is a file granularity; or The access requirement information includes identification information of the first file and identification information of a partition in the first file, and the first granularity is a partition granularity.

9. The system of claim 8, wherein, The system further includes a file path authentication module, The computing engine is configured to send an authentication request to the file path authentication module, the authentication request comprising authentication information, the authentication information being used to indicate the first user, a file path of the first file, and an access operation of the first user on the file path, the authentication information being obtained based on the access requirement information and account information of the first user, and the file path being used to indicate a storage location of the first file. The file path authentication module is configured to authenticate, based on first permission information and the authentication information, a permission of the first user to use the access operation to access the file path, the first permission information being used to indicate a user identity and an access operation that can access the file path, and send an authentication response to the computing engine after the authentication of the permission is passed, the authentication response comprising a temporary credential. The object file storage system is configured to correspondingly save the temporary credential, the file path, and an operation type of the access operation. The computing engine is further configured to access the first file based on the temporary credential, the access requirement information, and the file path.

10. The system of claim 7, wherein, The system further comprises a data filtering engine, and the data filtering engine comprises the administrator account information. The computing engine is configured to send an access instruction to the data filtering engine, the access instruction comprising a file path of the first file and the access requirement information, and the file path being used to indicate a storage location of the first file. The data filtering engine is configured to access the first file based on the administrator account information, the file path, and the access requirement information.

11. The system of any of claims 7-10, wherein, The computing engine is further configured to: authenticate, based on second permission information, account information of the first user, and the access requirement information, a permission of the first user to access the content, and the second permission information being used to indicate a user identity and an access operation that can access the content; and determine, after the authentication of the permission of the first user to access the content is passed, a granularity of accessing content in the first file based on the access requirement information.

12. The system of claim 11, wherein, The system further comprises a linkage permission module, The linkage permission module is configured to generate first permission information based on the second permission information, and the first permission information being used to indicate a user identity and an access operation that can access a file path of the first file, and the file path being used to indicate a storage location of the first file.

13. An apparatus for accessing a file, the apparatus comprising: The apparatus comprises: a communication unit configured to receive a data access request, the data access request comprising access requirement information, the access requirement information being used to indicate content in a first file that a first user needs to access, and the first file being stored in an object file storage system; a processing unit configured to, when a granularity of accessing the content in the first file is determined to be a first granularity based on the access requirement information, access the first file based on account information of the first user and the access requirement information. The processing unit is further configured to, when determining that the granularity of accessing the content in the first file is a second granularity based on the access demand information, access the first file based on the specified administrator account information and the access demand information, and the second granularity is smaller than the first granularity. In the first file is a structured data file, the first file stores data in a list form, the access demand information includes identification information of the first file and first information, the first information is used to indicate at least one row of the first file and / or at least one column of the first file, and the second granularity is a row-column granularity; or, The first file is a semi-structured data file, the first file includes at least one data segment, the data segment is used to save data with the same business attribute, the access demand information includes identification information of the first file and identification information of one or more data segments in the first file, and the second granularity is a data segment granularity.

14. The apparatus of claim 13, wherein, The access demand information includes identification information of the first file, and the first granularity is a file granularity; or, The access demand information includes identification information of the first file and identification information of a partition in the first file, and the first granularity is a partition granularity.

15. The apparatus of claim 14, wherein The communication unit is further configured to send an authentication request to a file path authentication module, the authentication request including authentication information, the authentication information being used to indicate the first user, a file path of the first file, and an access operation of the first user on the file path, the authentication information being obtained based on the access demand information and account information of the first user, the authentication request being used to trigger the file path authentication module to authenticate, based on first permission information and the authentication information, a permission of the first user to access the file path using the access operation, the file path being used to indicate a storage location of the first file, and the first permission information being used to indicate a user identity and an access operation that can access the file path; The communication unit is further configured to receive an authentication response sent by the file path authentication module after the permission authentication is passed, the authentication response including a temporary credential, the temporary credential, the file path, and an operation type of the access operation being stored in the object file storage system in correspondence; The processing unit is configured to access the first file based on the temporary credential, the access demand information, and the file path.

16. The apparatus of claim 13, wherein The communication unit is further configured to send an access instruction to a data filtering engine, the access instruction including the access demand information, the data filtering engine including the administrator account information, and the access instruction being used to trigger the data filtering engine to access the first file based on the administrator account information and the access demand information.

17. The apparatus of any one of claims 13-16, wherein, The processing unit is further configured to The second permission information is used for indicating a user identity and an access operation capable of accessing the content. After the permission of the first user to access the content is authenticated, a granularity of accessing the content in the first file is determined based on the access demand information.

18. The apparatus of claim 17, wherein, The processing unit is further configured to: generate first permission information based on the second permission information, the first permission information being used for indicating a user identity and an access operation capable of accessing a file path of the first file, the file path being used for indicating a storage location of the first file.

19. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a computer to implement the method in any one of claims 1-6.

20. A computer program product, characterised in that, The computer program product comprises a computer program stored in a computer readable storage medium, and the computer program is loaded by a processor to implement the method in any one of claims 1-6.

Images