Permissions system and method

Abstract

The disclosure relates to a system for managing user permissions for accessing a data store and a related method. The system comprises: a central permissions database storing user permissions for accessing one or more documents stored in a data store; an access-control microservice for managing the central permissions database; and one or more search services, each providing a search function for searching documents stored in the data store and having a local permissions database, wherein in response to a first document being uploaded to the data store by a user, the access-control microservice is configured to: assign a first permission to the first document based on identity information of the user; store the first permission in the central permissions database; and generate an update message comprising changes to the permissions stored in the central permissions database; wherein the one or more search services are each configured to: read the update message generated by the access control service; and store the first permission in the local permissions database.

Description

[0001] PERMISSIONS SYSTEM AND METHOD

[0002] FIELD OF THE INVENTION

[0003] The invention relates to methods and systems for managing access to documents in a data store by using a distributed permissions system. According to the invention, when permissions in a permissions database are updated, the updated permissions are propagated to downstream services in the system. The services store the permissions in a local native database.

[0004] BACKGROUND

[0005] When hosting several data sets from various sources with highly sensitive information, there is a need to keep the data sets as separate as possible to avoid any data spillage, and at the same time provide data discoverability with minimal latency for a user.

[0006] Traditionally this would be done by keeping data on different servers. This approach is both costly and inefficient, as some of the data used by multiple users might be overlap and therefore the data would be duplicated across various servers. This results in significant wasted storage space.

[0007] Further, according to such an approach, if the user performed a search in several databases, across millions of documents, it would be technically impossible to send millions of requests to validate permissions while the user is awaiting results.

[0008] Therefore, a need exists to provide an efficient means of searching a large data store without compromising security of the stored data.

[0009] SUMMARY OF INVENTION

[0010] Aspects of the disclosure provide a permissions system that resolve the aforementioned problems by rapidly distributing permissions throughout the system. Services across the system can therefore quickly and reliably access permissions when required. A first aspect of the disclosure provides a system for managing user permissions for accessing a data store, the system comprising: a central permissions database storing user permissions for accessing one or more documents stored in a data store; an access-control microservice for managing the central permissions database; and one or more search services, each providing a search function for searching documents stored in the data store and having a local permissions database, wherein in response to a first document being uploaded to the data store by a user, the access-control microservice is configured to: assign a first permission to the first document based on identity information of the user; store the first permission in the central permissions database; and generate an update message comprising changes to the permissions stored in the central permissions database; wherein the one or more search services are each configured to: read the update message generated by the access control service; and store the first permission in the local permissions database.

[0011] In this way, the disclosure provides a permissions system in which a microservice distributes permissions in a central permissions database by generating the first message, so that downstream services and their associated local databases can synchronise themselves with the central permissions database. This avoids the need for the downstream services to perform a lengthy and inefficient process of validating permissions in the central permissions database.

[0012] Further, the local permissions database is kept in sync with permissions in the central permissions database. The first search service listens for updates to the central permissions database, and recognises that the first message generated by the microservice indicates that a permission is to be updated. The local permissions stored at the first search service, i.e. in the local permissions database, are therefore kept up to date with the central permissions database even when new documents are uploaded, orwhen permissions otherwise change.

[0013] Another advantage is that the present system is scalable, since a plurality of search services with respective local permissions databases can listen, or detect, the first message, thereby allowing changes in the central permissions database to rapidly filter through to various search services. That is, changes to the central permissions database are propagated downstream to local permissions databases in milliseconds.

[0014] As used throughout, the search service may also be understood to mean a search engine.

[0015] The disclosure also provides for significantly faster searching of multiple data sets. Further, the system introduces redundancy in that if the permissions server is offline, then search queries may still be carried out, because the permissions are available locally at downstream services.

[0016] Further, in some examples, the microservice may generate a second message in response to a user’s permission to access a document being revoked. The second message may comprise an indication that a permission has been revoked, thereby allowing downstream services to update their database of local permissions to reflect the change.

[0017] In some advantageous examples, the first permission may be stored in a native format of the local permissions database.

[0018] In some advantageous examples, the one or more search services comprises a first search service configured to: receive, from a user, a search query relating to the first document; search the local permissions database for permissions assigned to the first document; and output a result, to the user, based on if the user has permission to access the first document.

[0019] As such, when the user inputs a search query relating to the first document, that is, searches for the first document, the first search service searches the local database for a corresponding permission, which allows a result to be returned to the user faster than if the search service had to search the central permissions database for user permissions before returning documents. Further, because the local database is kept in sync with the central permissions database, the user will reliably be able to access permitted documents. Further, the system is particularly advantageous when the data store comprises a plurality of documents, for example thousands, hundreds of thousands, or millions of documents, as it is quicker by several orders of magnitude to check permissions kept on the native database than it is to request the permissions from the permissions database.

[0020] It may be understood that the first search service could also be referred to as a search pattern service.

[0021] In some implementations, if the user has permission to access the first document then the result comprises the first document, or if the user does not have permission to access the first document, then the result comprises a message indicating that access to the first document is denied.

[0022] In some further examples, the data store is configured to store a plurality of documents, and the first search service is configured to search a subset of the plurality of documents, wherein each document in the subset of the plurality of documents has the same document type.

[0023] In this way, a first search service may search for relevant documents in the data store. For example, the first search service may be configured to search for documents of a same type. When referring to a document type, we intend to mean the format of the document, or the type of data stored in the document. For example, one type of document may be a word searchable PDF, and another type of document may be a spreadsheet.

[0024] Additionally or alternatively, the one or more search services further comprises a second search service configured to directly communicate with the access-control microservice, wherein the second search service is further configured to: receive, from the user, a request to access a second document in the data store; and send the request to the microservice.

[0025] Such a system is efficient where the user requests to access one document because relatively few resources are required to cross-check permissions to access one document in the first permissions database. Further the microservice is further configured to: search the central permissions database to determine if the user has permission to access the second document; and output a result to the second search service, based on if the user has permission to access the second document.

[0026] Accordingly, fast and secure access to the requested document can be provided.

[0027] It is to be noted that the results returned to the second search service are dependent on if the user has permission to access the requested document. If they do have permission, then the search service may output that access to the document is permitted. On the other hand, if the user does not have suitable permission to access the requested document, then the request would be denied, and a corresponding output may be provided.

[0028] In some implementations, if the user has permission to access the second document then the result comprises the second document, or if the user does not have permission to access the second document, then the result comprises a message indicating that access to the first document is denied.

[0029] In some examples, the identity information is included in a JSON Web Token.

[0030] Further, the data store is configured to store a plurality of documents, and the central permissions database is configured to store all permissions corresponding to the plurality of documents.

[0031] In this sense, it would be understood that the first permissions database is a central database, and that all of the permissions relating to the documents in the data store are stored in the permissions database. As such, when the native databases of search services are synced with the permissions database, it can confidently be established if a user has permission to access a particular document by searching the native database of the search service.

[0032] In some examples the permissions database comprises a Postgres database. Further, the local database may be any relational database management system (RDBMS), an ElasticSearch database or a any graph database.

[0033] Accordingly, the system is compatible with a wide variety of databases, and has wide applicability. Accordingly, rapid searching of a variety of different types of data stores can be implemented.

[0034] In some examples, the system may comprise a plurality of first search services, and each of these may have a respective different local database. That is, one first search service may be configured to communicate with a Postgres database, while another first search service may be configured to communicate with an ElasticSearch database.

[0035] It would therefore be understood that the disclosure is not limited to any specific kind of local database, but has wide applicability.

[0036] A second aspect of the disclosure provides a computer implemented method performed by an access control microservice in a system for managing user permissions for accessing a data store, the method performed in response to a first document being uploaded to a data store by a user, the method comprising: assigning a first permission to the first document based on identity information of the user; storing the first permission in a central permissions database; and generate an update message comprising changes to the permissions stored in the central permissions database.

[0037] A third aspect of the disclosure provides a computer readable storage medium comprising instructions which, when executed by a processor, cause the processor to perform a method according to the second aspect.

[0038] A fourth aspect of the disclosure provides an access control microservice configured to perform a method according to the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0039] The disclosure will be discussed with reference to the accompanying drawings, in which:

[0040] Figure 1 is a high-level schematic representation of a system for managing user permissions for accessing a data store;

[0041] Figure 2A shows a method performed by an access-control microservice;

[0042] Figure 2B shows a method performed by a search service;

[0043] Figure 3A shows a method performed by a search service with a local permissions database for accessing a document;

[0044] Figure 3B shows a method performed by a search service that directly accesses the central permissions database; and

[0045] Figure 3C shows a method performed by the access-control microservice.

[0046] DETAILED DESCRIPTION

[0047] The disclosure provides a permissions system that rapidly distributes permissions throughout the system, thereby facilitating reliable and quick access to the permissions when required. The distributed permissions system of the disclosure exploits an access-control microservice to maintain synchronization between a central permissions database and at least one local permissions database.

[0048] A system 100 according to the disclosure is shown in Figure 1. System 100 includes ingestion-service 105, access-control service 110, central permissions database 115, acl-updates queue 120, as well as search pattern service-a 125, search pattern service-b 130, search pattern service-c 140, their respective local permissions databases 145, 150, 155, and direct access service-a 160, direct access service-b 165, and direct access service-c 170.

[0049] The ingestion-service 105 ingests the documents to which permissions in the permissions system relate. When documents are ingested identity information of the user is used to assign permissions to the ingested documents. The central permissions database 115 is a central database which stores permissions relating to all documents that are stored in a system, and may be implemented as a Postgres database. The central permissions database 115 acts as a single source of truth for all permissions in the system. Hence, when a document is uploaded to a data store by a user, a permission relating to that document is stored in the central permissions database 115 and local permissions databases 145, 150, 155 are synchronized to the central permissions database.

[0050] Access-control service 110 may also be referred to as access-control microservice or central microservice. The central microservice 110 is communicatively connected to ingestion-service 105, central permissions database 115, acl-updates block 120 and direct access service. The central microservice 110 assigns a permission to an uploaded document and stores the permission in the central permissions database 115. The central microservice 110 also generates an update message which contains details of the changes made in the central permissions database. The update message is typically implemented as a Kafka message which is sent to the acl-updates block 120. The search pattern services 125, 230 and 140 listen for changes on the acl-updates block 120 and thereby update their respective local permissions database 145, 150 and 155. In this way, the disclosure provides a distributed permission system whereby permissions in a central permissions database are synchronized with respective local permissions databases in real-time, thereby avoiding the need for search services to query the central permissions database directly. This is especially advantageous when a user is searching across a large number of documents, which would have a corresponding large number of permissions to be checked. Instead of checking the permissions in the central database 115, the search pattern services 125, 130, and 140 can reliably check the relevant permissions locally without needing to exchange data with the central permissions database 115.

[0051] Each of the search pattern services, those being service-d 125, service-e 130 and service-f 140 are configured to perform a search across large numbers of documents, for example millions, or hundreds of millions of documents. The search pattern services listen for an update message generated by or on the acl-updates block 120, read the update message and store a relevant permission in a respective local permissions database 145, 150, 155. A relevant permission is a permission that relates to a document that a search pattern service is configured to search for. For example, a particular search pattern service may be configured to search for documents of a particular type or format. As discussed above, each search pattern service is communicatively connected to a local permissions database 145, 450, 155. The local permissions databases store a copy of the permissions in the central permissions database 115. Moreover, each local permissions database typically stores the permissions that are relevant to the documents that can be searched for by its respective search pattern service. This avoids unnecessary information being stored in a local permissions database.

[0052] Each local permissions database 145, 150 and 155 stores the relevant permissions in the native format of the local permissions database 145, 150, 155. For example, the local permissions database could be a Postgres database, an ElasticSearch database, or a TigerGraph database. Other types of database could certainly be used to implement the local permissions database, and the skilled person would understand that different kinds of databases have difference advantages and uses, and would use a local permissions database that suited their intended use-case. The local permissions databases also store the data, i.e. the documents, to which the permissions in that local permissions database relate. This further facilitates fast and convenient document retrieval, since the permissions are stored alongside the data.

[0053] Direct access services, such as service-a 160, service-b 165 and service-c 170 are a different group of services to the search pattern services discussed above, Direct access services interact directly with the central microservice 110 via a remote procedure call framework such as gRPC, and requests information relating to a single document. That is, the direct access services generate queries of if user X can access document Y. It is generally more efficient to use a direct access service when querying access to a single document. In any case, since the direct access services interact directly with central microservice 110, they are also able to quickly and reliably access up to date permissions from the central permissions database 115.

[0054] Referring now to Figure 2A, a method 200 performed by the central microservice 110 in response to a user uploading a document to a data store is shown. In step 210, the central microservice 110 assigns a first permission to the first document, i.e. the document being uploaded, based on identity information of the user. The identity information could take any suitable form, but in some cases a JSON Web Token may be used. However, it is emphasised that many types of identity information exist. For example, identity information could be implemented thorugh session based authentication, API keys or Mutual Transport Layer Security (mTLS).

[0055] In a next step 220, the first permission is stored in the central permissions database 110 and in step 230 the central microservice 110 generates an update message that comprises changes to the central permissions database 115. That is, in this example, the details of the first permission are included in the generated update message.

[0056] Turning now to Figure 2B, a method 250 performed by a search service is shown. After the central microservice 110 generates the update message at step 230, at step 260 the search service is configured to read the generated update message. The search service typically reads the generated update message by reading or listening for updates on the acl-update block 120. Then at step 270 the search service stores the first permission in the local permissions database. That is, the search service stores the first permission in the local permissions database associated the search service.

[0057] The process of retrieving a document from the data store will now be discussed with reference to Figures 3A, 3B and 3C.

[0058] Referring first to Figure 3A, an example method 300 for retrieving a first document is shown. The method may be performed by a search pattern service, such as service-d 125, service-e 130 or service-f 140. In step 310, the search service receives a search query relating from a user relating to the first document. For example, the user may request to access, view, edit or delete the first document.

[0059] At step 320 the search service searches the local permissions database for permissions assigned to the first document. The identity associated with the permissions may be compared to the identity of the user. To ascertain the level of access that the user has.

[0060] Then, at step 330 a result is output to the use, based on if the user has permission to access the first document. To elaborate, if the user sought access to the first document and the user does have permission to access the first document then the result output to the user is the first document itself. On the other hand, if the user does not have permission to access the first document, then the outputted result may be a message indicating to the use that they do not have permission to access the requested document or documents.

[0061] Referring now to Figure 3B, a method 340 of requesting access to a document via a search service that directly communicates with the central microservice 110 is shown. In step 350, the search service, such as service-a 160, service-b 165 or service-c 170 receives, from the user, a request to access a document in the data store. As noted above, requests from the direct access services typically relate to one document. However, it is feasible that requests from the direct access services could relates to tens or hundreds of documents, as these could still be processed at an acceptable speed.

[0062] At step 360, the request is sent to the central microservice 110.

[0063] Referring now to Figure 3C, a method 370 performed by the central microservice 110 following step 360 is shown. In step 380 the central microservice 110 searches the central permissions database 115 to determine if the user has permission to access the requested document. At step 390, depending on if the user has permission to access the requested document a result is output to the user. For example, if the user sought access to a document and the user does have permission to access that document then the result output to the user is the first document itself. On the other hand, if the user does not have permission to access the first document, then the outputted result may be a message indicating to the use that they do not have permission to access the requested document.

[0064] It has been demonstrated that by synchronising permissions from a central permissions database to local permissions databases, the need for downstream search services to search the central permissions database is avoided, thereby improving search efficiency and the speed with which results can be returned. Additionally, the permissions are stored locally in the native format of the local permissions database, thereby avoiding the need to contact the central permissions database when the search query is executed.

[0065] The methods and processes described above can be implemented as code (e.g., software code). The decoder discussed above may be implemented in hardware or software as is well-known in the art. For example, hardware acceleration using a specifically designed Field Programmable Gate Array (FPGA) may provide certain efficiencies.

[0066] For completeness, such code can be stored on one or more computer-readable media, which may include any device or medium that can store code and / or data for use by a computer system. When a computer system reads and executes the code stored on a computer-readable medium, the computer system performs the methods and processes embodied as code stored within the computer-readable storage medium. In certain embodiments, one or more of the steps of the methods and processes described herein can be performed by a processor (e.g., a processor of a computer system or data storage system).

Claims

CLAIMS1 . A system for managing user permissions for accessing a data store, the system comprising: a central permissions database storing user permissions for accessing one or more documents stored in a data store; an access-control microservice for managing the central permissions database; and one or more search services, each providing a search function for searching documents stored in the data store and having a local permissions database, wherein in response to a first document being uploaded to the data store by a user, the access-control microservice is configured to: assign a first permission to the first document based on identity information of the user; store the first permission in the central permissions database; and generate an update message comprising changes to the permissions stored in the central permissions database; wherein the one or more search services are each configured to: read the update message generated by the access-control service; and store the first permission in the local permissions database.

2. A system according to claim 1 , wherein the first permission is stored in a native format of the local permissions database.

3. A system according to claim 1 or 2, wherein the one or more search services comprises a first search service configured to: receive, from a user, a search query relating to the first document; search the local permissions database for permissions assigned to the first document; and output a result, to the user, based on if the user has permission to access the first document.

4. A system according to claim 3, wherein if the user has permission to access the first document then the result comprises the first document, or if the user does not have permission to access the first document, then the result comprises a message indicating that access to the first document is denied.

5. A system according to claim 3 or 4, wherein the data store is configured to store a plurality of documents, and the first search service is configured to search a subset of the plurality of documents, wherein each document in the subset of the plurality of documents has the same document type.

6. A system according to any one of claims 1 to 5, wherein the one or more search services further comprises a second search service configured to directly communicate with the access-control microservice, wherein the second search service is further configured to: receive, from the user, a request to access a second document in the data store; and send the request to the microservice.

7. A system according to claim 6, wherein the microservice is further configured to: search the central permissions database to determine if the user has permission to access the second document; and output a result to the second search service, based on if the user has permission to access the second document.

8. A system according to claim 7, wherein if the user has permission to access the second document then the result comprises the second document, or if the user does not have permission to access the second document, then the result comprises a message indicating that access to the second document is denied.

9. A system according to any one of claims 1 to 8, wherein the identity information is included in a JSON Web Token.

10. A system according to any one of claims 1 to 9, wherein the one or more documents comprises a plurality of documents, and the local permissions database is configured to store all permissions corresponding to the plurality of documents.

11. A system according to any one of claims 1 to 10, wherein the central permissions database comprises a Postgres database.

12. A system according to any one of claims 1 to 11 , wherein the local permissions database comprises a Postgres database, an ElasticSearch database or a TigerGraph database.

13. A computer implemented method performed by an access-control microservice in a system for managing user permissions for accessing a data store, the method performed in response to a first document being uploaded to a data store by a user, the method comprising: assigning a first permission to the first document based on identity information of the user; storing the first permission in a central permissions database; and generating an update message comprising changes to the permissions stored in the central permissions database.

14. A computer readable storage medium comprising instructions which, when executed by a processor, cause the processor to perform a method according to claim 13.

15. An access-control microservice configured to perform a method according to claim 13.

Images