Data storage
The database management system addresses the challenge of non-unique record keys by processing composite and extensible keys, ensuring efficient storage and retrieval of evolving identities, enhancing performance and data retrieval efficiency.
Patent Information
- Application Number
- GB2024002090
- Authority / Receiving Office
- GB · GB
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-02-15
- Publication Date
- 2025-08-20
AI Technical Summary
Conventional database management systems struggle with efficiently storing and querying data where record keys are not fixed and may comprise non-unique identifiers, leading to performance issues and inefficient data retrieval, particularly in applications involving unstructured and semi-structured data.
A database management system that processes composite and extensible record keys by dividing them into sections, determining their type, and managing overlaps and merges to maintain identities transparently, allowing efficient storage and retrieval of records associated with evolving identities.
Enables efficient storage and retrieval of records with composite and extensible keys, improving performance by maintaining identities throughout their lifecycle and facilitating quick retrieval of related records.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
Technical Field The present invention relates to improvements in data storage for databases, particularly though not exclusively to methods of managing databases using extensible record keys. Background Art A database management system (DBMS) is a term used to describe specialist data management software tasked with the management of databases. A typical DBMS provides for data in a database to be created, read, deleted, updated, and protected. A DBMS provides an interface between databases and applications, and is responsible for ensuring that data is organized and accessible. In the technical field of software engineering, data integration and data warehousing needs are generally seen as independent, solved by databases and stateless integration software such as message queues. While some general-purpose stores that combine databases and queues have emerged (e.g. Apache Kafka®), such stores lack the capabilities required to search and report on such data. The Applicant has appreciated a need in some applications involving the processing of entities extracted from unstructured and semi-structured data for the ability to operate on records that may have identifiers composed of multiple parts that are not always precise. Examples of applications that process entities extracted from unstructured and semi-structured data include, but are not limited to, customer data processing, natural language processing, and fraud and collusion detection. This presents an engineering problem for computer science, in particular regarding how data can be stored in relational databases where the value of an identity key is not precisely known o priori. Specifically, the problem is known to arise in applications involving querying data originating from multiple data sources, where even originally structured data may exhibit irregular structure once combined. In some applications such as modern customer data models, record keys are not fixed and may comprise identifiers that are not unique due to privacy-related anonymisation and / or being partially unknown. For example, each customer may have multiple identifiers that differ from traditional database keys as when more identifiers are collected throughout the customer lifecycle, they extend the identity rather than create new entities; some identifiers forming part of such composite extendable key may not be unique. Traditional databases do not have capabilities to efficiently model such data and maintain such evolving keys incrementally. Not only are such conventional databases limited in their treatment of record keys, but much of the existing specialised software known in the art perse assume the existence of a single unique identifier of an entity such as person or a business account, or an association between the entity and a set of related records that is assumed to never evolve. Examples of such systems may be found in WO2019010057, US2008172495, and US2017046734. Where multiple identifiers are supported, they are uniquely associated with an individual, without taking into account the reality of real-life customer data where such identifier may not be unique, such as anonymized click IDs shared amongst a plurality of ad platform users. An example of this may be found in US2009248653. As a result, existing applications that rely on real-world customer identification data (see, for example, US2018300748) resort to storing customer interactions in a general-purpose DBMS and making inferences at a separate analysis stage (e.g. for generating reports), associating various interactions not linked to respective identities (unless a singular unchangeable customer identifier is provided externally) based on available historical data. However, the Applicant has appreciated that such an approach has substantial performance implications as the database in a general-purpose DBMS would not have a singular identity stored as an entity and would instead have to recalculate it deductively each time the data are queried based on various identifiers used throughout stored interactions. This requires additional processing resources and slows down performance of systems reliant on the DBMS. This also has functional implications even when performance is not important, as in such case a standard DBMS data manipulation language cannot be used to return a subset of records corresponding to identities requested in the query as the database would have no prior knowledge of them in its schema, and DBMS data definition language would be of little use to enforce integrity of the data. The Applicant has appreciated therefore that improvements to the performance of database management systems and processes of databases may be of significant technical benefit. In particular, conventional DBMS systems known in the art perse do not meet expectations of data processing when used for offline reporting and may present significant limitations for real-time applications such as feeding real-time conversion and segment data to ad platforms. Summary of the Invention When viewed from a first aspect, embodiments of the present invention provide a computerised database management system configured to store data in a non-transitory computer-readable memory comprising a database, wherein the database is configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key, the database management system comprising: an interface configured to receive a request to insert a new record, said request comprising a new record tuple and a corresponding new record identity key; a key section processing module configured to: i) divide the new record identity key into one or more key sections, wherein metadata indicates whether each of said one or more key sections is respectively of a unique type; ii) process the key sections, wherein the key sections are processed by: a) for each key section indicated as being of a unique type, determining whether said key section overlaps with one or more of said identity keys stored in the database; and b) for each identity key that overlaps with said key section, loading the corresponding identity indexed by the overlapping identity key; and a database update module configured to: i) responsive to an identity being loaded: a) store the new record comprising the new record tuple and generate a new reference to associate the new record with the loaded identity indexed by the overlapping identity key; and b) responsive to determining that one or more key sections from the new record identity key are not already present in the overlapping identity key, update the overlapping identity key by combining the overlapping identity key and the key sections from the new record identity key not already present in the overlapping identity key. This first aspect of the invention extends to a computer-implemented method of storing data in a non-transitory computer-readable memory comprising a database, wherein the database is configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key, the method comprising: receiving a request to insert a new record, said request comprising a new record tuple and a corresponding new record identity key; dividing the new record identity key into one or more key sections, wherein metadata indicates whether each of said one or more key sections is respectively of a unique type; processing the key sections, wherein the key sections are processed by: i) for each key section indicated as being of a unique type, determining whether said key section overlaps with one or more of said identity keys stored in the database; and ii) for each identity key that overlaps with said key section, loading the corresponding identity indexed by the overlapping identity key; and responsive to an identity being loaded: i) storing the new record comprising the new record tuple and generating a new reference to associate the new record with the loaded identity indexed by the overlapping identity key; and ii) responsive to determining that one or more key sections from the new record identity key are not already present in the overlapping identity key, updating the overlapping identity key by combining the overlapping identity key and the key sections from the new record identity key not already present in the overlapping identity key. The first aspect of the invention further extends to a non-transitory computer-readable medium comprising instructions which, when executed by a processor, cause the processor to carry out a method of storing data in a non-transitory computer-readable memory comprising a database, wherein the database is configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key, the method comprising: receiving a request to insert a new record, said request comprising a new record tuple and a corresponding new record identity key; dividing the new record identity key into one or more key sections, wherein metadata indicates whether each of said one or more key sections is respectively of a unique type; processing the key sections, wherein the key sections are processed by: i) for each key section indicated as being of a unique type, determining whether said key section overlaps with one or more of said identity keys stored in the database; and ii) for each identity key that overlaps with said key section, loading the corresponding identity indexed by the overlapping identity key; and responsive to an identity being loaded: i) storing the new record comprising the new record tuple and generating a new reference to associate the new record with the loaded identity indexed by the overlapping identity key; and ii) responsive to determining that one or more key sections from the new record identity key are not already present in the overlapping identity key, updating the overlapping identity key by combining the overlapping identity key and the key sections from the new record identity key not already present in the overlapping identity key. The first aspect of the invention yet further extends to a computer software product comprising instructions which, when executed by a processor, cause the processor to carry out a method of storing data in a non-transitory computer-readable memory comprising a database, wherein the database is configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key, the method comprising: receiving a request to insert a new record, said request comprising a new record tuple and a corresponding new record identity key; dividing the new record identity key into one or more key sections, wherein metadata indicates whether each of said one or more key sections is respectively of a unique type; processing the key sections, wherein the key sections are processed by: i) for each key section indicated as being of a unique type, determining whether said key section overlaps with one or more of said identity keys stored in the database; and ii) for each identity key that overlaps with said key section, loading the corresponding identity indexed by the overlapping identity key; and responsive to an identity being loaded: i) storing the new record comprising the new record tuple and generating a new reference to associate the new record with the loaded identity indexed by the overlapping identity key; and ii) responsive to determining that one or more key sections from the new record identity key are not already present in the overlapping identity key, updating the overlapping identity key by combining the overlapping identity key and the key sections from the new record identity key not already present in the overlapping identity key. Thus it will be appreciated by those skilled in the art that embodiments of the present invention may provide a computer database management system for storing and retrieving records associated with an identity that is specified by composite keys that are redundant and extensible. It will be appreciated that 'redundant' means that a portion of the identity key can be used to uniquely look up the records associated with the identity. It will also be appreciated that 'extensible' means that extending the key does not break existing relationships of the records referencing a previous or partial key. The database management system maintains the identity key throughout its lifecycle as more records are inserted, maintaining evolving identities on the storage medium transparently to the user, ensuring related records can be quickly retrieved. It will be appreciated that the metadata that indicates whether a key section is of a unique type may comprise a respective flag for said key section that indicates whether it is a unique type and / or it may comprise a respective flag for said key section that indicates whether it is a non-unique type. In other words, while metadata may explicitly indicate that a key section is of a unique type, in some embodiments the metadata may indicate only non-unique types explicitly, such that an assumption can be made that a key section is of a unique type unless it is explicitly marked otherwise. Thus the phrase 'indicated as being of a unique type' (and similar) as used herein in respect of key sections should be understood to include the absence of an indication that the relevant key section is non-unique. In some embodiments, metadata for the new record identity key comprises one or more qualifiers, wherein the key sections are compared only to identity keys stored in the database that meet the one or more qualifiers, and / or to sections of the identity keys stored in the database that meet the one or more qualifiers. The or further metadata may indicate whether the stored identity keys (or sections thereof) match the qualifier(s). In other words, during the processing carried out (e.g. by the key section processing module), step ii) a) outlined above, the comparison can be limited only to stored key sections that meet certain constraints. This may advantageously reduce the amount of processing required, improving processing efficiency and speed. As outlined above, assuming that an identity is loaded as a result of an overlap between a section of the new record identity key and one of the stored identity keys, the new record is assigned to that identity (and the key may be updated to incorporate the sections of the corresponding new record identity key appearing for the first time). However, it is possible that the new record does not correspond to any existing identity. As such, in some embodiments, the database update module is further configured to: ii) responsive to no identity being loaded: a) generate a new identity indexed by the new record identity key; and b) store the new record comprising the new record tuple and generate a new reference to associate the new record with the new identity indexed by the new record identity key. Conversely, it is possible that multiple identities are loaded as a result of multiple overlaps. For example, if a new record identity key 'BD' is inserted with key sections 'B' and 'D', and there are two existing records with identity keys 'AD' and 'BC', then both of those existing identities overlap with the new one and can be merged into one. Thus, in some embodiments, the database update module is further configured to: iii) responsive to more than one identity being loaded: a) merge said loaded entities to generate a merged entity indexed by a merged identity key, said merged identity key being generated by combining the identity key corresponding to each loaded entity and any key sections from the new record identity key not already present in any identity key corresponding to the loaded entities; b) update the respective reference for the records previously associated with the loaded entities to associate said records with the merged entity; and c) store the new record comprising the record tuple and generate a new reference to associate the new record with the merged entity. Thus embodiments of the present invention may provide the means to combine records with extensible keys when the keys of two or more records sufficiently overlap after extending keys of one or more records. The combination of keys may be achieved, at least in some embodiments, by concatenating the overlapping identity key and the key sections from the new record identity key not already present in the overlapping identity key. In some embodiments, the key section processing module is further configured to apply a predetermined transformation to a key section responsive to determining that said key section matches a format criterion. In other words, if it is detected that a key section is a particular data format type (e.g. an email, a telephone number, or similar), then a transformation may be applied. In some such embodiments, the transformation comprises a surjective function. Those skilled in the art will appreciate that a surjective function is a function in which every element in the function's co-domain (the resultant domain after the function) maps to at least one element in the function's domain (the domain before the function). The format criterion may, in some embodiments, comprise a telephone number. In some such embodiments, the transformation comprises converting a key section corresponding to a telephone number into a binary representation of a last predetermined number of digits of said telephone number. The numeric value of this predetermined number of digits may be selected so as to ensure a sufficient degree of uniqueness. For example, in some embodiments, the predetermined number is 10. In the United Kingdom, for example, a telephone number (without the '+44' country prefix) is 11 digits, but where the first is a leading 'O'. Under different schema (e.g. employed in different countries), the number of digits in a phone number may vary and so a different predetermined number of digits may be chosen. The Applicant has appreciated that it may not be necessary to include the full telephone number to ensure a sufficient degree of uniqueness to identify a specific entity. In some potentially overlapping embodiments, the format criterion comprises an email address. In some such embodiments, the transformation comprises converting a key section corresponding to an email address into a lower-case representation of said email address. While the key sections could be processed and the identities created, updated, and / or merged (as appropriate) one at a time, in some embodiments the database update module performs its process after all key sections have been processed. By waiting until all of the key sections have been processed, the updating of the database can be completed efficiently, particularly where multiple identities are found to overlap with the new record identity key and thus are to be merged as per certain embodiments as outlined above. The database management system also provides means for notifying subscribers (in a sense commonly understood in computer science as the 'Publish-Subscribe' architecture) of new records and the latest states of the identities referenced to them within the database. Thus, in some embodiments, the database management system further comprises a subscription module configured to transmit a notification comprising data indicative of the new record to one or more subscribers. In some such embodiments, the notification comprises data indicative of the identity associated with the new record. Depending on the configuration set in respect of the subscriber(s), notifications may be provided either in real-time or in batches (or combinations of these, e.g. with some users wanting real-time notifications and others wanting batch notifications). Thus in some such embodiments, the subscription module is configured to transmit notifications in real-time. In some potentially overlapping embodiments, the subscription module is configured to transmit notifications in a batch process. Thus there may be provided the means to transmit newly recorded data to destinations, e.g. periodically or on demand in batches, as arranged for each destination-tenant pair independently. Aspects of the present invention further extend to a networked computer system comprising the database management system and the one or more subscribers. In embodiments of the present invention, the database management system and the one or more subscribers may be connected via a network link. The network link may be any suitable communications network, for example the Internet. As outlined above, metadata indicates whether each key section is of a unique or non-unique type, where non-unique type key sections are ignored for the purposes of indexing the identities (since non-unique types cannot be used to uniquely identify a given identity). The metadata may be expressly provided together with the request for the new record (alongside the new record tuple and new record identity key), but in some embodiments, the metadata is derived from the one or more key sections. For example, if a key section has the format of a telephone number or email address, these may be identified as being key sections of a unique type even if no specific flag was provided to explicitly indicate that key section as being of a unique type. The insertion of records is not the only function that makes use of the improvements provided by the present invention, as record queries may also be augmented by the approach described herein. In particular, in some embodiments, the interface is further configured to receive a query to retrieve a record, said query comprising a query identity key, and wherein the key section processing module configured to: i) divide the query identity key into one or more query key sections; ii) process the query key sections, wherein the key sections are processed by: a) for each query key section, determine whether said query key section overlaps with one or more of said identity keys stored in the database; and b) for each identity key that at least partially matches said query key section, loading and outputting the corresponding identity indexed by the matching identity key. The querying process is novel and inventive in its own right and thus, when viewed from a second aspect, embodiments of the present invention provide a computerised database management system configured to store data in a non-transitory computer-readable memory comprising a database, wherein: the database is configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key comprising one or more key sections, wherein at least one identity key comprises first and second key sections, said first key section derived from a first inserted record, and said second key section derived from a second inserted record, said first and second inserted records being associated with the identity indexed by said identity key; and the database management system comprises: an interface configured to receive query to retrieve a record, said query comprising a query identity key; a key section processing module configured to: i) divide the query identity key into one or more query key sections; ii) process the query key sections, wherein the key sections are processed by: a) for each query key section, determine whether said query key section overlaps with one or more of said identity keys stored in the database; and b) for each identity key that at least partially matches said query key section, loading and outputting the corresponding identity indexed by the matching identity key; and a database retrieval module configured to, responsive to an identity being loaded, load one or more stored records associated with each loaded identity and return said loaded one or more stored records in response to the query. This second aspect of the invention extends to a computer-implemented method of querying data stored in a non-transitory computer-readable memory comprising a database, wherein: the database is configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key comprising one or more key sections, wherein at least one identity key comprises first and second key sections, said first key section derived from a first inserted record, and said second key section derived from a second inserted record, said first and second inserted records being associated with the identity indexed by said identity key; and the method comprises: receiving a query to retrieve a record, said query comprising a query identity key; dividing the query identity key into one or more query key sections; processing the query key sections, wherein the key sections are processed by: i) for each query key section, determining whether said query key section overlaps with one or more of said identity keys stored in the database; and ii) for each identity key that at least partially matches said query key section, loading and outputting the corresponding identity indexed by the matching identity key; and responsive to an identity being loaded, loading one or more stored records associated with each loaded identity and returning said loaded one or more stored records in response to the query. The second aspect of the invention further extends to a non-transitory computer-readable medium comprising instructions which, when executed by a processor, cause the processor to carry out a method of querying data stored in a non-transitory computer-readable memory comprising a database, wherein: the database is configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key comprising one or more key sections, wherein at least one identity key comprises first and second key sections, said first key section derived from a first inserted record, and said second key section derived from a second inserted record, said first and second inserted records being associated with the identity indexed by said identity key; and the method comprises: receiving a query to retrieve a record, said query comprising a query identity key; dividing the query identity key into one or more query key sections; processing the query key sections, wherein the key sections are processed by: i) for each query key section, determining whether said query key section overlaps with one or more of said identity keys stored in the database; and ii) for each identity key that at least partially matches said query key section, loading and outputting the corresponding identity indexed by the matching identity key; and responsive to an identity being loaded, loading one or more stored records associated with each loaded identity and returning said loaded one or more stored records in response to the query. The second aspect of the invention yet further extends to a computer software product comprising instructions which, when executed by a processor, cause the processor to carry out a method of querying data stored in a non-transitory computer-readable memory comprising a database, wherein: the database is configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key comprising one or more key sections, wherein at least one identity key comprises first and second key sections, said first key section derived from a first inserted record, and said second key section derived from a second inserted record, said first and second inserted records being associated with the identity indexed by said identity key; and the method comprises: receiving a query to retrieve a record, said query comprising a query identity key; dividing the query identity key into one or more query key sections; processing the query key sections, wherein the key sections are processed by: i) for each query key section, determining whether said query key section overlaps with one or more of said identity keys stored in the database; and ii) for each identity key that at least partially matches said query key section, loading and outputting the corresponding identity indexed by the matching identity key; and responsive to an identity being loaded, loading one or more stored records associated with each loaded identity and returning said loaded one or more stored records in response to the query. Thus this second aspect of the invention provides a DBMS in which users can query records by a section of a key and retrieve the following records as if they had been inserted with the same key originally: 1. Records that had the specified key section present in their key at the time of insertion; 2. Records that had one or more of their own key sections, at the time of insertion, matching one or more key sections of another record, which, in its own turn, had one or more of its own key sections, at the time of insertion, matching one or more sections of the specified key; 3. Records that had one or more of their own key sections, at the time of insertion, matching one or more key sections of an intermediary record, which in turn had one or more of its own key sections, at the time of insertion, matching one or more key sections of another record, which, in its own turn, had one or more of its own key sections, at the time of insertion, matching one or more sections of the specified key; 4. Records that had one or more of their own key sections, at the time of insertion, matching one or more key sections of a first intermediary record in a chain of N intermediary records, each linked to the next by one or more matching key sections at insertion time, where the Nth intermediary record in the chain had one or more of its own key sections, at the time of insertion, matching one or more sections of the specified key. Thus, in accordance with this second aspect, records are associated via matching key sections, either directly or through one or more intermediary records linked in a chain, enabling retrieval through a specified key section based on insertion-time key matching associations. In some embodiments, the query comprises one or more record tuple constraints, wherein the database retrieval module is further configured to determine whether each loaded stored record matches the record tuple constraints and to return only the stored records that match said record tuple constraints. In a set of such embodiments, loaded stored records that do not match the record tuple constraints are discarded. When viewed from a third aspect, embodiments of the present invention provide a computer-implemented database configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key, wherein at least one identity key comprises first and second key sections, said first key section derived from a first inserted record, and said second key section derived from a second inserted record, said first and second inserted records being associated with the identity indexed by said identity key. This third aspect of the invention extends to a computerised database management system configured to store data in a non-transitory computer-readable memory comprising a database configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key, wherein at least one identity key comprises first and second key sections, said first key section derived from a first inserted record, and said second key section derived from a second inserted record, said first and second inserted records being associated with the identity indexed by said identity key. Optional features set out in respect of any aspect of the invention are also applicable to the other aspects of the invention as appropriate. In the context of this specification "comprising" is to be interpreted as "including". The physical storage of the data is selected depending on implementation needs and may be a computer disk, a higher-level logical storage or another suitable physical medium or software interface. The records and the identities are stored in a database. It should be understood that the records and identities may be stored in logically distinct databases, tables, arrays, or other such structures, or these may be collocated, as appropriate. Aspects of the invention comprising certain elements are also intended to extend to alternative embodiments "consisting" or "consisting essentially" of the relevant elements. Where technically appropriate, embodiments of the invention may be combined. Embodiments are described herein as comprising certain features / elements. The disclosure also extends to separate embodiments consisting or consisting essentially of said features / elements. Technical references such as patents and applications are incorporated herein by reference. Any embodiments specifically and explicitly recited herein may form the basis of a disclaimer either alone or in combination with one or more further embodiments. Brief Description of the Drawings Certain embodiments of the present invention will now be described with reference to the accompanying drawings, in which: Figs. 1A to D are block diagrams illustrating how data is organised in the database in accordance with an embodiment of the present invention; Fig. 2 is a flow chart illustrating a method of inserting a new record in accordance with an embodiment of the present invention; Fig. 3 is a flow chart illustrating a method of querying the records in accordance with an embodiment of the present invention; Fig. 4 is a block diagram illustrating the structure of a key section; and Figs. 5A to D are block diagrams illustrating the states of the identity and record data stores during operations carried out in accordance with embodiments of the invention. Detailed Description The described database management system provides a mechanism for managing data organised around a logically independent storage for two types of related objects - identities and records. The database management system provides one or more interfaces which provide mechanisms to insert, query and subscribe to the records. While each record belongs to an identity internally, the Identities themselves are created and modified by the database management system implicitly based on Record Identity Keys contained in the insertion and querying requests. Each identity consists of a database management system managed key that serves as a unique identifier of the identity and can be extended without losing ability to query an identity by the old key (the version of key prior to extension) or its constituent part (such as old key). Each section of the identity key may have metadata associated with the key section that is not a part of the key itself. Metadata may specify whether a given key section identifies an identity uniquely or may be shared by multiple identities. Figs. 1A to D are block diagrams illustrating how data is organised in the database in accordance with an embodiment of the present invention. This shows the logical organisation of the data and morphing of the identities and their keys as new records are inserted. Examples assume that none of the identity key sections have metadata that would identify them as non-unique. The database contains an identities table 100 and records table 102. The records are associated with the identities by reference 104, where the arrows in Figs. 1A to D indicate which identities and records are associated with one another. As can be seen in Fig. 1A, this exemplary database has an initial state in which the identities table 100 contains two identities, indexed by keys 'A' and 'BC' respectively. The identity with key 'A' is associated with two records in the records table 102, which are: 'Record 1' which has record tuple (al, bl, cl, dl,...); and 'Record 2' which has record tuple (a2, b2, c2, d2,...). The identity with key 'BC' is associated with one record in the records table 102, which is 'Record 3' which has record tuple (a3, b3, c3, d3,...). Insertion of Records Records are inserted into the database by accessing the database management system programming interface (or user interface) and transmitting the record tuple (a, b, c, d,...) and record identity key for the new record to be inserted. This process can be best understood by reference to both Figs. 1A to ID and Fig. 2. Fig. 2 is a flow chart, illustrating the process 200 for record insertion. Fig. 2 demonstrates how supplied identity key is processed before a record is saved, and the merge process is shown in Figs. lAto ID. After receiving a request 202 to insert a new record, the database management system attempts to locate an existing stored identity that would match the record identity key. First, the identity key contained in the record supplied for insertion, is divided into one or more key sections 204. A check 206 is made to determine if there are any unprocessed key sections, which there will be at least one on the first iteration of this process. The key sections may have metadata attached to them. Metadata may additionally or alternatively be derived from the key section value itself. If its value matches a known natural key type, i.e. representing a string containing a syntactically valid e-mail address or a valid phone number, the corresponding type may be added to metadata even if not supplied in the insertion request. Example of a key consisting of three sections provided with a record for insertion: Key section number: 1 II III Key section: 01234 567890 Name@domain.com 898f917e7333be8d Metadata ('type') Click identifier Metadata ('non-unique') This same key can have metadata derived for it, in which missing type metadata and 'non-unique' flags added by the system upon identifying syntactically correct phone and email, as well as a 'click identifier' type that was configured to be processed a non-unique key section type: Key section number: 1 II Ill Key section: 01234 567890 Name@domain.com 898f917e7333be8d Metadata ('type') Phone Email Click identifier Metadata ('non-unique') — — Yes When instructed by metadata, the database management system applies a transformation 208 to the key section prior to further processing, recording the non-transformed section in metadata for future reference, and using the transformed key section for further processing. One exemplary type of transformation is a surjective mapping function used to remove or reset bits in the 10 key section that do not serve for unique identification of the natural key represented as a key section. For example, if metadata specifies that a key section is a phone number, it may be transformed to binary representation of the last 10 digits, whereas an email can be either transformed to a lowercase string, or bits indicating upper- or lower-character case can be removed altogether. Taking the example key given previously, key sections can be transformed to normalised formats based on 15 the type specified in metadata. In the example given, the email address is converted to a synonymous lower-case representation, and extra non-identifying characters (leading zero and a space) are removed from the phone number: Key section number: 1 II Ill Key section: 1234567890 name@domain.com 898f917e7333be8d Metadata ('type') Phone Email Click identifier Metadata ('non-unique') — — Yes A check is made to determine whether the current key section is of a non-unique type 210. If the key section is marked in the metadata as being of a non-unique type, the process returns to step 206 where the process repeats for the next key section (unless there are no more key sections). If the key section is not marked as being of a non-unique type, the database management system performs a search of that key section amongst the stored identities in the identities table 102. The search is performed by comparing each (potentially transformed) key section from the new record identity key with each key section of each stored identity. An identity is considered to match the queried identity key if one of the new record identity key sections equals one of the stored identity key sections, and, if present, if any qualifiers specified in the key sections' metadata (such as key section type) match. If a match is found, the matching identity is loaded 212. The process then returns to step 206 where the process repeats for the next key section (unless there are no more key sections, as below). Once all key sections have been processed, the database management system checks how many identities were loaded 214. If a single identity was loaded, the new record tuple is stored and is associated with the identity that was loaded 216. An example of this is shown in Fig. IB, in which the new record being added with record tuple Record 4 = (a4, b4, c4, d4,...) and new record identity key 'AD'. The section 'A' of 'AD' overlaps with the existing record with identity key 'A' as discussed previously with reference to Fig. 1A. As such, the new Record 4 is stored in the records table 102 and is associated with the identity previously indexed by key 'A'. The identity key for that identity is updated at step 215 to append the new key section 'D' that was not previously present to the already-present key section 'A', resulting in a new key 'AD'. If more than one stored identity is found, these are merged into a single identity 216. The key of the merged identity is composed of the keys of the identities being merged (e.g. by concatenation, but other methods may be used), as well as key sections from the record being inserted that are not present in the merge identity key already. An example of this is shown in Fig. IC, in which the new record being added with record tuple Record 5 = (a5, b5, c5, d5,...) and new record identity key 'BD'. The section 'B' of 'BD' overlaps with the existing record with identity key 'BC' as discussed previously with reference to Fig. 1A. Additionally, the section 'D' of 'BD' overlaps with the existing record with identity key 'AD' as discussed previously with reference to Fig. IB. As such, the identities with keys 'AD' and 'BC' are merged into a single entity with key 'ADBC'. The new Record 5 is stored in the records table 102 and is associated with the new merged identity. Pre-merge identities are then dereferenced from the records, which are set to reference merged identity, or are transformed to pointers to the merged identity instead. If no Identities were found during search, a new identity is created 220 with the key set to the new record identity key supplied with in the record insertion request. An example of this is shown in Fig. ID, in which the new record being added with record tuple Record 6 = (a6, b6, c6, d6,...) and new record identity key 'E'. No section of 'E' overlaps with any existing stored record. As such, a new identity is created with the identity key 'E' in the identities table 100. The new Record 6 is stored in the records table 102 and is associated with the new identity indexed by key 'E'. Thus record tuples are written to the storage medium together with a reference to the appropriate identity as set out above. The reference points to either a single identity found by the search, a merged identity (if several identities were found in storage), or a newly-saved-to-storage new identity. The inserted record together with the associated identity (in its final form as stored on the storage medium after the algorithm steps defined above complete) may then be sent to subscribers configured for the database 222. This is done by calling the subscriber's application programming interface (API) or queuing a notification for future processing, either in memory or on a suitable storage medium. Querying of Records The database management system provides means for retrieving a set of records by supplying record identity key and / or record tuple value(s) to search and filter the records by. Fig. 3 is a flow chart illustrating a process 300 for querying the records in accordance with an embodiment of the present invention. If an identity key is supplied, it is first processed analogous to how it is done in the record insertion process outlined above. However, when running a query (rather than an insertion), the uniqueness of the key sections does not need to be checked as it is not relevant. After receiving a query 302 to find records matching record tuple constraints and / or a particular identity key, the database management system attempts to locate an existing stored records that match the query. First at step 304, the identity key contained in the query is divided into one or more key sections that may have metadata attached to them. A check 306 is made to determine if there are any unprocessed key sections, which there will be at least one on the first iteration of this process. As outlined previously, metadata may also be derived from the key section value itself. If its value matches a known natural key type, i.e. representing a string containing a syntactically valid e-mail address or a valid phone number, the corresponding type may be added to metadata even if not supplied in the insertion request. When instructed by metadata, the database management system applies a transformation to the key section at step 308 prior to further processing. For ease of explanation, the processes of deriving metadata and performing transformations to the key sections is not repeated here, however these may be the same or similar processes to those described previously in respect of the record insertion process 200. Then, the database management system performs a search of each key section amongst stored identities. The search is performed by comparing each (potentially transformed) key section with each key section of each stored identity. An identity is considered to match the queried identity key if one of the identity key sections equals one of the stored identity key sections, and, if present, if any qualifiers specified in the relevant key section's metadata (such as key section type) match. Any matching identities found are loaded at step 310. Once all key sections have been processed, at step 312 the database management system then loads from storage and returns the record(s) that have tuple parameters matching specified search conditions, as well as, if an identity key was provided, identity reference matching one of the identities located by the aforementioned identity search steps. If there are any records that don't match the supplied record tuple constraints, these are discarded at step 314. When requested, referenced identities are returned alongside the records at step 316. It can be seen therefore that stored records are treated as if they had originally been inserted with the key that they later obtain as a result of key merges and extensions due to identities extending as outlined previously. When a queried identity is searched, the records that are returned are: 1. Records that had the specified key section present in their key at the time of insertion; 2. Records that had one or more of their own key sections, at the time of insertion, matching one or more key sections of another record, which, in its own turn, had one or more of its own key sections, at the time of insertion, matching one or more sections of the specified key; 3. Records that had one or more of their own key sections, at the time of insertion, matching one or more key sections of an intermediary record, which in turn had one or more of its own key sections, at the time of insertion, matching one or more key sections of another record, which, in its own turn, had one or more of its own key sections, at the time of insertion, matching one or more sections of the specified key; 4. Records that had one or more of their own key sections, at the time of insertion, matching one or more key sections of a first intermediary record in a chain of N intermediary records, each linked to the next by one or more matching key sections at insertion time, where the Nth intermediary record in the chain had one or more of its own key sections, at the time of insertion, matching one or more sections of the specified key. Subscriptions The described database management system provides publish-subscribe capabilities. Multiple subscribers can be configured to receive notifications of new records stored by the database management system. The database management system configuration may specify conditions filtering the records sent to a Subscriber. After a new record is stored, the database management system evaluates the list of configured subscribers and prepares a list of notifications of the record that need to be delivered. Notifications consist of (or reference) the record, subscriber, and notification identity. The notification identity is formed from the identity as stored in the database; however, the subscriber configuration may specify instructions to modify the identity key based on the metadata for the key section. Thus, configuration may contain rules to selectively include sections of the identity key where metadata indicates the key section to be currently valid from the subscriber standpoint (such as being created in the last 90 days). Each such notification may be stored in the volatile or non-volatile memory. Configuration may specify real-time or batch-oriented subscribers. Notifications sent to real-time subscribers, using the programming interface or the protocol specified in the configuration, are sent without delay. Notifications to batch-oriented subscribers are stored in memory without further action until subscriber requests a batch of unprocessed notifications. Optionally, batches can be transmitted to subscribers when the size of an individual batch or all batches prepared for the destination reaches a predetermined threshold or regularly based on a schedule. When a subscriber requests a batch of unprocessed notifications, the database management system marks the notifications returned for processing with a unique reference of the subscriber's batch request to indicate processing status. In either case, subscriber may return notification processing results to the database management system synchronously or return an external reference (which may equal to the internal identifiers or sequential number of the notification in the database management system). Database management system receives asynchronous processing results from the subscribers after processing is completed, which are then stored in the notification record in the database. Data Storage The DBMS storage can be arranged in any suitable manner known in the art perse. However, in at least some exemplary embodiments, the DBMS storage can be organised to store identity keys and the record tuples in two independent storage spaces (e.g. in different files or areas of computer memory addressable as two distinct internally contiguous virtual memory address spaces). Thus there may be an identity key store 500 and a record store 502 (as shown in Figs. 5Ato 5D, described in more detail below), which for example may be used respectively for storing the identity table 100 and records table 102 described previously. In this specific example the stores 500, 502 each have address values ranging from 00 to FF, though these may each have different lengths and the stores need not be the same size. Each record is assumed to be stored as a fixed size tuple, whereas the length of the identity keys is flexible. As can be seen in Figs. 5A to 5D, identity keys each comprise an identity key header 504 that points to the starting address of the identity key, followed by one or multiple key sections 506, terminating with the end of identity key reference 508. The end of identity reference 508 is a pointer to section of memory that, in case if it follows the final section of the key, points to the key header address 504 of that same identity key. Otherwise, the end of identity reference 508 points to the non-contiguous address which contains further key sections. Each key section or an end of identity are preceded by a flag 510 indicating whether the following data is a key section (with the flag value '0') or an end of identity reference (with the flag value '1')- As shown in Fig. 4, each key section 400 has associated metadata 402, which may for example be represented as a fixed-size tuple, followed by the key section length 404 and key section value 406. Referring back to Figs. 5A to 5D, it can be seen each record tuple 512 is stored together with a reference 514 to its identity, pointing to the address of corresponding identity key header 504. This exemplary DBMS may operate as follows. When a record is inserted, if the DBMS search for a matching identity, as described previously with reference to Fig. 2, was not successful (e.g. a database is empty), the following values are sequentially written to the identity key storage: the identity key header 504 pointing to its own address (0 in an empty database); the key sections 506 (e.g. key section 1 and 2 in Fig. 5A); and the end of identity reference 508 (1-0000 in Fig. 5A). When a next record containing identity key that has sections that overlaps with an existing key, an existing key is extended as shown in Fig. 5B. This is performed by appending identity key to the end of key storage with the key header 504 pointing to the start of the key being extended (0000 at address 0100 in Fig. 5B), followed by the key section and the end of identity reference (1-0000 in Fig. 5B). The previous end of identity reference 508 is then changed to point to the address 504 of the new key section 506 appended (1- 0100 in Fig. 5B) and the new record tuple 512 is written to the record store, with its reference 514 pointing to the start address 504 of the identity key in the key store 500 (0000 in Fig. 5B). When a new record is inserted that does not have key sections that overlap with any of the previously existing key sections as shown in Fig. 5C, an address of the new identity (04A0, pointing to itself, in Fig. 5C) is appended to the end of the key store 500, followed by the key sections of the new identity key (0-Key section 1 in Fig. 5C) and end of identity 508 reference, pointing to the start address 504 of the key section 506 in the key store (1-04A0 in Fig. 5C). The record (Record 17 in Fig. 5C) is then written to the data store 502, with a reference 514 pointing to the new identity key start address (04A0 in Fig. 5C). When a record is inserted that has an identity key overlapping with more than one existing identity key, these identity keys are merged as shown in Fig. 5D. The merge is performed by changing the end of each identity key, except the last one matched, to point to the start of another matched identity key (1-0000 is changed to 1-04A0 in Fig. 5D), and the end of identity key of the last identity key is changed to point to the start of the merged identity key (1-04A0 is changed to 1-0000 in Fig. 5D). The new record (Record 161) is then written to the data store 502 alongside with the reference 514 to the merged identity key (0000); references to the keys that were merged are then updated (overwritten) to point to the starting address 504 of the merged identity key, rather than the start of its merged parts (Record 17 in Fig. 5D). The search therefore consists of matching parts of the identity key specified in a query condition with the key sections stored in the identity key store. When a key section in the store matches an identity key section, records containing a key reference pointing to the start of identity key record are retrieved from the data store and written to the output. A DBMS implementing the described storage mechanism benefits from reduced computational complexity of retrieving chains of interrelated records that didn't have an explicit shared key value at the time of being inserted to linearly proportional to the size of the database (O(n)) and having a constant memory size requirement independent on the size of the database or number of intermediate identity keys linking the related records at the time they were inserted. Conclusion Thus it will be appreciated by those skilled in the art that embodiments of the present invention may provide an improved database management system and method with the means for using extensible record keys consisting of one or multiple parts, which can be partially formed from non-unique identifiers. This may be achieved with combined means of record storage, transmitting, and routing. Embodiments of the present invention allow the use of imprecise identifiers to create and look up records. A database management system in accordance with embodiments of the present invention may possess the means to combine records with extensible keys when the keys of two or more records sufficiently overlap after extending keys of one or more records. Newly recorded data may be transmitted to destinations periodically or on demand in batches arranged for 5 each destination-tenant pair independently. A subset of customer identifiers may be transmitted to a destination chosen to maximise the attribution ratio based on rules such as identifier key sections validity periods. While specific embodiments of the present invention have been described in detail, it will be appreciated by those skilled in the art that the embodiments described in detail are not limiting on the scope of the 10 claimed invention.
Claims
1. A computerised database management system configured to store data in a non-transitory computer-readable memory comprising a database, wherein the database is configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key, the database management system comprising:an interface configured to receive a request to insert a new record, said request comprising a new record tuple and a corresponding new record identity key;a key section processing module configured to:i) divide the new record identity key into one or more key sections, wherein metadata indicates whether each of said one or more key sections is respectively of a unique type;ii) process the key sections, wherein the key sections are processed by:a) for each key section indicated as being of a unique type, determining whether said key section overlaps with one or more of said identity keys stored in the database; andb) for each identity key that overlaps with said key section, loading the corresponding identity indexed by the overlapping identity key; anda database update module configured to:i) responsive to an identity being loaded:a) store the new record comprising the new record tuple and generate a new reference to associate the new record with the loaded identity indexed by the overlapping identity key; andb) responsive to determining that one or more key sections from the new record identity key are not already present in the overlapping identity key, update the overlapping identity key by combining the overlapping identity key and the key sections from the new record identity key not already present in the overlapping identity key.
2. The database management system as claimed in claim 1, wherein the database update module is further configured to:ii) responsive to no identity being loaded:a) generate a new identity indexed by the new record identity key; andb) store the new record comprising the new record tuple and generate a new reference to associate the new record with the new identity indexed by the new record identity key.
3. The database management system as claimed in claim 1 or 2, wherein the database update module is further configured to:iii) responsive to more than one identity being loaded:a) merge said loaded entities to generate a merged entity indexed by a merged identity key, said merged identity key being generated by combining the identity key corresponding to each loaded entity and any key sections from the new record identity key not already present in any identity key corresponding to the loaded entities;b) update the respective reference for the records previously associated with the loaded entities to associate said records with the merged entity; andc) store the new record comprising the record tuple and generate a new reference to associate the new record with the merged entity.
4. The database management system as claimed in any preceding claim, wherein metadata for the new record identity key comprises one or more qualifiers, wherein the key section processing module is configured to compare each key section indicated as being of a unique type only to identity keys stored in the database that meet the one or more qualifiers, and / or to sections of the identity keys stored in the database that meet the one or more qualifiers.
5. The database management system as claimed in any preceding claim, wherein the key section processing module is further configured to apply a predetermined transformation to a key section responsive to determining that said key section matches a format criterion, optionally wherein the transformation comprises a surjective function.
6. The database management system as claimed in claim 5, wherein the format criterion comprises a telephone number, optionally wherein the transformation comprises converting a key section corresponding to a telephone number into a binary representation of a last predetermined number of digits of said telephone number, further optionally wherein the predetermined number is 10.
7. The database management system as claimed in claim 5 or 6, wherein the format criterion comprises an email address, optionally wherein the transformation comprises converting a key section corresponding to an email address into a lower-case representation of said email address.
8. The database management system as claimed in any preceding claim, wherein the database update module performs its process after all key sections have been processed.
9. The database management system as claimed in any preceding claim, further comprising a subscription module configured to transmit a notification comprising data indicative of the new record to one or more subscribers, optionally wherein the notification comprises data indicative of the identity associated with the new record.
10. The database management system as claimed in claim 9, wherein the subscription module is configured to transmit notifications in real-time and / or in a batch process.
11. The database management system as claimed in any preceding claim, wherein the interface is further configured to receive a query to retrieve a record, said query comprising a query identity key, and wherein the key section processing module configured to:i) divide the query identity key into one or more query key sections;ii) process the query key sections, wherein the key sections are processed by:a) for each query key section, determine whether said query key section overlaps with one or more of said identity keys stored in the database; andb) for each identity key that at least partially matches said query key section, loading and outputting the corresponding identity indexed by the matching identity key.
12. The database management system as claimed in any preceding claim, wherein the metadata is derived from the one or more key sections.
13. A computer-implemented method of storing data in a non-transitory computer-readable memory comprising a database, wherein the database is configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key, the method comprising:receiving a request to insert a new record, said request comprising a new record tuple and a corresponding new record identity key;dividing the new record identity key into one or more key sections, wherein metadata indicates whether each of said one or more key sections is respectively of a unique type;processing the key sections, wherein the key sections are processed by:i) for each key section indicated as being of a unique type, determining whether said key section overlaps with one or more of said identity keys stored in the database; andii) for each identity key that overlaps with said key section, loading the corresponding identity indexed by the overlapping identity key; andresponsive to an identity being loaded:i) storing the new record comprising the new record tuple and generating a new reference to associate the new record with the loaded identity indexed by the overlapping identity key; andii) responsive to determining that one or more key sections from the new record identity key are not already present in the overlapping identity key, updating the overlapping identity key by combining the overlapping identity key and the key sections from the new record identity key not already present in the overlapping identity key.
14. The method as claimed in claim 13, further comprising:responsive to no identity being loaded:i) generating a new identity indexed by the new record identity key; andii) storing the new record comprising the new record tuple and generating a new reference to associate the new record with the new identity indexed by the new record identity key.
15. The method as claimed in claim 13 or 14, further comprising:responsive to more than one identity being loaded:i) merging said loaded entities to generate a merged entity indexed by a merged identity key, said merged identity key being generated by combining the identity key corresponding to each loaded entity and any key sections from the new record identity key not already present in any identity key corresponding to the loaded entities;ii) updating the respective reference for the records previously associated with the loaded entities to associate said records with the merged entity; andiii) storing the new record comprising the record tuple and generating a new reference to associate the new record with the merged entity.
16. The method as claimed in any of claims 13 to 15, wherein metadata for the new record identity key comprises one or more qualifiers, wherein the method comprises comparing each key section indicated as being of a unique type only to identity keys stored in the database that meet the one or more qualifiers, and / or to sections of the identity keys stored in the database that meet the one or more qualifiers.
17. The method as claimed in any of claims 13 to 16, further comprising applying a predetermined transformation to a key section responsive to determining that said key section matches a format criterion, optionally wherein the transformation comprises a surjective function.
18. The method as claimed in claim 17, wherein the format criterion comprises a telephone number, optionally wherein the transformation comprises converting a key section corresponding to a telephone number into a binary representation of a last predetermined number of digits of said telephone number, further optionally wherein the predetermined number is 10.
19. The method as claimed in claim 17 or 18, wherein the format criterion comprises an email address, optionally wherein the transformation comprises converting a key section corresponding to an email address into a lower-case representation of said email address.
20. The method as claimed in any of claims 13 to 19, wherein storing the new record and updating the identity key is carried out after all key sections have been processed.
21. The method as claimed in any of claims 13 to 20, further comprising transmitting a notification comprising data indicative of the new record to one or more subscribers, optionally wherein the notification comprises data indicative of the identity associated with the new record.
22. The method as claimed in claim 21, comprising transmitting the notifications in real-time and / or in a batch process.
23. The method as claimed in any of claims 13 to 22, further comprising:receiving a query to retrieve a record, said query comprising a query identity key, and wherein the key section processing module configured to:dividing the query identity key into one or more query key sections;processing the query key sections, wherein the key sections are processed by:i) for each query key section, determining whether said query key section overlaps with any identity key stored in the database; andii) for each identity key that at least partially matches said query key section, loading and outputting the corresponding identity indexed by the matching identity key.
24. The method as claimed in any of claims 13 to 23, further comprising deriving the metadata from the one or more key sections.
25. A computer-implemented method of querying data stored in a non-transitory computer-readable memory comprising a database, wherein:the database is configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key comprising one or more key sections, wherein at least one identity key comprises first and second key sections, said first key section derived from a first inserted record, and said second key section derived from a second inserted record, said first and second inserted records being associated with the identity indexed by said identity key; andthe method comprises:receiving a query to retrieve a record, said query comprising a query identity key;dividing the query identity key into one or more query key sections;processing the query key sections, wherein the key sections are processed by:i) for each query key section, determining whether said query key section overlaps with one or more of said identity keys stored in the database; andii) for each identity key that at least partially matches said query key section, loading and outputting the corresponding identity indexed by the matching identity key; andresponsive to an identity being loaded, loading one or more stored records associated with each loaded identity and returning said loaded one or more stored records in response to thequery.
26. A non-transitory computer-readable medium or a computer software product comprising instructions which, when executed by a processor, cause the processor to carry out the method of any of claims 13 to 25.
27. A computer-implemented database configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key, wherein at least one identity key comprises first and second key sections, said first key section derived from a first inserted record, and said second key section derived from a second inserted record, said first and second inserted records being associated with the identity indexed by said identity key.
28. A computerised database management system configured to store data in a non-transitory computer-readable memory comprising a database, wherein:the database is configured to store a plurality of records and a plurality of identities, wherein each record comprises a respective record tuple associated with an identity via a reference, and wherein each identity is indexed by a respective identity key comprising one or more key sections, wherein at least oneidentity key comprises first and second key sections, said first key section derived from a first inserted record, and said second key section derived from a second inserted record, said first and second inserted records being associated with the identity indexed by said identity key; andthe database management system comprises:5 an interface configured to receive query to retrieve a record, said query comprising a queryidentity key;a key section processing module configured to:i) divide the query identity key into one or more query key sections;ii) process the query key sections, wherein the key sections are processed by:10 a) for each query key section, determine whether said query key sectionoverlaps with one or more of said identity keys stored in the database; andb) for each identity key that at least partially matches said query key section, loading and outputting the corresponding identity indexed by the matching identity key; and15 a database retrieval module configured to, responsive to an identity being loaded, load oneor more stored records associated with each loaded identity and return said loaded one or more stored records in response to the query.
29. The database management system as claimed in claim 28, further comprising the database management system as claimed in any of claims 1 to 12.