Database system

Abstract

A database system is described, comprising at least one database, at least one user interface, an authorization system, and an explanation system for the metadata fields, wherein the explanation system includes an agent that assists the at least one user in entering data records, wherein the agent has a large language model that is connected to a policy database and a metadata directory, and wherein the agent is configured to make suggestions for the metadata fields based on the policy database and the metadata directory.

Description

[0001] Database systems of the type mentioned above are known in the state of the art.

[0002] Large corporate database systems require comprehensive mechanisms to ensure the consistency of data records, especially with regard to metadata. These include clear standards and instructions, effective authorization systems, and regular user training. At the same time, these systems must remain flexible enough to adapt to changing requirements without compromising data quality.

[0003] The complexity of large database systems in large companies arises, among other things, from the multitude of actors involved, the diverse data types, and the high demands on consistency and reliability of the stored information. A central aspect of this complexity is the entry of data records, especially with regard to metadata, which is essential for the organization and findability of the data. Metadata serves as structured additional information to the actual data and helps to classify, describe, and link it.

[0004] In large organizations, the quality and consistency of metadata entry often varies greatly, as data records are entered by different users or systems. Different departments often have different definitions or standards for the use of metadata fields. This can lead to inconsistent or incomplete fields being filled out, significantly complicating data searches and analysis. Furthermore, the use of optional metadata fields or a lack of standardized entry processes can lead to redundant or erroneous entries.

[0005] Another complexity factor lies in the often dynamic and rapidly changing requirements of large companies. New business processes, regulatory requirements, or technical changes regularly require adjustments to the database structure and the definition of metadata fields. These changes must be reflected in existing workflows and user instructions to ensure long-term consistency. At the same time, there is the challenge of ensuring that historical data can continue to be interpreted correctly.

[0006] Finally, authorizations and access rights also play a crucial role. Different user groups often have different permissions to enter and edit data records. This can further jeopardize consistency if, for example, some users are unable to fill out mandatory fields or have restricted access to certain fields. To counteract this, complex authorization systems and user instructions are often necessary, which further increase complexity.

[0007] US 2019 / 0057147 A1 discloses a software platform that ingests, processes, and makes data from a variety of data sources available for search. It also describes a data portal through which users can quickly find information and enhance metadata to leverage the company's data assets. Various aspects of disclosed software platforms include technologies for retrieving metadata without touching the underlying data assets, technologies for using table or database provenance as a factor in determining document relevance, processes for receiving data transmitted from data sources, and processes for disclosing what a user must do to access specific data assets.

[0008] Inconsistent or poor-quality (meta)data can significantly impair the functionality of a database system. Inconsistent or missing metadata makes it difficult to find and correctly interpret data records, leading to incorrect analyses and decision-making. Furthermore, redundant data records often arise due to different spellings or terminology usage, increasing storage requirements and complicating data consolidation. Processes based on automated algorithms can produce incorrect results or fail due to inconsistencies, significantly reducing the efficiency and reliability of data processing.

[0009] Furthermore, such inconsistencies hinder collaboration between departments, as teams often have to refer to conflicting or inaccurate data. This leads to time-consuming manual corrections and reconciliations. In regulated industries, incomplete or inaccurate metadata can also lead to regulatory violations, which can result in financial penalties or legal consequences. In the long term, such problems harm system adoption: Users lose confidence in the reliability of the data and resort to inefficient, manual processes, thereby undermining the original benefits of the database system.

[0010] The task therefore arises of further developing database systems of the type mentioned above in such a way that higher metadata quality can be ensured.

[0011] The problem is solved by a database system according to claim 1. Further embodiments and developments are the subject of the dependent claims.

[0012] A database system is described with - at least one database in which data records and metadata associated with the data records are stored, with a plurality of metadata fields being provided for each data record, - at least one user interface for the input of data records including metadata by at least one user, - an authorization system through which users or user groups are assigned different roles as well as access rights and / or editing rights for the data records and / or metadata fields, whereby changes and queries to the at least one database are only permitted on this basis, - an explanation system to explain the metadata fields, where the explanation system comprises an agent that assists the at least one user in entering data records, the agent comprising a large language model connected to a policy database and a metadata dictionary, the agent being configured to make suggestions for the metadata fields based on the policy database and the metadata dictionary.

[0013] The large language model serves as the basis for a networked automatic suggestion of metadata entries based on semantic or lexematic information, which the user can either accept or reject.

[0014] Such a large language model is often referred to as a Large Language Model (LLM). An LLM is an artificial neural network trained to interpret and generate human language, and solve tasks based on it. It is based on the architecture of a so-called transformer model, which works particularly efficiently through parallel processing of large amounts of data. The basis of the functionality of an LLM is the analysis and processing of text in the form of mathematical representations, so-called vectors.

[0015] The model is first pre-trained with very large amounts of text data from various sources such as books, websites, or articles. It learns to recognize patterns and relationships in language, such as which words frequently occur together or how sentence structures are constructed. This learning process is based on the so-called "self-attention" mechanism, which allows the model to identify and prioritize relevant parts of a sentence or text. This allows it to better understand the context of words and deliver more precise results.

[0016] When using an LLM, an input, also called a prompt, is typically fed into the model. The model analyzes this input and generates an output based on its training. It uses probabilities to predict which word or string best matches the input. This enables tasks such as answering questions, writing texts, or translating into other languages.

[0017] A key feature of an LLM is its ability to generalize. Based on the knowledge it has learned, it can interpret inputs it hasn't seen during training.

[0018] The database system is designed to be populated by users assigned specific roles, each responsible for entering specific types of information.

[0019] The authorization system ensures that each user enters information according to their assigned role and relevant policies.

[0020] When logging into the database system, a check can be made to see whether the user has previously attended training on how to use the database system. The training is intended to explain the database system and corresponding processes to the user and to familiarize them with the various functions, functionalities, processes, and responsibilities. If the check is negative, training can be recommended or started automatically before the user can use the system. Once training has been completed, the user is granted access and editing rights appropriate to their role. This check can be based on an email address as an identifier together with a password before the actual user interface can be displayed. In this way, it is possible to check whether a user has been registered and to register whether a user has attended training prior to this login process in order to, if necessary, reactivate them.automatically redirect to training or grant access to the interface.

[0021] In a first further embodiment, it is provided that the authorization system is designed to check a system training status of the at least one user and to enforce system training for the at least one user.

[0022] The integrated agent assists the user in adding metadata via the user interface. Within a specific field, which can be displayed next to the metadata fields, for example, the user can enter questions about the requested metadata, promoting their understanding and the entry of correct metadata.

[0023] By connecting the agent to a policy database that stores specific rules associated with different types of metadata (e.g., data sets, responsible users, associated IT systems), specific explanations can be derived as to why certain metadata needs to be added, edited, etc.

[0024] In a further development, it is planned that the agent is a chatbot.

[0025] The chatbot is trained with guidelines and supports users by providing guidance and recommendations during the data entry process. Furthermore, the chatbot is regularly updated with metadata-related guidelines, allowing it to provide up-to-date advice. It is also capable of detecting inconsistent or incorrect information and automatically suggesting corrected data, thereby improving the accuracy and reliability of the database.

[0026] In a further refinement, it is planned that a metadata analysis system is connected to the metadata directory to check the policy conformity of the metadata.

[0027] In a further refinement, it is planned that the large language model is focused on the data of the policy database and the metadata directory by means of fine-tuning, prompt engineering and / or retrieval-augmented generation.

[0028] In a further refinement, a reporting module is provided for the creation of reports on the data sets, metadata and / or analyses.

[0029] In this case, the relationship between different metadata entities (e.g. IT systems, data domains, data, business objects or others) can, for example, be automatically represented in differently created reports that depict the semantic relationship in the form of a semantic network between all related objects and, in certain embodiments, also allow a further representation (drilldown) from one entity (e.g. domain level) to all related entities (e.g. data sets, use cases, etc.).

[0030] As new business requirements arise, reports can be edited to add additional automatically generated reports to the system and remove those that are no longer needed.

[0031] In a further further embodiment, it is provided that the user interface is designed to display different user interfaces depending on the role, the access rights and / or editing rights, wherein at least one first user interface is provided for data-entering users and at least one second user interface is provided for administrators.

[0032] In a further refinement, the large language model is designed to provide metadata suggestions to administrators.

[0033] In a further further embodiment, it is provided that a workflow module is provided which is configured to assign tasks for entering data records and / or metadata to a plurality of users.

[0034] In a further refinement, it is provided that the workflow module is connected to the metadata directory, wherein the workflow module is configured to send metadata to the metadata directory.

[0035] Further advantages, features, and details will become apparent from the following description, which – where appropriate with reference to the drawings – describes at least one embodiment in detail. Identical, similar, and / or functionally equivalent parts are provided with the same reference numerals.

[0036] They show schematically: Fig. 1 a database system with a database, Fig. 2 the structure of a data set, as well as Fig. 3 a flowchart of a procedure.

[0037] Fig. 1 shows a database system 2 with a database 4.

[0038] In the database 4, a variety of data can be stored together with associated metadata (see Fig. 2) are filed.

[0039] The database system 2 has user interfaces 6 and 8, with the help of which user B and administrator A can access the database 4.

[0040] If user B wishes to log in to the database system 2 via the user interface 6, a check is first performed to determine whether appropriate access already exists and whether user B has fulfilled all necessary training requirements. These can be repeated cyclically to ensure that user B remains regularly up to date. If this is not the case, appropriate training 10 is conducted, which may depend on the respective role of user B. If this check was successful, user B can access the database 4 according to the corresponding assigned user profile and the corresponding role and enter or query data and metadata.

[0041] Corresponding user inputs entered by user B in database 4 are forwarded to a chatbot 12, which is tasked with interpreting and checking the user input to determine whether user B's inputs are correct and compliant. Chatbot 12 can then provide appropriate suggestions to help user B enter the metadata correctly. Chatbot 12 can be based on a large language model.

[0042] The chatbot 12 obtains the corresponding suggestions from another Large Language Model 14, which is connected to the database 4 and has access to a metadata directory 16 in the database 4. The LLM 14 is supplied with metadata by a user group BG1, whereby the user group BG1 is responsible for defining the relevant metadata that may be stored in the database 4. This defined metadata is then stored in the metadata directory 16.

[0043] The chatbot 12 is further connected to a policy database 18 in which policies valid within the company are stored to ensure that the suggestions made by the chatbot 12 to user B are policy-compliant.

[0044] If User B cannot find a suitable selection of metadata for their needs, User B's metadata suggestions can be sent via LLM 14 via a distribution list 20 to a second user group BG2, which is tasked with reviewing User B's metadata suggestions and, if necessary, approving them. Automated information and confirmation processes can ensure that added responsible metadata providers can edit, confirm, or reject any information added in connection with them if this information is modified or originally saved. A change by User B can automatically trigger a notification to them. The second user group BG2 can partially or completely overlap with the first user group BG1.

[0045] Via user interface 8, administrator A can access database 4 and make any necessary changes to database 4 and the policies to ensure appropriate data quality, particularly with regard to the relevant metadata. A metadata analysis system 22 is used for this purpose, allowing administrator A to more quickly obtain insights into the metadata used and potential policy violations.

[0046] A reporting module 24 is used to generate appropriate reports from the data and metadata of the database 4, whereby the reports can be adapted to reflect new business requirements.

[0047] Database 4 can be linked to additional databases 26, for example, to map and link a company-wide network of different databases. These databases 26 are connected to the metadata directory 16 to enable automatic synchronization of metadata between the systems. For example, one system can contain information about the data domain and the business owner of a data set, while another can contain the ontological position and the associated IT systems for a business object with its ID as a unique identifier.

[0048] Fig. 2 shows the structure of a data set DS.

[0049] The data set DS essentially contains data D and metadata MD. The metadata MD contains a plurality of metadata fields MDF1 to MDF4, each of which has specific functions and may be filled with specific data types, e.g., words, characters, or status states. The metadata fields MDF1 to MDF4 can only be filled with a defined number and type of metadata, which are stored in the metadata directory 16 and defined via the policy database 18.

[0050] In this respect, user interface 6 differs from user interface 8. User interface 6 includes a selection of metadata fields that are searchable for the entities under consideration, e.g., various data objects and data sets. Once a searched entity is selected in this metadata field, all associated information at the viewed level, e.g., associated data sets, data domains, IT systems, and others, is displayed in an automatically generated report and additionally in a semantic network in which all relationships become visible.

[0051] User interface 8 is similar to user interface 6 in this respect, but allows administrators A, who are designated as such by database system 2, additional rights that allow them to edit various fields in the DS data records. When entries are changed, a message is sent to the relevant responsible persons in user group BG2, who must confirm or reject the change. A rejection can trigger a change process in which the administrator must make or suggest alternative entries within a specific timeframe. If these deadlines are not met by user group BG2, corresponding warning messages can be sent to user group BG2.

[0052] Administrator A can also receive suggestions from the Large Language Model 14 and the chatbot 12. Based on the analysis of related metadata MD and the performance of semantic and lexematic analyses of all metadata in the metadata directory 16, appropriate metadata information or responsible users can be suggested to be added to the metadata field under consideration, MDF1 to MDF4.

[0053] The general user B also benefits from this feature. User B is automatically suggested relationships between the information being sought and potentially useful related entities in the semantic network. These suggestions result from a statistical analysis of correctness based on the separation and correlation levels of the entity's metadata, as well as a linguistic (semantic and lexematic) analysis of the entity itself using an LLM.

[0054] Fig.3 shows a possible flowchart of a method for using the database system 2.

[0055] This assumes different users who have edit rights on database system 2 for their respective domains but only read rights for the entire system. Administrator A has additional rights to adjust references between data entities, which are not shown in the flowchart.

[0056] In a first step VS1, a first user, who is responsible for providing metadata MD, is assigned the task of entering metadata into database 4. However, the training process must first be completed.

[0057] In a second step (VS2), the first user completes the corresponding training and is authorized by the database system 2 to use the database system 2. The first user opens the database system 2 via the user interface 6 and checks the metadata stored about him or her in the system. If the metadata is correct, the first user can request that the metadata stored about him or her be adjusted. Should the first user have any questions, they can ask them via the corresponding chatbot 12.

[0058] In steps VS4 to VS7, steps VS1 to VS3 are repeated for a second user and a third user.

[0059] In step VS8, the third user completes a questionnaire specified by threshold values, which checks whether additional metadata is necessary. The corresponding threshold values ​​are stored in the metadata analysis system 22.

[0060] In a further step VS9, the third user checks whether the metadata suggestions from chatbot 12 are correct, with chatbot 12 assisting the third user in selecting the correct metadata. If the preliminary check is negative, the corresponding metadata no longer needs to be entered according to step VS10.

[0061] If the verification is positive, additional users receive a message in step VS11, triggered by the database system 2, requesting them to complete the corresponding training. These additional users are then activated after the training is completed. The additional users then complete the metadata in the database system 2 that still needs to be filled out. Any further questions can be answered via the chatbot 12.

[0062] In a subsequent step VS12, as soon as all users have completed their task, at least one administrator or workflow representative is informed that the relevant metadata has been collected and that the process can be completed.

[0063] In a subsequent step VS13, the list of recorded metadata is updated in the metadata directory 16 and is available for reporting via the reporting module 24. The corresponding reports can be generated automatically.

[0064] In a subsequent step VS14, it can be checked whether the corresponding metadata MD in the system has been updated in the metadata directory 16.

[0065] According to the final step VS14, a data asset assessment can be initiated during the next iteration of the process listed after process steps VS1 to VS11. Changes to the metadata, reports, hierarchical structure, and the like can be automatically incorporated.

[0066] Although the invention has been illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations may be derived therefrom by those skilled in the art without departing from the scope of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned by way of example are truly only examples and should not be construed as limiting the scope, possible applications, or configuration of the invention in any way.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without departing from the scope of protection defined by the claims and their legal equivalents, such as a further explanation in the description. List of reference symbols 2 Database system 4 Database 6, 8 User interface 10 Training 12 Chatbot 14 LLM 16 Metadata directory 18 Policy database 20 distributors 22 Metadata analysis system 24 Reporting module 26 other databases A Administrator B User BG1, BG2 user group D Data DS data set MD metadata MDF1 - MDF4 metadata field QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 2019 / 0057147 A1

[0007]

Claims

[1] Database system (2), with - at least one database (4) in which data records (DS) and metadata (MD) associated with the data records (DS) are stored, wherein a plurality of metadata fields (MDF1 - MDF4) are provided for each data record (DS), - at least one user interface (6, 8) for entering data records (DS) including metadata (MD) by at least one user (B, A), - an authorization system (10) by which users (B, A) or user groups (BG1, BG2) are assigned different roles as well as access rights and / or editing rights for the data records (DS) and / or metadata fields (MDF1 - MDF4), whereby changes and queries to the at least one database (4) are only permitted on this basis, - an explanation system (12, 14) for explaining the metadata fields (MDF1 - MDF4), characterized byin that the explanation system (12, 14) has an agent (12) which supports the at least one user (B, A) in entering data records (DS), wherein the agent (12) has a large language model (14) which is connected to a policy database (4) and a metadata directory (16), wherein the agent (12) is set up to make suggestions for the metadata fields (MDF1 - MDF4) on the basis of the policy database (4) and the metadata directory (16). [2] Database system (2) according to claim 1, characterized by that the authorization system (10) is designed to check a system training status of the at least one user (B, A) and to enforce system training for the at least one user (B, A). [3] Database system (2) according to claim 1 or 2, characterized by that the agent is a chatbot (12). [4] Database system (2) according to one of the preceding claims, characterized bythat a metadata analysis system (22) for checking policy conformity of the metadata (MD) is connected to the metadata directory (16). [5] Database system (2) according to one of the preceding claims, characterized by that the large language model (14) is focused on the data of the policy database (4) and the metadata directory (16) by means of fine-tuning, prompt engineering and / or retrieval-augmented generation. [6] Database system (2) according to one of the preceding claims, characterized by that a reporting module (24) is provided for generating reports on the data sets (DS), metadata (MD) and / or analyses thereof. [7] Database system (2) according to one of the preceding claims, characterized bythat the user interface (6, 8) is designed to display different user interfaces depending on the role, the access rights and / or the editing rights, wherein at least one first user interface (6) is provided for data-entering users (B, A) and at least one second user interface (8) is provided for administrators (8). [8] Database system (2) according to one of the preceding claims, characterized by that the large language model (14) is configured to provide metadata suggestions to administrators (A). [9] Database system (2) according to one of the preceding claims, characterized by that a workflow module is provided which is designed to assign tasks for entering data records (DS) and / or metadata (MD) to a plurality of users (B). [10] Database system (2) according to claim 9, characterized bythat the workflow module is connected to the metadata directory (16), wherein the workflow module is configured to send metadata (MD) to the metadata directory (16).

Images