Machine-learning system for contract intelligence automation using contract digitization into machine parseable objects and method thereof

Abstract

Proposed is a novel machine learning system (1) for contract intelligence automation, and corresponding method for training the machine learning system for automated text analysis and for applying it. A plurality of contracts (2) with a plurality of clauses (22) is received by the system (1), wherein wordings of equivalent clauses (22) vary across contracts (2); contract text (21) is read into a data processing system (15); clause text chunks (121) are identified, that are equivalent across contracts (2), and assigned a contract term category (122). Considering a respective semantic context (131) of each of the contracts (2), a semantic meaning (132) of each of the clause text chunks (121) is determined and encoded in a clause embedding space (141) and stored in vector database (14); data automation tasks can be performed using entries of the vector database (14), particularly automatic consistency monitoring (197) and outlier detection across contracts (2) and a monitoring of clause (22) nuances across contracts (2) e.g. via a graphical representation (16).

Description

[0001] P1478PC00

[0002] Machine- Learning System For Contract Intelligence Automation Using Contract Digitization Into Machine Parseable Objects And Method Thereof

[0003] Field of the Invention

[0004] The present invention relates to machine learning systems and methods providing contract intelligence automation using contract digitization into machine parseable objects having a structured automation layer. In particular, the present invention relates to systems and methods for contract intelligence automation including automated contract management, automated quantitative quality checks and portfolio insights. More particular, the invention relates to a novel system and method of training a machine learning system capable of automated text block recognition and text analysis, to an automated system and method of automated text block recognition and text analysis. In general, the invention relates to a machine learning system and method capable of automated text analysis, and to a machinelearning system and method for contract intelligence automation.

[0005] Background of the Invention

[0006] Contracts serve as the foundational frame-work and interaction basic for economical units and / or entities. They establish enforceable agreements or dictate terms of exchange and responsibilities. Nevertheless, while contracts in most economic sectors typically facilitate the exchange of goods and services, in the risk-transfer industry (e.g. insurance and reinsurance technology), the contract itself predominantly serves as the product. This suggests that contract digitization into machine parseable objects together with an automation layer on top, is poised to have significant impacts on the sector. The advantages of contract intelligence automation spreads among various areas including Contract Management (e.g. central accessibility of terms and conditions with qualitative checks before being legally bound), Contract Quality (e.g. automated quantitative checks of material risk wordings before be-coming legallybound) and Portfolio Insights (e.g. holistic insights of clause wordings across a legally bound portfolio). Further it relates to various data processing and data handling technical problems, Al and smart technology problems, sensible data handling, cybersecurity etc. This invention focuses on solving technical problems associated with the automation and machine-based intelligence of portfolio insights and, inter alia, aims to provide a blueprint of data science architecture behind the automation of contract intelligence.

[0007] Contract automation, as such, involves to using technology to create, manage, and execute legal agreements with minimal or even no human intervention. Implementing such systems involves both technical infrastructure requirements and specific requirements for the contracts themselves to be automatable. The technically required means for contract automation, inter alia, comprises: (A) Template Management System: (i) Template Engine: Must support dynamic insertion of variables, clauses, and conditional logic (e.g., Jinja, Liquid), (ii) Version Control: Track changes to templates over time, and (iii) Standard Clause Library: Reusable pre-approved clauses for consistency; (B) Document Automation Platform: (i)Data Integration: Connect to CRM, ERP, HR systems to pull relevant data (e.g., party names, prices), (ii) User Interface: Guided forms or questionnaires for users to generate contracts, (iii) Rule Engine: Define logic for which clauses or terms to include based on user input; (C) Electronic Signature Integration: (i) Cloud-based platform enabling to send, sign, and manage agreements electronically or at least supporting for electronic platforms like DocuSign, Adobe Sign, etc., (ii) Compliance with e-signature regulations (elDAS, ESIGN, UETA); (D) Contract Lifecycle Management (CLM) system: (i) Workflows: Routing for approvals, reviews, and negotiations, (ii) Alerts / Reminders: For renewals, obligations, expirations, (iii) Audit Trails: Full tracking of all edits, views, approvals; (E) Security and Compliance: (i) Data Encryption: At rest and in transit, (ii) Access Controls: Role-based permissions, (iii) Regulatory Compliance: GDPR, HIPAA, SOC 2, etc.; (F) Integration Capabilities: (i) APIs: RESTful APIs for third-party system integration, (ii) Webhooks: Real-time event notifications, (iii) SSO & Identity Management: Seamless user authentication. However, prior art systems have the disadvantage that they often already put requirements on the kind of contract itself. In the prior art, to be automatable, contracts are required to follow certain structural and content-related rules as (i) Modular Structure: Contracts should be built from modular clauses that can be reused and rearranged. Bespoke, one-off language should be avoided unless necessary; (ii) Standardized Language:Contracts must use clear, unambiguous terms to enable automation and logic-based decisions. Overly complex or nested clauses should be avoided unless supported by the automation logic; (iii) Parameterization: All variable elements (e.g., party names, dates, amounts) typically need to be identified and marked. Tags or placeholders are required for automation tools to insert data (e.g., {{EffectiveDate}}); (iv) Conditional Clauses: Logic for inclusion / exclusion of clauses (e.g., "Include arbitration clause if the value > $50,000") must be clearly defined for automation; (v) Metadata Tagging:

[0008] Contracts require tagging with key metadata (e.g., contract type, jurisdiction, duration) to support search, reporting, and workflows; (vi) Digital Compatibility: Often, file formats are required to be used that support automation (e.g., DOCX, XML, JSON for smart contracts). Another example is PDF, which is often acceptable for final versions but harder to automate at input stage.

[0009] Architecture and building structure, in its broadest sense, forms the foundation of the physical and social landscapes, providing not only functional spaces but also embodying cultural and aesthetic values. Similarly, the architecture and structure of intelligence in machine learning systems is pivotal as it underpins the efficiency, adaptability, and scalability of these advanced technologies that are increasingly shaping our digital experiences. At industry and general industrial production of any kind, buildings and office spaces architecture can play a critical role. It transcends the mere construction of physical spaces, embodying the organization's corporate culture and values. The buildings, though diverse in spatial design, typically maintain a consistent identity of a firm, for example, reflective of the firms's focus on long-term planning and commitment to quality, innovation, and sustainability. The latest years, due to radical advancements of technology, a new kind of architecture is emerging, the architecture of intelligence and automation. This new structural design, although more subtle in nature, can be crucial. It determines the agility and depth with which intelligent systems of an organization can learn, evolve, and respond to an everchanging environment, ultimately driving the frontier of what machines can achieve.

[0010] It is to mention that the concept of a contract is known across various jurisdictions around the world. At its core, a contract is an agreement between two or more parties that creates legally binding obligations. Although not a general prerequisite for all contracts to be valid, many contracts are provided and stored in atextual form such as a written document specifying rights and obligations pertaining to the two or more parties.

[0011] Particularly larger business organizations in some economic sectors accumulate a large number of active contracts that have been concluded with one or more parties. In addition, contracts in text-form may comprise large quantities of text, which may be complex, and phrases used in these contracts can be unique to the industry.

[0012] For managing contracts, business organizations therefore often rely on legal departments in order to professionally manage negotiations leading to contracts, to carefully draft the wording of each contract and for supervising and managing the life cycle of each contract. Despite the professional support of a legal department, the sheer number of contracts, which themselves may be complex and may comprise a large amount of text, often makes it impossible for one person or a small group of persons to be aware of all contracts and encompassed clauses within these contracts, and to provide precise information about certain aspects contained across various contracts. These challenges are often amplified by international contracts including involving parties being based in various jurisdictions. Legal developments such as amended laws, new laws, and in some jurisdictions a revised case law often necessitates a reformulation of clauses over time, such that substantially equivalent contracts drafted at different points of time may have to be formulated differently.

[0013] Not only on the occasion of a legal dispute between two parties it is essential for the content of a contract to be well known to a particular party, and for certainty about obligations and rights between the parties to be obtained, it may also be of high value or necessary to assess or determine a business strategy. Also in financial terms it may be a necessity to gain statistical insight about risks evolving from substantially equivalent clauses, that are included in a number of contracts.

[0014] To obtain such information promptly when needed is challenging, wherein required effort rises with the number of contracts on the one hand and text length and complexity of each particular contract on the other hand.In the following technical context it is assumed that a large number of contracts is to be analyzed with the help of data processing systems, i.e. computer systems, for the purpose of data automation, particularly automated contract management, wherein the contract texts exhibit technical peculiarities: It is assumed that a plurality of contracts are provided, wherein the contracts are provided in text form, such that each of the contracts comprises text with a plurality of clauses. Each clause in turn comprises at least one sentence in human language, and each sentence has a plurality of words. However, one or more of the clauses comprise at least two sentences. The most technical demanding problem for data automation, however, may be that it has to be assumed or is a given fact, that wordings and / or sentence structures of equivalent clauses vary across contracts.

[0015] Technical Objects of the Invention

[0016] It is one object of the present invention to provide a technical solution to ease data automation particularly for contracts with the above mentioned properties. It is a further object of the present invention to transform machinable data format of contract text chunk abstractions into a form that is human readable. It is another object of the present invention to automatically detect statistical outliers in different formulations of substantially equivalent contract clauses from a plurality of contracts. In general, it is one object of the present invention to provide a novel and improved method, and a system for automated for contract intelligence automation. The inventive system and method should further provide a scalable (in respect to processing quantity, quality and volume, as well as in respect to used processing capacity and technical architecture) automated analytics based on a pipeline that is scalable and actionable at the same time.

[0017] Summary of the Invention

[0018] According to the present invention, these objects are achieved, particularly, with the features of the independent claims. In addition, furtheradvantageous embodiments can be derived from the dependent claims and the related descriptions.

[0019] According to the present invention, the above-mentioned objects for an automated system, in particular for a machine learning system and method for contract intelligence automation and automated text analysis with sparse attention pattern providing low memory footprint and compute resource consumption, and more particularly for an automated system and method for contract intelligence automation using contract digitization into machine parseable objects with an automation layer, are achieved, particularly, in that the machine learning system receives and / or digitizes (Tl) a plurality of contracts, each of the contracts comprising text data with a plurality of clauses, each of the clauses comprising at least one sentence with a plurality of words and at least one of the clauses comprising at least two sentences, wherein wordings and / or sentence structures of equivalent clauses vary across at least two contracts; in that the machine learning system reads (T2) said text of received contracts into a persistence storage of a data processing system; in that the machine learning system determining (T4), by the data processing system, a respective semantic context of each of the contracts, and a semantic meaning of each of the clause text chunks based at least on a wording of the respective clause text chunk and based on the determined semantic context of the contract with the respective clause text chunk; in that the machine learning system encodes (T5), by means of the data processing system, each of the clause text chunks based on its respective determined semantic meaning into a respective numerical representation within a clause embedding space, each numerical representation being a single fixed-size vector and the clause embedding space representing a numerical space for encoded semantic meanings of clause text chunks; in that the machine learning system stores (T6) each of the numerical representations in a vector database of the persistence storage; and in that the machine learning system performs (T7), by using the data processing system, a training process of a machine learning unit with a predefined machine learning structure and with a plurality of tunable parameters, the machine learning unit having access to the vector database, the training process automatically adjusting values of said tunable parameters using an optimization structure optimizing a loss function value of a loss function, the loss function being based on a difference between a ground truth answer and an answer predicted by an inference with the machine learning unit in response to a predefined question, wherein the ground truth answer is based on theground-truth dataset. The invention has, inter alia, the advantage that it provides scalable, machine-automated analytics and appropriate output signaling based on an automated pipeline that is technically scalable (in respect to resource consumption and processing volume) and actionable at the same time combining in its technical architecture an automated sequence (pipeline drawing) of fast machine learning structures(ML) for the digital twin, a scalable transformer-based technology for consistencies and the targeted GenAI deep dives for depth. Thus, in respect to the scalability, the data processing system can, for example, be a computer device, a multi-core processor system, or a distributed computing system with distributed data processing nodes, in particular a micro-services-based distributed computer system.

[0020] One key idea of the embedding of clause text chunks is the mapping of an entire text chunk with, in most cases, more than one sentences to a single numerical vector of an embedding space. This contrasts with the word embeddings commonly used in transformer architectures with attention mechanisms and with word embedding methods such as the known "Word2vec", which stands for "word to vector". This idea is especially different from mapping sentence by sentence to a respective numerical representation, where only single sentences are used as text chunks. An inventive key idea is to map entire clause text chunks to an embedding space, wherein at least one of the clause text chunks contains at least two sentences, such that using a standard sentence transformer is insufficient to map the entire clause text chunk comprising at least two sentences to a single fixed size vector. One inventive key idea is therefore to provide an automated system using or based on a machine learning unit, the data processing system being capable of embedding entire clause text chunks with more than one sentences per clause text chunk to one single numerical representation in form of a location vector inside the embedding space. Hence, known techniques to map single words or single sentences into representing vectors of an embedding space are not sufficient to the application of the machine learning unit in this context. One consequence of using an entire clause text chunk to be represented by a respective vector in the clause embedding space is that the clause embedding space comprising vectors encoding clause text chunks is a different embedding space than an embedding space suitable for vectors encoding words or single sentences. It may be assumed that the clause embedding space in general needs to be of higher dimension than an embedding space for encoded sentences of a sentence transformer, since more content for clause text chunks has to be captured than for single sentences.Another key idea is that each word of a particular clause text chunk is contextually considered to determine the semantic meaning of the entire clause text chunk to be mapped into the clause embedding space. That means, raw semantic meanings of single words are not only refined by the semantic meaning of attending words within the same sentence of a clause text chunk, also surrounding sentences within a clause text chunk refine the meaning of another sentence. However, not only the words within a particular clause text chunk are regarded to encode the semantic clause meaning, but also a semantic context of the entire contract text that contains this clause refines the semantic meaning of the clause text chunk. Hence, the semantic understanding of each clause text chunk is contextualized by the semantic direction identified for the contract containing the particular clause text chunk. In other words, it is the goal to capture the semantic meaning of an entire clause, used in the machinable format of a clause text chunk, wherein a semantic context of the contract document is regarded.

[0021] One important technical effect of transforming entire clause text chunks to single fixed size vectors is that not only contextual relationships between words within a clause chunks but also between sentences within a clause text chunk are considered and in addition to that, the entire semantic context of the contract which contains such a clause chunk is considered additionally.

[0022] Every one of these numerical representations of the clause text chunks is stored in a vector database, wherein each numerical representation is a fixed-size vector, i.e. all vectors have the same dimension, which is the dimension of the clause embedding space, and all vectors are location vectors in the embedding space. Hence, a particular vector points into a certain direction and has a certain length within the clause embedding space, wherein length and direction of such a vector represents the semantic meaning of a clause, and is not only determined by the context that is set by the wording and sentence construction of a clause text chunk, but also refined by contextualizing the semantic background of the contract which contains the particular clause text chunk. The vector database may be refined in the fine-tuning process to obtain a fully pre-trained machine learning structure, and is a prerequisite for the application of the machine learning unit at inference time.A clause of the contract can also be called "proviso" and describes a logically coherent passage of a contract referring to a particular obligation, a particular right, a particular condition or the like. Clauses can be labeled manually as such for the training phase of the machine learning structure, to be later automatically identifiable at inference time of the machine learning structure. However, an automated labeling can e.g. also be realized by another pre-trained machine learning structure. The data processing system is then used to identify such clauses, wherein after identification the clauses in the logical contract context are used as clause text chunks for further computational processing steps. Ideally, all clauses are identified correctly as such and used as clause text chunks, whose semantic meaning is to be analyzed and encoded into a semantic vector space, the clause embedding space.

[0023] A clause is a semantically coherent segment of a contract. One contract in general contains a plurality of clauses. Clauses are formulated in the contract, clause text chunks are the text chunks of the contract's text body used by the data processing system based on the identified clauses. Since clauses are the formulation of terms, each of the clause text chunks can be assigned at least one contract term category. It may be the case, that each of the clause text chunks belongs to exactly one contract term category. The clauses are identified for the purpose of the training process of the machine learning system with a manual, semi-manual, or automatic process to create the ground truth dataset. For the purpose of inference with the pre-trained machine learning system, the identification process is done automatically. It is irrelevant to these purposes, whether the clause text chunks are formatted in a certain way within a contract, for instance as paragraph with line breaks, page breaks, or any other formatting means.

[0024] Preferably, a large amount of clause text chunks differing in sentence constructions and / or wordings for a particular contract term category is applied in order to diversify the training dataset for the machine learning unit to adjust its tunable parameters.

[0025] In an analog fashion to the relationship between a clause and a clause text chunk, the relation between a category of a clause and a contract term category is defined. Whether clauses are semantically equivalent is determined objectively by context and the contractually stipulated relationship between the parties of thecontract. Equivalent clauses across contracts may address the same issue but with different wordings and / or different sentence constructions. It is the purpose of annotating training data and after training of the computer system, to assign equivalent clauses the same contract term category. The contract term category is hence a machinable concept for the computer system. The contract term categories can be predefined and chosen from a given list of predefined contract term categories. In an alternative embodiment, the contract term categories are generated by the computer system automatically by grouping semantic meanings of clauses which are addressing equivalently the same issue, but may be formulated with different wordings and / or different sentence constructions across contracts.

[0026] Such a machine learning unit can be built architecturally in two phases: In the first phase, the pre-training phase, understanding language per se is aimed at. To this end, one or a plurality of artificial neural networks can be applied, that are pretrained to semantically process human language, followed by a fine-tuning phase, in which the machine learning system is further trained to understand language given a specific task in a domain-specific context. This fine-tuning phase is specifically adapted to the contract texts with clauses. Furthermore, the fine-tuning phase of the machine learning system can be specifically targeted to domain specific questions relating to the domain specific context. The result of completing the fine-tuning phase is the fully pre-trained machine learning structure.

[0027] The usage of a raw, regular pre-trained Large Language Model, however, would not be sufficient for said purposes since it can be assumed that received contracts use specific wordings that cannot be found and sufficiently represented by Large Language Models which predominantly have been trained with text available on the internet such as Wikipedia pages. For the data automation of contract texts using clause text chunks, a unique notion of similarity of clauses and therefore between numerical representations in the form of representing vectors within a clause embedding space has to be available. Hence, the applied machine learning structure with a basic semantic understanding of human language is fine-tuned by the training process to semantically understand specific and intricate language used in the contracts provided. The technical mean to do so is to use the specific clause embedding space especially adapted for the clause text chunks to capture the meaning of each of the clause text chunks as a whole.The process of fine-tuning of training time for the purpose of handling contract domain specific language and understanding is preferred over a method of retrieval-augmented generation performed at inference-time, since incorporating the available information from the ground truth data set allows for a specifically and carefully pre-trained machine learning structure, resulting in a machine learning structure carefully tailored to contract specific text and the concept of clauses.

[0028] Creating a vector database first, which is stored on a storage device, allows for comparing individual vectors within the vector database as well as comparing a newly created vector outside the vector database with one or more vectors from the vector database.

[0029] The present invention according to the independent claims provides a system and a method for automatically processing and analyzing a plurality of contracts. Each of the contracts comprises text, particularly for expressing various clauses in each contract. It is assumed that the large number of contracts is analyzed and processed and each of the contracts in turn comprises a large quantity of text which may semantically and linguistically complex and often reformulated the in language typically used legal formulations. Since contracts are meant to be read by humans, the composition of contract text reflects human understanding of language and content.

[0030] Thus, since the contract text is not in a standardized structure such as an SQL database, every contract can be assumed to exhibit an unstructured data format with regard to a machine learning system reading the text of a contract. This leads to the necessity of transformation of these unstructured data into structured data and a structured data format, that can be processed by a digital data processing system, adequately. A digital data processing system is made to process digital data, as e.g. numerical data, register addresses or basic Boolean operation, but not to process human readable words on a hardware-near level. The implication of this is a technical requirement of transforming human readable text into a machinable data and data format, that accounts for the underlying mechanism of human language including the concept of semantics being made up not only by the particular meaning of a particular word but instead a semantic meaning that is obtained by certain sequences of words within their context of a clause as part of the semantic context of a contract.That means that a mechanism for automatically processing and analyzing the text of the contracts is to be applied to the contracts not only for the isolated meaning of each of the words that jointly form the text of each contract, but beyond this, based in particular on each position of each word in a clause and semantic connections between words such that implicit relations between words that attend to each other in the text are accounted for.

[0031] Given a pre-trained machine learning system that is capable not only of reading text of contracts while also at least to some degree of understanding the semantic meaning of the clauses, it can be applied to specific tasks in the context of data automation for contract management and analysis. Inferencing with the pretrained machine learning structure following the embedding of the clause text chunks of the contracts provides an advantageous implementation of the concept of data automation, which comprises the collection of text data from the contracts, processing the data and analyzing the data automatically.

[0032] The fine-tuning process may be applied under the consideration of certain objectives during inference with the pretrained machine learning system, such as a question and answering module or system, possibly in the form of a chat bot, text classification, named entity recognition, or for quality checks according to predefined or pre-trained criteria. The specific task may also be called downstream task and may for instance be a question and answering task, where the clause of interest is formed as a question and the prediction of the relevant text in the contract is the answer of the model. This technically is done by stating that a certain word is the beginning of the answer, and that another word is the ending of the answer. Everything between these two positions is part of the answer. This in turn means that given the length of the context being L, there are L number of predictions made for the start and the end positions respectively, providing 2L predicted values. For each of these positions, the computer system has to output a certain value representing a likelihood of that position being a start or end position. This is particularly done by generating two vectors of length L and filling them with the value 0 if that position is neither a start, nor an end position, or with 1 if it is a start or end position.

[0033] The basic steps of embedding clause text chunks enable contract digitization into machine parseable objects. Together with an automation layer on topof the obtained machinable data, automated contract management can be applied. The automated contract management may include a centralized searchable vector database providing central accessibility of contract clauses such as terms and conditions with qualitative checks before, particularly before the contract becomes legally bound. In addition, automated contract quality checks can be performed to identify material risk in wordings, also particularly before the contract becomes legally bound. For an existing contract portfolio, portfolio insights such as holistic insights of clause wordings across a legally bound portfolio can be obtained in an at least partially automated manner. Certain kind of contracts, particularly insurance or reinsurance contracts, are traditionally dense and intricate documents, and are typically packed with complex information that often eludes conventional data processing. It supersedes the capability of a single human or a team of humans to be aware of all clauses of all contracts that are active and thus legally bounding. Applying the automation process and taking advantage of the semantics analysis of the pre-trained machine learning system when inferencing, however, allows for targeted search queries and automated tests to analyze and provide warnings.

[0034] According to an embodiment of the invention, the ground truth answer is a correct identification of a clause from the plurality of contracts based on a predefined question, wherein the predicted answer is a predicted identification of a clause text chunk, the predicted identification being obtained by an inference with the machine learning structure. In this case, the downstream task of the machine learning unit is a Q&A task where the clause of interest is formed as a question and the prediction of the relevant text in the contract is the answer of the model.

[0035] According to another embodiment, for a predefined question, a plurality of paraphrased question-variants with one corresponding ground truth answer is used for training. By providing the model and modeling process with a range of linguistic variations, a discovery performance of specific context, where the formulation of a question or a clause might not follow a typical pattern, can be optimized.

[0036] According to another embodiment, contract texts of at least two of the contracts are in different languages, wherein the machine learning unit is trained to be language-agnostic with regard to contract languages. This can be accomplished usingsaid clause embedding space, which is language-agnostic, since it is a space for numerical representations of semantic meanings in a mathematical numerical space.

[0037] The fine-tuning process of the machine learning structure and machine learning unit, respectively, preferably comprises at least one of the following clause text chunk related methods: Natural language inference, clause text chunk similarity, triplet dataset.

[0038] The natural language inference step takes two clause text chunks and determines whether the first of the clause text chunks entails the second of the clause text chunks semantically or if these clause text chunks contradict each other semantically, or neither of both. To this end, preferably a Siamese Network is used for parallel processing of the clause text chunks. A neural network comprised by the machine learning system may be trained by minimizing a cross entropy of metrics based on these transformed clause text chunks, preferably using the BERT architecture.

[0039] Also, the clause text chunk text similarity method applied to clause text chunks, wherein at least one of the clause text chunks comprises two or more sentences, can make use of a Siamese Network structure. This method in particularly involves the computation of a cosine similarity between numerical representations of two clause text chunks and the comparison of the predicted cosine similarity quantity with a ground truth value.

[0040] The triplet data set in contrast to the two aforementioned methods uses a set of three clause text chunks, namely the anchor clause text chunk, a related clause text chunk and an unrelated clause text chunk. The clause text chunk of one contract belonging to a particular contract term category is taken as the anchor clause text chunk, and another, equivalent clause text chunk belonging to the same contract term from another contract is used as the related clause text chunk, whereas a clause text chunk belonging to a different contract term category from the same or another contract is applied as the unrelated clause text chunk. The anchor clause text chunk, the related clause text chunk and the unrelated clause text chunk are then embedded to respective numerical representations, i.e. to three individual numerical vectors. The goal of the training process that adjusts the tunable parameters of the machine learning system is to minimize a vector distance between the numerical representationsof the anchor clause text chunk and the related clause text chunk, while on the other hand maximizing a vector distance between the numerical representations of the anchor clause text chunk and the unrelated clause text chunk.

[0041] According to an embodiment of the invention, for the training process of the machine learning system, a ground-truth triplet dataset is used, comprising an anchor clause, a related clause, and an unrelated clause, the anchor clause and the related clause being equivalent clauses and their corresponding clause text chunks belonging to the same contract term category, the unrelated clause not being equivalent to the anchor clause and its corresponding clause text chunk not belonging to the contract term category of the anchor clause text chunk; wherein the training process seeks to minimize a vector distance between numerical representations of i) the clause text chunk corresponding to the anchor clause and ii) the clause text chunk corresponding to the related clause, and to maximize a vector distance between numerical representations of a) the clause text chunk corresponding to the anchor clause and b) the clause text chunk corresponding to the unrelated clause.

[0042] According to a further embodiment of the invention, the electronic machine learning system for contract intelligence automation receives or captures via a data transmission interface (SI) a plurality of contracts, each of the contracts comprising text data with a plurality of clauses, each of the clauses comprising at least one sentence with a plurality of words and at least one of the clauses comprising at least two sentences, wherein wordings and / or sentence structures of equivalent clauses vary across at least two contracts; the electronic machine learning system stores (S2) said text of received contracts into a persistence storage of a data processing system, the processing system comprising a machine learning structure specifically pre-trained for identifying clauses in text of each contract as clause text chunks and for identifying equivalent clauses across contracts and assigning each of the identified clause text chunks an associated contract term category, with clause text chunks corresponding to equivalent clauses across contracts assigned the same contract term category; the electronic machine learning system identifies (S3), by the machine learning structure, clauses in each of the contracts as clause text chunks and assigning each of the identified clause text chunks an associated contract term category; the electronic machine learning system determines (S4), by the data processing system, a respective semantic context of each of the contracts, and a semantic meaning of each of theclause text chunks based at least on a wording of the respective clause text chunk and based on the determined semantic context of the contract with the respective clause text chunk; the electronic machine learning system encodes (S5), by means of the data processing system, each of the clause text chunks based on its respective determined semantic meaning into a respective numerical representation within a clause embedding space, each numerical representation being a single fixed-size vector for each of the clause text chunks and the clause embedding space representing a numerical space for encoded semantic meanings of clause text chunks; the electronic machine learning system stores (S6) each of the numerical representations in a vector database of the persistence storage; the electronic machine learning system receives or captures (S7) a question or a task; the electronic machine learning system performs (S8), by the data processing system, inference with the pre-trained machine learning structure using the vector database, wherein the input of the pre-trained machine learning system is the received question or task; and the electronic machine learning system outputs (S9) a result of the inference by generating an output signaling.

[0043] Receiving the question or the task can be conducted automatically by receiving a predefined question or predefined task, i.e. by capturing a predefined question or predefined task from a corresponding database of the persistence storage (which can e.g. also be structured in a decision tree structure) or by receiving a question or task being generated by an Al agent. On the other hand, the question or task can also be provided by user input via an interface, allowing the user to take advantage of the possibilities of prompt engineering techniques.

[0044] According to another embodiment, the electronic machine learning system comprises a storage module storing the numerical or alphanumerical representations of predefined contract term categories, wherein the numerical or alphanumerical representations of clause text chunks are compared with the numerical or alphanumerical representations of the predefined contract term categories for similarity, wherein the clause text chunks are assigned to the contract term categories according to a similarity condition. Since the numerical representation of a contract term category, as well as a numerical representation of a clause text chunk, are each numerical vectors, mathematical methods to compare vector similarity can be applied. Such methods may comprise one or more of the following: Cosine similarity between vectors, computing a dot product of the vectors, or searching for the nearest neighbor around the numerical or alphanumerical representation of each contractterm category to identify closest numerical or alphanumerical representations of clause text chunks belonging to the same contract term category. The goal is that clause text chunks belonging to the same contract term category with semantically similar but different wordings and / or sentence structures are encoded to vectors pointing to similar but different coordinate locations within the embedding space. Pointing to coordinate locations within the embedding with sufficient similarity, however, allows for assigning these differing clause text chunks coherently the same contract term category.

[0045] According to another embodiment, the similarity condition is evaluated by a nearest neighbor search or by computations of cosine similarities. Preferably a k-nearest neighbor method is applied.

[0046] According to another embodiment, the electronic machine learning system performs an automatic consistency check (S10) by detecting a statistical outlier in numerical representations of clause text chunks within a contract term category across contracts, and outputting an information about a detected outlier. The automatic consistency check (S 10), on a system level, thus forms an abnormality detector. In other words, a measure of deviation for each contract's wording from the 'norm' is provided.

[0047] According to another embodiment, the information about the detected outlier comprises the cause, according to which a wording or a sentence construction or both of a clause text chunk are categorized as statistical outlier.

[0048] According to another embodiment, the automatic consistency check (S10) combines detecting a statistical outlier in numerical representations of clause text chunks within a contract term category across contracts with the application of a wording table comprising predefined text tokens, the predefined text tokens being compared with tokens from text of the contracts for detecting deviations between the tokens from text of the contracts and the predefined text tokens from the wording table.

[0049] Typically, this may be applied, where critical information hidden in the wordings of clauses will not be influential enough in the final projection in the contextual space. As a tangible but hypothetical example, consider the case of agency ratingthresholds in a down-grading clause. "Should the security rating of a reinsurer be downgraded at any time below A+ as issued by Standard & Poor and / or A+ as issued by Bests, then the reinsured shall have the option to reduce or terminate such reinsurer's participation (on a basis to be determined by the reinsured) at any time on or after the date of the aforementioned downgrading." In this case, despite the fact that the downgrade trigger would be an A+ rating which is an outlier among the typical A-ratings, it may not be captured by the embeddings projection method. The semantics projections of such wordings are attributed to the clause as a whole and the minor difference of A- or A+ may not change the projections. Nevertheless, such changes are outliers, they translate to risk and can then be captured using the wording table comprising the token 'A-'.

[0050] According to another embodiment, the statistical outliers are detected by: Generating or otherwise computing, by the data processing system, a centroid point of coordinate locations, to which vectors representing clause text chunks of the plurality of contracts are pointing in the clause embedding space, by generating distances of said coordinate locations to the centroid point, and comparing each of the generated distances with a predefined threshold condition. The centroid is a geometric center of the end points of the location vectors in the clause embedding space, which is an n-dimensional Euclidean space. The centroid point is preferably generated as an arithmetic mean of the coordinate locations, to which the vectors representing clause text chunks are pointing to. Because of this, the distances of said coordinate locations to the centroid point are particularly Euclidean distances. Employing the Burr distribution as a model for these distances allows for the estimation of the tail behavior of the distribution of deviations, which is essential for detecting outliers effectively. A probability density function (pdf) of the Burr distribution comprises shape parameters. By fitting the parameters of the Burr distribution to the empirical distribution of Euclidean distances, outliers can be identified as those points whose distance exceeds a certain threshold, typically determined by the tail property of the fitted Burr distribution. This approach is particularly advantageous in the context of insurance contract wordings due to its sensitivity to extreme values and its flexibility in modeling the tail-heavy distributions often observed in such data. As a result, it provides a robust and reliable technical framework for detecting those contracts that require further scrutiny, thereby enhancing the overall efficiency and accuracy of outlier detection in a portfolio management process.According to another embodiment, the electronic machine learning system performs, using the data processing system, a dimensionality reduction of the vectors from the vector database to a reduced space, the reduced space having a dimension lower than the dimension of the embedding space and being bigger than zero and lower than five, and generates output signaling for and / or visualizes coordinate locations, to which vectors in the reduced are pointing. For instance, a reduced space with a dimensionality of 4 can be visualized via elements in a three-dimensional coordinate system and colors, establishing a heatmap visualization. On the other hand, a reduced space with dimensionality of 3 can either be visualized in a 3D coordinate system without colors of the data elements or in a 2D plane with colors. While vectors from the vector database can be used for such a dimensionality reduction, also, logits or logits mapped by a softmax function to the values of a resulting probability distribution as the output of an artificial neural network as part of the pre-trained machine learning system can be used in a dimensionality reduction process, preferably followed by a visualization step.

[0051] According to another embodiment, a graphical plot of coordinate locations, to which vectors in the reduced space are pointing, is generated, the plot containing contour lines enclosing clusters of coordinate locations with semantic similarities.

[0052] In an embodiment variant, the electronic machine learning system for contract intelligence automation and automated text analysis with sparse attention pattern providing a system architecture using low memory footprint and compute resource consumption, comprises a receiver, that receives a plurality of contracts, each of the contracts comprising text data with a plurality of clauses, each of the clauses comprising at least one sentence with a plurality of words and at least one of the clauses comprising at least two sentences, wherein wordings and / or sentence structures of equivalent clauses vary across at least two contracts; a data processing system capturing said text data of received contracts, the data processing system comprising a machine learning system specifically pre-trained for identifying clauses in the text data of each contract as clause text chunks and for identifying equivalent clauses across contracts and for assigning each of the identified clause text chunks an associated contract term category, with clause text chunks corresponding to equivalent clauses across contracts assigned the same contract term category,wherein the data processing system is configured to identify clauses in each of the contracts as clause text chunks and to assign each of the identified clause text chunks an associated contract term category, wherein the data processing system is configured to determine a respective semantic context of each of the contracts and a semantic meaning of each of the clause text chunks based at least on a wording of the respective clause text chunk and based on the determined semantic context of the contract with the respective clause text chunk, wherein the data processing system is configured to encode each of the clause text chunks based on its respective determined semantic meaning into a respective numerical representation within a clause embedding space, each numerical representation being a single fixed-size vector for each of the clause text chunks and the clause embedding space representing a numerical space for encoded semantic meanings of clause text chunks, and wherein the data processing system is configured to store each of the numerical representations in a vector database; a detector for detecting, as automatic wording consistency check, a statistical outlier in numerical representations of clause text chunks within a contract term category across contracts by performing an inference with the pre-trained machine learning structure using the vector database, the detector outputting an output signaling indicating a detected outlier.

[0053] Advantages and preferred embodiments of the system for automated text analysis can be derived by mutatis mutandis applying the features of the specification shown in connection with the inventive machine-learning based method and system for contract intelligence automation and automated text analysis.

[0054] The description herein, in particular with regard to the drawings below, is presented for purposes of illustration and understanding, and is not intended to be exhaustive or limited to the invention in the form disclosed. Various modifications and variations will be apparent to those of ordinary skill in the art. The embodiments, in particular with regard to the drawings, are chosen and described in order to best explain the principles of the invention, the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as suited to a particular contemplated use.

[0055] According to on embodiment variant of the present invention, there is provided a computer-readable storage medium comprising computer-executableprogram instructions stored thereon that when executed by a computer processor perform a method according to one of the aspects of the present invention.

[0056] Brief Description of the Drawings

[0057] The present invention will be explained in more detail below relying on examples and with reference to these drawings in which:

[0058] Figure 1 shows a block diagram, schematically illustrating a general structure of a received contract 2 with four (a-d) exemplary clause text chunks 121.

[0059] Figure 2 shows a block diagram, schematically illustrating a transformation process of contract clauses 22 to a clause embedding space 141 according to an embodiment of the invention.

[0060] Figure 3 shows a block diagram, schematically illustrating a process of creating the vector database 14 by transforming the machinable clause text chunks 121a and 121b of the contracts 2 a / b into associated numerical representations 133, which are numerical fixed size vectors 1411 a and 1411 b according to an embodiment of the invention. Fig. 3 shows the logical continuation of Fig. 2 in the process of creating a vector database 14 by the data processing system 15 from a plurality of contracts 2 as the exemplarily shown first contract 2 a and second contract 2 b. In each contract 2, clauses 22 are identified and their text 21 used as clause text chunks 121 as described by figure 2 by the data processing system 15. Every clause text chunk 121 is transformed into an associated embedding vector 1411, wherein all the embedding vectors 1411 are stored in a vector database 14. This way all received contracts 2 are semantically represented by numerical vectors 1411 in a high-dimensional clause embedding space 141, such that the data processing system 15 can process the automated contract analysis.

[0061] Figure 4 shows a block diagram, schematically illustrating a fine-tuning of the machine learning unit 11 for the automated contract analysis and therefore of completing a training of a machine learning unit 11 to be capable of automated textanalysis, according to an embodiment of the invention. Step T1 is the receiving and / or digitizing plurality of contracts 2; step T2 is the storing of contracts 2 to persistence storage 151; step T3 is the creating ground-truth dataset 12; step T4 is the determining semantic context 131 of contracts 2; step T5 is the encoding of clause text chunks 121; step T6 is the storing numerical representations 131 as single fixed-size vector 1411 in vector database 14; step T7 is the performing training process of machine learning unit 11; and optional step T8 (not shown) is the detecting a statistical outlier in numerical representations (consistency check).

[0062] Figure 5 shows a block diagram, schematically illustrating an automatic contract analysis using the fine-tuned and therefore pre-trained machine learning system 1 according to an embodiment of the invention. If the automated contract analysis is to be executed for the contracts 2 that have already been received in the training process and for which already embeddings in form of vectors 1411 have been generated from the clause text chunks 121, the vector database 14 can be used to analyze the clause text chunks 121 using the vectors 1411, which are associated with the respective contracts 2 they stem from. Step SI is the receiving and / or digitizing plurality of contracts 2; step S2 is the storing of contracts 2 to persistence storage 151; step S3 is the identifying clauses 22 in the contracts 2 as clause text chunks 121; step S4 is the determining semantic context 131 of contracts 2 and a semantic meaning 132 of clause text chunksl21; step S5 is the encoding of clause text chunks 121; step S6 is the storing numerical representations 133 as single fixed-size vector 1411 in vector database 14; step S7 is the receiving question or a task; step S8 is the performing inference with the machine learning structure 111; step S9 is the outputting or transmitting output signaling 118 as result of the inference; and the optional step S10 (not shown) is the detecting a statistical outlier 1331 in numerical representations 133 (consistency check).

[0063] Figure 6 shows a block diagram, schematically illustrating vector representations 1411 mapped to a reduce space 142 by dimensionality reduction according to an embodiment of the invention. Thus, figure 6 schematically shows a projection of embedded clause text chunks 121 into a plane. This projection is achieved by converting the distances between data points in the high-dimensional space 141 into conditional probabilities that represent similarities. The data are arranged in the two-dimensional space such that these similarities are preserved. The wordings of the clauses 22 are shaping the observed clusters 14222 and semanticallymeaningful clusters 14222 (like war, terrorism and nuclear (cf. dotted box in figure 7)) are found close to each other. A vector database 14 stores a plurality of vector 1411 embeddings, wherein each of the vectors 1411 is the result of an abstraction process from the text modality input to a respective numerical representation 133. Each vector 1411 comprises an array of numbers, wherein each component of such an array represents a learned feature. The numerical representation 1411 of a learned feature is in general noninterpretable for a human since the level of abstraction in the clause embedding space 141 has in general no useful correlation with the human mind.

[0064] Figure 7 shows a block diagram, schematically illustrating an exemplary projection of clauses-wordings contextual embeddings in a plane according to figure 6 in more details. The wordings of the same clause 2 are shaping the observed clusters 14222 and semantically meaningful clusters 14222 (like war and terrorism (cf. dotted box in figure 7)) are found close to each other. The right part of the image highlights how such representation can provide value combined with outlier detection T8 / S10. For instance the wording from the main area of war cluster 14222 is a typical war exclusion while the wording corresponds to the edges of the war exclusion cluster 14222, while it speaks about war it refers at the same time to voyages and plans pointing to the fact that it belongs to a contract that comes from some aviation specialty line.

[0065] Figure 8 shows a block diagram, schematically illustrating an exemplary loss function across the steps (T1-T7 / S1-S9) and epochs. The monitoring of loss decreases and stabilization during training, together with the validation loss decrease controls that the machine learning structure 111 is learning and converging towards a set of parameters that minimizes the error. In figure 8, epochs are not explicitly displayed. They are hidden in the sense, that the training and validation losses are computed in each (training) step (on the y-axis), and each epoch consists of the same number of steps. So for example, 500 (training) steps may be 5 epochs, or only 2, or even only a single epoch. However, this are only details of an example of how the training can actually be conducted. Further note that the thick black line is an added visual element to highlight convergence, but not an organic result of the training.

[0066] Figure 9 shows a block diagram, schematically illustrating an exemplary pipeline 19 of contract intelligence. It is composed of two main parts, the creation of the digital twin 191 and the consistency automation. From left to right, the contracts 2are optical character recognized 192, then the recognized data are normalized and layout information is extracted 1 3. Language detection 194 is applied before the final clause 22 extraction 195 for detecting the contract language 23. The clause 22 extraction 195 can e.g. comprise fine tuning 1951 of the machine learning structure 111, validation 1952, and automated document annotation by an automated document annotation system 1953. The step of clause extraction 195 is linked here with the latest version of a machine-learning structure 111, which can e.g. be trained in a HITL (human-in-the-loop) paradigm to cope with complex data. In terms of a HITL process and data generation, the machine learning structure 111 has to be provided with concrete examples of clauses 22 in the specific field, e.g. reinsurance clauses, herein referred as document annotation 1953. This can be achieved by experts (for example, digital experts) opening contracts in a Document Annotation System in place and highlighting texts with their corresponding clause titles. As soon as these annotations 1953 are available for the system 1, tuples of input / output examples data objects are created and the model training can start. The training data can e.g. be split into 3 parts: the train set, the validation set and the test set. The machine learning unit 11 uses the train set to do the actual parameter updates during the training step: this data is what the model learns from. The validation step 1952 is used during training to detect over / underfitting but is not used to update the model's parameters. Its use is limited to setting the cut-o’ value for the training step when overfitting is detected for example. The test set is used to finally assess the machine learning structure's performance on an entirely new and unseen dataset, this is used to determine if the machine learning structure 111 is still capable of generalization, even after fine-tuning 1951. Then, as soon as the digital twin 191 is created, the consistency automation 197 is enabled with prompt engineering 196 allowing actionable insights and / or automated control monitoring e.g. organized in a service dashboards or a SCADA (Supervisory Control and Data Acquisition) control system.

[0067] Figure 10 shows a block diagram, schematically illustrating exemplary the design paradigm of the present electronic machine learning system 1 for contract 2 intelligence automation enabling scalability, allowing horizontal breadth and targeted depth (icicle design paradigm).Figure 11 shows a block diagram, schematically illustrating an exemplary precision, recall and Fl score performance per term together with the 80% driven threshold.

[0068] Detailed Description of the Preferred Embodiments

[0069] The accompanying drawings incorporated in and forming part of the specification illustrate several aspects of the present invention, and together with the description, serve to explain in more detail, by way of example, the principles of the invention. The drawings are drafted schematically and not drawn to scale. The description herein, in particular with regard to the drawings below, is presented for purposes of illustration and understanding, and is not intended to be exhaustive or limited to the invention in the form disclosed. Various modifications and variations will be apparent to those of ordinary skill in the art. The embodiments, in particular with regard to the drawings, are chosen and described in order to best explain the principles of the invention, the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as suited to a particular contemplated use.

[0070] The training process of the machine learning unit 11 and its application using inference are closely interconnected. For this reason, exemplary features shown in the context of one of these two processes may be transferred by the person skilled in the art to the other process. A feature described in the context of one drawing may thus be transferred to the context of another drawing.

[0071] Figure 1 illustrates properties of a particular contract 2 received. In Figure 1 , an overview picture of a contract 2 on the left with a detailed concept of the same exemplary contract 2 on the right is shown. While a plurality of contracts 2 needs to be used for fine-tuning the electronic machine learning system for contract intelligence automation 1 with a basic language core module, at least one contract 2 is used for the pre-trained machine learning structure 11 during inference time as input to analyze the at least one contract 2 for data automation. The one or more contracts 2 used at inference time can be the same that were used to train the machine learning structure111 but can also be different. Independent of the application of a particular contract 1, be it the fine-tuning of the machine learning unit 11 or be it the usage of the pretrained machine learning structure 11 after the fine-tuning process for automated analysis of contract text 21, it is assumed that every contract 2 provided and received comprises contract text 21 , which is organized in clauses 22. Corresponding with the clauses 22 of each of the contracts 2, the machine learning system 1 segments the particular contracts 2 to clause text chunks 121. For illustration purposes in figure 1, for one single contract 2 four exemplary clause text chunks 121 are shown: the clause text chunks 121a, 121b, 121c, 121d, while typical contracts 2, as e.g. (re)insurance contacts or other type of contracts, comprise more clauses 22 than four. Each of the clause text chunks 121 can be associated with one of a plurality of predefined contract term categories 122. This may be done manually for generating the ground-truth dataset 12, in an alternative embodiment, the contract term categories 122 are determined by the machine learning unit 11 itself, for instance by grouping semantically similar clauses 22 together based on similarity conditions 124, for instance with a k-nearest neighbor search process 1241. Depending on the industry and field, from which the contracts 2 are originating, intricate details such as, for example in the (re)insurance technology, risk assessment metrics, actuarial calculations, or specific insurance or reinsurance terms that define the layers of coverage and shared liabilities can be found in the contracts 2. The following list shows examples of contract term categories 122, with which the clauses 22 can be associated: Contracting Parties; Geographical Scope; Arbitration; Governing Law Jurisdiction; Scope of cover: Lines of business and / or Business Covered; Coverage & Exclusions: War; Coverage & Exclusions: Nuclear Risks; Coverage & Exclusions: Terrorism; Coverage & Exclusions: Cyber; Coverage & Exclusions: Pollution; Premium / Rate; Ceding Commissions; Profit Commission; Claims Notification; Claims Cooperation; Loss Settlement; Assignment of contractual rights; Commutation of contractual rights; Confidentiality; Data Protection; Accounting; Currency; Translation.

[0072] Figure 2 shows schematically the process of transforming the machinable clause text chunks 121a and 121b into associated numerical representations 133, which are numerical fixed size vectors 1411a and 141 lb as shown in figure 2 schematically. The first clause text chunk 121a is transformed to the first numerical representation 133 in the form of a first vector 1411a, and the second clause text chunk 121 b is transformed to a second numerical representation 133 as the second numerical vector 1411b. This process is also called embedding, since the vectors 1411a and 1411b are locationvectors 1411 in a clause embedding vector space 141. The same process is repeated for any other clause text chunk 121 that has been extracted from a particular contract 2.

[0073] Figure 3 shows the logical continuation of figure 2 in the process of creating a vector database 14 by the data processing system 15 from a plurality of contracts 2 as the exemplarily shown first contract 2a and second contract 2b. In each contract 2, clauses 22 are identified and their text 21 used as clause text chunks 121 as described by figure 2 by the data processing system 15. Every clause text chunk 121 is transformed into an associated embedding vector 1411, wherein all the embedding vectors 1411 are stored in a vector database 14. This way all received contracts 2 are semantically represented by numerical vectors 1411 in a high-dimensional clause embedding space 141, such that the data processing system 15 is able to conduct an automated contract analysis. The schematic illustration of figure 3 is independent of the purpose of the application of the data processing system 15, be it the fine-tuning process of the machine learning unit 11 integrated in and available by the data processing system 15, or be it the application of the pre-trained machine learning structure 111 of the data processing system 15 at inference time for conducting an automated contract analysis such as an automated consistency detection process 197 / T8 / S10. A pretrained machine learning structure 111 capable of natural language processing and fine-tuned for peculiarities of a certain kind of contracts 2 is fed the text 21 of each contract 2a, 2b and potentially more contracts 2 as input data. The machine learning system 1 comprises a module for a basic semantic processing capabilities of human language, thus considering the contextual surroundings of every word 222 of a sentence 221. To interpret singular words 222 with regard to an overall semantic context 131 may be done with the help of an attention mechanism of a core module 17 for natural language processing. It is determined which word attends to another one to update and refine a semantic meaning 132 of a word 222 or token. An introduction of the transformer architecture using attention mechanism is e.g. provided by the following publication: Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, tukasz Kaiser, and lllia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'! 7) . Curran Associates Inc., Red Hook, NY, USA, 6000-6010.It should be recognized that other architectures or particular modifications of architectures based on the general term of the transformer architecture may also be suitable in the role of such a language core module 17. A suitable extension of the transformer architecture for instance is the "BERT" network architecture; other derivatives of the former such as "RoBERTa" can be used was well. Various architectures of a core module 17 for natural language processing capable of a semantic processing of human language may be implemented without departing from the spirit and the scope of the embodiments described. BERT stands for "Bidirectional Encoder Representations from Transformers". It comprises a contextual, multi headed attention based deep learning architecture. An important improvement of BERT over static word representations within a text based on simple lookups from a word-embedding table is that the embedding vectors are contextualized at each position given the entire text a word appears in. The BERT architecture is described in the following publication: Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1, pages 4171-4186, 2019.

[0074] Modifications of the BERT architecture are known, too. Particularly a modification of a pre-training step of a model of BERT with minor tweaks to (mostly) pre-training hyperparameters and removing the "next-sentence-prediction" training objective, making the model "next-token" only, is provided by at the modification called RoBERTa. Generally, RoBERTa models outperform pure BERT models (e.g. see Yinhan Liu, Myle Oft, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov. RoBERTa: A Robustly Optimized BERT pre-training Approach. 2019. https: / / doi.org / 10.48550 / arXiv.1907.! 1692).

[0075] In regular self-attention processes or mechanisms 134, the order of computational effort required to evaluate semantic relations between tokens (or words 222) scales in quadratic growth with the sequence length of input data and therefore with the number of tokens being considered in the self-attention mechanism 134. The availability of computational resources hence limits the number of tokens (or words 222) that can be analyzed together in their mutual relations. One possible solution to overcome this limitation is known in the art as the so called “ longformer" structure or architecture 1341, reducing the quadratic scaling with the number of tokens to be considered for mutual semantic relations to a linear or near-linear scaling. The19

[0076] longformer architecture 1341 is based on the key-ideas of the self-attention of a transformer architecture for a large language model, but allows for the processing of much larger text 21 quantities in terms of mutual relations between tokens and therefore between words 222 of the text 21 given a computational boundary. To this end, the longformer architecture 1341 combines a windowed attention mechanism using a sliding window on input data combining it with an adapted global attention mechanism, wherein the latter is to include global semantic information of the inputted text as a whole. This way, even longer documents with thousands of tokens or more can be analyzed by the attention mechanism 134 to detect mutual semantic relations between tokens and words 222. The longformer architecture is e.g. described in the following publication: Iz Beltagy, Matthew E. Peters, Arman Cohan. Longformer: The Long-Document Transformer. arXiv e-prints. Art. no. arXiv:2004.05150, 2020.

[0077] doi: 10.48550 / arXiv.2004.05150.

[0078] It can be shown that the longformer architecture 1341 consistently outperforms "RoBERTa" on long document tasks. This long-document-transformer 1341, or “longformer" is a drop-in modification to the attention components of the transformer architecture 134 to allow for longer input sequences. Such modification incorporates small local attention spans and increasingly wider attention spans in higher layers. Global attention is used to encode the task, be it classification or questionanswering, among others. The limitation of full attention matrices being memory complexity, scaling exponentially with input length, the longformer structure 1341 uses sparse attention patterns to ensure only linear scaling (in terms of input tokens) happens, decreasing the memory footprint and compute resources during inference and training. Alternative known modifications of the attention mechanism also aim at making context more scalable and to increase the context window size and can be used, for example: Reformer, Linformer, Sparse Attention Mechanisms, Blockwise Attention, Ring attention. Other alternatives known in the art can be applied as long as suitable to the underlying problem.

[0079] Figure 4 shows an embodiment variant of the fine tuning and therefore of completing a training of a machine learning unit 11 to be capable of automated text analysis. A first step is receiving (Tl) a plurality of contracts 2. The contracts 2 have specific properties: Each of the contracts 2 comprises text 21 with a plurality of clauses 22, each of the clauses 22 comprises at least one sentence with a plurality of words 222and at least one of the clauses 22 comprises at least two sentences. Wordings and / or sentence structures of equivalent clauses 22 vary across at least two contracts 2. Text 21 of the provided contracts 1 is read (T2) into a data processing system 15. This process may involve an optical character recognition step 192, if the contract text 21 is only available in a format that does not encode the text 21 directly in digital letters, such as in an image format (jpeg, png, tiff etc.) resulting from scanning a paper contract 2. A conversion of PDF format may also be applied 1 9; the goal is to obtain contract text 21 in directly a machine-readable text format, preferably plain text 21. After that, text data 21 that are received in such a way are normalized 193 and a layout information is extracted 193. Language detection 194 is applied before a final clause extraction 195. This is followed by the step of generating (T3) a ground-truth dataset 12 by identifying clauses 22 in said text 21 of each contract 2 as clause text chunks 121, identifying equivalent clauses 22 across contracts 2 and assigning each of the identified clause text chunks 121 an associated contract term category 122, with clause text chunks 121 corresponding to equivalent clauses 22 across contracts 2 being assigned the same contract term category 122; by the data processing system 15, a respective semantic context 131 of each of the contracts 2 is then automatically determined (T4), for instance by using the embedded clause text chunks 121 to determine a semantic context 131 of a particular contract 2. A semantic understanding of each contract text 21 is generated by the machine learning unit 11, and a semantic meaning 132 of each of the clause text chunks 121 based at least on a wording of the respective clause text chunk 121 and based on the determined semantic context 131 of the contract 2 with the respective clause text chunk 121 is determined. For a machinable format, an encoding (T5) step of each of the clause text chunks 121 is conducted by the data processing system 15 based on the respective determined semantic meaning 132 of each clause text chunk 121 into a respective numerical representation 133 within a clause embedding space 141, each numerical representation 133 being a single fixed-size vector 1411 and the clause embedding space 14 representing a numerical space 141 for encoded semantic meanings 132 of clause text chunks 121. After storing (T6) each of the numerical representations 133 in a vector database 14, a training process of the machine learning structure 111 can be performed (T7) by the data processing system 15. The machine learning structure 111 has a predefined structure and a plurality of tunable parameters. Using the vector database 14 as shown in figure 3, the training process automatically adjusts values of the tunable parameters according to the chosen optimization algorithm for optimizing a loss function value, the loss functionbeing based on a difference between a ground truth answer 116 and an answer 117 predicted by an inference with the machine learning structure 111 in response to a predefined question or task 115 (e.g. appropriately input parametrized), wherein the ground truth answer 1 16 is based on the ground-truth dataset 12 (see figure 8) . The specific values of tunable parameters of the machine learning structure 11 1 are determined by an automated adjustment process, preferably a back propagation method. Particularly in artificial neural networks, these parameter values are often called weights. These weights are adjusted in the process such that the value of a cost function is minimized, or the value of a value function is maximized, depending on the definition of the loss function applied. In other words, the metric used to assess the internal machine-learning structure's 111 performance is the so-called loss, which in case of this model, for example, can be a cross-entropy-loss and can be defined to be:

[0080]

[0081] where ytthe true label and ytis the predicted probability of the corresponding class. In general and depending on the used loss function, typically, the loss, e.g. a cross-entropy-loss, expressed by the loss function value, is to be minimized (cf. figure 8). The back propagation algorithm to tune the values of the parameters is accompanied by the approach of steepest decent minimization of a current loss function value. For example, an initial model performance and sanity can e.g. be conducted by the loss convergence curves. Apart from fine-tuning configurations related to number of epochs, evaluation frequency, learning rate and learning rate decay, no other hyperparameters can e.g. be set for the fine-tuning step. The number of epochs can e.g. be kept low (e.g. 3), if there are few samples only, however, evaluation can e.g. be set to happens e.g. every 5 steps, which has the advantage to provide a good way to monitor training success. Additionally, gradient accumulation (e.g. of size 32) can be used to be able to artificially increase the batch sizes from 1 which appears to be the maximum number of batches that can possibly fit on a specific data processing system 15 (e.g. having 16GB GPU). Generally, as illustrated in figure 8, a decreasing trend for the loss in each step can often be observed (e.g. each of a 32 gradient-accumulated updates), with some degree of (expected) noise. The fine-tuning can be prematurely terminated even before the exemplary full 3 epochs are completed, if the goal is achieved. After training, the machine-learning structure's 111 performance can e.g. be evaluated using metrics derived from a confusion matrix,such as accuracy, precision, recall, and Fl score. These metrics provide insights into the machine-learning structure's 111 effectiveness on unseen data. In accordance with predefined objectives, the identification performance threshold per term can be set to a defined value, as e.g. Fl Score = 80% . Figure 11 shows an example of a machinelearning structure's 111 precision, recall and Fl score per term, together with the predefined’ threshold.

[0082] The training process involves the careful tuning of numerous parameters and layers, through which the machine learning structure 111 of the machine learning unit 11 learns to make predictions based on input data. Over various training steps and epochs, a further diminishing value of the loss function during the training process suggests that the tunable parameters inside the machine learning structure 111 are converging to a desired state that generalizes the training datasets used. The target of this fine-tuning process of the machine learning unit 111 is to prepare it to be capable of handling the peculiar language of contract texts 21 and understand semantic meaning 132 of clauses 22, particularly such that they can be compared to each other. The machine learning structure 111 is therefore trained to comprehend the intricacies of a certain kind of contracts 2 and their unique clauses 22. Through extensive training on a corpus of literature and contract examples, the electronic machine learning system 1 for contract intelligence automation is fine-tuned 1951 to grasp the specific jargon, concepts, and structures inherent to the industrial or technical sector 1 511 the contracts 2 stem from. This is done with the help of ground truth data 12 of related and appropriate training dataset. The ground truth data 12 can e.g. be obtained by manually labeling original text 21 passages of the particular contract 2 or by automatically annotating these passages. Manual labeling can be achieved by human experts as such as digital lawyers who are opening contracts 2 in a document annotation 1953 system 19531 in place and highlight texts 21 with their corresponding clause 22 titles. Another example for automated labeling on the other hand is the labeling by another pre-trained machine learning structure 19532. As soon as these annotations are available, tuples of input / output examples data objects are created and the machine learning structure 111 training can start. The training data 1121 is preferably split into three parts: the training set 11211, the validation set 11212 and the test set 11213. The train set 11211 is used to perform the parameter updates during the training step. The validation set 11212 and validation step is used during the training process 112 to detect overfitting and underfitting, but is not used to update theparameter values. Its use is limited to setting the cut-off value for the training step when overfitting is detected for example. The test set 11213 is used to finally assess the machine-learning unit's 11 performance on an entirely new and unseen dataset to determine if the machine-learning structure 111 is still capable of generalization, even after the fine-tuning phase. Depending on the type of contract 2 that needs to be automatically managed by the fine-tuned and therefore completely pre-trained machine learning system 1, such training sets need to be choosing accordingly from this type of contract 2. When the contracts 2 to be managed come from a specific field, as e.g. the field of insurance contracts, the training sets 11211 should be provided accordingly from one or more field-specific contract 2, i.e. insurance contracts in this case. When the contracts 1 to be managed come from another field, e.g. the field of reinsurance with reinsurance contracts, accordingly the training sets 11211 to be used for the training process of the machine learning system 1 should be taken from one or more reinsurance contracts 2. A proper training data strategy in order to maintain a high level of performance is to be employed: Training data 1121 from past training steps are re-used to ensure that the machine-learning structure 111 retains knowledge from previous iterations and avoids catastrophic forgetting. This has the advantage in stabilizing the learning process and maintaining a consistent performance over time. A big amount of different clause inputs for the same term category 122 are applied so to diversify the training dataset 11211. This approach has the technical advantage in enhancing the machine learning system's 1 ability to generalize across various ways of expressing the same concept, thereby improving its robustness and technical reliability. Paraphrased question phrases 115 of the question-answer-pairs 115 / 116 are used to provide the machine-learning system 1 with a range of linguistic variations. This exposure has the technical advantage to achieve an optimized discovery performance of specific concepts where the formulation of the question 115 might not follow a typical pattern. A machine learning system 1 performance step is conducted on all past test sets ever generated. This comprehensive evaluation ensures that the machine learning system's 1 performance is consistently measured and that it continues to perform well on all previously seen data, thus preventing regression in its capabilities. Ensuring the accuracy and reliability of the machine learning structure 111 is well controlled, a rigorous validation framework is desired that leverages established machine learning key performance indicators (KPIs). This framework not only takes into account the internal deep neural network KPIs but at the same time ensures that the predictions align with the broader technical objectives.Once the machine learning system 1 is fully fine-tuned and thus fully pretrained, it can be used to either analyze the contracts 2 used for the training process 112 of the machine learning system 1 itself, or it can be applied to new contracts 2 that have not been used for the training process 112 of the machine learning system 1. Embeddings of clause text chunks 121 of one or more contracts 2 stored in the vector database 14 can be used as semantic dictionary and source of information about the machine read contracts 2.

[0083] Figure 5 shows an exemplary automated text analysis schematically. Again, the transformation of received contracts 2 from unstructured data into a structured, actionable format needs to be performed, initially. Contracts 2, in all kind of industrial or technical fields, are typically dense and intricate documents, packed with complex information that typically eludes conventional data processing. By converting these contracts into a digital twin 191, i.e. a precise, digital counter-part, each clause 22 is mapped out and structured in a way that the machine-learning system 1 can interpret and manipulate it. This process lays the basis for subsequent automation processes, enabling more efficient, accurate, and agile contract management in the digital environment. Further, if the automated contract analysis is to be executed for the contracts 2 that have already been received in the training process 112 and for which already embeddings in form of vectors 1411 have been generated from the clause text chunks 121, the vector database 14 can be used to analyze the clause text chunks 121 using the vectors 1411, which are associated with the respective contracts 2 they stem from. The following steps comprise the case that the vector database 14 for the contracts 2 to be analyzed is not yet available, for instance because new contracts 2, that are different from the contracts 2 that were used to train 112 the machine learning system 1, are received by the data processing system 15. In this latter case, the first step to be conducted is receiving (SI) a plurality of contracts 2, each of the contracts 2 comprising text 21 with a plurality of clauses 22, each of the clauses 22 comprising at least one sentence 221 with a plurality of words 222 and at least one of the clauses 22 comprising at least two sentences 221, wherein wordings and / or sentence structures of equivalent clauses 22 vary across at least two contracts 2; followed by reading (S2) said text 21 of received contracts 2 into the data processing system 15, the data processing system 15 comprising a machine learning unit 11 specifically pre-trained for identifying clauses 22 in text of each contract 2 as clause text chunks 121 and for identifying equivalent clauses 22 across contracts 2 and assigning each of the identified clausetext chunks 121 an associated contract term category 122, with clause text chunks 121 corresponding to equivalent clauses 22 across contracts 2 assigned the same contract term category 122; in turn followed by identifying (S3), by the machine learning system 1, clauses 22 in each of the contracts 2 as clause text chunks 121 and assigning each of the identified clause text chunks 121 an associated contract term category 122; and the determining (S4), by the data processing system 15, of a respective semantic context 131 of each of the contracts 2, and a semantic meaning 132 of each of the clause text chunks 121 based at least on a wording of the respective clause text chunk 121 and based on the determined semantic context 131 of the contract 2 with the respective clause text chunk 121; and the encoding (S5), by the data processing system 15, of each of the clause text chunks 121 based on its respective determined semantic meaning 132 into a respective numerical representation 133 within a clause embedding space 141, each numerical representation 133 being a single fixed-size vector 1411 for each of the clause text chunks 121 and the clause embedding space 141 representing a numerical space for encoded semantic meanings 132 of clause text chunks 121; the final step for generating the vector database 14 is the storing (S6) each of the numerical representations 133 in the vector database 14. Then, a question or a task 115 is received (S7), either by manual or machine prompting 196. A user prompt can be a search query or a request for portfolio analysis. Furthermore, performing (S8), by the data processing system 15, an inference with the pre-trained machine learning structure 111 using the vector database 14 is conducted, wherein the input of the pretrained machine learning structure 111 is the received question or task 115; finally, a result of the inference is outputted (S9) by appropriate output signaling.

[0084] Figure 6 (cf. figure 7) shows a projection of embedded clause text chunks 121 into a plane. This projection is achieved using exemplarily the t-Distributed Stochastic Neighbor Embedding (t-SNE) method, a technique that is particularly effective for visualizing high-dimensional datasets. The t-SNE method works by converting the distances between data points in the high-dimensional space 141 into conditional probabilities that represent similarities. The technical object here is to arrange the data in the two-dimensional space such that these similarities are preserved as much as possible. The wordings of the clauses 22 are shaping the observed clusters 14222, where semantically meaningful and / or associated clusters 14222 (like war and terrorism) are found close to each other. A vector database 14 stores a plurality of vector embeddings, wherein each of the vectors 1411 is the result ofan abstraction process from the text modality input to a respective numerical representation 133. Each vector 1411 comprises an array of numbers, wherein each component of such an array represents a learned feature. The numerical representation 133 of a learned feature is in general noninterpretable for a human since the level of abstraction in the clause embedding space 141 has in general no useful correlation with the human mind. The disconnect between how computer store data and how humans understand it is often called the semantic gap. Using vector embeddings that comprise an array of numbers as numerically encoded semantic meanings 132 of clause text chunks 121 cannot be comprehended by a human. To resolve this, a method is provided to make the inherent meaning of embeddings understandable for a human. To gain insight by a human in the result of the embedding processes, another transformation from the high dimensional clause embedding space 141 into a reduced space 142 with a smaller dimension comprehensible by a human is performed. To enable a more intuitive understanding of the data, for instance for visualization generation, a dimensionality reduction technique such as the aforementioned t-SNE can be applied. Other commonly known dimensionality reduction techniques are PCA, and U-MAP, where “PCA" stands for "principle component analysis" and excels by its simplicity and therefore low computational effort, but lacks an acceptable handling of nonlinear data sets. The aforementioned t-SNE method stands for "t-distributed the stochastic neighbor embedding" and is more suitable to accurately lower the dimensionality of nonlinear patterns in datasets while preserving an adequate level of information; the t-SNE method is based on the idea that data samples in the original dataset have a certain distance to each other, which should be reflected as good as possible in its representation of reduced dimensionality. UMAP in contrast to is based on a graph modeling of the original data and stands for "unified manifold approximation and projection". The distance 1412 of every highdimensional sample to another is looked at. The aim is again to obtain a lower dimensional representation that as good as possible matches these distances 1412. Other known methods for dimensionality reduction of data are so called "PaCMAP" and 'TriMap". Such representations in the reduced space can e.g. provide value when combined with an outlier detection, too. For instance, the wording from the main area of war cluster is a typical war exclusion while the wording corresponds to the edges of the war exclusion cluster. While it speaks about war it refers at the same time to voyages and plans pointing to the fact that it belongs to a contract 2 that comes from some aviation specialty line. In figure 6, the contour lines 161 of the plot encirclespecific semantically coherent regions of clauses 22. This means that equivalent clauses 22 whose corresponding clause text chunks 121 are grouped into the same contract term category 122, are located close to each other in the plane of the plot. The distance between different locations representing equivalent clauses of a number of contracts 2, i.e. of clause text chunks 121 belonging to the same contract term category 122, are attributable to the different formulations and / or sentence structures that are used across various contracts 2. Considering this fact, it is possible to visually evaluate and / or monitor the consistency of equivalent clauses 22 across various contracts 2 and to look for outliers 1331 visually, e.g. using a graphical representation 16. Such an outlier detection 197 (T8 / S10)can also be done automatically using the data processing system 15. A graphical plot or representation 16, however, can support the understanding of the outlier detection 1 7 mechanism and provides additional information about clause consistency 197 across contracts 2.

[0085] Once the term extraction 195 is complete and the wordings are projected into vector space 141 , outlier detection 197 becomes feasible. For example, by utilizing a vector space 141, as described above, and given a set of embeddings, {xi, X2 xn}, a centroid 1414 can be defined, c, as the arithmetic mean of the points. The Euclidean distance from each embedding to the centroid, d(xt,c) = - c7)2■ where m is the dimensionality of the embeddings, provides a measure of deviation for each contract's 2 wording from the "norm". Further, for example, employing the Burr distribution as a relation for these distances allows the machine-learning system 1 to estimate the tail behavior of the distribution of deviations, which is essential for detecting outliers 197 effectively. The probability density function of the Burr distribution can then be defined by (x; c, fc) = cfcxe-1 / (l + xe)fc+1, where c > 0 and k > 0 are shape parameters. By fitting the parameters of the Burr distribution to the empirical distribution of Euclidean distances, outliers can be identified as those points whose distance exceeds a certain threshold, typically determined by the tail property of the fitted Burr distribution. This approach has the advantage to be particularly advantageous in the context of large and complex contract 2 wordings, as e.g. typical for (re)insurance contracts, due to its sensitivity to extreme values and its flexibility in capturing the tailheavy distributions often observed in such data. As a result, it has the advantage to provide a robust technical framework for automatically detecting those contracts 2 that require further scrutiny, thereby enhancing the overall efficiency and accuracy of outlier detection 197 in a processing of the machine-learning system 1.In figure 6, clusters 14222 of locations 1413 representing vectors 1411 from the vector database 14 represent the contract term categories 132 in addition to the aforementioned exemplary contract term categories 132 of war, terrorism and nuclear (cf. dotted box in figure 7). Other contract term categories 132 may, for example, be: A territorial scope, a premium, an inspection of records, confidentiality, communicable disease, arbitration, and others.

[0086] In summary, the present invention is a machine-learning system 1 providing contract intelligence automation using contract digitization into machine parseable objects with an automation layer. The machine learning system 1 uses a specifically designed language model, which includes an automated model selection, domain experts' annotation data, fine tuning and training strategy, and validation of the selected machine learning structure 111. The approach provides automatable and scalable domain-specific (e.g. industrial / technical field as (re)insurance technology) analytics based on a pipeline that is economically scalable and actionable at the same time. That combines in a certain sequence (pipeline) fast machine learning (ML) for the digital twin, a scalable transformer-based technology for consistencies and the targeted GenAI deep dives for depth (icicle paradigm). Based on the fact that contract clauses 22 are typically long sentences or paragraph, the machine learning system 1 can adopt a specialized Longformer structure which consistently outperforms other known prior art systems on long document tasks. The inventive long-document-transformer is a drop-in modification to the attention components of the transformer architecture to allow for longer input sequences. Such modification incorporates small local attention spans and increasingly wider attention spans in higher layers. Global attention is used to encode the task, be it classification or question-answering (among others). The limitation of full attention matrices being memory complexity, scaling exponentially with input length, the technically proposed Longformer structure uses sparse attention patterns to ensure only linear scaling (in terms of input tokens) happens, decreasing the memory footprint and compute resources during inference and training.

[0087] While preferred embodiments of the present invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions will now occur to those skilled in the art without departing from the invention. It shouldbe understood that various alternatives to the embodiments of the invention described herein may be employed in practicing the invention. Numerous different combinations of embodiments described herein are possible, and such combinations are considered part of the present disclosure. In addition, all features discussed in connection with any one embodiment herein can be readily adapted for use in other embodiments herein. It is intended that the following claims define the scope of the invention and that methods and structures within the scope of these claims and their equivalents be covered thereby.List of reference numerals

[0088] Electronic machine learning system for contract intelligence automation 10 Receiver

[0089] 101 Data transmission interface

[0090] 11 Machine learning unit

[0091] 111 Machine learning structure

[0092] 1111 Output of machine learning structure

[0093] 112 Training process

[0094] 1121 Training data

[0095] 11211 Training set

[0096] 11212 Validation set

[0097] 11213 Test set

[0098] 113 Tunable Parameters

[0099] 114 Optimization structure

[0100] 1141 Loss function

[0101] 11411 Loss function value

[0102] 115 Predefined questions or tasks

[0103] 116 Ground truth answer

[0104] 117 Predicted answer by the machine learning unit 1171 Predicted identification of a clause text chunk 118 Output signaling of the machine learning unit 11 12 Ground-truth dataset

[0105] 121 Clause text chunk

[0106] 122 Contract term category

[0107] 123 Ground-truth triplet dataset

[0108] 1231 Anchor clause1232 Related clause

[0109] 1233 Unrelated clause

[0110] 124 Similarity condition

[0111] 1241 Nearest neighbor search

[0112] 1242 Computations of cosine similarities

[0113] 125 Wording table

[0114] 1251 Predefined text tokens of word table

[0115] 1252 Text tokens of a contract

[0116] 1253 Detected deviation

[0117] 13 Semantic parameters

[0118] 131 Semantic context of a contract

[0119] 132 Semantic meaning of a clause text chunk

[0120] 133 Numerical representation

[0121] 1331 Outlier of a numerical representation

[0122] 1332 Centroid point of coordinate locations

[0123] 134 Self-attention process

[0124] 1341 Long transformer structure

[0125] 14 Vector database

[0126] 141 Clause embedding space (high dimension)

[0127] 1411 Single fixed-size vector with numerical representation of a determined semantic meaning (i.e. of a clause text chunk)

[0128] 1412 Vector distance

[0129] 1413 Coordinate locations

[0130] 1414 Centroid point

[0131] 1415 Threshold condition

[0132] 1416 Dimension of clause embedding space142 Reduced clause space

[0133] 1421 Dimension of reduced space

[0134] 1422 Coordinate locations in the reduced space

[0135] 14221 Coordinate locations with semantic similarities 14222 Clusters of coordinate locations with semantic similarities

[0136] 1423 Vectors in the reduced space

[0137] 15 Data processing system / computing system

[0138] 151 Persistence storage

[0139] 16 Graphical representation

[0140] 161 Contour lines

[0141] 17 Language core module for natural language processing

[0142] 18 Detector

[0143] 181 Output signaling of the detector

[0144] 19 Automated analytics pipeline structure

[0145] 1 1 Digital twin

[0146] 192 Optical character recognition

[0147] 193 Normalization and extraction of layout information

[0148] 194 Language detection

[0149] 195 Clause 22 extraction

[0150] 1951 Fine tuning of the machine learning structure 111

[0151] 19511 Industrial and / or technical sector / field

[0152] 1952 Validation

[0153] 1953 Document annotation

[0154] 19531 Document annotation module or system 19532 Pre-trained machine learning structure for labelling 196 Prompt engineering197 Consistency automation and / or monitoring (outlier detection etc. (T8 / S10))

[0155] 1 8 Data refinement

[0156] 1981 Down grade trigger / down grade threshold

[0157] 199 Formatted text extraction (e.g. PDF2TXT), if required 2 Contract

[0158] 21 Text

[0159] 22 Clause

[0160] 221 Sentence

[0161] 222 Word / Token

[0162] 23 Contract language

[0163] T1 Receiving and / or digitizing plurality of contracts

[0164] T2 Storing of contracts to persistence storage

[0165] T3 Creating ground-truth dataset

[0166] T4 Determining semantic context of contracts

[0167] T5 Encoding of clause text chunks

[0168] T6 Storing numerical representations in vector database

[0169] T7 Performing training process of machine learning unit

[0170] T8 Detecting a statistical outlier in numerical representations (consistency check)

[0171] 51 Receiving and / or digitizing plurality of contracts

[0172] 52 Storing of contracts to persistence storage

[0173] 53 Identifying clauses in the contracts as clause text chunks

[0174] 54 Determining semantic context of contracts and a semantic meaning of clause text chunks

[0175] S5 Encoding of clause text chunks56 Storing numerical representations in vector database

[0176] 57 Receiving question or a task

[0177] 58 Performing inference with the machine learning structure

[0178] 59 Outputting or transmitting output signaling as result of the inference

[0179] S10 Detecting a statistical outlier in numerical representations (consistency check)

Claims

45Claims1. Automated method for a machine learning system ( 1 ) for contract intelligence automation and automated text analysis with sparse attention pattern providing low memory footprint and compute resource consumption, the automated method comprising:receiving and / or digitizing (Tl) a plurality of contracts (2), each of the contracts (2) comprising text data (21) with a plurality of clauses (22), each of the clauses (22) comprising at least one sentence (221 ) with a plurality of words (222) and at least one of the clauses (22) comprising at least two sentences (221), wherein wordings and / or sentence structures of equivalent clauses (21) vary across at least two contracts (2);reading (T2) said text (21) of received contracts (2) into a persistence storage (151) of a data processing system (15);creating (T3) a ground-truth dataset (12) by identifying clauses (22) in said text (21) of each contract (2) as clause text chunks (121), identifying equivalent clauses (22) across contracts (2) and assigning each of the identified clause text chunks (121) an associated contract term category (122), with clause text chunks (121) corresponding to equivalent clauses (22) across contracts (2) being assigned the same contract term category (122);determining (T4), by the data processing system (15), a respective semantic context (13 / 131) of each of the contracts (2), and a semantic meaning (13 / 132) of each of the clause text chunks (121) based at least on a wording of the respective clause text chunk (121) and based on the determined semantic context (131) of the contract (2) with the respective clause text chunk (121);encoding (T5), by the data processing system (15), each of the clause text chunks (121) based on its respective determined semantic meaning (132) into a respective numerical or alphanumerical representation (133) within a clause embedding space (141), each numerical or alphanumerical representation (133) being46a single fixed-size vector (1411) and the clause embedding space (141) representing a numerical space for encoded semantic meanings (132) of clause text chunks (121);storing (T6) each of the numerical or alphanumerical representations (1411) in a vector database (14) of the persistence storage (151); andperforming (T7), by using the data processing system (15), a training process (112) of a machine learning unit (11) with a predefined machine learning structure (111) and with a plurality of tunable parameters (113), the machine learning unit (11) having access to the vector database (14), the training process (112) automatically adjusting values of said tunable parameters (113) using an optimization structure (114) optimizing a loss function value (11411) of a loss function (1141), the loss function (1141) being based on a difference between a ground truth answer (116) and an answer (177) predicted by an inference with the machine learning unit (11) in response to a predefined question (115), wherein the ground truth answer (116) is based on the ground-truth dataset (12).

2. Automated method according to claim 1, wherein the ground truth answer is a correct identification of a clause (22) from the plurality of contracts (2) based on a predefined question (115) and wherein the predicted answer (117) is a predicted identification (1171) of a clause text chunk (121), the predicted identification (1171) being obtained by an inference with the machine learning unit (11).

3. Automated method according to one of the claims 1 to 2, wherein for a predefined question (115) a plurality of paraphrased question-variants with one corresponding ground truth answer (116) is used for training.

4. Automated method according to one of the claims 1 to 3, wherein text of at least two of the contracts (2) are in different languages (23), wherein the machine learning unit (11) is trained to be language-agnostic with regard to contract (2) languages (23).

5. Automated method according to one of the claims 1 to 4, wherein for the training process (112) of the machine learning unit (11) a ground-truth triplet dataset (123) comprising an anchor clause (1231), a related clause (1232), and an47unrelated clause (1233) is used, the anchor clause (1231) and the related clause (1232) being equivalent clauses (22) and their corresponding clause text chunks (121) belonging to the same contract term category (122), the unrelated clause (1233) not being equivalent to the anchor clause (1231) and its corresponding clause text chunk (121) not belonging to the contract term category (122) of the anchor clause text chunk (121); wherein the training process (112) minimizes a vector distance (1412) between the numerical or alphanumerical representations (133) of i) the clause text chunk (121) corresponding with the anchor clause (1231) and ii) the clause text chunk (121) corresponding to the related clause (1232), and maximizes a vector distance (1412) between the numerical or alphanumerical representations (1411) of a) the clause text chunk (121) corresponding to the anchor clause (1231) and b) the clause text chunk (121) corresponding to the unrelated clause (1233).

6. Automated method for automated for contract intelligence automation and automated text analysis with sparse attention pattern providing low memory footprint and compute resource consumption, comprising:- receiving (SI) a plurality of contracts (2), each of the contracts (2) comprising text data (21) with a plurality of clauses (22), each of the clauses (22) comprising at least one sentence (211) with a plurality of words (222) and at least one of the clauses (22) comprising at least two sentences (221), wherein wordings and / or sentence structures of equivalent clauses (21) vary across at least two contracts (2);- storing (S2) said text (21 ) of received contracts (2) into a persistence storage (151) of a data processing system (15), the processing system (15) comprising a machine learning structure (111) specifically pre-trained for identifying clauses (22) in text of each contract (2) as clause text chunks (121) and for identifying equivalent clauses (22) across contracts (2) and assigning each of the identified clause text chunks (121) an associated contract term category (122), with clause text chunks (121) corresponding to equivalent clauses (22) across contracts (2) assigned the same contract term category (122);- identifying (S3), by the machine learning structure (111), clauses (22) in each of the contracts (2) as clause text chunks (121) and assigning each of the identified clause text chunks (121) an associated contract term category (122);- determining (S4), by the data processing system (15), a respective semantic context (131) of each of the contracts (2), and a semantic meaning (132) of each of the clause text chunks (121) based at least on a wording of the respective clause text chunk (121) and based on the determined semantic context (131) of the contract (2) with the respective clause text chunk (131);- encoding (S5), by the data processing system (15), each of the clause text chunks (121) based on its respective determined semantic meaning (132) into a respective numerical representation (133) within a clause embedding space (141), each numerical representation (133) being a single fixed-size vector (141) for each of the clause text chunks (121) and the clause embedding space (141) representing a numerical space for encoded semantic meanings (132) of clause text chunks (121);- storing (S6) each of the numerical representations in a vector database (14) of the persistence storage (151);- receiving or capturing (S7) a question or a task (1142);- performing (S8), by the data processing system (15), inference with the pre-trained machine learning structure (111) using the vector database (14), wherein the input of the pre-trained machine learning system (111) is the received question or task (115); and- outputting (S9) a result (1111) of the inference by generating an output signaling.

7. Automated method according to claim 6, wherein the data processing system (15) comprises a storage (151) storing numerical representations (133) of predefined contract term categories (122), wherein the numerical representations (133) of clause text chunks (121) are compared with the numerical representations (133) of the predefined contract term categories (122) for similarity, wherein the clause text chunks (133) are assigned to the contract term categories (122) according to a similarity condition (124).

8. Automated method according to claim 7, wherein the similarity condition (124) is evaluated by a nearest neighbor search (1241) or by computations of cosine similarities (1242).

9. Automated method according to one of the claims 6 to 8, comprising: Performing an automatic consistency check (S10) by detecting a statistical outlier (1331) in numerical representations (133) of clause text chunks (121) within a contract term category (122) across contracts (2), and outputting an output signaling indicating a detected outlier (1331).

10. Automated method according to claim 9, wherein the output signaling about the detected outlier (1331) comprises the cause, according to which a wording or a sentence construction or both of a clause text chunk (121) are categorized as outlier (1331).

11. Automated method according to one of the claims 6 to 10, wherein the automatic consistency check (S10) combines detecting a statistical outlier (1331) in numerical representations (133) of clause text chunks (3) within a contract term category (122) across contracts (2) and applying a wording table (125) comprising predefined text tokens (1251), the predefined text tokens (1251) being compared with tokens (1252) extracted from text (21) of the contracts (2) for detecting deviations (1253) between the tokens from the text (21) of the contracts (2) and the predefined text tokens (1251) from the wording table (125).

12. Automated method according to one of the claims 6 to 11, wherein the statistical outliers (1331) are detected by: Generating, by the data processing system (15), a centroid point (144) of coordinate locations (1413), to which vectors (7) representing clause text chunks (132) of the plurality of contracts (2) are pointing in the clause embedding space (141), by generating distances (1412) of said coordinate locations (1413) to the centroid point (1414), and comparing each of the generated distances (1412) with a predefined threshold condition (1415).

13. Automated method according to one of the claims 6 to 12, comprising: Performing, by using the data processing system (15), a dimensionality reduction of the vectors (1411) from the vector database (14) to a reduced space (142), the reducedspace (142) having a dimension (1421) lower than the dimension (1416) of the clause embedding space (141) and being bigger than 0 and lower than 5, and signaling and / or visualizing coordinate locations (1413), to which vectors (1411) in the reduced space (142)are pointing.

14. Automated method according to claim 13, wherein a graphical representation (16) of coordinate locations ( 1422) , to which vectors ( 1423) in the reduced space (1421) are pointing, is generated, the graphical representation (16) containing contour lines (161) enclosing clusters (14222) of coordinate locations (1422) with semantic similarities (14221).

15. Electronic machine learning system (1) for contract intelligence automation and automated text analysis with sparse attention pattern providing a system architecture using low memory footprint and compute resource consumption, comprising- a receiver (10), that receives a plurality of contracts (2), each of the contracts (2) comprising text data (21) with a plurality of clauses (22), each of the clauses (22) comprising at least one sentence (221 ) with a plurality of words (222) and at least one of the clauses (22) comprising at least two sentences (221), wherein wordings (222) and / or sentence (221) structures of equivalent clauses (22) vary across at least two contracts (2);- a data processing system (15) capturing said text data (21) of received contracts (2), the data processing system (15) comprising a machine learning system (111) specifically pre-trained for identifying clauses (22) in the text data (21) of each contract (2) as clause text chunks (121) and for identifying equivalent clauses (22) across contracts (2) and for assigning each of the identified clause text chunks (121) an associated contract term category (122), with clause text chunks (121) corresponding to equivalent clauses (22) across contracts (2) assigned the same contract term category (122),wherein the data processing system (15) is configured to identify clauses (22) in each of the contracts (2) as clause text chunks (121) and to assign each of the identified clause text chunks (121) an associated contract term category (122),51wherein the data processing system (15) is configured to determine a respective semantic context (13 / 131) of each of the contracts (2) and a semantic meaning (132) of each of the clause text chunks (121) based at least on a wording of the respective clause text chunk (121) and based on the determined semantic context (131) of the contract (2) with the respective clause text chunk (121),wherein the data processing system (15) is configured to encode each of the clause text chunks (131) based on its respective determined semantic meaning (132) into a respective numerical representation (133) within a clause embedding space (141), each numerical representation (133) being a single fixed-size vector (1411) for each of the clause text chunks (121) and the clause embedding space (141) representing a numerical space for encoded semantic meanings (132) of clause text chunks (121), andwherein the data processing system (15) is configured to store each of the numerical representations (133) in a vector database (14);- a detector ( 18) for detecting, as automatic wording consistency check, a statistical outlier (1331) in numerical representations (133) of clause text chunks (3) within a contract term category (122) across contracts (2) by performing an inference with the pre-trained machine learning structure (111) using the vector database (14), the detector (18) outputting an output signaling indicating a detected outlier (1331).

16. A electronic machine learning system ( 1 ) according to claim 15, characterized in that the electronic machine learning system (1) comprises an scalable and actionable, automated analytics pipeline structure (19), wherein the contracts (2) are converted into a digital twin (1 1) building a precise, digital counter-part to the respective contract (2), each clause (22) to the digital twin (191) interpretable and manipulatable by the electronic machine learning system (1).

17. An electronic machine learning system (1) according to one of the claims 15 or 16, characterized in that the electronic machine learning system (1) combines in the pipeline structure (19) a sequence of fast machine learning (ML) for a scalable transformer-based digital twin (191) providing consistencies with a targeted GenAI deep dives for depth.

Images