System and method for generation of user advisories using generative ai in contract analysis

Abstract

A computer-implemented method and system for analyzing contract documents using multi-level prompting and retrieval-augmented generation (RAG). The method includes training a knowledge base with contract clauses, legal terminology, and industry-specific information. Upon receiving an input file containing contract-related text, including a main contract, statement of work (SOW), and addendum, a multi-level prompt generator creates a first-level prompt comprising abbreviated pages. A retrieval module obtains relevant information from the knowledge base using the first-level prompt, enabling a language model to generate an initial analysis. Based on this analysis, a second-level prompt retrieves additional contract-specific information. The language model then generates a detailed contract analysis and tailored user questions, which are classified into relevant categories. The method further includes performing risk analysis on the classified questions and contract analysis, creating an abbreviated contract form summarizing key terms, and generating an aggregated summary of the SOW and addendums. The system presents the analysis results, classified questions, risk assessment, abbreviated contract form, and aggregated summary through a user interface. The method improves contract analysis efficiency while maintaining comprehensiveness and accuracy in identifying contractual obligations and potential risks.

Description

PCT APPLICATIONFORSYSTEM AND METHOD FOR GENERATION OF USER ADVISORIES USING GENERATIVE Al IN CONTRACT ANALYSISInventors:Praful Saklani1 Sansome Street, Suite 3500, San Francisco, CA 94104 A citizen of the United StatesPedram Abrari1 Sansome Street, Suite 3500, San Francisco, CA 94104 A dual citizen of the United States and IranAchint Agarwal1 Sansome Street, Suite 3500, San Francisco, CA 94104 A citizen of the United StatesPrepared by: Mark StignaniUSPTO Registration No.: 36378TITLE OF INVENTIONSYSTEM AND METHOD FOR GENERATION OF USER ADVISORIES USINGGENERATIVE Al IN CONTRACT ANALYSISCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to provisional application number 63 / 617505, filed January 4, 2024, provisional application number 63 / 592992, filed October 25, 2023 and provisional application number 63 / 593004, filed October 25, 2023, the entire contents of which are incorporated herein by reference.BACKGROUND

[0002] Generative Al models lack an inherent grasp of control flows, rules, and logical reasoning, relying instead on statistical patterns from training data. Despite advances in research, enabling generative models to have explicit reasoning and rule-following capabilities remains a challenge.

[0003] Techniques such as chain-of-thought prompting only offer a framework for logical sequence of inferences rather than a complete solution.

[0004] While reinforcement learning allows models to adopt basic rule-based behaviors, they still struggle with intricate control flows.

[0005] Predominant generative Al models like GPT can produce fluent outputs but often falter in logical consistency, factual precision, and compliance with intricate rules.

[0006] The reliability of these models diminishes over extensive chains of inference due to the absence of an explicit rule model.

[0007] Alternative approaches using GenAI focusing on structured knowledge and reasoning, such as semantic parsers and theorem provers, offer more accuracy in modeling complex rules but may not be as fluent or encompassing in knowledge.

[0008] Training a Large Language Model (LLM) to understand contracts, addendums, and associated legal documents is a complex endeavor. One significant challenge is the inherent complexity and ambiguity in legal language. Legal documents often contain phrases that areintentionally broad to cover various scenarios or are couched in terms that have evolved specific meanings over centuries of legal practice. For a model, discerning the nuances and specific intentions behind such phrasings can be difficult.

[0009] Additionally, contracts often reference external laws, regulations, and previously agreed-upon terms. Without a comprehensive contextual understanding, an LLM might misinterpret a clause because it doesn't understand an external reference or how it interacts with the rest of the document. This interconnectedness of legal documents makes them particularly tricky for models that operate primarily on the immediate context they're given.

[0010] Moreover, legal language often evolves based on case law. That is, the interpretation of a phrase in a contract might depend on legal precedents set by courts in prior cases. Without access to this vast body of evolving case law, or without understanding the implications of a particular precedent, an LLM might not provide an accurate interpretation of a contract's terms.

[0011] Cultural and jurisdictional differences add another layer of complexity. Contracts drafted in different countries or legal systems might use similar terms but mean different things, based on the local legal tradition and practice. Training a model to understand these distinctions and provide accurate interpretations for diverse legal systems is challenging.

[0012] Lastly, the stakes in interpreting legal documents are incredibly high. Misinterpretations can lead to costly litigation, financial losses, or breaches of regulatory compliance. While LLMs can assist in understanding and drafting legal documents, relying solely on them without human oversight can pose significant risk The inventors have identified the foul problems with managing Context Windows in LLM

[0013] The first problem identified is a limited ability to process long sequences. LLMs can only process a certain number of tokens at a time, which limits their ability to understand and respond to complex or long-form text.

[0014] Another problem identified is difficulty tracking long-range dependencies. LLMs can struggle to track long-range dependencies in text, which can lead to errors in tasks such as question answering and summarization.

[0015] Another problem identified is increased risk of hallucination. LLMs can hallucinate, or generate false or misleading information, especially when given a large context window.

[0016] Another problem identified is increased computational cost. Training and deploying LLMs with large context windows are computationally expensive.

[0017] Another problem identified is reduced model portability. LLMs with large context windows are less portable and can be difficult to deploy on edge devices.

[0018] Another problem identified is increased risk of overfitting. LLMs with large context windows are more likely to overfit to the training data, which can lead to poor performance on unseen data.

[0019] Another problem identified is difficulty adapting to new domains. LLMs with large context windows can be difficult to adapt to new domains, such as medical or legal text.

[0020] Another problem identified is reduced interpretability. LLMs with large context windows are less interpretable, making it difficult to understand why they make certain predictions.

[0021] Another problem identified is increased difficulty in debugging. Debugging LLMs with large context windows is more difficult, as it can be challenging to identify the source of errors.

[0022] Another problem identified is reduced model efficiency. LLMs with large context windows are less efficient, meaning they can take longer to generate responses.

[0023] Another problem identified is increased memory requirements. Training and deploying LLMs with large context windows require more memory.

[0024] Another problem identified is reduced fairness. LLMs with large context windows can be biased, reflecting the biases in the training data.

[0025] Another problem identified is increased security risks. LLMs with large context windows can be more vulnerable to adversarial attacks.FIELD OF THE INVENTION

[0026] The field of the solution relates to computer implemented methods for the automated analysis or use of data, including use of a large language model (LLM), and to related computer implemented methods and systems as used to review and analyze large copra of contracts and other associated documents throughout the life span of the contracts.Introduction to LLMs:

[0027] LLMs are a subset of machine learning models specifically designed for understanding and generating human language. They are trained on vast amounts of text data, enabling them to perform tasks such as text generation, translation, summarization, and question-answering.LLMs are built using a type of architecture known as transformers, which have proved exceptionally proficient in handling language-based tasks.How LLMs Learn:

[0028] Training Data: LLMs begin by ingesting a plethora of text data. With this solution, the LLM is used to ingest contracts and their related documents.

[0029] Patterns and Probabilities: The model learns by identifying patterns in the data. For instance, it might learn that the word "whereas" often indicates preamble in contract text. Over time, the model refines its understanding, grasping more nuanced linguistic structures and contexts.

[0030] Parameters: An LLM has over a million adjusted parameters, which are akin to adjustable dials. During training, these parameters are tweaked to minimize the difference between the model's predictions and the actual data. Once trained, when given a new piece of text (a prompt), the LLM can generate an outcome of that text based on the patterns it has recognized during its training.The Context Window:

[0031] One of the challenges with LLMs is memory management. Given the vastness of language and the complexity of human context, a model can't simply remember everything it has ever seen. Instead, it operates within what's termed a "context window."

[0032] The context window is essentially the immediate span of tokens (words, characters, or sub words) the model considers when generating a response or understanding context. For example, GPT-4, one of OpenAI's LLM variants has a context window of ranging from 2048 32,000 tokens. The context window restricts the model's immediate "lookback." If a piece of information or context appears outside this window, the model won't directly consider it for its immediate response. So, handling long sets of texts is a problem If you feed an LLM a text that exceeds its context window, it will only consider the most recent tokens within that window. For instance, if you input a 3000-token text into a model with a 2048-token window, the earliest 952 tokens won't be directly considered in the model's response. Extending the context window is computationally complex and most LLM models restrict to maintain contextual relevance. Them context window ensures that the model gives more weight to immediate and relevant context. If you're asking the model about a specific topic like termination terms, it willfocus on the surrounding information, ensuring that the generated output is pertinent to the current contract or set of contracts.

[0033] Computational efficiency: as mentioned previously, confining the model's focus to a limited number of tokens, the context window ensures computational efficiency. Without this restriction, the task of real-time text generation or comprehension might become computationally prohibitive.

[0034] The present solution uses a plurality of methods and processes to overcome this context window issue. An aspect of the solution: A method is described wherein a processing system uses contract data, structured in a machine-readable language, to furnish new contractual analysis using an LLM to interpret and analyze contracts, addendums and other related documents attached by reference or other means.

[0035] Another Aspect of the solution: A method involves relaying contract data from one system via an LLM to a system that employs structured, machine-readable data. This system then evaluates and refines the LLM's output for an improved contract intelligence outcome experience.

[0036] Another Aspect of the solution: An LLM produces continuation data using feedback from a processing system that employs structured data. Here, the LLM uses this feedback as supplementally generated prompts to narrow and refine for superior text output. This method amplifies the quality of the processing system's output.

[0037] Another aspect of the solution. The solution can summarize state of the commercial relationship between two entities based on contractual agreements signed to date regardless of the contract origin. By using natural language risk assessment and negotiation playbook to assess commercial relationship risks associated with effective commercial terms, and to guide negotiation in order to mitigate them. The solution leverages historically signed contracts between parties to help the parties negotiate based on either paper, provide neutral risk context, draft agreements in either parties style or autogenerate relevant amendments, statements of work, and orders and allow assessments of critical points for who is required to sign.

[0038] Another aspect of the solution will be to have chatbot interactive sessions allowing user to ask questions about the commercial relationship and its agreements.BRIEF DESCRIPTION OF THE DRAWINGS

[0039] To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element is first introduced.

[0040] FIG. 1 illustrates a flowchart of the hardware of subject matter in accordance with one solution variants.

[0041] FIG. 2 illustrates a flow chart of the content and Al flow matter in accordance with one solution variant.

[0042] FIG. 3 illustrates a context window marshalling flow accordance with one solution variant.

[0043] FIG. 4 illustrates a contract object model in accordance with one solution variant.

[0044] FIG. 5 illustrates a contract playbook in accordance with one solution variant.

[0045] FIG. 6 illustrates a historical analysis in accordance with one solution variant.

[0046] FIG. 7 illustrates contract guidelines example in accordance with one solution variant.

[0047] FIG. 8 illustrates an example user interface and user experience (UXI) in accordance with one solution variant.DETAILED DESCRIPTION

[0048] The disclosed technology encompasses systems, methodologies, and / or computer program commodities at varying degrees of technical integration. Such a computer program commodity may comprise a machine-readable storage medium (or multiple mediums) bearing machine-executable instructions to prompt a processor to execute components of the specified technology.

[0049] This machine-readable medium is a physical entity capable of maintaining and storing instructions to be utilized by an instruction execution apparatus. The medium could be, for example, but not restricted to, electronic, magnetic, optical, electromagnetic, semiconductor storage devices, or a fusion of these. A non-limiting list of specific instances of the machine- readable medium includes portable computer diskettes, hard drives, RAM, ROM, EPROM or Flash memory, SRAM, CD-ROMs, DVDs, memory sticks, floppy disks, and mechanical devices like punch-cards or tangible structures with instructions. It should be clarified that theaforementioned medium does not consider transitory signals in isolation, like free-propagating electromagnetic waves or electrical signals over wires.

[0050] The machine-executable instructions detailed can be transferred to diverse computational devices from the machine-readable medium or an external computer or storage via networks like the Internet, LANs, WANs, or wireless networks. Such networks may integrate copper or optical fibers, wireless transmission mechanisms, routers, firewalls, switches, gateway computers, and edge servers. Within each computational device, a network interface or adapter fetches the instructions from the network, forwarding them for retention in the device's machine-readable medium.

[0051] Instructions facilitating operations of this technology might be encoded as assembler instructions, ISA instructions, machine codes, microcodes, firmware instructions, circuit configuration data, or code (both source and object) in diverse programming languages. Examples include but aren't restricted to object-oriented languages like Python, Java, C++, and procedural ones like the "C" language. These instructions might operate wholly on a local computer, partly on local and remote computers, or entirely remotely. Remote computers can be linked via networks, inclusive of the Internet via ISPs. In certain cases, hardware such as FPGAs or PLAs could employ the instructions, utilizing their state data to modify the hardware to actualize facets of the technology.

[0052] The technology's facets are expounded with reference to flowcharts and block diagrams of methods, systems, and computer program products per its solution variant s. Each block in these can be realized via machine-executable instructions. These instructions could be presented to a processor in general-purpose computers, specialized computers, or other programmable data apparatuses, crafting a machine that institutes the functions denoted in the diagrams. Furthermore, these instructions could be conserved within a machine-readable medium directing device to operate in a specific fashion. The instructions could also be loaded onto a computer or device to prompt a sequence of tasks producing a computer-driven process.

[0053] The depicted flowcharts and diagrams exhibit potential system, method, and product architectures and functionalities per the technology's solution variant s. It's essential to note that these blocks, or their combinations, can be realized by hardware systems specifically designed for those tasks or combinations of hardware and machine instructions.

[0054] While multiple solution variants have been detailed, skilled individuals will recognize that modifications can be made without diverging from the broader aspects of the technology.The term "user" here pertains to any individual or entity interacting with the described contract analysis and generation system. "User device" signifies any computational apparatus, from PCs to mobile devices like smartphones, laptops, wearables, and more, employed to access the system. The term "entity" encompasses organizations or user groups engaging with the system, such as businesses or companies either operating or accessing the system.

[0055] Displayed in the FIG. 1, one or more user devices are operationally linked through a network to one or more entity systems. Consequently, the user devices can transmit and retrieve data from the entity systems. In the depicted solution variant, several user devices might connect with the entity systems via the network. The system comprises a User Device 102, a Network 104, a SaaS Contract Platform 106, a Data Repository 108, an Al Module 110, a Training Data Repository 112, an Ingested Contracts 114, a Risk Recommendation 116, and a Contract Draft Generator 118.

[0056] FIG. l provides just one solution variant of a computer system. In alternative configurations, certain systems, devices, or servers might be amalgamated into a single unit, or be divided into multiple units. It should be inferred that the various servers, systems, and devices can be amalgamated in different variants while retaining analogous functionality.

[0057] For example, the network might be a dedicated distributive network which identifies specific network triggers. Moreover, the network could be a Global Area Network (GAN) like the Internet, a Wide Area Network (WAN), a Local Area Network (LAN), or any fusion of such networks. This network can facilitate wired, wireless, or combined communications. Additionally, the network might encompass a peer-to-peer communication system.

[0058] Both the entity systems and user devices possess data storage and processing capabilities, holding data pertinent to the system environment, including data devised by various system applications. The contract analysis and generation system's diverse applications and their associated data may reside within the memory of the entity systems and user devices. In some versions, users might engage with the contract analysis and generation system via a user interface. This interface is designed for simple contract generation.

[0059] FIG. 1 details singular modules and repositories for simplicity, but the solution and its variants comprise complex combinations of repositories, databases, and Al modules. These components work in concert to provide a comprehensive contract analysis and management system.

[0060] The user interface is designed for efficiency, offering single-click contract analysis. This streamlined approach allows users to quickly initiate comprehensive contract reviews without navigating through multiple menus or options.

[0061] A data repository houses an extensive collection of contract clauses. This repository serves a critical function by verifying the inclusion of all vital clauses in a contract. It acts as a safeguard, ensuring no essential elements are overlooked during contract creation or review.

[0062] One of the Al modules employs a model trained on a vast array of queries. This model is adept at determining whether human-input questions have default responses. When applicable, it intelligently inserts these pre-determined answers while providing contextual output, ensuring relevance and accuracy in the responses.

[0063] Another Al module is built around a database of keywords and links to various regulations, (e.g., Federal Acquisition Regulations (FAR). This module is fine-tuned to export unique and necessary contract wording that aligns with relevant regulations, ensuring compliance and precision in contract language.

[0064] Another Al model is trained to extract contract terms using a curated list of terms. This capability allows for rapid identification and isolation of key contractual elements, streamlining the review and drafting processes.

[0065] Another repository employs Named-Entity Recognition (NER) technology. Its primary function is to identify and exclude keywords that aren't considered variables in a contract from the working text. This refinement process helps focus attention on the most relevant and adaptable parts of the contract.

[0066] Another Al model in the solution variant is designed to classify anticipated contract issues into distinct categories or classes. This classification system aids in organizing and prioritizing potential contract concerns, allowing for more efficient problem-solving and risk management.

[0067] Another Al module handles comparative analysis, by conducting various types of comparisons, including:1. Comparing different contract templates2. Analyzing contracts against established templates3. Examining distinct versions of the same contract

[0068] Moreover, this Al module can dissect party details, transforming them into structured lists and contract templates.

[0069] Another Al module would be a multilevel prompt generator.

[0070] The contract lifecycle management module is engineered to oversee every stage of a contract's lifecycle. This includes creation, negotiation, approval, execution, and post-execution management, ensuring approach to contract administration.

[0071] In more advanced implementations, the system can integrate with external Enterprise Resource Planning (ERP) and Customer Relationship Management (CRM) systems. These systems serve as rich data repositories, retaining crucial information about users, entities, and their associated devices. This integration allows for a more comprehensive view of the contractual landscape within an organization's broader operational context.

[0072] Certain modules within the system, particularly the party management module, can access these ERP and CRM systems using established Application Programming Interface (API) techniques. This interoperability ensures smooth data flow and consistency across different organizational systems.

[0073] It's worth noting that these ERP and CRM systems might be hosted on various platforms - they could reside on centralized entity systems or be distributed across individual user devices, offering flexibility in deployment and data management strategies.

[0074] Additionally, the SaaS Contract Platform 106 and the Al Module 110 work together to add and enhance the solution response using Retrieval- Augmented Generation (RAG) in the following stages of contract lifecycles. During indexing, contract documents are preprocessed, chunked, and indexed in a vector database using dense embeddings. During data retrieval a query is received, relevant sections from the indexed contracts arc retrieved based on semantic similarity. The retrieved contract sections are used to augment the input to a large language model (LLM) and the LLM in the SaaS Contract Platform 106 generates responses to queries, grounded in the specific language and context of the retrieved contract sections.

[0075] An example of using this SaaS Contract Platform 106 would perform as follows: The system begins by maintaining a comprehensive knowledge base that encompasses various contract clauses, legal terminology, and industry-specific information that serves as the foundation for analysis. When a user initiates the process, they can upload multiple contract- related documents, including the main contract, statement of work (SOW), and any relevant addendums through a user interface. Within the Al Module 110, the multi-level promptgenerator then creates an initial first-level prompt based on the uploaded documents, which includes abbreviated versions of key pages to ensure efficient processing. Using this prompt, the retrieval module searches the knowledge base to gather relevant information for preliminary analysis. The language model, specifically trained for contract analysis, generates an initial assessment of the documents using this retrieved information. Based on this preliminary analysis, the system creates a more focused second-level prompt to gather additional specific contract-related details, leading to another round of information retrieval. The language model then processes this comprehensive set of information to produce a detailed contract analysis and generates tailored user questions relevant to the specific contract scenario. These questions are systematically classified into appropriate categories for better organization and understanding. The system performs a thorough risk analysis based on both the classified questions and the contract analysis results. To make the information more accessible, the system creates an abbreviated version of the contract that highlights key terms, conditions, and obligations from all documents. Finally, the system generates a comprehensive summary of the SOW and addendums, emphasizing critical deliverables, timelines, and responsibilities, providing users with a clear understanding of their contractual obligations and requirements.

[0076] Another variant of the solution comprises the following aspects. To effectively leverage contracts as enterprise assets through artificial intelligence, a systematic preprocessing approach is required that transforms raw contract documents into structured, analysis-ready data assets. This transformation process involves multiple stages: first, contracts must be cleansed and organized, including deduplication and family relationship establishment across document sets; second, provision models must be developed that capture key business use cases and requirements; third, these provision models must be decomposed into structured hierarchies of provisions, clauses, and terms that map to specific business problems and objectives. Only after this preparation can generative Al effectively analyze contract paragraphs by mapping them to the appropriate clause-level components within the provision models, enabling accurate playbook analysis, risk assessment, and other analytical functions. This structured approach ensures that Al systems can maintain contextual accuracy while processing complex contractual relationships and obligations at enterprise scale.

[0077] An example of how this variant of the solution works is as follows: To ingest and analyze a contract using Al, SaaS Contract Platform 106 begins with uploading the raw contract document via the Ingested Contracts 114, which is cleansed by removing duplicatesand identifying relationships with related documents. Next, a provision model via Al Module 110 is developed based on key business use cases and requirements, such as compliance, liability, or service level terms. The contract is then decomposed into a structured hierarchy, where provisions (such as payment obligations or confidentiality) are broken down into clauses that define specific terms, conditions, and obligations. The Al Module 110 maps these clauses to corresponding elements in the provision model, aligning each part of the contract with relevant business objectives. Once mapped, the system performs an automated playbook analysis, identifying risks, assessing deviations from company standards, and highlighting areas that may require negotiation. Finally, the Al Modules 110 generates risk and compliance reports, providing a comprehensive analysis of the contract, including key terms, risks, and recommendations for negotiation or modifications. This process enables Al to conduct accurate and context-aware analysis of complex contracts, ensuring alignment with business goals.

[0078] FIG. 2 illustrates a high-level complex control flow showing how a variant of the solution would work in assessing contract risk. According to an example of the solution, the method includes analyzing an Agreement Collection 216 comprising a Master Services Agreement MSA 202 and Amendment 204. The Effective Terms 220 are extracted comprising Assignment 206, Payment Term 208, Price Increase 210, and Other Terms 212. The GENAI 224 checks Check Rule A.2.1 226, a Check Rule A.2.2 228, a Check Rule A.2.3 230, a Check Rule Pl 232, and a Check Rule PT. l 234 against the extracted Effective Terms 220, example categorizes the Agreement Collection 216 into three responses High Risk (GC / Deal Desk ) Approval Needed 214, Medium Risk (Deal Desk Approval) 218, and Low Risk (No Approvals 222 for review and clearance for the Effective Terms 220

[0079] Although the example routine depicts a particular sequence of operations, the sequence may be altered without departing from the scope of the present disclosure. For example, some of the operations depicted may be performed in parallel or in a different sequence that does not materially affect the function of the routine. In other examples, different components of an example device or system that implements the routine may perform functions at substantially the same time or in a specific sequence. As another example of the solution variant, a set of hardware and software would ingest a set of contracts, parsing their constituent parts, supplemental documents like amendments, statements of work, appendices, codices and other correspondence and associated communications. The parser breaks the documents into effectiveterms, (e.g. Assignments, Payment terms, Pricing, other key items) These terms are then fed into a set of generative Al rules that are associated with assessing the risks related to various states or effective terms in a particular contracts and outputs a risk rating for the particular contract or contract effective term that is then routed to an appropriate authority for review. (e.g. General Counsel, Deal Desk or normal no / low risk outcome. The platform then generates three types of prompts.1. Summarize Contracts prompt - this prompt will create a summary of the lineage of contracts for a business relationship, including the original agreement and any follow-on amendments.2. Summarize Orders prompt - summarizes all the orders executed for a business relationship like products sold / purchased renewal dates, etc.3. Summarize Playbook prompt - summarizes the organization’s negotiation playbook converted from raw form into defined rules by SI.

[0080] The solution then feeds the contracts into the LLM in a fashion that minimizes hallucination or improper context to be drawn from the representative parts of the contracts being analyzed.

[0081] Displayed in the FIG. 3., the solution variant that uses context window marshalling process to avoid accuracy issues using a First in First out Queueing process. FIG. 3 illustrates an example routine for summarization of an Aggregated Contract Documents 314. Although the example routine depicts a particular sequence of operations, the sequence may be altered without departing from the scope of the present disclosure. For example, some of the operations depicted may be performed in parallel or in a different sequence that does not materially affect the function of the routine. In other examples, different components of an example device or system that implements the routine may perform functions at substantially the same time or in a specific sequence. The process of the solution variant displayed comprises: the Aggregated Contract Documents 314 comprising a Master Service Agreement MSA 302 and is amending documents Amendment 1 304, Amendment 2 308 and other order documents Order 1 306, Order 2 312, and any other documents that would further comprise context or information related to the Aggregated Contract Documents 314 is fed to a Summarization Step 316 to create abbreviated versions of each document or page from Aggregated Contract Documents 314. The Abbreviated Pages 318 and the Next Abbreviated Pages 322 are fed into the LLM Prompt 330 to generate an LLM Response for MSA and Amendment summary 332. The reason for creatingAbbreviated Pages 318 is to keep the LLM Prompt 330 size to fit into the LLM Context Window (FIFO Queue) s 320 and to eliminate hallucinations caused by overflowing the LLM Context Window (FIFO Queue) 320 which would not normally fit a very large Aggregated Contract Documents 314. Presently, commercial LLMs had context windows ranging from about 8,000 to 32,000 tokens (roughly 6,000 to 24,000 words). This also lowers the cost of using a more expensive LLM.[00821 Further refinements on the process above comprises:1. Contracts and their parts are submitted to the LLM in a queue. This queue can be implemented in a variety of ways, such as a database table, a message queue, or a simple file. Also, contracts can be submitted to the queue in a variety of ways, such as through a web interface, an API, or a batch processing system. The specific method of submission will be different based on the desired user input mode.2. The LLM analyzes the first contract in the queue. The LLM is trained on a specific dataset of contract rules and patterns in order to perform this task effectively.3. Once the LLM has finished analyzing the contract, it generates a report. This report can include information such as the key terms of the contract, the risks associated with the contract, and any potential legal issues.4. The LLM then removes the analyzed contract from the queue and moves on to the next contract.5. This process repeats until all of the contracts in the queue have been analyzed.6. After the contracts are ingested, the system weights various risk rules associated with the risk profile associated with general contract risk and risk tolerances of the particular entity using the LLM. Upon request, the system generates a risk report. This report can include information such as the key terms of the contract, the risks associated with the contract, and any potential legal issues. The specific format and content of the report will depend on the specific implementation of the LLM.

[0083] One of the significant benefits to the staging of the ingestion and analysis of a document as demonstrated by this solution is demonstrated by this example. An MSA (master services agreement) as shown in FIG. 3 is optimized for an LLM solution by ingesting the MSA and its aggregated contracts in a selected order to keep all relevant documents in the LLM.

[0084] FIG. 4 describes a contract object model for a variant of the present solution.

[0085] A contract object model in the context of an LLM (Large Language Model) analysis platform typically refers to a structured representation of the key elements, relationships, and constraints within legal contracts or agreements. This model serves as a standardized framework for analyzing and processing contract data using Al and natural language processing techniques The model displayed only lists some of the metadata and commercial terms. The solution object model is structured to capture the key elements and attributes of a contract while also being flexible enough to be used across diverse types of contracts. Examples of this solutions objects further comprise Contract Metadata 402 which in turn further comprises a Terms and Renewal 404, a Party 406, an Other party 408, a Third Party 410, a Doc ID 412, a Parent Doc ID 414, a Parent Doc Title 416, a Parent Doc Effective Date 418, a Signature 420, among other metadata elements. The Standard Contract Terms 422 further comprise an Assignment 424, an Audit Rights 426, an Early Termination Fee 428, an Exclusivity 430, a Liability Limits 432, a Most Favored Nation 434, a Geographic Restriction 436 among other standard contract terms the model then resides in the Account Corpora of Contracts 438 that may be associated with a specific account for the solution variant where multiple accounts are held within a particular variant of the solution. Other useful metadata is listed below.

[0086] Other Contract Metadata:1. Contract ID: A unique identifier for the contract.2. Title: The title or name of the contract.3. Date Created: The date the contract was created.4. Last Modified: The date of the last modification.5. Contract Type: The type of the contract (e.g., lease, employment, or vendor).6. Status: Current status (e.g., active, expired, terminated).

[0087] Parties Involved:1. Primary Party: The main party or entity entering into the contract.2. Secondary Party: The other party or parties involved.3. Representative: Legal representatives or contact persons for each party.4. Company Level / Group5. Account Maps6. Associated 3rd Party product or service providers.

[0088] Key Commercial Terms:1. Duration: Start and end date of the contract.2. Payment Terms: Details about payment amounts, frequencies, and methods.3. Termination Clauses: Conditions under which the contract can be terminated.4. Fees associated with the performance of the contract other than payment (termination fees / audit fees / etc.)5. Obligations: Responsibilities and duties of each party.6. Rights: Rights granted to each party under the contract.7. Export / Off shore Restrictions8. Most Favored Nation9. Exclusivity10. Geographical Reach11. Audit Requirements12. Licensing Requirements

[0089] Common or Unique Clauses and Provisions:1. Confidentiality: Details about confidentiality and non-disclosure.2. Liability: Clauses related to indemnity, limits of liability, etc.3. Governing Law: The jurisdiction and laws governing the contract.4. Dispute Resolution: Mechanisms for resolving disputes (e.g., arbitration).

[0090] Attachments and other Related Documents1. Related Documents: Links or references to external documents or previous versions.2. Schedules: Detailed schedules or appendices attached to the contract.3. Statements of Work or Orders

[0091] Review and Analysis:• Risks Values and Risk Change values• Recommendations and previous recommendations• Al Summary of the aggregated contract

[0092] Signatures:• Signature Primary: Digital or scanned signature of the primary party.• Signature Secondary: Digital or scanned signature of the secondary party.• Third Party Approvals

[0093] This object model is modular, so sections can be added, removed, or modified based on the specific type of contract and the requirements of the organization. Solution also facilitates the LLM's ability to understand, analyze, and generate content related to contracts.

[0094] FIG. 5 is an example of a playbook generated by solution variant that is designed to guide less experienced negotiators through complex contract discussions. This feature analyzes contract terms to identify potential risks, suggests optimal negotiation strategies, and provides customized guidance based on historical data and the specific context of each negotiation. The playbook assists in compliance with regulatory requirements and internal policies while presenting multiple negotiation scenarios to the user help prepare for various outcomes. By leveraging machine learning algorithms, it continuously improves its guidance, incorporating feedback on negotiation outcomes to refine future recommendations. This feature not only reduces risk exposure for the company but also accelerates the learning curve for new negotiators, ensuring consistency in negotiation approaches across the organization. Ultimately, the negotiation playbook generator serves as both a protective measure against unfavorable terms and an educational tool, actively supporting the negotiation process and safeguarding the company's interests. Examples of Risk Points and Guidelines beyond the exemplar drawing would also comprise:

[0095] Solution Risk Point Identification:1. Flagged Unique Contract Type: Without clarity on the type of the contract, (e.g. a services contract format used for product delivery.)2. Undefined or Ambiguous Parties: Often 3rd party obligations creep into contract language, by not clearly identifying all involved parties can lead to future disputes.3. Unclear or Conflicted Duration: Without a well-defined duration and renewal terms, there may be unintended contract extensions or premature terminations.4. Conflicting Payment Terms: Ambiguous payment conditions can result in financial discrepancies.5. Termination Ambiguities: Unclear termination conditions might lead to unintended penalties or breach of contract.6. Loose or Missing Confidentiality Clauses: Inadequate confidentiality or non-disclosure terms can risk leakage of sensitive information.7. Governing Law: Agreeing to jurisdictions that are hostile to the company is a significant risk point.8. Undefined Dispute Mechanisms: Without clear dispute resolution methods, disagreements can become protracted or costly.9. Overlooked or Unique Special Provisions: Missing out on unique or unusual terms can introduce unexpected obligations or rights.10. Unchecked Liabilities: Not assessing potential liabilities can expose the organization to undue risks.[00961 Solutions Categories for Guidelines for Procurement:1. Clearly Define the Contract Type: Limiting the types of contracts to be accepted.2. Identify All Parties and update negotiation playbook: Ensure all entities involved in the contract are clearly listed.3. Establish Clear Contract Duration: Limit contracts to the minimums and maximums.4. Detail Payment Terms: Lists of cleared payment amounts, frequencies, methods, and conditions.5. Clear Termination Clauses: List the minimum and maximum grounds for contract termination and any associated penalties.6. Confidentiality Minimum Guidelines: Ensure robust clauses for confidentiality or nondisclosure are in place.7. Governing Law as listed: Clearly state the legal jurisdiction governing the contract.8. Permitted Dispute Resolution Mechanisms9. Flagging Special Provisions:10. Flagging other solution Risks and Liabilities as needed

[0097] FIG. 6 is an example of a historical look back at previous contracts with an entity or entity group that would inform the negotiations of a current contract. Other aspects of historical risk extracted from historical contracts also comprise:1. Problem Clauses: Previous contracts might contain clauses that have historically resulted in disputes or unfavorable outcomes.2. Performance on Payment: Past contracts may have ambiguous or inconsistent payment terms and performance that caused financial discrepancies or disputes.3. Problem or Missing Definitions: Definitions that have caused historical disputes.4. Risky Termination Clauses: Historical termination clauses were either too rigid or too lenient, affecting the parties involved in negative formats.5. Mismatched Expectations: Previous contracts might have set expectations that were not met, leading to dissatisfaction or breaches.6. Overlapping Jurisdictions: Historical contracts may have had unclear or conflicting jurisdiction clauses, complicating dispute resolutions.7. Problematic Obligations: Past contracts might not have clearly delineated the responsibilities of each party.8. Inadequate Dispute Resolution Mechanisms: Previous agreements may have lacked a clear mechanism for handling disputes, leading to prolonged or unresolved disagreements.9. Unfavorable Auto Renewal Terms: Past contracts might have auto-renewal terms that were not in a party's favor or went unnoticed.10. Missed Deadlines or Milestones: Historical instances where critical deadlines or milestones were missed, leading to penalties or breaches.11. Weak Indemnity Clauses: Historical contracts may have had inadequate indemnity clauses, leading to financial burdens.

[0098] FIG. 7 is an example of Guidelines for a new draft contract based on the solution's assessment. This example recommends three areas of concern for the negotiators.

[0099] Summarize Contracts prompt - this prompt will create a summary of the lineage of contracts for a business relationship, including the original agreement and any follow-on amendments .

[0100] Summarize Orders prompt - summarizes all the orders executed for a business relationship like products sold / purchased renewal dates, etc.

[0101] Summarize Playbook prompt - summarizes the organization’s negotiation playbook converted from raw form into defined rules by Al.

[0102] FIG. 8 shows a variant of the solution user experience (UXI) constructing an addendum to an Aggregated Contract Documents 314 and utilizing the SaaS Contract Platform 106 to consider all historical information, current commitments and current requirements of all the various documents in a large contract set that utilizes the LLM Response for MSA and Amendment summaries 332 to bring together all outstanding items of concern, Contract Metadata 402, Standard Contract Terms 422 among Other Terms 212 to cause the Contract Draft Generator 118 to create a draft for a user to then review and send to the other parties to the contract.ADDITIONAL CONSIDERATIONS

[0103] While the preceding description outlines various solution variants in detail, it's important to recognize that the legal scope of the invention is ultimately defined by the claims listed at the end of this patent. The detailed description serves as an illustrative example, and it is not exhaustive of all potential solution variants. Due to the impracticality of describing every conceivable solution variant, alternate configurations may exist — whether using current technologies or those developed after this patent’s filing — that still fall within the scope of the claims.

[0104] Throughout this specification, references to singular instances may also include plural instances, and vice versa. Likewise, while operations of methods are described separately, they can be performed concurrently or in a different sequence than presented. Components or functionalities described as separate in example configurations may be combined, while those presented as a single entity may be divided into multiple components. These and other modifications or improvements remain within the bounds of the described invention.

[0105] In certain solution variants, logic, routines, subroutines, applications, or instructions may be executed via software (e.g., code on a non-transitory, machine-readable medium) or hardware (e.g., special-purpose processors). In a hardware context, these routines can be physical, tangible units configured in specific ways, such as through a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). Alternatively, they may leverage general-purpose processors configured temporarily via software to execute specific operations. Decisions on whether to implement routines in dedicated hardware, software, or hybrid solutions may depend on cost, complexity, or other constraints.

[0106] For purposes of clarity, "hardware module" should be understood to mean a tangible entity that can either be physically constructed or configured (permanently or temporarily) to operate in a specific manner. If temporarily configured via software, a general-purpose processor may act as various hardware modules at different times. This flexibility enables the same processor to perform multiple functions dynamically, depending on the system’s current needs.

[0107] Inter-module communication between hardware modules may occur through signal transmission over circuits or buses. When modules are instantiated at different times, data can be stored and retrieved from shared memory structures, enabling asynchronous operation. Forinstance, a hardware module may execute an operation and store its results in memory, allowing another module to access and process the stored information later.

[0108] The operations of methods described in various solution variants may be partially or fully implemented by one or more processors. These processors may be physically located within a single machine or distributed across multiple systems, enabling distributed processing. In some cases, these systems may be in a centralized location, like a data center, while in other cases, they could be spread across multiple geographic locations. When processors are distributed, they may communicate and coordinate their tasks via networked infrastructure, forming a cohesive processing system.

[0109] Terminology used herein, such as "processing," "computing," or "calculating," refers to the manipulation of data in physical forms, such as electrical, magnetic, or optical quantities. When the specification refers to "one solution variant" or "a solution variant," it indicates that the described feature may be applicable to at least one possible solution variant. This should not imply that all instances of the phrase refer to the same solution variant.

[0110] Additionally, terms like "comprises," "including," and their variants are intended to imply non-exclusive inclusion. For instance, a method that "comprises" certain elements is not limited to those elements alone and may include other components not explicitly listed.Similarly, "or" should be interpreted as inclusive unless otherwise specified, meaning A or B could be true individually or simultaneously.

[0111] The descriptions provided are intended as illustrative, non-exhaustive examples. They do not define every possible solution variant, as doing so would be impractical, if not impossible. Moreover, technological advancements and alternate configurations may arise that still fall within the invention’s defined scope.

Claims

CLAIMSWhat is claimed is:

1. A method of contract analysis using multi-level prompting and retrieval-augmented generation (RAG), comprising: training a knowledge base containing contract clauses, legal terminology, and industryspecific information; applying a retrieval module; using a multi-level prompt generator; engaging a language model fine-tuned on contract analysis, generation, and summarization; and interacting with a user interface: receiving an input file containing contract-related text, including a main contract and other contract documents further comprising a statement of work (SOW) and an addendum; generating, by the multi-level prompt generator, a first-level prompt based on the input file, the first-level prompt further comprises a set of abbreviated pages; retrieving, by the retrieval module, a set of relevant information from the knowledge base using the first-level prompt; generating, by the language model, an initial analysis of the input file using the set of relevant information; based on the initial analysis, generating a second-level prompt to request a set of additional relevant information comprising more specific contract-related information; retrieving the set of additional relevant information using the second-level prompt; generating, by the language model, a detailed contract analysis and a set of tailored user questions based on the retrieved information from both prompting levels; classifying the tailored user questions into categories relevant to contract analysis; performing risk analysis on the classified questions and contract analysis; creating an abbreviated form of the contract by summarizing key terms, conditions, and obligations from the main contract, SOW, and addendums; generating an aggregated summary of the SOW and addendums, highlighting critical deliverables, timelines, and responsibilities;1) presenting the results of the analysis, classified questions, risk assessment, abbreviated contract form, and an aggregated SOW and addendum summary through the user interface.

2. The method of claim 1 further comprising: the set of abbreviated pages are further sized to fit within a context window of a large language model.

3. The method of claim 2 further comprising: selecting the order and size of the set of abbreviated pages to create an abbreviated representation of a set of essential contract terms that fits inside of a large learning model (LLM) context window.

4. The method of claim 3 further comprising using a contract object model to identify a set of metadata of the input file.

5. The system of claim 4 further comprising using the contract object model to further identify a set of standard contract terms.

6. A system for contract analysis using multi-level prompting and retrieval- augmented generation (RAG), comprising: a knowledge base configured to store contract clauses, legal terminology, and industryspecific information; a retrieval module; a multi-level prompt generator; a language model fine-tuned on contract analysis, generation, and summarization; a user interface; and a processor configured to: receive an input file containing contract-related text, including a main contract and other contract documents further comprising a statement of work (SOW) and an addendum; generate, via the multi-level prompt generator, a first-level prompt based on the input file, the first-level prompt further comprising a set of abbreviated pages;retrieve, via the retrieval module, a set of relevant information from the knowledge base using the first-level prompt; generate, via the language model, an initial analysis of the input file using the set of relevant information; generate, based on the initial analysis, a second-level prompt to request a set of additional relevant information comprising more specific contract-related information; retrieve the set of additional relevant information using the second-level prompt; generate, via the language model, a detailed contract analysis and a set of tailored user questions based on the retrieved information from both prompting levels; classify the tailored user questions into categories relevant to contract analysis; perform risk analysis on the classified questions and contract analysis; create an abbreviated form of the contract by summarizing key terms, conditions, and obligations from the main contract, SOW, and addendums; generate an aggregated summary of the SOW and addendums, highlighting critical deliverables, timelines, and responsibilities; and present the results of the analysis, classified questions, risk assessment, abbreviated contract form, and an aggregated SOW and addendum summary through the user interface.

7. The system of claim 6, wherein the set of abbreviated pages are sized to fit within a context window of a large language model.

8. The system of claim 7, wherein the processor is further configured to select the order and size of the set of abbreviated pages to create an abbreviated representation of a set of essential contract terms that fits inside of a large learning model (LLM) context window.

9. The system of claim 8, further comprising a contract object model configured to identify a set of metadata of the input file.

10. The system of claim 9, wherein the contract object model is further configured to identify a set of standard contract terms.

11. An apparatus for contract analysis using multi-level prompting and retrieval-augmented generation (RAG), the apparatus comprising: at least one processor; at least one memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to: maintain a knowledge base containing contract clauses, legal terminology, and industryspecific information; execute a retrieval module; operate a multi-level prompt generator; implement a language model fine-tuned on contract analysis, generation, and summarization; provide a user interface; receive an input file containing contract-related text, including a main contract and other contract documents further comprising a statement of work (SOW) and an addendum; generate, via the multi-level prompt generator, a first-level prompt based on the input file, the first-level prompt further comprising a set of abbreviated pages; retrieve, via the retrieval module, a set of relevant information from the knowledge base using the first-level prompt; generate, via the language model, an initial analysis of the input file using the set of relevant information; generate, based on the initial analysis, a second-level prompt to request a set of additional relevant information comprising more specific contract-related information; retrieve the set of additional relevant information using the second-level prompt;generate, via the language model, a detailed contract analysis and a set of tailored user questions based on the retrieved information from both prompting levels; classify the tailored user questions into categories relevant to contract analysis; perform risk analysis on the classified questions and contract analysis; create an abbreviated form of the contract by summarizing key terms, conditions, and obligations from the main contract, SOW, and addendums; generate an aggregated summary of the SOW and addendums, highlighting critical deliverables, timelines, and responsibilities; and present the results of the analysis, classified questions, risk assessment, abbreviated contract form, and an aggregated SOW and addendum summary through the user interface.

12. The apparatus of claim 11, wherein the set of abbreviated pages are sized to fit within a context window of a large language model.

13. The apparatus of claim 12, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to select the order and size of the set of abbreviated pages to create an abbreviated representation of a set of essential contract terms that fits inside of a large learning model (LLM) context window.

14. The apparatus of claim 13, wherein the apparatus further comprises a contract object model configured to identify a set of metadata of the input file.

15. The apparatus of claim 14, wherein the contract object model is further configured to identify a set of standard contract terms.

16. A method for enterprise contract analysis using structured provision modeling, comprising: receiving a corpus of enterprise contract documents; performing document pre-processing operations further comprising: deduplicating a set of redundant contract documents, establishing a set of document familyrelationships between a plurality of related contracts, amendments, and supporting documents, and organizing each of the documents from the corpus of enterprise contract documents into at least a structured collection based on a set of predefined business relationships; creating a standardized provision model that: map to a specific business use case, define hierarchical relationships between a plurality of a unique contract’s provisions, clauses, and terms, and capture a set of key business requirements and obligations; analyzing contract paragraphs using generative Al by mapping a set of extracted contract text to a corresponding clause-level components within the provision model: maintaining contextual relationships between the set of extracted mapped contract text and validating the mapping of the set of extracted mapped contract text against established business rules; performing a plurality of analysis operations on the mapped contract data, further comprising generating a set of risk assessments based on the standardized provision model maps; creating a negotiation playbook derived from a plurality of historical patterns further comprising identifying deviations from the standard contract terms, analyzing a set of obligation fulfillment parameters across a set of contract families; and generating a plurality of reports and recommendations based on the plurality of analysis operations wherein the plurality of reports and recommendations maintain a grounding in the original contract language while providing a plurality of enterpriselevel insights about business relationships and obligations.

Images