Intelligent tool and method supporting the patent granting process

Abstract

Nowadays the subject matter of patent applications often relates to disruptive technologies, which lead to the development of new products, systems and methods at a fast pace. As a result, inventors need to have a fast outcome to their complex patent applications, in order to decide if and how their novel ideas can be exploited. Unfortunately, nowadays, the examination and patent granting procedures remain slow when compared to the fast pace of technological evolution. The Information and Communication Technology (ICT) tool and method proposed by this invention uses Artificial Intelligence (AI) technology to retrieve prior art documentation, support patent application drafting, and assess novelty and inventive step with effectively reasoned and reliable technical argumentation. The tool supports end-users, such as inventors, patent attorneys and patent examiners, from their own perspective.

Description

[0001] INTELLIGENT TOOL AND METHOD SUPPORTING THE PATENT GRANTING PROCESS DESCRIPTION

[0002] 1. TECHNICAL FIELD

[0003] The present application relates to the field of Information and Communication Technology (ICT) and in particular to an intelligent system integrating tool and methods involving Artificial Intelligence (Al) procedures such as fuzzy logic computing, neural networks, machine learning and quantum computing.

[0004] 2. BACKGROUND ART

[0005] United States Patent US 11,966,688 Bl, AI-Based Method and System For Drafting Patent Applications, Gal Ehrlich, Apr. 23, 2024, and United States Patent Application, US 2024 / 0303416 Al, AI-Based Method and System For Drafting Patent Applications, Gal Ehrlich, Sep. 12, 2024, disclose the use of Al for the purpose of drafting a patent application document.

[0006] InstructPatentGPT: training patent language models to follow instructions with human feedback, Jieh-Sheng Lee, Springer 8 April 2024, conceptualizes a system of reinforcement learning from human feedback, for Patent document retrieval and patent claim generation. The following list contains prior art documents helpful for understanding the technical background of the invention.

[0007] 1. LLMs to the Moon? Deng, X., Bashlovkina, V., Han, F., Baumgartner, S., Bendersky, M. (2023), Reddit Market Sentiment Analysis with Large Language Models Companion Proceedings of the ACM Web Conference 2023

[0008] 2. Grohs, M., Abb, L., Elsayed, N., Rehse, J. (2023) Large Language Models can accomplish Business Process Management Tasks, ArXiv abs / 2307.09923

[0009] 3. Process Modeling With Large Language Models, Kourani, Humam et al., ArXiv abs / 2403.07541 (2024)

[0010] 4. Specialist or Generalist? Instruction Tuning for Specific NLP Tasks, Shi, C., Su, Y, Yang, C., Yang, Y, Cai, D. (2023), ArXiv abs / 2310.15326

[0011] 5. A Self-enhancement Approach for Domain-specific Chatbot Training via Knowledge Mining and Digest, Zhang, R., Gao, L., Zheng, C., Fan, Z., Lai, G., Zhang, Z., Ai, F., Yang, Y, Yang, H. (2023), ArXiv abs / 2311.10614

[0012] 6. Patent Classification Using BERT-for-Patents on USPTO, Chikkamath, R., Parmar, V., R., Otiefy, Y, Endres, M. (2022), Proceedings of the 2022 5th International Conference on Machine Learning and Natural Language Processing

[0013] 7. ChatGPT vs. Google Gemini: Assessing AI Frontiers for Patent Prior Art Search Using European Search Reports, Renukswamy Chikkamath, Ankit Sharma, Christoph Hewel and Markus Endres, Second International Workshop on Semantic Technologies and Deep Learning Models for Scientific, Technical and Legal Data May 26th or 27th, 2024 / Hersonissos, Greece held at ESWC 2024, https: / / ceur- ws.org / Vol-3697 / paper2.pdf8. Patent Classification Using BERT-for-Patents on USPTO Authors: Renukswamy Chikkamath, Vishvapalsinhji Ramsinh Parmar, Yasser Otiefy, Markus Endres, MLNLP '22: Proceedings of the 2022 5th International Conference on Machine Learning and Natural Language Processing

[0014] 9. Leveraging the BERT algorithm for Patents with TensorLlow and BigQuery, November 2020, Rob Srebrovic, Jay Yonamine

[0015] 10. An empirical study on patent novelty detection: A novel approach using machine learning and natural language processing, R Chikkamath, M Endres, L Bayyapu, C Hewel, 2020 Seventh International Conference on Social Networks Analysis 11. Unveiling the inventive process from patents by extracting problems, solutions and advantages with natural language processing, Expert Systems with Applications, Vito Giordano, Giovanni Puccetti, Filippo Chiarello, Tommaso Pavanello, Gualtiero Fantoni, Volume 229, Part A, 2023, ISSN 0957-4174

[0016] 12. Natural language processing to identify the creation and impact of new technologies in patent text: Code, data, and new measures, Sam Arts, Jianan Hou, Juan Carlos Gomez, Research Policy, Volume 50, Issue 2, 2021, 104144, ISSN 0048-7333

[0017] 13. PatentGPT: A Large Language Model for Intellectual Property, Zilong Bai, Ruiji Zhang, Linqing Chen, Qijun Cai, Yuan Zhong, Cong Wang, Yan Fang, Jie Fang, Jing Sun, Weikuan Wang, Lizhi Zhou, Haoran Hua, Tian Qiu, Chaochao Wang, Cheng Sun, Jianping Lu, Yixin Wang, Yubin Xia, Meng Hu, Haowen Liu, Peng Xu, Licong Xu, Fu Bian, Xiaolong Gu, Lisha Zhang, Weilei Wang, Changyang Tu, Patsnap LLM Team, https: / / arxiv.org / abs / 2404.18255v5, June 6, 2024

[0018] 14. How small Chinese Al start-up DeepSeek shocked Silicon Valley, Financial Times, Eleanor Olcot in Beijing and Zijing Wu in Hong Kong January 252025 Furthermore there are publicly available platforms, which use Al to support prior art retrieval and / or patent application drafting, for example the platforms developed by QUESTEL, XLSCOUT, IPRALLY, PatentPal and WIPO.

[0019] However the tool and methods described herein are not disclosed by the available prior art.

[0020] 3. TECHNICAL PROBLEM

[0021] Disruptive technologies, such as Al, allow for fast technological advancements in almost all fields of technology, that is in engineering, medicine, pharmaceuticals, chemistry, energy, environment, food etc. As a result, inventors want to protect their novel ideas as soon as possible, in order to proceed with the exploitation of the novel ideas.

[0022] Unfortunately, there are substantial delays inherent to the patent granting procedure. The delays are inherent to the procedures followed by patent attorneys and patent examiners in retrieving and evaluating relevant documents, when assessing novelty and inventive step. The evaluation of complex multidisciplinary documents (i.e. patent application, prior art documents) in leading edge fields, become even more difficult to retrieve and subsequently analyze, from databases comprising of millions of highly complex documents. In addition to typical research challenges, prior art documents may exist only in languages not comprehended at least by some of the parties involved in the patent granting procedure. Nowadays, the task of analysis and evaluation of complex documents requires multidisciplinary expertise for each single document. As a result, it can take several years before a final decision is taken by a patent office, when the work process is based onconventional search and examination tools, whereby the patent granting procedure is manpower intensive with the effective scientific inference based primarily on human analysis and reasoning.

[0023] This time lag between the evolution of novel ideas and the patent granting procedure, is seen as an obstacle in the development of new products and procedures, and creates huge financial losses to inventors, investors and entrepreneurs alike. It is not uncommon that patents are granted and then challenged successfully by competitors, due to the limitations of the conventional patent processes. The gap between technology evolution and formal innovation recognition, leads to cost increases and time delays in delivering much needed products like medicines and food. Therefore, a solution is required to assist inventors, patent examiners and patent attorneys in concluding their responsibilities with high degree of accuracy and in pace with the technological evolution cycle.

[0024] 4. BRIEF DESCRIPTION OF FIGURES

[0025] Figure 1. Document collection for LLM Training

[0026] Figure 1 is a graphical representation of major processable document categories collected and organized to support Large Language Model (LLM) Training, (see the Training Data section of the invention and the second embodiment).

[0027] Figure 2. Document Preparation for LLM Training

[0028] Figure 2 is a graphical representation of the main document pre-processing activities, in order to become data suitable for LLM Training (see the Training Data section of the invention and the second embodiment).

[0029] Figure 3. Loading LLM Training Data

[0030] Figure 3 is a graphical representation of main types of document pre-processing in order to convert them to plain text ready to be used as LLM Training data. Said documents have already been processed as depicted in Figure 2 (see the sections of Training Data, the Training Method, and the Quality Assurance (QA) Method).

[0031] This Figure indicates graphically the process steps according to the present invention for training data production and the steps where quality measures are produced. Data from such measurements support data handling and critical decisions for training. For a more detailed graphical representation of this process see Figures 6.1 to 6.5.

[0032] Figure 4. LLMs Trained by the invention

[0033] Figure 4 is a graphical representation of several types of Trained LLMs, which are produced by the training methods of this invention. The flow of the blocks when followed from left to right of the graph indicate the progressive stages of deeper and more specialized training required to develop an LLM trained for the purpose of the scope indicated by the block label. The vertical block to the left side represents the notion that the specialized training procedures for all LLMsadhere to the same high standards of LLM training, the Quality Assurance (QA) controls, the information security and information services, which are common for all.

[0034] Figure 5. Tool Network and Logical Architecture

[0035] Figure 5 is a graphical representation of the network and logical architecture of the Development, QA and Production Environments. It presents a graphical depiction of access rights controls for users and user roles to the environments and to LLMs trained with information and procedures controlled by ownership rights and privacy requirements.

[0036] Figure 6. LLM Training and Line Tuning

[0037] Figure 6 provides a generic illustration of the process for LLM Training and Fine Tuning. Figure 6.1 to Figure 6.5 illustrate in more detail the process flow of Figure 6 for specific training scope, as specified in the second line of the title of each Figure 6.1 to 6.5.

[0038] Figure 6.1. LLM Training and Fine Tuning. Training LLMs to develop the Tool Baseline.

[0039] Figure 6.2. LLM Training and Fine Tuning. Training the Tool with the Guidelines of a Jurisdiction

[0040] Figure 6.3. LLM Training and Fine Tuning. Training the Tool for Professionals Figure 6.4. LLM Training and Fine Tuning. Training the Tool with Private and / or Proprietary Documents.

[0041] Figure 6.5. LLM Training and Fine Tuning. Training the Tool with Private and / or Proprietary Guidelines.

[0042] The detailed descriptions of these processes are documented in the Training Method section of this invention and in the QA method of this invention.

[0043] 5. DISCLOSURE OF INVENTION

[0044] When a patent application is prepared to be submitted for evaluation according to the present invention, the tool and method of this invention assist the claim assessment by retrieving relevant prior art documentation and suggesting technically worded reasoning. By assessing the set of patent claims with regard to novelty and inventive step ahead of the actual examination process, the invention gives to the user early confidence whether a patent granting may be feasible or not. In particular the invention by simulating the anticipated dialogue between the applicant and the patent examiner, enables identification of claimed subject matter which is either explicitly disclosed or implicitly derived from logical inferences based on prior art documentation or not. Special tools and methods of this invention, suitable for specific end-user groups, utilize as training data the communication between examiners and applicants and the valid sets of question and responses in relevant technical domains, properly organized to support high precision technical reasoning and text synthesis.

[0045] The invented intelligent tool and method as claimed support the drafting of a concrete patent application with higher chance of patent granting. The text drafting is achieved in amuch shorter time than what is common today. In addition it provides properly worded valid argumentation, which can be used to speed up the patent examination procedure. The tool and method reduce the total duration and workload, due to the fact that the amount of repetitive examination steps with regard to compliance with patenting requirements, which take up a considerable valuable time and resources, are executed efficiently and provide the resulting text in the required format. The drafting of a patent application document is supported by the invented tool and method, by responding to user guidance and requests. Furthermore the tool and method assist the end-users to document the claims of a patent application, while end-users can always utilize any conventional tools available, to perform complementary validation and verification, if required.

[0046] The invention utilizes a number of appropriate pre-trained Large Language Models (LLM), as one Model does not meet the requirements of all technical domains and all end user groups. Each model is trained and optimized for processing patent related information of a specific technical domain, e.g. a model trained in the Electromechanical field is trained with the relevant core scientific and technical texts and the related prior art documents. Information irrelevant to patents is forgotten according to the methods and the techniques of this invention. Depending on the technical domains supported by a model, the metadata and information weights are optimized for the relevant domain resulting in low degree of Al hallucinations. The use of the invented tool requires technical domain knowledge as well as a level of expertise in AL

[0047] Each of said pre-trained LLMs is selected to be suitable for a specific technical field. The model receives focused training for the Patent Legal Framework (Articles, Rules, Case Law and Guidelines) of a specific jurisdiction and technical documentation. The invented tool by design incorporates high quality standards in an innovative way. The LLMs integrate technologies such as Quantum computing, advanced security and information protection, Text Tokenization, specialized User Interfaces, Advanced Computer Programming. As a result the implementation of the invention is achieved with the data produced by the methods of this invention and Al expertise required for training and validating said LLMs. Furthermore quality assurance ensures that quality standards are met.

[0048] Furthermore the invention is designed to absorb the technical evolution and Patent Legal Framework updates. The LLMs are updated by an iterative process in a systematic way and enhance the offered services with new user requirements and the evolving Patent Legal Framework. The combination of the aforementioned features distinguishes the invention from the prior art.

[0049] The invention supports with its methods and tools different end-user scenarios. Types of end-users who have different requirements and expectations are:

[0050] • Professionals who require fast response with precise outcome. They are also interested for additional capabilities such as strong privacy, security, compliance to policies and procedures.

[0051] ° A patent Applicant, a patent Attorney and a patent Examiner are typical endusers and among themselves have different priorities, different work approach, and different logic in their patent document analysis.° Professionals typically employ custom tools for their work, which utilize the same technical document but analyze the text from a different perspective. The invented tool supports seamless information exchange with the custom tools. ° Researchers benefit for the precise responses of the invented tool. Additionally they may be inspired by unexpected, “out of the box” responses due to hallucinations.

[0052] • Students / Trainees, who will use the invented tool for education, familiarization with patent processes and personal development. For this end-user group results of moderate precision provided by a quick response are satisfactory.

[0053] The invented tool also considers the requirement to make efficient use of resources. Methods and techniques are implemented to train the LLM efficiently resulting in an effective patent tool.

[0054] It is evident that the user objectives determine both training resources, Production Environment architecture and resources for the tools. As a result different LLMs and / or LLM architecture may be employed to support different user roles while complying to ethical standards.

[0055] The invention described herein, is disruptive with regard to the current well-established patent granting processes with significant impact on products and services relevant to the patenting process, in the following sense:

[0056] • Patent offices will be able to process patent claims faster, since their examiners will retrieve more pertinent prior art documentation with the invented tool. The inclusion of Artificial Intelligence in the Patent Search and Examination will accelerate the Patent Granting Procedures. The resulting patents will have a higher presumption of validity leading to a lower enforcement effort. As a result Patent Offices will be able to serve better the public by processing higher number of patent applications per calendar year with adherence to established quality procedures, and increasing legal certainty.

[0057] • Patent Attorney offices, will benefit from using the invented Patent Intelligent processing Tools and Methods. The Attorney Offices will be able to process more patent applications in the same average duration they schedule for one patent application. Therefore they will be able to allocate resources in a more efficient and effective way and produce improved quality patent application documents with regard to more precise prior art documentation and well worded technical reasoning.

[0058] • Patent applicants make best use of their time, budget, facilities and other resources resulting from the higher speed of patent granting procedures. Such results will have an immediate effect in the “time to market” for new products such as: critical medicines for pandemics and other deceases, improvements to nutritional food production and food distribution, environmental protection and industries such as ICT, space and astrophysics.

[0059] • The well founded argumentation produced with the assistance of the invented tool, for the proposal to grant a patent, acts as a deterrent to ill-founded fillings for oppositions or infringement procedures.5.1 Invented Tool System Architecture

[0060] The invented tool supports methods and techniques, which offer Information Services for the end-users. The tool comprises three working environments. Each environment comprises of an associated tool and methods. In a progressive approach here-below it is presented the architecture of each environment and the applicable methods and techniques for each environment. The tool associated with each environment is implemented by a server farm. In the embodiment section there are descriptions of instances with specific parameters and resource requirements for each working environment.

[0061] The invented tool in its generic form comprises the following three environments:

[0062] • A Development Environment where the data preparation, programming and LLM selection and training is performed.

[0063] • A Quality Assurance (QA) Environment where the specially trained models are subjected to quality procedures to validate suitability for usage. Identified faults and corrective actions are returned to the Development Environment as improvement requests. Throughout this description ISO 9001 terminology regarding quality is respected.

[0064] • A Production Environment whereby the final product is produced by cleaning the LLM environment from any data, code, and parameter settings required for training and QA, before it is released to Production.

[0065] 5.1.1 The Development Environment

[0066] The Development Environment concerns, the LLM Selection method, and the LLM Training Method. The LLM Training Method incorporates the LLM training Data Preparation, and the LLM Training techniques.

[0067] To achieve this outcome, the Development Environment incorporates and / or interfaces with required services, which do not form part of the invention. Examples of required services are Digital Identity Management, Information Security, Quality Assurance, Version Management, Document Management, Information Technology Services Management, Performance Management, and Change Management, Performance Analysis, Anomalies Detection, Information Correlations and Contextualization, etc. Such services are not part of the invention and thus they are not described in detail. They are though necessary Information System Operations and Artificial Intelligence Operations, required for the proper function of the Invented Tool.

[0068] 5.1.1.1 LLM Selection Method

[0069] The invention utilizes pre-trained LLMs, which are available for use as general purpose Al tools. The LLM selection, appropriate for a specific technical domain, is one of the initial decisions made. Depending on performance metrics utilized in the Quality Assurance Environment, when the results are not at the required level, more than one models may be trained, to compare results and make the final model selection, which will be the model deployed for production. Each LLM is then trained with patent specific information. The training method also assures that the model is trained with a sample of scientific and technical documents relevant to the corresponding Technical Domains.

[0070] Critical factors in the selection of the pre-trained LLM are:

[0071] • The ability to receive efficiently both manual training and automated training. • The ability to complete training with optimization of key parameter values (i.e. task duration, memory size, CPU and GPU size)• The ability to process the complexity of the information on which it will be trained using the minimum resources. Considerations are the model ability to either receive training on the basis of special technical language and terminology in which the information is expressed.

[0072] • For specific technical fields, which for instance require both text and image analysis, such as Pharmaceuticals and Chemistry, separate LLMs, each one suitable for a specific type of content (text, chemical formulas, imaging, etc.), are selected.

[0073] • The model’s ability to utilize Quantum Computing technology is a significant consideration when the training data have a higher than usual degree of complexity or they involve a disproportionately number of parameters and / or a long series of inferences.

[0074] These critical factors contribute to the preselected LLM architecture and determine parameters like what data will be selected for training, the training approach and the preparation of training data. Ideally a model is pre-trained on the technical knowledge of a selected technical domain. In case there are reasons to select a general purpose model with insufficient training in technical domain, the same procedures of this invention are used to train the LLM with the technical knowledge data pertaining to the technical domain at hand.

[0075] The invention utilizes a number of appropriate LLMs, as one Model does not meet the requirements of all technical domains. Each model is trained and optimized for processing patent related information in a specific technical domain.

[0076] 5.1.1.2 LLM Training Method

[0077] The present invention is based on a special LLM training method. Said method incorporates a set of known training techniques, inventive information selection procedures, inventive data structured by the design of this invention and specialized quality assurance methods and techniques. This section contains a systematic presentation of the training strategy and the human involvement, which leads to two elements contributing to the invention:

[0078] • An inventive approach for document pre-processing.

[0079] • An inventive method in document selection and categorization by qualitative and quantitative criteria, so that the documents are converted into training data, which training data are a technical feature of this invention.

[0080] Said training data, a technical feature of this invention, are utilized by training procedures and training techniques following a selected training approach.

[0081] 5.1.1.2.1 Training Strategy

[0082] To train an LLM for a specific Technical domain it requires one or more training cycles with the methods and techniques of this invention. For each training round a strategy document defines and governs the training approach and the related training parameters. The invention utilizes six elements in an inventive combination to develop an LLM Trained for a specific Technical Domain, adapting the training parameters and the quality standards to train a specific LLM for the expectations and use cases of a specific user group. The key elements used in the training strategy are:

[0083] • The definition of the final training objective and the training approach governing the methods and the procedures based on the end user expectations, within time limitations and available computing resources.

[0084] • The definition of a set of innovative training Data, as described below in the section of “Innovative Training data pre-processing”.• The definition of a set of innovative quantitative criteria and qualitative criteria to segment the training data into sets suitable for effective LLM training, as described below in the sections of “qualitative criteria” and “quantitative criteria”.

[0085] • The selection of appropriate training techniques to meet the training objectives, as described below in the section of “training techniques”.

[0086] • The definition of a set of Quality Assurance procedures based on Test data and performance thresholds, representing in a measurable form the user expectations for critical parameters. Representative critical parameters are the information precision target for LLM responses, training and operating cost, production system responses time and work duration, in order to approve the production system. • The selection of an appropriate Large Language Model (LLM) and the system architecture, as described below in the sections of the “Quality Assurance (QA) Environment”.

[0087] The said training approach will determine how the documents selected by the qualitative and quantitative criteria will be subdivided in smaller size training document subsets. The aim is to optimize the resources for training by minimizing the duration of training procedures which require more resources in time and / or expensive hardware and / or services. The training approach determines at which iterations the LLM is subject to quality measurements to assess whether it has reached the predefined performance level. The training experts determine whether the LLM is trained sufficiently, on the basis of measured parameters. If further training is required, they decide what adjustment is required in training.

[0088] 5.1.1.2.2 The Human Intellect Factor

[0089] This invention considers that during the tool development users are utilizing the methods and tools in the context of development work roles. The definition of roles is in the section of Reference Signs and Terminology List. Development users requiring Patent Expertise from different technical domains are necessary to be engaged in the formalization of the training strategy and in execution of the LLM Training Method. In addition to their expertise in Patents these experts will need to have qualifications in Al, LLM Architectures, and specific LLM Training. They form a joined team with other experts from the Development Environment, so that said team has strong qualifications and expertise in LLM technology, programming, and the required Information Systems Technologies.

[0090] This team of multidisciplinary expertise defines the training strategy elements for each round of training. The aim of the team is to increase the ability of the Model to respond on user prompts with detailed and accurate information. Some of the key responsibilities of these roles are:

[0091] • They perform specialized training on a model pre-trained as a general purpose Al tool, in order to achieve specialization in the Patent granting processes and procedures.

[0092] • They introduce the localization training, as described below, which is the specialization of the trained tool on the Patent legal framework (Articles, Rules, Case Law and Guidelines) valid for a specific jurisdiction.

[0093] • They fine tune the tool trained by this invention to improve its performance and to become a specialized Al tool for usage in the intellectual property domain.5.1.1.2.3 Training Data

[0094] The information required for training is in processable documents. For these documents to become training data they are pre-processed and evaluated as described below. Figure 1 provides an illustration of selection of documents for LLM Training. The document content and structure is processed and is converted in plain text format files. By this processing method the objective is to clean documents from information not suitable for LLM training such as errors, information duplication, and redundant information (for example text layout and references to figures), for the purpose of converting said documents into training data, while preserving the essence of the information. Figure 2 provides an illustration of document preparation for LLM training. The original document is preserved in its original format and is stored in library linked to the training data. Likewise images, possible multimedia information associated to the document is enhanced with metadata and separated from the text to be stored in special file libraries with appropriate narrative, cross-references and indexes. Special records are created to be used for bibliographic data, Unique Author Identification (UAI), and classification / indexing attributes, which are also stored in structured auxiliary files. Information extracted from the rest of the documents of the Global Dossier are organized to form valid query and answer sets based on the technical domain documents and the Patent Legal Framework.

[0095] The aforementioned data after been cleaned and organized are used to train specific types of LLMs, with methods described below. The choice of the pre-trained LLMs is based on whether they can be trained for text and / or multimedia patent data. It is conceivable that multimedia data can be used in training depending on the capabilities of the LLM to be trained. The cleaning process is adapted to the intended use of the LLM. According to the invention the auxiliary files and the proprietary data are generated during the conversion of documents in the technical domain to “LLM training data”. This technique, which forms part of the invention as claimed, enables to achieve the training data infrastructure required by said specific types of pre-trained LLMs.

[0096] According to the invention the aforementioned training data are utilized in accordance to the training strategy implementing the end user objectives. The proprietary data produced by the invention is cleaned and valuated, considering the training objective of a specific technical domain. This process forms part of a Data Governance process, which is defined and applied by a Data Officer. Said Data Governance process takes into account the LLM training parameters (training cost, training duration, use of services), which are affected by at least the data requirements for technical, compliance and ethical attributes. The LLM training produces different intermediate outcomes, which may be suitable for some user communities and not for others. For example a professional engaged in patent procedures has different expectations than a trainee on patent processes. The invention provides the required level of information services and technical approach for each identified user community.

[0097] 5.1.1.2.4 Innovative Training data pre-processing

[0098] The documents, which are selected to train the LLMs, have to meet certain criteria, which are explained below. They have to be processed to meet the criteria required to be converted in training data, suitable for exploitation by the methods of the present invention. The documents are grouped according to the International Patent Classification (IPC) and Cooperative Patent Classification (CPC) schemes. A document subset of each class is classified for use in supervised training. There are qualitative and quantitative criteria used to decide the document selection. There is assignment of qualitative documentcriteria like prioritization and significance, which segment the training sets is subsets. There is special consideration for border line conditions and exceptions, so that information is not missed due to training parameter settings. All these considerations about the document selection methods form part of the inventive concept and are necessary to implement the LLM Training.

[0099] 5.1.1.2.4.1 Quantitative criteria.

[0100] The exact quantities to meet specific training objectives are detailed and documented in the training Strategy. Typical quantity criteria are referenced in the bullets below, while examples of quantity values are referenced in the embodiment section.

[0101] • The complete set of training documents for a specific technical domain are segmented in three non-overlapping subsets utilizing one or more of the values of the qualitative criteria described in the next section. For each subset die training team utilizes the quantitative criteria to select a percentage of the overall available documents to be used for training. The said percentage is defined by the subject matter experts depending on the characteristics of the available documents and the training performance objectives. When this selection results in a subset that is too large for the purposes of the training, this subset shall be subdivided into smaller subsets at the discretion of the expert based on the quality criteria of the documents. For subsets where the number of relevant documents remains too large and it is impossible to reduce the size, documents are selected randomly as the members of the subset have relatively similar impact on training.

[0102] • The first subset is used for supervised training and its size is determined by the training experts and depends on parameters such as:

[0103] ° the LLM model and its complexity

[0104] ° the model architecture,

[0105] ° the complexity of a technical domain determined either through measurable parameters, or on contextual knowledge and heuristics.

[0106] ° the number of training steps required to fine-tune training to achieve performance objectives, such as text summarisation, response to desk actions, discovery of the level of technique, etc. during supervised training.

[0107] • Given the type of information required in quality control scenarios, a low percentage of the remaining documents are separated by the training specialists forming the second subset of training, for use in quality control procedures. The qualitative document criteria, described in the next section, assist in the selection of documents for quality testing scenarios. For a subset with an insufficient number of documents for test scenarios, either said subset is merged with another relevant subset, or that subset shall be augmented with test data compiled by the training and quality teams.

[0108] • The remaining documents, which are not used for supervised training and quality control, constitute the third training documents subset and are used for automated training. The training team calculates the size of each subset by ensuring that the automated training subset is the largest of the three. If a sufficient number ofdocuments cannot be assembled, the training team shall include documents that are in a hierarchically higher class of IPC / CPC.

[0109] The goal is to have a viable LLM training process, using the allocated resources with acceptable quality test results.

[0110] 5.1.1.2.4.2 Qualitative criteria.

[0111] Qualitative criteria are applied to the set of documents in a technical domain to identify the subset of documents with significantly different formulations of expressing scientific thought designating a specific technical solution. The selected documents of a particular technical domain are used to perform domain specific fine tuning for the LLM. These qualitative criteria are:

[0112] • Importance and uniqueness of the document. This criterion determines the importance of the document expressing complex ideas and concepts, with domainspecific unique expressions that go beyond skillful language expressions and demonstrate a deep understanding of the concepts related to the domain or relevant patent class, and / or

[0113] • Document age. Used to categorize documents into age groups based on whether scientific expression and syntax has changed significantly over these time periods, and whether there are changes in terminology, standards / units of measurement, and / or

[0114] • Author Importance. Characterizes the author's entity (researcher and / or organization) based on the number of patents and / or number of publications held, and / or

[0115] • Training and QA category. The training team assigns a category grade to selected documents using quantitative criteria described above to determine the intended use of each document in supervised training or automated training or quality testing scenarios; and / or

[0116] • Training Significance rating. The training team assigns a significance rating to the documents included in a subset in order to determine the significance of the documents in achieving the training objectives of that subset. The Significance rating also determines the significance of the document in order to avoid errors in LLM training developed by the structure and technical expressions contained in the document. This criterion helps to identify documents which use subtle variations of words and terminologies, which express concepts specific to the technical domain and the patent class.

[0117] The present invention is even applicable to complex technical fields such as biotechnology, medical technology and pharmaceuticals. Depending on the complexity of the technical field, the training expert team may use other qualitative criteria, which are related to the complexity of the technical characteristics for the related technical fields.

[0118] These qualitative criteria are not part of the invention, but the fact that qualitative criteriaare utilized in the manner identified above to support the improvement of LLM training is a technical feature of the present invention.

[0119] 5.1.1.2.2.4.3 Borderline Conditions and Exceptions

[0120] • The selection of training documents is performed with the purpose of LLM training. According to the invention, the training team should exclude technical documents, which cause problems in Al parameters setting when used in LLM training, thus developing a curated set of training documents. For instance, LLM is trained with current technical knowledge, while not including very old documents based on outdated technology. Depending on the quality criteria for the training, documents with units of measurement and standards that are outdated are either undergoing unit conversion or not being used.

[0121] • A registry is kept for each exception handling and each boundary condition. Similarly, records shall be kept of the selected documents and any pre-processing actions. The records are analyzed to monitor progress, e.g. to record the completion of exception handling decisions for similar patents, and to measure the duration of the training process steps in order to improve the efficiency of the training. These registries assist in decision making and facilitate the planning of training work. • Evaluating the suitability of the data based on end-user expectations is one method of improving the training parameters of LLM. Such parameters include the determination of the hallucination level and / or the degree of error in responses. Parameters like these help to validate the performance of the model and test whether the model is aligned with current technical knowledge. Said registries assist the decision making and facilitate planning.

[0122] 5.1.1.2.5 LLM Training Approach

[0123] The selected LLMs receive both supervised training by knowledge area experts and unsupervised training; both methods utilize the said specially prepared data. Further training fine tuning improvements are introduced by using pairs of valid questions and responses. The combination of training techniques, unique data and quality measurement procedures improve the LLM training and develop an innovative approach, which permits model performance validation and assure suitability for the purpose of use. The specifics of the training method are presented below. Figure 3 provides an illustration of training data used for LLM training including the aspects of data security, quality control, and data management.

[0124] 5.1.1.2.5.1 Training Techniques

[0125] The proposed invention utilizes pre-trained LLMs, which are available for use as general purpose Al tools. They are then trained with patent specific information; the training method also assures that the model is trained on the basic technical documents relevant to the Technical Domain.

[0126] The invention utilizes, depending on the needs, two tuning training techniques: the Instruction Tuning and the Parameter Tuning. These techniques are described below:• Instruction tuning. This technique aims at improving the model performance for specific tasks. With this technique the model is trained on pairs of valid queries and responses by using the content of the Global Dossier. The choice of data is crucial especially for tasks like summarization and translation and thus it requires a joint team of technical staff and patent experts. This joined team adjusts the model weights creating a new LLM version with improved capabilities. One key consideration is to achieve these objectives within acceptable time and with sufficient resources suitable for the complexity of the technical domain. For instance, a patent classified in a mechanical or electrics domains involves sufficient quantified parameters and thus less complexity than a patent in plants, medicine and medical fields, where heuristics may be involved. Thus at this stage and for specific technical domains quantum computing may be utilized to complete the fine tuning tasks and the operational tasks with reasonable response time.

[0127] • Parameter tuning. This technique optimizes the LLM training by freezing some of the parameters and updating only the rest, and thus achieving more manageable memory requirements, while preserving the past model training. This technique does not overwrite or change values, such as knowledge weights, on which the model has already been trained, thus avoiding the loss of previously learned information. The accidental change of trained parameters from past training is identified as “catastrophic forgetting”, which is avoided by this technique. The parameter fine tuning handles technical issues like remaining within available storage resources during LLM training.

[0128] The team of patent specialists and Al experts additionally performs various training techniques that depend on the technical field. These training techniques include Chain-of-Thought learning, Reinforcement Learning, and hybrid methods, etc. These techniques are not part of the invention and, therefore, are not described in detail.

[0129] However, the combination of all of the above techniques to optimize training resources and LLM performance for specific technical fields is an inventive concept as described in the present application.

[0130] 5.1.1.2.5.2 Training Procedures

[0131] The completion of supervised and unsupervised training establishes an initial baseline, meaning that there is an initial outcome that is already suitable for end-users with specific requirements. LLM Localization procedure trains the model in applicable Patent Legal Framework for a specific jurisdiction. This makes the model capable to address specific requests for claim validation. To increase efficiency in the model performance, an additional training procedure is executed. With this additional procedure, documents from the Global Dossier are used to train the model and introduce valid pairs of queries and answers, with information extracted from the correspondence between the examiners and the inventors. The combined effect of the training procedures improves the model precision in responses and reduces hallucination. In addition the model can augment its training with proprietary information for patent related matters and with proprietary administrative procedures, enabling the model to respond for scenarios of use specific to the owner of the proprietary information.

[0132] The training methods and techniques are also combined to produce an LLM with lower training duration and lower cost in training but with a higher degree of hallucination. Such a tool is suitable for scenarios where the user wants to be supported in brain storming and“out of the box” thinking, based on scientific approaches that are not conventional and are not restricted by the established approach.

[0133] Figure 6 provides a generic illustration of the process for LLM Training and Fine Tuning. Figure 6.1 to Figure 6.5 illustrate in more detail the process flow of Figure 6 for specific training scope, as specified in the second line of the title of each one of the Figure 6.1 to 6.5.

[0134] 5.1.2 The Quality Assurance (QA) Environment

[0135] The model performance for each purpose of usage is validated and fine-tuned in the Quality Assurance Environment, where the model is subjected to an extended set of tests with predefined procedures and methods. They validate the suitability for purpose of usage, as defined in the training strategy. Depending on the training objectives, more than one LLM are trained and statistics of their relative performance support decisions about the best model, which is suitable for specific end-user groups and user roles.

[0136] The invention provides methods for Quality Assurance (QA). In said methods the team of experts performs Quality Measurement Evaluation on each trained LLM by utilizing qualitative and quantitative tests, based on the selected data sets for the QA Environment. The outcome of the tests is returned to the Production Environment to trigger decisions by the team of experts, which improve the LLM training. After each LLM training cycle, the QA test sequences are repeated until the best performing LLM, which meets the predefined quality expectations, is identified. The outcome of these tests is used for verification and validation of the trained model regarding its suitability for the target end-user community. The QA Environment is a totally separate environment from the Development Environment, to avoid unwanted interference from parameters and procedures required for developers, which has negative impact on the measurements necessary by the quality processes and procedures. It is also necessary that the roles implemented in the Development Environment are not replicated in the QA Environment, as in the QA needs to mimic the security and other constrains implemented in the Production Environment. Any Developer user and Test user operating in the QA Environment utilize the tool Application interfaces with alternating end-user profiles and alternating roles than any profiles / roles they may have had in the Production Environment.

[0137] 5.1.2.1 Training Method. Quality Assurance

[0138] Qualitative assessments always involve human expert engagement. Quantitative assessments primarily utilize Metrics and can receive a fair degree of automation, but involve at a lesser degree human evaluation as well. Utilizing these methods, the experts execute the required Quality procedures as defined in the training strategy and execute standardized experiments with pre-selected data sets, to validate the model performance. The second document subset, developed during the production of training document subsets, is dedicated to the use of documents within QA scenario. The said document subset is used to test the Trained LLM for performance within the predefined measurable quality criteria. The LLM is subjects to quality tests and the findings of the trained LLM responses are evaluated by the quality method. Any findings in the QA Environment are returned to the Production Environment as requests for improvement. The training work continues producing new and improved Model for the QA, in an iterative approach, till the model reaches the performance level defined in the training strategy. This iterative Training and QA process continues enhancing the model cognitive abilities until the QA team approves the quality in the Model responses as appropriate for the patent examinationprocess. Optionally at different rounds of this iteration cycle, intermediate versions of the model are produced for exploitation by specific user groups. Quantum Computing accelerates the training and QA process in specific demanding technical domains.

[0139] In the Development Environment, when the model performance meets the defined user expectations, it is ready to be deployed for Quality Assurance. The Quality Assurance Environment of the invented tool is able to accommodate Test-users operating the model in test mode with different roles. They utilize user accounts designed with privileges and security access appropriate for each role, by allocating appropriate access rights, so that they replicate a “real life” use scenario. In this situation, the LLM is assessed whether it meets standardized performance requirements and protects the security of information and data of the system, before being released to production. The quality procedures engaged in LLM training, develop systemic and measurable criteria, leading to systematic adherence to quality of the invented tool. Such criteria are utilized in critical decisions such as whether the LLM has the efficiency and delivers the user experience expected by the endusers. When the quality measurements meet the criteria of a specific use case, the LLM is subject to version control and is prepared for transfer to the Production Environment as a trained LLM. Consequently in the Production environment the end-user finds several versions of LLMs trained for different use cases. The quality approach defined in this invention is a distinctive feature, when compared with the prior art. The aim is for the deployed model to provide a significant degree of assurance that it meets or exceeds the criteria for the purpose of usage.

[0140] This approach to quality is to be applied for the final product as well as for any product components as referenced above, which product components offer suitable functionality for specific use scenarios. So any product component has the same development and quality approach, with just the specific product parameters been adjusted for the purpose to be fit for usage. For instance for Researchers a model that has a reasonable degree of hallucination is useful to trigger “out of the box” thinking responses and reasoning with combinations of normally unrelated entities. For example, a researcher at the stage of drafting ideas for an innovation is not interested in the legal framework and the relevant regulations of the patenting process. The researcher is primarily particularly interested in how research ideas may be formulated when they are considered as a potential innovation. In these cases, end users prefer a faster response from the invented tool. Also, a model that has a degree of hallucinations may be useful for inducing unpredictable reasoning in researchers through combinations of seemingly unrelated technical features. The methods according to the present invention support these varying end-user requirements, with the assumption that the tools used are appropriate for the end-user's purpose.

[0141] When the QA measurements indicate that an LLM meets the predefined expectations, it is ready to be used in the Production Environment. Before transferring the LLM to the Production Environment it is cleaned from any code, data and settings, which were used for the purpose of training and assessing compliance to quality.

[0142] In case QA results indicate that models reach training saturation without achieving the said quality objectives, the Quality experts may decide to deploy multiple models for complex technical domains. As an example it is possible that some LLMs may respond efficiently for the search for prior art, while for text synthesis and text summarization another LLM and / or LLM architecture may have better performance.In this case the LLMs form a modular architecture and utilize appropriate APIs, whereby a master LLM interacts with different slave LLMs. As a result the tool performance is optimized, by synthesizing the results from different LLMs, forming a comprehensive response for the end-user. However this modular architecture does not form part of the invention as claimed.

[0143] 5.1.2.2 Outcomes from Training and QA Methods

[0144] During the Training and QA procedures, several partly trained LLMs are produced as usable tools, as explained below. Such partly trained LLMs are produced by adjusting the parameters defined in the training strategy, which governs the fine tuning of the LLM. Each partly trained LLM achieves the defined objectives by following a scientific approach with high adherence to quality in training LLMs specifically with Patent related information and procedures. This invention provides the appropriate methods and tools for implementing what is required by the different end-user groups.

[0145] As described above, there are different outcomes from the LLM training and fine tuning. The innovative approach to LLM training defined by this invention includes in particular as typical outcomes the following LLMs, which are specialized for specific patent procedures:

[0146] • LLM Trained on specific technical domains

[0147] • LLM Trained in specific Patent Legal Framework procedures of one jurisdiction.

[0148] • The previous two bullets compose a baseline of LLM suitable for Patent examination procedures, (see Figure 6.1 and Figure 6.2)

[0149] • LLM Trained as baseline for one or more technical domains with enhanced training based on valid query and response pairs with special structure to enable analysis of complex reasoning supported by intermediate analysis steps. Said special structure for each query and response pair, is the decomposition of complex technical arguments, used in official correspondence. Such complex technical arguments are decomposed in a series of questions and valid response pairs for each step of the argumentation. This way the complexity of the argumentation is simplified, and a cluster of known pairs with valid questions and response can be formed. This model utilizes the training data related to a specific technical domain including information from the Global Dossier required for training in enhanced reasoning as well as additional information defined by this invention, such as metadata, to fine tune the model responses and increase the precision of the information provided to professionals as end-users, (see Figure 6.3)

[0150] • LLM trained at professional level with knowledge augmented by proprietary patent related information, (see Figure 6.4)

[0151] • LLM trained at professional level with augmentation of proprietary data and additional augmentation of proprietary administrative procedures, (see Figure 6.5) Figure 4 provides an illustration of outcomes from LLM Training and a representation of the progressive improvements of the LLM capabilities.

[0152] 5.1.3 The Production Environment

[0153] The invented tool is trained to process patent information according to the articles and rules of a specific jurisdiction. Thus the invention utilizes a Data Governance framework. Data governance framework is commonly defined as a set of rules, processes, and responsibilities that dictate how data is managed (i.e. collected, organized, stored, and used), and how data is protected from security risks. By allocating specific access rights touser roles the system is protected from unauthorized data updates and erroneous introduction of new documents. Additionally the invented tool includes any related Patent Legal Framework as well as ethics and compliance considerations. The data management relevant to the LLM training is described in the section titled “LLM training method”. The Production Environment utilizes the cleaned and trained LLMs, which are the outcome of successful QA results. In the Production Environment each user is assigned appropriate identity and profiles. Developer-users and Test-users, who had as main objective Training and QA, may still operate in the Production Environment as Productionusers, after replacing their former profiles and identities by assignment of new user identities and profiles, appropriate for this environment. Production environment users may have a range of different profiles, associated to different user roles and are granted access to one or more of the trained LLM versions developed in the Production Environment. Figure 5 provides an illustration of the network and logical architectures for the computing environments described in this invention.

[0154] 5.1.4 Tool Life-Cycle Management

[0155] The invention realizes that the proposed Tools and Methods are influenced by two major industrial developments. The first is the Software Development Life-Cycle, which affects the Software Components and the second is the Production System Operation Life-Cycle which is affected by the Artificial Intelligence Developments and Operations. Both have a dynamic roadmap and using the most current and / or effective version is in the interest of the end-users. To improve the tool quality and effectiveness, the invented tool incorporates a Life-Cycle Management Method.

[0156] Said management method incorporates data collection and operating procedures for the purpose of periodic assessment for further improvements as well as for assessing the need to update the tool with the technological improvements and the revised Patent Legal Framework. A non-exhaustive list of related attributes for supporting said Life-Cycle Management is the following:

[0157] • CMDB

[0158] • Logging

[0159] • Real Time and Historical data collection

[0160] • Metrics

[0161] • Performance Analysis

[0162] • Anomalies Detection

[0163] • Information Correlations and Contextualization

[0164] • RunBooks and ChatOps

[0165] Consequently the tools in the Production Environment have a systematic approach in collecting usage statistics. Teams of experts evaluate the performance data to form an assessment whether the tools and methods are still fit for the purpose of usage. When there is deviation either due to technology evolution or to important changes in the Patent Legal Framework, or the applicable security and administrative policy and / or procedures, a retraining process is initiated. Obviously due to significantly lower volume of data this new cycle of training is a less complicated process that the initial LLM training. At this round of re-training, the models are trained on valid prompts and responses produced by their operation so far, increasing further their response precision. The tool is subject to the same training methods with suitable data to improve its performance and technical architecture.The invented tool implements IT services adhering to a modular architecture following Information Technology Infrastructure Library (ITIL) recommendations for IT Service Management. As a result the IT services adhere to specific Key Performance Criteria, expressing the user performance expectations. This scientific approach expedites the human work efforts and supports the end-users in their follow on steps to define an innovative idea. Failures, Anomalies and Performance Issues are systematically recorded in order to support the Life-Cycle Management decisions. Such decisions lead to generation of new evolution cycles for the invented tool. This is an inventive characteristic of the tool design, which is an essential element for sustaining the effectiveness of the invented concepts, especially in fast evolving technological domains.

[0166] When considering the technological improvements, one particular area of interest is the anticipated development of Generative Artificial Intelligence (GenAI) and Retrieval Augmented Generation (RAG). It is anticipated that the technological maturity will pennit in reasonable time duration the GenAI and RAG to generate text expressing ideas with high assurance that a Patent Claim based on said text has high expectations to be granted. In such a case the invented tool through its Life-Cycle Management is considered to be ready to incorporate the GenAI / RAG technologies. For example, when a Patent Claim proposed by the inventor is identified by the invented tool as not been new and inventive against existing prior art, a new Claim can be proposed by the GenAI / RAG, which has high probability to be granted.

[0167] 5.1.5 Benefits by Using the invented tool

[0168] The invented tool is used to assist the end-users in the following procedures:

[0169] • Prior art documentation retrieval including documents written in unknown languages to the user, which are included in diverse sources not otherwise known to the user. Retrieval of such documentation assures thoroughness of search for prior art,

[0170] • Objective intelligent scientific analysis at response speeds and within time frames that are impossible for manual labor. The tool delivers repeatedly high quality results,

[0171] • Creating scientifically documented assessments about novelty and inventive step, based on the tool’s ability to apply contextual understanding. Said assessments are formulated with unique linguistic expressions appropriate to a specific Patent class,

[0172] • Instruct the tool to prepare text sections that could be used in a patent application document. In addition the model can be instructed to list all sources and cross- references taken into account for preparing said text section according to enduser requirements.

[0173] • Integrating the methods of the invention with other tools and in formulating complex procedures by using modular architecture. Modular architecture enables development of suitable interfaces, such as web-interfaces and Application Programming Interface (API), to permit both cooperation among the invented tool components and cooperation of the tool with other systems.

[0174] • Utilizing multi-layer security. Security is built by design in the tools and in the process automation. This security concept protects data at rest and in transit, identifies and validates end-users and devices, maintains appropriate logs to trace events for analysis in case of a suspected security incident, and complies with the applicable security standards and guidelines. The security component is designedto evolve in response to changes in the threat environment and the evolution of applicable standards and regulations.

[0175] • Utilizing IT services for adherence to user performance criteria, by implementing IT Service Management (ITSM). The methods proposed by the invention are structured to be components of IT services, consumed by the end-users maintaining the expected service performance criteria.

[0176] 6. DESCRIPTION OF EMBODIMENT

[0177] A number of exemplary non limiting embodiments are presented below. The first embodiment is system oriented. The second embodiment is related to LLM training for a specific technical field.

[0178] 6.1 First Embodiment.

[0179] Figure 5 provides a visual representation of the network and logical architecture of the computing environments implementing the invented tool.

[0180] The invented tool comprises of the following components:

[0181] • Centralized control and management unit for information security, communications security, policy enforcement, administration, Quality Assurance, Global Policy and Procedures. In this embodiment (see Figure 5) the architecture represents a diagram of the environments required for use of the invented tool by either a patent related organization (O) and / or a Researcher / Inventor / Patent Consultant (E). Each enduser has controlled access to specific components of the overall architecture. Said components are resident in the Development environment. Each end-user may perform more than one role.

[0182] • The Development environment with VRAM as defined in paragraph (A), other technical characteristics as described below and as represented graphically by Cloud 0 and 1 in Figure 5.

[0183] • The QA environment with VRAM as defined in paragraph (B), other technical characteristics as described below and as represented graphically by Cloud 2 in Figure 5. When the LLM reaches the required performance an instance of a Trained LLM is cleaned, e.g. from data and code required for training and QA processes. The cleaned LLM is transferred to the Production environment.

[0184] • The Production environment with VRAM as defined in paragraph (C), other technical characteristics as described below and as represented graphically by Cloud 3 in Figure 5. It is the Production environment that provides access to enduser communities for professional exploitation. The LLM version in the Production environment does not permit changes on the LLM parameters.

[0185] • Production environment represented graphically by Clouds 4 and 5 in Figure 5, form an example of the invented tool, which is isolated at hardware level on systems accessed exclusively by specific end-users, such as large corporation, Patent Offices, etc. Cloud 4 is a Private Cloud typically utilized by an Organization, which has selected a cloud solution for implementation of their Production environment, utilizing proprietary Technical Documents and / or bespoke Patent application procedures. Accordingly the graphical representation of cloud 5 is a Private Production environment for an organization, which has opted for an onpremise solution for their Production environment.By way of example, a valid option for the implementation of the invented tool is to use rented server farms with the most updated CPU technology and with the following types of 980 resources:

[0186] • 2.5 Terabytes of VRAM

[0187] • 200 Terabytes storage space. The storage space is shared among all environments and space is dynamically allocated. Said storage space is allocated under strict security rules to isolate the resources of the 3 operating environments described 985 below.

[0188] • The communication infrastructure with its security and firewall components is dictated by applicable laws, policies and standards.

[0189] The resources described below are needed for an exemplary implementation of the 990 invented tool.

[0190] A. Development Environment with 1.3 Terabyte of VRAM. This environment utilizes a number of pre-trained LLMs, which are trained and fine tuned for patent related procedures, using the methods described in paragraph 4.1.1. Examples of pre-trained 995 LLMs, which can be used are listed below:

[0191] • GPT3 with allocation of 350 GB VRAM

[0192] • Bloom with allocation of 355 GB VRAM

[0193] • Llama-2-70b with allocation of 140 GB VRAM

[0194] 1000 • Falcon-40b with allocation of 80 GB VRAM

[0195] • MPT-30b with allocation of 60 GB VRAM

[0196] Obviously, for each technical field, the optimum list of pre-trained LLMs is selected at the implementation time.

[0197] 1005

[0198] B. A QA Environment with 800 GB of VRAM, which makes requests to dynamically increase the VRAM size with VRAM from the pool, which is non allocated to another environment. For example when there is work concentration in the QA environment there is less emphasis in the Development environment, so VRAM can be re-allocated 1010 from one environment to the other.

[0199] The planning is to be able to operate at least 2 of the above models with higher VRAM requirements simultaneously in QA. When required, more models are employed in the QA Environment.

[0200] 1015

[0201] C. A typical size of VRAM for the Production Environment is around 600 GB.

[0202] D. A Private Production environment proprietary to a specific user community, e.g. a large corporation, a Patent Office, which wants to protect their intellectual property data, can 1020 be realized either on Private Cloud or On-Premises. These organizations determine the architecture of their environments.

[0203] 6.2 Second embodiment

[0204] This embodiment is presented as an implementation scenario demonstrating 1025 implementation methods and techniques for the selection of training documents and innovative techniques to produce proprietary training data.6.2.1 Implementation Scenario

[0205] 1030 In the Development Environment described in the first embodiment section, a team of Al technical experts and experts in Patent examination in the Mechanical and Electrical technical domains select documents to be used for performing fine-tuning training of a selected pre-trained LLM preserving ethical and legal guidelines.

[0206] 1035 In this embodiment, the pre-processing of documents to become training data is presented.

[0207] Said document pre-processing presentation is not a comprehensive implementation procedure. It demonstrates the logic involved and focuses in an innovative step of this invention, which is the generation of Registries and Meta Data. A visual representation of the process flow including Meta Data is in Figures 6.1 to 6.5.

[0208] 1040

[0209] The experts decide for example that for the selected technical domain, 15% of the related documentation will be used for supervised training, 65% for automated training and 20% for QA procedures. Based on the decided percentages the full set of available documents of the particular technical domain are allocated numerical values such as the values of the 1045 qualitative criteria listed below:

[0210] • Supervised Training: numerical Value 1

[0211] • Automated Training: numerical Value 2

[0212] • QA Procedures: numerical Value 3

[0213] 1050

[0214] Based on the allocated numerical value assigned to each document, the subsets for supervised training, automated training and QA procedures are generated.

[0215] The training team then decides which qualitative criteria are to be allocated on each 1055 document of the subset for supervised training. This decision is based on logical interpretations regarding the contextual understandings of said technical domain. The associated numerical values are allocated on the selected documents for supervised training. An example of qualitative criteria and numerical values is demonstrated in the list below.

[0216] 1060

[0217] Author Importance (for training objectives) with qualitative values

[0218] • Very Important Author: numerical value 2,

[0219] • Average Author Importance: numerical value 1,

[0220] • Rarely seen Author: numerical value 0.

[0221] 1065

[0222] Document Category with qualitative values

[0223] • Important document for manual training: numerical value 2,

[0224] • To be used for automated training: numerical value 0,

[0225] • To be used for QA: numerical value 1.

[0226] 1070

[0227] Document Priority with qualitative values

[0228] • To be used in the initial training batch: numerical value 1,

[0229] • To be used for QA improvements: numerical value 0.

[0230] 1075 Document Significance with qualitative values

[0231] • Document with significant concepts and / or expressions: numerical value 1,Document with no significant importance: numerical value 0, Document with potential to create training errors and issues, which is still of interest: numerical value -1.

[0232] 1080

[0233] The selected documents are cleaned from harmful and erroneous content.

[0234] The selected documents for either supervised or automated training are converted to plain text, so that they are suitable as training data.

[0235] 1085 For the documents selected for either supervised or automated training there is a number of processing procedures executed. These procedures generate metadata, which is stored in appropriately formatted data bases and registries.

[0236] Three basic types of metadata are generated, while more types are produced for specific 1090 categories of raw data:

[0237] • Descriptive metadata e.g. data describing the information such as titles, volume data, links to bibliographic information described below,

[0238] • Administrative metadata e.g. data that enable administrators to impose rules for access rights, data processing, data masking, data maintenance, etc.

[0239] 1095 • Structural metadata e.g. data about the structure of tables, images, video, complex formulas and mathematical expressions, etc.

[0240] Said metadata is used for procedures such as the items listed below:

[0241] • Retrieval of exact documents and / or exact document components (images, 1100 drawings, etc.)

[0242] ° Support deeper insight in the textual content

[0243] ° Facilitate document cross referencing

[0244] ° Assist in data discovery

[0245] • Data organization to support processes and procedures, like the qualitative criteria 1105 described above.

[0246] • Information security to prevent unauthorized and / or unethical data access and manipulation

[0247] A data base is used for example for storing bibliographic data. Bibliographic information is 1110 stored in specialized bibliographic data bases due to the information complexity. The stored data includes links to the original document. Such architecture permits the retrieval of said original document. To reduce errors during document retrieval, the tool employs “Unique Author Identification” (UAI), when UAI is available, to select only documents by the specific Author and avoid confusion with synonyms or other similarities.

[0248] 1115

[0249] When a registry for said metadata is formatted as a table, the table may have the following table column headings:

[0250] • Document ID

[0251] 1120 • List of Unique Author IDs or Author names

[0252] • List of references to Drawings

[0253] • List of references to Tables

[0254] • List of references to formulas

[0255] • Number of pages

[0256] 1125 • List of bibliographic referencesReference to Qualitative Criteria

[0257] Comments

[0258] Registries and data bases with Meta information about documents are used for processes 1130 and procedures related but not limited to the methods listed below

[0259] • Conversion of documents to LLM training data

[0260] • LLM Training and fine-tuning

[0261] • Prior art retrieval

[0262] • LLM response precision

[0263] 1135 • Support and improve data analytics

[0264] • Text synthesis

[0265] • Text analysis

[0266] The descriptions above are only preferred embodiments of the present invention, which are 1140 not used to limit the present invention. For a person skilled in the art, the present invention may have various changes and variations. Any modifications, equivalent substitutions, improvements etc. within the principle of the present invention as defined by the claims, will be considered as falling within the protection scope of the present invention.1145 References Signs and Terminology List

[0267] Acronym Description

[0268] Al Artificial Intelligence

[0269] API Application Programming Interface

[0270] 1150 ChatOps Is the use of chat clients. It is also known as conversion driven DevOps as well as conversation driven collaboration. Examples of chat clients are chatbots and other real-time communication tools to facilitate IT Operations and Software Development.

[0271] CMDB Configuration Management Data Base

[0272] 1155 CPU Central Processing Unit

[0273] CPC The Cooperative Patent Classification (CPC) is an extension of the IPC and is jointly managed by the EPO and the US Patent and Trademark Office. End-User The user who works with the invented tool operable to perform the invented method. End-Users normally work in the Production Environment. The 1160 End-User may also work in the Quality Environment to assess suitability of methods and tools for use, prior to releasing them in the Production Environment.

[0274] Ethical Standards The right governance of Al exploitation is one of the great challenges of our time, as it is a new technology generating products and services with 1165 big impact to many aspects of life on the planet. One of the more comprehensive definitions for Ethical Use of Al are the Ten Core Principles, which lay out a human-rights centered approach to the Ethics of Al, defined by UNESCO. See https: / / www.unesco.org / en / artificial- intelligence / recommendation-ethics.

[0275] 1170 Global Dossier As defined by WIPO, the Global Dossier is a secure Public Access Dossier service, offering online access to the file histories of related applications from participating IP Offices.

[0276] ICT Information and Communication Technology

[0277] IPC The International Patent Classification (IPC)

[0278] 1175 Invented Method It is also expressed simply as the method and the methods. The term is a collective expression for the methods invented for the three working environments, to achieve the outcomes described in the disclosure of this invention and in the claims of this invention.

[0279] Invented Tool It is also simply expressed as the tool. The term has been used in the 1180 document as a short form of expression. It represents the three tools as defined in the relative documents sections, which are resident in the three environments (Development, QA and Production) defined by the invention and the associated methods for each environment.

[0280] ITIL Information Technology Infrastructure Library

[0281] 1185 ITSM Information Technology Service Management

[0282] LLM Large Language Model

[0283] QA Quality Assurance

[0284] Roles User roles are defined as the different way end-users may interact with the system to achieve certain tasks. A role is defined as an entity that can be 1190 shared between one or more actual end-users. One end-user may also be in more than one roles. Examples of roles are: Researcher, Patent Attorney, Patent Applicant, Patent Examiner, System Developer, LLM Trainer, LLM quality testerRunBook RunBook is a set of standardized written procedures for completing 1195 repetitive information

[0285] Tokenization Tokenization, in the realm of Artificial Intelligence (Al), refers to the process of converting input text into smaller units or 'tokens' such as words or subwords. This is foundational for Natural Language Processing (NLP) tasks, enabling Al to analyze and understand human language.

[0286] 1200 User Any person or process that utilizes the functionality of the tools available by the patented tools to achieve an objective

[0287] VRAM Video Random Access Memory

[0288] UAI Unique Author Identification

Claims

Claims1. LLM training method for creating an intelligent tool to support the patent granting process, where the method includes:- selecting a set of pre-trained LLMs that are suitable for training in a predetermined technical field,- defining the architecture of the preselected LLMs based on predetermined critical factors,- determining the training strategy for each of the preselected LLMs, according to the complexity of the respective technical field, the training techniques of the respective LLM based on the end user's requirements, the available computing resources, and the training duration,- selecting classified patent-related documents in the selected technical field, as well as non-classified technical field related documents, and documents on the patent legal framework, in order to form a set of documents for the training of each LLM,- using measurable qualitative and quantitative criteria for the selection of documents for LLM training,- using qualitative criteria to partition said documents into three nonoverlapping subsets, based on percentages determined by quantitative criteria, creating a first subset for supervised training, a second subset for quality assurance procedures, and a third subset for the automated training for each LLM,- using qualitative criteria for selecting documents from the first and third subsets from a technical domain, for fine-tuning each LLM using iterative training cycles for progressive improvement of its learning,- defining quality assurance procedures to be applied to each training cycle, characterized in that unique proprietary data is created from metadata of said documents, said metadata adapted for:- bibliographic management of documents to achieve a low degree of Al hallucinations,- data management for- the creation of said qualitative and quantitative criteria of documents, - the creation of cross-references and indexes, and- the composition of valid questions and answers sets, and - data security to ensure- the reliability of documents- the use of data within the rules of ethical use of ML models,and where said metadata is stored in structured auxiliary files.

2. The LLM training method according to claim 1, wherein from each selected training document the data, images, and / or existing multimedia information related to the document are enhanced with metadata, separated from the text and stored in dedicated file libraries with appropriate annotation, cross-references, and indexes.

3. The LLM training method according to claim 2, wherein the content and structure of the documents are prepared as data for LLM training by cleaning the documents of harmful and / or erroneous information that impedes LLM learning, and by converting said documents into plain text.

4. The LLM training method according to claim 1, wherein legal and administrative documents of the patent legal framework relevant to a particular jurisdiction are prepared as data for LLM training by cleaning said documents of harmful and / or incorrect information that hinders LLM training and by converting said documents into plain text.

5. The LLM training method according to anyone of the claims 2 to 4, wherein the respective LLM is trained with said data and the performance of the LLM is gradually improved by using at least one of the techniques of instruction fine-tuning, parameter fine-tuning, chain-of-thought learning, reinforcement learning, and hybrid methods.

6. The LLM training method according to claim 5, where the responses of the LLM are optimized by training the LLM with valid question-answer pairs derived from official correspondence during the patent granting process.

7. The LLM training method accordingto claim 6, wherein the LLM is iteratively updated using data related to the current version of the patent legal framework, published documents related to technological developments since the previous LLM training cycle, and published documents related to patents since the previous LLM training cycle.

8. The LLM training method according to claims 6 or 7, wherein the responses from the trained LLMs are subject to quality control for compliance with predetermined measurable quality criteria, wherein said quality control includes:- defining the quality approach and quality management procedures in the LLM training strategy,- selecting the best performing of the preselected LLMs,- developing different user roles by creating user profiles and access rights, - performing quality assurance tests with human supervision, both repetitive and automatic, until acceptable LLM performance is achieved based on enduser quality requirements, so that the optimal LLM can be transferred to the production environment.

9. The LLM training method according to claim 8, where key performance indicators are created, which are influenced by the Software Development Life Cycle and / or the Production Systems Operation Life Cycle, and which are compared to predetermined measurable end-user expectations to determine the need to update the LLM training.

10. The LLM training method according to any of claims 1 to 9, wherein the LLM has been trained to operate within certain quality criteria, and to:- identify the prior art related to a technical solution described in a set of claims of a patent application which is disclosed in patent related documents and / or other relevant documents,- evaluate the novelty and inventive step of the drafted claims in relation to the prior art, and to:- assist in drafting parts or the whole of a patent application, and / or - simulate the expected dialogue between the applicant and a patent examiner, and / or- identify the claimed subject matter, which is classified as either explicitly disclosed or implicitly derived by logical reasoning from the prior art, or not.

11. An intelligent tool operable to perform the methods described in any of claims 1 to 10, comprising:- a process management unit suitable for the methods of claims 1 to 10, - data storage and video RAM (VRAM) suitable for selected LLMs,- an information and communications security module suitable for protecting the secrecy when using the tool,- a modular architecture with application programming interfaces and web interfaces for cooperation between the components of the claimed tool and for cooperation of the tool with other systems,- a server farm for the development environment, specialized for data preparation, software development, LLM training, and LLM fine-tuning, - a server farm for the quality assurance environment, specialized for the execution of quality management methods, and for the validation and verification of LLM performance, and- a server farm for the production environment, the server farm comprising of the operational tools specialized for the use of trained LLMs by end users.

12. The tool according to claim 11, wherein said server farms use advanced processing units to accelerate specialized training in complex technical fields, complex technical reasoning, and prior art retrieval.

13. The tool according to any of claims 11 and 12, wherein said server farms include architectures based on quantum computing technology.

Images