Method for indexing documents and for information retrieval therefrom

Abstract

The present invention relates to a computer-implemented method for indexing at least a portion of a document. The method comprising determining a first representation using one or more keywords of said portion of the document, said keywords being selected by a model as being relevant syntagms; indexing said first representation thereby updating a first index; obtaining a summary for said portion of the document; from said summary, determining a second representation as a pair of a vectorial embedding of said summary and a vectorial embedding of said portion of the document; indexing said second representation thereby updating a second index.

Description

[0001] METHOD FOR INDEXING DOCUMENTS AND FOR INFORMATION

[0002] RETRIEVAL THEREFROM

[0003] TECHNICAL FIELD

[0004] [1] The present disclosure relates to the field of computer programs and systems. More specifically, the present disclosure relates to a method, system and program for indexing a document and / or a portion thereof. Furthermore, the present disclosure relates to a method, system and program for information retrieval from one or more documents.

[0005] BACKGROUND

[0006] [2] Significant increase in the volume of data in combination with a need to update them rapidly enough make traditional search and indexing systems struggle. On the one hand, the challenge is to keep the scalability. On the other hand, the challenge is to continuously updating data without causing latency problems. This is particularly true in systems requiring real-time or near-real-time data access.

[0007] [3] Keyword-based indexing solutions, while being fast, often fail to capture the full context of information. This results in less accurate and less relevant search results as well as incomplete search results.

[0008] [4] Hierarchic indexation techniques are also often used but may lead to a poor definition of indexation layers resulting in redundant or incomplete information.

[0009] [5] Semantic indexing frameworks have emerged as powerful tools for query resolution, offering superior contextual understanding and intent recognition. These systems are particularly adept at handling complex or natural language queries, accommodating synonyms and lexical variations with high resilience. Moreover, they can support multilingual queries effectively, assuming the underlying embedding models have been trained on diverse linguistic datasets. However, these systems are inherently limited by their reduced precision for exact keyword matches and their inability to effectively handle rare or domain-specific terms, posing challenges in applications requiring high specificity. [6] A significant limitation identified in the prior art is the reliance on static, sizebased chunk segmentation, which prevents from semantic coherence, leading to overly dense embedding spaces where single chunks may encompass multiple distinct topics. Such fragmentation results in suboptimal vector representations, necessitating hierarchical indexing as a compensatory mechanism. However, the hierarchical indexation needs substantial computational and memory overhead without fully resolving the issue of context fragmentation.

[0010] SUMMARY

[0011] [7] This disclosure relates to a computer-implemented method for indexing at least a portion of a document. The method comprises determining a first representation using one or more keywords of said portion of the document. Said keywords are selected by a model as being relevant syntagms. The method further comprises indexing said first representation thereby updating a first index. The method further comprises obtaining a summary for said portion of the document. The method, from said summary, determines a second representation as a pair of a vectorial embedding of said summary and a vectorial embedding of said portion of the document. The method then indexes said second representation. Thereby, the method updates a second index.

[0012] [8] Surprisingly, the Applicant found that combining a first representation using keyword approach and a second representation resulting from a summary of the document and the document itself, leads to unexpected results in terms of efficiency in indexing documents. Moreover, the Applicant found that using this technique in the framework of information retrieval, in a system combining a LLM together with a RAG architecture implementing at least one of parsing services, data storage services and / or indexing services, lead to very accurate and relevant responses to the queries.

[0013] [9] The method proposes a hybrid indexing solution combining keyword indexing and a semantic hierarchical structure. This improves search relevance while maintaining performance and scalability. In examples, the method may comprise storing, in a database, said portion in correspondence to the obtained summary.

[0010] It is also provided a computer-implemented method for indexing a document. The method comprises partitioning the document into one or more portions, each portion being defined by a respective context. The method further comprises indexing each portion using the method discussed above for indexing at least a portion of a document.

[0014]

[0011] It is also provided a computer-implemented method for information retrieval from one or more documents. The method comprises providing one or more groups of first indexes and second indexes, each group being obtained for a document of the one or more documents using the method for indexing a document. The method further comprises providing a request to retrieve information, determining, using the first indexes of the provided one or more groups, a first list of documents in relation to the requested information; and determining, using the second indexes of the provided one or more groups, a second list of documents in relation to the requested information. The method yet further comprises generating a response to said request using the determined first list of documents and the determined second list of documents. In one embodiment, the architecture underlying the provision of the request to retrieve information and the generation of response is a RAG structure implementing at least one of parsing services, data storage services and / or indexing services. The parsing service for document management used according to the invention supports uploading of documents in various formats, including PDF, Word, PowerPoint, and Excel, with secure download management to ensure data confidentiality; extracts relevant information through methods such as OCR for scanned documents, structured data extraction from spreadsheets, and advanced text analysis to capture key content; segments documents into semantically coherent chunks using a hierarchical and semantic-aware chunking mechanism, enabling finer-grained indexing and enhanced information retrieval while preserving document structure and context. The data storage service that may be used according to the invention for securely storing and managing parsed document data, ensures data protection and availability through mechanisms such as redundancy and backups; enables efficient document retrieval using unique identifiers for rapid access and querying; regularly transmits stored data to an indexing service, facilitating ongoing updates for optimized search and analysis within a retrieval-augmented generation system. The indexing service that may be used when implementing the invention, for optimizing document search and retrieval, creates and updates a dual -hi erar chi cal index structure integrating keyword-based and semantic indexing methods to enhance query precision and relevance; represents document chunks as vector embeddings to enable context-aware similarity searches and ranking of results based on relevance criteria such as keyword frequency and metadata importance; supports scalability and adaptability by allowing the dynamic integration of new indexes or deletion of existing ones without data loss, ensuring system evolution in response to growing data volumes and changing user requirements.

[0015]

[0012] According to examples, the information retrieval method includes one or more of the following features, taken alone or in any possible combination:

[0016] Obtaining two reduced lists of documents by filtering each of the first list of documents and the second list of documents using a relevance score measure, the relevance score measure representing the relevance of a document to the request, wherein the generating of a response to said request using the determined first list of documents and the determined second list of documents comprises using said two reduced lists of documents; obtaining a further reduced list of documents by filtering said reduced first list and said reduced second list, wherein the generating of a response to said request using the determined first list of documents and the determined second list of documents comprises using said further reduced list of documents;

[0017] - the relevance score measure is a BM25 score;

[0018] - the provided request is in a human-language syntax; and / or

[0019] - the method further comprises providing a large language model (LLM), and the generating of a response comprises outputting a response using said provided LLM, the determined first number of documents and the determined second number of documents.

[0020]

[0013] The present disclosure also relates to a computer program comprising instructions for performing any method as above.

[0021]

[0014] The present disclosure further pertains to a (e.g., non-transitory) computer readable storage medium having recorded thereon the computer program.

[0015] Such a non-transitory program storage device can be, without limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device, or any suitable combination of the foregoing. It is to be appreciated that the following, while providing more specific examples, is merely an illustrative and not exhaustive listing as readily appreciated by one of ordinary skill in the art: a portable computer diskette, a hard disk, a ROM, an EPROM (Erasable Programmable ROM) or a Flash memory, a portable CD-ROM (Compact-Disc ROM).

[0022]

[0016] The present disclosure further pertains to a system comprising a processor coupled to a memory and preferably a graphical user interface, the memory having recorded thereon the computer program.

[0023]

[0017] It is further provided a device comprising a computer readable storage medium having recorded thereon the computer program. The device may form or serve as a non- transitory computer-readable medium, for example on a cloud-based platform, or the like. The device may alternatively comprise a processor coupled to the data storage medium. The device may thus form a computer system in whole or in part (e.g. the device is a subsystem of the overall system). The system may further comprise a graphical user interface coupled to the processor.

[0024] DEFINITIONS

[0025]

[0018] In the present invention, the following terms have the following meanings:

[0026]

[0019] The terms “adapted” and “configured” are used in the present disclosure as broadly encompassing initial configuration, later adaptation or complementation of the present device, or any combination thereof alike, whether effected through material or software means (including firmware).

[0027]

[0020] A “Retrieval-Augmented Generation” or RAG model is an architecture model. This hybrid model combines two steps of information retrieval and text generation to improve the quality and relevance of the responses produced adapted to the context. The first step is an information retrieval step. The goal of this first step is querying a database or index in order to extract relevant documents in response to a request. The second step is the text generation in which the extracted relevant documents is used by an advanced text generation model, like an LLM, to generate a fluid, context-sensitive response to the query. In this way, the extracted data is used to enrich the context of the generated response. The extracted data may be provided as part of a prompt with which an LLM is called. Instead of relying solely on knowledge derived from the training data, a RAG architecture exploits relevant information and connects static LLMs with real-time data retrieval. By leveraging large document datasets efficiently, RAG improves the relevance and accuracy of answers, facilitating the accurate extraction of information and the generation of context-sensitive tailored to the context.

[0028]

[0021] As known per se in the field of computer science and database management (see element or a plurality of elements” it is meant obtaining or determining a group of one or more indexes for said element or each element of said plurality of elements. The indexing further comprises storing said obtained group of one or more indexes in a local or a remote database. Thereby indexing seek to facilitate fast and accurate information retrieval.

[0029]

[0022] A “syntagm” or “syntagma” is a linguistic unit consisting of a set of linguistic forms (phonemes, words, or phrases) that are in a sequential relationship to one another. In linguistics, a syntagma is an elementary constituent segment within a text. Such a segment can be a phoneme, a word, a grammatical phrase, a sentence, or an event within a larger narrative structure, depending on the level of analysis.

[0030]

[0023] The term “processor” should not be construed to be restricted to hardware capable of executing software and refers in a general way to a processing device, which can for example include a computer, a microprocessor, an integrated circuit, or a programmable logic device (PLD). The processor may also encompass one or more Graphics Processing Units (GPU), whether exploited for computer graphics and image processing or other functions. Additionally, the instructions and / or data enabling to perform associated and / or resulting functionalities may be stored on any processor- readable medium such as, e.g., an integrated circuit, a hard disk, a CD (Compact Disc), an optical disc such as a DVD (Digital Versatile Disc), a RAM (Random-Access Memory) or a ROM (Read-Only Memory). Instructions may be notably stored in hardware, software, firmware or in any combination thereof.

[0031]

[0024] By “providing X” to a method, it is meant that the method obtains or receives X or becoming able to have access to X. This may be realized, for example, when X become available by being stored / transferred / downloaded to a memory to which the method has access.

[0032] BRIEF DESCRIPTION OF THE DRAWINGS

[0033]

[0025] The present disclosure will be better understood, and other specific features and advantages will emerge upon reading the following description of particular and non-restrictive illustrative embodiments, the description making reference to the annexed drawings wherein:

[0034]

[0026] Figure 1 presents a general flowchart of the indexing method applied on a portion of document;

[0035]

[0027] Figure 2 present a general flowchart of the indexing method for a document;

[0036]

[0028] Figure 3 presents an example flowchart of the information retrieval method;

[0037]

[0029] Figure 4 presents a schematic of a RAG architecture;

[0038]

[0030] Figure 5 presents a global view of different services (or components of the method);

[0039]

[0031] Figure 6 presents a schematic diagram of the data flow among different components;

[0040]

[0032] Figure 7 presents a global view of different components;

[0041]

[0033] Figures 8A-D present an example of chunking;

[0042]

[0034] Figure 9 presents an example of information retrieval using a hierarchical index; and

[0035] Figure 10 shows an example of the system.

[0043]

[0036] On the figures, the drawings are not to scale, and identical or similar elements are designated by the same references.

[0044] ILLUSTRATIVE EMBODIMENTS

[0045]

[0037] The present description illustrates the principles of the present disclosure. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the disclosure and are included within its scope.

[0046]

[0038] Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.

[0047]

[0039] Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein may represent conceptual views of illustrative circuitry embodying the principles of the disclosure. Similarly, it will be appreciated that any flow charts, flow diagrams, and the like represent various processes which may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0048]

[0040] The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared.

[0049]

[0041] It should be understood that the elements shown in the figures may be implemented in various forms of hardware, software or combinations thereof. Preferably, these elements are implemented in a combination of hardware and software on one or more appropriately programmed general-purpose devices, which may include a processor, memory and input / output interfaces.

[0050]

[0042] In reference to Figure 1, it is provided a computer-implemented method for indexing at least one portion of a document. By a portion of a document, it is meant a chunk or a partition of said document, i.e., a continuous subset of the document. In examples, said portion may be the whole / totality of said document, a chapter / section / page of said document, or a paragraph or a sentence of said document. In other examples, said portion may be a context-based partition of said document. In other words, each partition is a collection of a plurality of words, sentences, paragraphs, or pages of said document sharing a common context and in sequential relationship to another. Said context-based partition may be obtained using any adequate context-based partitioning method. By a “context” it is meant a consistent set of propositional assumptions about which something can be shared. Such a set forms a theory of some topic, e.g., a theory of mechanics, a theory of the weather in winter, etc. In that sense, a context is called a "Microtheory." The scope of a context (the theory associated with the context) is the set of objects over which its predictions hold. In other words, the context of a document (or a portion thereof) refers to what is being discussed in said document (or said portion thereof).

[0051]

[0043] The method comprises, as step S10, determining a first representation (of said portion) using one or more keywords of said portion of the document. In other words, said first representation (of said portion of the document) is a keyword-based representation. Said keywords are selected by a model as being relevant (e.g., most representative) syntagms for said portion of the document. By “being relevant” it is meant that said keywords are representing (e.g., the most representing) words for said portion. Said keywords may comprise proper names, names, numbers, adjectives, and / or verbs. Said model for selecting the keywords is configured to receive a document or a portion thereof and output one or more selected keywords. In examples, said model (which is configured to select the keywords) may be a part-of-speech tagging (POST) model as known in the field.

[0052]

[0044] The document may be a text document. Alternatively, or additionally, the document may contain audio, image and / or video. In examples where the document contains image, audio or video file, the method may further comprise, prior to step S10, converting said file, or the at least a portion thereof, into a text file. Such a conversion may be done according to any known method in the field.

[0053]

[0045] The method, in step S20, comprises indexing said first representation thereby updating a first index. By updating a first index it is meant creating and initializing said first index if the index is not existing. On the other hand, said updating a first index may comprise updating an already-existing index if the index has been already created and has a value. The latter, for example, may be the case where the document (and therefore said at least portion thereof) is modified (e.g., due to one or more modifications in the document by one or more users). This modification necessitates to update the indexed first representation of said updated portion. The first index may be equivalently referred to as Index 1.

[0054]

[0046] The method, in step S30, obtains a summary for said portion of the document. The method may obtain said summary using any known method in the field of artificial intelligence (Al) and Large Language Models (LLMs).

[0055]

[0047] The method, then in step S40, determines, from said obtained summary in step S30, a second representation. The second representation is a pair of a vectorial embedding of said summary and a vectorial embedding of the document; said second representation is a semantic-based representation. As known, the semantic representation aims to embed the data (e.g., documents or portions thereof) into a vector space, i.e., maps the data into a vectorial embedding. This allows each data point to provide information (text, sentence, documents, etc.) and become a coordinate point. As discussed below in reference to the information retrieval method and Figure 3, during search time, the query input is processed into a vector via said embedding into the same vector space. The vectorial embedding may be obtained using any known vectorial embedding model known in the prior art.

[0056]

[0048] The method, in step S50, comprises indexing said second representation thereby updating a second index. Similar to the first index discussed above, by updating a second index it is meant creating and initializing said second index if the index has not existed before, or updating an already-existing index if the index has been already created and has a value. As the second representation comprises a pair of vectorial embeddings, the second index forms a two-level index, each level related to one of said two vectorial embeddings, i.e., one for the summary of the portion and another for the portion. The second index may be equivalently referred to as Index 2.

[0057]

[0049] This constitutes an improved solution to index a document (or a portion thereof). The advantage over the known methods is at least twofold: first, using a keyword-based indexing in parallel to semantic indexing improves the drawback of semantic indexing using vectorial embedding representations in capturing the details when they deal with particular categories like proper nouns. Imposing to a query an exact match with one or more keywords meaningfully improves the quality compared to pure semantic indexing querying.

[0058]

[0050] Secondly, using a two-level vectorial index which uses a vectorial embedding of a summary in addition to vectorial embedding of the portion of the document is of high interest as the information of the document is provided through two various sources, the summary and the document itself.

[0059]

[0051] In examples of the method, the steps of indexing may be performed together and upon obtaining the first representation and the second representation. In other words, the method may perform steps S20 and S50, upon completion of steps S10, S30, and S40. In these examples, the method may comprise one step of indexing, e.g., S60, as follows: indexing said first representation and said second representation thereby updating a first index and a second index, respectively. The advantage of parallelizing the indexing steps improves the indexing time.

[0060]

[0052] In examples, the method may further comprise storing, in a database, said portion in correspondence to the obtained summary. This summary provides a lightweight piece of data to assess the corresponding file and improves the time needed to access the relevant information inside the file.

[0061]

[0053] In reference to Figure 2, it is further provided a computer-implemented method for indexing a document. The method comprises, in Step S210, partitioning the document into one or more portions. Each portion is defined by a respective context. In other words, Step 210 comprises a context-based partitioning. The method then, in Step S220, indexes each portion using the indexing method discussed above. In other words, the method obtains a first / second representation for each portion of the document and for the document and then updates the existing respective first / second index using the abovediscussed method (or the examples thereof).

[0062]

[0054] By partitioning a document, it is meant to divide said document into a plurality (i.e., at least two) of portions. Each two (e.g., consecutive) partitions may overlap. In other words, each location at the document may belong to one or more partitions.

[0063]

[0055] This method constitutes an improved solution as each portion is defined by a context rather than a mere length of the portion. In other words, the method performs partitioning in according to at least a context of the portions to be obtained. Additionally, the method may set a size (e.g., an upper bound) for each portion. This is beneficial to limit the size of the portions regarding the computing power at hand. This avoids obtaining too long portions which cannot be analyzed in total due to hardware or software restrictions.

[0064]

[0056] In examples, the document belongs to a library. The library may comprise a plurality of documents. The library may be a local library, i.e., a library stored in a local storage, or a remote library, i.e., a library which is accessible via a network, for example a library located on a cloud. The indexing method may be applied to a subset (e.g., each) of the plurality of documents. This provides a searchable library index for said plurality of documents for a user.

[0065]

[0057] In reference to Figure 3, it is further provided a computer-implemented method for information retrieval from one or more documents. The one or more documents have been provided to the method by providing an access to a local storage or a remote storage platform. In other words, the method has access to read any of the one or more documents. The mandatory steps are presented in solid lines while the optional steps are presented in dotted lines.

[0058] The information retrieval method comprises, in Step S310, providing one or more groups each obtained for a document (of said one or more documents) using the indexing method discussed above. In other words, each group comprises a first index, or Index 1, obtained by indexing a respective first representation, and a second index, or Index 2, obtained by indexing a respective second representation, as discussed above.

[0066]

[0059] The retrieval method comprises, in Step S320, providing a request (i.e., a query or prompt) to retrieve information. In examples, the request may be in a human-language syntax (e.g., a question). The request may be provided by a user, for example, via a keyboard, a mouse, or another haptic or vocal device. The request may also be provided using an audio to text converter. In such examples, the user may recite the request to a microphone and the method converts the vocal request to a text to be provided to the method.

[0067]

[0060] Next, in Step S330, the method determines a first list of documents in relation to the requested information using the first indexes of the one or more groups. In other words, the method retrieves / selects the first list of documents from the provided one or more documents according to the requested information and using the first indexes. In yet other words, the method exploits the one or more keywords related to each first index to assess the relevance of a respective document to said first index to the request (e.g., the text of the request). This comparison between the request and said keywords establishes a matching measure between the keywords of a first index (e.g., indexed in a document) and the wording of the request.

[0068]

[0061] Similarly, in Step S340, the method determines a second list of documents in relation to the requested information using the second indexes of the one or more groups. In other words, the method retrieves / selects the second list of documents from the provided one or more documents according to the second indexes. In yet other words, the method assesses a similarity between a representation of the request, for example a vectorial embedding and said pair of vectorial embeddings of the second index.

[0069]

[0062] In examples, Step S340 may comprise applying a vectorial embedding to the provided request in order to obtain a representation of said request. The method then may compare said representation of the request to the indexed second representation of each provided one or more documents to obtain the second list of documents. This enables comparing the context of the request with the context of the one or more documents.

[0070]

[0063] As discussed above, said second representation is a pair of a vectorial embedding of a summary of and a vectorial embedding of said portion of the document. In examples, the method may first compare said representation of the request (which is a vectorial embedding) to the vectorial embedding of the summary of each document. Then, if the similarity between the vectorial embedding of the request and the vectorial embedding of summary of a document is higher than a threshold, the method may compare the vectorial embedding of the request and the vectorial embedding of said document (as a whole). This two-level retrieval allows processing large volumes of data more efficiently, by focusing on the relevant documents after an initial filtering by summary.

[0071]

[0064] The method then, as Step S350, generates a response to said request using the determined first list of documents and the determined second list of documents. In examples, a large language model (LLM) may be provided, and the generating of a response comprises outputting a response using said provided LLM. The LLM is inputted by the determined first number of documents and the determined second number of documents. This provides a RAG architecture in which the extracted data (i.e., the first list and the second list of documents) is used to enrich the context of the generated response.

[0072]

[0065] In examples, the method for information retrieval may comprise one or more offline steps and one or more online steps. The offline steps, or preparatory steps, comprise preparation of documents including parsing, chunking, storing and indexing the document. In other words, offline steps include preparing the portions for the method of indexing, and indexing them according to the examples discussed above.

[0073]

[0066] On the other hand, online steps comprise treatment of the query, determining the most appropriate documents for the response (i.e., the first list and the second list), and response generation using the prompt and the documents to generate a response.

[0067] Such a separation is beneficial in improving the latency of the method when realtime performance is needed.

[0074]

[0068] In examples, the method comprises, prior to the generating the response to obtain two reduced list of documents. In other words, the method reduces each of the first list and the second list by (a first step of) filtering according to relevance score measure, for example a BM25 score obtained from a Rank-BM25 model (see https: / / en.wikipedia.org / wiki / Okapi_BM25). The relevance score measure represents the relevance of a document to the request. Such a filtering may keep the K (e.g., 100, 50, 20, 10 or 5) most relevant items of each list. In such examples, the generation of a response to said request using the determined first list of documents and the determined second list of documents comprises using said two reduced lists of documents. In other words, the method uses the reduced two lists of documents instead of the two determined first and second lists. This improves computational efficiency in response generation.

[0075]

[0069] Back to Figure 3, the first list obtained at Step S330 may be filtered to obtain the first reduced list in Step S335. Similarly, the second list obtained at Step S340 may be filtered to obtain the second reduced list in Step S345. Steps S335 and S335 may be executed upon terminating the execution of both Step S330 and Step S340. Alternatively, Step S335 may be executed upon termination of Step S330 and before or in parallel with the execution of Step S340. Step S345 may then be executed after the termination of the execution of Step S340. The obtained two reduced lists by Steps S335 and S345 may then be directly used in Step S350 (not shown) or be reduced again in Step S355 as explained below.

[0076]

[0070] In examples, the method may obtain a further reduced list of documents by filtering said reduced first list and said reduced second list. Such a second step of filtering may be according to the same relevance measure as of the first step, for example a BM25 score. To obtain said further reduced list, the method may accumulate / combine the first reduced list and the second reduced list, and then select the K (e.g., 100, 50, 20, 10 or 5) most relevant documents of the combined list. In such examples, the generating of a response to said request using the determined first list of documents and the determined second list of documents comprises using said further reduced list. In other words, the method uses the further reduced list of documents (which is a subset of the union of the first and second determined reduced lists) instead of the two determined first and second lists. This further improves computational efficiency in response generation.

[0077]

[0071] Back to Figure 3, the first reduced list obtained in Step S335, and the second reduced list obtained in Step S345, are provided to Step S355 to be further reduced to a single list. This further reduced single list is then used for generating a response in Step 350.

[0078]

[0072] Figure 10 shows an example of the system, wherein the system is a client computer system, e.g., a workstation of a user. The client computer of the example comprises a central processing unit (CPU) 1010 connected to an internal communication BUS 1000, a random-access memory (RAM) 1070 also connected to the BUS. The client computer is further provided with a graphical processing unit (GPU) 1110 which is associated with a video random access memory 1100 connected to the BUS. Video RAM 1100 is also known in the art as frame buffer. A mass storage device controller 1020 manages accesses to a mass memory device, such as hard drive 1030. Mass memory devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM disks 1040. Any of the foregoing may be supplemented by, or incorporated in, specially designed ASICs (application-specific integrated circuits). A network adapter 1050 manages accesses to a network 1060. The client computer may also include a haptic device 1090 such as cursor control device, a keyboard or the like. A cursor control device is used in the client computer to permit the user to selectively position a cursor at any desired location on display 1080. In addition, the cursor control device allows the user to select various commands, and input control signals. The cursor control device includes a number of signal generation devices for input control signals to system. Typically, a cursor control device may be a mouse, the button of the mouse being used to generate the signals. Alternatively or additionally, the client computer system may comprise a sensitive pad, and / or a sensitive screen.

[0073] The computer program may comprise instructions executable by a computer, the instructions comprising means for causing the above system to perform the method. The program may be recordable on any data storage medium, including the memory of the system. The program may for example be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. The program may be implemented as an apparatus, for example a product tangibly embodied in a machine- readable storage device for execution by a programmable processor. Method steps may be performed by a programmable processor executing a program of instructions to perform functions of the method by operating on input data and generating output. The processor may thus be programmable and coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. The application program may be implemented in a high- level procedural or object-oriented programming language, or in assembly or machine language if desired. In any case, the language may be a compiled or interpreted language. The program may be a full installation program or an update program. Application of the program on the system results in any case in instructions for performing the method. The computer program may alternatively be stored and executed on a server of a cloud computing environment, the server being in communication across a network with one or more clients. In such a case a processing unit executes the instructions comprised by the program, thereby causing the method to be performed on the cloud computing environment.

[0079]

[0074] An example implementation of the method is now discussed

[0080]

[0075] The implementation focuses on information management to handle the difficulties to extract precise information from a large amount of document data. The traditional approach to information retrieval is not effective due to its lack of contextual! zati on and limited ability to handle complex queries. These traditional methods can pose major barriers in obtaining accurate information from a large volume of data.

[0081]

[0076] The implementation uses a Retrieval-Augmented Generation (RAG) architecture which combines information retrieval and response generation methods to provide more accurate answers to complex and contextual queries, thus overcoming the constraints of traditional methods. By leveraging large document datasets in an efficient way, the RAG architecture improves the relevance and accuracy of responses, making it easier to extract information accurately and produce context-sensitive answers. This improves information management efficiency.

[0082]

[0077] Architecture of the system

[0083]

[0078] The implemented architecture exploits several interconnected services that facilitate the processing and generation of answers based on the library of documents. This architecture follows the RAG model, which combines information retrieval and text generation to provide accurate, contextualized answers.

[0084]

[0079] An overview of the main components and data flow within the system is now discussed. This overview shows how the system, using the RAG model, combines search and text generation to transform data into accurate, tailored answers. Coordination between the various stages ensures efficient document and query management, providing a robust solution for information retrieval and generation.

[0085]

[0080] Introduction to the RAG architecture

[0086]

[0081] The RAG architecture is a hybrid model that combines two steps to improve the quality and relevance of the answers produced. This model combines information retrieval and text generation to deliver more accurate, context-sensitive answers. These two processes interact with each other. The aim of the information retrieval stage is to query a database or index to extract relevant documents in response to a request. Once the relevant documents have been collected, the text generation uses an advanced text generation model to generate a fluid, context-sensitive response. The model is capable of capturing and summarizing data in order to provide a coherent response to the queries posed.

[0087]

[0082] By combining search and generation, the RAG model exploits available data to improve the accuracy and relevance of responses, making the system more efficient and intelligent.

[0083] Figure 4 presents an example of a RAG structure in which the response 450 to question 410 is obtained by inputting to LLM 420 said question 410 as well as the relevant knowledge 440 obtained from the database 430.

[0088]

[0084] System overview

[0089]

[0085] In reference to Figure 5, a global view of the system is presented that highlights the interaction between its various components to generate precise and context-sensitive answers.

[0090]

[0086] Document preparation: Documents are first analyzed to extract the relevant information. This preparation stage is beneficial, as it transforms the raw data into a structured and optimized format, ready to be used for the next processing steps. By ensuring rigorous extraction and structuring, this stage lays the foundations for reliable analysis and accurate responses, with a direct impact on overall system performance.

[0091]

[0087] Data storage: Once the documents have been prepared, the data is stored securely, while remaining easily accessible. Thanks to this storage, data is preserved and quickly available for future operations, while maintaining its integrity and security.

[0092]

[0088] Indexing and organization: Next, stored data is structured through an indexing process which transforms documents into vector representations. This step facilitates search and access to relevant information, thus considerably improving data retrieval and enabling faster, more precise searches.

[0093]

[0089] Request processing: Once a request, e.g., query prompt 510 is received, it is examined to identify the data required for the response. An embedding model embeds the query so that a similarity search can be done in the database based on the similarity with the query (i.e., the embedding of the query). The index is used by the system, as shown in 520, to locate the most appropriate documents 530. This step ensures that responses are based on the most appropriate and contextually relevant data.

[0094]

[0090] Answer generation: By exploiting the collected documents, the system associates this data with the prompt to produce an answer using an advanced text generation model 540. This model generates coherent, relevant answers by taking into account both the context provided by the documents 530 and the query prompt 510. This is achieved as shown by the combination 545 of the context of the documents 530 and the query 510.

[0095] System services

[0096]

[0091] Different services (or equivalently, components) of the implementation are now discussed.

[0097] 1. Parsing service

[0098]

[0092] The parsing service is the system's entry point for document management. It allows users to upload various types of files, which are then processed to extract the relevant information required for further system operations. The parsing service offers the following features:

[0099] - Document uploading: Users can upload files in various formats, such as PDF files (.pdf), Word documents (.doc / .docx), slides (.ppt / .pptx), and spreadsheets (.xls / .xlsx). The system ensures secure download management to preserve data confidentiality and avoid security risks.

[0100] - Parallel Analysis: Files are processed in parallel, meaning that several documents can be analyzed simultaneously. This optimizes system performance by reducing waiting times and enabling rapid document processing, even when a large number of files are submitted.

[0101] Information extraction: After uploading, the parsing service extracts relevant information from the files. This includes text recognition for scanned documents (OCR), structured data extraction from spreadsheets, and text analysis to extract key information.

[0102] - Data structuring: The extracted information is then structured according to predefined formats, facilitating its integration into the system.

[0103]

[0093] The parsing service may play an important role for several reasons.

[0104]

[0094] The quality of extracted data may be of importance to the smooth running of the system. Accurate, well -structured data ensures that subsequent analyses are reliable, and that decisions based on this data are justified. Errors or inconsistencies in parsed data can lead to incorrect results and adversely affect system performance.

[0095] On the other hand, the chunking process significantly improves the quality of document indexing. By partitioning documents into portions or chunkshelps each portion or chunk be precisely representedas a vector. This enables finer-grained indexing, with more complete and contextually relevant information, improving the quality of subsequent search and analysis. Chunking also helps to manage memory efficiently and speed up processing operations by facilitating data access and manipulation.

[0105] 2. Data storage service

[0106]

[0096] The data storage service is responsible for storing documents after they have been parsed. The data storage ensures secure storage of data and rapid, efficient access for further processing.

[0107]

[0097] The data storage service offers several important features:

[0108] - Data Storage: The service ensures that parsed documents are stored securely, protecting data against loss and corruption. It uses backup and redundancy mechanisms to ensure continuous data availability.

[0109] Search: The service enables efficient searching of stored documents using their unique identifiers. This facilitates rapid access to specific documents and reduces the time needed to locate relevant information.

[0110] Send for Indexing: The data storage service regularly sends documents to be indexed, enhancing data search and analysis in the system. This ensures that new information is taken into account in analytical processes.

[0111]

[0098] The data backup service is important for several reasons. The service ensures the protection of documents while guaranteeing their continued availability for future analysis and operations, thus avoiding service interruptions. By enabling rapid access to documents via unique identifiers, and regularly sending documents for indexing, the service optimizes system performance. Rapid access to data enables efficient execution of queries and analyses, improving system responsiveness and user satisfaction.

[0112] 3. Indexing service

[0099] The indexing service is responsible for organizing documents to optimize searching and rapid access to information. By transforming documents into structured indexes, this service improves query efficiency and the quality of returned results.

[0113]

[0100] The indexing service offers two main functionalities:

[0114] Index creation and updating: The service regularly receives documents from the database and indexes them. The index created lists the identifiers (IDs), their source, and their vector representations. This makes it possible to track the origin of documents, and to represent them in digital form for easier algorithmic processing.

[0115] Index search: The service enables fast, efficient searching by querying the index rather than the complete database. Identifiers provide direct access to relevant documents, while vector representations are used to measure similarity with queries, optimizing response time and relevance of results.

[0116]

[0101] The indexing service is of high importance for several reasons. By structuring data efficiently, indexing reduces the time needed to search the database. This increases system responsiveness and enhances the user experience. Furthermore, indexing enables documents to be ranked according to their relevance, using criteria such as keyword frequency and metadata importance. This ensures that the most appropriate results are returned for each query.

[0117]

[0102] The design of the indexing service aims to adjust not only to increasing data volumes, but also to changing system requirements. This ensures that the system is able to adapt to growing and changing needs, while maintaining optimum performance.

[0118]

[0103] Indexing structure Indexing is designed to integrate two types of searches: keyword search and semantic search. This dual approach enhances the accuracy and relevance of results.

[0119]

[0104] The keyword search identifies the most relevant documents based on the exact terms searched. In keyword search, the documents are indexed according to keyword frequency, thus enabling fast and efficient searching.

[0105] Semantic indexing organizes the documents in a multi-level hierarchy. This makes it possible to perform a general search on the first level, which facilitates a more precise search on the second level. This hierarchy improves the accuracy of results by refining the initial search results to find the most relevant documents.

[0120]

[0106] Hierarchical semantic indexing enables high accuracy by refining results across multiple search levels. This contrasts with conventional methods which, while effective for simple searches, do not always capture the contextual relationships between terms. Furthermore, and despite the fact that the hierarchical approach requires a little more time to handle queries, a hierarchical approach delivers more precise results and is better adapted to users' particular needs. Results obtained using hybrid indexing are generally of higher quality, as they offer a deeper contextual understanding and a more precise search.

[0121]

[0107] The implementation allows data growth and the integration of new indexes or the deletion of existing ones without loss of information. This flexibility ensures that the system can evolve to meet growing needs, while maintaining data quality and consistency.

[0122]

[0108] 3. Model service and data flow

[0123]

[0109] The model service is responsible for processing user requests.

[0124]

[0110] In reference to Figure 6, the data flow and interactions in the model service are now discussed.

[0125]

[0111] First, model service 600 receives (Step SI) a query 610 from the user 605. The model service 600 queries the indexing service 650 to identify documents relevant to the query. The indexing service performs a quick index search and returns the identifiers (IDs) of the relevant documents (Step S2). Using the IDs provided, the indexing service 650 retrieves the corresponding documents from the data storage service 670 (Step S3) and returns them to the model service 600 (Steps S4 and S5).

[0126]

[0112] The model service 600 then uses the retrieved documents as context to generate an answer to the user's query (Step S6). The generation model can be based on advanced technologies such as GPT (Generative Pre-trained Transformer) which are capable of producing fluid, relevant text. The generated response is returned to the user 605 as a stream 680, enabling an interactive and responsive experience.

[0127]

[0113] Furthermore, and in reference to Figure 6, the data flow can be summarized as below: i. Uploading (Step S7) the documents by a user or an expert 606 may upload one or more documents. ii. Document analysis using parsing service 690: This step receives uploaded documents and analyses their content to extract information, then sends them to the data storage service 670 (Step S8). iii. Data storage service 670 stores data extracted by the parsing service 690 in a database for later access. iv. Search preparation: The indexing service 650 regularly retrieves (Step S4) stored documents from the data storage service 670 and creates indexes (Step S3) for these documents to facilitate future searches. v. Document Search and Retrieval: Model Service 600 receives a search request and consults the indexing service. The indexing service searches for relevant documents in the index and retrieves the full content of found documents from the data storage service. vi. Answer Generation: The model service 600 compiles information from retrieved documents to formulate an answer, then returns this answer to the user in real time.

[0128]

[0114] Figure 7 shows a global view of the system and the data flow between the elements and services as follows:

[0129] - Step 1.0: Interacting with a user (receiving a query / response stream);

[0130] Step 1.1.0: Requesting for a search for index;

[0131] Step 1.1.1 : Search by Indexing service;

[0132] Step 1.1.2: Requesting the data storage service for the search by the index

[0133] Step 1.1.3: Searching by the data storage service; Step 1.1.5: Returning the search results;

[0134] Step 2.0: Uploading a file by a user;

[0135] Step 2.1 : Analyzing and partitioning of the document by the parsing service (Chunking a large text into multiple chunks by a chunking model);

[0136] - Step 2,2: Storing the chunks by sending them to the data storage service;

[0137] - Step 2,3: Storing the documents / chunks;

[0138] - Step 2,4: Sending the documents / chunks to be indexed by the indexing service; and

[0139] - Step 2,5: Indexing the documents / chunks.

[0140] Implementation Steps

[0141] 1. Parsing documents

[0142]

[0115] Parsing, or syntactic analysis, is the process by which a document is analyzed to extract and structure information in a meaningful way. In the context of document management, parsing is of importance because it allows unstructured texts to be converted into usable formats, thus facilitating the search, analysis and use of data contained in documents.

[0143]

[0116] The parsing must take into account the semantics and hierarchy of the document. This means that the text structure, with its headings, sub -headings and paragraphs, must be preserved to faithfully reflect the logical and thematic organization of the content.

[0144]

[0117] Several variants of parsing method have been implemented and tested.

[0145]

[0118] In a first variant, at the beginning of the parsing process, Langchain reads the entire content of the documents, then this content is divided into parts of a defined (e.g., uniform / constant) size. This sorting is done without taking into account the semantic or logical context of the text. This variant partitions large amounts of text into small segments, which improves the efficiency of the indexing process. It also makes further processing easier, especially for analyses or searches in a large data set. However, by ignoring context, chunking can interrupt essential sentences or ideas, which can lead to a loss of meaning. The resulting segments may not reflect the logical relationships or continuity of the text, making analysis and information extraction more complex.

[0146]

[0119] In a second variant, Langchain SemanticChunker is implemented for intelligent page slicing that considers the meaning and flow of information. The second variant is able to detect thematic transitions and create segments that preserve the context. This release provides enhanced document understanding and organization and improves the relevance of extracted information by maintaining semantic context, facilitating complex textual analyses and contextual searches. Despite these improvements, the second variant does not always preserve the original structural hierarchy of the document, which can be problematic for formally structured texts such as contracts or technical reports.

[0147]

[0120] A third variant allows text to be cut while respecting the hierarchical structure of the document. This aims to reconcile granularity and structural integrity. The third variant may be implemented using a library known in prior art, e.g., Unstructured library. The third variant comprises analyzing the document structure elements, such as headings, subheadings, and sections, while respecting the hierarchical organization of the content.

[0148]

[0121] The third variant significantly improves the quality of the extraction by respecting the original structure of the document while preserving hierarchical organization. This is particularly beneficial for complex documents and allows more efficient data extraction.

[0149]

[0122] Figure 8A-D show an example application of the parsing. Figure 8A shows the original document, while Figures 8B-D show parsing according to the first, second, and the third variant discussed above. As may be seen, the chunking according to the third variant preserves better the coherency of the context in each chunk.

[0150] 2. Indexing

[0151]

[0123] Indexing plays an important role in facilitating efficient and fast document search by the user.

[0152]

[0124] Several variants of indexing have been implemented and tested.

[0153]

[0125] A first variant of indexing service includes two types of indexes: Facebook Al Similarity Search (FAISS) index and word frequency index. The FAISS index is used for semantic search. FAISS is a powerful tool for managing and searching high-dimensional vectors. Nevertheless, FAISS does not allow dynamic addition of new data to the index without a complete rebuild, which hinders its ability to evolve based on the new information added. The word frequency index is implemented manually (i.e., without using a specialized library) or using a library. The text is first filtered using specific tags via a spacy model to keep only the words deemed useful for searches. Then, the frequency of the remaining words is calculated.

[0154]

[0126] A second variant of indexing service uses Milvus index. Milvus index is a distributed vector management system that provides increased scalability and efficiency for semantic search. Milvus allows new data to be dynamically added to the index without requiring a complete rebuild. Despite its effectiveness for semantic search, Milvus Index has limitations when texts (or chunks thereof) are very large. Embedding models may have difficulty in correctly representing these chunks, which can reduce the quality of the search.

[0155]

[0127] A third variant has a hierarchical approach. In this approach, the indexing is improved by introducing a summary level at the first level of the index. The first level of the index contains summaries of the chunks. This helps to determine which files to explore in more detail at the second level of the index, where the chunks themselves are stored. This hierarchy helps to better explore the relevant files and improve query response. The hierarchical approach with two levels may be also called a two-level indexing.

[0156]

[0128] The hierarchical approach allows processing large volumes of data more efficiently, by focusing on the relevant chunks after an initial filtering by summary. Thereby, it improves the precision of the search and reduces the time needed to access the relevant information. Nevertheless, as the summary level does not allow for detailed textual searching, the two-level indexing has limitations in terms of keyword searching,

[0157]

[0129] Figure 9 presents an example of a two-level indexing. Each of the plurality of first level indexes 910 refers to a summary of a document. While each of the plurality of the second level indexes 950 refers to a document or a chunk / portion of a document. For example, the summary index 915 refers to a first source document 901 while the summary index 925 refers to the second source document 902. The first source document itself corresponds to the group 955 of the second level indexes, and the second source document corresponds to the group 965 of the second level indexes.

[0158]

[0130] Upon providing an embedding 905 of a query (e.g., about how to ask for a day off at work), the method first retrieves summaries 915 and 925 as being relevant to the query. At the next level, the method looks up the documents these summaries refer to, i.e., the first source document 901 and the second source document 902, and the respective second level indexes. Thereby it obtains the respective results 955 in the first source 901 and 965 in the second source 902. The second level indexes may be related to portions of the source document which facilitates the relevance analysis of the retrieved results. For example, the second level indexes of the results 955 (respectively, 965) may comprise 9551 and 9552 (respectively, 9651, 9652, and 9653) each referring to a specific portion / chunk of the source 901 (respectively, source 902).

[0159]

[0131] A fourth variant is a variant of the third variant discussed above by adding a second index. The fourth variant combines the Milvus index for semantic search with the Whoosh index for keyword search. Whoosh is a full-text search engine written in Python, which improves keyword search.

[0160]

[0132] First, the text is filtered in order to keep only the words relevant for searches. The filtering is done using specific tags by a part-of-speech tagging (POST) model. The tagging model used is Stanza model (see http s : / / stanfordnlp , ithub . i o / stanza / ) . After searching both indexes (i.e., Whoosh and Milvus indexes), the results are filtered using a relevance score measure. The implementation uses Rank-BM25 model (see https: / / en.wikipedia.org / wiki / Okapi BM25) to select the K most relevant items. Rank- BM25 is a ranking algorithm based on term frequency and document length, which helps to refine the results and identify the most relevant documents for the user query.

[0161]

[0133] The fourth variant provides complete coverage of semantic and keyword search needs, thus allowing increased flexibility and precision in document retrieval. The combined use of Milvus and Whoosh indexes, as well as the application of Rank-BM25 for result refinement, may result in an increase in search time as the system must query two separate indexes and perform an additional ranking with Rank-BM25, which may prolong the process. However, this increase in search time is outweighed by the substantial improvements in accuracy and the offered quality of results.

[0162] 3. Database

[0163]

[0134] Data management and backup are important to ensure the integrity and accessibility of the information processed by our system. Here are the details of the different approaches used for data backup in the different variants of the indexing system.

[0164]

[0135] In a first variant, the data is saved as files, for example JSON files. The JSON format is lightweight, easy to read and write, which simplifies data serialization and deserialization. Furthermore, the hierarchical structure of data with JSON makes it easier to organize and use. However, JSON format needs full loading. In other words, full JSON file loading is required for each operation, which can be inefficient for large amounts of data. Furthermore, in JSON format, data search, filter or update operations are manual, which can become complex and inefficient as the volume of data increases. This results in longer response times and less efficient data handling.

[0165]

[0136] In a second variant, the data is saved in a NoSQL database, for example MongoDB. MongoDB allows good scalability to manage large volumes of data and scale horizontally by adding additional servers, thus meeting growing storage and processing needs. Furthermore, MongoDB's shows further advantages:

[0166] Its document data structure makes it easy to add new fields or modify existing structures without affecting data integrity.

[0167] It improves the indexing process by making it easier to manage indexed and non-indexed documents. It allows retrieving non-indexed documents one by one and adding them to the index without loading all the data with each query.

[0168] It optimizes performance by avoiding the full data load for each query. Search and retrieval operations are faster and more efficient, improving system response times.

[0169]

[0137] The second variant proposes an efficient way to manage indexed and nonindexed documents and the process of submitting documents for indexing. This provides smooth and fast data management while having high flexibility and efficiency and meeting the growing demands of the indexing service.

[0170] 4. Chunk evaluation

[0171]

[0138] To ensure that only the most relevant chunks are kept for indexing, the database service uses LLMjudge. This language model is used to evaluate the relevance of chunks based on their usefulness.

[0172]

[0139] LLMjudge assigns a score to document chunks based on their quality and relevance. This score determines whether a chunk should be retained for indexing. Only chunks that exceed a predefined threshold (i.e., a relevance threshold) in the evaluation score are retained for indexing, ensuring that the indexed data is the most useful.

[0173]

[0140] Using LLMjudge improves the quality of indexed data by filtering out irrelevant chunks, leading to better search performance and more accurate information retrieval. To further improve the efficiency, LLMjudge may be tuned. A tuning process is discussed later below.

[0174]

[0141] Documents not yet indexed are first evaluated by LLMjudge. Those that exceed the relevance threshold are then sent for indexing. Once the documents are indexed, they are marked as already indexed in the database to avoid repeated processing. This process is performed regularly to ensure that new data is integrated into the system. Evaluation and indexing can be initiated automatically at defined intervals or manually, if necessary, thus ensuring that the index is continuously updated with new data.

[0175] 5. Model service

[0176]

[0142] Model Service is the central component that handles user queries and generates responses based on documents retrieved from the index.

[0177]

[0143] In a first variant, the service receives a query from the user and passes it to the index to search for relevant documents. The query is used to query the index, which returns matching documents. Using the documents retrieved from the index as context, a text generation tool, for example Mistral Al language model, generates the response in streaming. This process allows producing a fluid and relevant response in real time, based on the contextual information provided.

[0144] A second variant is a variant of the first variant by introducing new features aimed at improving the quality and accuracy of responses. The second variant proposes query reformation to reformulate the user’s query before sending it to the index. Reformulation helps correct typos and clarify the question, which allows for more accurate and efficient searching in the index, improving the relevance of the results. Furthermore, the second variant uses a chunk evaluation in the data storage system as discussed above.

[0178]

[0145] The implementation is implemented using Python programming language for backend development and machine learning model training. Python offers a rich ecosystem of libraries and frameworks for data processing and model development.

[0179]

[0146] For parsing, the implementation uses the following software packages:

[0180]

[0147] The implementation uses Unstructured to analyze and extract raw textual data. This tool transforms unstructured data into a format that can be used for further processing. The implementation further uses Langchain for sequencing and integrating the various stages of text data processing. This facilitates the creation of sophisticated, integrated workflows.

[0181]

[0148] In database services, the implementation uses MongoDB for storing and managing extracted data. MongoDB provides a flexible structure for parsed documents, allowing dynamic and scalable data management.

[0182]

[0149] For indexing, the implementation uses the following software packages:

[0183]

[0150] The implementation uses Milvus for vector embedding management and large- scale similarity search. Milvus enables vector embedding processing and searching, providing advanced search capabilities for large volumes of data.

[0184]

[0151] The implementation further uses Whoosh for indexing documents, allowing for fast and efficient searching. Whoosh relies on word frequency to index and retrieve relevant documents from the indexed data.

[0185]

[0152] For text generation, the implementation uses the following software packages:

[0186]

[0153] The implementation uses Mistral Al text generation, based on an advanced language model that offers high-level performance. Mistral Al provides an improved price-quality ratio, guaranteeing optimum performance at a competitive cost.

[0154] The implementation further uses Hugging Face for embedding models and language models (LLM). Hugging Face provides high-quality embedding models used for indexing and semantic chunking methods in the parsing service discussed above. The implementation also uses Hugging Face LLMs for text generation, providing advanced capabilities for natural language processing. Indeed, Hugging Face provides python libraries that allow running the Mistral Al model. Any other text generation algorithm known in the field may be also used.

[0187]

[0155] Testing and validating the implementation is now discussed.

[0188]

[0156] A two-step validation protocol is implemented. In other words, the validation protocol adjusts the language model used for the evaluation and the complete analysis of the generated chunks. The implementation may use LLMJudge which is an advanced language model to evaluate the quality of chunks generated by the parsing process. Before performing a full evaluation, LLMJudge needs to be carefully tuned to ensure accurate and reliable results. To do so, the implementation comprises an evaluation and tuning phase using manually evaluated chunks as discussed below.

[0189]

[0157] The testing process is as follows:

[0190] LLMJudge Tuning

[0191]

[0158] A small sample of chunks is prepared and manually evaluated to test and tune LLMJudge. This step is important to calibrate the tool, ensuring that it can evaluate chunks accurately and in line with human criteria for relevance and quality. To tune LLMJudge, a correlation between manual and automatic assessments is used as the main criterion. A correlation of 78% was obtained, indicating a good match between human and LLMJudge judgments. This high concordance rate attests to the reliability of the model to assess chunk quality, ensuring that the adjustments made align the model’s performance with human expectations.

[0192] Manual Evaluation

[0193]

[0159] Sample chunks are manually reviewed and scored to provide accurate benchmarks for adjusting / tuning the LLMJudge prompt. This evaluation helps refine the prompt to match human expectations when evaluating documents. Full Evaluation

[0194]

[0160] After LLMJudge is tuned (or adjusted), a full evaluation is performed on all chunks generated by the parsing process. At this point, LLMJudge evaluates the chunks using the adjusted criteria to measure the completeness of information and the proper separation of data in each chunk. This step aims to ensure that the generated chunks meet the defined quality criteria and to validate the effectiveness of the parsing process on the entire data set.

[0195]

[0161] The results of the full evaluation were compared between the different versions of the parsing process. For this comparison, the average of the evaluations of all documents performed by LLMJudge was used as the main criterion.

[0196]

[0162] The tests showed that the third variant, using the Unstructured library, achieved the best overall performance. The third variant has demonstrated an increased ability to maintain document structure and extract sensitive and important information with increased accuracy, confirming its effectiveness and robustness compared to previous versions.

Claims

CLAIMS1. A computer-implemented method for indexing at least a portion of a document (D), the method comprising: determining (S10) a first representation using one or more keywords of said portion of the document, said keywords being selected by a model as being relevant syntagms; indexing (S20) said first representation thereby updating a first index (Index i); obtaining (S30) a summary for said portion of the document (D); from said summary, determining (S40) a second representation as a pair of a vectorial embedding of said summary and a vectorial embedding of said portion of the document (D); indexing (S50) said second representation thereby updating a second index (Index 2).

2. The method of claim 1, further comprising: storing, in a database, said portion in correspondence to the obtained summary.

3. A computer-implemented method for indexing a document (D) comprising: partitioning (S210) the document into one or more portions, each portion being defined by a respective context, indexing (S220) each portion using the method of any of claims 1 to 2.

4. A computer-implemented method for information retrieval from one or more documents, the method comprising: providing (S310) one or more groups of first indexes and second indexes, each group being obtained for a document partitioned and indexed using the method of claim 3; providing (S320) a request to retrieve information;determining (S330), using the first indexes of the provided one or more groups, a first list of documents in relation to the requested information; determining (S340), using the second indexes of the provided one or more groups, a second list of documents in relation to the requested information; and generating (S350) a response to said request using the determined first list of documents and the determined second list of documents.

5. The method of claim 4, wherein the architecture underlying the provision of the request to retrieve information and the generation of response is a RAG structure implementing at least one of parsing services, data storage services and / or indexing services.

6. The method of any of claim 4 or 5, further comprising, prior to the generating of a response: obtaining (S335, S345) two reduced lists of documents by filtering each of the first list of documents and the second list of documents using a relevance score measure, the relevance score measure representing the relevance of a document to the request; wherein the generating of a response to said request using the determined first list of documents and the determined second list of documents comprises using said two reduced lists of documents.

7. The method of any of claim 4 or 5, further comprising: obtaining (S355) a further reduced list of documents by filtering said reduced first list and said reduced second list; wherein the generating of a response to said request using the determined first list of documents and the determined second list of documents comprises using said further reduced list of documents.

8. The method of any of claims 4 to 6, wherein the relevance score measure is a BM25 score.

9. The method of any of claims 4 to 8, wherein the provided request is in a humanlanguage syntax.

10. The method of any of claims 4 to 9, wherein the method further comprises using a large language model (LLM), and the generating of the response comprises using said LLM together with a RAG architecture implementing at least one of parsing services, data storage services and / or indexing services, the determined first list of documents and the determined second list of documents.

11. A computer program comprising instructions for performing the method according to any one of claims 1 to 10.

12. A computer readable storage medium having recorded thereon the computer program of claim 11.

13. A system comprising a processor coupled to a memory, the memory having recorded thereon the computer program of claim 12.

Images