System and method for document relevance investigation - Patents.com

Abstract

Disclosed herein is a system 102 and method for investigating the relevance of documents. The system 102 extracts documents from one or more data sources 210 based on a request from a user. The system 102 then obtains user intent information 108 and user queries 110 from the user. The system 102 then analyzes each document with respect to the user intent information 108 to determine each document's relevance level. The relevance level is indicated in the form of a ranking score. The system ranks the documents and displays them to the user in order of their scores. The system 102 also highlights important excerpts from the documents and provides one or more responses 222 to one or more user queries 110 submitted by the user for each document. Based on the received responses, the user provides feedback to further train the system, thereby achieving better accuracy.

Description

[Technical field]

[0001] The present disclosure relates to the field of data analysis, and more particularly, to analyzing documents extracted from various sources to examine document relevance. [Background technology]

[0002] The following description includes information that may be useful in understanding the present disclosure. No admission is made that any information provided herein is prior art or relevant to the present claims, or that any publication specifically or implicitly referenced is prior art.

[0003] Preparing a report for a survey-based study requires manual extraction and parsing of a large number of documents and is therefore a very tedious task. An individual has to spend a long time to find the relevant documents and, after finding the relevant documents, has to study each and every one of them to decide whether to include the document in the study or not. For example, an individual wants to prepare a Clinical Evaluation Report (CER) that describes the clinical evaluation of a medical device. For this, the individual must first identify clinical data from existing literature, clinical experience, clinical trials, or any combination of the three. Then, the individual must evaluate the relevance, applicability, quality, and significance of the data based on the collected data and, finally, articulate a conclusion in the CER. Moreover, since these two steps are essential to the quality of the report that is prepared, these steps must be performed with utmost care. Thus, when performed manually, the entire process is very time-consuming and mentally exhausting.

[0004] Thus, there is a need for methods and systems that reduce the manual effort involved in identifying and evaluating data and investigating the relevance of documents. Summary of the Invention

[0005] The present disclosure overcomes one or more shortcomings of the prior art and provides additional advantages that are discussed throughout the present disclosure. Additional features and advantages are realized through the techniques of the present disclosure. Other embodiments and aspects of the present disclosure are described in detail herein and are considered a part of the claimed disclosure.

[0006] In one non-limiting embodiment of the present disclosure, a method for investigating the relevance of documents is disclosed. The method includes extracting a plurality of documents from one or more data sources based on a user request. The method further includes receiving a user input from a user, the user input includes at least one of user intent information and one or more user queries. The method further includes generating a ranking score for each of the plurality of documents via a domain-specific ranking module by correlating the user intent information with respect to the plurality of documents, the ranking score indicating a relevance level of each of the plurality of documents with respect to the user intent information. The method further includes highlighting one or more excerpts in each document while displaying the plurality of documents based on the ranking score. The method further includes exploring user feedback corresponding to the relevance of each document based on the ranking score and the highlighted one or more excerpts to train the domain-specific ranking module.

[0007] In yet another non-limiting embodiment of the present disclosure, a system for investigating relevance of documents is disclosed. The system includes an extracting unit configured to extract a plurality of documents from one or more data sources based on a user request. The system further includes a receiving unit configured to receive a user input from a user, the user input including at least one of user intent information and one or more user queries. The system further includes a score generating unit configured to generate ranking scores corresponding to the plurality of documents via a domain-specific ranking module by correlating the user intent information with respect to the plurality of documents, each ranking score indicating a relevance level of the document with respect to the user intent information. The system further includes a highlighting unit configured to highlight one or more excerpts in each document while displaying the plurality of documents based on the ranking scores. The system further includes a feedback unit configured to search for user feedback corresponding to the relevance of each document based on the ranking scores and the highlighted one or more excerpts. The system further includes a training unit configured to train the domain-specific ranking module based on the user feedback.

[0008] The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the exemplary aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description. [Brief description of the drawings]

[0009] The embodiments of the present disclosure itself, as well as preferred modes of use, further objects and advantages thereof, will be best understood by reference to the following detailed description of an exemplary embodiment when read in conjunction with the accompanying drawings, in which one or more embodiments are described, by way of example only, with reference to the accompanying drawings in which: [Figure 1A]FIG. 1A illustrates an environment 100 of a system for investigating document relevance according to one embodiment of the present disclosure. [Figure 1B] FIG. 1B illustrates an environment 100 of a system for investigating document relevance according to one embodiment of the present disclosure. [Figure 1C] FIG. 1C illustrates an environment 100 of a system for investigating document relevance according to one embodiment of the present disclosure. [Diagram 2] FIG. 2 illustrates a block diagram 200 of a system for investigating document relevance according to one embodiment of the present disclosure. [Diagram 3] FIG. 3 illustrates an example embodiment 300 illustrating a domain-specific ranking module, according to an embodiment of the present disclosure. [Figure 3A] FIG. 3A illustrates a dependency tree 300A corresponding to the first sentence, according to one embodiment of the present disclosure. [Figure 3B] FIG. 3B illustrates a dependency tree 300B that corresponds to the second sentence, according to one embodiment of the present disclosure. [Figure 3C] FIG. 3C illustrates an attention matrix 300C corresponding to the first sentence, according to one embodiment of the present disclosure. [Figure 3D] FIG. 3D illustrates an attention matrix 300D corresponding to the second sentence, according to one embodiment of the present disclosure. [Figure 4] FIG. 4 illustrates a flowchart 400 of a method for checking document relevance according to one embodiment of the present disclosure.

[0010] The figures depict embodiments of the present disclosure for illustrative purposes only. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods described herein may be employed without departing from the principles of the disclosure set forth herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The foregoing has outlined rather broadly the features and technical advantages of the present disclosure in order that the detailed description of the disclosure that follows may be better understood. It should be appreciated by those skilled in the art that the conception and specific embodiments disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present disclosure.

[0012] The novel features which are believed to be characteristic of the present disclosure, both as to its organization and method of operation, together with further objects and advantages, will be better understood from the following description when considered in connection with the accompanying figures, in which: It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description only and is not intended as a definition of the limits of the present disclosure.

[0013] Disclosed herein is a system and method for investigating the relevance of documents. The task of preparing research reports, such as evaluation reports, literature review reports, qualitative reports, and quantitative reports, is a very tedious and time-consuming task. An individual must spend a long time collecting data relevant to the individual's research. The individual must then analyze the collected data and investigate the relevance, applicability, quality, and significance of the data. However, the initial step of collecting the data and the subsequent step of analyzing the data are very tedious and can often take days to complete when done manually. Since the tasks of collecting the data and analyzing the data are very essential with respect to the quality of the report produced, these tasks need to be performed with utmost care. When performing the tasks manually, there is always room for improvement since it is not humanly possible for an individual to meticulously analyze every single document.

[0014] The present disclosure provides a system to assist a user in the task of collecting and analyzing documents. The system extracts documents from a data source based on a request from a user. The extracted documents are loosely based on the search the user wants to perform. The system then receives the user's intent behind performing the research along with a specific query the user wants to be answered by the system with respect to the extracted documents. The query may be domain specific and customized by the user based on the user's needs and research domain. For example, a user working in the biomedical domain will have different queries compared to a user working in the nanomaterials domain or the transportation domain. The system then analyzes each document with respect to the user's intent and determines the relevance level of each document. The relevance level may be indicated in the form of a ranking score. The system ranks the documents and displays the documents to the user in order of the ranking score. This allows the user to view the documents in order of relevance. Furthermore, the system also highlights important excerpts from the documents, helping the user to focus only on the highlighted excerpts instead of reading the entire document. Furthermore, the system also provides responses to the queries submitted by the user with respect to each and every document. In this way, it becomes relatively easy for a user to decide whether to include or exclude documents from his / her research. The time taken to reach such a decision is significantly reduced, since there is no need to study the entire document to investigate its relevance. The detailed operation and description of the system are described in the following paragraphs.

[0015] According to an embodiment of the present disclosure, Figures 1A-1C show an exemplary environment 100 of a system for investigating document relevance. It should be noted by those skilled in the art that the exemplary environment 100 of Figure 1A is described considering that a user wants to create a clinical evaluation report (CER) in the biomedical domain. Furthermore, those skilled in the art may understand that the system 102 may also be implemented in various environments other than that shown in Figure 1A.

[0016] According to one embodiment of the present disclosure, the exemplary environment 100 is described in conjunction with Figure 2, which illustrates a block diagram 200 of the system 102. Although the present disclosure is described with the system 102 being implemented on a server, it can be understood that the system 102 may be implemented as a tool in a variety of computing systems, such as a laptop computer, a desktop computer, a notebook, a workstation, a mainframe computer, a server, a network server, a cloud-based computing environment, etc.

[0017] In one implementation, the system 102 may include an I / O interface 202, a processor 204, a memory 206, and a unit 208. The memory 206 may be communicatively coupled to the processor 204 and the unit 208. Furthermore, the memory 206 may store the user intent information 108, one or more queries 110, one or more responses 222, and one or more tools 224. The meaning and use of each of the stored quantities is explained in the following paragraphs. The processor 204 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operational instructions. Among other functions, the processor 204 is configured to fetch and execute computer-readable instructions stored in the memory 206. The I / O interface 202 may include various software and hardware interfaces, such as a web interface, a graphical user interface, and the like. The I / O interface 202 may enable the system 102 to communicate with other computing devices, such as web servers and external data servers (not shown). The I / O interface 202 may facilitate communications within a wide variety of network and protocol types, including, for example, wired networks, such as LAN, cable, and wireless networks, such as WLAN, cellular, or satellite. The I / O interface 202 may include one or more ports for connecting multiple devices to each other or to another server.

[0018] In one implementation, the units 208 may include an extracting unit 231, a receiving unit 232, a score generating unit 233, a highlighting unit 234, a response generating unit 235, a feedback unit 236, a training unit 237, and a determining unit 238. According to embodiments of the present disclosure, these units 231-238 may include hardware components, such as a processor, a microprocessor, a microcontroller, an application specific integrated circuit, for performing various operations of the system 102. It should be understood that according to various embodiments of the present disclosure, the processor 204 may perform all the functions of the units 231-238.

[0019] Referring to FIG. 1A, the environment 100 shows a system 102 for extracting a plurality of documents from one or more data sources 210 based on a user request. According to FIG. 2, the extraction of the documents is performed by an extraction unit 231. In one embodiment, the user request includes a set of keywords 104 associated with the plurality of documents. For example, if the subject of the CER that the user wants to create is a report related to "quantitative determination of access alpha-fetoprotein (AFP) in human serum as an aid to managing patients with non-seminoma testicular cancer", the user provides a set of keywords 104 that may include "AFP", "human", and "testicular cancer". The extraction unit 231 based on the set of keywords extracts the plurality of documents. In another embodiment, the user request includes the user providing a plurality of unique IDs 106 associated with the plurality of documents. The plurality of unique IDs are provided to the extraction unit 231, which extracts a plurality of documents corresponding to the plurality of unique IDs 106. According to the exemplary environment 100 depicted in FIG. 1A, the extracted plurality of documents is 250.

[0020] Once the plurality of documents are extracted, the receiving unit 232 receives a user input. In one embodiment, the user input includes user intent information 108 that indicates the intent behind creating the CER using natural language. According to the exemplary environment 100, the user intent information 108 may read as follows: "The Access Alpha-Fetoprotein (AFP) Assay is a paramagnetic particle, chemiluminescent immunoassay for use in the Access InoM Assay System for the quantitative determination of AFP in human serum as an aid in the management of patients with non-seminoma testicular cancer, maternal serum and amniotic fluid to aid in the detection of fetal neural tube closure defects (ONTD)." As shown in FIG. 1A and FIG. 1C, the user input further includes one or more user queries 110 relating to the plurality of documents. For example, the one or more user queries 110 may include questions such as "What is the intent of the document?", "What is the device used for?", "What are the types of participants in the study?", etc. It should be noted by those skilled in the art that the one or more user queries 110 are customizable based on the needs of the user. That is, the one or more user queries 110 may be compiled or drafted based on the type of research the user is conducting and the domain the research is in. For example, according to the exemplary environment 100, the research is in the biomedical domain and the one or more queries 110 relate to the biomedical domain. However, a user whose research is in the nanomaterials domain may have a different set of queries than those described above and may customize the queries based on the needs of the user.

[0021] To investigate the relevance of the multiple documents with respect to the user intent information 108, the score generation unit 233 correlates the user intent information 108 with the multiple documents and generates a ranking score for each document. The ranking score indicates the relevance level of each document with respect to the user intent information 108. According to the exemplary environment 100 depicted in FIG. 1A, the user intent information 108 is correlated with each of the 250 extracted documents.

[0022] 3, to correlate the user intent information 108 for multiple documents, the score generation unit 233 employs a domain-specific ranking module 300. The domain-specific ranking module 300 is stored in the memory 206 as one of the one or more tools 224. Details of the domain-specific ranking module 300 are described with an example in the following paragraphs.

[0023] In step 302 of the domain-specific ranking module 300, a sentence pair is input. The sentence pair includes a first sentence from the user intent information 108 and a second sentence from one of the multiple documents. For example, the sentence pair may include: Sentence 1 - AFP is detected in patients with NSTC Sentence 2 - Elevated AFP revealed mixed germ cell tumours.

[0024] In step 304 of the domain-specific ranking module 300, a first dependency tree 300A corresponding to the first sentence and a second dependency tree 300B corresponding to the second sentence are generated. As illustrated in Figures 3A and 3B, the first and second dependency trees 300A, 300B represent the grammatical structure of the first and second sentences and reveal their syntactic structure.

[0025] In step 306 of the domain-specific ranking module 300, a domain-specific advanced language model is employed to encode a first set of tokens (or words) corresponding to the first sentence and a second set of tokens (or words) corresponding to the second sentence in a domain-specific manner so as not to change the meaning of the first sentence and the second sentence. The operation of the domain-specific advanced language model for the first sentence and the second sentence is described in Table 1 and Table 2, respectively.

[0026] [Table 1]

[0027] [Table 2]

[0028] However, if a normal language model is used instead of an advanced domain-specific language model, the tokens of the first sentence will be generated as ['a','##f','##p','is','detected','in','patients','with','n','##s','##t','##c','.'] and the tokens of the second sentence will be generated as ['elevated','a','##f','##p','revealed','mixed','ge','##rm','cell','tu','##mour','##s','.']. Thus, the meaning of the first and second sentences will be lost.

[0029] In step 308 of the domain-specific ranking module 300, a first attention matrix corresponding to the set of words in the first sentence and a second attention matrix corresponding to the set of words in the second sentence are created to establish an explicit dependency between the set of words in the first sentence and the set of words in the second sentence. The attention matrices corresponding to the set of words in the first sentence and the set of words in the second sentence are depicted in Figures 3C and 3D, respectively, with the highlighted cells being for attention weights.

[0030] In step 310 of the domain-specific ranking module 300, a mutual attention technique is employed to generate a semantic representation corresponding to the first sentence and the second sentence. The mutual attention considers synonyms and synonyms of words at different positions so that the similarity between words is not overlooked due to the positioning of the words or due to the use of synonyms or synonyms. For example, the first sentence "AFP is detected in patients with NSTC" can also be expressed as "NSTC patients are detected with AFP levels", and the second sentence "Elevated AFP revealed mixed germ cell tumours" can also be expressed as "Mixed germ cell tumours are detected with high AFP".

[0031] In step 312 of the domain-specific ranking module 300, the set of words of the first sentence and the set of words of the second sentence are fine-tuned to generate semantically meaningful sentence representations or embeddings. Fine-tuning is accomplished by providing the sets of words of the first sentence and the second sentence to a dual-branch network with shared parameters to minimize a loss function. This dual-branch network with shared parameters is pre-trained to minimize the loss function by providing a model with labeled sentence pairs of similarity feedback and minimizing the loss between the semantic representations of the sentences.

[0032] In step 314 of the domain-specific ranking module 300, ranking scores are generated for the refined first sentence and the second sentence by employing similarity measurement techniques, such as cosine similarity, Manhattan distance similarity, Euclidean distance similarity, etc. However, it may be noted by those skilled in the art that similarity measurement techniques other than those mentioned above may also be used.

[0033] The procedure described in steps 302-314 is performed iteratively to determine a ranking score for each of the plurality of documents according to the user intent information 108.

[0034] Once the ranking scores for the documents have been generated, the documents are displayed in order of their ranking scores, as depicted in FIG. 1B. In one embodiment, the document with the highest ranking score may be displayed at the top. However, in another embodiment, the document with the highest ranking score may be displayed at the bottom. Furthermore, as shown in FIG. 1B, each document is displayed with its unique ID and its ranking score. Although the ranking scores depicted in FIG. 1B are out of 100, it may be noted by one skilled in the art that the ranking scores may be generated or expressed in other formats.

[0035] Further, for each displayed document, the highlighting unit 234 highlights one or more excerpts that are important in terms of the user intent information 108. It can be understood by those skilled in the art that the one or more excerpts may include one or more paragraphs and / or one or more sentences. To highlight one or more important paragraphs, the domain-specific ranking module 300 is employed by the highlighting unit 234 to identify paragraphs in the document that have the highest relevance to the user intent information 108. Further, to highlight one or more important sentences of the important paragraph, the attention weight of each token (or word) of the sentence of the important paragraph and the user intent information 108 are calculated by the score generating unit 233. Based on the attention weight, each token is assigned a score. Based on the score, the start and end of the important part of the most relevant paragraph is obtained to be highlighted as an important sentence by the highlighting unit 234.

[0036] Highlighting one or more excerpts allows a user to easily identify the sections of the document that are most relevant to the user's intended goal as defined by the user in user intent information 108 .

[0037] Further, as shown in FIG. 1C, the system 102 allows the user to view one or more responses to the one or more user queries 110 provided by the user by selecting a response tab as shown. In FIG. 1B, responses to the one or more user queries 110 are provided by a response generation unit 235 employing a domain-specific advanced language model for question and answering (QA) stored as one or more tools 224 in the memory 206 of the system 102. The generated one or more responses 222 are stored in the memory 206. Further, based on the generated one or more responses, the feedback unit 236 allows the user to provide feedback on the one or more responses. For example, as shown in the exemplary environment 100 of FIG. 1A, if the user is not satisfied with the provided response or if the response generation unit fails to provide a response, the user is provided with an option to verify the response provided by the response generation unit 235 or to modify the response. Based on the feedback, the training unit 237 retrains the advanced language model for question and answering (QA).

[0038] 1A, the system 102 further allows the user to select one or more related documents by providing a decision tab. The decision tab is controlled by the decision unit 238 and provides the user with the option to either "include" or "exclude" a particular document. However, if the user cannot decide, the decision unit 238 allows the user to reconsider the particular document by providing the user with a "can't decide" option.

[0039] Thus, the system 102 through the interaction of its various components makes it easier for viewers to identify relevant documents with minimal time and effort, thereby improving the user experience. Furthermore, because the domain-specific ranking module 300 employs a combination of various techniques, the manner in which ranking scores for each of the multiple documents are generated is very comprehensive and accurate.

[0040] FIG. 4 illustrates a method 400 for investigating the relevance of multiple documents according to one embodiment of the present disclosure.

[0041] 4, method 400 includes one or more blocks describing a method for managing information in a manufacturing plant. Method 400 may be described in the general context of computer-executable instructions. Generally, computer-executable instructions may include routines, programs, objects, components, data structures, procedures, modules, and functions that perform particular functions or implement particular abstract data types.

[0042] The order in which method 400 is described is not intended to be construed as a limitation, and any number of the described method blocks may be combined in any order to implement the method. Additionally, individual blocks may be deleted from the method without departing from the spirit and scope of the described subject matter.

[0043] At block 402, the method 400 may include retrieving a number of documents from one or more data sources 210 based on a user request.

[0044] At block 404 , the method 400 may include receiving user input from a user, the user input including at least one of user intent information 108 and one or more user-queries 110 .

[0045] At block 406, the method 400 may include generating a ranking score for each of the plurality of documents by correlating the user intent information 108 for the plurality of documents.

[0046] At block 408, the method 400 may include highlighting one or more excerpts in each document while displaying the multiple documents based on the ranking scores.

[0047] At block 410, the method 400 may include exploring user feedback corresponding to the relevance of each document based on the ranking score and the highlighted excerpt or excerpts to train a domain-specific ranking module.

[0048] At block 412, the method 400 may include generating one or more responses 222 corresponding to the one or more user-queries 110 by using a domain-specific advanced language model for question and answering (QA).

[0049] At block 414 , the method 400 may include searching for user feedback corresponding to the accuracy of the one or more answers 222 with respect to the corresponding one or more user-queries 110 .

[0050] At block 416, the method 400 may include training a domain-specific advanced language model for question answering (QA) based on user feedback regarding the accuracy of the one or more answers 222.

[0051] A description of an embodiment having several components in communication with each other does not imply that all such components are required, rather, a variety of optional components are described to illustrate the wide variety of possible embodiments of the present disclosure.

[0052] It will be apparent that when a single device or article is described herein, two or more devices / articles (whether they cooperate or not) may be used in place of the single device / article. Similarly, when two or more devices or articles (whether they cooperate or not) are described herein, it will be apparent that a single device / article may be used in place of two or more devices or articles, or that a different number of devices / articles may be used in place of the number of devices or programs shown. Alternatively, the functionality and / or features of a device may be embodied by one or more other devices not explicitly described as having such functionality / features. Thus, other embodiments of the present disclosure need not include the device itself.

[0053] Finally, the language used herein has been selected primarily for readability and explanatory purposes, and may not be selected to define or limit the inventive subject matter. Accordingly, it is intended that the scope of the disclosure be limited not by this detailed description, but rather by any claims that issue on an application based hereon. Accordingly, the embodiments of the present disclosure are intended to illustrate, but not limit, the scope of the disclosure, which is defined in the following claims.

[0054] While various aspects and embodiments are disclosed herein, other aspects and embodiments will be apparent to those of ordinary skill in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Claims

1. 1. A processor-implemented method for document relevance testing, comprising: Retrieving (402) a plurality of documents from one or more data sources (210) based on a user request; receiving (404) user input from a user, the user input including at least one of user intent information (108) and one or more user queries (110); generating (406) a ranking score for each of the plurality of documents via a domain-specific ranking module by correlating the user intent information (108) with each of the plurality of documents, the ranking score indicating a level of relevance of each of the plurality of documents with respect to the user intent information (108), and correlating the user intent information (108) with each of the plurality of documents includes: For each of the plurality of documents, determining a first sentence from the user intent information (108) and a second sentence from the plurality of documents; encoding (306) a first set of tokens corresponding to the first sentence and a second set of tokens corresponding to the second sentence by employing a domain-specific advanced language model, the first set of tokens corresponds to a first set of words in the first sentence, and the second set of tokens corresponds to a second set of words in the second sentence. Step (306); generating (308) a first attention matrix corresponding to the first set of words and a second attention matrix corresponding to the second set of words to establish dependencies between the first set of words and the second set of words; generating (310) a semantic representation corresponding to the first sentence and the second sentence by employing mutual attention; and refining (312) the first set of words and the second set of words by providing the first set of words and the second set of words to a dual-branch network to determine a refined first sentence and a refined second sentence, The dual-branch network is trained to determine the fine-tuned first sentence and the fine-tuned second sentence by minimizing a loss function between a semantic representation corresponding to the first sentence and the semantic representation corresponding to the second sentence; the ranking scores of the corresponding documents are determined by employing at least one similarity measurement technique on the refined first sentence and the refined second sentence. Step (312): generating (406) highlighting (408) one or more excerpts in each document while displaying the plurality of documents based on the ranking scores; searching 410 for user feedback corresponding to the relevance of each document based on the ranking score and the highlighted excerpt or excerpts to train the domain-specific ranking module; A method comprising:

2. generating (412) one or more responses (222) corresponding to the one or more user queries (110) by using a domain-specific advanced language model for question and answering (QA); searching (414) for user feedback corresponding to the accuracy of the one or more responses with respect to the corresponding one or more user queries (110); training (416) the advanced language model for question answering (QA) based on the user feedback on the accuracy of the one or more responses (222); The method of claim 1 further comprising:

3. The method of claim 1 , wherein the user request includes a set of keywords (104) associated with the plurality of documents or a plurality of unique IDs (106) associated with the plurality of documents.

4. Highlighting 408 the one or more excerpts in each document may include: generating a second ranking score for each paragraph in each document with respect to the user intent information (108); selecting at least one paragraph from each document having a highest ranking score with respect to said user intent information (108); highlighting at least one important paragraph in each document by Highlighting 408 the one or more excerpts in each of the documents may include: Calculating attention weights of words in the at least one important paragraph and the user intent information (108); assigning a score to each of the plurality of words; obtaining a start point and an end point from the at least one significant paragraph based on the score; and highlighting at least one important sentence of the at least one important paragraph by The method of claim 1.

5. 1. A system for investigating document relevance, the system comprising:

1. A processor comprising: Retrieving documents from one or more data sources (210) based on a user request; receiving user input from a user, wherein the user input includes at least one of user intent information (108) and one or more user queries (110); generating ranking scores corresponding to the plurality of documents via a domain-specific ranking module (300) by correlating the user intent information (108) with each of the plurality of documents; wherein each ranking score indicates a level of relevance of a document with respect to said user intent information (108); wherein, to correlate the user intent information (108) with each of the plurality of documents, the processor: For each of the plurality of documents, determining a first sentence from the user intent information (108) and a second sentence from the plurality of documents; encoding a first set of tokens corresponding to the first sentence and a second set of tokens corresponding to the second sentence by employing a domain-specific advanced language model; wherein the first set of tokens corresponds to a first set of words in the first sentence and the second set of tokens corresponds to a second set of words in the second sentence; generating a first attention matrix corresponding to the first set of words and a second attention matrix corresponding to the second set of words to establish dependencies between the first set of words and the second set of words; generating a semantic representation corresponding to the first sentence and the second sentence by employing mutual attention; and fine-tuning the first set of words and the second set of words by providing the first set of words and the second set of words to a dual-branch network to determine a fine-tuned first sentence and a fine-tuned second sentence; wherein the dual-branch network is trained to determine the fine-tuned first sentence and the fine-tuned second sentence by minimizing a loss function between a semantic representation corresponding to the first sentence and a semantic representation corresponding to the second sentence; and The ranking scores of the corresponding documents are determined by employing at least one similarity measurement technique on the refined first sentence and the refined second sentence; It is structured as follows: highlighting one or more excerpts in each document while displaying the plurality of documents based on the ranking scores; searching for user feedback corresponding to the relevance of each document based on the ranking score and the highlighted excerpt or excerpts; A system comprising: a processor configured to train the domain-specific ranking module (300) based on the user feedback.

6. The processor, generating one or more responses (222) corresponding to the one or more user queries (110) by using a domain-specific advanced language model for question and answering (QA); searching for user feedback corresponding to the accuracy of the one or more answers (222) with respect to the corresponding one or more user queries (110); 6. The system of claim 5, further configured to train the advanced language model for question answering (QA) based on the user feedback regarding the accuracy of the one or more responses (222).

7. The system of claim 5 , wherein the user request includes a set of keywords (104) associated with the plurality of documents or a plurality of unique IDs (106) associated with the plurality of documents.

8. To highlight the one or more excerpts in each document, the processor: generating a second ranking score for each paragraph in each document with respect to the user intent information (108); selecting at least one paragraph from each document having a highest ranking score with respect to said user intent information (108); Highlight at least one important paragraph in each document by Calculating attention weights of words in the at least one important paragraph and the user intent information (108); assigning a score to each of the plurality of words; obtaining a start point and an end point from the at least one significant paragraph based on the score; and 6. The system of claim 5, further configured to highlight at least one important sentence of the at least one important paragraph by:

9. The method of claim 1 , wherein the plurality of documents is displayed in ascending or descending order of the ranking score of each of the plurality of documents.

10. A non-transitory computer readable medium having computer executable instructions stored thereon, the instructions, when executed by a processor, causing the processor to Retrieving (402) a plurality of documents from one or more data sources (210) based on a user request; receiving (404) user input from a user, the user input including at least one of user intent information (108) and one or more user queries (110); generating (406) a ranking score for each of the plurality of documents via a domain-specific ranking module by correlating the user intent information (108) with each of the plurality of documents, the ranking score indicating a level of relevance of each of the plurality of documents with respect to the user intent information (108), and correlating the user intent information (108) with each of the plurality of documents includes: For each of the plurality of documents, determining a first sentence from the user intent information (108) and a second sentence from the plurality of documents; encoding (306) a first set of tokens corresponding to the first sentence and a second set of tokens corresponding to the second sentence by employing a domain-specific advanced language model, the first set of tokens corresponds to a first set of words in the first sentence, and the second set of tokens corresponds to a second set of words in the second sentence. Step (306); generating (308) a first attention matrix corresponding to the first set of words and a second attention matrix corresponding to the second set of words to establish dependencies between the first set of words and the second set of words; generating (310) a semantic representation corresponding to the first sentence and the second sentence by employing mutual attention; and refining (312) the first set of words and the second set of words by providing the first set of words and the second set of words to a dual-branch network to determine a refined first sentence and a refined second sentence, The dual-branch network is trained to determine the fine-tuned first sentence and the fine-tuned second sentence by minimizing a loss function between a semantic representation corresponding to the first sentence and the semantic representation corresponding to the second sentence; the ranking scores of the corresponding documents are determined by employing at least one similarity measurement technique on the refined first sentence and the refined second sentence. Step (312): generating (406) highlighting (408) one or more excerpts in each document while displaying the plurality of documents based on the ranking scores; searching 410 for user feedback corresponding to the relevance of each document based on the ranking score and the highlighted excerpt or excerpts to train the domain-specific ranking module; A non-transitory computer readable medium for causing operations including:

11. The processor, generating one or more answers (222) corresponding to the one or more user queries (110) by using a domain-specific advanced language model for question and answering (QA); searching for user feedback corresponding to the accuracy of the one or more answers (222) with respect to the corresponding one or more user queries (110); training the advanced language model for question answering (QA) based on the user feedback regarding the accuracy of the one or more responses (222); 11. The non-transitory computer readable medium of claim 10, further configured to:

12. The non-transitory computer-readable medium of claim 10 , wherein the user request includes a set of keywords (104) associated with the plurality of documents or a plurality of unique IDs (106) associated with the plurality of documents.

13. The processor, generating a second ranking score for each paragraph in each document with respect to the user intent information (108); and selecting at least one paragraph from each document having a highest ranking score with respect to said user intent information (108); highlighting at least one important paragraph in each document by calculating attention weights of a plurality of words of the at least one important paragraph and the user intent information (108); assigning a score to each of the plurality of words; and obtaining a start point and an end point from the at least one significant paragraph based on the score; highlighting at least one important sentence of the at least one important paragraph by 11. The non-transitory computer-readable medium of claim 10, configured to highlight the one or more excerpts in each document by:

Images