Method and system for improving interpretability of RAG technology text generation content

Abstract

The invention relates to the technical field of natural language processing, and particularly discloses a method and system for improving the interpretability of RAG technology text generation content, and the method comprises the steps: determining a normalized weight value of each document fragment according to the semantic similarity between each document fragment associated with a problem text and the problem text in a retrieval stage of the RAG technology; the normalized weight value represents the contribution degree of the document fragment to the generated text; the method comprises the following steps of: for a generation stage of an RAG technology, obtaining a target document fragment on which each generated word in a generated text depends in a generation process and an attention weight value corresponding to the target document fragment; the attention weight value represents the importance degree of the target document fragment on which the generated word depends in the generation process. By transparentizing the retrieval process and the generation process of the RAG technology and providing clear and traceable explanation of the text generation content, the interpretability and the transparency of the RAG technology in the field with high credibility requirements can be improved.

Description

Background Art

[0002] RAG (Retrieval-Augmented Generation) technology is a method that has made important progress in the field of natural language processing in recent years. It combines information retrieval (retrieval) and text generation (generation), and generates more accurate and relevant text content by retrieving relevant documents or information fragments from an external knowledge base and using this information as input to the generation model. In the information retrieval part, RAG technology retrieves relevant documents or fragments from a large knowledge base (such as a document collection, database, web page content, etc.) by querying input data. Common retrieval methods include traditional TF-IDF or BM25-based retrieval, and semantic retrieval methods based on deep learning models (such as BERT, T5, etc.) can also be used. The latter can retrieve relevant documents based on semantic similarity. In the text generation part, the retrieved document fragments will be passed to the text generation model (such as GPT, T5, BART, etc.), and the generation model will generate natural language text by processing these documents. The generated text can be an answer to a question, an article summary, a reply to a conversation, etc.

[0003] Although RAG provides richer contextual information, the generated text often lacks sufficient interpretability, especially when the source of the retrieval results is unclear. The generated text may be a synthesis of multiple retrieved documents, and it is impossible to clearly indicate which specific information is used to generate the output. This is an urgent problem to be solved for some applications that require high interpretability (such as the medical field). Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a method and system for improving the interpretability of RAG technical text generated content.

[0005] In a first aspect, the present invention provides a method for improving the interpretability of RAG technical text generated content, and the technical solution of the method is as follows:

[0006] In the retrieval stage of the RAG technology, a normalized weight value of each document fragment is determined according to the semantic similarity between each document fragment associated with the question text and the question text; wherein the normalized weight value represents the contribution of the document fragment to the generated text corresponding to the question text;

[0007] For the generation stage of the RAG technology, the target document fragment that each generated word in the generated text depends on during the generation process and the attention weight value corresponding to the target document fragment are obtained; wherein the attention weight value represents the importance of the target document fragment that the generated word depends on during the generation process; the target document fragment is at least one document fragment among all document fragments.

[0008] The beneficial effects of the method for improving the interpretability of RAG technical text generation content of the present invention are as follows:

[0009] The method of the present invention can improve the interpretability and transparency of RAG technology in fields with high credibility requirements by making the retrieval process and generation process of RAG technology transparent and providing clear and traceable explanations of text generation content.

[0010] Based on the above scheme, a method of improving the interpretability of RAG technical text generated content of the present invention can also be improved as follows.

[0011] In an optional manner, the method further includes:

[0012] The question text is searched to obtain each document fragment associated with the question text.

[0013] In an optional manner, the step of determining the normalized weight value of each document fragment according to the semantic similarity between each document fragment associated with the question text and the question text includes:

[0014] Using the cosine similarity calculation formula, calculate the semantic similarity between each document fragment associated with the question text and the question text;

[0015] Determine the semantic similarity corresponding to any document fragment associated with the question text as the relevance score of the document fragment, until the relevance score of each document fragment is obtained;

[0016] The relevance scores of all document fragments are normalized to obtain a normalized weight value for each document fragment.

[0017] In an optional manner, the step of obtaining the target document fragment on which each generated word in the generated text depends during the generation process and the attention weight value corresponding to the target document fragment includes:

[0018] The text generation model is used to determine the target document fragment that each generated word in the generated text depends on during the generation process, and the Attention mechanism is used to calculate the attention weight value corresponding to the target document fragment that each generated word in the generated text depends on during the generation process.

[0019] In an optional manner, the generation process of the generated text includes at least one logical step, each logical step corresponds to a generated word and a target document fragment that the generated word depends on during the generation process and an attention weight value corresponding to the target document fragment.

[0020] In an optional manner, the method further includes:

[0021] Generate a first heat map or a first weight map representing the contribution degree of each document fragment to the generated text corresponding to the question text according to the normalized weight value of each document fragment;

[0022] According to the target document fragment that any generated word in the generated text depends on during the generation process and the attention weight value corresponding to the target document fragment, a second heat map or a second weight map representing the importance of the target document fragment that any generated word depends on during the generation process is generated.

[0023] In an optional manner, the method further includes:

[0024] Bidirectionally annotate any generated word with the target document fragment corresponding to the generated word and the attention weight value corresponding to the target document fragment to show the association between the generated word and the corresponding target document fragment.

[0025] In a second aspect, the present invention provides a system for improving the interpretability of RAG technical text generated content, and the technical solution of the system is as follows:

[0026] It includes: a first processing module and a second processing module;

[0027] The first processing module is used to: determine the normalized weight value of each document fragment according to the semantic similarity between each document fragment associated with the question text and the question text in the retrieval stage of the RAG technology; wherein the normalized weight value represents the contribution of the document fragment to the generated text corresponding to the question text;

[0028] The second processing module is used for: for the generation stage of the RAG technology, obtaining the target document fragment that each generated word in the generated text depends on during the generation process and the attention weight value corresponding to the target document fragment; wherein the attention weight value represents the importance of the target document fragment that the generated word depends on during the generation process; the target document fragment is at least one document fragment among all document fragments.

[0029] The beneficial effects of the system for improving the interpretability of RAG technical text generation content of the present invention are as follows:

[0030] The system of the present invention can improve the interpretability and transparency of RAG technology in areas with high credibility requirements by making the retrieval process and generation process of RAG technology transparent and providing clear and traceable explanations of text generation content.

[0031] In a third aspect, a technical solution of an electronic device of the present invention is as follows:

[0032] The invention comprises a memory, a processor and a program stored in the memory and running on the processor. When the processor executes the program, the steps of the method for improving the interpretability of the content generated by RAG technical text according to the present invention are implemented.

[0033] In a fourth aspect, the present invention provides a computer-readable storage medium having the following technical solution:

[0034] Instructions are stored in the computer-readable storage medium. When the computer-readable storage medium reads the instructions, the computer-readable storage medium executes the steps of the method for improving the interpretability of RAG technical text generated content according to the present invention.

[0035] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0037] Figure 1 A flow chart of an embodiment of a method for improving the interpretability of RAG technical text generation content of the present invention;

[0038] Figure 2 A schematic diagram of the structure of an embodiment of a system for improving the interpretability of RAG technical text generation content according to the present invention;

[0039] Figure 3 The figure is a schematic structural diagram of an embodiment of an electronic device of the present invention. DETAILED DESCRIPTION

[0040] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0041] Figure 1A flow chart of an embodiment of a method for improving the interpretability of RAG technical text generated content provided by the present invention is shown. The method for improving the interpretability of RAG technical text generated content can be executed by an electronic device such as a terminal device or a server. The terminal device can be any fixed or mobile terminal such as a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The server can be a single server or a server cluster composed of multiple servers. Any electronic device can implement the method for improving the interpretability of RAG technical text generated content by means of a processor calling computer-readable instructions stored in a memory. As Figure 1 As shown, the following steps are included:

[0042] S1. In the retrieval stage of the RAG technology, a normalized weight value of each document fragment is determined according to the semantic similarity between each document fragment associated with the question text and the question text.

[0043] Among them, the question text is: the text content corresponding to the question input by the user. In this embodiment, the question text defaults to the question text in the medical field, such as "How to effectively prevent seasonal influenza?", "How much water should you consume every day to stay healthy?", etc. The document fragment is: a document fragment associated with the question text, and the number of document fragments is at least one. Semantic similarity represents: the cosine similarity between the semantic vector corresponding to the question text and the language vector corresponding to the document fragment. The normalized weight value represents the contribution of the document fragment to the generated text corresponding to the question text. The larger the normalized weight value of a document fragment, the greater the contribution of the document fragment to the generated text.

[0044] S2. For the generation stage of the RAG technology, obtain the target document fragment that each generated word in the generated text depends on during the generation process and the attention weight value corresponding to the target document fragment.

[0045] Among them, the text generation model used by the RAG technology generates a generated text corresponding to the question text according to each document fragment associated with the question text. The generated text can be a text containing only the final answer or result, or it can be all texts including logical steps. In this embodiment, it defaults to all texts including logical steps. The target document fragment is the document fragment that the generated word depends on during the generation process, and the target document fragment is at least one document fragment among all document fragments. The target document fragments that different generated words depend on during the generation process may be different, which is determined according to the actual situation and is not limited here. For any target document fragment that a generated word depends on during the generation process, there is an attention weight value for the generated word respectively. The attention weight value represents the importance of the target document fragment that the generated word depends on during the generation process. The larger the attention weight value of a target document fragment for the corresponding generated word, the greater the importance of the target document fragment in the generation process of the corresponding generated word.

[0046] In an optional manner, the method further includes:

[0047] The question text is searched to obtain each document fragment associated with the question text.

[0048] Specifically, after receiving the question text input by the user, the pre-trained Transformer model is used to convert the question text into a high-dimensional question semantic vector for representation. The vector retrieval technology is used to retrieve document fragments similar to the question semantic vector in the vector knowledge base. It should be noted that in this embodiment, the number of document fragments can be determined by setting a threshold value of vector similarity.

[0049] Among them, the pre-trained Transformer models include: BERT, RoBERTa, etc. The vector retrieval technology is: FAISS or HNSW. Through the above retrieval methods, the most relevant information to the question content can be efficiently obtained from the knowledge base.

[0050] In an optional manner, the step of determining the normalized weight value of each document fragment according to the semantic similarity between each document fragment associated with the question text and the question text includes:

[0051] The cosine similarity calculation formula is used to calculate the semantic similarity between each document segment associated with the question text and the question text.

[0052] Specifically, the cosine similarity calculation formula is used to calculate the semantic similarity between any document fragment associated with the question text and the question text, and the above process is repeated until the semantic similarity between each document fragment associated with the question text and the question text is calculated.

[0053] It should be noted that the cosine similarity calculation formula is:

[0054]

[0055] Among them, q represents the vector representation of the question text, d i The vector representation of the i-th document fragment associated with the question text, Sim(q,d i ) is the semantic similarity between the question text and the i-th document fragment associated with the question text, ||q|| represents the Euclidean norm of q, ||d i || means d i The Euclidean norm of ; by calculating the semantic similarity, the relevance between the document fragment and the question text can be quantified.

[0056] The semantic similarity corresponding to any document segment associated with the question text is determined as the relevance score of the document segment, until the relevance score of each document segment is obtained.

[0057] Specifically, for each document fragment d i , use the following formula to calculate its relevance score:

[0058] Score(d i )=Sim(q,d i )

[0059] Among them, Score(d i ) means d i The relevance score is used to reflect the influence of the document fragment on the generated text during the generation process.

[0060] The relevance scores of all document fragments are normalized to obtain a normalized weight value for each document fragment.

[0061] Specifically, for each document fragment d i , use the following formula to calculate its normalized weight value:

[0062]

[0063] Where n represents the number of document fragments associated with the question text, Normalized Weight (d i ) represents the normalized weight value of the i-th document fragment.

[0064] In an optional manner, the step of obtaining the target document fragment on which each generated word in the generated text depends during the generation process and the attention weight value corresponding to the target document fragment includes:

[0065] The text generation model is used to determine the target document fragment that each generated word in the generated text depends on during the generation process, and the Attention mechanism is used to calculate the attention weight value corresponding to the target document fragment that each generated word in the generated text depends on during the generation process.

[0066] Among them, text generation models include: GPT, T5, BART, etc. The Attention mechanism can help users understand the dependencies behind each generated word. In the process of generating text, the model uses Attention to decide which document fragments it "pays attention to". In this embodiment, the attention weight value is calculated based on Dot-Product Attention:

[0067]

[0068] Among them, wt represents the tth generated word; score(w t ,d i ) indicates w t With d i The matching degree between them is measured using the dot product; α t,i Indicates d i For w t The attention weight value.

[0069] It should be noted that, in order to ensure transparency during the generation process, this embodiment indicates which retrieval segments the text segment (e.g., each word) is related to each time the text is generated. In order to track the decision basis of the generation process, it is necessary to indicate the relationship between the generated word and the retrieved related document segment at each step of the generation model. t When recording the target document fragment set D it depends on t ={d 1 ,d 2 ,...,d k}, the contribution weight of each fragment is α t,i . Specifically:

[0070] w t =f(Input,D t ,α t,1 ,a t,2 ,...,a t,k )

[0071] Where f represents the generation function of the text generation model (such as Transformer), D t represents the set of target document fragments related to the tth generated word, α t,i Represents the target document fragment d i To generate word wt The contribution weight (attention weight value) of .

[0072] In an optional manner, the generation process of the generated text includes at least one logical step, each logical step corresponds to a generated word and a target document fragment that the generated word depends on during the generation process and an attention weight value corresponding to the target document fragment.

[0073] In order to further improve the interpretability, this embodiment divides the text generation process into multiple logical steps, each of which corresponds to a generated word and the target document fragment that the generated word depends on during the generation process and the attention weight value corresponding to the target document fragment, and provides a detailed reasoning path and retrieval fragment description for each step. Each logical step is represented as:

[0074] GeneratedStep t =(w t ,{(d i ,α t,i )|i=1,2,...,k})

[0075] Among them, Generated Step t is the generated text content generated by the tth logical step, including the generated word w t , generate word w t The corresponding target document fragment set D t and the attention weight α t,i , k is the number of target document fragments. It should be noted that each logical step generates a word, that is, the t-th logical step generates the t-th generated word.

[0076] In an optional manner, the method further includes:

[0077] According to the normalized weight value of each document fragment, a first heat map or a first weight map is generated to characterize the contribution degree of each document fragment to the generated text corresponding to the question text.

[0078] Among them, the first heat map and the first weight map can visually display the contribution degree of each document fragment.

[0079] According to the target document fragment that any generated word in the generated text depends on during the generation process and the attention weight value corresponding to the target document fragment, a second heat map or a second weight map representing the importance of the target document fragment that any generated word depends on during the generation process is generated.

[0080] The second heat map and the second weight map can visually display the importance of the target document fragment that the generated word depends on during the generation process. The second heat map and the second weight map corresponding to each generated word are different.

[0081] In an optional manner, the method further includes:

[0082] Bidirectionally annotate any generated word with the target document fragment corresponding to the generated word and the attention weight value corresponding to the target document fragment to show the association between the generated word and the corresponding target document fragment.

[0083] By establishing a bidirectional annotation between the generated words and the document fragment, each generated word or sentence can be traced back to its source document fragment. t After that, the source document fragment is marked. i and its correlation score α t,i :

[0084] w t ={(d i ,α t,i )|i=1,2,...,k}

[0085] Specifically, another technical solution in this embodiment includes the following steps:

[0086] S10. Search the question text in the medical field to obtain each document fragment associated with the question text.

[0087] S20. In the retrieval stage of the RAG technology, a normalized weight value of each document fragment is determined based on the semantic similarity between each document fragment associated with the question text and the question text; wherein the normalized weight values ​​respectively represent the contribution of the document fragment to the generated text corresponding to the question text.

[0088] S30. For the generation stage of the RAG technology, the target document fragment that each generated word in the generated text depends on during the generation process and the attention weight value corresponding to the target document fragment are obtained; wherein the attention weight value represents the importance of the target document fragment that the generated word depends on during the generation process; the target document fragment is at least one document fragment among all document fragments.

[0089] S40. Generate a first heat map or a first weight map representing the contribution of each document fragment to the generated text corresponding to the question text according to the normalized weight value of each document fragment; generate a second heat map or a second weight map representing the importance of the target document fragment on which any generated word in the generated text depends during the generation process according to the target document fragment on which any generated word in the generated text depends during the generation process and the attention weight value corresponding to the target document fragment.

[0090] S50, bidirectionally annotate any generated word with the target document fragment corresponding to the generated word and the attention weight value corresponding to the target document fragment to show the association between the generated word and the corresponding target document fragment.

[0091] The technical solution of this embodiment can improve the interpretability and transparency of RAG technology in areas with high credibility requirements by making the retrieval and generation processes of RAG technology transparent and providing clear and traceable explanations of text generation content.

[0092] Figure 2 FIG. 2 is a schematic diagram showing a structure of an embodiment of a system 200 for improving the interpretability of RAG technical text generated content provided by the present invention. Figure 2 As shown, the system 200 includes: a first processing module 210 and a second processing module 220;

[0093] The first processing module 210 is used to: determine the normalized weight value of each document fragment according to the semantic similarity between each document fragment associated with the question text and the question text in the retrieval stage of the RAG technology; wherein the normalized weight value represents the contribution degree of the document fragment to the generated text corresponding to the question text;

[0094] The second processing module 220 is used for: for the generation stage of the RAG technology, obtaining the target document fragment that each generated word in the generated text depends on during the generation process and the attention weight value corresponding to the target document fragment; wherein the attention weight value represents the importance of the target document fragment that the generated word depends on during the generation process; the target document fragment is at least one document fragment among all document fragments.

[0095] In an optional manner, the method further includes: a retrieval module; the retrieval module is used to:

[0096] The question text is searched to obtain each document fragment associated with the question text.

[0097] In an optional manner, the first processing module is specifically used to:

[0098] Using the cosine similarity calculation formula, calculate the semantic similarity between each document fragment associated with the question text and the question text;

[0099] Determine the semantic similarity corresponding to any document fragment associated with the question text as the relevance score of the document fragment, until the relevance score of each document fragment is obtained;

[0100] The relevance scores of all document fragments are normalized to obtain a normalized weight value for each document fragment.

[0101] In an optional manner, the second processing module 220 is specifically used for:

[0102] The text generation model is used to determine the target document fragment that each generated word in the generated text depends on during the generation process, and the Attention mechanism is used to calculate the attention weight value corresponding to the target document fragment that each generated word in the generated text depends on during the generation process.

[0103] In an optional manner, the generation process of the generated text includes at least one logical step, each logical step corresponds to a generated word and a target document fragment that the generated word depends on during the generation process and an attention weight value corresponding to the target document fragment.

[0104] In an optional manner, the method further includes: a visualization module; the visualization module is used to:

[0105] Generate a first heat map or a first weight map representing the contribution degree of each document fragment to the generated text corresponding to the question text according to the normalized weight value of each document fragment;

[0106] According to the target document fragment that any generated word in the generated text depends on during the generation process and the attention weight value corresponding to the target document fragment, a second heat map or a second weight map representing the importance of the target document fragment that any generated word depends on during the generation process is generated.

[0107] In an optional manner, the method further includes: an association module; the association module is used to:

[0108] Bidirectionally annotate any generated word with the target document fragment corresponding to the generated word and the attention weight value corresponding to the target document fragment to show the association between the generated word and the corresponding target document fragment.

[0109] It should be noted that the beneficial effects of the system for improving the interpretability of the content generated by the RAG technical text provided by the above-mentioned embodiment are the same as the beneficial effects of the method for improving the interpretability of the content generated by the RAG technical text, which will not be repeated here. In addition, when the system provided by the above-mentioned embodiment realizes its functions, it only uses the division of the above-mentioned functional modules as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided by the above-mentioned embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0110] Among them, the system for improving the interpretability of the content generated by RAG technical text of the present invention can be a computer program (including program code) running in a computer device. For example, the system for improving the interpretability of the content generated by RAG technical text of the present invention is an application software that can be used to execute the corresponding steps of the method for improving the interpretability of the content generated by RAG technical text of the present invention.

[0111] In some embodiments, the system for improving the interpretability of the content generated by RAG technical text of the present invention can be implemented in a combination of software and hardware. As an example, the system for improving the interpretability of the content generated by RAG technical text of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the method for improving the interpretability of the content generated by RAG technical text of the present invention. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (ASIC, Application Specific Integrated Circuit), DSP, programmable logic device (PLD, Programmable Logic Device), complex programmable logic device (CPLD, Complex Programmable Logic Device), field programmable gate array (FPGA, Field-Programmable Gate Array) or other electronic components.

[0112] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The name of a module does not limit the module itself in some cases.

[0113] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any one of the above-mentioned methods for improving the interpretability of content generated by RAG technical text is implemented. That is, an electronic device according to an embodiment of the present invention may include but is not limited to: a processor and a memory; a memory for storing a computer program; a processor for executing the method for improving the interpretability of content generated by RAG technical text as shown in any embodiment of the present invention by calling a computer program.

[0114] In an optional embodiment, an electronic device is provided, such as Figure 3 As shown, Figure 3 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0115] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0116] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3In the figure, only one thick line is used to represent the bus 4002, but this does not mean that there is only one bus or one type of bus.

[0117] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this.

[0118] The memory 4003 is used to store the application code (computer program) for executing the solution of the present invention, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the application code stored in the memory 4003 to implement the content shown in the above method embodiment.

[0119] Among them, the electronic device can also be a terminal device, and the terminal device can be any terminal device that can install applications and access web pages through applications, including at least one of a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart TV, and a smart car device.

[0120] It should be noted that Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0121] A computer-readable storage medium according to an embodiment of the present invention stores a computer program on the computer-readable storage medium, and when the computer program is executed by a processor, any of the above methods for improving the interpretability of content generated by RAG technical text is implemented.

[0122] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0123] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the above-mentioned method for improving the interpretability of the content generated by the RAG technical text.

[0124] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0125] It should be understood that the flow charts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the module, the program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0126] The computer-readable storage medium provided by the embodiment of the present invention may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or component.

[0127] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0128] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the disclosure scope involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) by each other.

[0129] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0130] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0131] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for improving the interpretability of RAG technical text generation content, characterized in that: include: In the retrieval stage of the RAG technology, a normalized weight value of each document fragment is determined according to the semantic similarity between each document fragment associated with the question text and the question text; wherein the normalized weight value represents the contribution of the document fragment to the generated text corresponding to the question text; For the generation stage of the RAG technology, the target document fragment that each generated word in the generated text depends on during the generation process and the attention weight value corresponding to the target document fragment are obtained; wherein the attention weight value represents the importance of the target document fragment that the generated word depends on during the generation process; the target document fragment is at least one document fragment among all document fragments.

2. The method for improving the interpretability of RAG technical text generation content according to claim 1, characterized in that: Also includes: The question text is searched to obtain each document fragment associated with the question text.

3. The method for improving the interpretability of RAG technical text generation content according to claim 1, characterized in that: The step of determining a normalized weight value of each document fragment according to the semantic similarity between each document fragment associated with the question text and the question text comprises: Using the cosine similarity calculation formula, calculate the semantic similarity between each document fragment associated with the question text and the question text; Determine the semantic similarity corresponding to any document fragment associated with the question text as the relevance score of the document fragment, until the relevance score of each document fragment is obtained; The relevance scores of all document fragments are normalized to obtain a normalized weight value for each document fragment.

4. The method for improving the interpretability of RAG technical text generation content according to claim 1, characterized in that: The step of obtaining the target document fragment on which each generated word in the generated text depends during the generation process and the attention weight value corresponding to the target document fragment comprises: The text generation model is used to determine the target document fragment that each generated word in the generated text depends on during the generation process, and the Attention mechanism is used to calculate the attention weight value corresponding to the target document fragment that each generated word in the generated text depends on during the generation process.

5. The method for improving the interpretability of RAG technical text generation content according to claim 1, characterized in that: The generation process of the generated text includes at least one logical step, each logical step corresponds to a generated word and a target document fragment that the generated word depends on during the generation process and an attention weight value corresponding to the target document fragment.

6. The method for improving the interpretability of RAG technical text generation content according to any one of claims 1 to 5, characterized in that: Also includes: Generate a first heat map or a first weight map representing the contribution degree of each document fragment to the generated text corresponding to the question text according to the normalized weight value of each document fragment; According to the target document fragment that any generated word in the generated text depends on during the generation process and the attention weight value corresponding to the target document fragment, a second heat map or a second weight map representing the importance of the target document fragment that any generated word depends on during the generation process is generated.

7. The method for improving the interpretability of RAG technical text generation content according to any one of claims 1 to 5, characterized in that: Also includes: Bidirectionally annotate any generated word with the target document fragment corresponding to the generated word and the attention weight value corresponding to the target document fragment to show the association between the generated word and the corresponding target document fragment.

8. A system for improving the interpretability of RAG technical text generation content, characterized in that: include: a first processing module and a second processing module; The first processing module is used to: determine the normalized weight value of each document fragment according to the semantic similarity between each document fragment associated with the question text and the question text in the retrieval stage of the RAG technology; wherein the normalized weight value represents the contribution of the document fragment to the generated text corresponding to the question text; The second processing module is used for: for the generation stage of the RAG technology, obtaining the target document fragment that each generated word in the generated text depends on during the generation process and the attention weight value corresponding to the target document fragment; wherein the attention weight value represents the importance of the target document fragment that the generated word depends on during the generation process; the target document fragment is at least one document fragment among all document fragments.

9. An electronic device, characterized in that: The electronic device includes a processor, the processor is coupled to a memory, the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the electronic device implements the method for improving the interpretability of RAG technical text generation content as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the computer-readable storage medium implements the method for improving the interpretability of RAG technical text generation content as described in any one of claims 1 to 7.

Images