User question answering processing method and device based on uploaded document

Abstract

The invention provides a user question answering processing method and device based on an uploaded document, and relates to the technical field of intelligent question answering. The method comprises the following steps: replacing a picture link with a token to obtain a document analysis content; processing the document analysis content to obtain a document slice vector, and adding the picture content added with the identifier and the picture link to a document target position according to the document slice vector to obtain updated document analysis content; writing the updated document analysis content and the document slice vector into a vector library, and obtaining at least one target content for answering the user question from the vector library; and replacing the picture link in each piece of target content with a token, splicing each piece of target content after the token is replaced, and calling the large model to process the spliced content to obtain a model output result. According to the method and the device provided by the embodiment of the invention, the accuracy of the answer result aiming at the user question can be improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent question-answering technology, specifically to a method and apparatus for processing user questions and answers based on uploaded documents. Background Technology

[0002] Retrieval Augmented Generation (RAG) is an AI framework that combines information retrieval and text generation. It works through two core steps: first, retrieving relevant document fragments from an external knowledge base; and then generating a response using a large model based on the retrieval results. This approach overcomes the limitations of traditional language models that rely solely on training data, enabling dynamic access to the latest information and significantly improving the accuracy and real-time performance of the generated content.

[0003] In existing technologies, when a program processes a document, the first problem is that images in the document are usually replaced with links. When the large model summarizes relevant content, it is easy to modify the content of the links, resulting in the inability to display relevant image content in the final answer. The second problem is that when the user's question is related to the image content, the lack of parsing of the image content makes it impossible to retrieve relevant content, resulting in inaccurate answers to the user's question.

[0004] Regarding the first issue, existing technologies generally reduce the probability of large models generating incorrect links through prompt word engineering. This involves explicitly instructing the large model in the prompt words to retain or correctly process Markdown-formatted image links. For example, the model might be instructed to "retain all Markdown syntax for  in the response."

[0005] Its drawback is that language models (LLMs) are essentially probabilistic models, and the models cannot always perfectly understand and maintain complex formats. They may lose or modify characters and other content, leading to broken links.

[0006] The effectiveness of this method is significantly correlated with the parameter size of the large model. Models with smaller parameters have limitations in understanding and executing complex prompts, leading to a higher frequency of such problems.

[0007] Regarding the first issue, the existing technology's program logic directly appends the relevant content parsed from the image to the image link to ensure that the relevant content can be retrieved during the search.

[0008] Its drawback is that the program's stitching is too simple, and large models may not be able to distinguish which part is the image content and which part is the original document text, resulting in the final result containing content that does not exist in the document. Summary of the Invention

[0009] To address the problems in the prior art, embodiments of the present invention provide a method and apparatus for processing user questions and answers based on uploaded documents, which can at least partially solve the problems existing in the prior art.

[0010] On one hand, this invention proposes a user question answering method based on uploaded documents, including:

[0011] The uploaded document is parsed to obtain image links, and the image links are replaced with tokens to obtain the parsed document content;

[0012] The document parsing content is processed to obtain a document slice vector. Based on the document slice vector, the image content with added identifier and the image link are added to the target position of the document to obtain the updated document parsing content.

[0013] The updated document parsing content and the document slice vector are written into a vector library, and at least one target content for answering user questions is obtained from the vector library; each target content includes the image content, the image link, and the document parsing content at the target location in the document;

[0014] Replace the image link in each of the target contents with the token, concatenate each target content after replacing the token, and call the large model to process the concatenated content to obtain the model output result.

[0015] The document slice vector includes the preceding slice vector and the following slice vector; correspondingly, determining the target location of the document includes:

[0016] Calculate the first vector similarity between the image content vector corresponding to the image content and the image preceding slice vector, and calculate the second vector similarity between the image content vector and the image subsequent slice vector;

[0017] The position of the slice of the vector corresponding to the larger value of the first vector similarity and the second vector similarity in the document is determined as the target position of the document.

[0018] The step of obtaining the target content for answering user questions from the vector library includes:

[0019] Obtain the user's question and vectorize it.

[0020] Based on the vectorized user question, a multi-path retrieval is performed in the vector database to obtain the target content.

[0021] The step of concatenating each target content after replacing the token includes:

[0022] Each target content after the token replacement is concatenated according to the preset concatenation order.

[0023] Prior to the step of calling the large model to process the concatenated content, the user question answering method based on the uploaded document further includes:

[0024] Generate a large model hint word, and append the large model hint word before the concatenated content;

[0025] The large model is invoked to process the concatenated content containing prompts from the large model.

[0026] The method for processing user questions based on uploaded documents, after the step of calling the large model to process the concatenated content and obtain the model output, further includes:

[0027] Replace the token with the image link, parse the image link to obtain the image, and display the image at the target location in the document.

[0028] On one hand, the present invention proposes a user question answering device based on uploaded documents, comprising:

[0029] The first acquisition unit is used to parse the uploaded document, obtain the image link, and replace the image link with a token to obtain the document parsing content;

[0030] The second acquisition unit is used to process the document parsing content to obtain a document slice vector, and add the marked image content and image link to the target position of the document according to the document slice vector to obtain the updated document parsing content;

[0031] The third acquisition unit is used to write the updated document parsing content and the document slice vector into a vector library, and to acquire at least one target content for answering user questions from the vector library; each target content includes the image content, the image link, and the document parsing content at the target position in the document;

[0032] The fourth acquisition unit is used to replace the image link in each of the target contents with the token, concatenate each target content after replacing the token, and call the large model to process the concatenated content to obtain the model output result.

[0033] In another aspect, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the following method:

[0034] The uploaded document is parsed to obtain image links, and the image links are replaced with tokens to obtain the parsed document content;

[0035] The document parsing content is processed to obtain a document slice vector. Based on the document slice vector, the image content with added identifier and the image link are added to the target position of the document to obtain the updated document parsing content.

[0036] The updated document parsing content and the document slice vector are written into a vector library, and at least one target content for answering user questions is obtained from the vector library; each target content includes the image content, the image link, and the document parsing content at the target location in the document;

[0037] Replace the image link in each of the target contents with the token, concatenate each target content after replacing the token, and call the large model to process the concatenated content to obtain the model output result.

[0038] This invention provides a computer-readable storage medium, comprising:

[0039] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the following method:

[0040] The uploaded document is parsed to obtain image links, and the image links are replaced with tokens to obtain the parsed document content;

[0041] The document parsing content is processed to obtain a document slice vector. Based on the document slice vector, the image content with added identifier and the image link are added to the target position of the document to obtain the updated document parsing content.

[0042] The updated document parsing content and the document slice vector are written into a vector library, and at least one target content for answering user questions is obtained from the vector library; each target content includes the image content, the image link, and the document parsing content at the target location in the document;

[0043] Replace the image link in each of the target contents with the token, concatenate each target content after replacing the token, and call the large model to process the concatenated content to obtain the model output result.

[0044] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the following method:

[0045] The uploaded document is parsed to obtain image links, and the image links are replaced with tokens to obtain the parsed document content;

[0046] The document parsing content is processed to obtain a document slice vector. Based on the document slice vector, the image content with added identifier and the image link are added to the target position of the document to obtain the updated document parsing content.

[0047] The updated document parsing content and the document slice vector are written into a vector library, and at least one target content for answering user questions is obtained from the vector library; each target content includes the image content, the image link, and the document parsing content at the target location in the document;

[0048] Replace the image link in each of the target contents with the token, concatenate each target content after replacing the token, and call the large model to process the concatenated content to obtain the model output result.

[0049] The present invention provides a user question answering method and apparatus based on uploaded documents. The method parses the uploaded document to obtain image links, replaces the image links with tokens, and obtains document parsing content. The document parsing content is processed to obtain document slice vectors. Based on the document slice vectors, image content with added identifiers and image links are added to the target position in the document to obtain updated document parsing content. The updated document parsing content and the document slice vectors are written into a vector library, and at least one target content for answering user questions is obtained from the vector library. Each target content includes the image content, the image link, and document parsing content at the target position in the document. The image links in each target content are replaced with the tokens. Each target content after token replacement is concatenated, and a large model is called to process the concatenated content to obtain the model output result. This method can improve the accuracy of answering user questions. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0051] Figure 1 This is a flowchart illustrating a user question answering method based on uploaded documents, provided in an embodiment of the present invention.

[0052] Figure 2 This is a flowchart illustrating a user question answering method based on uploaded documents, provided in another embodiment of the present invention.

[0053] Figure 3This is a flowchart illustrating a user question answering method based on uploaded documents, provided in another embodiment of the present invention.

[0054] Figure 4 This is a schematic diagram of a user question answering device based on uploaded documents provided in an embodiment of the present invention.

[0055] Figure 5 This is a schematic diagram of the physical structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments and descriptions of the present invention are used to explain the present invention, but are not intended to limit the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.

[0057] Explanation of relevant terms:

[0058] RAG: Retrieval-Augmented Generation.

[0059] Semantic slicing: Dividing the original document into several chunks according to semantic integrity for vector retrieval and generation tasks.

[0060] Token: A short text character format provided in this embodiment of the invention, used to represent different image links.

[0061] Figure 1 This is a flowchart illustrating a user question answering method based on uploaded documents according to an embodiment of the present invention, as shown below. Figure 1 As shown, the user question answering method based on uploaded documents provided in this embodiment of the invention includes:

[0062] Step S1: Parse the uploaded document to obtain the image link, and replace the image link with the token to obtain the parsed document content.

[0063] Step S2: Process the document parsing content to obtain a document slice vector. Based on the document slice vector, add the marked image content and image links to the target position in the document to obtain the updated document parsing content.

[0064] Step S3: Write the updated document parsing content and the document slice vector into the vector library, and obtain at least one target content for answering the user's question from the vector library; each target content includes the image content, the image link, and the document parsing content at the target location in the document.

[0065] Step S4: Replace the image link in each of the target contents with the token, concatenate each target content after replacing the token, and call the large model to process the concatenated content to obtain the model output result.

[0066] In step S1 above, the device parses the uploaded document to obtain an image link, and replaces the image link with a token to obtain the parsed document content. The device can be a computer device executing this method, specifically an intelligent question-answering large-scale model platform (integrating a large model). The acquisition, storage, use, and processing of data in this application's technical solution all comply with relevant regulations.

[0067] like Figure 2 As shown, the document uploader can be the user who raised the question. During document parsing, images can be saved to the server and corresponding image links can be generated, such as https: / / xxxxxx.png.

[0068] The analysis is as follows:

[0069]  Similar to the hold-out method, there are multiple ways to divide a dataset D into k subsets. To reduce the differences introduced by different sample partitions, k-fold cross-validation usually involves randomly using different partitions and repeating the process p times. The final evaluation result is the mean of these p k-fold cross-validation results. Common examples include…

[0070] A corresponding token is generated for each image link for replacement to prevent a single image link from being split into multiple segments during semantic segmentation. The explanation is as follows:

[0071] After performing regular expression extraction, the token reference table is generated as follows:

[0072] "(URL=0)":"".

[0073] "(URL=1)":"".

[0074] In this context, URL=0 and URL=1 are tokens.

[0075] The document parsing content is as follows:

[0076] (URL=0)

[0077] Similar to the hold-out method, there are multiple ways to divide a dataset D into k subsets. To reduce the differences introduced by different sample partitions, k-fold cross-validation usually involves randomly using different partitions and repeating the process p times. The final evaluation result is the mean of the results of these p k-fold cross-validation iterations. For example, common methods include…”.

[0078] In step S2 above, the device processes the parsed document content to obtain document slice vectors. Based on these document slice vectors, it adds the identified image content and image links to the target location in the document, thus updating the parsed document content. The processing of the parsed document content includes semantic slicing and vectorization.

[0079] The document content after the links are replaced (the parsed document content) undergoes semantic-level integrity slicing to ensure the integrity of image links and their corresponding descriptions, preventing the complete semantic content from being split into two parts. The explanation is as follows:

[0080] Segmenting the sentences yields the following content:

[0081] (URL=0)

[0082] Similar to the hold-out method, there are also multiple ways to divide a dataset D into k subsets.

[0083] To reduce the differences introduced by different sample partitions, k-fold cross-validation usually involves randomly using different partitions and repeating it p times. The final evaluation result is the mean of the results of these p k-fold cross-validation attempts. For example, common methods include...

[0084] Calculate vectors based on sentences.

[0085] Sentences within each paragraph are compared pairwise using vector similarity. Sentences with similarity below a set threshold are grouped into two segments, as shown below:

[0086] (URL=0)

[0087] Similar to the hold-out method, there are multiple ways to divide a dataset D into k subsets. To reduce the differences introduced by different sample partitions, k-fold cross-validation usually involves randomly using different partitions and repeating the process p times. The final evaluation result is the mean of the results of these p k-fold cross-validation iterations. For example, common methods include…

[0088] Content extraction was performed on the image using a multimodal large model, resulting in the following image content:

[0089] "This image illustrates the process of 10-fold cross-validation. The dataset D is divided into 10 subsets (D1 to D10)."

[0090] The image content is vectorized, for example, to obtain [0.0049567847,0.004308458,-0.01197175……].

[0091] The parsed document content after semantic slicing is vectorized to obtain document slice vectors. Specifically, this includes vectorizing the preceding image slices and the subsequent image slices to obtain the preceding image slice vector A1 and the subsequent image slice vector A2.

[0092] The document slice vector includes the image preceding slice vector and the image following slice vector; correspondingly, determining the document target position includes:

[0093] Calculate the first vector similarity between the image content vector corresponding to the image content and the image preceding slice vector, and calculate the second vector similarity between the image content vector and the image subsequent slice vector;

[0094] The location of the slice in the document corresponding to the vector with the larger similarity between the first and second vectors is determined as the document target location. If the second vector similarity is greater than the first vector similarity, then the location of the subsequent image slice in the document corresponding to the subsequent image slice vector is determined as the document target location.

[0095] Add the marked image content and image link to the target location in the document, and the updated document parsing content will be as follows:

[0096] "" Similar to the hold-out method, there are multiple ways to divide a dataset D into k subsets. To reduce the differences introduced by different sample partitions, k-fold cross-validation usually involves randomly using different partitions and repeating the process p times. The final evaluation result is the mean of the results of these p k-fold cross-validation iterations. For example, common methods include…

[0097] The image with the added label reads: "[This image shows the process of 10-fold cross-validation. Dataset D is divided into 10 subsets (D1 to D10)].]".

[0098] The image link is "(https: / / xxxxxxx.png)". ".

[0099] It's important to note that if the target location in the document is determined by the position of subsequent image slices within the document, then the image content and link with the added identifier will be appended before the target location. That is, "(https: / / xxxxxxx.png)". "Similar to the method of leaving out" continues.

[0100] If the position of the preceding image slice in the document is determined as the document target position, then the image content with the added identifier and the image link are added after the document target position. That is, the last sentence in the preceding image slice content follows "![This image illustrates the process of 10-fold cross-validation. Dataset D is divided into 10 subsets (D1 to D10)".

[0101] In step S3 above, the device writes the updated document parsing content and the document slice vector into a vector library, and obtains at least one target content for answering the user's question from the vector library; each target content includes the image content, the image link, and the document parsing content at the target position in the document. Writing the updated document parsing content and the document slice vector into the vector library requires attention to the fact that the document slice vector here includes not only the preceding and following image slice vectors, but also other slice vectors within the document. The storage structure in the vector library is as follows:

[0102] {

[0103] "_index": "kbv2-test",

[0104] "_id": "1",

[0105] "_score": 1,

[0106] "_source": {

[0107] "kb_id": "1898970011892219904",

[0108] "file_id": "1898970011892219904",

[0109] "knowledge_id": "1898970011892219904",

[0110] "is_title": 0,

[0111] "parent_title": "2.2.2 Cross-validation",

[0112] "content": "![This image illustrates the process of 10-fold cross-validation. Dataset D is divided into 10 subsets (D1 to D10).] (https: / / xxxxxxx.png)" Similar to the hold-out method, there are multiple ways to divide a dataset D into k subsets. To reduce the differences introduced by different sample partitions, k-fold cross-validation usually involves randomly using different partitions and repeating the process p times. The final evaluation result is the mean of these p k-fold cross-validation results.

[0113] "content_vector": [

[0114] 0.0049567847,

[0115] 0.004308458,

[0116] -0.01197175,

[0117] 0.016822197,

[0118] -0.02143252,

[0119] 0.04470368,

[0120] -0.019333176,

[0121] -0.019195965,

[0122] 0.038748678,

[0123] -0.05118009,

[0124] 0.018784327,

[0125] 0.00467207,

[0126] 0.059056062,

[0127] 0.019374339,

[0128] 0.011807096,

[0129] 0.014174003,

[0130] -0.030762939,

[0131] -0.010386951,

[0132] -0.031833194,

[0133] 0.0114366235,

[0134] -0.009433327 ]

[0136] }

[0137] }

[0138] The step of obtaining the target content for answering user questions from the vector library includes:

[0139] Obtain the user's question and vectorize it; for example... Figure 3 As shown, user questions can be obtained by extracting user input.

[0140] The target content is obtained by performing multi-path retrieval based on the vectorized user question in the vector database. The vectorized user question is represented as follows:

[0141] [0.0049567847,0.004308458,-0.01197175,0.016822197,-0.02143252,0.04470368,-0.019333176,-0.019195965,0.038748678,-0.05118009,0.0187843 27, 0.00467207, 0.059056062, 0.019374339, 0.011807096, 0.014174003, -0.030762939, -0.010386951, -0.031833194, 0.0114366235, -0.009433327…...).

[0142] The user questions, after being vectorized, are used for multi-path recall within the vector database, as explained below:

[0143] The corresponding content was obtained through vector search and full-text search, and the target content is represented as follows:

[0144]  Similar to the hold-out method, there are multiple ways to divide a dataset D into k subsets. To reduce the differences introduced by different sample partitions, k-fold cross-validation usually involves randomly using different partitions and repeating the process p times. The final evaluation result is the mean of these p k-fold cross-validation results.

[0145] In step S4 above, the device replaces the image links in each of the target contents with the token, splices each target content after replacing the token, and calls the large model to process the spliced ​​content to obtain the model output result.

[0146] Regular expressions are used to extract image links from the recall results (target content). A corresponding token is generated for each image link for replacement. The relationship between the token and the image link is recorded through a token reference table. After replacing the image link with a shorter token, the probability of link address errors caused by large model integration is reduced.

[0147] After performing regular expression extraction, the token reference table is generated as follows:

[0148] {

[0149] "",

[0150] ""

[0151] }

[0152] The target content after replacing the token is shown below:

[0153] "![This image illustrates the process of 10-fold cross-validation. The dataset D is divided into 10 subsets (D1 to D10). (URL=0)" Similar to the hold-out method, there are multiple ways to divide the dataset D into k subsets. To reduce the differences introduced by different sample partitions, k-fold cross-validation usually involves randomly using different partitions and repeating the process p times. The final evaluation result is the mean of the results of these p k-fold cross-validation iterations.

[0154] The step of concatenating each target content after replacing the token includes:

[0155] Each target content after the token replacement is concatenated according to the preset concatenation order.

[0156] The concatenated content is represented as follows:

[0157] (Other related content)...

[0158] This image illustrates the process of 10-fold cross-validation. The dataset D is divided into 10 subsets (D1 to D10). (URL=0) Similar to the hold-out method, there are multiple ways to divide a dataset D into k subsets. To reduce the differences introduced by different sample partitions, k-fold cross-validation usually involves randomly using different partitions and repeating the process p times. The final evaluation result is the mean of these p k-fold cross-validation results.

[0159] (Other related content)...

[0160] The preset splicing order can be the order in which all target content appears in the uploaded document.

[0161] Before the step of calling the large model to process the concatenated content, the user question answering method based on the uploaded document further includes:

[0162] Generate a large model hint word, and append the large model hint word before the concatenated content;

[0163] The large model is invoked to process the concatenated content containing prompts from the large model.

[0164] The content after combining the keywords with large model prompts is shown below:

[0165] Based on the information above, answer the user's question, with the following requirements:

[0166] 1) If the above known information does not contain any content corresponding to the user's question, then reply directly with "Based on the knowledge I have learned in the knowledge base, I am still unable to answer your question";

[0167] (Other related content)...

[0168] This image illustrates the process of 10-fold cross-validation. The dataset D is divided into 10 subsets (D1 to D10). (URL=0) Similar to the hold-out method, there are multiple ways to divide a dataset D into k subsets. To reduce the differences introduced by different sample partitions, k-fold cross-validation usually involves randomly using different partitions and repeating the process p times. The final evaluation result is the mean of the results of these p k-fold cross-validation iterations.

[0169] (Other related content)...

[0170] After the step of calling the large model to process the concatenated content and obtain the model output, the user question answering method based on the uploaded document further includes:

[0171] Replace the token with the image link, parse the image link to obtain the image, and display the image at the target location in the document.

[0172] The streaming response from the large model is cached. If token-related characters appear, they are cached and not returned to the front-end interface until the token is concatenated. After the token is concatenated, it is replaced with the corresponding image link and returned to the front-end. This avoids outputting the characters in the intermediate link, and the front-end will directly output the image to improve the user experience during Q&A. The specific explanation is as follows:

[0173] The model output is as follows:

[0174] This image illustrates the process of 10-fold cross-validation. The dataset D is divided into 10 subsets (D1 to D10). (URL=0) Similar to the hold-out method, there are multiple ways to divide a dataset D into k subsets. To reduce the differences introduced by different sample partitions, k-fold cross-validation usually involves randomly using different partitions and repeating the process p times. The final evaluation result is the mean of these p k-fold cross-validation results.

[0175] Replace the token with the image link as follows:

[0176] "" Similar to the hold-out method, there are multiple ways to divide the dataset D into k subsets. To reduce the differences introduced by different sample partitions, k-fold cross-validation usually involves randomly using different partitions and repeating the process p times. The final evaluation result is the mean of the results of these p k-fold cross-validation iterations.

[0177] The front-end uses Markdown format to parse and display the image links of the corresponding images, and then displays the corresponding images to the user at the target location in the document.

[0178] The user question answering method based on uploaded documents provided in this embodiment of the invention has the following beneficial technical effects:

[0179] Supports image content retrieval: Supports searching by image content, improving search results.

[0180] Improved accuracy of large model summaries: Improves the quality of large model output data and reduces the probability of link address errors caused by large model integration.

[0181] Improved user experience: When returning results in a streaming manner, unreadable characters such as image links will not appear; images will be output directly, thus improving the user experience.

[0182] Multimodal semantic information fusion embedding: The image content extracted from the multimodal large model is concatenated into the content parsed in the document in a specific format to ensure that the retrieval of image content is provided without affecting the normal content display.

[0183] Token-based context optimization: Replace links in the context with tokens to optimize the context length and improve the accuracy of large models.

[0184] Link restoration based on token caching: The content corresponding to the token is replaced back into the output of the large model to prevent the large model from tampering with the link content during processing.

[0185] The user question answering method based on uploaded documents provided in this invention involves parsing the uploaded document to obtain image links, replacing the image links with tokens to obtain document parsing content; processing the document parsing content to obtain document slice vectors; adding tagged image content and image links to the target position of the document according to the document slice vectors to obtain updated document parsing content; writing the updated document parsing content and the document slice vectors into a vector library; and obtaining at least one target content for answering user questions from the vector library; each target content includes the image content, the image link, and document parsing content at the target position of the document; replacing the image links in each target content with the tokens; concatenating each target content after token replacement; and calling a large model to process the concatenated content to obtain the model output result, which can improve the accuracy of the answering results for user questions.

[0186] In the above optional embodiments, the document slice vector includes the image preceding slice vector and the image following slice vector; correspondingly, determining the document target position includes:

[0187] Calculate the first vector similarity between the image content vector corresponding to the image content and the preceding slice vector of the image, and calculate the second vector similarity between the image content vector and the subsequent slice vector of the image; the above embodiments can be referred to for explanation, and will not be repeated here.

[0188] The position of the slice of the vector corresponding to the larger value of the first vector similarity and the second vector similarity in the document is determined as the target position in the document. This can be referred to the above embodiment for explanation, and will not be repeated here.

[0189] In the above optional embodiments, obtaining the target content for answering the user's question from the vector library includes:

[0190] The user question is obtained and then vectorized; this can be described with reference to the above embodiments and will not be repeated here.

[0191] Based on the vectorized user question, a multi-path retrieval is performed in the vector database to obtain the target content. This can be referred to the above embodiment for further explanation, and will not be repeated here.

[0192] In the above optional embodiments, the step of concatenating each target content after replacing the token includes:

[0193] Each target content after the token replacement is concatenated according to a preset concatenation order. This can be referred to the above embodiment for explanation, and will not be repeated here.

[0194] In the above optional embodiments, before the step of calling the large model to process the spliced ​​content, the user question answering method based on the uploaded document further includes:

[0195] Generate a large model prompt word and append the large model prompt word before the concatenated content; refer to the above embodiment for explanation, and will not be repeated here.

[0196] The large model is invoked to process the concatenated content containing prompts from the large model. This can be explained in the above embodiment and will not be repeated here.

[0197] In the above optional embodiments, after the step of calling the large model to process the concatenated content and obtain the model output result, the user question answering method based on the uploaded document further includes:

[0198] The token is replaced with the image link, the image link is parsed to obtain the image, and the image is displayed at the target location in the document. This can be referred to the above embodiment for further explanation, and will not be repeated here.

[0199] Figure 4 This is a schematic diagram of a user question answering device based on uploaded documents provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the user question answering device based on uploaded documents provided in this embodiment of the invention includes a first acquisition unit 401, a second acquisition unit 402, a third acquisition unit 403, and a fourth acquisition unit 404, wherein:

[0200] The first acquisition unit 401 is used to parse the uploaded document, obtain image links, and replace the image links with tokens to obtain document parsing content; the second acquisition unit 402 is used to process the document parsing content to obtain document slice vectors, and add the identified image content and image links to the document target position according to the document slice vectors to obtain updated document parsing content; the third acquisition unit 403 is used to write the updated document parsing content and the document slice vectors into a vector library, and obtain at least one target content for answering user questions from the vector library; each target content includes the image content, the image link, and the document parsing content at the document target position; the fourth acquisition unit 404 is used to replace the image links in each target content with the tokens, concatenate each target content after replacing the tokens, and call a large model to process the concatenated content to obtain the model output result.

[0201] Specifically, the first acquisition unit 401 in the device is used to parse the uploaded document, obtain image links, and replace the image links with tokens to obtain document parsing content; the second acquisition unit 402 is used to process the document parsing content to obtain document slice vectors, and add the identified image content and image links to the document target position according to the document slice vectors to obtain updated document parsing content; the third acquisition unit 403 is used to write the updated document parsing content and the document slice vectors into a vector library, and obtain at least one target content for answering user questions from the vector library; each target content includes the image content, the image link, and the document parsing content at the document target position; the fourth acquisition unit 404 is used to replace the image links in each target content with the tokens, concatenate each target content after replacing the tokens, and call a large model to process the concatenated content to obtain the model output result.

[0202] The user question answering device based on uploaded documents provided in this embodiment of the invention parses the uploaded document to obtain image links, replaces the image links with tokens, and obtains document parsing content; processes the document parsing content to obtain document slice vectors, adds the identified image content and image links to the target position of the document according to the document slice vectors, and obtains updated document parsing content; writes the updated document parsing content and the document slice vectors into a vector library, and obtains at least one target content for answering user questions from the vector library; each target content includes the image content, the image link, and the document parsing content at the target position of the document; replaces the image links in each target content with the tokens, concatenates each target content after token replacement, and calls a large model to process the concatenated content to obtain the model output result, which can improve the accuracy of the answering results for user questions.

[0203] The embodiments of the present invention provide a user question answering device based on uploaded documents, which can be used to execute the processing flow of the above method embodiments. Its functions will not be repeated here, but can be referred to the detailed description of the above method embodiments.

[0204] Figure 5 This is a schematic diagram of the physical structure of a computer device provided in an embodiment of the present invention, such as... Figure 5 As shown, the computer device includes: a memory 501, a processor 502, and a computer program stored in the memory 501 and executable on the processor 502. When the processor 502 executes the computer program, it implements the following method:

[0205] The uploaded document is parsed to obtain image links, and the image links are replaced with tokens to obtain the parsed document content;

[0206] The document parsing content is processed to obtain a document slice vector. Based on the document slice vector, the image content with added identifier and the image link are added to the target position of the document to obtain the updated document parsing content.

[0207] The updated document parsing content and the document slice vector are written into a vector library, and at least one target content for answering user questions is obtained from the vector library; each target content includes the image content, the image link, and the document parsing content at the target location in the document;

[0208] Replace the image link in each of the target contents with the token, concatenate each target content after replacing the token, and call the large model to process the concatenated content to obtain the model output result.

[0209] This embodiment discloses a computer program product, which includes a computer program that, when executed by a processor, implements the following method:

[0210] The uploaded document is parsed to obtain image links, and the image links are replaced with tokens to obtain the parsed document content;

[0211] The document parsing content is processed to obtain a document slice vector. Based on the document slice vector, the image content with added identifier and the image link are added to the target position of the document to obtain the updated document parsing content.

[0212] The updated document parsing content and the document slice vector are written into a vector library, and at least one target content for answering user questions is obtained from the vector library; each target content includes the image content, the image link, and the document parsing content at the target location in the document;

[0213] Replace the image link in each of the target contents with the token, concatenate each target content after replacing the token, and call the large model to process the concatenated content to obtain the model output result.

[0214] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the following method:

[0215] The uploaded document is parsed to obtain image links, and the image links are replaced with tokens to obtain the parsed document content;

[0216] The document parsing content is processed to obtain a document slice vector. Based on the document slice vector, the image content with added identifier and the image link are added to the target position of the document to obtain the updated document parsing content.

[0217] The updated document parsing content and the document slice vector are written into a vector library, and at least one target content for answering user questions is obtained from the vector library; each target content includes the image content, the image link, and the document parsing content at the target location in the document;

[0218] Replace the image link in each of the target contents with the token, concatenate each target content after replacing the token, and call the large model to process the concatenated content to obtain the model output result.

[0219] Compared with existing technologies, the user question answering method based on uploaded documents provided in this invention involves parsing the uploaded document to obtain image links, replacing the image links with tokens to obtain document parsing content; processing the document parsing content to obtain document slice vectors; adding tagged image content and image links to the target position of the document according to the document slice vectors to obtain updated document parsing content; writing the updated document parsing content and the document slice vectors into a vector library; and retrieving at least one target content for answering user questions from the vector library; each target content includes the image content, the image link, and document parsing content at the target position of the document; replacing the image links in each target content with the tokens; concatenating each target content after token replacement; and calling a large model to process the concatenated content to obtain model output results, which can improve the accuracy of answering user questions.

[0220] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0221] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0222] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0223] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0224] In the description of this specification, the references to terms such as "an embodiment," "a specific embodiment," "some embodiments," "for example," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0225] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for processing user questions and answers based on uploaded documents, characterized in that, include: The uploaded document is parsed to obtain image links, and the image links are replaced with tokens to obtain the parsed document content; The document parsing content is processed to obtain a document slice vector. Based on the document slice vector, the image content with added identifier and the image link are added to the target position of the document to obtain the updated document parsing content. The updated document parsing content and the document slice vector are written into a vector library, and at least one target content for answering user questions is obtained from the vector library; each target content includes the image content, the image link, and the document parsing content at the target location in the document; Replace the image link in each of the target contents with the token, concatenate each target content after replacing the token, and call the large model to process the concatenated content to obtain the model output result.

2. The user question answering method based on uploaded documents according to claim 1, characterized in that, The document slice vector includes the image preceding slice vector and the image following slice vector; correspondingly, determining the document target position includes: Calculate the first vector similarity between the image content vector corresponding to the image content and the image preceding slice vector, and calculate the second vector similarity between the image content vector and the image subsequent slice vector; The position of the slice of the vector corresponding to the larger value of the first vector similarity and the second vector similarity in the document is determined as the target position of the document.

3. The user question answering method based on uploaded documents according to claim 1, characterized in that, The step of obtaining the target content for answering user questions from the vector library includes: Obtain the user's question and vectorize it. Based on the vectorized user question, a multi-path retrieval is performed in the vector database to obtain the target content.

4. The user question answering method based on uploaded documents according to claim 2, characterized in that, The step of concatenating each target content after replacing the token includes: Each target content after the token replacement is concatenated according to the preset concatenation order.

5. The user question answering method based on uploaded documents according to claim 1, characterized in that, Before the step of calling the large model to process the concatenated content, the user question answering method based on the uploaded document further includes: Generate a large model hint word, and append the large model hint word before the concatenated content; The large model is invoked to process the concatenated content containing prompts from the large model.

6. The user question answering method based on uploaded documents according to any one of claims 1 to 5, characterized in that, After the step of calling the large model to process the concatenated content and obtain the model output, the user question answering method based on the uploaded document further includes: Replace the token with the image link, parse the image link to obtain the image, and display the image at the target location in the document.

7. A user question answering device based on uploaded documents, characterized in that, include: The first acquisition unit is used to parse the uploaded document, obtain the image link, and replace the image link with a token to obtain the document parsing content; The second acquisition unit is used to process the document parsing content to obtain a document slice vector, and add the marked image content and image link to the target position of the document according to the document slice vector to obtain the updated document parsing content; The third acquisition unit is used to write the updated document parsing content and the document slice vector into a vector library, and to acquire at least one target content for answering user questions from the vector library; each target content includes the image content, the image link, and the document parsing content at the target position in the document; The fourth acquisition unit is used to replace the image link in each of the target contents with the token, concatenate each target content after replacing the token, and call the large model to process the concatenated content to obtain the model output result.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 6.

Images