Information generation method and device, equipment, storage medium and computer program product

Abstract

The invention relates to the field of natural language processing, and discloses an information generation method, device and equipment, a storage medium and a computer program product.The method comprises the steps that in response to input content of a large model, retrieval is carried out in a preset private knowledge base according to the input content, a retrieval result is obtained, and based on the input content, the retrieval result is used as a generation reference, and the large model is generated; response information is generated through the large model; according to the method and the device, the knowledge in the preset private knowledge base is retrieved to enable the large model to refer to answer, so that the large model has the capability of answering private questions, the defect that the large model is trained by public corpora and cannot answer private knowledge related questions in the prior art is overcome, and the answering accuracy of the large model is improved.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and more particularly to an information generation method, apparatus, device, storage medium, and computer program product. Background Technology

[0002] Currently, large models perform well in tasks such as text generation and text-to-image generation. However, because large models are trained on public corpora, they cannot answer questions related to private knowledge, which leads to a tendency for large models to give incorrect answers. Summary of the Invention

[0003] The main objective of this invention is to provide an information generation method, apparatus, device, storage medium, and computer program product, aiming to solve the technical problem that large models, trained on public corpora, cannot answer questions related to private knowledge, thus leading to the tendency for large models to give incorrect answers.

[0004] To achieve the above objectives, the present invention provides an information generation method, the information generation method comprising:

[0005] In response to the input content of the large model, a search is performed in a preset private knowledge base based on the input content to obtain the search results;

[0006] Based on the input content and using the search results as a generation reference, response information is generated through the large model.

[0007] Optionally, before responding to the input content of the large model and performing a search in a preset private knowledge base based on the input content to obtain the search results, the method further includes:

[0008] The document to be processed is parsed to obtain formatted data;

[0009] The formatted data is structured into a knowledge base to construct a pre-defined private knowledge base.

[0010] Optionally, the step of structuring the formatted data to construct a preset private knowledge base includes:

[0011] The formatted data is augmented to obtain augmented data;

[0012] An index is constructed on the enhanced data, and a preset private knowledge base is constructed based on the index. The index includes at least one of a search engine, a vector engine, a relational database engine, and a graph database engine.

[0013] Optionally, the enhanced data includes question-answer pairs, table summaries, image descriptions, and knowledge graph triples. The step of performing data augmentation on the formatted data to obtain the enhanced data includes:

[0014] The large model generates question-and-answer pairs corresponding to the text data in the formatted data.

[0015] The large model generates table summaries corresponding to the tabular data in the formatted data.

[0016] The image descriptions corresponding to the image data in the formatted data are generated using a multimodal large model.

[0017] The large model generates knowledge graph triples corresponding to the text data in the formatted data.

[0018] Optionally, parsing the document to be processed to obtain formatted data includes:

[0019] The document to be processed is subjected to multimodal recognition to obtain multimodal data, and the multimodal data is formatted to obtain formatted data. The multimodal recognition includes at least two of the following: table recognition, formula recognition, image recognition, and text recognition.

[0020] Optionally, the step of performing multimodal recognition on the document to be processed to obtain multimodal data, and formatting the multimodal data to obtain formatted data, includes:

[0021] Perform layout analysis on the document to be processed to obtain the layout analysis results;

[0022] Based on the layout analysis results, multimodal recognition is performed on the document to be processed to obtain multimodal data;

[0023] The multimodal data is used to restore the layout, and the restored layout is obtained.

[0024] Based on the restored version, the multimodal data is formatted to obtain formatted data.

[0025] Optionally, the step of performing layout analysis on the document to be processed to obtain layout analysis results includes:

[0026] Obtain a document image of the document to be processed, and perform document orientation detection on the document to be processed based on the document image;

[0027] The layout analysis of the document to be processed is performed based on the document graph and the document orientation detection results to obtain the layout analysis results.

[0028] Optionally, in response to the input content of the large model, the search is performed in a preset private knowledge base based on the input content to obtain search results, including:

[0029] In response to the input content of the large model, multiple search statements are generated based on the input content through the large model;

[0030] Based on the multiple search statements, multi-path knowledge retrieval is performed in a preset private knowledge base to obtain search results.

[0031] Optionally, the step of performing multi-path knowledge retrieval in a preset private knowledge base based on the multiple search statements to obtain search results includes:

[0032] Based on the multiple search statements, at least one of the search engines, vector engines, relational database engines, and graph database engines in the preset private knowledge base is invoked to perform the search;

[0033] Based on the search results, structured and unstructured data in the preset private knowledge base are retrieved through multiple channels.

[0034] The retrieved text and vector searches are then re-ranked to obtain the search results.

[0035] Optionally, in response to the input content of the large model, multiple search statements are generated through the large model based on the input content, including:

[0036] In response to the input content of the large model, multi-step reasoning is performed based on the input content to obtain multi-step reasoning data;

[0037] Based on the multi-step reasoning data, multiple search statements are generated through the large model.

[0038] Optionally, after generating response information using the large model based on the input content and the search results as a generation reference, the process further includes:

[0039] The chain of evidence for the large model's response is determined based on the search results and the response information.

[0040] The chain of evidence for the response is checked to determine whether it is sufficient, and the authenticity of the response information is judged based on the test results.

[0041] Optionally, after generating response information using the large model based on the input content and the search results as a generation reference, the process further includes:

[0042] Obtain the evaluation metrics corresponding to each information generation process;

[0043] The information generation process is evaluated based on the evaluation indicators to obtain evaluation results.

[0044] Furthermore, to achieve the above objectives, the present invention also proposes an information generation device, the information generation device comprising:

[0045] The retrieval module is used to respond to the input content of the large model, and to search in a preset private knowledge base based on the input content to obtain the retrieval results;

[0046] The generation module is used to generate response information based on the input content and the search results as a generation reference, using the large model.

[0047] Optionally, the information generation device further includes:

[0048] The database construction module is used to parse the document to be processed and obtain formatted data; and to perform knowledge structuring on the formatted data to build a preset private knowledge base.

[0049] Optionally, the database building module is further configured to perform data augmentation on the formatted data to obtain augmented data; construct an index of the augmented data; and construct a preset private knowledge base based on the index, wherein the index includes at least one of a search engine, a vector engine, a relational database engine, and a graph database engine.

[0050] Optionally, the enhanced data includes question-answer pairs, table summaries, image descriptions, and knowledge graph triples. The database construction module is further configured to generate question-answer pairs corresponding to the text data in the formatted data using the large model; generate table summaries corresponding to the table data in the formatted data using the large model; generate image descriptions corresponding to the image data in the formatted data using the multimodal large model; and generate knowledge graph triples corresponding to the text data in the formatted data using the large model.

[0051] Optionally, the database construction module is further configured to perform multimodal recognition on the document to be processed to obtain multimodal data, and to format the multimodal data to obtain formatted data. The multimodal recognition includes at least two of table recognition, formula recognition, image recognition, and text recognition.

[0052] Furthermore, to achieve the above objectives, the present invention also proposes an information generation device, which includes a memory, a processor, and an information generation program stored in the memory and executable on the processor, wherein the information generation program is configured to implement the information generation method described above.

[0053] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing an information generation program, which, when executed by a processor, implements the information generation method as described above.

[0054] In addition, to achieve the above objectives, the present invention also provides a computer program product, the computer program product including an information generation program, which, when executed by a processor, implements the information generation method as described above.

[0055] In this invention, a method is disclosed that responds to the input content of a large model, searches a preset private knowledge base based on the input content, obtains search results, and generates response information through the large model based on the input content and the search results as a generation reference. This invention enables the large model to answer private questions by retrieving knowledge from the preset private knowledge base, thereby overcoming the defect in the prior art where the large model is trained on public corpora and cannot answer questions related to private knowledge, and improving the accuracy of the large model's answers. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of the structure of the information generation device for the hardware operating environment involved in the embodiments of the present invention;

[0057] Figure 2 This is a flowchart illustrating the first embodiment of the information generation method of the present invention;

[0058] Figure 3 This is a flowchart illustrating the second embodiment of the information generation method of the present invention;

[0059] Figure 4 This is a flowchart illustrating the third embodiment of the information generation method of the present invention;

[0060] Figure 5 This is a flowchart illustrating the document parsing process of an embodiment of the information generation method of the present invention.

[0061] Figure 6 This is a flowchart illustrating the fourth embodiment of the information generation method of the present invention;

[0062] Figure 7 This is an architecture diagram of an embodiment of the information generation method of the present invention;

[0063] Figure 8 This is a structural block diagram of the first embodiment of the information generation device of the present invention.

[0064] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0065] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0066] Reference Figure 1 , Figure 1This is a schematic diagram of the information generation device structure of the hardware operating environment involved in the embodiments of the present invention.

[0067] like Figure 1 As shown, the information generation device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen, and optionally, it may also include a standard wired interface or a wireless interface. In this invention, the wired interface of the user interface 1003 may be a USB interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0068] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the information generation device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0069] like Figure 1 As shown, the memory 1005, which is identified as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an information generation program.

[0070] exist Figure 1 In the information generation device shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the user equipment; the information generation device calls the information generation program stored in the memory 1005 through the processor 1001 and executes the information generation method provided in the embodiment of the present invention.

[0071] Based on the above hardware structure, an embodiment of the information generation method of the present invention is proposed.

[0072] Reference Figure 2 , Figure 2 This is a flowchart illustrating a first embodiment of the information generation method of the present invention. The first embodiment of the information generation method of the present invention includes:

[0073] Step S10: In response to the input content of the large model, a search is performed in the preset private knowledge base based on the input content to obtain the search results.

[0074] It should be understood that the execution subject of this embodiment may be an information generation device with data processing, network communication and program running functions, such as a server, or other electronic devices that can achieve the same or similar functions. This embodiment does not limit this.

[0075] It should be noted that in this embodiment and other embodiments, the large model can be a large language model (LLM). A large language model can be a deep learning model trained on a large-scale dataset. A large language model can generate natural language text, deeply understand the meaning of the text, and handle various natural language tasks, such as text summarization, question answering, and translation.

[0076] The input content of the large model includes, but is not limited to, text content, image content, and audio content. This input content can be entered by the user or automatically generated by the server according to the user's actual needs. This embodiment does not impose any restrictions on this.

[0077] It is understood that the preset private knowledge base can be pre-built based on the documents to be processed uploaded by the user (for example, in the case of an online document library, the user can upload judgment case documents to build a preset private knowledge base related to the case), or it can be automatically generated by the server based on user information (for example, in the case of an online document library, if the user is detected to be a lawyer, a preset private knowledge base related to law will be automatically built). This embodiment does not limit this; the search results include, but are not limited to, the private knowledge obtained by the search.

[0078] In a specific implementation, the search is performed in a preset private knowledge base based on the input content. Obtaining the search results can be achieved by first determining the preset private knowledge base, and then searching within it based on the input content. Determining the preset private knowledge base can involve identifying the user's corresponding private knowledge base, or it can involve parsing the input content and using the associated private knowledge base as the preset private knowledge base based on the parsing results. This embodiment does not impose any limitations on this approach. Searching within the preset private knowledge base based on the input content can be performed using techniques such as semantic analysis, keyword matching, and similarity calculation. Of course, to obtain more reliable search results, multi-step reasoning and multi-path recall techniques can also be used to search within the preset private knowledge base based on the input content.

[0079] Step S20: Based on the input content and using the search results as a generation reference, generate response information through the large model.

[0080] It should be understood that after receiving the search results, the large model will conduct in-depth analysis and understanding of the input content and the search results. Based on this in-depth analysis and understanding, the large model will generate response information corresponding to the input content.

[0081] For ease of understanding, the following examples are provided, but they do not limit the invention. In one example, assume the server is a web document server, the user is a lawyer, the preset private knowledge base is a private legal knowledge base, and the scenario is that the lawyer uses a large model and the private legal knowledge base to assist in their work when handling cases. The information generation method includes the following steps:

[0082] 1. When lawyers are handling specific cases, they may encounter some legal issues that require in-depth research or analysis (e.g., the applicability of a specific legal provision or the basis for a judgment in a case). Lawyers can use these issues as input for a larger model.

[0083] 2. Lawyers or law firms can maintain at least one private legal knowledge base. This private legal knowledge base includes a large number of legal provisions, cases, legal documents, as well as the experience summaries and case analyses of the legal team. When the online document database server receives input from a lawyer, it can search the private legal knowledge base based on keywords in the input (such as the name of the legal provision, the case number, etc.) to find the most relevant legal provisions, cases, or documents. The search process involves technologies such as semantic analysis of the text content, keyword matching, and case similarity calculation. The online document database server can return the relevant legal provisions, cases, or documents found as search results to the large model. The search results can include the original text of the legal provisions, the judgment summary of the case, and the key points of the legal documents.

[0084] 3. After receiving the search results, the large-scale model will combine the lawyer's input with the retrieved relevant materials for in-depth analysis and understanding. The model can analyze the legal logic, the basis of case judgments, and the interpretation of legal documents within these materials. Based on this analysis and understanding, it will generate a preliminary legal suggestion or analytical approach. The large-scale model can then generate at least one complete response message for the lawyer based on the generated legal suggestion or analytical approach, as well as its own language generation capabilities. The response message may include legal suggestions, case analysis reports, draft legal documents, etc.

[0085] In the above examples, the response information can provide lawyers with an analysis of the applicability of specific legal provisions, the basis for judgments in similar cases, and interpretations of relevant legal documents, helping lawyers to better understand the legal background of the case and formulate more effective legal strategies.

[0086] The Retrieval Augmented Generation (RAG) technology in this embodiment provides essential external knowledge base capabilities for large-scale model applications, supporting both C-end products (clients) and B-end applications (enterprises). For example, in C-end products, it provides underlying knowledge base support for document question-and-answer functions in online document libraries, cloud storage, browsers, smart office applications, and smart search products, as well as for intelligent digital human products.

[0087] In this embodiment, a response to the input content of the large model is disclosed. Based on the input content, a search is performed in a preset private knowledge base to obtain search results. Based on the input content, the search results are used as a generation reference to generate response information through the large model. This embodiment enables the large model to answer questions by retrieving knowledge in the preset private knowledge base, thereby overcoming the defect in the prior art where the large model is trained on public corpus and cannot answer questions related to private knowledge, and improving the accuracy of the large model's answers.

[0088] Reference Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the information generation method of the present invention, based on the above. Figure 2 The first embodiment shown presents a second embodiment of the information generation method of the present invention.

[0089] In the second embodiment, before step S10, the method further includes:

[0090] Step S01: Parse the document to be processed to obtain formatted data.

[0091] It should be noted that the documents to be processed include, but are not limited to, data from intelligent document parsing, operational data generated by the operation system, and log data recorded by the log system; this embodiment does not impose any limitations on these. Of course, in this embodiment, updated knowledge can be injected into the large model through a knowledge base to achieve factual updates.

[0092] It is understood that parsing the document to be processed to obtain formatted data can be done by first recognizing the document to be processed and then formatting the data obtained from the recognition. Recognition of the document to be processed includes, but is not limited to, table recognition, formula recognition, image recognition, and text recognition. Formatting includes, but is not limited to, standardized naming and unified data format. This embodiment does not limit these aspects.

[0093] Step S02: Perform knowledge structuring on the formatted data to construct a preset private knowledge base.

[0094] It should be understood that knowledge structuring of formatted data includes, but is not limited to, extracting key information, constructing knowledge structures, data augmentation, and building indexes, and this embodiment does not impose any limitations on these aspects.

[0095] For ease of understanding, the following examples are provided, but are not intended to limit the invention. In one example, assuming the server is a web document server, the user is a lawyer, and the preset private knowledge base is a private legal knowledge base, the steps for pre-constructing the private legal knowledge base include:

[0096] 1. The documents to be processed obtained by the online document repository server include:

[0097] Lawyers upload pending documents related to their cases or legal fields, including contracts, judgments, legal provisions, case analyses, expert opinions, etc.

[0098] 2. The online document server parses the document to be processed to obtain formatted data, including:

[0099] The online document server first performs text recognition on the document to be processed, then establishes a unified naming standard for key information in the document (such as case number, legal clause number, party name, etc.), and converts the parsed data into a unified data format, such as JSON, XML, etc., to facilitate subsequent processing and storage.

[0100] 3. The online document repository server performs knowledge structuring on formatted data, including:

[0101] (1) Extract key information:

[0102] Legal entity identification: Identify legal entities in a document, such as parties, courts, and legal clauses, and assign them unique identifiers;

[0103] Relationship extraction: Analyze sentences and paragraphs in a document to extract relationships between legal entities, such as the relationship between the plaintiff and the defendant, or the relationship between legal clauses and case judgments.

[0104] (2) Constructing a knowledge structure:

[0105] Establish a classification system: Based on the characteristics and needs of the legal field, establish a reasonable classification system, such as classifying by legal field, case type, legal procedure, etc.

[0106] Constructing a knowledge graph: Using technologies such as graph databases or relational databases, the extracted legal entities and relationships are represented and stored in the form of a graph to form a knowledge graph. In the knowledge graph, legal entities are nodes, and the relationships between them are edges.

[0107] 4. The online document server constructs a private legal knowledge base based on the knowledge structuring results.

[0108] In this embodiment, by parsing the document, formatting the data, and structuring the knowledge to pre-build a preset private knowledge base, a reliable private knowledge base can be pre-built, thereby improving the accuracy of the model's answers during subsequent information generation.

[0109] Reference Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the information generation method of the present invention. Based on the second embodiment described above, a third embodiment of the information generation method of the present invention is proposed.

[0110] In the third embodiment, step S01 includes:

[0111] Step S011: Perform multimodal recognition on the document to be processed to obtain multimodal data, and format the multimodal data to obtain formatted data. The multimodal recognition includes at least two of the following: table recognition, formula recognition, image recognition, and text recognition.

[0112] It should be understood that, in order to achieve multimodal parsing, in this embodiment, the document to be processed is subjected to multimodal recognition, and the multimodal data is formatted to obtain formatted data. The multimodal recognition includes at least two of table recognition, formula recognition, image recognition, and text recognition. The formatting process includes, but is not limited to, standardized naming and data format unification. This embodiment does not impose any restrictions on these aspects.

[0113] Furthermore, in order to reorganize the identified and processed document content into a layout structure similar to the original document, step S011 includes: performing layout analysis on the document to be processed to obtain layout analysis results; performing multimodal recognition on the document to be processed based on the layout analysis results to obtain multimodal data; restoring the layout of the multimodal data to obtain the restored layout; and formatting the multimodal data based on the restored layout to obtain formatted data.

[0114] It is understood that layout analysis includes, but is not limited to, element identification and position determination. Element identification can refer to identifying various elements in a document, such as identifying tables using table recognition algorithms, formulas using formula recognition algorithms, images using image recognition algorithms, and text using Optical Character Recognition (OCR) technology. Position determination can refer to determining the relative positions of these elements within the original document, such as determining the relative positions of these elements within the original document by analyzing the document's layout structure and the relative relationships between elements. This embodiment does not impose any limitations on this.

[0115] Page layout restoration can reorganize document content based on the identified elements and their position information using typesetting and layout algorithms to form a page structure similar to the original document.

[0116] Furthermore, to improve the accuracy of layout analysis, the step of performing layout analysis on the document to be processed and obtaining layout analysis results includes: acquiring a document image of the document to be processed, and performing document orientation detection on the document to be processed based on the document image; performing layout analysis on the document to be processed based on the document graph and the document orientation detection results to obtain layout analysis results.

[0117] For ease of understanding, please refer to Figure 5 This description is provided but does not limit the scope of the invention. Figure 5 This is a flowchart illustrating a document parsing process according to an embodiment of the information generation method of the present invention. The parsing of the document to be processed includes: acquiring a document image of the document to be processed, and performing document orientation detection on the document based on the document image; performing layout analysis on the document based on the document graph and the document orientation detection results to obtain layout analysis results; performing table recognition, formula recognition, image recognition, and text recognition on the document based on the layout analysis results to obtain multimodal data; restoring the layout of the multimodal data to obtain the restored layout; and formatting the multimodal data based on the restored layout to obtain formatted data.

[0118] In the third embodiment, step S02 includes:

[0119] Step S021: Perform data augmentation on the formatted data to obtain augmented data.

[0120] It should be understood that, in order to facilitate subsequent extended recall, this embodiment also performs data augmentation on the formatted data and constructs an index for the augmented data.

[0121] Furthermore, to improve the data augmentation effect, the augmented data includes question-answer pairs, table summaries, image descriptions, and knowledge graph triples. Step S021 includes: generating question-answer pairs corresponding to the text data in the formatted data using the large model; generating table summaries corresponding to the table data in the formatted data using the large model; generating image descriptions corresponding to the image data in the formatted data using the multimodal large model; and generating knowledge graph triples corresponding to the text data in the formatted data using the large model.

[0122] For ease of understanding, the following examples are provided, but are not intended to limit the invention. In one example, based on the current text, LLM is used to generate question-answer pairs for expanding recall; based on the current table, LLM is used to generate a table summary for expanding table recall; based on the current image, a multimodal large model is used to generate an image caption and description for image recall; based on the current text, LLM is used to generate knowledge graph triples for constructing a graph database.

[0123] Step S022: Construct an index for the enhanced data and construct a preset private knowledge base based on the index. The index includes at least one of a search engine, a vector engine, a relational database engine, and a graph database engine.

[0124] It should be understood that building an index for enhanced data can involve indexing both structured and unstructured data. For example, parsed document data can be segmented into blocks, and then a vector engine can be built using a vectorization model.

[0125] This embodiment performs data augmentation on formatted data and builds an index for the augmented data, which facilitates subsequent extended recall and improves the accuracy of the model's responses during the subsequent information generation process.

[0126] Reference Figure 6 , Figure 6 This is a flowchart illustrating the fourth embodiment of the information generation method of the present invention. Based on the above embodiments, a fourth embodiment of the information generation method of the present invention is proposed.

[0127] In the fourth embodiment, step S10 includes:

[0128] Step S101: In response to the input content of the large model, generate multiple search statements through the large model based on the input content.

[0129] It should be understood that, in order to improve retrieval efficiency and accuracy, in this embodiment, multiple retrieval statements are first generated based on the input content using a large model, and then multiple knowledge retrievals are performed in a preset private knowledge base based on the multiple retrieval statements to obtain retrieval results.

[0130] In its implementation, after receiving the input content, the large model generates multiple search queries related to the input content based on its training data and algorithms. For example, if the input content is "contract dispute," the following search queries can be generated: "Legal provisions related to contract disputes," "Case analysis of contract termination conditions," "Expert interpretation of breach of contract liability," and "Past judgments on contract disputes," etc.

[0131] Furthermore, in order to improve the accuracy of the search statements, step S101 includes: responding to the input content of the large model, performing multi-step reasoning based on the input content to obtain multi-step reasoning data; and generating multiple search statements through the large model based on the multi-step reasoning data.

[0132] Understandably, in order to improve the accuracy of search statements, this embodiment obtains multiple search statements by performing multi-step reasoning on the input content.

[0133] In practical implementation, for example, multi-step reasoning is performed through frameworks such as ReAct and workflow to obtain multi-step reasoning data, and multiple search statements are generated through a large model based on the multi-step reasoning data.

[0134] Step S102: Based on the multiple search statements, perform multi-path knowledge retrieval in a preset private knowledge base to obtain search results.

[0135] It should be understood that multi-path knowledge retrieval based on multiple search statements in a preset private knowledge base to obtain search results can be achieved by calling at least one of the search engines, vector engines, relational database engines, and graph database engines in the preset private knowledge base based on multiple search statements; multi-path retrieval of structured and unstructured data in the preset private knowledge base based on search results; and re-ranking the retrieved text and vector searches to obtain search results.

[0136] In specific implementations, for example, based on the planning capabilities of large models, retrieval queries are generated according to the context. Based on text search, vector search, graph search, and SQL search, multi-path recall is performed on structured and unstructured data. After recall, text retrieval and vector retrieval are reranked.

[0137] In this embodiment, multiple search statements are first generated based on the input content using a large model. Then, based on these multiple search statements, multi-path knowledge retrieval is performed in a preset private knowledge base to obtain search results, thereby improving search efficiency and accuracy.

[0138] In the fourth embodiment, after step S20, the method further includes:

[0139] Step S30: Determine the answer evidence chain of the large model based on the search results and the response information.

[0140] It should be understood that, in order to facilitate the determination of the authenticity of the large model's response, the large model also provides reference search results when responding. Based on the search results and response information, it is helpful to determine the evidence chain of the large model's response.

[0141] Understandably, determining the answer evidence chain of a large model based on search results and response information can involve establishing correlations between search results and response information, obtaining correlation analysis results, and then determining the answer evidence chain of the large model based on the correlation analysis results.

[0142] Step S40: Detect whether the chain of evidence for the response is sufficient, and determine the authenticity of the response information based on the detection result.

[0143] It should be understood that whether the evidence chain of the answer is sufficient can be detected by using a preset truth-detection model. The preset truth-detection model can be pre-set, for example, it can be a pre-set neural network model.

[0144] In practice, if the chain of evidence is sufficient, the response is considered true; if the chain of evidence is insufficient, the response is considered false.

[0145] Furthermore, in order to evaluate the performance of each aspect of the information generation process, after step S20, the method further includes: obtaining the evaluation indicators corresponding to each information generation process; evaluating each information generation process according to the evaluation indicators, and obtaining the evaluation results.

[0146] For ease of understanding, please refer to Figure 7 This description is provided but does not limit the scope of the invention. Figure 7This is an architecture diagram of an embodiment of the information generation method of the present invention. In the diagram, structured data such as charts, tables, formulas, and text are extracted through data parsing. Then, knowledge structuring is performed to establish knowledge storage formats such as relational databases, graph databases, vector engines, and search engines. After user input, the large model's planning capabilities are utilized to call various engines for knowledge retrieval in a timely manner. The retrieval and generation results are reflected upon, and multi-step reasoning is employed to ultimately provide a reliable answer. Specifically, this includes the following (in no particular order):

[0147] 1. Intelligent document parsing and formatted output:

[0148] By analyzing the layout of PDF documents, tables, formulas, images, and text are recognized, and finally, formatted output is generated.

[0149] 2. Knowledge structuring:

[0150] Data augmentation: Based on the current text, use LLM to generate question-answer pairs to expand recall; based on the current table, use LLM to generate a table summary to expand table recall; based on the current image, use a multimodal large model to generate image captions and descriptions to enhance image recall; based on the current text, use LLM to generate knowledge graph triples to build a graph database.

[0151] Build indexes for both structured and unstructured data. For example, after parsing document data, split it into blocks and then build a vector engine using a vectorization model.

[0152] 3. Knowledge Recall:

[0153] Leveraging the planning capabilities of a large model, a retrieval query is generated based on the context. Multi-path recall is performed on structured and unstructured data using text search, vector search, graph search, and SQL search. After recall, text and vector retrieval data are re-ranked for fine-tuning.

[0154] 4. Multi-step reasoning:

[0155] Use frameworks such as ReAct and workflow to perform multi-step retrieval and recall knowledge.

[0156] 5. Factual hallucinations / reflections:

[0157] Determine whether the results generated by the large model respect the facts cited.

[0158] 6. Evaluation System:

[0159] Human-based SBS end-to-end evaluation.

[0160] Based on benchmark-based automated evaluation, the evaluation is conducted in modules. For example, the evaluation of large model capabilities corresponding to the large model in the figure, the information extraction accuracy and recall corresponding to knowledge structuring, the context relevance (accuracy and recall) corresponding to knowledge recall, the context relevance (accuracy and recall) corresponding to multi-step reasoning, and the answer fidelity and answer relevance corresponding to response information.

[0161] 7. Operating System:

[0162] Knowledge management and operation system;

[0163] Multi-function inference plugin management system.

[0164] 8. Large Model:

[0165] Data from intelligent document parsing, operational data generated by the operation system, and log data recorded by the log system can all be used as training data for the model; domain pre-training; domain fine-tuning.

[0166] As shown in the figure, in this embodiment, knowledge retrieval can also be managed via API plugins.

[0167] In addition, refer to Figure 8 The present invention also proposes an information generation device, the information generation device comprising:

[0168] The retrieval module 10 is used to respond to the input content of the large model, and to perform a retrieval in a preset private knowledge base based on the input content to obtain retrieval results.

[0169] It should be noted that in this embodiment and other embodiments, the large model can be a large language model (LLM). A large language model can be a deep learning model trained on a large-scale dataset. A large language model can generate natural language text, deeply understand the meaning of the text, and handle various natural language tasks, such as text summarization, question answering, and translation.

[0170] The input content of the large model includes, but is not limited to, text content, image content, and audio content. This input content can be entered by the user or automatically generated by the server according to the user's actual needs. This embodiment does not impose any restrictions on this.

[0171] It is understood that the preset private knowledge base can be pre-built based on the documents to be processed uploaded by the user (for example, in the case of an online document library, the user can upload judgment case documents to build a preset private knowledge base related to the case), or it can be automatically generated by the server based on user information (for example, in the case of an online document library, if the user is detected to be a lawyer, a preset private knowledge base related to law will be automatically built). This embodiment does not limit this; the search results include, but are not limited to, the private knowledge obtained by the search.

[0172] In a specific implementation, the search is performed in a preset private knowledge base based on the input content. Obtaining the search results can be achieved by first determining the preset private knowledge base, and then searching within it based on the input content. Determining the preset private knowledge base can involve identifying the user's corresponding private knowledge base, or it can involve parsing the input content and using the associated private knowledge base as the preset private knowledge base based on the parsing results. This embodiment does not impose any limitations on this approach. Searching within the preset private knowledge base based on the input content can be performed using techniques such as semantic analysis, keyword matching, and similarity calculation. Of course, to obtain more reliable search results, multi-step reasoning and multi-path recall techniques can also be used to search within the preset private knowledge base based on the input content.

[0173] The generation module 20 is used to generate response information based on the input content and the search results as a generation reference, through the large model.

[0174] It should be understood that after receiving the search results, the large model will conduct in-depth analysis and understanding of the input content and the search results. Based on this in-depth analysis and understanding, the large model will generate response information corresponding to the input content.

[0175] For ease of understanding, the following examples are provided, but they do not limit the invention. In one example, assume the server is a web document server, the user is a lawyer, the preset private knowledge base is a private legal knowledge base, and the scenario is that the lawyer uses a large model and the private legal knowledge base to assist in their work when handling cases. The information generation method includes the following steps:

[0176] 1. When lawyers are handling specific cases, they may encounter some legal issues that require in-depth research or analysis (e.g., the applicability of a specific legal provision or the basis for a judgment in a case). Lawyers can use these issues as input for a larger model.

[0177] 2. Lawyers or law firms can maintain at least one private legal knowledge base. This private legal knowledge base includes a large number of legal provisions, cases, legal documents, as well as the experience summaries and case analyses of the legal team. When the online document database server receives input from a lawyer, it can search the private legal knowledge base based on keywords in the input (such as the name of the legal provision, the case number, etc.) to find the most relevant legal provisions, cases, or documents. The search process involves technologies such as semantic analysis of the text content, keyword matching, and case similarity calculation. The online document database server can return the relevant legal provisions, cases, or documents found as search results to the large model. The search results can include the original text of the legal provisions, the judgment summary of the case, and the key points of the legal documents.

[0178] 3. After receiving the search results, the large-scale model will combine the lawyer's input with the retrieved relevant materials for in-depth analysis and understanding. The model can analyze the legal logic, the basis of case judgments, and the interpretation of legal documents within these materials. Based on this analysis and understanding, it will generate a preliminary legal suggestion or analytical approach. The large-scale model can then generate at least one complete response message for the lawyer based on the generated legal suggestion or analytical approach, as well as its own language generation capabilities. The response message may include legal suggestions, case analysis reports, draft legal documents, etc.

[0179] In the above examples, the response information can provide lawyers with an analysis of the applicability of specific legal provisions, the basis for judgments in similar cases, and interpretations of relevant legal documents, helping lawyers to better understand the legal background of the case and formulate more effective legal strategies.

[0180] The Retrieval Augmented Generation (RAG) technology in this embodiment provides essential external knowledge base capabilities for large-scale model applications, supporting both C-end products (clients) and B-end applications (enterprises). For example, in C-end products, it provides underlying knowledge base support for document question-and-answer functions in online document libraries, cloud storage, browsers, smart office applications, and smart search products, as well as for intelligent digital human products.

[0181] In this embodiment, a response to the input content of the large model is disclosed. Based on the input content, a search is performed in a preset private knowledge base to obtain search results. Based on the input content, the search results are used as a generation reference to generate response information through the large model. This embodiment enables the large model to answer questions by retrieving knowledge in the preset private knowledge base, thereby overcoming the defect in the prior art where the large model is trained on public corpus and cannot answer questions related to private knowledge, and improving the accuracy of the large model's answers.

[0182] Other embodiments or specific implementations of the information generation device described in this invention can be found in the above-described method embodiments, and will not be repeated here.

[0183] Furthermore, this embodiment of the invention also proposes a storage medium storing an information generation program, which, when executed by a processor, implements the information generation method described above.

[0184] Furthermore, this invention also proposes a computer program product, including an information generation program, which, when executed by a processor, implements the information generation method described above.

[0185] The specific implementation of the computer program product of the present invention is basically the same as the embodiments of the information generation method described above, and will not be repeated here.

[0186] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0187] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0188] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0189] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

[0190] This invention discloses A1, an information generation method, the information generation method comprising:

[0191] In response to the input content of the large model, a search is performed in a preset private knowledge base based on the input content to obtain the search results;

[0192] Based on the input content and using the search results as a generation reference, response information is generated through the large model.

[0193] A2. The information generation method as described in A1, before responding to the input content of the large model and performing a retrieval in a preset private knowledge base based on the input content to obtain the retrieval results, further includes:

[0194] The document to be processed is parsed to obtain formatted data;

[0195] The formatted data is structured into a knowledge base to construct a pre-defined private knowledge base.

[0196] A3. The information generation method as described in A2, wherein the step of structuring the formatted data into a knowledge base and constructing a preset private knowledge base includes:

[0197] The formatted data is augmented to obtain augmented data;

[0198] An index is constructed on the enhanced data, and a preset private knowledge base is constructed based on the index. The index includes at least one of a search engine, a vector engine, a relational database engine, and a graph database engine.

[0199] A4. The information generation method as described in A3, wherein the enhanced data includes question-answer pairs, table summaries, image descriptions, and knowledge graph triples, and the step of performing data augmentation on the formatted data to obtain the enhanced data includes:

[0200] The large model generates question-and-answer pairs corresponding to the text data in the formatted data.

[0201] The large model generates table summaries corresponding to the tabular data in the formatted data.

[0202] The image descriptions corresponding to the image data in the formatted data are generated using a multimodal large model.

[0203] The large model generates knowledge graph triples corresponding to the text data in the formatted data.

[0204] A5. The information generation method as described in A2, wherein parsing the document to be processed to obtain formatted data includes:

[0205] The document to be processed is subjected to multimodal recognition to obtain multimodal data, and the multimodal data is formatted to obtain formatted data. The multimodal recognition includes at least two of the following: table recognition, formula recognition, image recognition, and text recognition.

[0206] A6. The information generation method as described in A5, wherein performing multimodal recognition on the document to be processed to obtain multimodal data, and formatting the multimodal data to obtain formatted data, includes:

[0207] Perform layout analysis on the document to be processed to obtain the layout analysis results;

[0208] Based on the layout analysis results, multimodal recognition is performed on the document to be processed to obtain multimodal data;

[0209] The multimodal data is used to restore the layout, and the restored layout is obtained.

[0210] Based on the restored version, the multimodal data is formatted to obtain formatted data.

[0211] A7. The information generation method as described in A6, wherein the step of performing layout analysis on the document to be processed to obtain layout analysis results includes:

[0212] Obtain a document image of the document to be processed, and perform document orientation detection on the document to be processed based on the document image;

[0213] The layout analysis of the document to be processed is performed based on the document graph and the document orientation detection results to obtain the layout analysis results.

[0214] A8. The information generation method as described in any one of A1 to A7, wherein responding to the input content of the large model, performing a search in a preset private knowledge base based on the input content to obtain search results includes:

[0215] In response to the input content of the large model, multiple search statements are generated based on the input content through the large model;

[0216] Based on the multiple search statements, multi-path knowledge retrieval is performed in a preset private knowledge base to obtain search results.

[0217] A9. The information generation method as described in A8, wherein the step of performing multi-path knowledge retrieval in a preset private knowledge base based on the multiple retrieval statements to obtain retrieval results includes:

[0218] Based on the multiple search statements, at least one of the search engines, vector engines, relational database engines, and graph database engines in the preset private knowledge base is invoked to perform the search;

[0219] Based on the search results, structured and unstructured data in the preset private knowledge base are retrieved through multiple channels.

[0220] The retrieved text and vector searches are then re-ranked to obtain the search results.

[0221] A10. The information generation method as described in A8, wherein in response to the input content of the large model, multiple search statements are generated through the large model based on the input content, including:

[0222] In response to the input content of the large model, multi-step reasoning is performed based on the input content to obtain multi-step reasoning data;

[0223] Based on the multi-step reasoning data, multiple search statements are generated through the large model.

[0224] A11. The information generation method as described in any one of A1 to A7, after generating response information based on the input content and using the retrieval results as a generation reference through the large model, further includes:

[0225] The chain of evidence for the large model's response is determined based on the search results and the response information.

[0226] The chain of evidence for the response is checked to determine whether it is sufficient, and the authenticity of the response information is judged based on the test results.

[0227] A12. The information generation method as described in any one of A1 to A7, further comprising, after generating response information based on the input content and using the retrieval results as a generation reference through the large model:

[0228] Obtain the evaluation metrics corresponding to each information generation process;

[0229] The information generation process is evaluated based on the evaluation indicators to obtain evaluation results.

[0230] The present invention also discloses B13, an information generation device, the information generation device comprising:

[0231] The retrieval module is used to respond to the input content of the large model, and to search in a preset private knowledge base based on the input content to obtain the retrieval results;

[0232] The generation module is used to generate response information based on the input content and the search results as a generation reference, using the large model.

[0233] B14. The information generation apparatus as described in B13, further comprising:

[0234] The database construction module is used to parse the document to be processed and obtain formatted data; and to perform knowledge structuring on the formatted data to build a preset private knowledge base.

[0235] B15. The information generation device as described in B14, wherein the database building module is further configured to perform data enhancement on the formatted data to obtain enhanced data; construct an index of the enhanced data; and construct a preset private knowledge base based on the index, wherein the index includes at least one of a search engine, a vector engine, a relational database engine, and a graph database engine.

[0236] B16. The information generation device as described in B15, wherein the enhanced data includes question-answer pairs, table summaries, image descriptions, and knowledge graph triples, and the database construction module is further configured to generate question-answer pairs corresponding to the text data in the formatted data through the large model; generate table summaries corresponding to the table data in the formatted data through the large model; generate image descriptions corresponding to the image data in the formatted data through the multimodal large model; and generate knowledge graph triples corresponding to the text data in the formatted data through the large model.

[0237] B17. The information generation device as described in B14, wherein the database construction module is further configured to perform multimodal recognition on the document to be processed to obtain multimodal data, and to perform formatting processing on the multimodal data to obtain formatted data, wherein the multimodal recognition includes at least two of table recognition, formula recognition, image recognition and text recognition.

[0238] The present invention also discloses C18, an information generation device, the information generation device comprising: a memory, a processor, and an information generation program stored in the memory and executable on the processor, wherein the information generation program, when executed by the processor, implements the information generation method described above.

[0239] The present invention also discloses D19, a storage medium storing an information generation program, which, when executed by a processor, implements the information generation method described above.

[0240] The present invention also discloses E20, a computer program product, the computer program product including an information generation program, which, when executed by a processor, implements the information generation method as described above.

Claims

1. An information generation method characterized by comprising: The information generation method comprises: in response to input content of a large model, searching in a preset private knowledge base according to the input content to obtain a search result; based on the input content, generating response information through the large model by taking the search result as a generation reference.

2. The information generation method according to claim 1, wherein Before the step of in response to input content of a large model, searching in a preset private knowledge base according to the input content to obtain a search result, the method further comprises: parsing a to-be-processed document to obtain formatted data; performing knowledge structuring on the formatted data to construct a preset private knowledge base.

3. The information generation method according to claim 2, characterized by, The step of performing knowledge structuring on the formatted data to construct a preset private knowledge base comprises: performing data enhancement on the formatted data to obtain enhanced data; constructing an index of the enhanced data, and constructing a preset private knowledge base according to the index, wherein the index comprises at least one of a search engine, a vector engine, a relational database engine, and a graph database engine.

4. The information generation method according to Claim 3, characterized by, The enhanced data comprises question and answer pairs, table summaries, picture descriptions, and knowledge graph triples, and the step of performing data enhancement on the formatted data to obtain enhanced data comprises: generating question and answer pairs corresponding to text data in the formatted data through the large model; generating table summaries corresponding to table data in the formatted data through the large model; generating picture descriptions corresponding to picture data in the formatted data through a multi-modal large model; generating knowledge graph triples corresponding to text data in the formatted data through the large model.

5. The information generation method according to Claim 2, characterized by, The step of parsing a to-be-processed document to obtain formatted data comprises: performing multi-modal recognition on the to-be-processed document to obtain multi-modal data, and performing formatting processing on the multi-modal data to obtain formatted data, wherein the multi-modal recognition comprises at least two of table recognition, formula recognition, picture recognition, and text recognition.

6. The information generation method according to Claim 5, characterized by, The step of performing multi-modal recognition on the to-be-processed document to obtain multi-modal data, and performing formatting processing on the multi-modal data to obtain formatted data comprises: performing layout analysis on the to-be-processed document to obtain a layout analysis result; performing multi-modal recognition on the to-be-processed document based on the layout analysis result to obtain multi-modal data; performing layout restoration on the multi-modal data to obtain restored layout; performing formatting processing on the multi-modal data based on the restored layout to obtain formatted data.

7. An information generation apparatus characterized by comprising: The information generation device comprises: a search module configured to search in a preset private knowledge base according to input content of a large model in response to the input content to obtain a search result; a generation module configured to generate response information through the large model by taking the search result as a generation reference based on the input content.

8. An information generation apparatus characterized by comprising: The information generation device comprises a memory, a processor, and an information generation program stored on the memory and executable on the processor, wherein the information generation program is executed by the processor to implement the information generation method according to any one of claims 1 to 6.

9. A storage medium, characterized by The storage medium stores an information generation program that, when executed by a processor, implements the information generation method according to any one of claims 1 to 6.

10. A computer program product, characterised in that, The computer program product includes an information generation program that, when executed by a processor, implements the information generation method according to any one of claims 1 to 6.

Images