Data processing, question answering method and device, electronic equipment and storage medium

Abstract

The application provides a data processing method, a question and answer method, a device, an electronic device and a storage medium. The data processing method comprises: generating a question and answer pair based on a target document; the question and answer pair comprises a question that can be answered based on the target document and an answer corresponding to the question; updating a knowledge base based on the question and answer pair; the knowledge base is used to provide a reference question and answer pair for a target question to be answered; the target question and the reference question and answer pair are used to generate a target answer of the target question. The application can realize automatic knowledge base updating.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing, question-and-answer method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of Artificial Intelligence (AI) technology, Large Language Models (LLMs) based on AI are increasingly being applied to areas such as user question answering. For example, for complex and cumbersome cloud service platforms, an intelligent assistant can be built based on a LLM to assist users in using the platform. However, due to the illusion problem, LLMs can lead to poor accuracy in outputting answers.

[0003] Retrieval-Augmented Generation (RAG) is a technique that improves the illusion problem in large language models. The core idea of ​​RAG is to first retrieve documents or knowledge fragments related to the question based on a limited knowledge base, and then obtain the final answer based on the retrieved documents or knowledge fragments.

[0004] However, how to construct the aforementioned knowledge base is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a data processing, question-answering method, apparatus, electronic device, and storage medium that can realize automated knowledge base updates.

[0006] In a first aspect, this application provides a data processing method, the method comprising:

[0007] Based on the target document, a question-answer pair is generated; the question-answer pair includes: a question that can be answered based on the target document, and the answer to the question;

[0008] Based on the question-answer pair, the knowledge base is updated; the knowledge base is used to provide reference question-answer pairs for the target question to be answered; the target question and the reference question-answer pair are used to generate the target answer to the target question.

[0009] Optionally, generating question-answer pairs based on the target document includes:

[0010] The content of the target document at the target granularity is determined; the target granularity includes at least one of the following: document granularity, paragraph granularity, and block granularity; the block granularity is smaller than the paragraph granularity.

[0011] The question-answer pair is generated based on the content at the target granularity.

[0012] Optionally, the target granularity includes: the chunk granularity, and the step of determining the content of the target document's target granularity based on the target document includes:

[0013] The target document is segmented to obtain the paragraph content at the paragraph level of the target document;

[0014] Based on the data type of the paragraph content, the block content at the block granularity is determined; the data type is text data or table data; the block content belongs to the same data type.

[0015] Optionally, determining the chunk content at the chunk granularity based on the data type of the paragraph content includes:

[0016] Based on the data type of the paragraph content, the paragraph content is divided into at least one initial block; the content of the initial block belongs to the same data type.

[0017] Based on the number of lexical units in the initial segment and the preset number of lexical units, the segment content at the segment granularity is determined, wherein the number of lexical units at the segment granularity is less than or equal to the preset number of lexical units.

[0018] Optionally, the target document is a new version document with updated content compared to historical versions. Generating the question-answer pair based on the content at the target granularity includes:

[0019] Based on the content at the target granularity and the content at the target granularity in the historical version document, determine the chunks of content required to update the knowledge base from the target document;

[0020] The question-answer pair is generated based on the chunks of content required to update the knowledge base.

[0021] Optionally, generating the question-answer pair based on the chunks of content required to update the knowledge base includes:

[0022] In response to the fact that the data type of the segmented content is a table type, supplementary information of the segmented content is obtained based on the segmented content and the target document; the supplementary information includes: context information of the segmented content in the target document, and / or, title information to which the segmented content belongs;

[0023] The question-answer pair is generated based on the supplementary information of the segmented content and the segmented content itself.

[0024] Optionally, before generating question-answer pairs based on the target document, the method further includes:

[0025] Get the initial document;

[0026] The initial document is preprocessed to obtain the target document;

[0027] The preprocessing includes at least one of the following operations: converting the format of the initial document to the target format, correcting the headings at all levels of the initial document, deleting images from the initial document, and converting the identifiers of referenced documents in the initial document into the access addresses of the referenced documents.

[0028] Optionally, the knowledge base is a vector database, and updating the knowledge base based on the question-answer pairs includes:

[0029] The question-and-answer pairs are encoded to obtain the encoded vector data of the question-and-answer pairs;

[0030] The vector data is stored in the knowledge base.

[0031] Secondly, this application provides a question-and-answer method, the method comprising:

[0032] Obtain the target question to be answered;

[0033] Based on the target question, a reference question-answer pair is retrieved from a knowledge base; the knowledge base is as described in any of the first aspects; the reference question-answer pair includes: a reference question and a reference answer;

[0034] Based on the reference question-answer pair and the target question, a target answer to the target question is generated.

[0035] Optionally, generating the target answer to the target question based on the reference question-answer pair and the target question includes:

[0036] Based on the reference question-answer pair and the target question, target prompt words are obtained;

[0037] The target prompt is input into the question-answering model to obtain the target answer.

[0038] Thirdly, this application provides a smart assistant, which includes: a client and a server;

[0039] The client is used to receive the target question to be answered and send the target question to the server;

[0040] The server is configured to retrieve reference question-answer pairs from a knowledge base based on the target question; generate a target answer to the target question based on the reference question-answer pairs and the target question; and output the target answer through the client; the knowledge base is a knowledge base as described in any of the first aspects; the reference question-answer pairs include: a reference question and a reference answer.

[0041] Fourthly, this application provides a cloud service platform, which includes: the intelligent assistant as described in the third aspect;

[0042] The intelligent assistant is used to retrieve reference question-answer pairs from a knowledge base based on a target question; and to generate a target answer to the target question based on the reference question-answer pairs and the target question; the knowledge base is a knowledge base as described in any of the first aspects; the reference question-answer pairs include: a reference question and a reference answer;

[0043] The target question is used to ask questions related to the cloud service platform; the reference question-and-answer pair is generated based on the target document; the target document is used to describe relevant knowledge about the cloud service platform.

[0044] Fifthly, this application provides a data processing apparatus, the apparatus comprising:

[0045] A processing module is used to generate question-answer pairs based on a target document; the question-answer pairs include: questions that can be answered based on the target document, and answers corresponding to the questions;

[0046] An update module is used to update the knowledge base based on the question-answer pair; the knowledge base is used to provide reference question-answer pairs for the target question to be answered; the target question and the reference question-answer pairs are used to generate the target answer for the target question.

[0047] Sixthly, this application provides an electronic device, including: a processor and a memory; the processor and the memory are communicatively connected.

[0048] The memory stores computer-executed instructions;

[0049] The processor executes computer execution instructions stored in the memory to implement the method as described in any one of the first and / or second aspects.

[0050] In a seventh aspect, this application provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the method as described in any one of the first and / or second aspects.

[0051] Eighthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in either the first aspect or the second aspect.

[0052] The data processing, question-answering method, apparatus, electronic device, and storage medium provided in this application can generate question-answer pairs from a target document. Based on these pairs, a knowledge base can be updated, enabling it to provide reference question-answer pairs for subsequent questions. This, in turn, allows for the generation of the target answer based on the reference question-answer pairs and the target question. This method achieves automated updating of the knowledge base based on the target document, providing reference question-answer pairs for the subsequent question-answering process and laying the foundation for improving the accuracy of answers to the target question. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 A flowchart illustrating a data processing method provided in this application;

[0055] Figure 2 This application provides a flowchart illustrating a method for generating question-answer pairs based on target documents.

[0056] Figure 3 A flowchart illustrating a question-and-answer method provided in this application;

[0057] Figure 4 A schematic diagram of the structure of an intelligent assistant provided in this application;

[0058] Figure 5 A schematic diagram of the structure of a cloud service platform provided in this application;

[0059] Figure 6 A flowchart illustrating another data processing method provided in this application;

[0060] Figure 7 A schematic diagram of the structure of a data processing device provided in this application;

[0061] Figure 8 A schematic diagram of the structure of a question-and-answer device provided in this application;

[0062] Figure 9This is a schematic diagram of the hardware structure of an electronic device provided in this application.

[0063] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0065] The following is a brief explanation of some of the terms and concepts used in this application:

[0066] Large language models refer to deep learning models trained on massive amounts of data. They possess strong language understanding and generation capabilities and can be applied to natural language processing tasks across various scenarios.

[0067] The illusion problem refers to the phenomenon where large language models generate inaccurate answers (such as fabricated or misleading answers) due to a lack of accurate references or insufficient training data.

[0068] Large language models can be applied to areas such as user question answering. For example, for complex and cumbersome cloud service platforms, a smart assistant can be built based on a large language model. This smart assistant can assist and guide users in using the cloud service platform.

[0069] However, due to the illusion problem of large language models, the answers provided by existing intelligent assistants based on large language models have poor accuracy.

[0070] The inventors discovered through research that RAG technology can improve the illusion problem in large language models. RAG is a technique that combines retrieval and generation to improve the performance of natural language processing tasks. The core idea of ​​RAG is to first retrieve documents or knowledge fragments related to the question based on a limited knowledge base, and then provide these retrieved documents or knowledge fragments to a sequence-to-sequence (seq2seq) generative model to generate the final answer.

[0071] By combining RAG with a large language model, the large language model can generate answers based on the retrieved documents, thus improving the accuracy of the generated answers.

[0072] However, how to construct the aforementioned knowledge base is a problem that urgently needs to be solved.

[0073] Therefore, this application proposes a data processing method for automating knowledge base updates. This method updates the knowledge base using question-and-answer pairs, which in turn provide relevant reference question-and-answer pairs for subsequent target questions, laying the foundation for improving the accuracy of the answers to those target questions. Optionally, the execution entity of this data processing method can be any electronic device with processing capabilities, such as a server or a terminal device.

[0074] The technical solutions of this application will be described in detail below with reference to specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0075] Figure 1 This is a flowchart illustrating a data processing method provided in this application. Figure 1 As shown, the method may include the following steps:

[0076] S101. Generate question-answer pairs based on the target document.

[0077] Optionally, the aforementioned target document can be used to describe relevant knowledge of any target object. For example, the target object can be an electronic product, software product (e.g., operating system, application, etc.). Alternatively, the target object can also be, for example, a cloud service platform. For example, the relevant knowledge of the target object may include: user manuals, functional specifications, product introductions, safety usage guidelines, and other documents related to the target object. This application does not limit the content described in the target document.

[0078] The question-and-answer pair mentioned above may include: a question that can be answered based on the target document, and the answer to that question.

[0079] For example, taking the aforementioned target document as a user manual for a cloud service platform, the question that this target document can answer could be: "What operating system versions does the cloud service platform client support?" The corresponding answer could be: "The range of operating systems supported by the cloud service platform client includes: server operating systems…".

[0080] It should be understood that this application does not limit the number of question-answer pairs corresponding to the aforementioned target document. For example, an electronic device can generate at least one question-answer pair based on the target document.

[0081] Optionally, the electronic device may receive the target document input manually, for example, through an Application Programming Interface (API) or a Graphical User Interface (GUI). Alternatively, the electronic device may also automatically detect updates to relevant documents related to the aforementioned target object to obtain the target document.

[0082] As one possible implementation, electronic devices could generate the aforementioned question-and-answer pairs based on a target document using a large language model. For example, the electronic device could obtain prompt words based on the target document and input these prompt words into the large language model to obtain question-and-answer pairs. For example, the prompt words could include "[target document], please generate several pairs of questions that can be answered based on the target document, and the corresponding answers to those questions."

[0083] As another possible implementation, the electronic device could first divide the target document into multiple sub-documents. Then, for each sub-document, the electronic device could generate question-and-answer pairs based on that sub-document. Optionally, the specific implementation of the electronic device generating question-and-answer pairs based on the sub-document can refer to the method of generating question-and-answer pairs based on the target document, and will not be elaborated here.

[0084] As another possible implementation, the electronic device could first determine the differences between the target document and historical versions of the document, based on the target document and historical versions. Then, the electronic device could generate question-answer pairs based on these differences. For example, the electronic device could obtain prompt words based on these differences and input them into a large language model to obtain question-answer pairs.

[0085] S102. Update the knowledge base based on question-answer pairs.

[0086] Optionally, the knowledge base can be used to provide reference question-and-answer pairs for the target question to be answered. These reference question-and-answer pairs belong to the aforementioned question-and-answer pairs generated based on the target document. The target question and the reference question-and-answer pairs can be used to generate the target answer to the target question. In other words, based on the target question to be answered, a reference question-and-answer pair can be obtained from the knowledge base, and based on the target question and the reference question-and-answer pairs, the target answer to the target question can be generated.

[0087] It should be understood that the executing entity of the data processing method provided in this application and the executing entity used to perform "based on the above-mentioned target question to be answered, obtain reference question-answer pairs from the knowledge base, and generate the target answer of the target question based on the target question and the reference question-answer pairs" can be the same executing entity or different executing entities.

[0088] It should be understood that this application does not limit the type of the aforementioned knowledge base. For example, the knowledge base can be a vector database (which can be used to store vector data). Alternatively, the knowledge base can also be a scalar database (which can be used to store scalar data).

[0089] Optionally, after acquiring the question-and-answer pair, the electronic device can, for example, store the question-and-answer pair in the knowledge base to update the knowledge base. In some embodiments, taking the knowledge base as a vector database, the electronic device can, for example, encode the question-and-answer pair after acquiring it to obtain the encoded vector data, and then store the vector data in the knowledge base. For example, taking the question-and-answer pair as text data, the electronic device can encode the question-and-answer pair to obtain the encoded vector data by referring to any existing method for converting text into text feature vectors, which will not be elaborated further in this application.

[0090] Alternatively, in some embodiments, the knowledge base may be a relational database, an in-memory database, or a similar type of database. This application does not limit the type of the knowledge base.

[0091] In this embodiment, a question-and-answer pair can be generated from the target document. Based on this pair, the knowledge base can be updated, enabling it to provide reference question-and-answer pairs for the target question to be answered. This, in turn, allows for the generation of the target answer based on the reference question-and-answer pair and the target question. This method achieves automated updating of the knowledge base based on the target document, providing reference question-and-answer pairs for the subsequent question-and-answer process and laying the foundation for improving the accuracy of the answers to the target question.

[0092] The following section provides a detailed explanation of how electronic devices can generate question-and-answer pairs based on target documents:

[0093] Figure 2 This application provides a flowchart illustrating a method for generating question-answer pairs based on a target document. Figure 2 As shown, as one possible implementation, step S101 above may include the following steps:

[0094] S201. Determine the target granularity of the target document.

[0095] The target granularity mentioned above may include, for example, at least one of the following: document granularity, paragraph granularity, and block granularity.

[0096] Taking the target granularity as an example, the content of the document granularity can be the complete content of the target document. In other words, an electronic device can use the complete content of the target document as the content of the document granularity.

[0097] Taking a target granularity including paragraph granularity as an example, the content of the paragraph granularity can be the content of any paragraph in the target document. Optionally, the electronic device can, for example, segment the target document to determine each paragraph of the target document. Then, the electronic device can use the content of each paragraph as the content of the paragraph granularity of the target document. For example, taking a target document including paragraph identifiers as an example, the electronic device can, for example, segment the target document based on the paragraph identifiers of the target document.

[0098] The granularity of the above-mentioned chunks is smaller than that of the paragraphs. That is, the content of a chunk is less than the content of a paragraph. In some embodiments, this chunk granularity may also be referred to as Chunk granularity.

[0099] Taking the aforementioned target granularity as an example, including the chunk granularity, in some embodiments, the electronic device may first segment the target document to obtain the paragraph content at the paragraph granularity. Then, the electronic device may determine the chunk content at the aforementioned chunk granularity based on the data type of the paragraph content.

[0100] The data types mentioned above can be, for example, text data or table data. All the content blocks mentioned above belong to the same data type. For example, any content block can be text data, or it can be table data.

[0101] Optionally, paragraph content at any paragraph level within the target document may include content of one data type (e.g., only text data, or only table data). Alternatively, the paragraph content may include multiple data types (e.g., text data and table data).

[0102] Taking a paragraph that includes both text and tabular data as an example, the paragraph content could be as follows:

[0103] "Cloud service platforms are computing resource pools built on virtualization technology and distributed architecture. They provide users with elastically scalable infrastructure services. A comparison of the functional modules of common cloud service platforms is shown in Table 1 below:"

[0104] Table 1

[0105] Service Categories Main functions Technical characteristics Computing services Cloud servers and container services Supports elastic scaling and resource isolation Storage services Object storage, file storage Provides data redundancy and cross-region replication Network services Load balancing, etc. Achieve network isolation and traffic control Database service Relational databases, etc. Supports automatic backup and read / write separation Security services Access control, etc. Provide identity authentication and traffic scrubbing

[0106] The cloud service platform adopts a multi-availability zone deployment architecture to ensure service availability, supports automated operation and maintenance through standardized application programming interfaces, and implements security measures such as data encryption and access control in accordance with industry standards. All types of services provide detailed usage monitoring and logging functions.

[0107] Taking the paragraph content described in the example above as an example, the table data included in this paragraph is Table 1. The text data included in this paragraph is all the content in this paragraph other than Table 1.

[0108] Optionally, the electronic device may determine the chunk content at the aforementioned chunk granularity based on the number of tokens at the chunk granularity.

[0109] For example, an electronic device can first divide the paragraph content into at least one initial chunk based on the data type of the paragraph content. Then, the electronic device can determine the chunk content at the chunk granularity based on the number of tokens in the initial chunks and a preset number of tokens (i.e., the preset number of tokens). The number of tokens at the chunk granularity is less than or equal to the preset number of tokens.

[0110] For example, the contents of the initial blocks mentioned above can belong to the same data type.

[0111] For example, an electronic device can group content of the same data type within the aforementioned paragraph into a single initial block. For instance, assuming the paragraph content includes both text and table data, the electronic device can use the text data as one initial block and the table data as another. Alternatively, assuming the paragraph content consists only of text data, the electronic device can use that text data as one initial block (i.e., the electronic device can use the entire paragraph content as one initial block).

[0112] For example, the preset number of tokens mentioned above could be the number of tokens pre-stored in the electronic device.

[0113] Optionally, the number of preset tokens corresponding to the initial blocks of different data types can be the same or different, and this application does not impose any restrictions on this.

[0114] For example, for any initial block, the electronic device can determine whether the number of tokens in the initial block is less than or equal to the aforementioned preset number of tokens. If so, it indicates that the information described by the content of the initial block is already relatively simple, and the electronic device can, for example, use the content of the initial block as the block content at the aforementioned block granularity.

[0115] If not, it indicates that the information described in the initial block is complex. Therefore, the electronic device can further divide the initial block into multiple sub-blocks until the number of tokens in each sub-block is less than or equal to the preset number of tokens. Then, the electronic device can use the content of the sub-block as the block content at the block granularity.

[0116] The above method allows for the acquisition of initial chunks belonging to the same data type based on the paragraph content's data type, laying the foundation for ensuring that subsequent chunks at different granularities belong to the same data type. By determining the number of lexical units in the initial chunks and a preset number of lexical units, the content of each chunk at the granular level is made less than or equal to the preset number of lexical units. This ensures that the information described by the chunk content at the granular level is relatively singular, thereby improving the accuracy of subsequent question-and-answer pairs obtained based on that chunk content.

[0117] Alternatively, electronic devices may directly use the aforementioned initial blocks as the block content at the block granularity.

[0118] Alternatively, electronic devices can divide paragraph content into multiple chunks based on the number of tokens in the paragraph content, and use these chunks as a single chunk granularity.

[0119] In some embodiments, the electronic device may, for example, divide the target document into equal parts based on a preset number of tokens, such that the number of tokens in each block is equal to the preset number of tokens, and use the block content as the block content at the block granularity.

[0120] S202. Generate question-answer pairs based on the target granularity of the content.

[0121] For example, taking the target granularity as including document granularity and / or paragraph granularity, an electronic device can obtain prompt words based on the content of the target granularity and input the prompt words into a large language model, so that the large language model can generate question-answer pairs based on the content of the target granularity.

[0122] Taking the above target granularity as an example, the electronic device can optionally obtain prompt words based on the content of the segment granularity and input the prompt words into the large language model, so that the large language model can generate question-answer pairs based on the content of the segment granularity.

[0123] Alternatively, electronic devices can incrementally update the knowledge base based on content at the chunk level.

[0124] For example, taking the aforementioned target document as a new version document with updated content compared to historical versions, in some embodiments, the electronic device may, for example, determine the chunks of content required to update the knowledge base from the target document based on the content at the target granularity and the content at the target granularity in the historical versions.

[0125] For example, the electronic device may first compare the target document with the aforementioned historical version document at the paragraph level to identify paragraphs where content has been updated. Then, the electronic device may identify, for example, chunks of content within those updated paragraphs that have been updated compared to the historical version document. The electronic device may then use these updated chunks of content as the chunks required to update the knowledge base.

[0126] Then, the electronic device can generate the above question-and-answer pairs based on the chunks of content required to update the knowledge base.

[0127] By using the above method, question-answer pairs are generated by updating the required chunks of content in the knowledge base, which enables incremental updates of question-answer pairs. This reduces the computational load of calculating question-answer pairs, improves the efficiency of question-answer pair calculation, and thus improves the update efficiency of the knowledge base.

[0128] Optionally, the electronic device can directly obtain prompt words based on the chunks of content required for updating the knowledge base, and input the prompt words into the large language model, so that the large language model can generate question-answer pairs based on the chunks of content required for updating the knowledge base.

[0129] Alternatively, in some embodiments, the electronic device may first determine whether the data type of the segmented content is a table type. Then, based on the determination result of whether the data type of the segmented content is a table type, the question-answer pair generation method is determined.

[0130] For example, because table types typically require context to represent descriptive information, if the data type of the aforementioned content chunk is table-based, the electronic device can, for instance, first obtain supplementary information about the content chunk based on the content chunk and the target document. Then, the electronic device can generate the question-and-answer pair based on the supplementary information and the content chunk itself.

[0131] The supplementary information mentioned above may include, for example, the context information of the content in the target document, and / or the title information to which the content in the content belongs.

[0132] Taking the aforementioned supplementary information, including the context information of the content segment in the target document, as an example, an electronic device may, for instance, use the text data of a first number of tokens preceding the content segment and / or the text data of a second number of tokens following the content segment as the context information of the content segment, based on the position of the content segment in the target document. The first and second token numbers may be the same or different, and this application does not impose any limitation on this.

[0133] Taking the aforementioned supplementary information, including the title information of the content block, as an example, an electronic device can, for instance, determine the paragraph to which the content block belongs based on the content block. Then, the electronic device can determine the title information of the title to which the paragraph belongs (i.e., the title information of the content block) based on the target document. For instance, the title information may include, for example, the name of the title, the name of the parent title to which the title belongs, and other information.

[0134] If the data type of the aforementioned chunked content is not a table type, for example, the electronic device can obtain prompt words based on the chunked content, input the prompt words into a large language model, and enable the large language model to generate question-answer pairs based on the chunked content.

[0135] Using the above method, electronic devices can generate question-and-answer pairs based on supplementary information and the content itself when the data type of the segmented content is table-type, thus improving the accuracy of generating question-and-answer pairs based on table-type segmented content.

[0136] In this embodiment, question-answer pairs are generated based on the target granularity of the target document's content. This makes the question-answer pairs generated based on more detailed content, increasing their richness. Therefore, it can improve the accuracy of providing reference question-answer pairs for the target question based on these question-answer pairs, thereby improving the accuracy of question-answering.

[0137] As one possible implementation, the electronic device can preprocess the initial document before generating question-answer pairs based on the target document, for example, by using the preprocessed initial document as the target document.

[0138] For example, an electronic device can first obtain an initial document, and then preprocess the initial document to obtain the target document.

[0139] Optionally, the electronic device can receive manually input initial documents via API or GUI. Alternatively, the electronic device can automatically detect updated documents and use them as initial documents. It should be understood that this application does not limit the implementation method of the electronic device obtaining initial documents.

[0140] The preprocessing described above may include, for example, at least one of the following operations: converting the format of the initial document to the target format, correcting the headings at all levels of the initial document, deleting images from the initial document, and converting the identifiers of referenced documents in the initial document into access addresses of the referenced documents.

[0141] Taking the preprocessing described above, which includes converting the format of the initial document to the target format, as an example, assuming the initial document is in Hyper Text Markup Language (HTML) format, an electronic device can, for example, convert the initial document to Markdown (a lightweight markup language) format. It should be understood that the implementation of converting the initial document format to the target format described above can refer to any existing file format conversion method, and will not be elaborated upon here.

[0142] Taking the aforementioned preprocessing, including correcting the headings at each level of the initial document, as an example, the electronic device can, for instance, use a preset heading enhancement algorithm to correct the headings at each level of the initial document, thereby improving the completeness and accuracy of the headings. Alternatively, the electronic device can, for instance, input the initial document into a large language model, and through this large language model, correct the headings at each level of the initial document based on its content.

[0143] Taking the aforementioned preprocessing, including deleting images from the initial document, as an example, the electronic device can, for instance, recognize the content of the initial document and delete image-formatted content. By deleting images from the initial document, the amount of data in the initial document is reduced, which in turn reduces the amount of data in the target document obtained subsequently based on the initial document, thereby improving the efficiency of subsequent data processing based on the target document.

[0144] Taking the preprocessing described above, which includes converting the identifier of a referenced document in the initial document into an access address of the referenced document, as an example, the electronic device can determine the access address of the referenced document based on the identifier of the referenced document and the mapping relationship between the identifier of the referenced document and the access address of the document. For example, the mapping relationship between the document identifier and the access address of the document can be a mapping relationship pre-configured in the electronic device. Alternatively, the electronic device can also request the access address of the referenced document from a device storing the mapping relationship. For example, the access address can be a Uniform Resource Locator (URL) address.

[0145] By converting the identifier of the referenced document into the access address of the referenced document, and using it as the target document, the content of the referenced document can be obtained based on the access address when the target document is used subsequently (e.g., to generate question-answer pairs based on the target document). This lays the foundation for obtaining highly accurate question-answer pairs based on the target document.

[0146] This application also provides a question-answering method. Optionally, the entity executing this question-answering method may be, for example, a question-answering system. Figure 3 This is a flowchart illustrating a question-and-answer method provided in this application. Figure 3 As shown, the method includes the following steps:

[0147] S301. Obtain the target question to be answered.

[0148] For example, the question-and-answer system can receive the aforementioned target question input by the user through an API or GUI.

[0149] S302. Based on the target question, retrieve reference question-answer pairs from the knowledge base.

[0150] The knowledge base mentioned above can be any of the knowledge bases described in the foregoing embodiments. The reference question-and-answer pairs mentioned above can include: a reference question and a reference answer.

[0151] For example, taking the knowledge base as a vector database, the question-answering system can encode the target question to obtain a feature vector corresponding to the target question. Then, the electronic device can retrieve information from the knowledge base based on this feature vector, determine the vector of questions matching the feature vector, and use the question corresponding to this vector as the reference question. The question-answering system can then, for example, obtain the reference answer through this reference question and the mapping relationship between questions and answers.

[0152] For example, the mapping relationship between the question and the answer can be shown in Table 2 below:

[0153] Table 2

[0154] question Answer Question 1 Answer 1 Question 2 Answer 2 Question 3 Answer 3 …… ……

[0155] Assuming the question-answering system determines the reference question as question 2 using the above method, and taking the mapping relationship shown in Table 2 as an example, the question-answering system can determine the reference answer as answer 2.

[0156] Optionally, the question-answering system can retrieve the implementation of reference question-answer pairs from the knowledge base based on the aforementioned target question. For example, it can also refer to any other existing data retrieval method, which will not be elaborated here.

[0157] S303. Based on the reference question-answer pair and the target question, generate the target answer to the target question.

[0158] Optionally, the question-answering system can, for example, derive target prompts based on the aforementioned reference question-answer pairs and the target question. Then, the question-answering system can input these target prompts into the question-answering model to obtain the aforementioned target answer.

[0159] For example, a question-and-answer system can combine the aforementioned reference question-and-answer pair and the target question using a preset concatenation method to obtain the target prompt word. For example, the preset concatenation method can be to add the reference question-and-answer pair and the target question to a preset statement to obtain the target prompt word. For example, the target prompt word can be as follows:

[0160] "Based on the reference question and answer pair [Reference Question and Answer Pair Content], please provide the answer to this question [Target Question Content]".

[0161] For example, the question-answering model described above can be any existing large language model or machine learning model, and this application does not limit the question-answering model. For example, taking a large language model as an example, the question-answering system can input the target prompt word into the large language model, so that the large language model can generate the target answer to the target question based on the target prompt word.

[0162] In some embodiments, after generating the target answer, the question-answering system may output the target answer so that the user can view it in a timely manner.

[0163] In this embodiment, by retrieving the reference question-answer pair corresponding to the target question from the knowledge base, the target answer to the target question can be generated based on the target question and the reference question-answer pair, providing a reference for the generation of the target answer and thus improving the accuracy of the target answer.

[0164] This application also provides a smart assistant. Figure 4 This is a structural diagram of a smart assistant provided in this application. Figure 4 As shown, optionally, the smart assistant may include: a client and a server.

[0165] For example, the aforementioned client can be deployed on a user terminal device. This user terminal device can be any electronic device such as a mobile phone, computer, or tablet.

[0166] This client can be used to receive the target question to be answered and send the target question to the server.

[0167] It should be understood that this application does not limit the deployment environment of the client, nor the communication method between the client and the server.

[0168] Accordingly, the server can receive the target question.

[0169] The server can retrieve reference question-and-answer pairs from the knowledge base based on the target question. Then, based on the reference question-and-answer pairs and the target question, the server can generate the target answer for the target question and output the target answer through the client mentioned above.

[0170] As mentioned above, the knowledge base can be as described in any of the foregoing embodiments. The aforementioned question-and-answer pairs may include: a reference question and a reference answer.

[0171] The intelligent assistant provided in this application can be used to execute the question-and-answer method embodiments described in any of the foregoing embodiments. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0172] This application also provides a cloud service platform. Figure 5 This is a schematic diagram of the structure of a cloud service platform provided in this application. Figure 5 As shown, the cloud service platform may include a smart assistant as described in the foregoing embodiments. In some embodiments, the cloud service platform may provide cloud services. It should be understood that this application does not limit the cloud services provided by the cloud service platform. The cloud services provided by the cloud service platform may, for example, refer to the cloud services that any existing cloud service platform can provide.

[0173] The cloud service platform can use this intelligent assistant to retrieve reference question-and-answer pairs from the knowledge base based on the target question, and generate a target answer for the target question based on the reference question-and-answer pairs and the target question. As mentioned above, the knowledge base can be the knowledge base described in any of the foregoing embodiments. The aforementioned reference question-and-answer pairs can include: a reference question and a reference answer.

[0174] In this implementation, the target question can be used to inquire about issues related to the cloud service platform. The reference question-and-answer pair can be generated based on the target document. This target document can be used to describe relevant knowledge about the cloud service platform.

[0175] The cloud service platform provided in this application can execute the question-and-answer method embodiments described in any of the foregoing embodiments through a smart assistant. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0176] The following example, using the aforementioned target document as an example of the security usage guidelines for cloud service platforms, illustrates the updating and question-and-answer process of this knowledge base:

[0177] First, the electronic device can synchronize the initial document to an Open Data Processing Service (ODPS). Then, the electronic device can retrieve the same initial document and differentiate it into documents in different languages. That is, a single document ID corresponds to documents in multiple languages. The electronic device can then retrieve all the most recently updated documents from these documents and perform the following operations on each updated document:

[0178] Figure 6 A flowchart illustrating another data processing method provided in this application. Figure 6 As shown, for example, an electronic device can perform step 601, converting an HTML document into Markdown format. Furthermore, the Markdown document can be cleaned. Specifically, this document cleaning process may include, for example, determining whether the content of each level of headings in the document is correct, deleting image content from the document, and completing web link sections (e.g., replacing web link sections with actual URL addresses), etc.

[0179] Electronic devices can, for example, acquire question-and-answer pairs corresponding to content at three granularities: document granularity, paragraph granularity, and chunk granularity.

[0180] Taking document granularity as an example, electronic devices can input the complete content of the target document into a large language model, and instruct the large language model through prompt words to generate question-answer pairs based on the target document (e.g., generate a question-answer pair).

[0181] For example, the question-answer pairs generated based on the target document can be:

[0182] Question: What operating system versions does the cloud service platform client support?

[0183] Answer: The document outlines the range of operating systems supported by the cloud service platform's clients, including server operating systems. It lists detailed supported operating system versions and kernel versions, reminding users to ensure their server operating system and kernel version are within the specified support range to ensure that clients and various plugins function correctly and provide security protection.

[0184] Electronic devices can, for example, segment a target document to obtain content at the paragraph level (i.e., paragraphs). For any given paragraph, the electronic device can input the content of that paragraph into a large language model and instruct the large language model using prompt words to generate a question-and-answer pair based on that paragraph (e.g., generate a single question-and-answer pair).

[0185] For example, for any paragraph, the paragraph content may include: the document name of the target document to which the paragraph belongs, and the first-level heading of the paragraph (e.g., it may also include second-level headings, third-level headings, etc.). For example, a paragraph may include the following content:

[0186] "Document Name": "Operating Systems Supported by the Client"

[0187] "Level 1 Heading": "Client Supported Operating Systems"

[0188] "Second-level heading: Operating system versions supported by ransomware protection"

[0189] Third-level heading: "Operating System Versions Supported by Server Ransomware Protection"

[0190] "Paragraph content": "Only when the server's operating system version and kernel version simultaneously meet the requirements in the table below will the server support the network attack protection function provided by the xx plugin...".

[0191] For example, a question-and-answer pair generated based on a paragraph could be:

[0192] Question: Client compatibility list: What are the supported operating systems and corresponding kernel versions?

[0193] Answer: Supports multiple operating systems and specific kernel versions...

[0194] In step S602, the electronic device may, for example, split the paragraph into initial blocks of table type or initial blocks of text type based on any paragraph.

[0195] For example, taking a table as the data type of the initial block, the initial block may include the following content:

[0196] "Document Name": "Client Supported Operating Systems"

[0197] "Level 1 Heading": "Client Supported Operating Systems"

[0198] "Second-level heading": "Operating system versions supported by ransomware protection"

[0199] Third-level heading: "Operating System Versions Supported by Server Ransomware Protection"

[0200] "Data type": "table"

[0201] "Initial block content": "(Table)...".

[0202] For example, taking the initial block's data type as text, the initial block may include the following content:

[0203] "Document Name": "Client Supported Operating Systems"

[0204] "Level 1 Heading": "Client Supported Operating Systems"

[0205] "Second-level heading": "Operating system versions supported by ransomware protection"

[0206] Third-level heading: "Operating System Versions Supported by Server Ransomware Protection"

[0207] "Data type": "text"

[0208] "Initial chunk content": "(text)...".

[0209] In steps S603 and S604, the electronic device may, for example, calculate the number of tokens for each initial block, and then divide the blocks at a granular level based on this number of tokens. That is, the electronic device may divide the table and text of a document paragraph into blocks to obtain the block content. In step S605, for example, the electronic device may input the content at this block granularity into a large language model, and instruct the large language model through prompt words to generate question-answer pairs based on the content at this block granularity (multiple question-answer pairs can be generated, for example, 10 question-answer pairs).

[0210] For example, question-answer pairs generated based on chunk-level content granularity may include:

[0211] Question 1: For different operating systems, are there detailed lists of version numbers and kernel version numbers?

[0212] Answer 1: Based on the format of the paragraph content, it should provide a detailed list of version numbers and kernel version numbers for different operating systems.

[0213] Question 2: How can I determine if my operating system kernel version is compatible with a specific service?

[0214] Answer 2: The kernel version number needs to be compared with the support list provided by the specified service to determine compatibility.

[0215] Question 3...

[0216] In steps S606 and S607, the electronic device can, for example, encode the question-answer pairs generated at the three granularities mentioned above using an encoding model, and store them in the vector database (i.e., the knowledge base). For example, the electronic device can write the encoding results and information at different granularities into the knowledge base. Furthermore, the electronic device can also write the encoding results and information at different granularities into the ODPS and record the data writing status. If an error message is found, the data can be rewritten.

[0217] In some embodiments, the electronic device may also perform knowledge base construction optimization. Optionally, the knowledge base construction optimization method may refer to any existing database optimization method. For example, the electronic device may optimize the knowledge base construction using a user-defined function (UDF) to accelerate the process.

[0218] For example, in the process of converting HTML web pages to Markdown format, data processing efficiency and knowledge base optimization efficiency can be improved by using hints (a method to improve data processing efficiency by adjusting parameters such as concurrency and resource allocation to optimize task execution performance).

[0219] In some embodiments, the electronic device may further optimize the updating efficiency of the knowledge base by using a thread pool for medium-volume network requests (e.g., network requests to write encoded question-and-answer pairs into the knowledge base in batches).

[0220] In some embodiments, considering that the large language model may experience rate limiting during data processing, potentially leading to data loss when generating question-and-answer pairs from chunked content, the electronic device can perform correctness verification on the generated question-and-answer pairs. For example, the electronic device can verify the format and completeness of the content to determine if the question-and-answer pairs are correct. If the question-and-answer pairs pass the correctness verification, they can be stored in the knowledge base. If the question-and-answer pairs fail the correctness verification, the electronic device can re-input the chunked content into the large language model, re-obtain the question-and-answer pairs through the large language model, and rewrite the obtained question-and-answer pairs into the knowledge base.

[0221] In this embodiment, preprocessing the initial document, such as format standardization, ensures the accuracy of subsequent question-answer pair retrieval based on the preprocessed target document. By combining modules such as a large language model, ODPS, and a vector database, automated updates and maintenance of the knowledge base are achieved. Incremental updates reduce wasted computational resources. Updating the knowledge base using question-answer pairs at three granularities—document, paragraph, and chunk—improves its stability. UDF concurrency optimization and thread pool optimization enhance data processing efficiency. Maintaining knowledge base metadata (e.g., question-answer pairs already written to the knowledge base), monitoring data changes, and automatically retrying failed writes further improves the stability of the knowledge base.

[0222] Figure 7 This is a schematic diagram of the structure of a data processing device provided in this application. Figure 7 As shown, the data processing device 50 may include: a generation module 51 and an update module 52. Wherein,

[0223] The generation module 51 is used to generate question-answer pairs based on the target document. The question-answer pairs include: questions that can be answered based on the target document, and the answers to those questions.

[0224] The update module 52 is used to update the knowledge base based on the question-answer pair. The knowledge base is used to provide reference question-answer pairs for the target question to be answered; the target question and the reference question-answer pairs are used to generate the target answer to the target question.

[0225] Optionally, the generation module 51 is specifically used to determine the content of the target document at a target granularity based on the target document; and to generate the question-answer pair based on the content at the target granularity. The target granularity includes at least one of the following: document granularity, paragraph granularity, and block granularity; the block granularity is smaller than the paragraph granularity.

[0226] Taking the target granularity as an example, specifically the block granularity, the generation module 51 is used to segment the target document to obtain the paragraph content of the target document at the paragraph granularity; and to determine the block content of the block granularity based on the data type of the paragraph content. Wherein, the data type is text data or table data; and the block content belongs to the same data type.

[0227] Optionally, the generation module 51 is specifically used to split the paragraph content into at least one initial block based on the data type of the paragraph content; and to determine the block content at the block granularity based on the number of tokens in the initial block and a preset number of tokens. Wherein, the content of the initial blocks belongs to the same data type; and the number of tokens in the block granularity is less than or equal to the preset number of tokens.

[0228] Taking the target document as an example, which is a new version document with updated content compared to the historical version document, optionally, the generation module 51 is specifically used to determine the block content required to update the knowledge base from the target document based on the content at the target granularity and the content at the target granularity in the historical version document; and generate the question-answer pair based on the block content required to update the knowledge base.

[0229] Optionally, the generation module 51 is specifically configured to, in response to the data type of the segmented content being a table type, obtain supplementary information of the segmented content based on the segmented content and the target document; and generate the question-answer pair based on the supplementary information of the segmented content and the segmented content. The supplementary information includes: context information of the segmented content in the target document, and / or, the title information to which the segmented content belongs.

[0230] Optionally, the data processing device 50 may further include an acquisition module 53, configured to acquire an initial document before generating question-answer pairs based on the target document; and preprocess the initial document to obtain the target document. The preprocessing includes at least one of the following operations: converting the format of the initial document to the target format, correcting the headings at each level of the initial document, deleting images from the initial document, and converting the identifiers of referenced documents in the initial document into access addresses of the referenced documents.

[0231] Optionally, the update module 52 is specifically used to encode the question-answer pair to obtain the encoded vector data of the question-answer pair; and to store the vector data in the knowledge base.

[0232] The data processing apparatus 50 provided in this application is used to execute the aforementioned data processing method embodiments. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0233] Figure 8 This is a schematic diagram of a question-and-answer device provided in this application. Figure 8 As shown, the question-and-answer device 60 may include: an acquisition module 61, a retrieval module 62, and a generation module 63. Among them,

[0234] Module 61 is used to obtain the target question to be answered.

[0235] The retrieval module 62 is used to retrieve reference question-answer pairs from a knowledge base based on the target question. The knowledge base is as described in any of the foregoing embodiments; the reference question-answer pairs include: a reference question and a reference answer.

[0236] The generation module 63 is used to generate a target answer to the target question based on the reference question-answer pair and the target question.

[0237] Optionally, the generation module 63 is specifically used to obtain target prompt words based on the reference question-answer pair and the target question; and input the target prompt words into the question-answering model to obtain the target answer.

[0238] The question-answering device 60 provided in this application is used to execute the aforementioned question-answering method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0239] Figure 9 This is a schematic diagram of the hardware structure of an electronic device provided in this application. Figure 9 The illustrated electronic device 70 includes a memory 71, a processor 72, and a communication interface 73. The memory 71, processor 72, and communication interface 73 are communicatively connected to each other. For example, the memory 71, processor 72, and communication interface 73 can be connected via a network. Alternatively, the electronic device 70 may also include a bus 74. The memory 71, processor 72, and communication interface 73 are communicatively connected to each other via the bus 74. Figure 9 It is an electronic device 70 in which the memory 71, processor 72, and communication interface 73 are connected to each other via bus 74.

[0240] The memory 71 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 71 may store programs, and when the program stored in the memory 71 is executed by the processor 72, the processor 72 and the communication interface 73 are used to perform the data processing and / or question-and-answer methods described in any of the foregoing embodiments. The memory may also store data required for the data processing and / or question-and-answer methods.

[0241] The processor 72 can be a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), graphics processing unit (GPU), or one or more integrated circuits.

[0242] Processor 72 can also be an integrated circuit chip with signal processing capabilities. In implementation, the data processing and / or question-and-answer methods of this application can be completed through integrated logic circuits in the hardware of processor 72 or through software instructions. The aforementioned processor 72 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the following embodiments of this application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the following embodiments of this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 71. Processor 72 reads the information in memory 71 and, in conjunction with its hardware, completes the data processing and / or question-and-answer method of this application.

[0243] Communication interface 73 uses transceiver modules, such as, but not limited to, transceivers, to enable communication between electronic device 70 and other devices or communication networks. For example, data sets can be acquired through communication interface 73.

[0244] When the aforementioned electronic device 70 includes a bus 74, the bus 74 may include a path for transmitting information between various components of the electronic device 70 (e.g., memory 71, processor 72, communication interface 73).

[0245] This application also provides a computer-readable storage medium, which may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. Specifically, the computer-readable storage medium stores program instructions, which are used in the methods described in the above embodiments.

[0246] This application also provides a program product including executable instructions stored in a readable storage medium. At least one processor of an electronic device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the electronic device to implement the data processing and / or question-and-answer methods provided in the various embodiments described above.

[0247] The term "multiple" in this document refers to two or more. The term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the preceding and following related objects; in formulas, " / " indicates a "division" relationship. Additionally, it should be understood that in the description of this application, words such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order.

[0248] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0249] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A data processing method, characterized by, The method includes: Based on the target document, a question-answer pair is generated; the question-answer pair includes: a question that can be answered based on the target document, and the answer to the question; Based on the question-answer pair, the knowledge base is updated; the knowledge base is used to provide reference question-answer pairs for the target question to be answered; the target question and the reference question-answer pair are used to generate the target answer to the target question.

2. The method of claim 1, wherein, The generation of question-answer pairs based on the target document includes: The content of the target document at the target granularity is determined; the target granularity includes at least one of the following: document granularity, paragraph granularity, and block granularity; the block granularity is smaller than the paragraph granularity. The question-answer pair is generated based on the content at the target granularity.

3. The method of claim 2, wherein, The target granularity includes: the block granularity; the step of determining the target granularity of the target document based on the target document includes: The target document is segmented to obtain the paragraph content at the paragraph level of the target document; Based on the data type of the paragraph content, the block content at the block granularity is determined; the data type is text data or table data; the block content belongs to the same data type.

4. The method of claim 3, wherein, The determination of the chunk content at the chunk granularity based on the data type of the paragraph content includes: Based on the data type of the paragraph content, the paragraph content is divided into at least one initial block; the content of the initial block belongs to the same data type. Based on the number of lexical units in the initial segment and the preset number of lexical units, the segment content at the segment granularity is determined, wherein the number of lexical units at the segment granularity is less than or equal to the preset number of lexical units.

5. The method according to claim 3 or 4, characterized in that, The target document is a new version document with updated content compared to historical versions. Generating the question-answer pair based on the content at the target granularity includes: Based on the content at the target granularity and the content at the target granularity in the historical version document, determine the chunks of content required to update the knowledge base from the target document; The question-answer pair is generated based on the chunks of content required to update the knowledge base.

6. The method according to claim 5, characterized in that, The process of generating the question-answer pair based on the chunked content required to update the knowledge base includes: In response to the fact that the data type of the segmented content is a table type, supplementary information of the segmented content is obtained based on the segmented content and the target document; the supplementary information includes: context information of the segmented content in the target document, and / or, title information to which the segmented content belongs; The question-answer pair is generated based on the supplementary information of the segmented content and the segmented content itself.

7. The method according to any one of claims 1-4, characterized in that, Before generating question-answer pairs based on the target document, the method further includes: Get the initial document; The initial document is preprocessed to obtain the target document; The preprocessing includes at least one of the following operations: converting the format of the initial document to the target format, correcting the headings at all levels of the initial document, deleting images from the initial document, and converting the identifiers of referenced documents in the initial document into the access addresses of the referenced documents.

8. The method according to any one of claims 1-4, characterized in that, The knowledge base is a vector database, and updating the knowledge base based on the question-answer pairs includes: The question-and-answer pairs are encoded to obtain the encoded vector data of the question-and-answer pairs; The vector data is stored in the knowledge base.

9. A question-and-answer method, characterized in that, The method includes: Obtain the target question to be answered; Based on the target question, a reference question-answer pair is retrieved from a knowledge base; the knowledge base is the knowledge base as described in any one of claims 1-8; the reference question-answer pair includes: a reference question and a reference answer; Based on the reference question-answer pair and the target question, a target answer to the target question is generated.

10. The method according to claim 9, characterized in that, The step of generating a target answer to the target question based on the reference question-answer pair and the target question includes: Based on the reference question-answer pair and the target question, target prompt words are obtained; The target prompt is input into the question-answering model to obtain the target answer.

11. A smart assistant, characterized in that, The smart assistant includes: a client and a server; The client is used to receive the target question to be answered and send the target question to the server; The server is configured to retrieve reference question-answer pairs from a knowledge base based on the target question; generate a target answer to the target question based on the reference question-answer pairs and the target question; and output the target answer through the client; the knowledge base is the knowledge base as described in any one of claims 1-8; the reference question-answer pairs include: a reference question and a reference answer.

12. A cloud service platform, characterized in that, The cloud service platform includes: the intelligent assistant as described in claim 10; The intelligent assistant is used to retrieve reference question-answer pairs from a knowledge base based on a target question; and to generate a target answer to the target question based on the reference question-answer pairs and the target question; the knowledge base is the knowledge base as described in any one of claims 1-8; the reference question-answer pairs include: a reference question and a reference answer; The target question is used to ask questions related to the cloud service platform; the reference question-and-answer pair is generated based on the target document; the target document is used to describe relevant knowledge about the cloud service platform.

13. A data processing apparatus, characterized in that, The device includes: A processing module is used to generate question-answer pairs based on a target document; the question-answer pairs include: questions that can be answered based on the target document, and answers corresponding to the questions; An update module is used to update the knowledge base based on the question-answer pair; the knowledge base is used to provide reference question-answer pairs for the target question to be answered; the target question and the reference question-answer pairs are used to generate the target answer for the target question.

14. An electronic device, characterized in that, include: Processor and memory; The processor is communicatively connected to the memory; The memory stores computer instructions; The processor executes computer instructions stored in the memory to implement the method as described in any one of claims 1-9.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-9.

16. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1-9.

Images