Data question and answer method and device, electronic equipment and storage medium

Abstract

The invention provides a data question and answer method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the steps of obtaining a to-be-processed problem, a target session graph of a target session to which the to-be-processed problem belongs and a dynamic index space of the target session; according to the target session graph and the dynamic index space, determining a target retrieval mode for retrieving the to-be-processed problem; and generating a target answer of the to-be-processed question based on the target retrieval mode. According to the embodiment of the invention, retrieval from a static global index space can be avoided when each to-be-processed problem is retrieved, and the target retrieval mode of the to-be-processed problem is determined in combination with the session context, the historical session interaction state and the dynamic index space of the to-be-processed problem; and the target answer of the to-be-processed question is generated based on the determined target retrieval mode, so that the accuracy of generating the target answer and the retrieval efficiency are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a data question-answering method and apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the continuous evolution of artificial intelligence technology, significant breakthroughs have been achieved in the field of natural language processing. Interactive and exploratory querying and analysis using natural language and structured databases has gradually become an important development direction for intelligent question answering. In this process, the rapid development of Large Language Models (LLMs) and Multimodal Large Models has provided a more powerful foundation for understanding and processing complex user intents, further promoting the practice and application of intelligent data access.

[0003] To effectively combine the generative capabilities of large-scale models with the accuracy of external knowledge sources, Retrieval-Augmented Generation (RAG) has been proposed and widely applied as a key technological paradigm. This paradigm typically transforms user-input natural language questions into retrieval instructions, retrieves relevant information fragments from a pre-built knowledge base as context, and then guides large models to generate structured query statements, natural language answers, or other forms of output, thereby improving the reliability and relevance of the response.

[0004] In database-oriented intelligent question-answering systems, when responding to user queries, the system often performs a retrieval operation independently based on the current single question, retrieving relevant information from a static global index, without considering the possible continuous conversational context or historical interaction state of the query. While this approach achieves a certain degree of integration between knowledge retrieval and model generation, providing a basic framework for natural language interaction with databases, the accuracy of the response data provided to users is difficult to guarantee. Summary of the Invention

[0005] To address this issue, the present invention provides a data question-answering method and apparatus, electronic device, and computer-readable storage medium to solve the problem of low accuracy of response data provided to users in the prior art due to the lack of consideration for continuous conversation context or historical interaction states.

[0006] To achieve the above objectives, the first aspect of the present invention provides a data question-answering method, the method comprising: acquiring a question to be processed, a target session graph of a target session to which the question to be processed belongs, and a dynamic index space of the target session; determining a target retrieval method for retrieving the question to be processed based on the target session graph and the dynamic index space; and generating a target answer for the question to be processed based on the target retrieval method.

[0007] Optionally, determining a target retrieval method for retrieving the question to be processed based on the target session graph and the dynamic index space includes: determining historical questions of the question to be processed in the target session graph and determining the semantic similarity between the question to be processed and the historical questions; determining the depth value and edge type of the question to be processed in the target session graph; determining the historical retrieval score of the historical answer corresponding to the historical question in the dynamic index space; and determining the target retrieval method for retrieving the question to be processed based on the semantic similarity, the depth value, the edge type, and the historical retrieval score.

[0008] Optionally, determining the target retrieval method for retrieving the problem to be processed based on the semantic similarity, the depth value, the edge type, and the historical retrieval score includes: constructing a first prompt word for determining the target retrieval method based on the semantic similarity, the depth value, the edge type, and the historical retrieval score; inputting the first prompt word into a retrieval prompt model to obtain the target retrieval method output by the retrieval prompt model.

[0009] Optionally, the target retrieval method includes dynamic retrieval; generating the target answer to the question to be processed based on the target retrieval method includes: performing the dynamic retrieval on the question to be processed to generate the target answer to the question to be processed; performing the dynamic retrieval on the question to be processed to generate the target answer to the question to be processed includes: encoding the question to be processed to obtain a question feature vector corresponding to the question to be processed; determining a first feature similarity between the question feature vector and a plurality of multimodal feature vectors stored in the dynamic index space; determining a preset number of target feature vectors from the plurality of multimodal feature vectors according to the plurality of first feature similarities and a preset number of answers; and decoding the preset number of target feature vectors to obtain the target answer corresponding to the preset number of target feature vectors.

[0010] Optionally, the target retrieval method includes multimodal retrieval; generating a target answer to the question to be processed based on the target retrieval method includes: performing the multimodal retrieval on the question to be processed to generate a target answer to the question to be processed; performing the multimodal retrieval on the question to be processed to generate a target answer to the question to be processed includes: constructing a second prompt word based on the question to be processed; inputting the second prompt word into a multimodal generation model to obtain a target answer output by the multimodal generation model, wherein the target answer includes any one of image, video, and interactive content.

[0011] Optionally, the target retrieval method includes global retrieval; generating the target answer to the question to be processed based on the target retrieval method includes: performing the global retrieval on the question to be processed to generate the target answer to the question to be processed; performing the global retrieval on the question to be processed to generate the target answer to the question to be processed includes: encoding the question to be processed to obtain a question feature vector corresponding to the question to be processed; determining a second feature similarity between the question feature vector and multiple feature vectors stored in a preset shared vector storage space; determining a preset number of target feature vectors from the multiple feature vectors based on the multiple second feature similarities and a preset number of answers; and decoding the preset number of target feature vectors to obtain the target answer corresponding to the preset number of target feature vectors.

[0012] Optionally, the method further includes: creating the dynamic index space when the first session question in the target session is obtained; adding the target answer to the dynamic index space when the target session is detected to be running; and destroying the dynamic index space when the target session is detected to have ended.

[0013] Secondly, the present invention provides a data question-answering device, comprising: an acquisition module configured to acquire a question to be processed, a target session graph of a target session to which the question to be processed belongs, and a dynamic index space of the target session; a determination module configured to determine a target retrieval method for searching the question to be processed based on the target session graph and the dynamic index space; and a generation module configured to generate a target answer for the question to be processed based on the target retrieval method.

[0014] Thirdly, the present invention provides an electronic device comprising: one or more processors; a memory storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-described data question-and-answer method; and one or more I / O interfaces connected between the processors and the memory, configured to enable information interaction between the processors and the memory.

[0015] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described data question-and-answer method.

[0016] The present invention has the following advantages:

[0017] The data question-answering method provided by this invention obtains the target session graph of the target session to which the question belongs, thereby understanding the current conversation context and historical session interaction status of the question. Each session also has a corresponding dynamic index space, which avoids searching from the static global index space when searching for each question, reducing search computation and improving search efficiency. By determining the target search method for the question based on the target session graph and dynamic index space, and performing the search based on the target search method, the target answer for the question is generated. This means that the search for the question is no longer limited to static global search, but rather combines the conversation context, historical session interaction status, and dynamic index space to determine the target search method for the question, and generates the target answer based on the determined target search method, improving the accuracy of the generated target answer and the search efficiency.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0020] Figure 1 A flowchart of a data question-answering method provided in an embodiment of the present invention;

[0021] Figure 2 A flowchart illustrating a data question-answering method provided in an embodiment of the present invention;

[0022] Figure 3 A block diagram of a data question-and-answer device provided in an embodiment of the present invention;

[0023] Figure 4 This is a block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0024] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0025] As used in this invention, the term "and / or" includes any and all combinations of one or more of the associated enumerated entries.

[0026] The terminology used in this invention is for describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms "a" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0027] When the terms “comprising” and / or “made of” are used in this invention, the presence of the said feature, integral, step, operation, element and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or groups thereof is not excluded.

[0028] Unless otherwise specified, all terms used in this invention (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in common dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined by the invention.

[0029] In a first aspect, embodiments of the present invention provide a data question-and-answer method.

[0030] See Figure 1 , Figure 1 A flowchart of a data question-answering method provided in an embodiment of the present invention specifically includes the following steps:

[0031] Step 102: Obtain the problem to be processed, the target session graph of the target session to which the problem to be processed belongs, and the dynamic index space of the target session.

[0032] In this context, the "question to be processed" refers to any question raised by the user in the target session. The question can be text, an image, or a video, or at least two of these three. The specific form of the question can be determined based on the actual application, and this invention does not impose any limitations on it. The "target session" refers to the session generated between the user and the intelligent question-answering system. It should be noted that a target session typically includes at least one round of conversation, but if the question to be processed is the first conversational question raised by the user, the target session is empty. The "target session graph" refers to the session context graph corresponding to the target session, including multiple entity nodes and multiple edges. In practical applications, the target session graph can be a tree structure or a network structure of a knowledge graph; in this embodiment of the invention, a tree structure is preferred. The target session graph is updated in real time based on the conversation flow between the user and the intelligent question-answering system, and can be used to characterize the progress, conversation context, and historical interaction status of the target session between the user and the intelligent question-answering system. The entity nodes in the target conversation graph include the questions and answers used in the conversational interaction between the user and the intelligent question-answering system. Edges represent the relationships between two entity nodes, including relationships between questions, between questions and answers, between answers, and the user's intent to interact with the intelligent question-answering system. It should be noted that the vector representation of each entity node in the target conversation graph is no longer a simple text embedding, but rather a context-fused representation of each entity node, as detailed in Formula 1 below:

[0033] Formula 1

[0034] in, For entity nodes The vector representation of , The textual semantic vector representation of questions and answers. For semantic vector representation of visualization types, A structured vector representation of the data range. Indicates a fully connected layer. .

[0035] A dynamic index space refers to a space created separately for each session to store the relevant data obtained from queries executed in each session. The dynamic index space can be a temporary vector index library, vector index table, etc., and its specific form can be determined according to the actual application; this invention does not limit it. The dynamic index space can be used to store feature vectors of relevant data during the execution of the target session. This relevant data includes query answers, search scores, and multimodal data uploaded by the user related to the target session (such as images, videos, charts, documents, etc.). In practical applications, a dynamic index space is created for each session.

[0036] In one specific embodiment of the present invention, the method further includes: upon obtaining the first session question in the target session, creating the dynamic index space; upon detecting that the target session is running, adding the target answer to the dynamic index space; and upon detecting that the target session has ended, destroying the dynamic index space.

[0037] Specifically, when the intelligent question-answering system receives a question to be processed, if there is no related session in the existing sessions, it indicates that the question to be processed is a new question posed by the user to the intelligent question-answering system, meaning it is the first session question in the target dialogue. At this point, a corresponding dynamic index space is created for the target session, and the question to be processed is encoded to obtain a question feature vector, which is a vectorized representation of the question. This question feature vector is stored in the dynamic index space. During the execution of the target session, the answers generated by the intelligent question-answering system for each round of the session are also vectorized, and the vectorized answers are added to the dynamic index space. Furthermore, upon detecting the end of the target session, the dynamic index space is destroyed to promptly release system resources and avoid resource congestion.

[0038] In addition, after obtaining the question to be processed, the intelligent question answering system will update the question to be processed to the target session graph, and after obtaining the target answer to the question to be processed, update the target answer to the target session graph, and determine the relationship between the question to be processed and the target answer, that is, generate the edge between the entity node of the question to be processed and the entity node of the target answer.

[0039] In practical applications, after acquiring a question to be processed, the intelligent question-answering system can determine the target session to which the question belongs among multiple existing sessions. If the target session to which the question belongs cannot be determined, it means that if the question is the first session question in a new session, then a corresponding target session graph and dynamic index space are generated for the question. If the target session to which the question belongs is determined among multiple existing sessions, the target session graph and dynamic index space corresponding to the target session are obtained. In subsequent processes, based on the target session graph and dynamic index space, the target retrieval method for searching the question is determined, and the target answer for the question is further determined.

[0040] In this embodiment of the invention, a corresponding conversation graph and dynamic index space are created for each conversation running in the intelligent question-answering system. During the subsequent retrieval of the question to be processed, the context of the question to be processed can be determined according to the conversation graph, and the relevant data of the target conversation to which the question to be processed belongs can be obtained according to the dynamic index space, thereby improving the accuracy of subsequent retrieval of the question to be processed and the generation of the target answer.

[0041] Step 104: Based on the target session graph and the dynamic index space, determine the target retrieval method for retrieving the question to be processed.

[0042] After obtaining the target session graph and dynamic index space corresponding to the target session to which the problem to be processed belongs, the target retrieval method for searching the problem to be processed can be determined based on the relevant data stored in the target session graph and dynamic index space. The target retrieval method refers to the final determined implementation method for searching the problem to be processed. In this embodiment of the invention, the retrieval methods include global retrieval, multimodal retrieval, and dynamic retrieval, and the target retrieval method is at least one of these three retrieval methods. Global retrieval refers to the method of searching within a preset shared vector storage space. In practical applications, a database is pre-set, storing a large amount of knowledge data, including but not limited to text, images, and videos. The knowledge data stored in the database is vectorized and then stored in the shared vector storage space. The shared vector storage space can be a globally shared vector index library, vector index table, etc. This invention does not limit the specific form of the shared vector storage space. Multimodal retrieval refers to a retrieval method used to generate target answers in multimodal data formats. That is, multimodal retrieval can be used to generate target answers in the form of images, videos, interactive content, etc. Interactive content refers to the generated target answer being an image or video, and containing elements that can be interacted with by the user, such as displaying a chart when the mouse hovers over it. Dynamic retrieval refers to a method of retrieving information from a dynamic index space.

[0043] The following explains the specific implementation of determining the target retrieval method.

[0044] In a specific embodiment of the present invention, determining a target retrieval method for retrieving the question to be processed based on the target session graph and the dynamic index space includes: determining historical questions of the question to be processed in the target session graph and determining the semantic similarity between the question to be processed and the historical questions; determining the depth value and edge type of the question to be processed in the target session graph; determining the historical retrieval score of the historical answer corresponding to the historical question in the dynamic index space; and determining the target retrieval method for retrieving the question to be processed based on the semantic similarity, the depth value, the edge type, and the historical retrieval score.

[0045] Here, "historical questions" refers to session questions preceding the question to be processed in the target session. There can be one or more historical questions, but each historical question must be at least the previous session question that includes the question to be processed. Semantic similarity measures the semantic similarity between the question to be processed and historical questions. Depth value refers to the depth value of the question to be processed within the tree-structured target session graph. Edge type refers to the type of edges associated with the question to be processed in the target session graph, including but not limited to drill-down, roll-up, perspective, switching visualization types, and requesting video generation. Historical answers are the answers generated by the intelligent question-answering system for historical questions. Historical retrieval score is the relevance score between the historical answer and the context of the historical question in the target session, used to measure the relevance between the historical answer and the context of the historical question in the target session.

[0046] Specifically, the historical questions of the problem to be processed are identified in the target session graph, and the semantic similarity between the problem to be processed and the historical questions is determined, such as by calculating the cosine similarity or Euclidean distance between them. The depth value of the problem to be processed in the target session graph and the edge type of the edges associated with it are determined, and the historical retrieval scores of the historical answers for the historical questions are obtained from the dynamic index space corresponding to the target session. Therefore, by combining the semantic similarity between the problem to be processed and the historical questions, the depth value of the problem to be processed in the target session graph, the edge type, and the historical retrieval scores of the historical answers, the target retrieval method for searching the problem to be processed is comprehensively determined, thereby improving the accuracy of determining the target retrieval method for searching the problem to be processed.

[0047] In practical applications, the model can be invoked to process the semantic similarity, depth value, edge type, and historical retrieval score mentioned above, and the target retrieval method for the problem to be processed can be determined through the model. The specific implementation method is as follows:

[0048] In a specific embodiment of the present invention, determining the target retrieval method for retrieving the problem to be processed based on the semantic similarity, the depth value, the edge type, and the historical retrieval score includes: constructing a first prompt word for determining the target retrieval method based on the semantic similarity, the depth value, the edge type, and the historical retrieval score; inputting the first prompt word into a retrieval prompt model to obtain the target retrieval method output by the retrieval prompt model.

[0049] Here, the first prompt term refers to the prompt term constructed to determine the target retrieval method. The retrieval prompt model refers to the model used to determine the target retrieval method for the question to be processed. In practical applications, the retrieval prompt model can be a large language model or other models, etc., depending on the actual application situation. This invention does not limit the type of retrieval prompt model.

[0050] Specifically, first suggestion terms are constructed based on semantic similarity, depth value, edge type, and historical retrieval score. These first suggestion terms are then input into the retrieval suggestion model, which analyzes and processes them to output a target retrieval method for the problem to be addressed. Furthermore, to improve the efficiency of first suggestion term construction, a suggestion term template can be pre-set. The determined semantic similarity, depth value, edge type, and historical retrieval score are then filled into the corresponding positions in the suggestion term template to generate the first suggestion terms.

[0051] This invention constructs a first suggestion term by determining the semantic similarity between the question to be processed and historical questions, the depth value and edge type of the question to be processed in the target session graph, and the historical retrieval score of the historical answers corresponding to the historical questions. The first suggestion term is then processed based on a retrieval suggestion model to obtain the target retrieval method output by the retrieval suggestion model, thereby improving the accuracy of determining the target retrieval method. Furthermore, in this invention, the retrieval method for the question to be processed is no longer limited to a single global retrieval method, further improving the accuracy of generating the target answer for the question to be processed based on the target retrieval method in subsequent processes.

[0052] Step 106: Generate the target answer to the question to be processed based on the target retrieval method.

[0053] As mentioned above, the retrieval methods include dynamic retrieval, multimodal retrieval, and global retrieval. The target retrieval method is at least one of dynamic retrieval, multimodal retrieval, and global retrieval. After determining the target retrieval method, the retrieval is performed according to the target retrieval method, and the target answer to the question to be processed is generated.

[0054] In one specific embodiment of the present invention, generating a target answer to the question to be processed based on the target retrieval method includes: performing the dynamic retrieval on the question to be processed to generate a target answer to the question to be processed.

[0055] Specifically, when the target retrieval method is determined to be dynamic retrieval, the problem to be processed is dynamically retrieved, and the target answer to the problem to be processed is generated.

[0056] Furthermore, in a specific embodiment provided by the present invention, performing the dynamic retrieval on the question to be processed to generate the target answer to the question to be processed includes: encoding the question to be processed to obtain a question feature vector corresponding to the question to be processed; determining a first feature similarity between the question feature vector and a plurality of multimodal feature vectors stored in the dynamic index space; determining a preset number of target feature vectors from the plurality of multimodal feature vectors based on the plurality of first feature similarities and a preset number of answers; and decoding the preset number of target feature vectors to obtain the target answer corresponding to the preset number of target feature vectors.

[0057] Here, the problem feature vector refers to the feature vector obtained by encoding the problem to be processed, that is, the vector representation of the problem to be processed; the multimodal feature vector refers to the feature vector of multimodal data stored in the dynamic index space, that is, the vector representation of multimodal data, including text, images, videos, etc. The multimodal feature vectors stored in the dynamic index space are a subset of the feature vectors in the shared vector storage space. First feature similarity refers to the similarity between the problem feature vector and the multimodal feature vectors in the dynamic index space. The preset number of answers is the number of target answers corresponding to the problem to be processed, pre-set by the user. The target feature vector refers to the feature vector with the highest first feature similarity among multiple multimodal feature vectors and the problem feature vector.

[0058] Specifically, the retrieval suggestion model determines the target retrieval method for the question to be processed based on the first suggestion word. When the retrieval suggestion model determines that the question to be processed needs further analysis based on the first suggestion word, it determines and outputs the target retrieval method as dynamic retrieval. When the retrieval method for the question to be processed is determined to be dynamic retrieval, an encoder encodes the question to be processed, obtaining the question feature vector corresponding to the question. Based on the question feature vector, a retrieval is performed in the dynamic index space of the target session to which the question belongs. The first feature similarity between the question feature vector and the multimodal feature vectors stored in the dynamic index space is determined. For example, cosine similarity, Euclidean distance, etc., can be calculated between the question feature vector and the multimodal feature vectors. Based on the first feature similarity corresponding to each multimodal feature vector, a preset number of target feature vectors with high first feature similarity are selected from the multiple multimodal feature vectors. A decoder then decodes the multiple target feature vectors to obtain the target answer corresponding to each target feature vector.

[0059] It should be noted that the multimodal feature vectors stored in the dynamic index space consist of feature vectors obtained from global retrieval and feature vectors obtained after vectorizing the answers generated by multimodal retrieval. Therefore, if the question to be processed is the first session question in the target session, the dynamic index space is empty, and the target retrieval method for retrieving the question to be processed will not be dynamic retrieval.

[0060] In this embodiment of the invention, during the execution of the target session, dynamic retrieval of the question to be processed is performed. This allows the retrieval of the question to be processed to be performed only in the dynamic index space, without having to perform retrieval in the shared vector storage space that stores a large number of multimodal feature vectors, which greatly reduces the computational load of the retrieval. Moreover, the multimodal feature vectors stored in the dynamic index space are all feature vectors of the answers corresponding to each session question in the target session, which have a high correlation with the question to be processed. Therefore, dynamic retrieval of the question to be processed based on the dynamic index space can improve the contextual relevance and accuracy of the target answer to the question to be processed.

[0061] In a specific embodiment of the present invention, generating a target answer to the question to be processed based on the target retrieval method includes: performing the multimodal retrieval on the question to be processed to generate a target answer to the question to be processed.

[0062] Specifically, when the target retrieval method is determined to be multimodal retrieval, multimodal retrieval is performed on the question to be processed, and the target answer to the question to be processed is generated.

[0063] Furthermore, in a specific embodiment provided by the present invention, performing multimodal retrieval on the question to be processed and generating a target answer to the question to be processed includes: constructing a second prompt word based on the question to be processed; inputting the second prompt word into a multimodal generation model to obtain a target answer output by the multimodal generation model, wherein the target answer includes any one of images, videos, and interactive content.

[0064] The second prompt word refers to the prompt word constructed to generate the target answer to the question to be processed. The multimodal generation model refers to the model used to generate the target answer to the question to be processed, and the target answer generated based on the multimodal generation model is an image, video, or interactive content.

[0065] Specifically, the retrieval suggestion model determines the target retrieval method for the question to be processed based on the first suggestion word. When the retrieval suggestion model determines that images, videos, or interactive content need to be generated based on the first suggestion word (e.g., the first suggestion word contains keywords such as "draw" or "generate video"), or when the multimodal target answer generated based on global retrieval and dynamic retrieval is inaccurate (e.g., the retrieval score is low), then the target retrieval method is determined and output as multimodal retrieval. When the target retrieval method for the question to be processed is determined to be multimodal retrieval, a second suggestion word is constructed based on the question to be processed. In practical applications, a template for constructing the second suggestion word can be pre-set, and the question to be processed is filled into the template to obtain the second suggestion word. The second suggestion word is then input into the multimodal generation model to obtain the target answer output by the multimodal generation model.

[0066] For example, for a problem related to "sales trend review", static images such as bar charts, line charts, pie charts, and scatter plots can be generated by constructing a second prompt word based on a multimodal generation model; a video script can be generated by constructing a second prompt word based on a multimodal generation model, and then narration audio can be generated using speech synthesis technology. The video script and narration audio can be synthesized to obtain an narration video related to the problem; images or videos with interactive elements (such as charts that display values ​​when hovered) can be generated by constructing a second prompt word based on a multimodal generation model.

[0067] In this embodiment of the invention, when the target retrieval method is determined to be multimodal retrieval, the target answer in a multimodal form required by the user can be generated by constructing a second prompt word and calling a multimodal generation model. This is no longer limited to a single structured data and improves the diversity of the generated target answer.

[0068] In a specific embodiment of the present invention, generating a target answer to the question to be processed based on the target retrieval method includes: performing the global retrieval on the question to be processed to generate a target answer to the question to be processed.

[0069] Specifically, when the target retrieval method is determined to be global retrieval, a global retrieval is performed on the question to be processed, and the target answer to the question to be processed is generated.

[0070] Furthermore, in a specific embodiment provided by the present invention, performing the global retrieval on the question to be processed to generate the target answer to the question to be processed includes: encoding the question to be processed to obtain a question feature vector corresponding to the question to be processed; determining a second feature similarity between the question feature vector and a plurality of feature vectors stored in a preset shared vector storage space; determining a preset number of target feature vectors from the plurality of feature vectors based on the plurality of second feature similarities and a preset number of answers; and decoding the preset number of target feature vectors to obtain the target answer corresponding to the preset number of target feature vectors.

[0071] The second feature similarity refers to the similarity between the question feature vector and the feature vectors in the shared vector storage space. Specifically, the retrieval suggestion model determines the target retrieval method for the question to be processed based on the first suggestion word. When the retrieval suggestion model determines that the question to be processed is a new session started by the user, or when dynamic retrieval of the question to be processed fails, the target retrieval method is determined and output as global retrieval. Similar to the dynamic retrieval method described above, the question to be processed is encoded by an encoder to obtain the question feature vector corresponding to the question to be processed, and a retrieval is performed in the shared vector storage space based on the question feature vector. The second feature similarity between the question feature vector and the feature vectors in the shared vector storage space is determined respectively. Based on the second feature similarity corresponding to each feature vector, target feature vectors with high second feature similarity of a preset number of answers are selected from multiple feature vectors. The decoder decodes multiple target feature vectors to obtain the target answer corresponding to each target feature vector.

[0072] In this embodiment of the invention, global search is used first only when a user starts a new session, and then global search is used when dynamic search fails, thereby reducing the computational load of searching for the problem to be processed and improving the efficiency and accuracy of searching for the problem to be processed.

[0073] It should be noted that, when the target retrieval method is determined to be global retrieval or multimodal retrieval, after obtaining the target answer corresponding to the question to be processed, it is also necessary to perform vectorization processing on the target answer and add the target feature vector corresponding to the target answer incrementally to the dynamic index space corresponding to the target session.

[0074] In practical applications, the target retrieval method can be at least two of dynamic retrieval, multimodal retrieval, and global retrieval. After obtaining the answer to the question to be processed according to different retrieval methods, all the answers obtained can be directly used as the target answer to the question to be processed. Alternatively, based on the multiple answers obtained, the relevance between each answer and the question to be processed can be further determined, and a preset number of answers can be selected from the multiple answers based on the relevance as the target answer to the question to be processed. The specific implementation method can be determined according to the actual application situation, and this invention does not limit it here.

[0075] Furthermore, CLIP (Contrastive Language-Image Pre-training) can be used as an encoder. The CLIP encoder encodes knowledge data in a pre-created database and matches corresponding text-image sample pairs through contrastive learning. Specifically, the similarity between sample pairs can be determined by calculating the cosine similarity between the sample pairs.

[0076] In addition, the intelligent question-answering system also possesses online learning capabilities. In practical applications, users can provide feedback on the target answer output by the intelligent question-answering system. Based on this feedback, the system can reduce the preference weight of the target retrieval method corresponding to the target answer, thereby optimizing itself. If an execution error occurs during the generation of the target answer to the question, the intelligent question-answering system can record the error and optimize itself based on this record.

[0077] This invention achieves the following: By acquiring the target session graph of the target session to which the question to be processed belongs, the current conversation context and historical session interaction status of the question to be processed can be known. Each session also corresponds to a dynamic index space, which avoids searching from the static global index space when searching for each question to be processed, reducing search computational power and improving search efficiency. By determining the target search method for the question to be processed based on the target session graph and dynamic index space, and performing a search based on the target search method, the target answer for the question to be processed is generated. This means that the search for the question to be processed is no longer limited to static global search, but rather combines the conversation context, historical session interaction status, and dynamic index space to determine the target search method for the question to be processed, and generates the target answer based on the determined target search method, improving the accuracy of the generated target answer and the search efficiency.

[0078] The following is in conjunction with the appendix Figure 2 The data question-answering method provided by this invention will be further explained and described below. Figure 2 A flowchart of a data question-answering method provided in an embodiment of the present invention is shown below. Figure 2 As shown, users can input questions to be processed on the user side, such as "Review of last quarter's sales trends." After receiving the user's input, the routing engine in the intelligent question-answering system determines the target session to which the question belongs within the existing sessions. If the target session to which the question belongs cannot be determined, it means that the question is from a new session initiated by the user, and in this case, a target session graph and dynamic index space corresponding to the question are created. If the target session to which the question belongs is determined, the target session graph and dynamic index space corresponding to the target session are obtained. Furthermore, the routing engine determines the historical questions, the depth value, and edge type of the question in the target session graph, and the historical answers and historical retrieval scores of the historical answers in the dynamic index space, thus determining the semantic similarity between the question and the historical questions. The semantic similarity between the question to be processed and historical questions, the depth value, edge type, and historical retrieval score of the question to be processed in the target session graph are filled into the preset prompt word template to generate the first prompt word. The first prompt word is then input into the retrieval prompt model to obtain the target retrieval method used to retrieve the question to be processed.

[0079] When the target retrieval method is global retrieval, the problem to be processed is encoded to obtain the problem feature vector corresponding to the problem to be processed. Based on the problem feature vector, a retrieval is performed in the shared vector storage space. Specifically, the similarity between the problem feature vector and multiple feature vectors in the shared vector storage space is determined. Based on the similarity corresponding to each feature vector, a preset number of target feature vectors are selected from the multiple feature vectors. The incremental target feature vectors are merged into the dynamic index space of the target session. The target feature vectors are decoded to obtain the target answer corresponding to each target feature vector. The target answer is then output to the user terminal through the routing engine. The target answer can be text, video, or image, etc.

[0080] When the target retrieval method is multimodal retrieval, a second prompt word is constructed based on the question to be processed. This second prompt word is input into the multimodal generation model to obtain the target answer output by the multimodal generation model. The target answer is then output to the user terminal via the routing engine. The target answer can be a video, an image, or an image or video with interactive elements, etc. The target answer is encoded to obtain the target feature vector of the target answer, and the incremental target feature vector is merged into the dynamic index space of the target session.

[0081] When the target retrieval method is dynamic retrieval, the problem to be processed is encoded to obtain the problem feature vector corresponding to the problem to be processed. Based on the problem feature vector, a retrieval is performed in the dynamic index space. Specifically, the similarity between the problem feature vector and multiple multimodal feature vectors in the dynamic index space is determined. Based on the similarity corresponding to each multimodal feature vector, a preset number of target feature vectors are selected from the multiple multimodal feature vectors. The target feature vectors are decoded to obtain the target answer corresponding to each target feature vector. The target answer is then output to the user terminal through the routing engine. The target answer can be text, video, or image, etc.

[0082] It should be noted that the answers generated for each session question are also dynamically updated in the target session graph. After the target session ends, the dynamic index space of the target session is destroyed to release system resources in a timely manner.

[0083] This invention achieves the following: by obtaining the target session graph of the target session to which the question to be processed belongs, the current conversation context and historical session interaction status of the question to be processed can be known. At the same time, each session also corresponds to a dynamic index space, which can avoid searching from the static global index space when searching for each question to be processed, reducing search computing power and improving search efficiency. By determining the target search method for searching the question to be processed based on the target session graph and dynamic index space, the accuracy of determining the target search method is improved, further ensuring the accuracy of the generated target answer.

[0084] By dynamically retrieving questions to be processed, it is not necessary to search for each question in the shared vector storage space, thus reducing the workload and computational resources of retrieval; by performing multimodal retrieval of questions to be processed, it provides users with diverse target answers, improving the user experience; by performing global retrieval of questions to be processed, it ensures the successful completion of the retrieval of questions to be processed even if dynamic retrieval or multimodal retrieval fails, thus ensuring the versatility and robustness of the intelligent question answering system.

[0085] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0086] Secondly, embodiments of the present invention provide a data question-and-answer device.

[0087] The acquisition module 302 is configured to acquire the problem to be processed, the target session graph of the target session to which the problem to be processed belongs, and the dynamic index space of the target session;

[0088] The determination module 304 is configured to determine the target retrieval method for retrieving the question to be processed based on the target session graph and the dynamic index space.

[0089] The generation module 306 is configured to generate the target answer to the question to be processed based on the target retrieval method.

[0090] Optionally, the determining module 304 is further configured to:

[0091] In the target session graph, identify the historical questions of the question to be processed and determine the semantic similarity between the question to be processed and the historical questions;

[0092] Determine the depth value and edge type of the problem to be processed in the target session graph;

[0093] Determine the historical retrieval score of the historical answer corresponding to the historical question in the dynamic index space;

[0094] Based on the semantic similarity, the depth value, the edge type, and the historical retrieval score, a target retrieval method is determined for retrieving the problem to be processed.

[0095] Optionally, the determining module 304 is further configured to:

[0096] Based on the semantic similarity, the depth value, the edge type, and the historical retrieval score, a first prompt word is constructed to determine the target retrieval method;

[0097] Input the first suggestion word into the retrieval suggestion model to obtain the target retrieval method output by the retrieval suggestion model.

[0098] Optionally, the target retrieval method includes dynamic retrieval;

[0099] The generation module 306 is further configured to:

[0100] The dynamic retrieval is performed on the problem to be processed to generate the target answer to the problem to be processed;

[0101] The generation module 306 is further configured to:

[0102] The problem to be processed is encoded to obtain the problem feature vector corresponding to the problem to be processed;

[0103] Determine the first feature similarity between the problem feature vector and the multiple multimodal feature vectors stored in the dynamic index space;

[0104] Based on multiple first feature similarities and a preset number of answers, determine the preset number of target feature vectors from the multiple multimodal feature vectors;

[0105] The target feature vectors corresponding to the preset number of target feature vectors are decoded respectively to obtain the target answers.

[0106] Optionally, the target retrieval method includes multimodal retrieval;

[0107] The generation module 306 is further configured to:

[0108] The multimodal retrieval is performed on the problem to be processed to generate the target answer to the problem to be processed;

[0109] The generation module 306 is further configured to:

[0110] A second prompt word is constructed based on the problem to be solved;

[0111] The second prompt word is input into the multimodal generation model to obtain the target answer output by the multimodal generation model, wherein the target answer includes any one of image, video and interactive content.

[0112] Optionally, the target retrieval method includes global retrieval;

[0113] The generation module 306 is further configured to:

[0114] The global search is performed on the problem to be processed to generate the target answer to the problem to be processed;

[0115] The generation module 306 is further configured to:

[0116] The problem to be processed is encoded to obtain the problem feature vector corresponding to the problem to be processed;

[0117] Determine the second feature similarity between the problem feature vector and multiple feature vectors stored in the preset shared vector storage space;

[0118] Based on multiple second feature similarities and a preset number of answers, determine the preset number of target feature vectors from the multiple feature vectors;

[0119] The target feature vectors corresponding to the preset number of target feature vectors are decoded respectively to obtain the target answers.

[0120] Optionally, the device further includes:

[0121] A module is configured to create the dynamic index space upon receiving the first session question in the target session;

[0122] An add module is configured to add the target answer to the dynamic index space when the target session is detected to be running.

[0123] The destruction module is configured to destroy the dynamic index space when the target session is detected to have ended.

[0124] This invention achieves the following: By acquiring the target session graph of the target session to which the question to be processed belongs, the current conversation context and historical session interaction status of the question to be processed can be known. Each session also corresponds to a dynamic index space, which avoids searching from the static global index space when searching for each question to be processed, reducing search computational power and improving search efficiency. By determining the target search method for the question to be processed based on the target session graph and dynamic index space, and performing a search based on the target search method, the target answer for the question to be processed is generated. This means that the search for the question to be processed is no longer limited to static global search, but rather combines the conversation context, historical session interaction status, and dynamic index space to determine the target search method for the question to be processed, and generates the target answer based on the determined target search method, improving the accuracy of the generated target answer and the search efficiency.

[0125] The apparatus provided in the embodiments of the present invention has functions or includes modules that can be used to perform the methods described in the first aspect of the method embodiments above. The specific implementation and technical effects can be referred to the description of the method embodiments above. For the sake of brevity, they will not be repeated here.

[0126] It should be noted that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.

[0127] See Figure 4 This invention provides an electronic device comprising:

[0128] One or more processors 401;

[0129] The memory 402 stores one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the above-described data question-and-answer method.

[0130] One or more I / O interfaces 403 are connected between the processor and the memory and configured to enable information exchange between the processor and the memory.

[0131] Among them, processor 401 is a device with data processing capabilities, including but not limited to central processing unit (CPU); memory 402 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); I / O interface (read-write interface) 403 is connected between processor 401 and memory 402, and can realize information interaction between processor 401 and memory 402, including but not limited to data bus (Bus).

[0132] In some embodiments, the processor 401, memory 402, and I / O interface 403 are interconnected via a bus, and thus connected to other components of the computing device.

[0133] This embodiment also provides a computer-readable medium storing a computer program thereon. When the program is executed by a processor, it implements the data question-and-answer method provided in this embodiment. To avoid repetition, the specific steps of the data question-and-answer method will not be repeated here.

[0134] This invention also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described data question-and-answer method.

[0135] Those skilled in the art will understand that all or some of the steps, systems, or apparatuses in the methods, systems, and apparatuses described above can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0136] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0137] Those skilled in the art will understand that although some embodiments described herein include certain features that are included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this embodiment and form different embodiments.

[0138] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A data question answering method, characterized in that, include: Obtain the problem to be processed, the target session graph of the target session to which the problem to be processed belongs, and the dynamic index space of the target session; Based on the target session graph and the dynamic index space, determine the target retrieval method for retrieving the question to be processed; The target answer to the question to be processed is generated based on the target retrieval method described above.

2. The method as described in claim 1, characterized in that, Based on the target session graph and the dynamic index space, a target retrieval method for searching the question to be processed is determined, including: In the target session graph, identify the historical questions of the question to be processed and determine the semantic similarity between the question to be processed and the historical questions; Determine the depth value and edge type of the problem to be processed in the target session graph; Determine the historical retrieval score of the historical answer corresponding to the historical question in the dynamic index space; Based on the semantic similarity, the depth value, the edge type, and the historical retrieval score, a target retrieval method is determined for retrieving the problem to be processed.

3. The method as described in claim 2, characterized in that, Based on the semantic similarity, the depth value, the edge type, and the historical retrieval score, a target retrieval method is determined for retrieving the problem to be processed, including: Based on the semantic similarity, the depth value, the edge type, and the historical retrieval score, a first prompt word is constructed to determine the target retrieval method; Input the first suggestion word into the retrieval suggestion model to obtain the target retrieval method output by the retrieval suggestion model.

4. The method according to any one of claims 1-3, characterized in that, The target retrieval method includes dynamic retrieval; Based on the target retrieval method, the target answer to the question to be processed is generated, including: The dynamic retrieval is performed on the problem to be processed to generate the target answer to the problem to be processed; The dynamic retrieval of the problem to be processed generates the target answer to the problem to be processed, including: The problem to be processed is encoded to obtain the problem feature vector corresponding to the problem to be processed; Determine the first feature similarity between the problem feature vector and the multiple multimodal feature vectors stored in the dynamic index space; Based on multiple first feature similarities and a preset number of answers, determine the preset number of target feature vectors from the multiple multimodal feature vectors; The target feature vectors corresponding to the preset number of target feature vectors are decoded respectively to obtain the target answers.

5. The method according to any one of claims 1-3, characterized in that, The target retrieval method includes multimodal retrieval; Based on the target retrieval method, the target answer to the question to be processed is generated, including: The multimodal retrieval is performed on the problem to be processed to generate the target answer to the problem to be processed; Performing the multimodal retrieval on the question to be processed to generate the target answer to the question to be processed includes: A second prompt word is constructed based on the problem to be solved; The second prompt word is input into the multimodal generation model to obtain the target answer output by the multimodal generation model, wherein the target answer includes any one of image, video and interactive content.

6. The method according to any one of claims 1-3, characterized in that, The target retrieval method includes global retrieval; Based on the target retrieval method, the target answer to the question to be processed is generated, including: The global search is performed on the problem to be processed to generate the target answer to the problem to be processed; The global search is performed on the question to be processed to generate the target answer to the question to be processed, including: The problem to be processed is encoded to obtain the problem feature vector corresponding to the problem to be processed; Determine the second feature similarity between the problem feature vector and multiple feature vectors stored in the preset shared vector storage space; Based on multiple second feature similarities and a preset number of answers, determine the preset number of target feature vectors from the multiple feature vectors; The target feature vectors corresponding to the preset number of target feature vectors are decoded respectively to obtain the target answers.

7. The method according to any one of claims 1-3, characterized in that, The method further includes: Upon obtaining the first session problem in the target session, the dynamic index space is created; During the detection of the target session running, the target answer is added to the dynamic index space; If the target session is detected to have ended, the dynamic index space is destroyed.

8. A data question-and-answer device, characterized in that, include: The acquisition module is configured to acquire the problem to be processed, the target session graph of the target session to which the problem to be processed belongs, and the dynamic index space of the target session; The determination module is configured to determine the target retrieval method for searching the question to be processed based on the target session graph and the dynamic index space. The generation module is configured to generate the target answer to the question to be processed based on the target retrieval method.

9. An electronic device, characterized in that, include: One or more processors; A memory having stored one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method according to any one of claims 1-7; One or more I / O interfaces are connected between the processor and the memory and configured to enable information interaction between the processor and the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images