Generative question answering method and device based on AI

Abstract

The invention discloses a generative question answering method and device based on AI. The method comprises the steps that a multi-mode question and answer request input by a user is received, the multi-mode question and answer request is converted into a text description language, and the multi-mode question and answer request comprises at least one of a text, voice, an image and a video; analyzing the text description language and generating corresponding intention data, and retrieving and acquiring corresponding preliminary response information in a knowledge base based on the intention data; and dynamically correcting the preliminary response information according to a preset rule to generate corrected response information. The technical problem of inaccurate question and answer response generated in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to an AI-based generative question-answering method and device. Background Art

[0002] With the rapid development of artificial intelligence technology, generative question-answering systems are increasingly used in various fields. Existing generative question-answering systems usually receive users' question-answering requests through a single modality (such as text or voice) and generate corresponding responses based on pre-set rules or models. However, due to the diversity and complexity of user inputs and the limitations of question-answering systems in processing multimodal information, the generated responses often lack accuracy and flexibility. In addition, current question-answering systems often fail to fully understand users' intentions when faced with multimodal inputs, which results in the retrieved answers deviating from user needs and reducing user experience.

[0003] Especially when dealing with multimodal data including voice, images, and videos, traditional question-answering systems lack the ability to uniformly parse data of different modalities, and it is difficult to efficiently convert and process multimodal information through natural language. Therefore, how to generate accurate answers that meet user needs under multimodal input has become a difficult problem in existing technologies.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide an AI-based generative question-answering method and device to at least solve the technical problem of inaccurate question-answering responses generated in the prior art.

[0006] According to one aspect of an embodiment of the present invention, there is provided an AI-based generative question-answering method, comprising: receiving a multimodal question-answering request input by a user, converting the multimodal question-answering request into a text description language, wherein the multimodal question-answering request comprises at least one of the following: text, voice, image, and video; parsing the text description language and generating corresponding intent data, and based on the intent data, retrieving and acquiring corresponding preliminary response information in a knowledge base; and dynamically correcting the preliminary response information according to preset rules to generate corrected response information.

[0007] According to another aspect of an embodiment of the present invention, there is also provided an AI-based generative question-and-answer device, comprising: a conversion module, configured to receive a multimodal question-and-answer request input by a user, and convert the multimodal question-and-answer request into a text description language, wherein the multimodal question-and-answer request comprises at least one of the following: text, voice, image, and video; a parsing module, configured to parse the text description language and generate corresponding intent data, and based on the intent data, retrieve and obtain corresponding preliminary response information from a knowledge base; and a correction module, configured to dynamically correct the preliminary response information according to preset rules or user feedback, and generate corrected response information.

[0008] In an embodiment of the present invention, a multimodal question and answer request input by a user is received, and the multimodal question and answer request is converted into a text description language, wherein the multimodal question and answer request includes at least one of the following: text, voice, image, and video; the text description language is parsed and corresponding intent data is generated, and based on the intent data, corresponding preliminary response information is retrieved and obtained in the knowledge base; the preliminary response information is dynamically corrected according to preset rules to generate corrected response information. Through the above scheme, the technical problem of inaccurate question and answer responses generated in the prior art is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0010] Figure 1 is a flow chart of an AI-based generative question answering method according to an embodiment of the present invention;

[0011] Figure 2 is a flow chart of another AI-based generative question answering method according to an embodiment of the present invention;

[0012] Figure 3 is a flow chart of a method for converting a user's voice input into a text description language in real time through a voice recognition model according to an embodiment of the present invention;

[0013] Figure 4 is a flowchart of a method for parsing a text description language and generating corresponding intent data according to an embodiment of the present invention;

[0014] Figure 5 is a flowchart of a method for dynamically modifying the preliminary response information according to a preset rule according to an embodiment of the present invention;

[0015] Figure 6 is a flow chart of a method for converting an image into a text description language according to an embodiment of the present invention;

[0016] Figure 7 is a schematic diagram of the structure of an AI-based generative question-answering device according to an embodiment of the present invention;

[0017] Figure 8 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0019] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0020] According to an embodiment of the present invention, a method embodiment of an AI-based generative question answering method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0021] Generative Question Answering is a question answering system that uses artificial intelligence models (usually deep learning models) to generate natural language responses. Unlike traditional question answering systems (such as retrieval-based methods), generative question answering systems do not just select from existing answers, but generate new, natural language responses based on the user's questions.

[0022] Artificial Intelligence (AI) is a branch of computer science that focuses on developing systems that can simulate or perform intelligent human behavior. The goal of AI is to enable machines to learn, reason, understand language, solve problems, and make decisions. The following are some key concepts and areas of artificial intelligence.

[0023] Multimodality, in the field of artificial intelligence and computer science, usually involves different types of data input, such as text, voice, image, video, etc. Multimodal technology aims to combine these different inputs to improve the system's understanding and responsiveness.

[0024] Figure 1 is a generative question answering method based on AI according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0025] Step S102: receiving a multimodal question and answer request input by a user, and converting the multimodal question and answer request into a text description language, wherein the multimodal question and answer request includes at least one of the following: text, voice, image, and video.

[0026] In some embodiments, the user's voice input is converted into the text description language in real time through a speech recognition model. For example, the speech input is nonlinearly mapped layer by layer using the speech recognition model to obtain the result of the nonlinear mapping; the probability value of the category of the speech unit in the speech input is calculated based on the result of the nonlinear mapping, and the probability distribution of the speech unit is determined based on the probability value; the category of the speech unit is determined based on the probability distribution, and the speech input is converted into the text description language based on the determined category. This embodiment performs nonlinear mapping of the speech signal layer by layer through the speech recognition model, ensuring the accurate capture and analysis of complex speech features. The method of probability value calculation and category determination makes the recognition of speech units more accurate, thereby improving the accuracy and fluency of text conversion. The real-time conversion in this application not only improves the user experience, but also lays a solid foundation for the efficient response of the multimodal question-and-answer system, and promotes the naturalness and intelligence of human-computer interaction.

[0027] In some embodiments, images uploaded by users are analyzed through image recognition and object detection algorithms, key information is extracted and the corresponding text description language is generated. This embodiment not only improves the efficiency of information extraction, but also ensures accurate understanding of the image content. By converting visual information into readable text descriptions, users can more conveniently obtain and share information, thereby enhancing the interactive experience. In addition, this method provides rich contextual support for the multimodal question-answering system, improving the intelligence level and responsiveness of the system.

[0028] In some embodiments, video analysis technology is used to parse the video frames in the video uploaded by the user frame by frame to extract visual information, and speech recognition technology is used to process the audio content in the video, and the video as a whole is converted into the text description language based on the visual information and the audio content. This application not only improves the ability to understand complex information, but also ensures the richness and accuracy of the text description. Users can more easily obtain key details of the video content, which enhances the accessibility and shareability of information. At the same time, this technology provides more comprehensive contextual support for the multimodal question-and-answer system, significantly improving the system's intelligent interaction capabilities and user experience.

[0029] Step S104, parse the text description language and generate corresponding intent data, and based on the intent data, retrieve and obtain corresponding preliminary response information in the knowledge base.

[0030] First, generate intent data. Use a natural language processing tool to segment the text description language, and perform part-of-speech tagging on the segmented results; extract important entities in the text through a named entity recognition model, and mark the identified important entities as key features; based on the key features, combined with the user's historical query records, current session context and user portraits, use a long short-term memory network or a transformer model to process the context data of the text description language to identify the intent data. This application uses a natural language processing tool to segment and tag the text, making the text structure clearer. Then, important entities are extracted through a named entity recognition model and marked as key features to highlight key information. In addition, combined with the user's historical query records, current session context and user portraits, the context data of the text is processed through a long short-term memory network or a transformer model, which can accurately identify the user's intent. In summary, this application not only improves the accuracy of intent recognition, but also enhances the system's responsiveness to user needs, thereby optimizing user experience and interaction efficiency.

[0031] Next, generate preliminary response information. Parse the intent data to extract key search terms; based on the key search terms, retrieve and obtain corresponding preliminary response information from the knowledge base. By accurately identifying the needs of users, the system can quickly retrieve information related to key search terms from the knowledge base to ensure the relevance and accuracy of the response. This application not only improves the efficiency of information retrieval, but also optimizes the user's feedback experience, allowing them to obtain the required answers or suggestions in a timely manner.

[0032] Step S106: dynamically modify the preliminary response information according to preset rules to generate modified response information.

[0033] In the case of detecting inappropriate or inaccurate original words in the preliminary response information, the original words are replaced with synonyms or professional terms in the knowledge base; and the format of the preliminary response information is adjusted according to the user's preference or response format requirements. This solution significantly improves the accuracy and professionalism of the response by intelligently replacing inappropriate or inaccurate words in the preliminary response information. The use of synonyms or professional terms in the knowledge base ensures the clarity and authority of information delivery. In addition, the format of the information is adjusted according to the user's preference and response format requirements, which enhances the user experience and makes the response more in line with the user's expectations.

[0034] Figure 2 is a flow chart of another AI-based generative question answering method according to an embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:

[0035] Step S202: convert the multimodal question and answer request into a text description language.

[0036] In the embodiment of the present application, the multimodal question and answer request is a voice input, and the user's voice input is converted into a text description language in real time through a voice recognition model, such as Figure 3 As shown, the conversion method includes the following steps:

[0037] Step S2022, classifying the speech units of the speech input based on the probability distribution.

[0038] The classifier consists of a multi-layer neural network, and the neurons in each layer process the output features passed from the previous layer. The input fusion features are first transformed linearly to generate intermediate results in each layer. The weight matrix and bias vector linearly map the features and convert them to higher dimensions or different feature spaces. Subsequently, the linear results are processed nonlinearly through activation functions, so that the model can capture complex patterns and feature relationships and enhance the expressiveness of features. As the number of network layers increases, this layer-by-layer nonlinear mapping can mine deeper semantic information and further strengthen the correlation between the fusion features and the speech unit categories.

[0039] For example, in the linear transformation process of each layer, the corresponding intermediate results are generated by combining the index of the current network layer, the number of multi-scale features (used to capture information at different levels), the weight matrix, the output features of the previous layer (i.e., the input of the current layer), and the bias vector of the current scale. After that, the activation function performs nonlinear processing on these intermediate results to obtain the nonlinear mapping output of each layer. Through multi-scale feature mapping, layer-by-layer linear transformation and nonlinear activation, the present application can extract rich feature information from different levels and capture feature patterns in local and global ranges. In addition, the linear transformation flexibly adjusts the input features through the weight matrix and bias vector, and the activation function introduces nonlinearity, thereby enhancing the classifier's ability to learn complex patterns and features. Ultimately, the result of the nonlinear mapping allows the network to understand the features more deeply and improve the overall expression ability and classification effect.

[0040] In the last layer of the network, the features processed by multiple layers are mapped to the output space, corresponding to different categories of speech units. Each category will receive a score indicating the relevance of the input feature to the category. These scores are converted into probability values ​​through the Softmax activation function. For example, the weight matrix of the last layer of the network maps the high-level semantic information extracted by the previous layer to the category space, and adjusts the score in combination with the bias vector to make it more suitable for the specific classification task. This application converts complex feature information into scores for each category by utilizing the weight matrix and bias vector of the last layer, generating linear outputs related to each category. The Softmax function normalizes these scores to generate probability values ​​between 0 and 1, and ensures that the sum of the probabilities of all categories is 1.

[0041] Through the Softmax function, the probability value of each category represents the confidence of the classifier that the input feature belongs to that category. In addition, the introduction of the category correlation weight matrix and the temperature parameter can further improve the performance of the model. The correlation weight matrix captures the dependencies between different categories, and the temperature parameter controls the smoothness of the output distribution, making the model more robust during training. When the temperature is higher, the probability distribution is smoother, avoiding premature focus on a certain category; when the temperature is lower, the output of the classifier is clearer, improving the decision accuracy. In this way, through the Softmax function and the correlation weight matrix, the model can generate a reasonable and balanced probability distribution to ensure more accurate classification decisions.

[0042] After generating the category probability distribution, the system will use the maximum likelihood method to classify the speech units and select the category with the highest probability as the final classification result. Specifically, the classifier scans the probability distribution of the Softmax output and selects the category corresponding to the maximum probability value as the prediction result. This application uses the maximum likelihood method to ensure that the classifier can make the best choice from multiple possible categories. For example, based on the probability distribution of the Softmax output, the classifier will select the most likely category as the final classification result.

[0043] Through this process, the classifier processes the input features layer by layer, generates the probability distribution of each category, and achieves accurate classification of speech units by maximizing the likelihood method. This not only extracts effective information from the input data, but also makes the best classification decision in the complex feature representation.

[0044] Step S2024: Perform speech recognition on the speech input according to the classification result of the speech unit to obtain a text description language.

[0045] After completing the classification of speech units, the system will further analyze the sequence structure of the speech input based on the probability distribution results generated by the classifier. Specifically, the system will use the probability values ​​corresponding to each previously classified speech unit, and through sequence matching technologies such as dynamic time warping (DTW), match the classification results with the known vocabulary or language model to identify the complete text description language.

[0046] First, the system will perform contextual analysis on adjacent speech units based on the classification results of speech units to ensure the continuity and consistency of recognition. This process is optimized through the grammatical rules and semantic associations of the language model. For example, the model can reasonably correct and complete adjacent speech units based on factors such as sentence structure and word frequency, and finally obtain text output that conforms to natural language expression.

[0047] During this process, the system will also combine the prediction mechanism of the language model to infer and supplement the ambiguous or incomplete parts of the voice input according to the context. In addition, the system may further use post-processing mechanisms, such as spelling correction and grammar correction, to ensure that the final generated text description language is accurate. Post-processing technology is based on a vocabulary or external language database to improve the overall text accuracy and readability.

[0048] Finally, after this step, the voice input is converted into a standardized text description language for use in the subsequent generative question-answering module.

[0049] Step S204, parse the text description language and generate corresponding intent data.

[0050] like Figure 4As shown, the method for parsing the text description language and generating corresponding intent data includes the following steps:

[0051] Step S2042: word segmentation and part-of-speech tagging of the text description language.

[0052] 1) Participle.

[0053] Word segmentation is the process of splitting continuous text description language into meaningful vocabulary units. Since many languages ​​(such as Chinese) do not have clear word boundaries, the system uses rule-based or machine learning word segmentation tools (such as Jieba word segmentation, SpaCy, etc.) to segment the text. Specifically, by inputting the text description language into the word segmentation model, the model identifies complete vocabulary units based on the dictionary and language rules accumulated during the training process. For example, the input "I have a cat on the sofa" will be segmented into independent word units such as "I", "have", "one", "cat", "in", "sofa", and "on".

[0054] 2) Part-of-speech tagging.

[0055] After the word segmentation is completed, the segmented words are tagged with parts of speech, that is, each word is assigned its corresponding grammatical category (such as noun, verb, adjective, etc.). The present embodiment adopts a part-of-speech tagging tool (such as an HMM model or a CRF model) to mark each word in combination with the grammatical structure of the context. Specifically, the word segmentation result is processed using a part-of-speech tagging model, such as marking "cat" as a noun (N) and "have" as a verb (V), thereby providing grammatical and semantic information for subsequent processing. This annotation not only helps the system understand the role of each word, but also provides a basis for named entity recognition and intent understanding.

[0056] Step S2044, named entity recognition.

[0057] 1) Entity extraction.

[0058] Named Entity Recognition (NER) is the process of extracting entities of a specific category (such as names of people, places, organizations, etc.) from text. The system uses a pre-trained NER model (such as BERT-BiLSTM-CRF, etc.) to further analyze the text after word segmentation and part-of-speech tagging to identify important entities. Specifically, for example, in the sentence "The user uploaded a photo of a cat", the NER model can identify "cat" as the "animal" class in the entity category and mark it as a key feature. The system labels each identified entity with a category and provides input for subsequent feature processing.

[0059] 2) Entities are marked as key features.

[0060] The identified important entities (e.g., "cat", "photo") are labeled as key features by the system for subsequent context understanding and user intent recognition. The labeling of key features is based on the importance of the entity and its relevance in the current query or session. For example, animal names can be labeled as key features in some image classification scenarios.

[0061] Step S2046, identifying intent by combining historical records with context.

[0062] 1) Contextual data processing.

[0063] After extracting key features from the text description, the system combines these features with the user's historical query records, the context of the current session, and the user profile. The user profile is a model built based on the user's past behaviors, interests, habits, and personalized information, aiming to better understand the user's current needs and preferences. Specifically, the system analyzes the user's historical query records, such as previous image or text query information, historical session content, and preferences inferred from the user profile. By integrating this data, the system can build a richer contextual environment to provide data support for intent recognition.

[0064] 2) Identify intent.

[0065] In order to process complex contextual data and identify the user's intention, a speech intent parsing model composed of a long short-term memory network (LSTM) and a transformer model (such as BERT, GPT, etc.) can be used. This model processes sequence data, can capture long-distance dependencies, and performs deep semantic analysis in combination with contextual information. Specifically, LSTM can be used to process data with time dependencies, such as the user's query history or contextual data in continuous sessions. LSTM can save contextual information over a long span of time through its internal memory units, helping to identify the user's intention in the current query. When processing large-scale data or more complex contexts, the system can use a transformer model (such as BERT, GPT), which can efficiently understand the meaning of each word in the text in different contexts through the self-attention mechanism. The system inputs the text description language and its key features, and the model can identify the user's intention data based on the context. For example, the system can recognize that the "cat" in the image uploaded by the user is associated with the "pet" in his historical query, and judge that the user may be looking for relevant information about pet care. Based on the results of the LSTM or transformer model analysis, the system identifies the user's current query intention. For example, by identifying the key entity "cat" in the text and combining it with the historical query record "pet supplies", the system may infer that the user's intention is to buy pet supplies. The system then generates a corresponding response or recommendation based on this intention.

[0066] The training process of the speech intention analysis model is as follows: First, the speech intention analysis model is used to extract the primary embedding vector group of the speech recognition text and the auxiliary embedding vector group of the manual recognition text. For the speech recognition text, the model generates its feature representation, namely the primary embedding vector group, which captures the characteristics of the speech text in the multi-dimensional semantic space. Similarly, for the manual recognition text, the model generates an auxiliary embedding vector group to represent its semantic information. By extracting these embedding vectors, the model can capture the intrinsic characteristics of the text in the high-dimensional semantic space, providing a basis for subsequent processing steps. Then, according to the preset expansion rank criterion, the primary embedding vector group and the auxiliary embedding vector group are respectively multi-dimensionally upgraded. Subsequently, a multi-channel convolution kernel is used to couple and fuse the features of the vector group to obtain the coupled primary embedding group and the auxiliary embedding group. Next, a nonlinear activation function is applied to further enhance the richness of the semantic representation, and the activated primary embedding group and the auxiliary embedding group are obtained. Finally, the activated primary embedding group and the auxiliary embedding group are normalized to ensure the scale consistency of the data, and finally a multi-dimensional feature matrix is ​​obtained to form the subsequent optimization input.

[0067] Subsequently, the primary feature matrix and the auxiliary feature matrix are subjected to high-order matrix decomposition respectively. First, these matrices are expanded along the direction of the expansion dimension to form multiple sub-matrices. Each sub-matrix is ​​subjected to singular value decomposition to extract the core subspace representation and the factor matrix. The core subspace representation condenses the key information of the matrix, and the factor matrix captures the local structure of its features. Next, the principal component difference between the primary feature matrix and the auxiliary feature matrix is ​​calculated. By measuring the similarity of the core subspace representation and the difference of the principal components in the factor matrix, the subtle changes between the two sets of data can be accurately captured. This allows a more accurate comparison of the differences between speech recognition text and manual recognition text. Finally, the loss function is updated based on the difference between the two to obtain a trained speech intent parsing model.

[0068] This application can extract key information and identify the user's intention from the text description language generated from the images uploaded by the user. The system combines word segmentation, part-of-speech tagging, named entity recognition, and context processing technology to ensure that the user's needs can be accurately captured and the corresponding question-answering or recommendation results can be generated. The solution provided by this application has broad application prospects in the fields of intelligent question-answering systems, virtual assistants, and personalized recommendation systems.

[0069] Step S206: based on the intention data, retrieve and obtain corresponding preliminary response information in the knowledge base.

[0070] In order to convert user intent into an actionable search request, the intent data is first parsed to extract key search terms. The parsing process relies on semantic understanding models or keyword extraction algorithms, such as word embedding technologies based on TF-IDF (term frequency-inverse document frequency) or deep learning (such as Word2Vec, BERT, etc.). Specifically, the system first analyzes the semantic structure of the intent data and identifies important information. For example, for the sentence "I want to buy a toy for my cat", the system will parse the core intent as "buy a toy", and "cat" as a specific object is associated with this intent.

[0071] After that, the key search terms are extracted. During the parsing process, the system extracts the keywords that are most directly related to the user's intent as the key search terms for the search query. Specifically, the key search terms extracted by the system may include "cat", "toy", and "buy". These words focus on expressing the core needs of the user. Through the keyword extraction model, the system can ensure that the selected words can cover the main aspects of the intent while avoiding interference from redundant information.

[0072] A knowledge base is a database that stores relevant information, including various knowledge contents that users may query. These contents may include FAQ, product information, technical documents, encyclopedia knowledge, etc. The structure of the knowledge base can be in the form of a relational database, a document storage system (such as Elasticsearch), or a knowledge graph based on a graph database. The system can establish a knowledge base containing various products, services, or knowledge contents. For example, a pet-related knowledge base may include product information (such as toys, food), care suggestions, and user reviews.

[0073] The system uses key search terms to query the knowledge base. The retrieval method can be based on traditional keyword matching or semantic search technology to ensure that the user's intention is semantically associated with the information in the knowledge base, so as to find the most relevant results. Specifically, for example, the system performs precise matching in the knowledge base based on the extracted key search terms such as "cat", "toy", and "purchase", and returns relevant documents or records containing these keywords. For example, the system retrieves a list of products related to "cat toys". If the system uses more advanced semantic search algorithms, the retrieval process is not limited to literal matching, but uses natural language understanding (NLU) technology to find results that match the semantics of the user's intention. For example, even if "cat toys" does not directly appear in some product descriptions in the knowledge base, the system can identify content related to pet toys.

[0074] In the preliminary search stage, the system returns multiple results related to the search terms. To improve accuracy, the system can further filter or sort the search results based on the user's historical query records, personal preferences, or other contextual information. Specifically, the system can sort the results based on the user's historical preferences (such as frequently purchased categories of goods), making it easier for users to find relevant products. In addition, the system can filter out outdated, invalid, or irrelevant content. For example, if a user has previously shown interest in high-end cat toys, the system may prioritize returning relevant information about high-end toys.

[0075] After obtaining multiple candidate results in the knowledge base, the preliminary response information is generated based on these candidate results, including content that the user may be interested in, such as related product information, FAQ answers, etc. For example, assuming that the system retrieves product information related to "cat toys" from the knowledge base, the preliminary response information may include the name of the toy, price, user reviews, etc. This information will be used as the system's initial suggestions or answers to the user.

[0076] In order to facilitate subsequent processing or display, the system will structure the retrieved preliminary response information. The structured preliminary response information can be sorted and classified according to different needs, and passed to the front end for presentation as needed. Specific implementation: The system can sort the retrieved multiple products by price, evaluation or sales volume to form an ordered list of products. In addition, if the preliminary response information involves different types of answers (such as FAQ answers and product recommendations), the system can classify and organize the results by category.

[0077] Through the above steps, the present invention realizes the whole process from intention data analysis to key search term extraction, and then to retrieval and acquisition of preliminary response information from the knowledge base. This method ensures that the system can extract key information from the user's natural language input, and through intelligent retrieval technology, quickly and accurately obtain corresponding response information from the knowledge base to meet the user's needs.

[0078] Step S208: dynamically modify the preliminary response information according to preset rules to obtain modified response information.

[0079] like Figure 5 As shown, the method for dynamically modifying the preliminary response information according to a preset rule includes the following steps:

[0080] Step S2082, define preset rules.

[0081] Preset rules are defined based on different application scenarios, business requirements, and user historical behaviors. The rules can be simple filtering conditions, sorting rules, or complex logical judgments, which can be divided into the following categories:

[0082] 1) User preference rules: Based on the user's historical interaction data (such as the user's previous purchase records, query habits, etc.), provide users with information that is more in line with their preferences. For example, users prefer cost-effective products or frequently search for a certain type of product. 2) Context-related rules: Adjust the initial response information according to the context of the current session. For example, if the user is currently looking for pet supplies, the system will give priority to pet-related products. 3) Business rules: Adjust the response information based on the business party's priority, promotional activities, inventory status, etc. For example, products that are currently on promotion will be displayed first, or the system recommends the nearest store based on geographic location. 4) Quality assurance rules: Ensure that the returned response information meets the accuracy, timeliness and consistency requirements.

[0083] Preset rules can be defined in various forms such as rule engines, decision trees, machine learning models, etc. For ease of management and rapid application, rules are usually defined in a structured format (such as JSON, XML, etc.) and stored in the system's rule base. The system can dynamically load these rules according to different scenarios and make corresponding corrections to the initial response information.

[0084] Step S2084, dynamically correct the preliminary response information.

[0085] After obtaining the initial response information, the system will dynamically modify the information according to the preset rules. The modification methods include filtering, sorting, enhancing or deleting.

[0086] 1) Modification based on user preferences.

[0087] The system will personalize the initial response information based on the user's historical behavior and preferences. For example, if the user usually prefers lower-priced items, the system will adjust the order of items based on the user's historical query or purchase history, placing lower-priced items at the front of the list. At the same time, the system may filter out high-priced items that do not conform to the user's consumption habits.

[0088] 2) Correction based on contextual information.

[0089] The system will also adjust the response based on the context of the current session. For example, if the user is currently asking questions about pet care, the system will prioritize products or information that are more relevant to this topic.

[0090] Specific implementation: In the case of a query for "cat toys", if the user's previous query involves pet health, the system may prioritize displaying some cat toys with health benefits and include relevant health tips in the recommended content.

[0091] 3) Modification based on business rules.

[0092] Business rules dynamically influence the initial response information to ensure that the content returned by the system meets the current business goals or market needs. For example, the system can prioritize products that are currently on sale, or mark products with limited inventory as "sold out soon." In addition, if there are specific limited-time offers or delivery conditions in the user's area, the system can adjust the product list accordingly based on these business rules.

[0093] 4) Amendment of quality assurance rules.

[0094] To ensure the accuracy of the response information, the system can apply quality assurance rules to automatically correct or remove expired, erroneous or invalid content. The system can regularly check the inventory status of products and remove products that are sold out or about to be removed from the initial response information. In addition, the system will update product information to ensure that data such as price and description are up to date to prevent users from obtaining outdated information.

[0095] 5) Implementation of dynamic correction algorithm.

[0096] Depending on the preset rules, the system uses the rule engine to parse the preset logic and correct the preliminary response information. The rule engine can quickly apply complex business logic and support real-time adjustments. Alternatively, through user historical data and preferences, the machine learning model can automatically learn and optimize the rules, allowing the system to more accurately recommend response information that meets user needs. Alternatively, the system can set a priority or weight for each rule, and dynamically adjust the display order of the preliminary response information through weighted sorting. For example, if the user tends to view promotional items, the weight of this type of item can be set higher.

[0097] The revised response information is the result of dynamic adjustment according to preset rules. The system will structure it for display to users or for further processing. For example, the revised response information will be sorted according to the adjusted priority, such as products are sorted by price or promotional discounts. At the same time, the system will classify similar information to ensure that users can quickly browse different types of content. For example, the system can display recommended cat toys in groups according to their functions (such as educational, bite-resistant, etc.).

[0098] The embodiment of the present invention also provides another AI-based generative question-answering method. Figure 2 The difference between the corresponding embodiments is that the multimodal question and answer request in this embodiment is an image or video input. Therefore, this embodiment will describe in detail how to analyze the image uploaded by the user, extract key information and generate the corresponding text description language through image recognition and object detection algorithms when the multimodal question and answer request is an image input. The other parts will not be described in detail. Figure 6As shown, the method for converting an image into a text description language comprises the following steps:

[0099] Step S602: image processing.

[0100] When the input is an image, image recognition and object detection algorithms are used to analyze the images uploaded by the user. When the input is a video, image recognition and object detection algorithms are used to analyze the video frames in the video. The specific steps are as follows:

[0101] 1) Object detection algorithm.

[0102] Use pre-trained deep learning models (such as YOLO, Faster R-CNN, SSD, etc.) for object detection. For example, extract image features through convolutional neural networks (CNN). The CNN model gradually extracts spatial features in the image through multiple layers of convolution and pooling operations. Generate candidate regions through the region proposal network (RPN), which may contain the objects to be detected. Classify the candidate regions and predict the bounding box of the object in each region, and output information including object category, confidence, and bounding box coordinates.

[0103] Step S604: image recognition.

[0104] After the object is detected, a more detailed image recognition is performed. Convolutional neural networks (CNN) or other deep learning models (such as ResNet, Inception, etc.) can be used for object recognition. The feature vector of the object is extracted to facilitate the subsequent text description generation. The detected objects are classified into predefined categories, such as "cat", "dog", "car", etc. After obtaining the results of object detection and recognition, this module is responsible for extracting key information from them to generate text description language. Specifically, all detected objects and their categories and positions are extracted from the output of object detection. For example, for an image, objects such as "cat", "ball", "sofa" may be recognized. Attribute information of the object, such as color, size, etc., is extracted. For example, the recognized "red ball" may contain attribute information of color and shape. The extracted object information and attributes are sorted to construct a preliminary text description. For example, for the detected object, the generated description can be: "A black cat sits next to a red ball." After that, the corresponding text description language is generated based on the extracted key information. Specifically, multiple text description templates can be predefined so that they can be dynamically filled according to the detected objects and their attributes. According to the identified objects and their attributes, the corresponding information is extracted from the template and filled into the predefined text description. For example: Generate a description from the detected results: Input information: Object category = "cat"; Attribute = "black"; Next object = "red ball". Generate text: "A black cat sits next to a red ball." To improve the naturalness and fluency of the generated text, the system can use natural language processing (NLP) technology to optimize the generated text, including grammar correction and sentence reconstruction.

[0105] Through the above steps, the present invention provides an effective implementation scheme combining image recognition and object detection algorithms, which can extract key information from images uploaded by users and generate corresponding text description languages. This method not only improves the multimodal question-answering system's ability to understand image input, but also enhances the user's interactive experience, and is suitable for various intelligent application scenarios.

[0106] This application also provides an AI-based generative question-answering device, such as Figure 7 As shown, it includes: a conversion module 72, configured to receive a multimodal question and answer request input by a user, and convert the multimodal question and answer request into a text description language, wherein the multimodal question and answer request includes at least one of the following: text, voice, image, and video; a parsing module 74, configured to parse the text description language and generate corresponding intent data, and based on the intent data, retrieve and obtain corresponding preliminary response information in the knowledge base; a correction module 76, configured to dynamically correct the preliminary response information according to preset rules or user feedback, and generate corrected response information.

[0107] It should be noted that the AI-based generative question-answering device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the AI-based generative question-answering device provided in the above embodiment and the AI-based generative question-answering method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0108] Figure 8 FIG. 1 shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present disclosure. It should be noted that: Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0109] like Figure 8 As shown, the electronic device includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0110] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0111] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A generative question-answering method based on AI, characterized in that: include: Receiving a multimodal question and answer request input by a user, and converting the multimodal question and answer request into a text description language, wherein the multimodal question and answer request includes at least one of the following: text, voice, image, and video; Parsing the text description language and generating corresponding intent data, and based on the intent data, retrieving and acquiring corresponding preliminary response information in a knowledge base; The preliminary response information is dynamically modified according to preset rules to generate modified response information.

2. The method according to claim 1, characterized in that Converting the multimodal question-answering request into a text description language includes at least one of the following: Converting the user's voice input into the text description language in real time through a voice recognition model; Analyze the images uploaded by users through image recognition and object detection algorithms, extract key information and generate the corresponding text description language; Video analysis technology is used to parse the video frames in the video uploaded by the user frame by frame to extract visual information, and speech recognition technology is used to process the audio content in the video, and the entire video is converted into the text description language based on the visual information and the audio content.

3. The method according to claim 1, characterized in that Parsing the text description language and generating corresponding intent data includes: Using a natural language processing tool to segment the text description language, and performing part-of-speech tagging on the segmented results; Extract important entities in the text through the named entity recognition model and mark the identified important entities as key features; Based on the key features, combined with the user's historical query records, current session context and user portrait, a long short-term memory network or a transformer model is used to process the context data of the text description language to identify the intent data.

4. The method according to claim 1, characterized in that: Based on the intention data, the corresponding preliminary response information is retrieved and obtained in the knowledge base, including: Parsing the intent data to extract key search terms; Based on the key search terms, corresponding preliminary response information is retrieved and obtained from the knowledge base.

5. The method according to claim 1, characterized in that Dynamically modifying the preliminary response information according to a preset rule to generate modified response information includes at least one of the following: In the case of detecting that the original word in the preliminary response information is inappropriate or inaccurate, replacing the original word with a synonym or professional term in the knowledge base; The format of the preliminary response information is adjusted according to the user's preference or response format requirement.

6. The method according to claim 1, characterized in that The user's voice input is converted into the text description language in real time through the voice recognition model, including: Using the speech recognition model to perform layer-by-layer nonlinear mapping on the speech input to obtain a nonlinear mapping result; Calculating a probability value of a category of a speech unit in the speech input based on a result of the nonlinear mapping, and determining a probability distribution of the speech unit based on the probability value; Based on the probability distribution, the category of the speech unit is determined, and based on the determined category, the speech input is converted into the text description language.

7. A generative question-answering device based on AI, characterized in that: include: A conversion module is configured to receive a multimodal question and answer request input by a user, and convert the multimodal question and answer request into a text description language, wherein the multimodal question and answer request includes at least one of the following: text, voice, image, and video; A parsing module, configured to parse the text description language and generate corresponding intent data, and based on the intent data, retrieve and obtain corresponding preliminary response information in a knowledge base; The correction module is configured to dynamically correct the preliminary response information according to preset rules or user feedback to generate corrected response information.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 6.

9. A computer device, characterized in that: include: Memory and processor, The memory stores a computer program; The processor is used to execute the computer program stored in the memory, and when the computer program is run, the processor is enabled to execute the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images