User Q&A method, device, electronic device, storage medium and program product

Abstract

The present application provides a user Q&A method, device, electronic device, storage medium and program product, which relates to the field of artificial intelligence technology. By searching for text analysis and displaying pictures through the device ID, user questions and text answer templates, a picture-and-text style answer is realized. The present application provides a "foolproof" method for guiding users to use or operate devices in the field of device Q&A applications such as household appliances, and at the same time proposes an applicable human-computer interaction process. The present application matches the text answer template according to the device ID, and realizes the rapid acquisition of the preliminary answer to the user's question. Through further searching with the device ID and user questions, a solution containing text analysis and displayed pictures is obtained, making the solution more vivid, which is beneficial to improving user understanding and improving the accuracy and efficiency of user Q&A.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular, to a user question and answer method, device, electronic device, storage medium, and program product. Background Art

[0002] The existing user question and answer for household appliances mainly includes instruction manual question and answer. Instruction manual question and answer is essentially a special document question and answer or knowledge question and answer.

[0003] However, in the field of instruction manual question and answer for household appliances and other devices, the existing document question and answer or knowledge question and answer has the following disadvantages. There are a large number of pictures in the instruction manual documents of household appliances and other devices to assist users in operation. The text and pictures form a relatively complete information set, while general knowledge question and answer is usually presented to users in text form.

[0004] Whether it is instruction manual question and answer or document knowledge question and answer, their common deficiencies are as follows: The answers given only contain text and cannot meet the guiding needs of users to use household appliances and other devices in a "foolish" way. For user question retrieval, multiple (K) candidate knowledge need to be obtained and combined with a large language model for knowledge question and answer. Since there is a contradiction between the value of K and the retrieval efficiency, it is difficult to achieve a balance between a good question and answer effect and a low-latency interaction experience, which affects the user experience. When the corresponding instruction manual or question and answer template of the device model does not exist, there is a situation where the user's question cannot be answered or the answer quality is not high. Therefore, the efficiency and accuracy of the existing user question and answer for household appliances are low. Summary of the Invention

[0005] This application provides a user question and answer method, device, electronic device, storage medium, and program product to solve the defect of low efficiency and accuracy of the instruction manual question and answer of household appliances in the prior art, and to improve the efficiency and accuracy of the user question and answer of household appliances.

[0006] In a first aspect, this application provides a user question and answer method, including: obtaining a user question and a device ID corresponding to the user question; in the case where the device ID exists in a standard question database, matching the user question with the standard question database to determine a standard question, and determining a text answer template corresponding to the user question in a pre-constructed question-answer template database through the standard question; classifying the user question to determine the question type of the user question; in the case where the determined question type is a question type that needs to be answered in combination with pictures, obtaining an initial display picture of the device input by the user; searching for the instruction manual content of the device based on the device ID, and using at least one of the instruction manual content of the device and the text answer template as text data; fusing and organizing the user question, the initial display picture, and the text data to obtain the text analysis and the display picture to determine the solution plan.

[0007] In one embodiment, the user Q&A method further includes: when the device ID does not exist in the standard question database, obtaining the device information corresponding to the device ID, and searching for the instruction manual content of the device online according to the device information; generating a first prompt instruction according to the device ID and the user question, and obtaining a text analysis from the instruction manual content of the device according to the first prompt instruction to determine the solution.

[0008] In one embodiment, the user Q&A method further includes: when it is determined that the question type is not a question type that needs to be answered in combination with pictures, matching the device ID with the device instruction manual database to obtain the instruction manual content of the device; generating a first prompt instruction according to the device ID and the user question, and obtaining a text analysis from the instruction manual content of the device according to the first prompt instruction to determine the solution.

[0009] In one embodiment, the user Q&A method further includes: when it is determined that the question type is a question type that needs to be answered in combination with pictures, matching the device ID with the picture database to determine the picture information corresponding to the device ID, where the picture information includes a picture ID and a picture description; generating a second prompt based on the picture information, the user question, and the text answer template; re-arranging the picture information and the user question based on the second prompt to obtain a text analysis containing the picture ID; obtaining a display picture from the picture database according to the picture ID; and obtaining a solution based on the display picture and the text analysis.

[0010] In one embodiment, matching the user question with the standard question database to determine the standard question includes: converting the user question into a user question text vector, obtaining the similarity between the user question text vector and each standard question text vector in the standard question database, and taking the question corresponding to the standard question text vector with the maximum similarity as the standard question.

[0011] In a second aspect, the present application further provides a user Q&A device, including: a device identification module, configured to obtain a user question and a device ID corresponding to the user question; a Q&A template matching module, configured to match the user question with a standard question database to determine a standard question when the device ID exists in the standard question database, and determine a text answer template corresponding to the user question in a pre-constructed question-answer template database through the standard question; a question type determination module, configured to classify the user question to determine the question type of the user question; a picture-and-text Q&A module, configured to obtain an initial display picture of the device input by the user when it is determined that the question type is a Q&A type that needs to be answered in combination with pictures; search the instruction manual content of the device based on the device ID, and use at least one of the instruction manual content of the device and the text answer template as text data; fuse and organize the user question, the initial display picture and the text data to obtain the text analysis and the display picture to determine the solution.

[0012] In a third aspect, the present application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements any one of the above user Q&A methods.

[0013] In a fourth aspect, the present application further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements any one of the above user Q&A methods.

[0014] In a fifth aspect, the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements any one of the above user Q&A methods.

[0015] The user Q&A method, device, electronic device, storage medium, and program product provided by the present application search for text analysis and display pictures through the device ID, user question, and text answer template, realizing a picture-and-text answer. The present application provides a "foolproof" method for guiding users to use or operate devices in the field of device Q&A applications such as household appliances, and at the same time proposes an applicable human-computer interaction process. The present application matches the text answer template according to the device ID, realizing the rapid acquisition of a preliminary answer to the user question. Through further search by the device ID and the user question, a solution containing text analysis and display pictures is obtained, making the solution more vivid, which is conducive to improving user understanding and improving the accuracy and efficiency of user Q&A. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] To more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0017] Figure 1 It is one of the schematic flowcharts of the user Q&A method provided by the present application.

[0018] Figure 2 It is a schematic diagram of the standard question database provided by the present application.

[0019] Figure 3 It is a schematic diagram of the question-answer template database provided by the present application.

[0020] Figure 4 It is a schematic diagram of the device instruction manual database provided by the present application.

[0021] Figure 5 It is a schematic diagram of the picture database provided by the present application.

[0022] Figure 6 It is the second schematic flowchart of the user Q&A method provided by the present application.

[0023] Figure 7 It is the third schematic flowchart of the user Q&A method provided by the present application.

[0024] Figure 8 It is a schematic structural diagram of the user Q&A device provided by the present invention.

[0025] Figure 9 It is a schematic structural diagram of the electronic device provided by the present application. Detailed implementation manners

[0026] To make the objectives, technical solutions and advantages of the present application clearer, the following will clearly and completely describe the technical solutions in the present application in conjunction with the accompanying drawings in the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0027] The following will describe Figures 1 - 9 the user Q&A method, device and electronic device of the present application.

[0028] Figure 1 It is one of the schematic flowcharts of the user Q&A method provided by the present application, as shown in Figure 1As shown in the figure, the user Q&A method includes S100 to S400, and the specific steps are as follows.

[0029] S100: Obtain the user's question and the device ID corresponding to the user's question.

[0030] As Figure 6 shown in the figure, the execution entity of the embodiment of the present application is a user Q&A device, and the user Q&A device includes a device identification module, a Q&A template matching module, a question type determination module, and an illustrated Q&A module.

[0031] For the Q&A of the instructions of devices such as household appliances, usually the user needs to input information such as the device model, such as the device ID, etc. However, due to reasons such as the hidden position of the label on the device and the missing label, it is not easy to find the corresponding device ID. Therefore, optionally, the device identification module can identify the device ID according to the device picture taken by the user. Further, the device category can also be identified through the device picture.

[0032] For example, the device identification module includes a device identification model, whose input is the device picture and the output is the device ID and the device category (such as rice cooker, air fryer, etc.).

[0033] The device identification model can be constructed based on the following steps.

[0034] (1) Obtain sample device pictures: Collect the sample device pictures corresponding to each sample device ID, and perform cleaning, screening, etc. on the sample device pictures to eliminate low-quality pictures such as blurred devices and low device ratios. To improve the classification effect of the device identification model for device pictures of different qualities, it is necessary to perform data augmentation preprocessing on the sample device pictures to obtain the preprocessed sample device pictures. The data augmentation preprocessing includes at least adding background noise, rotation, or spatial distortion.

[0035] (2) Construct a sample data set corresponding to the sample device ID, the sample device category, and the preprocessed sample device pictures.

[0036] (3) Based on the constructed sample data set above, train a preset device picture classification model. The preset picture classification model includes EfficientNetV2, Swin Transformer, etc. Train the preset device picture classification model to recognize the mapping relationship between the sample device ID, the sample device category, and the sample device picture. Until the preset device picture classification model can accurately output the corresponding device ID and device category according to the sample device picture, end the training, and obtain the device identification model.

[0037] S200: When the device ID exists in the standard question database, match the user question with the standard question database to determine the standard question, and determine the text answer template corresponding to the user question in the pre-constructed question-answer template database through the standard question.

[0038] The question-answer template matching module checks whether the device ID exists in the pre-constructed standard question database. If it exists, match the user question with the questions in the standard question database, and return the top 1 similarity (maximum similarity) and the ID of the standard question. Further, match the text answer template in the preset question-answer template database according to the ID of the standard question. The text answer template includes the preset text answer to the user question. The text answer template is used to provide a preliminary solution for the user question. The user question can be answered step by step according to the format of the text answer template.

[0039] As Figure 2 shown, the standard question database includes the mapping relationship between the standard question text vector, the ID of the question, and the preset device ID. If the device ID belongs to the preset device ID in the standard question database, find the standard question corresponding to the maximum similarity in the standard question database according to the user question. Further, find the text answer template corresponding to the standard question in the pre-constructed question-answer template database. For example, as Figure 3 shown, the question-answer template database includes the Q&A template data tables of multiple preset device IDs. Each Q&A template data table of a preset device ID includes the IDs of multiple questions and the corresponding answer templates. Match the ID of the preset device according to the ID of the standard question, and further match the answer template, and finally realize determining the text answer template corresponding to the user question through the standard question.

[0040] S300: Classify the user question to determine the question type of the user question.

[0041] As Figure 7As shown, the user questions are classified to determine the types of the user questions. For example, a question classification model is constructed. (1) Collect sample user questions: Accumulate sample user questions that users often ask, including business actual data, Internet data, etc. (2) Data preprocessing and question category annotation: Remove special characters, punctuation marks, etc. from the sample user questions and annotate the question types. For example, define multiple basic question types, such as product information questions (e.g., What is the rated power of this device?), device usage questions (e.g., How to make an appointment for cooking porridge / rice / soup?), troubleshooting questions (e.g., The display on my rice cooker shows C2. What should I do?), device maintenance questions (e.g., What is the warranty service situation of the rice cooker?), cleaning and maintenance questions (e.g., How should the rice cooker be cleaned?). (3) Lexical segmentation and word embedding: Decompose the sample user questions into multiple tokens, and then perform word embedding on each token to convert it into a numerical form that can be learned by a preset model, obtaining numerical samples for model training. (4) Training of the question classification model: Obtain a preset model. For example, the preset model is a Bidirectional Encoder Representations from Transformers (BERT) model. The BERT model is a complex classification model based on transformers that can understand the context information of words. Train the BERT model according to the numerical samples to learn the patterns and associations between the sample user questions and their question types. After training, a question classification model is obtained. Automatically determine the question type of the user question according to the question classification model.

[0042] Furthermore, the user questions may be diverse and ambiguous, resulting in inaccurate judgment of the user types. For example, the user question is The rice cooker doesn't work well. What should I do? To improve the accuracy of understanding user questions, when the question type cannot be determined, use a large language model to judge the question type, such as letting a more powerful large language model judge through prompt words, or further asking the user to determine the question type.

[0043] S400: In the case where the determined question type is a question type that needs to be answered in combination with pictures, obtain the initial display picture of the device input by the user; search for the instruction manual content of the device based on the device ID, and use at least one of the device instruction manual content and the text answer template as text data; fuse and organize the user question, the initial display picture, and the text data to obtain a text analysis and a display picture to determine the solution.

[0044] Specifically: The question type output by the question classification model can be obtained. Determine the solution according to the question type. The solution includes a solution with text analysis and a display picture (a solution with both pictures and texts).

[0045] For example, if it is determined that the problem type is at least one of equipment usage problems, troubleshooting problems, or equipment repair problems, then it is determined that the user type is a problem type that requires a picture to answer. Input the equipment ID, user question, and text answer template into the picture-and-text Q&A module. The picture-and-text Q&A module searches according to the equipment ID, user question, and text answer template, obtains a text analysis and display pictures to determine the solution.

[0046] For example, the user's question is how to steam rice with a rice cooker? The text answer template is Step 1: Plug in the power supply; Step 2: Press the timer button; Step 3: Press the steaming button. Search for the corresponding display pictures and detailed text analysis according to the equipment ID, user question, and text answer template. Insert the display pictures and detailed text analysis into the corresponding steps of the text answer template to obtain the solution.

[0047] The user Q&A method provided by the embodiments of the present application searches for text analysis and display pictures through the equipment ID, user question, and text answer template, realizing picture-and-text answers. The present application provides a "foolproof" method for guiding users to use or operate equipment in the Q&A application field of home appliances and the like, and at the same time proposes an applicable human-computer interaction process. The present application matches the text answer template according to the equipment ID, realizing the rapid acquisition of a preliminary answer to the user's question. Through further search by the equipment ID and user question, a solution containing text analysis and display pictures is obtained, making the solution more vivid, which is beneficial to improving user understanding and improving the accuracy and efficiency of user Q&A.

[0048] Based on the above embodiments, the user Q&A method further includes the following steps.

[0049] In the case where the equipment ID does not exist in the standard question database, obtain the equipment information corresponding to the equipment ID, and search for the instruction manual content of the equipment through the network according to the equipment information.

[0050] Generate a first prompt instruction according to the equipment ID and the user question, and obtain text analysis from the instruction manual content of the equipment according to the first prompt instruction to determine the solution.

[0051] As Figure 7 shown, the user Q&A device may further include an instruction manual network search module and a general Q&A module. If the equipment ID cannot be matched in the standard question database, obtain the equipment information corresponding to the equipment ID (for example, the equipment ID and equipment category). Input the equipment information into the instruction manual network search module. The instruction manual network search module searches for the instruction manual content of the equipment through the network according to the equipment information, and sends the instruction manual content of the equipment to the general Q&A module.

[0052] The manual online search module relies on a search engine and implements searches by calling the search engine's Application Programming Interface (API), such as Baidu, Google, etc. The search results of the manual online search module include structured data, such as JSON data, and the content of the device's manual is obtained by parsing the JSON data.

[0053] The general Q&A module is used for general device-related knowledge Q&A and other situations. Based on understanding the entire content of the device's manual, it can correctly understand the user's intention for questions with diverse and ambiguous language expressions from the user, and give comprehensive and reasonable answers.

[0054] Considering the complexity of question-answer template construction and the difficulty of covering all user questions with standard questions, this application designs a more general manual Q&A strategy. The general Q&A module of this application uses the reading comprehension and dialogue generation capabilities of a large language model to obtain text analysis from the content of the device's manual and answer the user's questions.

[0055] The general Q&A module obtains the device ID, device questions, and the content of the device's manual, and generates a first prompt instruction according to the device ID and the user's question through prompt engineering. The first prompt instruction is input into the trained large model to stimulate the reading comprehension and dialogue capabilities of the trained large model. The trained large model obtains text analysis from the content of the device's manual according to the first prompt instruction to determine the solution plan.

[0056] For example, the content of the first prompt instruction is as follows:

[0057] 0. User question (question);

[0058] 1. You are a powerful assistant who can both answer questions related to the manual and conduct casual Q&A;

[0059] 2. When the user's question has nothing to do with the usage questions of household appliances such as air fryers and rice cookers, ignore the content of points 3 and 4 and answer the user's question normally;

[0060] 3. Otherwise, answer the question concisely and professionally according to the "content of the device's manual";

[0061] 4. Content of the device's manual (manual_content);

[0062] 5. Answer directly without repeating the question. Try to elaborate in points and use Chinese for the answers.

[0063] This application solves the user Q&A problem when the device ID cannot be retrieved in the standard question database by searching the instruction manual content of the device online, meeting the needs of diverse device instruction manual Q&A. By constructing the first prompt instruction to automatically obtain the solution, it provides a "foolproof" method to guide users to use or operate the device, simplifying the user Q&A process.

[0064] This application provides hierarchical answers to user questions of different question types, taking into account both the response speed of the Q&A and good accuracy.

[0065] Based on the above embodiments, the user Q&A method further includes the following steps.

[0066] In the case where it is determined that the question type is not a question type that needs to be answered in combination with pictures, match the device ID with the device instruction manual database to obtain the instruction manual content of the device.

[0067] Generate the first prompt instruction according to the device ID and the user question, and obtain the text analysis from the instruction manual content of the device according to the first prompt instruction to determine the solution.

[0068] If it is determined that the question type is not a question type that needs to be answered in combination with pictures, send the device ID and the user question to the general Q&A module.

[0069] The general Q&A module can also obtain the instruction manual content of the device from the preset device instruction manual database according to the device ID. As Figure 4 shown, the device instruction manual database includes a one-to-one mapping relationship between the preset device ID and the content of the instruction manual. The device instruction manual database uses the Remote Dictionary Service (Redis) database framework. Considering the limited number of instruction manual documents (more than ten thousand) in actual business applications, all models of device instruction manuals share one data table.

[0070] In order to minimize the latency of the Q&A service as much as possible and effectively reduce the number of disk input / output operations (Input / Output, I / O) for frequently reading the device instruction manual database, the device instruction manual database stores the device ID and its corresponding instruction manual content, rather than the address of the instruction manual document. In the initialization stage of the device instruction manual database, all the instruction manual content is read in sequence by the preloading method and saved in the device instruction manual database. In this way, when providing the formal Q&A service, the Q&A can be quickly carried out according to the instruction manual content, saving the consumption time of loading the instruction manual document.

[0071] After the general Q&A module matches the instruction manual content of the device, a first prompt instruction is generated based on the device ID and the user's question, and a text analysis is obtained from the instruction manual content of the device according to the first prompt instruction to determine the solution plan.

[0072] This application obtains the instruction manual content of the device through a pre-constructed device instruction manual database, avoiding online search and improving the efficiency of obtaining the instruction manual content of the device.

[0073] Based on the above embodiments, the user Q&A method further includes the following steps.

[0074] In the case where it is determined that the question type is a question type that needs to be answered in combination with pictures, the device ID is matched with the picture database to determine the picture information corresponding to the device ID, and the picture information includes a picture ID and a picture description.

[0075] A second prompt is generated based on the picture information, the user's question, and the text answer template.

[0076] Based on the second prompt, the picture information and the user's question are rearranged to obtain a text analysis containing the picture ID.

[0077] The display picture is obtained from the picture database according to the picture ID.

[0078] A solution plan is obtained based on the display picture and the text analysis.

[0079] As Figure 5 shown, a picture database is pre-constructed, and the picture database includes picture data tables for multiple preset device IDs. Each picture data table for a preset device ID includes a one-to-one mapping relationship among a preset picture ID, a text description, and picture encoding data. The picture encoding data includes Markdown (a simple markup language).

[0080] As Figure 7 shown, the device ID is matched with the preset device IDs in the picture database to obtain relevant picture information.

[0081] A second prompt is generated according to the picture information, the user's question, and the text answer template. For example, the second prompt is as follows:

[0082] 0. User question (question);

[0083] 1. Text answer template (answer);

[0084] 2. Picture information (picture_desc);

[0085] 3. You are a powerful assistant, please regenerate the answer according to the above user question and text answer template;

[0086] 4. You need to add the picture ID for the operation of the auxiliary device to the generated answer. You can refer to the picture information in Point 2, and the output format is strictly required as follows;

[0087] 5. It must be elaborated step by step according to the format of the text answer template;

[0088] 6. The picture ID must be enclosed in square brackets for subsequent picture restoration;

[0089] 7. Answer directly without repeating the question. Try to elaborate in points, and the answer should be in Chinese.

[0090] Among them, question represents the user's question. answer represents the text answer template. picture_desc represents the picture ID and picture description.

[0091] Input the second prompt into the trained large model. The trained large model rearranges the picture information and the user's question to obtain a text analysis containing the picture ID.

[0092] Obtain the display picture from the picture database according to the picture ID, and get the solution plan based on the display picture and the text analysis.

[0093] Perform post-processing on the generated text analysis containing the picture ID. Query the picture database according to the picture ID to obtain the picture encoding data corresponding to the picture ID, decode the picture encoding data, and restore the display picture of the auxiliary device operation. Arrange the display picture into the text answer template to form a vivid and illustrated solution plan that combines "text guidance and pictures of auxiliary operation devices".

[0094] Furthermore, the trained large model includes a language large model, or an open-source language model, such as the QWen series, Chat Generative Pretrained Transformer (ChatGPT), General Language Model (GLM) 4, etc.

[0095] This application constructs the second prompt, obtains the text analysis containing the picture ID according to the second prompt, and then obtains a vivid and illustrated solution plan, making the solution plan more vivid and conducive to "foolishly" guiding users to operate and use the device.

[0096] Based on the above embodiments, search according to the device ID, user question, and text answer template to obtain the text analysis and display picture to determine the solution plan, including the following steps.

[0097] Obtain the initial display picture of the device input by the user.

[0098] Search for the device's manual content based on the device ID, and use at least one of the device's manual content and the text answer template as text data.

[0099] Integrate and organize the user's question, the initial display image, and the text data to obtain a text analysis and a display image to determine the solution.

[0100] The initial display image includes the device's operation panel, display screen, etc. First, build an image recognition and object detection model to identify the operation buttons or fault areas of the household appliance device. Second, guide the user to take a photo to obtain the initial display image. Through the pre-built image recognition and object detection model, identify the positions and identifications of the function buttons or interfaces of the device in the initial display image. Then, obtain the device's manual content and / or text answer template according to the device ID to get the text data corresponding to the device ID. Send the user's question, the text data, and the recognized initial display image into the multimodal large model, and comprehensively understand by combining multiple modal information such as text and image to provide a text analysis and a display image to provide the corresponding solution for the user. The multimodal large model can be GPT-4O, GLM-4V, etc.

[0101] Furthermore, based on the real-time scene provided by the user's camera, use the Augmented Reality (AR) engine and Simultaneous Localization and Mapping (SLAM) technology to collect the environmental data of the device through the camera, and detect the device position and spatial changes in real time, and overlay virtual information (including labels, arrows, operation guides, etc.) on the device in the real scene. The system pre-generates 3D models for the device or operation, and these 3D models are loaded and rendered in real time according to the scene when the user requests. For example, show how to set a timer for a household appliance device through buttons.

[0102] This application provides an efficient solution with both text and pictures by integrating and organizing the user's question, the initial display image, and the text data to obtain a text analysis and a display image to determine the solution.

[0103] Based on the above embodiments, match the user's question with the standard question database to determine the standard question, including: convert the user's question into a user's question text vector, obtain the similarity between the user's question text vector and each standard question text vector in the standard question database, and use the question corresponding to the standard question text vector with the maximum similarity as the standard question.

[0104] Such as Figure 2As shown, the standard question database includes a standard question vector table for multiple preset device IDs. The standard question vector table for each preset device ID includes a one-to-one mapping relationship between multiple standard question text vectors and the IDs of multiple questions. If the device ID belongs to the preset device IDs in the standard question database, the standard question with the highest similarity is searched for in the standard question database according to the user's question. Based on the standard question, the corresponding text answer template of the standard question is further found in the pre-constructed question-answer template database.

[0105] The standard question database is constructed, and the question sources include actual business data, e-commerce platform review data, etc. The standard question database adopts the open-source Facebook AI Similarity Search (Faiss) vector database framework to construct a standard question vector data table indexed by preset device IDs for devices such as household appliances. Among them, the index of the standard question vector data table uses the default index ID (question ID) in the Faiss library, and the value is the standard question text vector. In the Faiss library, once the construction of the standard question vector data table is completed, its default question ID remains fixed and does not change. Therefore, the question corresponding to the standard question text vector can be uniquely determined according to the question ID. The standard question text vector is obtained by vectorizing the question through an embedding model. For example, the vector dimension is set to 1024. The embedding model can be the pre-trained BGE Large Chinese Model (bge-large-zh). When matching the standard question, for a certain device ID, the user's question is converted into a user question text vector. For example, the user's question is converted into a user question text vector according to the embedding model to facilitate matching with the standard question text vectors in the standard question database. The matching algorithm uses the classic Flat search method in Faiss, and its advantage is high matching accuracy without the need for complex preprocessing or training. The obtained similarity is a scalar. In order to minimize the latency of the question and answer as much as possible, this application only matches the standard question text vector with the highest similarity. The question corresponding to the standard question text vector with the highest similarity is used as the standard question. According to the standard question database, the ID of the standard question is obtained. The corresponding text answer template of the standard question is retrieved from the question-answer template database according to the ID of the standard question.

[0106] The question-answer template database adopts the Redis database framework. The Redis service has a high-throughput I / O design and sub-millisecond operation latency, which is suitable for frequent question-answer template retrieval and real-time question and answer services.

[0107] By obtaining the standard question text vector with the maximum similarity and then obtaining the standard question, the present application realizes the maximization of reducing the Q&A latency and improves the efficiency of user Q&A.

[0108] Furthermore, the preset device IDs of the standard question database, question-answer template database, device instruction manual database, and picture database of the present application are the same, that is, the preset device ID in any of the above databases can be directly used for data retrieval in other databases.

[0109] The user Q&A device provided by the present application will be described below. The user Q&A device described below can be mutually corresponded and referred to the user Q&A method described above.

[0110] As Figure 8 shown, a user Q&A device includes: a device identification module 801, configured to obtain a user question and a device ID corresponding to the user question.

[0111] A Q&A template matching module 802, configured to match the user question with the standard question database to determine the standard question when the device ID exists in the standard question database, and determine the text answer template corresponding to the user question in the pre-constructed question-answer template database through the standard question.

[0112] A question type determination module 803, configured to classify the user question to determine the question type of the user question.

[0113] A picture-and-text Q&A module 804, configured to obtain the initial display picture of the device input by the user when it is determined that the question type is a Q&A type that needs to be answered in combination with pictures; search for the instruction manual content of the device based on the device ID, and use at least one of the instruction manual content of the device and the text answer template as text data; fuse and organize the user question, the initial display picture, and the text data to obtain a text analysis and a display picture to determine the solution.

[0114] The user Q&A device provided by the embodiment of the present application searches for text analysis and display pictures through the device ID, user question, and text answer template, realizing a picture-and-text answer. The present application provides a "foolproof" method for guiding users to use or operate devices in the Q&A application field of household appliances and other devices, and at the same time proposes an applicable human-computer interaction process. The present application matches the text answer template according to the device ID, realizing the rapid acquisition of the preliminary answer to the user question. Through further search by the device ID and the user question, a solution containing text analysis and display pictures is obtained, making the solution more vivid, which is conducive to improving user understanding and improving the accuracy and efficiency of user Q&A.

[0115] In one embodiment, the user Q&A device further includes an instruction manual online search module and a general Q&A module. The instruction manual online search module is configured to: when the device ID does not exist in the standard question database, obtain the device information corresponding to the device ID, and search for the instruction manual content of the device online according to the device information. The general Q&A module is configured to: generate a first prompt instruction according to the device ID and the user's question, and obtain a text analysis from the instruction manual content of the device according to the first prompt instruction to determine a solution plan.

[0116] In one embodiment, the illustrated Q&A module 804 is further configured to: when it is determined that the question type is not a question type that needs to be answered in combination with pictures, match the device ID with the device instruction manual database to obtain the instruction manual content of the device. The general Q&A module is configured to: generate a first prompt instruction according to the device ID and the user's question, and obtain a text analysis from the instruction manual content of the device according to the first prompt instruction to determine a solution plan.

[0117] In one embodiment, the illustrated Q&A module 804 is further configured to: when it is determined that the question type is a question type that needs to be answered in combination with pictures, match the device ID with the picture database to determine the picture information corresponding to the device ID, where the picture information includes a picture ID and a picture description; generate a second prompt based on the picture information, the user's question, and the text answer template; re-arrange the picture information and the user's question based on the second prompt to obtain a text analysis containing the picture ID; obtain a display picture from the picture database according to the picture ID; and obtain a solution plan based on the display picture and the text analysis.

[0118] In one embodiment, the Q&A template matching module 802 is configured to: convert the user's question into a user question text vector, obtain the similarity between the user question text vector and each standard question text vector in the standard question database, and use the question corresponding to the standard question text vector with the maximum similarity as the standard question.

[0119] Figure 9 An exemplary schematic diagram of the physical structure of an electronic device is shown as Figure 9As shown in the figure, the electronic device may include: a processor 910, a communications interface 920, a memory 930, and a communication bus 940. Among them, the processor 910, the communications interface 920, and the memory 930 complete communication with each other through the communication bus 940. The processor 910 may call the logical instructions in the memory 930 to execute the user Q&A method, which includes: obtaining a user question and a device ID corresponding to the user question; in the case where the device ID exists in the standard question database, matching the user question with the standard question database to determine a standard question, and determining a text answer template corresponding to the user question in a pre-constructed question-answer template database through the standard question; classifying the user question to determine the question type of the user question; in the case where it is determined that the question type is a question type that needs to be answered in combination with a picture, obtaining an initial display picture of the device input by the user; searching for the instruction manual content of the device based on the device ID, and taking at least one of the instruction manual content of the device and the text answer template as text data; fusing and arranging the user question, the initial display picture, and the text data to obtain a text analysis and a display picture to determine a solution plan.

[0120] In addition, when the logical instructions in the above-mentioned memory 930 can be implemented in the form of software function units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0121] On the other hand, the present application also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the user Q&A method provided by the above-mentioned various methods. The method includes: obtaining a user question and a device ID corresponding to the user question; in the case where the device ID exists in the standard question database, matching the user question with the standard question database to determine a standard question, and determining a text answer template corresponding to the user question in a pre-constructed question-answer template database through the standard question; classifying the user question to determine the question type of the user question; in the case where it is determined that the question type is a question type that needs to be answered in combination with a picture, obtaining an initial display picture of the device input by the user; searching for the instruction manual content of the device based on the device ID, and using at least one of the instruction manual content of the device and the text answer template as text data; integrating and arranging the user question, the initial display picture and the text data to obtain a text analysis and a display picture to determine a solution plan.

[0122] On another aspect, the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the user Q&A method provided by the above-mentioned various methods. The method includes: obtaining a user question and a device ID corresponding to the user question; in the case where the device ID exists in the standard question database, matching the user question with the standard question database to determine a standard question, and determining a text answer template corresponding to the user question in a pre-constructed question-answer template database through the standard question; classifying the user question to determine the question type of the user question; in the case where it is determined that the question type is a question type that needs to be answered in combination with a picture, obtaining an initial display picture of the device input by the user; searching for the instruction manual content of the device based on the device ID, and using at least one of the instruction manual content of the device and the text answer template as text data; integrating and arranging the user question, the initial display picture and the text data to obtain a text analysis and a display picture to determine a solution plan.

[0123] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0124] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. A user Q&A method, characterized in that, Including: Obtain a user question and a device ID corresponding to the user question; When the device ID exists in the standard question database, match the user question with the standard question database to determine a standard question, and determine a text answer template corresponding to the user question in a pre-constructed question-answer template database through the standard question; Classify the user question to determine the question type of the user question; When it is determined that the question type is a question type that needs to be answered in combination with a picture, obtain an initial display picture of the device input by the user; Search for the instruction manual content of the device based on the device ID, and use at least one of the instruction manual content of the device and the text answer template as text data; Fuse and organize the user question, the initial display picture, and the text data to obtain a text analysis and a display picture to determine a solution plan.

2. The user Q&A method according to claim 1, wherein The method further includes: When the device ID does not exist in the standard question database, obtain device information corresponding to the device ID, and search for the instruction manual content of the device through the network according to the device information; Generate a first prompt instruction according to the device ID and the user question, and obtain a text analysis from the instruction manual content of the device according to the first prompt instruction to determine a solution plan.

3. The user Q&A method according to claim 1, wherein The method further includes: When it is determined that the question type is not a question type that needs to be answered in combination with a picture, match the device ID with a device instruction manual database to obtain the instruction manual content of the device; Generate a first prompt instruction according to the device ID and the user question, and obtain a text analysis from the instruction manual content of the device according to the first prompt instruction to determine a solution plan.

4. The user question and answer method according to claim 1, wherein, The method further includes: When it is determined that the question type is a question type that needs to be answered in combination with a picture, match the device ID with a picture database to determine picture information corresponding to the device ID, where the picture information includes a picture ID and a picture description; Generate a second prompt based on the picture information, the user question, and the text answer template; Re-arrange the picture information and the user question based on the second prompt to obtain the text analysis containing the picture ID; Obtain the display picture from the picture database according to the picture ID; Obtain the solution plan based on the display picture and the text analysis.

5. The user Q&A method according to claim 1, characterized in that The matching the user question with the standard question database to determine a standard question includes: Convert the user question into a user question text vector, obtain the similarity between the user question text vector and each standard question text vector in the standard question database, and use the question corresponding to the standard question text vector with the largest similarity as the standard question.

6. A user Q&A device, characterized in that, Including: A device recognition module for obtaining a user question and a device ID corresponding to the user question; A question-and-answer template matching module, configured to, when the device ID exists in a standard question database, match the user question with the standard question database to determine a standard question, and determine a text answer template corresponding to the user question in a pre-constructed question-answer template database through the standard question; A question type determination module, configured to classify the user question to determine the question type of the user question; A picture-and-text question-and-answer module, configured to, when it is determined that the question type is a question-and-answer type that needs to be answered in combination with pictures, obtain an initial display picture of the device input by the user; search for the instruction manual content of the device based on the device ID, and use at least one of the instruction manual content of the device and the text answer template as text data; fuse and organize the user question, the initial display picture and the text data, and obtain a text analysis and a display picture to determine a solution.

7. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, wherein, When the processor executes the computer program, the user question-answering method according to any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the user question-answering method according to any one of claims 1 to 5 is implemented.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the user question-answering method according to any one of claims 1 to 5 is implemented.

Images