Question and answer processing method and intelligent device

Abstract

The present invention discloses a question-answer processing method and an intelligent device, which relates to the technical field of natural language processing, wherein the method comprises using sample data to pre-train a question-answer generation model and a machine reading comprehension model in linkage; inputting the text data to be tested into the question-answer generation model to generate a target question and a benchmark answer; inputting the text data to be tested and the target question into the machine reading comprehension model to obtain a predicted answer; verifying the predicted answer, and if the predicted answer is successfully verified, storing a matching pair of the target question and the predicted answer. The scheme of the present invention can improve the accuracy of question-answer interaction of intelligent devices.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a question-answering processing method and an intelligent device. Background Art

[0002] In some human-computer interaction scenarios, smart devices such as smart TVs and smart speakers can support dialogue functions, and users can interact with smart devices through question-and-answer sessions. For example, a user asks a smart speaker, "What's the weather like today?", and the smart device replies, "It's sunny today, and the temperature is 0℃ to 15℃." Another example is that a user asks a smart refrigerator, "What's in the freezer?", and the smart refrigerator replies, "Fish, beef, and ice cream."

[0003] A model framework may be pre-installed in the smart device, and a prediction model may be generated by training the model framework for question-answering. The prediction model is used to output answers that match the questions asked by the user. The accuracy of the prediction model directly affects the accuracy of the smart device's question-answering. If the prediction model is not accurate enough, the smart device may give wrong answers or incomprehensible replies after responding to the user's questions. Summary of the invention

[0004] In order to solve the problems in the above-mentioned background technology, the present invention provides a question and answer processing method and an intelligent device.

[0005] A first aspect provides a smart device, including:

[0006] A memory configured to store matching pairs of questions and answers constructed after question-answering training;

[0007] The controller is configured to execute:

[0008] Use sample data to pre-train the question-answer generation model and the machine reading comprehension model.

[0009] Inputting the text data to be tested into the question-answer generation model to generate a target question and a benchmark answer;

[0010] Inputting the text data to be tested and the target question into the machine reading comprehension model to obtain a predicted answer;

[0011] The predicted answer is verified, and if the predicted answer is successfully verified, a matching pair of the target question and the predicted answer is written into the memory associated with the machine reading comprehension model.

[0012] In some embodiments, the controller is configured to perform the linkage training as follows:

[0013] Fine-tune the first pre-trained model using the sample data to obtain an elementary question-answer generation model;

[0014] Fine-tune the second pre-trained model using the sample data to obtain an elementary machine reading comprehension model;

[0015] Initiate joint training for the entry-level question-answering generation model and machine reading comprehension model.

[0016] In some embodiments, the controller is configured to perform the linkage training as follows:

[0017] First, we use the elementary machine reading comprehension model as a benchmark, optimize the parameters of the elementary question-answer generation model, and obtain an advanced question-answer generation model.

[0018] Then, based on the advanced question-answering generation model, the parameters of the primary machine reading comprehension model are optimized to obtain an advanced machine reading comprehension model.

[0019] In some embodiments, the controller is configured to obtain an advanced question-answer generation model in the following manner:

[0020] Using the sample data to train the preliminary question-answer generation model, and calculating a first loss value of the preliminary question-answer generation model; the sample data includes a description text, a sample question, and a sample answer;

[0021] Inputting the description text and the first question and the first answer output by the preliminary question-answer generation model into the preliminary machine reading comprehension model, and calculating a second loss value of the preliminary machine reading comprehension model;

[0022] Calculate a first weighted loss value by using the first loss value and its corresponding first weight, the second loss value and its corresponding second weight;

[0023] Parameters of the preliminary question-answer generation model are optimized according to the first weighted loss value.

[0024] In some embodiments, the controller is configured to obtain an advanced machine reading comprehension model in the following manner:

[0025] Inputting the sample data into the elementary machine reading comprehension model and the advanced question-answer generation model respectively;

[0026] Calculating a third loss value of the advanced question-answer generation model, and calculating a fourth loss value of the elementary machine reading comprehension model;

[0027] Calculate a second weighted loss value by using the third loss value and its corresponding third weight, the fourth loss value and its corresponding fourth weight;

[0028] Parameters of the preliminary machine reading comprehension model are optimized according to the second weighted loss value.

[0029] In some embodiments, the controller is further configured to perform:

[0030] Use the first crawler strategy to obtain hot search keywords;

[0031] Using a second crawler strategy, obtaining target text data related to the hot search keyword from network resources;

[0032] The target text data is preprocessed and saved as the text data to be tested.

[0033] In some embodiments, the controller is further configured to verify the predicted answer in the following manner:

[0034] Calculating the confidence of the predicted answer output by the advanced machine reading comprehension model;

[0035] If the confidence level is greater than or equal to a preset threshold, and the predicted answer is consistent with the reference answer output by the advanced question-answer generation model, then the predicted answer is determined to be successfully verified.

[0036] In some embodiments, the controller is further configured to execute: if the confidence level is less than the preset threshold, or the predicted answer is inconsistent with the reference answer, discarding the predicted answer.

[0037] In some embodiments, the controller is further configured to perform:

[0038] If the smart device further includes a sound collector and a sound player, the first question of receiving a user's voice input through the sound collector is:

[0039] Searching the memory for a first target matching pair to which the first question belongs, and controlling the audio player to broadcast an answer in the first target matching pair;

[0040] and / or,

[0041] If the smart device further includes a display, receiving a second question input by a user, and controlling the display to display the second question,

[0042] The memory is searched for a second target matching pair to which the second question belongs, and the display is controlled to display an answer in the second target matching pair.

[0043] A second aspect provides a question-answering processing method, the method comprising:

[0044] Use sample data to pre-train the question-answer generation model and the machine reading comprehension model.

[0045] Inputting the text data to be tested into the question-answer generation model to generate a target question and a benchmark answer;

[0046] Inputting the text data to be tested and the target question into the machine reading comprehension model to obtain a predicted answer;

[0047] The predicted answer is verified, and if the predicted answer is successfully verified, a matching pair of the target question and the predicted answer is stored.

[0048] Other embodiments of the second aspect may be adapted to refer to the first aspect described above and will not be described in detail here.

[0049] In this application, the bottom layer of the smart device includes two models. The first is a question-answering generation model for automatically extracting questions and their answers from text; the second is a machine reading comprehension model. The text and questions are input into the machine reading comprehension model. The machine reading comprehension model searches and locates the answers corresponding to the questions from the text based on text semantic understanding and keyword relevance. This application implements a linkage training mechanism for the two models. The so-called linkage training means taking model 2 as a benchmark, optimizing the parameters of model 1, and realizing the advancement of model 1. Then, taking the advanced model 1 as a benchmark, optimizing the parameters of model 2, and realizing the advancement of model 2. Then, the advanced model 1 and model 2 are used for prediction and verification. In the prediction and verification stage, the text data to be tested is input into the question-answering generation model, and the target question and benchmark answer can be extracted from the text. For example, if the text data to be tested is "Movie A is directed by and starred by actor X", one of the extracted target questions is "Who is the director of Movie A", and the benchmark answer is "Actor X". Then the text data to be tested and the target question are input into its reading comprehension model, and the output predicted answer is verified. If the verification is successful, the matching pair of "Who is the director of Movie A" + "Actor X" is written into the memory, and the accurate question-answering prediction result is stored in the library. When the smart device receives the question asked by the user, it queries the answer corresponding to the question from the library and feeds the answer back to the user, thereby improving the accuracy of the question-answering interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 An exemplary diagram of a speech recognition network architecture is shown;

[0051] Figure 2 The diagram exemplarily shows a scenario of a user performing a question-and-answer interaction with a smart speaker;

[0052] Figure 3 The diagram exemplarily shows a scenario of a user performing a question-and-answer interaction with a smart refrigerator;

[0053] Figure 4 A scene diagram showing a question-and-answer interaction between a user and a display device is exemplified;

[0054] Figure 5 A flowchart of the question-answering processing method is exemplified;

[0055] Figure 6 The flowchart of the first stage of linkage training is shown as an example;

[0056] Figure 7 The flowchart of the second stage of linkage training is shown as an example;

[0057] Figure 8 The flowchart of the third stage of linkage training is shown as an example;

[0058] Fig. 9 The flowchart for prediction and verification using Model1_2 and Model1_2 is shown as an example;

[0059] Fig.10 The flowchart of the collection and preprocessing of the text data to be tested is shown as an example;

[0060] Fig.11 An example of a question and answer setting page for a smart device is shown;

[0061] Fig.12 An exemplary diagram showing the change effect of the question and answer setting page when the smart device is not connected to the network after the switch control is turned on;

[0062] Fig.13 The schematic diagram of the structural configuration of the first type of intelligent device is exemplarily shown;

[0063] Fig.14 The schematic diagram of the structural configuration of the second intelligent device is exemplarily shown;

[0064] Fig.15 The schematic diagram of the structural configuration of the third type of intelligent device is shown as an example. DETAILED DESCRIPTION

[0065] In order to make the purpose and implementation method of the present application clearer, the exemplary implementation method of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0066] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0067] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0068] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0069] In some application scenarios, smart devices can support human-computer interaction, especially the dialogue function, which in turn involves the question-and-answer function, in which a user asks a question to a smart device, and the smart device queries the answer and then gives an answer. Through the question-and-answer interaction between the user and the smart device, on the one hand, it can answer questions for the user and realize knowledge transfer, and on the other hand, it is conducive to the evolution of the question-and-answer system of smart devices. Among them, the smart devices are not limited to terminal types such as smart TVs, smart speakers, smart refrigerators, and smart phones.

[0070] This application may involve voice control and interaction of smart devices, such as users asking questions to smart devices in the form of voice. In the field of voice control, entities refer to things that exist objectively and can be distinguished from each other, including specific people, things, objects, organizations, abstract concepts, etc.; knowledge graphs are essentially a semantic network that can represent the semantic relationships between entities. In the knowledge graph, entities are vertices or nodes, and relationships are edges. Knowledge graphs can be constructed in a variety of ways. The focus of the embodiments of this application is not on how to construct knowledge graphs, so this will not be described in detail.

[0071] See also Figure 1 , the smart device is used to receive input voice information and output the processing results of the voice information. The voice recognition service device is an electronic device deployed with a voice recognition service, the semantic service device is an electronic device deployed with a semantic service, and the business service device is an electronic device deployed with a business service. The electronic devices here may include servers, computers, etc. The voice recognition service, semantic service (also called semantic engine) and business service here are web services that can be deployed on electronic devices, wherein the voice recognition service is used to recognize audio as text, the semantic service is used to perform semantic analysis on the text, and the business service is used to provide specific services such as the weather query service of Moji Weather, the music query service of QQ Music, etc. In some embodiments, Figure 1 In the illustrated architecture, there may be multiple physical service devices deployed with different business services, or one or more functional services may be integrated into one or more physical service devices.

[0072] In some embodiments, the following Figure 1 The process of processing voice information input into the smart device in the illustrated architecture is described by way of example. Taking the information input into the smart device as a query statement input by voice as an example, the above process may include the following three processes:

[0073] [Speech Recognition]

[0074] After receiving the query statement input by voice, the smart device may upload the audio of the query statement to the voice recognition service device, so that the voice recognition service device recognizes the audio as text through the voice recognition service and returns it to the smart device. In one embodiment, before uploading the audio of the query statement to the voice recognition service device, the smart device may perform denoising on the audio of the query statement, and the denoising process here may include steps such as removing echoes and environmental noise.

[0075] [Semantic understanding]

[0076] The smart device uploads the text of the query statement recognized by the speech recognition service to the semantic service device, so that the semantic service device performs semantic analysis on the text through the semantic service to obtain the business field, intent, etc. of the text.

[0077] [Semantic Response]

[0078] The semantic service device sends a query instruction to the corresponding business service device based on the semantic analysis result of the query text to obtain the query result given by the business service. The smart device can obtain the query result from the semantic service device and output it. As an embodiment, the semantic service device can also send the semantic analysis result of the query statement to the smart device, so that the smart device can output the feedback statement in the semantic analysis result. It should be noted that Figure 1 The architecture shown is only an example and does not limit the protection scope of the present application. In the embodiment of the present application, other architectures can also be used to implement similar functions. For example, all or part of the three processes can be completed by the smart terminal, which will not be described in detail here.

[0079] In some application scenarios, such as Figure 2 In the scenario example, the smart speaker can be configured with a sound collector and a sound player. The sound collector is, for example, a microphone or other sound receiving device, and the sound player is, for example, a built-in speaker. The user asks the smart speaker "What is the highest mountain in the world?" The smart speaker collects the voice information 1, and analyzes and responds to the voice information 1 with reference to the aforementioned voice processing architecture. After learning that the user intends to ask a knowledge question and a specific question, the smart speaker searches for the answer to the question in the knowledge base or question-and-answer base, and announces the answer "Mount Everest" through the sound player.

[0080] In some application scenarios, such as Figure 3 In the scenario example, the smart refrigerator can be configured with a sound collector and a sound player. When the user is making a lunch recipe, he needs to know what existing ingredients are stored in the refrigerator. The user asks the smart refrigerator "What's in the freezer?" The smart speaker collects the voice information 2, parses and responds to the voice information 2. After knowing that the user intends to query the ingredients in the freezer, it queries the local ingredient library for the types of ingredients stored in the freezer, and plays the answer "fish, beef and chicken" through the sound player.

[0081] In some application scenarios, such as Figure 4 In the scenario example, the display device may be configured with hardware devices such as a display. Based on the capability configuration of the display device, the user may input questions through a remote control, an electronic / physical keyboard, a mouse, touch, voice, etc. The display device responds to the question input by controlling the display to display the question on a dialogue page. For example, the question displayed is "Who is the heroine of movie A?" If the display device finds the answer to the question in the knowledge base or the question-and-answer library, the answer is displayed on the dialogue page. For example, the answer displayed is "Actor X".

[0082] Some of the knowledge bases or question-and-answer bases of smart devices rely on manual construction, maintenance, and data expansion. Once the user's questions exceed the knowledge reserves in the base, such as obscure questions or questions involving some emerging network terms, the smart device may output wrong answers or fail to respond to incomprehensible questions. Smart devices can also build knowledge bases or question-and-answer bases by training models, but insufficient model accuracy will also lead to low question-and-answer accuracy.

[0083] In some embodiments, the present application provides a dual-model linkage training and prediction mechanism, thereby improving the accuracy of the predicted question + answer matching pairs, and writing the correct question and answer matching pairs into the question and answer library, thereby achieving data expansion of the question and answer library through the accumulation of question and answer matching pairs.

[0084] In some embodiments, the smart device also includes a memory for storing data in the question and answer library. The smart device can receive a first question input by user voice through a sound collector, and query the first target matching pair to which the first question belongs in the memory. Since the matching pair is used to record the correspondence between questions and answers, after determining the first target matching pair, the answer to the first question can be obtained. The smart device controls the sound player to play the answer in the first target matching pair, thereby realizing voice question and answer interaction.

[0085] In some embodiments, the smart device may include a display for presenting a user interface. The user can input the second question through devices such as a remote control, an electronic / physical keyboard, a mouse, etc. The smart device controls the display to synchronously display the text information of the second question, and at the same time queries the second target matching pair to which the second question belongs in the memory, and then controls the display to display the answer in the second target matching pair, thereby realizing question-and-answer interaction in the form of a UI.

[0086] In some embodiments, the smart device further includes a controller, which is configured with an operating system and a logic program, etc. The controller is connected to the memory and other hardware devices of the smart device to realize the control and function implementation of the smart device. Figure 5 An exemplary question-answer processing method is provided, through which the construction and data expansion of the question-answer database are realized. The method is configured to be executed by a controller and includes the following program steps as a whole:

[0087] Step S501, using sample data, pre-training the question-answer generation model and the machine reading comprehension model in a linked manner.

[0088] In this application, the bottom layer of the smart device includes two training models, namely the question-answering generation model and the machine reading comprehension model. Among them, the question-answering generation model is mainly used to automatically extract possible questions and their answers from the text. For example, if a text is described as "Movie A is directed and starred by actor X", then question 1 that can be extracted from the text is "Who is the director of Movie A?", and the answer to question 1 is "Actor X", and question 2 that can be extracted is "Who is the star of Movie A?", and the answer to question 2 is "Actor X".

[0089] The Machine Reading Comprehension (MRC) model is a technology that uses algorithms to enable computers to understand the semantics of an article and then find answers to questions in the article. The text and question sentences are input into the machine reading comprehension model. The machine reading comprehension model is based on the correlation between the text and the question. For example, the word "river" appears in the question sentence, and the text contains the keyword "Yangtze River". Although the two keywords are not exactly the same, their semantic encodings are similar and have a high correlation. Therefore, the word "Yangtze River" in the text and its adjacent context sentences will become the focus of the model when answering questions, and then find and locate the answer to the question and output the prediction result.

[0090] The sample data is a set of general data developed for training question-answer generation models and machine reading comprehension models, and the sample data includes description text, sample questions and sample answers. The sample data can be repeatedly called for training at different stages of the linkage training mechanism.

[0091] In some embodiments, for the convenience of description, the question-answer generation model is named Model1, and the machine reading comprehension model is named Model2. The so-called linkage training means that Model2 is used as a benchmark to optimize the parameters of Model1, so that the question-answer generation model evolves from the elementary Model1_1 to the advanced Model1_2; then Model1_2 is used as a benchmark to optimize the parameters of Model2, so that the machine reading comprehension model evolves from the elementary Model2_1 to the advanced Model2_2. Then, Model1_2 and Model2_2 are used to perform the following prediction and verification process.

[0092] Step S502: input the text data to be tested into the question-answer generation model to generate a target question and a benchmark answer.

[0093] The text data to be tested is the data used by the intelligent device to make predictions based on Model1_2 and Model2_2. It is generally used to expand the knowledge reserve of the question-answer library. The text data to be tested can be manually formulated or can be a large amount of open world text from the Internet. By inputting the text data to be tested into Model1_2, the target question and its corresponding benchmark answer can be automatically generated and output.

[0094] Step S503: input the text data to be tested and the target question into the machine reading comprehension model to obtain a predicted answer.

[0095] The text data to be tested and the target question generated by Model1_2 are input into Model2_2, and Model2_2 matches the answer to the target question from the text data to be tested, that is, the predicted answer.

[0096] Step S504: verify the predicted answer.

[0097] Step S505, determining whether the predicted answer is successfully verified. If the predicted answer is successfully verified, executing step S506; if the predicted answer fails to be verified, executing step S507.

[0098] Step S506: write the matching pair of the target question and the predicted answer into a memory associated with the machine reading comprehension model.

[0099] Step S507: discard the predicted answer.

[0100] When the predicted answer verification succeeds, that is, the target question and the predicted answer are matched successfully, the matching pair is allowed to be stored in the question and answer database; conversely, if the predicted answer verification fails, the predicted answer is directly discarded, that is, the target question and the predicted answer are not allowed to be written into the question and answer database, thereby ensuring the accuracy of the data in the question and answer database.

[0101] In some embodiments, the dual-model linkage training mechanism involved in the aforementioned step S501 can be mainly divided into the following three stages:

[0102] (a) If Figure 6 As shown, in the first stage of the linkage training mechanism, two initial pre-training models need to be called, named the first pre-training model and the second pre-training model, where the first pre-training model is the initial model strategy of the question-answering generation model, and the second pre-training model is the initial model strategy of the machine reading comprehension model.

[0103] In some embodiments, the first pre-training model and the second pre-training model can be appropriately selected according to the training requirements. For example, the first pre-training model can select mT5-PEGASUS, and the second pre-training model can select BERT. Among them, mT5-PEGASUS is based on mT5 as the basic architecture and initial weights. The Tokenizer (word segmenter) is first improved in combination with the characteristics of Chinese, and then PEGASUS is imitated to construct the pre-training task, and then a new version of the mT5 model is trained. BERT (Bidirectional Encoder Representation from Transformers) is a pre-training model based on deep bidirectional language representation. The architecture and principles of the mT5-PEGASUS and BERT pre-training models can refer to the prior art, and this application will not go into details.

[0104] In some embodiments, reference Figure 6 , input the sample data into the first pre-trained model and the second pre-trained model respectively, use the sample data to fine-tune the first pre-trained model to obtain an elementary question-answer generation model, namely Model1_1; and use the sample data to fine-tune the second pre-trained model to obtain an elementary machine reading comprehension model, namely Model2_1.

[0105] In the first stage, there are two concepts involved, "pre-training" and "fine-tuning". Pre-training refers to building a model in advance and performing training tasks on the model. During this period, the parameters of the model are continuously optimized. When the parameters of the model meet the training requirements, the model parameters are saved to obtain a pre-trained model; fine-tuning refers to designing the downstream neural network structure by yourself on the basis of the pre-trained model trained by others, so as to apply your own data set (i.e. the aforementioned sample data) and tasks to the pre-trained model, and in the training process, the model parameters are transitioned to the processing links that are adapted to the characteristics of your own data set and tasks. The implementation methods of pre-training and fine-tuning of the model can refer to the corresponding existing technologies, and this application will not go into details.

[0106] (ii) Figure 7 As shown, in the second stage of the linkage training mechanism, the sample data is continued to be used to fine-tune Model1_1, and the first loss value of Model1_1 is calculated based on the sample data and the prediction results of Model1_1, and the first loss value is represented by loss1; at the same time, the description text in the sample data and the first question and the first answer predicted by Model1_1 are input into Model2_1, and Model2_1 predicts and outputs the first answer ′ based on the description text and the first question, and the second loss value of Model2_1 is calculated based on the first answer and the first answer ′, and the second loss value is represented by loss2.

[0107] Reference Figure 7 The second stage is to optimize the parameters of Model1_1 based on Model2_1. Therefore, the model parameters of Model1_1 are solidified in this stage. The first weighted loss value is calculated using loss1 and its corresponding first weight w1, and loss2 and its corresponding second weight w2. The first weighted loss value is expressed as Loss_1, that is, Loss_1 = w1*loss1 + w2*loss2. Then, Loss_1 is used to perform feedback training on Model1_1. By debugging and optimizing the parameters of Model1_1, the loss of Model1_1 is reduced. When the loss of Model1_1 drops below the threshold, the model parameters can be saved, and the advanced question-answer generation model, namely Model1_2, is obtained.

[0108] Among them, the sum of w1 and w2 is 1, and the values ​​of w1 and w2 are not limited. In some embodiments, for example, several groups of {weight 1, weight 2} can be pre-set based on experience, and the model can be debugged using several groups of {weight 1, weight 2} respectively, and a group of data {weight 1′, weight 2′} with the lowest model loss is selected as {w1, w2}.

[0109] (III) Figure 8 As shown, in the third stage of the linkage training mechanism, the sample data are respectively input into Model1_2 and Model2_1 to continue fine-tuning training, the third loss value of Model1_2 is calculated, the first loss value is represented as loss3, and the fourth loss value of Model2_1 is calculated, the fourth loss value is represented as loss4.

[0110] Reference Figure 8The third stage is to optimize the parameters of Model2_1 based on Model1_2. Therefore, the model parameters of Model1_2 are solidified in this stage. The second weighted loss value is calculated using loss3 and its corresponding third weight w3, and loss4 and its corresponding fourth weight w4. The second weighted loss value is expressed as Loss_2, that is, Loss_2=w3*loss3+w4*loss4. Then, Loss_2 is used to perform feedback training on Model2_1. By debugging and optimizing the parameters of Model2_1, the loss of Model2_1 is reduced. When the loss of Model2_1 drops below the threshold, the model parameters can be saved, and the advanced machine reading comprehension model, namely Model2_2, is obtained.

[0111] Among them, the sum of w3 and w4 is 1, and the values ​​of w3 and w4 are not limited. In some embodiments, for example, several groups of {weight 3, weight 4} can be pre-set based on experience, and the model can be debugged using several groups of {weight 3, weight 4} respectively, and a group of data {weight 3′, weight 4′} with the lowest model loss is selected as {w3, w4}.

[0112] After going through the above three stages, Model1_2 and Model2_2 are finally obtained. Furthermore, the following question-answer prediction and verification are performed using the text data to be tested, Model1_2 and Model2_2.

[0113] In some embodiments, Fig. 9 In the example of, the text data to be tested is first input into Model1_2, and Model1_2 extracts and generates the target question and the benchmark answer from the text data to be tested; then the text data to be tested and the target question are input into Model2_2 together, and Model2_2 searches for the predicted answer matching the target question from the text data to be tested; the confidence of the predicted answer output by Model2_2 is calculated, and it is determined whether the confidence is greater than or equal to a preset threshold; if the confidence is less than the preset threshold, it means that the accuracy and reliability of the predicted answer are low, and the predicted answer is discarded, that is, the predicted question and answer are not stored; if the confidence is greater than or equal to the preset threshold, it means that the predicted answer has a certain degree of credibility, and then the predicted answer is further detected to be consistent with the benchmark answer; if the predicted answer is inconsistent with the benchmark answer, the predicted answer is discarded; if the predicted answer is consistent with the benchmark answer, the matching pair of the target question and the predicted answer is written into the memory corresponding to the question and answer library.

[0114] That is to say, only when the confidence of the predicted answer meets the preset threshold requirement and the predicted answer is consistent with the benchmark answer, will the predicted answer verification be considered successful, and the target question + predicted answer will be allowed to be written into the question and answer database; if the confidence of the predicted answer is lower than the preset threshold, or the predicted answer is inconsistent with the benchmark answer, indicating that the predicted answer is unreliable, then the predicted answer verification is considered to have failed, the target question + predicted answer is not allowed to be stored, and the predicted answer is directly discarded.

[0115] In some embodiments, the discard rate of answers is counted, where discard rate = number of discarded answers / total number of predicted answers. If the discard rate does not exceed the threshold, it may be an individual failure case, for example, due to invalid or incorrect text data to be tested. If the discard rate is higher than the threshold, it may be due to insufficient accuracy of Model1_2 and Model2_2. In this case, according to the second and third stages of the aforementioned linkage training mechanism, the question and answer generation model and the machine reading comprehension model are parameterized and optimized to improve model accuracy.

[0116] In some embodiments, if only manual maintenance and data expansion of the question and answer database are relied upon, the knowledge coverage of the question and answer database may be limited, knowledge updates may not be timely, etc., which may be reflected in the appearance of the question and answer, where the smart device may give wrong answers, give replies to questions that it does not understand, etc. In this regard, open world texts on the Internet can be obtained as the source of the text data to be tested, so as to realize the expansion of the question and answer database using network resources.

[0117] In some embodiments, a crawler strategy can be formulated to crawl information resources on the network. The crawler strategy defines crawling rules to achieve crawling / filtering of information resources that meet the screening conditions on the network, such as hot news, current affairs, etc. Figuratively speaking, a web crawler is similar to a detector. It can simulate human behavior to "wander" around various websites, explore network data, and bring back information that meets the conditions, thereby realizing automatic data collection. Web crawlers are an important part of search engines. After the web crawler selects a target URL that meets the conditions, it adds the target URL to the crawler queue. The search engine extracts the URL from the crawler queue in order, downloads the web page corresponding to the URL to the local computer, and stores it in a specified path. The specified path is, for example, a preset downloaded web page library.

[0118] In some embodiments, the smart device reads the web page data from the specified path, extracts text data from the web page data, and preprocesses the text data to obtain the text data to be tested, and then stores the text data to be tested into a text library, so that the smart device can retrieve the text data to be tested from the text library when performing prediction and verification according to Model1_2 and Model2_2, thereby realizing data expansion of the question and answer library.

[0119] In some embodiments, the crawler strategy generally sets a timing mechanism, that is, the crawler action is performed once every preset time interval, so the text library is updated regularly. The controller detects that the text library is updated, obtains the newly stored text data to be tested from the text library, and Fig. 9 In the example process, the successfully verified question and answer matching pairs are stored in the question and answer database to expand the data of the question and answer database.

[0120] In some embodiments, after extracting text data from web page data captured by the crawler, the text data is preprocessed, and the preprocessing is not limited to sensitive word filtering, invalid text cleaning, data segmentation, etc. Among them, sensitive word filtering is to screen the sensitive words involved in the text, and the sensitive words can be directly deleted, or the sensitive words can be replaced with non-sensitive words with the same or similar meanings; invalid text cleaning is to filter out invalid text, and the invalid text is, for example, empty text, garbled text, etc., which are useless for expanding the question and answer library data; data segmentation is to divide a long text into several short texts, such as dividing the text into several sentences according to punctuation marks (such as commas, periods, semicolons, etc.), and storing each sentence as a text data to be tested. The preprocessing method of text data is not limited to the examples of this embodiment.

[0121] In some embodiments, since the amount of data generated by the entire network every moment is very large, the crawler's crawling target can be set in a targeted manner, for example, the crawler mainly crawls hot news and consultations released by mainstream media, thereby reducing the amount of data obtained by the crawler and preventing the question-and-answer library data expansion program from occupying too many processing resources of the smart device. The crawler strategy can be adaptively formulated according to the knowledge expansion requirements for the text library, such as the knowledge level covered, the knowledge field focused on, etc.

[0122] In some embodiments, Fig.10As shown, the bottom layer of the smart device may include a search engine, a preprocessing module and a memory for storing a text library. The search engine may include a strategy formulation module and a crawler module, and the strategy formulation module is used to define and save the first crawler strategy and the second crawler strategy; the crawler module is used to execute: first, the first crawler strategy is used to obtain and organize the hot search keywords generated at all times from the specified path, and the specified path is not limited to media platforms such as Weibo, Baidu, and hot search lists; then, the second crawler strategy is used to obtain target text data related to the hot search keywords from network resources, and specifically, the target text data can be captured from the specified mainstream media and websites according to the hot search keywords, such as some real-time news websites, entertainment news websites, Weibo, Baidu, Toutiao, etc.; the preprocessing module is used to preprocess the target text data, and the preprocessing is not limited to data cleaning, data segmentation, etc., and then the text data to be tested obtained after preprocessing is written into the text library. In this embodiment, the crawled text data focuses on the real-time hot search hotspots in the network world, ensuring that the knowledge in the text library keeps pace with the times and is automatically updated.

[0123] In some embodiments, the present application involves two categories of databases, one is a question and answer library for storing question + answer matching pairs, and the other is a text library for storing text data to be tested. The data in the question and answer library and the text library can be stored in the same memory or in different memories.

[0124] In some embodiments, when the question-and-answer library is expanded using the open world text on the Internet, the smart device should be connected to the Internet. The smart device can be connected to a wired network or wireless network such as WiFi, data traffic (such as 3G, 4G, 5G, etc.), hotspot network, etc. to ensure the effective execution of the crawler action.

[0125] In some embodiments, if the network connection of the display device is disconnected, the crawler action is temporarily interrupted. At this time, the timing can be started until the network connection is detected to be restored and then the timing is stopped. The disconnection time T currently recorded by the detection timer is S Whether the time threshold is exceeded. If the network is disconnected for a period of T S If the time threshold is exceeded, it means that the smart device has been disconnected for a long time. In order to avoid missing the hot search data generated by the online world during the period when the smart device is disconnected, the historical target text data matching the historical hot search keywords within the time range of [T1, T2] can be queried, where T1 represents the time when the smart device is disconnected from the network, and T2 represents the time when the smart device resumes network connection again. T2=T1+T S If the network is disconnected for a period of T S If the time threshold is not exceeded, it means that the smart device is restored to the Internet soon after being disconnected. During this period, the hot search keywords in the online world may not have been changed. In this case, there is no need to search the historical target text data, and the current real-time hot search keywords and their corresponding new target text data can be continued to be captured.

[0126] In the aforementioned technical solution, it is mentioned that when the smart device expands the data of the question and answer library, it supports two expansion modes, one is manual regular maintenance, and the other is automatic expansion of the open world text based on the network. The user can select or switch the expansion mode of the question and answer library according to the question and answer requirements and usage scenarios. In this regard, in some embodiments, the smart device can add a question and answer setting page, such as Fig.11 As shown, taking the case where the smart device is a smart phone as an example, the question and answer setting page of the smart phone includes inquiry information 111 and switch control 112. The inquiry information 111 is used to inquire whether to expand the question and answer library based on the network. In some application scenarios, the question and answer knowledge field that the user is interested in is relatively professional and not fully disclosed in the online world. It requires specific personnel to perform manual maintenance and data expansion. In this case, the user can turn off the switch control 112, that is, enable the conventional manual maintenance expansion mode; if the user is a network expert and often pays attention to popular information in the online world, the user is more inclined to turn on the switch control 112, that is, enable the expansion mode of automatically expanding the question and answer library based on the open world text of the network. The question and answer setting page may also include other setting options related to question and answer interaction, such as voice settings, sound settings, etc.

[0127] In some embodiments, if the smart device detects the user input operation of turning on the switch control 112, it further detects the current network status of the device. If the smart device is currently in a network connected state, it executes a crawler action to grab the target text data from the network world, stores it in the text library after preprocessing, and then controls Model1_2 and Model2_2 to obtain the newly stored text data to be tested from the text library to perform the prediction and verification process; if the smart device is currently not connected to the network, Fig.12 As shown, in the question and answer settings page, the user is prompted "The device is not currently connected to the network, please set up or check the network connection", and the Go to Settings control 121 is displayed. When the user triggers the Go to Settings control 121, the UI of the smart device jumps to the network settings page, and the user can set up the network connection in the network settings page, or check the device network connection.

[0128] In some embodiments, Fig.13An example of a structure configuration of an intelligent device is provided, wherein the intelligent device at least includes a controller 131, a memory 132, a sound collector 133, a sound player 134, and a communicator 135. The controller 131 is configured to execute the aforementioned answer processing method; the memory 132 is used to store data in the question and answer library (or can also be used to store data in the text library); the sound collector 133 is used to collect the user's voice information, including collecting the question sentence input by the user; the sound player 134 is used to broadcast the answer sentence; and the communicator 135 is used to connect to the server to realize the use of a web crawler to capture the required target text data. In this embodiment, the intelligent device only supports voice question and answer interaction.

[0129] In some embodiments, Fig.14 The second type of smart device is illustrated in the structural configuration, and the smart device at least includes a controller 131, a memory 132, a communicator 135, and a display 136. The controller 131 is configured to execute the aforementioned answer processing method; the memory 132 is used to store the data in the question and answer library (or can also be used to store the data in the text library); the communicator 135 is used to connect to the server to achieve the target text data required by the web crawler; the display 136 is used to display the user interface, including displaying questions and answers, the aforementioned question and answer setting page and the network setting page. In this embodiment, the smart device only supports UI question and answer interaction.

[0130] In some embodiments, Fig.15 The third type of intelligent device is shown in the following example. The intelligent device includes at least a controller 131, a memory 132, a sound collector 133, a sound player 134, a communicator 135, and a display 136. The controller 131 is configured to execute the aforementioned answer processing method; the memory 132 is used to store data in the question and answer library (or can also be used to store data in the text library); the sound collector 133 is used to collect the user's voice information, including collecting the question statements input by the user; the sound player 134 is used to play the sound, including broadcasting the answer statement; the communicator 135 is used to connect to the server to realize the use of a web crawler to capture the required target text data; the display 136 is used to display the user interface, including displaying questions and answers, the aforementioned question and answer setting page, and the network setting page. The user supports voice question and answer interaction and UI question and answer interaction.

[0131] The human-computer interaction and UI display interface provided in this application are only exemplary and shall be adapted to different types of smart devices. The structure and configuration of smart devices are not limited to Figure 13 to Figure 15Examples. The pre-trained models and their fine-tuning, mT5-PEGASUS models, BERT, loss-based model optimization training, confidence calculation methods and other contents involved in the embodiments of this application can all refer to the prior art, and this application will not go into details one by one. Based on the linkage training and prediction verification mechanism of the dual models of this application, the initial pre-trained model architecture of the promised generation model and the machine reading comprehension model is not limited to mT5-PEGASUS&BERT. Those skilled in the art can select the type of suitable pre-trained model according to actual requirements. When managing a text library, the crawler strategy can be flexibly set. By limiting different crawling rules, the crawler can feedback network data that meets the conditions. The conditions are not limited to timed crawlers, network sources, data volume, etc. The method of pre-processing text data before warehousing is not limited to the examples of this application.

[0132] In some embodiments, the present application further provides a computer storage medium, which may store a program. When the computer storage medium is located in a smart device, the program may include the program steps of the question-and-answer processing method configured by the aforementioned controller when executed. Among them, the computer storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0134] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. According to the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are intended to better explain the content of this disclosure, so that those skilled in the art can better use the embodiments.

Claims

1. A smart device, characterized in that: include: A memory configured to store matching pairs of questions and answers constructed after question-answering training; The controller is configured to execute: Use sample data to fine-tune the first pre-trained model to obtain an elementary question-answering generation model; Fine-tune the second pre-trained model using the sample data to obtain an elementary machine reading comprehension model; The primary question-answer generation model is trained using the sample data to calculate a first loss value of the primary question-answer generation model; the sample data includes a description text, a sample question, and a sample answer; the description text and the first question and the first answer output by the primary question-answer generation model are input into the primary machine reading comprehension model to calculate a second loss value of the primary machine reading comprehension model; a first weighted loss value is calculated using the first loss value and its corresponding first weight, the second loss value and its corresponding second weight; the parameters of the primary question-answer generation model are optimized according to the first weighted loss value to obtain an advanced question-answer generation model; Inputting the sample data into the elementary machine reading comprehension model and the advanced question-answer generation model respectively; Calculating a third loss value of the advanced question-answer generation model, and calculating a fourth loss value of the primary machine reading comprehension model; calculating a second weighted loss value using the third loss value and its corresponding third weight, the fourth loss value and its corresponding fourth weight; optimizing parameters of the primary machine reading comprehension model according to the second weighted loss value to obtain an advanced machine reading comprehension model; Inputting the text data to be tested into the advanced question-answer generation model to generate target questions and benchmark answers; Inputting the text data to be tested and the target question into the advanced machine reading comprehension model to obtain a predicted answer; Calculating the confidence of the predicted answer; If the confidence level is greater than or equal to a preset threshold, and the predicted answer is consistent with the reference answer output by the advanced question-answer generation model, the predicted answer verification is determined to be successful, and the matching pair of the target question and the predicted answer is written into the memory associated with the machine reading comprehension model.

2. The smart device according to claim 1, characterized in that: The controller is also configured to execute: Use the first crawler strategy to obtain hot search keywords; Using a second crawler strategy, obtaining target text data related to the hot search keyword from network resources; The target text data is preprocessed and saved as the text data to be tested.

3. The smart device according to claim 1, characterized in that: The controller is also configured to execute: If the confidence level is less than the preset threshold, or the predicted answer is inconsistent with the reference answer, the predicted answer is discarded.

4. The smart device according to claim 1, characterized in that: The controller is also configured to execute: If the smart device further includes a sound collector and a sound player, the first question of receiving a user's voice input through the sound collector is: Searching the memory for a first target matching pair to which the first question belongs, and controlling the audio player to broadcast an answer in the first target matching pair; and / or, If the smart device further includes a display, receiving a second question input by a user, and controlling the display to display the second question, The memory is searched for a second target matching pair to which the second question belongs, and the display is controlled to display an answer in the second target matching pair.

5. A question-answering processing method, characterized in that: The method comprises: Use sample data to fine-tune the first pre-trained model to obtain an elementary question-answering generation model; Fine-tune the second pre-trained model using the sample data to obtain an elementary machine reading comprehension model; The primary question-answer generation model is trained using the sample data to calculate a first loss value of the primary question-answer generation model; the sample data includes a description text, a sample question, and a sample answer; the description text and the first question and the first answer output by the primary question-answer generation model are input into the primary machine reading comprehension model to calculate a second loss value of the primary machine reading comprehension model; a first weighted loss value is calculated using the first loss value and its corresponding first weight, the second loss value and its corresponding second weight; the parameters of the primary question-answer generation model are optimized according to the first weighted loss value to obtain an advanced question-answer generation model; Input the sample data into the primary machine reading comprehension model and the advanced question-answer generation model respectively; calculate the third loss value of the advanced question-answer generation model, and calculate the fourth loss value of the primary machine reading comprehension model; calculate the second weighted loss value using the third loss value and its corresponding third weight, the fourth loss value and its corresponding fourth weight; optimize the parameters of the primary machine reading comprehension model according to the second weighted loss value to obtain the advanced machine reading comprehension model; Inputting the text data to be tested into the advanced question-answer generation model to generate target questions and benchmark answers; Inputting the text data to be tested and the target question into the advanced machine reading comprehension model to obtain a predicted answer; Calculating the confidence of the predicted answer; If the confidence level is greater than or equal to a preset threshold, and the predicted answer is consistent with the reference answer output by the advanced question-answer generation model, the predicted answer verification is determined to be successful, and a matching pair of the target question and the predicted answer is stored.

Images