Question and answer data collection method, device, medium and system for smart home linkage

Abstract

The embodiment of the present application discloses a method, device, medium and system for collecting question-answering data for smart home linkage, the method comprising: obtaining the state parameters of the smart home device; sending the state parameters of the smart home device to a cloud device, and receiving a first type of question sent by the cloud device, the first type of question being associated with a target scenario, the target scenario being determined by the cloud device based on the state parameters of the smart home device; pushing the first type of question to the user and collecting the user's reply statements to establish a first corpus; sending the first corpus to the cloud device so that the cloud device can train and update the question-answering model based on the first corpus. In this way, since the first corpus is associated with the scenario, the updated question-answering model not only has basic question-answering capabilities, but can also realize more accurate interactive question-answering in combination with the scenario, thereby increasing user stickiness and achieving the purpose of accurate question-answering in smart scenarios.

Description

Technical Field

[0001] The present application relates to the field of smart home technology, and in particular to a method, device, medium and system for collecting question-answer data for smart home linkage. Background Art

[0002] An intelligent dialogue system is a program or agent that uses artificial intelligence (AI) to engage in meaningful conversations or responses with humans in natural, conversational language. These systems utilize intelligent algorithms such as machine learning, deep learning, reinforcement learning, and transfer learning to enable machines to understand human language and communicate effectively with humans, understanding intent expressed in human language and executing tasks or providing responses.

[0003] In the relevant technology, there are two common models used in intelligent dialogue systems: pipeline models and end-to-end models. Pipeline models execute different modules sequentially to achieve dialogue goals. These modules are highly coupled and interdependent, resulting in poor flexibility and limited functionality, making them difficult to adapt to new domains. End-to-end models, on the other hand, encapsulate all modules and train them using supervised learning. This requires a large amount of training data and lacks data processing, resulting in the inability to provide answers to specific questions during overall training. This leads to limitations and low user engagement. Summary of the Invention

[0004] This application provides a method, device, medium and system for collecting question-and-answer data for smart home linkage, which not only has basic question-and-answer capabilities, but can also achieve more accurate interactive question-and-answer in combination with scenarios, increase user stickiness, and thus achieve the purpose of accurate question-and-answer in smart scenarios.

[0005] The technical solution of this application is achieved as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for collecting question-and-answer data for smart home linkage, which is applied to a terminal device. The method includes:

[0007] Get the status parameters of smart home devices;

[0008] Sending the state parameters of the smart home device to a cloud device, and receiving a first type of question sent by the cloud device, wherein the first type of question is associated with a target scenario, and the target scenario is determined by the cloud device based on the state parameters of the smart home device;

[0009] Pushing the first type of questions to users and collecting the users' answer statements to establish a first corpus;

[0010] The first corpus is sent to the cloud device, so that the cloud device trains and updates the question-answering model based on the first corpus.

[0011] In a second aspect, the embodiments of the present application further provide another method for collecting question-and-answer data for smart home linkage, which is applied to a cloud device and includes:

[0012] Receive status parameters of smart home devices sent by terminal devices;

[0013] Determining a target scenario according to the state parameters of the smart home device, and determining a first type of question from a first preset question set according to the target scenario;

[0014] Sending the first type of questions to the terminal device;

[0015] Receive the first corpus returned by the terminal device, and train and update the question-answering model based on the first corpus.

[0016] In a third aspect, an embodiment of the present application further provides a terminal device, which includes an acquisition unit, a first sending unit, a first receiving unit, and a corpus building unit, wherein:

[0017] The acquiring unit is configured to acquire status parameters of the smart home device;

[0018] The first sending unit is configured to send the status parameters of the smart home device to a cloud device;

[0019] The first receiving unit is configured to receive a first type of question sent by the cloud device, wherein the first type of question is associated with a target scenario, and the target scenario is determined by the cloud device according to the state parameter of the smart home device;

[0020] The corpus building unit is configured to push the first type of questions to the user and collect the user's answer sentences to build a first corpus;

[0021] The first sending unit is further configured to send the first corpus to the cloud device, so that the cloud device trains and updates the question-answering model based on the first corpus.

[0022] In a fourth aspect, an embodiment of the present application further provides a cloud device, which includes a second receiving unit, a determining unit, a second sending unit and a training unit, wherein:

[0023] The second receiving unit is configured to receive the status parameters of the smart home device sent by the terminal device;

[0024] The determining unit is configured to determine a target scenario according to the state parameters of the smart home device, and determine a first type of question from a first preset question set according to the target scenario;

[0025] The second sending unit is configured to send the first type of questions to the terminal device;

[0026] The second receiving unit is further configured to receive the first corpus returned by the terminal device;

[0027] The training unit is configured to train and update the question-answering model based on the first corpus.

[0028] In a fifth aspect, an embodiment of the present application further provides another terminal device, the terminal device including a first memory and a first processor, wherein:

[0029] a first memory, configured to store a computer program capable of running on the first processor;

[0030] The first processor is configured to execute the question-and-answer data collection method for smart home linkage as described in the first aspect when running the computer program.

[0031] In a sixth aspect, an embodiment of the present application further provides another cloud device, the cloud device including a second memory and a second processor, wherein:

[0032] a second memory, configured to store a computer program capable of running on the second processor;

[0033] The second processor is used to execute the question-and-answer data collection method for smart home linkage as described in the second aspect when running the computer program.

[0034] In the seventh aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program. When the computer program is executed by a first processor, it implements the question-and-answer data collection method for smart home linkage as described in the first aspect, or when the computer program is executed by a second processor, it implements the question-and-answer data collection method for smart home linkage as described in the second aspect.

[0035] In an eighth aspect, an embodiment of the present application provides a question-and-answer data collection system, which includes the terminal device described in the third aspect or the fifth aspect, the cloud device described in the fourth aspect or the sixth aspect, and the smart home device.

[0036] The embodiments of the present application provide a method, device, medium, and system for collecting question-answering data for smart home linkage, which obtains state parameters of smart home devices; sends the state parameters of the smart home devices to a cloud device, and receives a first type of question sent by the cloud device, wherein the first type of question is associated with a target scenario, and the target scenario is determined by the cloud device based on the state parameters of the smart home devices; pushes the first type of question to the user and collects the user's reply sentences to establish a first corpus; and sends the first corpus to the cloud device so that the cloud device trains and updates the question-answering model based on the first corpus. In this way, by establishing a first corpus based on the first type of questions associated with the target scenario and the corresponding user's reply sentences, the cloud device can update the question-answering model based on the first corpus, so that when training data is insufficient, the corpus obtained from daily conversations can be used as training data for updating the question-answering model. Therefore, the updated question-answering model not only has basic question-answering capabilities, but also can realize more accurate interactive question-answering in combination with scenarios, thereby increasing user stickiness and achieving the purpose of accurate question-answering in smart scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of an architecture based on a pipeline model provided in an embodiment of the present application;

[0038] Figure 2 A schematic diagram of an architecture based on an end-to-end model provided in an embodiment of the present application;

[0039] Figure 3 A flowchart of a method for collecting question-and-answer data for smart home linkage provided in an embodiment of the present application;

[0040] Figure 4 A flowchart of another method for collecting question-and-answer data for smart home linkage provided in an embodiment of the present application;

[0041] Figure 5 A schematic diagram of the structure of a question-and-answer data collection system provided in an embodiment of the present application;

[0042] Figure 6 A flowchart of another method for collecting question-and-answer data for smart home linkage provided in an embodiment of the present application;

[0043] Figure 7 A flowchart of another method for collecting question-and-answer data for smart home linkage provided in an embodiment of the present application;

[0044] Figure 8 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0045] Figure 9A schematic diagram of the structure of another terminal device provided in an embodiment of the present application;

[0046] Figure 10 A schematic diagram of the structure of a cloud device provided in an embodiment of the present application;

[0047] Figure 11 A schematic diagram of the structure of another cloud device provided in an embodiment of the present application;

[0048] Figure 12 A schematic diagram of the composition structure of another question-and-answer data collection system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It should be understood that the specific embodiments described herein are only used to explain the related applications and are not intended to limit the applications. It should also be noted that for ease of description, only the portions relevant to the related applications are shown in the drawings.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0051] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0052] It should be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0053] An intelligent dialogue system is a program or agent that uses artificial intelligence (AI) to engage in meaningful conversations or responses with humans in natural, conversational language. These systems utilize intelligent algorithms such as machine learning, deep learning, reinforcement learning, and transfer learning to enable machines to understand human language and communicate effectively with humans, understanding intent expressed in human language and executing tasks or providing responses.

[0054] Currently, intelligent dialogue systems can be divided into two categories: task-oriented and non-task-oriented. Task-oriented systems are task-oriented, designed to guide users to complete specific tasks; non-task-oriented systems primarily interact with humans and provide responses and entertainment. Task-oriented intelligent dialogue systems typically employ two approaches: pipeline and end-to-end. Their core concept is to identify and analyze the semantic similarity between input text and data in a corpus, and to achieve interaction through overall similarity matching.

[0055] See also Figure 1 , which shows a schematic diagram of an architecture based on a pipeline model provided by an embodiment of the present application. Figure 1 As shown, the basic architecture of a pipeline approach is provided. This architecture includes a human-computer interaction unit, a natural language understanding unit, a dialogue state tracking unit, a dialogue policy learning unit, and a natural language generation unit. During a conversation, the human-computer interaction unit obtains messages conveyed by the user. The natural language understanding unit then parses and maps these messages into predefined semantic slots and performs dialogue intent detection. The dialogue state tracking unit manages the input of each dialogue turn and the historical dialogue, outputting the current dialogue state to ensure the robustness of the dialogue system. The dialogue policy learning unit generates the next action based on the state tracker, and the natural language generation unit maps the selected action and generates a response. The main drawback of traditional pipeline approaches is the high degree of coupling between units (modules). The input of each unit depends on the output of another unit. When adjusting a unit or updating it with new data, all units need to be adjusted to achieve the global optimality. Furthermore, changing semantic slots and features requires extensive manual work.

[0056] With the advancement of deep learning technology, dialogue systems have also made great progress. Using large amounts of data to learn feature representation and response generation strategies has simplified the manual operations in dialogue systems. The most commonly used structure is the end-to-end generative model, which uses a module and interacts with structured external data. Figure 2 , which shows a schematic diagram of an architecture based on an end-to-end model provided by an embodiment of the present application. Figure 2 As shown in Figure 1, this paper presents the basic architecture of an end-to-end approach, specifically a network-based, end-to-end trainable task-oriented dialogue system. This basic architecture includes an intent network, belief tracking, a database operator, a policy network, and a generator network. The entire system learns a mapping from dialogue history to system responses, trained using an encoder-decoder model. However, its drawbacks include the requirement for a large amount of training data and the lack of robustness due to the neglect of syntactic information and exploration of grammatical relationships in the training data.

[0057] In related technologies, pipeline models execute different modules sequentially to achieve conversational goals. Modules are highly coupled and interdependent. When a module is adapted to a new environment or updated with new data, all other modules must be adjusted accordingly to ensure global optimization. Constructing corresponding semantic slots for transitions between scenarios requires significant manual effort, and the model lacks flexibility and limited functionality, making it difficult to adapt to new domains. End-to-end models encapsulate all modules and communicate with external structured data through interactive modules to complete conversational tasks. However, supervised learning training requires a large amount of annotated data. Acquiring specific conversational data in actual conversational projects, especially during the warm-up phase of a new domain, is extremely difficult and labor-intensive. Training data lacks exploration of syntactic information, making it difficult to find effective solutions for specific scenarios during overall training.

[0058] Based on this, an embodiment of the present application provides a data collection method for use with a terminal device. The basic concept of the method is as follows: obtaining state parameters of a smart home device; sending the state parameters of the smart home device to a cloud device, and receiving a first type of question sent by the cloud device, wherein the first type of question is associated with a target scenario, and the target scenario is determined by the cloud device based on the state parameters of the smart home device; pushing the first type of question to the user and collecting the user's response statements to establish a first corpus; and sending the first corpus to the cloud device so that the cloud device can train and update a question-answering model based on the first corpus.

[0059] The present application also provides a data collection method for use with a cloud device. The basic concept of the method is as follows: receiving status parameters of smart home devices from a terminal device; determining a target scenario based on the status parameters of the smart home devices, and determining a first category of questions from a first set of preset questions based on the target scenario; sending the first category of questions to the terminal device; receiving a first corpus returned by the terminal device, and training and updating a question-answering model based on the first corpus.

[0060] In this way, the terminal device establishes a first corpus through the first type of questions associated with the target scenario and the corresponding user's reply statements. The cloud device can update the question-answering model based on the first corpus. The question-answering model can provide reply statements or control instructions for smart home devices based on the user's reply statements, so that when the training data is insufficient, the corpus obtained from the daily conversation process can also be used as training data for updating the question-answering model. Therefore, the updated question-answering model not only has basic question-answering capabilities, but can also achieve more accurate interactive question-answering in combination with the scenario, thereby increasing user stickiness and achieving the purpose of accurate question-answering in smart scenarios.

[0061] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0062] In one embodiment of the present application, see Figure 3 , which shows a flow chart of a method for collecting question and answer data for smart home linkage provided by an embodiment of the present application. Figure 3 As shown, the method may include:

[0063] S101: Obtain status parameters of smart home devices.

[0064] S102: Send the status parameters of the smart home device to the cloud device, and receive the first type of questions sent by the cloud device.

[0065] It should be noted that in this embodiment of the application, the first type of question is associated with a target scenario, and the target scenario is determined by the cloud device based on the status parameters of the smart home device. In other words, when collecting question and answer data in this embodiment of the application, it is associated with the status of the smart home device, that is, it is a smart home linkage question and answer data collection method.

[0066] It should also be noted that the execution subject of the question-and-answer data collection method provided in the embodiment of the present application is a terminal device. In addition, the question-and-answer data collection method provided in the embodiment of the present application is applied to a question-and-answer data collection system. In the question-and-answer data collection system, in addition to the terminal device, it can also include a cloud device and a number of smart home devices. Among them, the terminal device can not only collect data through interaction with the user to establish a corpus, but also can realize the control of the smart home device and conduct daily conversations with the user through interaction with the user, the cloud device and the smart home device.

[0067] It should also be noted that the terminal device may include one or more smart terminal devices embedded in different areas of the home, and each of the one or more smart terminal devices is connected to the cloud device to send data to the cloud device or execute instructions issued by the cloud device. The terminal device is also connected to various smart home devices in the home to control the status of the smart home devices. In addition, the terminal device may also have a voice broadcast function and include a display screen, such as a smart screen.

[0068] It should also be noted that the terminal device can obtain the status parameters of each smart home device in real time (or periodically or at specific times, or even when certain preset conditions are met) and send them to the cloud device. For example, the status parameters of the smart home device may include: the switch status of the door lock, the temperature set by the air conditioner, the indoor temperature measured by the temperature sensor, the indoor humidity measured by the humidity sensor, the temperature inside the refrigerator, the switch status of the curtains and lights, etc. In this way, after receiving the status parameters of the smart home device, the cloud device can determine whether the current scene meets a target scene based on the status parameters of the smart home device, and then send the first type of questions associated with the target scene to the terminal device.

[0069] For example, the target scene may include a "return home" scene, a "sleep" scene, a "study" scene, etc. For example, when it is determined based on the status parameters of the smart home device that the door lock is unlocked, the target scene may be determined to be a "return home" scene. At this time, the cloud device sends the first type of questions related to the "return home" scene to the terminal device. In this case, the first type of questions may include "Do I need to turn on the air conditioner?", "Do I need to open the curtains?", etc.

[0070] It should also be noted that, in the embodiment of the present application, the first type of questions associated with the target scenario usually carry the scenario label of the target scenario, so that the cloud device can quickly determine the first type of questions that need to be sent.

[0071] In addition, the terminal device can also obtain the user's voice instructions in real time, and convert the user's voice instructions into text information and send it to the cloud device. The cloud device can also parse the text information to determine whether the text information is associated with the target scene. For example, when the user's voice instruction is "I'm home", the cloud device can also determine that the current scene is the "home" scene and send the first type of questions under the "home" scene to the terminal device. In other words, the embodiment of the present application can not only obtain the status parameters of the smart home device and send them to the cloud device, so that the cloud device can determine the target scene and send questions based on them, but also determine the target scene and send questions based on the user's voice instruction.

[0072] S103: Push the first type of questions to the user and collect the user's answer sentences to establish a first corpus.

[0073] It should be noted that after receiving the first type of questions sent by the cloud device, the terminal device pushes the first type of questions to the user and collects the user's response statements. In the embodiment of the present application, the method of pushing the first type of questions to the user may include: pushing the first type of questions to the user by voice broadcast, or pushing the first type of questions to the user by screen display.

[0074] It should also be noted that since the first type of questions are related to the target scenario, that is, they are closely related to the current scenario, they need to be pushed to the user in a highly real-time manner. Therefore, the embodiment of the present application preferably uses voice broadcast to push the first type of questions to the user, so that the user can listen to the first type of questions broadcast by the terminal device in real time and make corresponding responses, so that the terminal device can quickly collect the user's response statement.

[0075] In some embodiments, collecting the user's reply statement may include: determining the user's reply statement from two preset options; wherein the two preset options include "OK" and "No need".

[0076] It should be noted that when the first type of questions are pushed to the user, the questions pushed in the embodiment of the present application are usually affirmative or negative questions. In this case, the user only needs to answer "OK" or "No" (or "OK" or "No" or other affirmative or negative answer statements).

[0077] In some embodiments, after collecting the user's reply statement, the method may further include:

[0078] Convert the response statement into text information;

[0079] Send text messages to cloud devices and receive reply statements or control instructions returned by the cloud devices;

[0080] Push the reply statement to the user; or send the control instruction to the smart home device, the control instruction is used to instruct the smart home device to control its own status.

[0081] It should be noted that after collecting the user's response, the terminal device converts the response into text information, such as Chinese characters, and saves the pushed question and the user's response in the terminal device's local memory. The push question and the user's response are then sent to the cloud device within a certain period of time. The cloud device generates a corresponding response statement or control instruction based on the received text information and sends it to the terminal device.

[0082] In an embodiment of the present application, the reply statement may include information about the hardware status of the smart home device, for example: the current temperature is 26 degrees, or the current time is 9 o'clock in the evening, etc. Such reply statements can be directly broadcast or displayed in text form on the screen of the terminal device; control instructions are usually used to change the status parameters of the smart home device. At this time, the terminal device needs to send the control instructions to the smart home device so that the smart home device can control its own status, such as opening the curtains, or adjusting the air conditioner temperature to 23 degrees.

[0083] In this way, the embodiment of the present application can establish a first corpus based on the first type of questions and the reply statements replied by the user. Specifically, in some embodiments, the establishment of the first corpus can include:

[0084] According to the first type of questions and the corresponding answer sentences, a first dialogue response sequence is formed;

[0085] The first dialogue response sequence is labeled with a scene label of the target scene to obtain a first corpus.

[0086] It should be noted that when building the first corpus, a first dialogue response sequence is first formed based on the first type of questions and their corresponding responses. The first dialogue response sequence is then labeled with the target scenario's scene label to generate the first corpus, which is then stored in the local memory of the terminal device. In other words, the first corpus consists of a plurality of first dialogue response sequences labeled with the target scenario's scene label.

[0087] It should also be noted that the first corpus may include not only the first type of questions and corresponding reply statements, but also the reply statements or control instructions returned by the cloud device after the reply statements are sent to the cloud device; as well as the content of the user's possible further conversations with the terminal device.

[0088] S104: Send the first corpus to the cloud device, so that the cloud device trains and updates the question-answering model based on the first corpus.

[0089] It should be noted that the embodiment of the present application collects corpus data during daily conversations between the terminal device and the user to obtain the first corpus. That is to say, during the conversation, the first corpus will be continuously expanded. The embodiment of the present application can send the first corpus to the cloud device at every preset time period, or send the first corpus to the cloud device when the total amount of the first conversation response sequence in the first corpus reaches a preset threshold, or can also send the first corpus to each cloud device in real time. After receiving the first corpus, the cloud device trains and updates the question-answering model based on the first corpus. The question-answering model can be used to provide a reply statement to the terminal device based on the user's reply statement or provide control instructions for smart home devices. In addition, the question-answering model can also analyze the status parameters of the smart home device, determine the target scenario and the corresponding first category of questions, and automatically analyze the user's habit characteristics to generate more target scenarios and corresponding first category of questions.

[0090] Since the first corpus is generated through daily conversations with users and the question-answering model is updated based on the first corpus, the model can accurately analyze the current user's behavioral habits, so that it can push and answer questions more accurately to users in different target scenarios, with greater accuracy.

[0091] In some embodiments, the method may further include:

[0092] Receive the second type of questions sent by the cloud device; wherein the second type of questions are not related to the target scenario and are randomly sent by the cloud device;

[0093] Push the second type of questions to users and collect their responses to establish a second corpus;

[0094] Send the second corpus to the cloud device.

[0095] It should be noted that in this embodiment of the present application, in addition to receiving the first type of questions associated with the target scenario, the terminal device also receives the second type of questions that are not associated with the target scenario, and the second type of questions are randomly sent by the cloud device. In other words, the second type of questions are not associated with the target scenario, do not carry the scene tag of the target scenario, and can be randomly pushed to the user during fragmented time.

[0096] It should also be noted that for the second type of questions, the method of collecting the user's reply statements and the operations performed after collecting the user's reply statements are consistent with the above description of the first type of questions. The difference is that when the second type of questions are pushed to the user, random questions are pushed in a fragmented time period and have no close relationship with the current scenario. Therefore, when pushing the second type of questions to the user, there is a stronger randomness. Therefore, the embodiment of the present application preferably uses text display to push the second type of questions to the user, so as to obtain the second type of questions and corresponding reply statements in a fragmented time period.

[0097] In this way, the embodiment of the present application can establish a second corpus based on the second type of questions and the reply statements replied by the user. Specifically, in some embodiments, the establishment of the second corpus can include:

[0098] According to the second type of questions and the corresponding answer sentences, a second dialogue response sequence is formed;

[0099] A second corpus is obtained according to the second dialogue response sequence.

[0100] It should be noted that when building the second corpus, a second dialogue response sequence is first constructed based on the second type of questions and their corresponding responses, thereby generating the second corpus, which is then stored in the local memory of the terminal device. In other words, the second corpus consists of several second dialogue response sequences. Unlike the first corpus, the second dialogue response sequences in the second corpus do not have scenario tags that indicate the target scenario. In other words, the second corpus is a more generalized basic corpus.

[0101] It should also be noted that the second corpus may not only include the second type of questions and the corresponding reply statements, but also include the reply statements or control instructions returned by the cloud device after the reply statements are sent to the cloud device; as well as the content of the user's possible further conversations with the terminal device.

[0102] It should be noted that the embodiment of the present application also collects corpus data during daily conversations between the terminal device and the user to obtain a second corpus. That is, during the conversation, the second corpus will be continuously expanded. The embodiment of the present application can send the second corpus to the cloud device at preset time intervals, or send the second corpus to the cloud device when the total number of second dialogue response sequences in the second corpus reaches a preset threshold, or can also send the second corpus to each cloud device in real time. After receiving the second corpus, the cloud device can train and update the question-answering model based on the second corpus.

[0103] In addition, since the embodiment of the present application can obtain the first corpus and the second corpus, the first corpus and / or the second corpus can be sent to the cloud device simultaneously or at different times for training and updating the question-answering model.

[0104] Since the second corpus is generated through daily conversations with users and the question-answering model is updated based on the second corpus, the question-answering model has stronger generalization ability and greater universality.

[0105] In some embodiments, the method may further include:

[0106] Receive user's voice data and convert the voice data into text information;

[0107] Send the status parameters and text information of the smart home device to the cloud device, and receive the reply statement or control command returned by the cloud device;

[0108] Push the reply statement to the user; or send a control instruction to the smart home device, where the control instruction is used to instruct the smart home device to control its own status.

[0109] It should be noted that in the embodiments of the present application, the terminal device can not only proactively push questions to the user, but also, even when not proactively pushing questions to the user, receive the user's voice data, convert the voice data into text information, and send the smart home device's status parameters and text information to the cloud device. The cloud device generates a corresponding reply statement or control instruction based on the text information and the smart home device's status parameters, and sends it to the terminal device. This allows the terminal device to respond appropriately to the user's voice data or control the status of the smart home device based on the scenario information.

[0110] In addition, the voice data initiated by the user and the corresponding reply statements or control instructions can also be sent to the cloud device as collected corpus data for training and updating the question-answering model.

[0111] In some embodiments, sending the control instruction to the smart home device may include:

[0112] Send control commands directly to smart home devices; or

[0113] Based on the control instruction, after receiving the user's confirmation instruction operation, the control instruction is sent to the smart home device.

[0114] It should be noted that when sending a control command to a smart home device, the command can be sent directly, or after receiving a confirmation instruction from the user. That is, before sending a control command to a smart home device, the user can be asked for confirmation first to avoid misoperation.

[0115] In addition, in some embodiments, before obtaining the status parameters of the smart home device, the method may further include:

[0116] The status parameters of the smart home devices are collected and pushed in a preset manner; wherein the preset manner includes a voice playback manner and / or a screen display manner.

[0117] It should be noted that embodiments of the present application can also collect status parameters of smart home devices in real time or periodically, and announce the status parameters of the smart home devices through voice broadcast or screen display. For example, the current indoor temperature, humidity, and other information can be announced through voice broadcast at regular intervals, or displayed on the screen of the terminal device. This allows users to directly obtain certain status information without asking, and allows users to actively initiate voice data to perform a series of hardware control operations, such as heating a water heater, turning on or off an air conditioner or fresh air system, and turning lights and curtains on and off.

[0118] This embodiment provides a data collection method, which is applied to a terminal device, by obtaining state parameters of a smart home device; sending the state parameters of the smart home device to a cloud device; and receiving first-category questions sent by the cloud device, wherein the first-category questions are associated with a target scenario, and the target scenario is determined by the cloud device based on the state parameters of the smart home device; pushing the first-category questions to a user and collecting the user's response sentences to establish a first corpus; and sending the first corpus to the cloud device so that the cloud device trains and updates a question-answering model based on the first corpus. In this way, by establishing a first corpus based on the first-category questions associated with the target scenario and the corresponding user's response sentences, the cloud device can update the question-answering model based on the first corpus. In the case of insufficient training data, the cloud device can also use the corpus obtained from daily conversations as training data for updating the question-answering model. As a result, the updated question-answering model not only has basic question-answering capabilities but also can achieve more accurate interactive question-answering in combination with scenarios, thereby increasing user stickiness and achieving the goal of accurate question-answering in smart scenarios. In addition, the embodiment of the present application also obtains a second corpus that is independent of the scenario, which is used to update the question-answering model, so that the model can be more generalized and universal; the embodiment of the present application also collects the status information of smart home devices, which can not only push the status information of smart home devices, but also perform a series of control operations on smart hardware devices according to the user's voice commands.

[0119] In another embodiment of the present application, see Figure 4 , which shows a flow chart of another smart home linkage question and answer data collection method provided by an embodiment of the present application. Figure 4 As shown, the method may include:

[0120] S201: Receive status parameters of smart home devices sent by a terminal device.

[0121] S202: Determine a target scenario based on the status parameters of the smart home device, and determine a first type of question from a first preset question set based on the target scenario.

[0122] S203: Send the first type of questions to the terminal device.

[0123] S204: Receive the first corpus returned by the terminal device, and train and update the question-answering model based on the first corpus.

[0124] It should be noted that the execution subject of the data collection method provided in the embodiment of the present application is a cloud device, such as a cloud server, a cloud platform, etc. In addition, the question-and-answer data collection method provided in the embodiment of the present application is applied to a question-and-answer data collection system. In the question-and-answer data collection system, in addition to the cloud device, it can also include at least one terminal device and several smart home devices. Among them, the cloud device can be connected to at least one terminal device, receive various data sent by the terminal device, and send various data to the terminal device, such as reply statements or control instructions, so that the terminal device can realize dialogue interaction with the user and control of the smart home device.

[0125] It should also be noted that after receiving the state parameters of the smart home devices from the terminal device, the cloud device determines the target scenario based on the state parameters of the smart home devices, for example, determining the current scenario as "returning home" based on the unlocking of the entrance door lock. Then, based on the target scenario, the cloud device determines a first category of questions from the first question set, sends the first category of questions to the terminal device, and then receives a first corpus returned by the terminal device. The question-answering model is trained and updated based on the first corpus. The first corpus may include a plurality of first dialogue response sequences consisting of the first category of questions and corresponding user responses, and the first dialogue response sequences have a scenario label of the target scenario.

[0126] In some embodiments, the method may further include:

[0127] In the preset fragmented time period, randomly select the second type of questions from the second preset question set;

[0128] Sending the second type of questions to the terminal device;

[0129] Receive the second corpus returned by the terminal device, and train and update the question-answering model based on the second corpus.

[0130] It should be noted that in the embodiment of the present application, the cloud device will not only send the first type of questions to the terminal device based on the status parameters of the smart home device sent by the terminal device, but will also randomly select the second type of questions from the second preset question set during a preset fragmented time period and send them to the terminal device, and receive the second corpus returned by the terminal device for training and updating the question-answering model.

[0131] That is to say, the corpus used to train and update the question-answering model may include not only a first corpus related to the scenario, but also a generalized second corpus that is independent of the scenario.

[0132] In some embodiments, the first preset question set is different from the second preset question set, and the method may further include:

[0133] Determine at least one question to be pushed in real time, and mark the at least one question to be pushed in real time with a scenario tag and a hardware trigger condition, to construct a first preset question set; or,

[0134] A fragmented interaction mode is set according to the status parameters of the smart home device, at least one question to be randomly pushed is generated, and a second preset question set is constructed according to the at least one question to be randomly pushed.

[0135] It should be noted that in the embodiment of the present application, the first preset question set and the second preset question set are different. The first preset question set can be constructed by determining at least one question to be pushed in real time, and marking each question with a scenario tag and a hardware trigger condition to form the first preset question set. In other words, the first preset question set includes at least one question to be pushed in real time, and each question to be pushed in real time is marked with a scenario tag and a hardware trigger condition.

[0136] The second set of preset questions may be constructed by setting a fragmented interaction mode according to the state parameters of the smart home device, generating at least one question to be randomly pushed, and constituting the second set of preset questions. In other words, the second set of preset questions includes at least one question to be randomly pushed.

[0137] In addition, the first preset question set may include, in addition to at least one question to be pushed in real time, a corresponding preset answer statement. The second preset question set may include, in addition to at least one question to be pushed randomly, a corresponding preset answer statement.

[0138] In some embodiments, determining the first type of questions from the first preset question set may include:

[0139] Searching for a target question from the first preset question set, which is marked with a scene tag indicating a target scene and for which the state parameters of the smart home device meet the hardware trigger condition of the target scene;

[0140] The target question found in the query is determined as the first type of question.

[0141] It should be noted that the first type of questions are determined from the first preset question set, and the target questions in the first preset question set that are marked with scene tags indicating the target scene and whose status parameters of the smart home device meet the hardware trigger conditions of the target scene are determined as the first type of questions.

[0142] In some embodiments, determining the second type of questions from the second preset question set may include:

[0143] In the preset fragmented time period, randomly select a question from the second preset question set;

[0144] The randomly selected questions are identified as the second type of questions.

[0145] It should be noted that the second type of questions are determined from the second preset question set. During the preset fragmented time period, questions are randomly selected from the second preset question set and determined as the second type of questions.

[0146] In some embodiments, the method may further include:

[0147] Acquire a training sample set, wherein the training sample set includes at least one dialogue response sequence sample, and the dialogue sequence sample includes questions and answers from the first preset question set and / or the second preset question set;

[0148] Performing text feature extraction on at least one dialogue response sequence sample in the training sample set to generate a word vector feature matrix corresponding to the at least one dialogue response sequence sample;

[0149] If the dialogue response sequence sample is associated with the state parameters of the smart home device, the word vector feature matrix corresponding to the dialogue response sequence is marked with a hardware label;

[0150] A preset model is trained using at least one dialogue response sequence sample and the corresponding word vector feature matrix, and the trained model is determined as a question-answering model.

[0151] It should be noted that, before updating the question-answering model, embodiments of the present application also first determine an initial question-answering model based on the training sample set. Specifically, the questions and answers from the first preset question set and / or the second preset question set can be combined into at least one dialogue response sequence sample. It will be appreciated that the dialogue response sequence sample from the first preset question set has a scene label for the target scenario, while the dialogue response sequence sample from the second preset question set does not have a scene label for the target scenario.

[0152] Text feature extraction is performed on the dialogue response sequence samples in the training sample set to generate a word vector feature matrix corresponding to the dialogue response sequence samples. For example, a text feature extraction function is used to extract features from the question and answer separately, and each word that appears is treated as a column of features to form a vocabulary; and a term frequency (TF) feature matrix is ​​formed based on the frequency of occurrence of each character or word in the question and answer sentences, and the vocabulary is multiplied by the TF feature matrix to obtain a word vector feature matrix. Those skilled in the art can perform feature extraction on the dialogue response sequence in any feasible manner known in the art, and this embodiment of the present application is not specifically limited to this.

[0153] The word vector feature matrix associated with the state parameters of the smart home device is marked with the relevant hardware label, and then the preset model is trained using at least one dialogue response sequence sample and the corresponding word vector feature matrix, and the trained model is determined as the question-answering model, that is, the initial question-answering model. In addition, in the embodiment of the present application, the preset model is preferably a recurrent neural network (RNN) model. Those skilled in the art can train the question-answering model in any feasible manner known in the art, and the embodiment of the present application does not specifically limit this.

[0154] In some embodiments, the method may further include:

[0155] Performing text feature extraction on at least one dialogue response sequence in the first corpus and / or the second corpus to generate a word vector feature matrix corresponding to the at least one dialogue response sequence;

[0156] If the dialogue response sequence is associated with the state parameters of the smart home device, the word vector feature matrix corresponding to the dialogue response sequence is marked with a hardware label;

[0157] The preset model is trained and updated using at least one dialogue response sequence and the corresponding word vector feature matrix to obtain a question-answering model.

[0158] It should be noted that after receiving the first corpus and / or the second corpus sent by the terminal device, the cloud device updates the question model based on the first corpus and / or the second corpus. Specifically, text feature extraction can be performed on at least one dialogue response sequence in the first corpus and / or the second corpus to generate a word vector feature matrix corresponding to at least one dialogue response sequence; hardware labels can be marked on the word vector feature matrix that has an association relationship with the status parameters of the smart home device; finally, the preset model is trained and updated using the at least one dialogue response sequence and the corresponding word vector feature matrix to obtain a question-answering model, that is, a question-answering model that has been updated based on the first corpus and / or the second corpus.

[0159] That is to say, the update of the question-answering model in the embodiment of the present application is to update the question-answering model using at least one dialogue response sequence from the first corpus and / or the second corpus and the corresponding word vector feature matrix, and determine the updated question-answering model as the question-answering model.

[0160] In some embodiments, the method may further include:

[0161] Receive text messages sent by terminal devices;

[0162] Use the question-answering model to process text information and generate reply sentences or control instructions;

[0163] Send the reply statement or control instruction to the terminal device.

[0164] It should be noted that after receiving the text information sent by the terminal device, the cloud device can use the question-answering model to process the text information, thereby generating corresponding reply statements or control instructions and sending them to the terminal device.

[0165] It should also be noted that there are two possible scenarios for text messages sent by terminal devices. The first scenario may include: after the cloud device sends the first or second type of question to the terminal device, the terminal device converts the text message based on the user's reply; the second scenario may include: the terminal device converts the voice data directly initiated by the user into a text message, that is, before the terminal device actively pushes the question to the user.

[0166] For the first scenario, a corresponding reply or control instruction is generated based on the question sent and the user's response. For example, if the question pushed to the user is "Do you need to open the curtains?" and the user responds "Yes," a control instruction to open the curtains can be generated. Another example is, if the question pushed to the user is "Do you want to know the current indoor temperature?" and the user responds "Yes," a reply related to the current indoor temperature can be generated, such as "The indoor temperature is currently 26 degrees."

[0167] In the second case, corresponding reply instructions or control instructions are generated based on the status parameters of the smart home device. For example, if a user in the living room utters the voice data "Set the air conditioner to 24 degrees," the current scene may be related to the living room, so a control instruction can be generated to set the air conditioner in the living room to 24 degrees, without setting the air conditioner temperature in the bedroom or other rooms. In addition, the user's location can be determined by infrared sensors or the location of specific data collection devices interacting with the user.

[0168] In some embodiments, processing text information using a question-answering model to generate a reply statement or control instruction may include:

[0169] Perform semantic intent analysis on text information through the question-answering model to obtain analysis results;

[0170] If the analysis result is an inquiry intent, then determine candidate reply sentences corresponding to the text information, and determine a reply sentence from the candidate reply sentences;

[0171] If the analysis result is an imperative intent, the hardware tag corresponding to the text information is determined, and a control instruction is generated based on the hardware tag.

[0172] It should be noted that when processing text information to generate reply statements or control instructions, the embodiments of the present application can also perform semantic intent analysis on the text information through a question-answering model to determine whether the analysis result is an inquiry intent or an imperative intent.

[0173] Inquiry intent indicates that the user intends to inquire about the status information or other questions of smart home devices, such as the current temperature, what music is playing, etc. For inquiry intent, first determine the candidate reply sentences corresponding to the text information, and then determine the reply sentence from the candidate reply sentences.

[0174] For example, if the text message is "What is the current temperature of the air conditioner?", it can be seen that the text message indicates an inquiry about the current temperature of the air conditioner, which is an interrogative intention. The reply at this time should be "The current temperature of the air conditioner is XX degrees." The cloud device will send the reply statement for replying to the current temperature of the air conditioner to the terminal device, and the terminal device will push the reply statement, such as voice broadcast or display on the screen: The current temperature of the air conditioner is 26 degrees.

[0175] If the text message is "Where is the capital of China", it can be seen that the text message is an inquiry intention, but it is irrelevant to the status parameters of the smart home device. The reply at this time should be "The capital of China is Beijing". The cloud device will send the reply statement used to answer the question to the terminal device, and the terminal device will push the reply statement, such as voice broadcast or display on the screen: The capital of China is Beijing.

[0176] An imperative intent indicates that the user intends to directly perform a certain action. For an imperative intent, the hardware tag corresponding to the text information is first determined, and then the corresponding control instruction is generated based on the hardware tag.

[0177] For example, if the text message is "Turn off the air conditioner," it can be seen that the text message is a command intent. At this time, the cloud device sends the control instruction for turning off the air conditioner to the terminal device, and the terminal device sends the control instruction to the air conditioner to turn it off. At the same time, a voice broadcast can also be performed: the air conditioner has been turned off.

[0178] In some embodiments, after obtaining the question-answering model, the method may further include:

[0179] Generate the first type of supplementary questions and the second type of supplementary questions through the dialogue model; the first type of supplementary questions carries the scene label;

[0180] Updating the first set of preset questions based on the first type of supplementary questions; and

[0181] The second preset question set is updated according to the second category of supplementary questions.

[0182] It should be noted that after the question-answering model is updated in the embodiment of the present application, the question-answering model will also generate a new set of questions based on its own update, namely the first type of supplementary questions and the second type of supplementary questions, and the first type of supplementary questions carry scenario labels and are used to update and expand the first preset question set; the second type of supplementary questions do not carry scenario labels and are generalized question types, which are used to supplement the second preset question set.

[0183] This embodiment provides a data collection method, applied to a cloud device, that receives status parameters of smart home devices sent by a terminal device; determines a target scenario based on the smart home device status parameters, and based on the target scenario, determines a first category of questions from a first preset question set; sends the first category of questions to the terminal device; receives a first corpus returned by the terminal device, and trains and updates a question-and-answer model based on the first corpus. In this way, the cloud device updates the question-and-answer model based on the first corpus. The question-and-answer model can provide a response statement or control instructions for the smart home device based on the user's response statement. Because the first corpus is associated with the target scenario, the updated question-and-answer model can achieve the goal of accurate question-and-answer in the smart scenario, increase user stickiness, and improve the robustness and robustness of the system to which the cloud device belongs, thereby increasing user stickiness. In addition, the cloud device also updates the question-and-answer model based on a second corpus that is independent of the scenario, making the model more generalized and universally applicable. Furthermore, because hardware tags are added during the training and updating of the question-and-answer model to associate the conversation with the smart home scenario, the generated responses can be classified into two types: response statements and control instructions, thereby achieving different purposes such as daily conversations or controlling the status of smart home devices, thereby optimizing the user experience.

[0184] In another embodiment of the present application, the question-answer data collection method for smart home linkage provided by the embodiment of the present application can also be based on Figure 5 The question-answering data collection system shown in FIG (also called a "dialogue system") is implemented. Figure 5 , which shows a schematic diagram of the composition structure of a question-answer data acquisition system 50 provided in an embodiment of the present application, such as Figure 5 As shown, the question-and-answer data collection system 50 may include a terminal device (also referred to as a "smart terminal") 80, a cloud device (also referred to as a "cloud platform," "data center," etc.) 100, and several smart home devices. Specifically, the terminal device 80 may include a data collection and interaction module, a smart home module, and a scenario response module, while the cloud device 100 may include a language understanding module (also referred to as a language processing module) and a self-learning module. The data collection method provided in the present embodiment will be described in detail below, combining the various components of the data collection system 50 and their specific functional implementations.

[0185] The data collection and interaction module is used to receive and push questions sent by the cloud device 100. The pushed questions include questions and answers about the status of smart home devices in the scene. The user only needs to make a "yes / no" selection. The terminal device 80 will save the pushed questions and the user's selection, execute the reply, and / or perform corresponding control operations on the smart home devices. For example, if the pushed question is "Do you want to know the current indoor temperature?", when the user replies "yes", the reply can be executed through voice, such as voice broadcast "The current indoor temperature is 26 degrees Celsius"; or through screen display, such as displaying "The current indoor temperature is 26 degrees Celsius" on the smart screen of the terminal device 80. Similarly, the question can also be pushed in the form of voice or text. In this way, the pushed questions and the user's answers are saved in the local storage of the terminal device, and the basic dialogue corpus in the home scene can be obtained. This basic dialogue corpus is sent to the cloud device 100 as the data basis for self-learning.

[0186] In addition, the conversations in the home scene have a strong correspondence with the areas, and the ways of pushing questions can be divided into real-time push and random push. For the real-time push method, it can be triggered by the actions of smart home devices, for example: when the user returns home, he opens the door or the user sends a voice message "I'm home", the real-time question broadcast in the "home" scene is immediately triggered, and the interactive form of voice broadcast can be used to ask the user whether to turn on the lights, whether to turn on the air conditioner, etc., and the user gives feedback by answering "OK / No"; or when the user enters the study or the user sends a voice message "I study", the real-time question broadcast in the learning scene is immediately triggered, and the interactive form of voice broadcast can be used to ask the user whether to change the lighting mode, etc., and the user gives feedback by answering "OK / No"; establish a basic dialogue corpus for specific scenarios (i.e., the first corpus in the aforementioned embodiment) and save it.

[0187] It should also be noted that there can be multiple terminal devices 80, which can be respectively set in the living room, bedroom, study, kitchen, etc.; or there can be only one terminal device 80. The embodiment of the present application can also determine the specific area where the user is located by using an infrared sensor device.

[0188] For the random push method, questions can be randomly pushed to the smart screen of the terminal device, and data can be collected by utilizing the user's fragmented time to build a more generalized basic corpus of home scenes (i.e., the second corpus in the aforementioned embodiment) and save it.

[0189] The data collection and interaction module sends the stored first corpus and second corpus to the cloud device 100 .

[0190] The data collection and interaction module is also used for voice and text conversion. Through this module, the voice information input by the user can be converted into text information such as Chinese characters, and stored and transmitted.

[0191] The smart home module is used to collect status parameters of smart home scenes in real time, such as water heater water temperature, air quality and humidity, refrigerator internal temperature, curtain and light switch status, etc.

[0192] The scene response module collects the status of smart home devices in real time through the smart home module. It can not only broadcast the status through voice, but also perform a series of hardware control operations based on the input voice, such as heating the water heater, turning on or off the air conditioner or fresh air system, turning on and off lights and curtains, etc.

[0193] On the one hand, the self-learning module in the cloud device 100 automatically analyzes the second corpus, fine-tunes and updates the question-answering model (the question-answering model before the update can be called the basic network model), and performs targeted optimization on the question-answering model; on the other hand, when the data acquisition and interaction module saves the first corpus, it will also mark the first dialogue response sequence in the first corpus with a scene label according to the different smart home device scenarios. The self-learning module then performs question-answering analysis on the first dialogue response sequence marked with the scene label and refines the question-answering model, thereby achieving the purpose of accurate question-answering in smart scenarios. The language understanding module in the cloud device 100 also extracts and classifies the user's input speech (i.e., the text information sent by the data acquisition and interaction module) based on the aforementioned question-answering model, and performs slot filling. At the same time, it uses syntactic analysis to detect intent and perform similarity matching with relevant entries in the external knowledge base.

[0194] That is to say, when obtaining the first corpus and / or the second corpus, the embodiment of the present application fully combines the specific scenario and the smart home status, and sends such a first corpus and / or the second corpus to the cloud device 100. After the cloud device 100 trains and updates the question-answering model accordingly, when subsequently pushing questions and answering questions to users, it can be more in line with the user's habits and needs in specific scenarios.

[0195] Based on the above-mentioned data acquisition system 50 and the specific functional implementation of each component and module, the data acquisition method provided by the embodiment of the present application is further described below. The embodiment of the present application can construct different question sets (i.e., the first preset question set and the second preset question set in the aforementioned embodiment) by manually constructing question sets, which are divided into real-time push mode and random push mode according to the question push mechanism. The question set constructed in the real-time push mode needs to add scene tags and hardware trigger conditions. According to the status parameters of the smart home devices reported in real time by the smart home module, the first type of questions are determined from the first preset question set and the corresponding first type of questions are pushed. The fragmented interactive mode is set by saving the status information of the smart home devices, and the second type of questions are automatically determined from the second preset question set during the fragmented time period and pushed to the smart screen of the terminal device. The questions include but are not limited to a series of questions such as "Do you want to know the current temperature in the room?" and "Do you need to turn up the speaker volume?"

[0196] The user can select questions and corresponding answers through the smart screen of the terminal device 80, and can simply answer "OK / No". The terminal device 80 saves each question and answer pair selected by the user (a question and answer pair is a pair of questions and answers, that is, a dialogue response sequence) and the current state of the smart home device to its own memory. The dialogue response sequence related to the target scenario is first marked with a scenario label. In this way, these dialogue response sequences are used to update the question and answer model. The question and answer model can extract questions that users are interested in and frequently use in the target scenario by analyzing and extracting words and phrases, and obtain solutions to these questions. For example, for the home scene, when the user is asked "Do you need to turn on the air conditioner?", the user always replies "No", which means that the user usually does not need to turn on the air conditioner in the home scene. In the subsequent home scene, the question "Do you need to turn on the air conditioner?" can be not pushed to the user; if the user is asked "Do you need to turn on the air conditioner?", the user always replies "OK" and sets the air conditioner temperature to 26℃ each time, then in the subsequent home scene, after pushing the question "Do you need to turn on the air conditioner?" to the user and confirming that the user replies "OK", the air conditioner temperature can be directly adjusted to 26℃, or after triggering the home scene, there is no need to push the question to the user, and the air conditioner can be directly turned on and the temperature set to 26℃.

[0197] The terminal device 80 sends the first corpus and / or the second corpus stored in its own memory to the cloud device 100. The processor of the cloud device 100 can calculate and store the questions and answers (or operations) in a slot-value pair mode and perform training. The training process is as follows: Figure 6 , which shows a flow chart of another method for collecting question and answer data for smart home linkage provided by an embodiment of the present application. Figure 6 As shown, the method may include:

[0198] S601: Receive a dialogue response sequence sent by a terminal device.

[0199] It should be noted that the terminal device 80 obtains the first dialogue response sequence with scenario tags and the second dialogue response sequence without scenario tags via real-time push and random push, respectively. The dialogue response sequences with scenario tags constitute the first corpus, while the dialogue response sequences without scenario tags constitute the second corpus. At predetermined intervals, such as weekly or monthly, the terminal device 80 sends the first and / or second corpus stored in it to the cloud device 100, allowing the cloud device 100 to update the question-answering model.

[0200] S602: Extract text features from the dialogue response sequence and obtain a word vector feature matrix.

[0201] The cloud device 100 may include a language understanding module. After receiving the first dialogue response sequence and / or the second dialogue response sequence, the language understanding module extracts questions from the dialogue response sequence to form a question set. The module then first uses a text feature extraction function to extract features, disregarding the order of text appearance. Each word that appears is treated as a separate feature column to form a vocabulary. A TF feature matrix is ​​constructed based on the frequency of occurrence of each character or word in each question sentence. Each row of the TF feature matrix represents a training text. The vocabulary and the TF feature matrix are multiplied together to obtain a word vector feature matrix (also called a primary word vector feature) for the question. Similarly, the same method is used to extract corresponding features from the reply sentence to obtain a word vector feature matrix for the reply sentence.

[0202] In the TF feature matrix, the frequency of each word is determined according to the following formula:

[0203]

[0204] S603: Determine whether the dialogue response sequence is related to the status parameters of the smart home device.

[0205] It should be noted that in smart home scenarios, conversation response behavior may be related to the state parameters of smart home devices in the environment. Therefore, after extracting text features from the conversation response sequence, it is determined whether the conversation response sequence is related to the state parameters of the smart home devices.

[0206] If the judgment result is yes, execute step S604; otherwise, execute step S605.

[0207] S604: Mark the vocabulary related to the state parameters of the smart home with related hardware tags.

[0208] It should be noted that after obtaining the word vector and word frequency feature matrix (i.e., the word vector feature matrix), the words related to the state parameters of the smart home are marked with hardware-related labels. For example, a row or a column is added to the word vector feature matrix to mark the hardware labels. In this way, the word vector feature matrix carries the hardware labels.

[0209] S605: Neural network model training.

[0210] It should be noted that during model training, both the dialogue response sequence and the word vector feature matrix are input into a question-answering model (preferably a recurrent neural network) for training. The first dialogue response sequence also carries the scenario label. The neural network has an encoder-decoder structure, consisting of an intent analysis encoder and a semantic encoder. The intent analysis encoder obtains the intent of the current dialogue through semantic analysis of long sentences. The semantic encoder estimates the word vectors and / or hardware-related labels of candidate response sentences through a random sliding window and fully connected layers. The decoder generates the corresponding response sentence or hardware-related labels through a fully connected layer.

[0211] In this way, since the trained dialogue response sequences carry different scenario labels or do not carry scenario labels, the trained primary word feature vectors are marked with hardware labels or are not marked with hardware labels, so that the dialogue model can accurately capture the user's intention according to the input dialogue response sequence or more accurately capture the current scenario and predict whether the dialogue response sequence is related to the hardware status. On the one hand, it can accurately respond to the user's input voice data and perform corresponding operations; on the other hand, in different scenarios, it can trigger push questions that are more in line with the user's preferences in the current scenario and intelligently control the status of smart home devices.

[0212] In addition, in the embodiment of the present application, the data acquisition and interaction module can use a voice software development kit (SDK) to convert voice information into text information and send it to the cloud device 100, for example, converting Mandarin into text information.

[0213] The language understanding module of the cloud device 100 divides the text information sent by the data collection and user interaction module into words and phrases, and converts the character string into a word frequency feature matrix for storage. For the long Chinese sentences input into the computer, they are first converted into computer-recognizable feature vectors before subsequent operations can be performed. First, the text feature extraction function is used to extract features and generate a TF feature matrix. The TF feature matrix is ​​constructed by the frequency of occurrence of each character or word, and the TF feature matrix is ​​converted into a feature vector through the Word2Vec method as the input of the question-answering model. After training, the question-answering model performs semantic and intent analysis on the input feature vector, for example: confirming whether the sentence is an imperative intention or an interrogative intention, and obtaining candidate reply word vectors and / or hardware labels. The support vector machine (SVM classifier) ​​can also be used as the decision layer to classify the final generated reply.

[0214] Specifically, see Figure 7 , which shows a flow chart of another method for collecting question and answer data for smart home linkage provided by an embodiment of the present application. Figure 7 As shown, the method may include:

[0215] S701. The data collection and interaction module sends text information to the cloud device.

[0216] S702: The language understanding module performs language understanding and feature transformation on the text information.

[0217] S703. The question-answering model performs semantic and intent analysis on the feature vector.

[0218] S704: SVM classifier performs classification.

[0219] S705: Generate daily conversations and output answer sentences.

[0220] S706: Generate control instructions and execute related operations.

[0221] It should be noted that after obtaining the user's voice information, the data collection and interaction module converts the voice information into text information and sends it to the cloud device 100. The language understanding module of the cloud device 100 can perform language understanding and feature transformation on the text information through the question-answering model to obtain a feature vector, and then input the feature vector into the question-answering model. According to the question-answering model, the feature vector is subjected to semantic and intent analysis to confirm whether the voice information represented by the feature vector is an imperative intent or an interrogative intent, and obtain candidate reply word vectors and / or hardware labels, and classify the final reply through the SVM classifier.

[0222] Specifically, if it is an inquiry-type intention, a corresponding reply will be made through the processing of the cloud device 100 and the status parameters of the smart home collected in real time by the smart home module, such as generating an answer statement for daily conversation and sending it to the terminal device. The terminal device outputs the answer statement through the scene response module, for example, through voice or displaying it on the smart screen of the terminal device to respond to the user; if it is a command intention, a control instruction for controlling the smart home device will be generated and sent to the terminal device. The scene response module sends the control instruction to the smart home device through the smart home module, so that the smart home device executes the relevant command action and changes its own status parameters.

[0223] In short, the smart home linkage question-and-answer data collection method provided in the embodiments of the present application can be applied to a question-and-answer data collection system. The question-and-answer data collection system can include a data collection and interaction module. By pushing smart home status question-and-answer questions to terminal devices in a scene, users only need to make a "yes / no" selection. The terminal device's memory stores the user's selected question-and-answer test questions and performs reply or control operations to obtain a basic dialogue corpus in the home scene as the data foundation for self-learning. The data collection system in the home scene has a strong correspondence with the terminal device area. The data collection and interaction module pushes questions to the terminal device in two modes: real-time push and random push. By setting a real-time push mechanism related to hardware actions, including but not limited to, when the user opens the front door when returning home, a real-time question broadcast is immediately triggered. In the form of voice broadcast interaction, the user is asked whether to turn on the lights or the air conditioner. The user responds by answering "yes / no" to the control system to establish a basic dialogue corpus in a specific scene (first corpus). Setting a random push method to push questions to the smart screen utilizes the user's fragmented time for data collection to build a more generalized basic corpus of home scenes (second corpus).

[0224] The data collection and interaction module marks the questions in the first corpus according to different hardware attribution scenarios, and then sends the first corpus and the second corpus to the cloud device. The self-learning module of the cloud device automatically analyzes the second corpus, fine-tunes and updates the question-answering model, and performs targeted optimization of the question-answering model. On the other hand, it performs question-answering analysis on the first dialogue response sequence in the first corpus and refines the question-answering model to achieve the purpose of accurate question-answering in intelligent scenarios.

[0225] The data collection and user interaction module can also realize voice and text conversion. Through this module, user voice input can be converted into text and stored and transmitted.

[0226] The scenario-based scenario response module collects the status parameters of smart home devices in the smart home scenario in real time through the smart home module, such as water heater water temperature, air quality and humidity, refrigerator internal temperature, curtain and light switch status, etc. It can not only broadcast the status through voice, but also perform a series of control operations according to the input voice, such as heating the water heater, turning on or off the air conditioner or fresh air system, turning on and off lights and curtains, etc.

[0227] The cloud device can also include a language understanding module, which uses a question-answering model to extract and classify features of the user's input and fill in slots. At the same time, it uses syntactic analysis to detect intent and perform similarity matching with related entries in the external knowledge base.

[0228] The embodiment of the present application first constructs and pushes questions, manually constructs a question set, and is divided into two categories according to the push method. The construction methods of real-time push and random push question sets are different. Real-time push questions need to add scene tags and hardware trigger conditions, and the status data of the smart home reported in real time by the smart home module are used to push corresponding questions. The fragmented interaction method is set to randomly push question sets based on the status information of the smart hardware platform and smart hardware devices stored in the memory, and the backend automatically pushes questions to the smart screen terminal during fragmented time periods. Questions include but are not limited to a series of questions such as "Do you want to know the current temperature in the room?" and "Do you need to turn up the speaker volume?" The data acquisition and interaction module can use the independently developed voice SDK to convert Mandarin into text information on the terminal device as the input of the language processing module of the cloud device.

[0229] Then user feedback and local storage are carried out. Users can choose questions and corresponding answers through the smart screen terminal, and can answer "OK / No". The terminal device's memory will save the question and answer pairs selected by the user each time and the current status parameters of the smart home device. The question and answer pairs related to the scene are first marked with the scene label. Subsequently, through the analysis and extraction of words and phrases, the user's interests and common questions in the scene, as well as the solutions to the problems, can be obtained.

[0230] The conversation model is then trained on the cloud device. The terminal device reports the data in the local memory to the cloud device. The cloud device's processor calculates and stores the question and answer (or operation) in a slot-value pair format. The training process is as follows:

[0231] Step 1: Input the dialogue response sequence into the text feature extraction module of the cloud device. Question extraction forms a question set. First, feature extraction is performed using the text feature extraction function. Regardless of the order of text appearance, each word is treated as a separate feature column to form a vocabulary. The frequency of occurrence of each word or phrase in each question statement is used to construct a TF feature matrix, with each row representing a training text. This matrix then generates primary word vector features. Similarly, primary feature extraction is performed on the answer statement.

[0232] Step 2: In the smart home scenario, the dialogue response behavior may be related to the state parameters of the smart home devices in the environment. After obtaining the word vector and word frequency matrix, mark the hardware-related hardware tags.

[0233] Step 3: Input the dialogue response sequence and primary word vector features into the recurrent neural network model for training. The neural network has an encoder-decoder structure. The encoding structure consists of two parts: the intent analysis encoder obtains the intent of the current dialogue through semantic analysis of long sentences. The semantic encoder estimates the word vectors of candidate response sentences or hardware-related labels through a random sliding window and fully connected layers. The decoder generates the corresponding response sentence or hardware-related labels through a fully connected layer.

[0234] The language understanding module of the cloud device can segment the sentences input by the data collection and interaction module into words and phrases, convert the character strings into word frequency matrices for storage, and convert the long Chinese sentences input into computer-recognizable feature vectors for subsequent operations. First, the text feature extraction function is used to extract features and generate a TF matrix. The feature matrix is ​​constructed by the frequency of occurrence of each character or word. The word frequency matrix is ​​converted into a feature vector through the Word2Vec method as the input of the recurrent neural network. The recurrent neural network uses the pre-trained model of the cloud device for semantic and intent analysis. For example, whether the sentence is a command or inquiry, etc., the candidate reply word vector or hardware-related label is obtained, and the SVM classifier is used as the decision layer to classify the final action of the system.

[0235] The scene response module responds accordingly to the inquiry intent through cloud device processing and the smart home status information collected by the smart home module. The command intent needs to issue control instructions through the smart home module to perform related operations.

[0236] The above embodiments have been used to illustrate the specific implementation of the aforementioned embodiments. It can be seen that the present invention proposes a method and system for collecting question-and-answer data for smart home linkage, which not only possesses basic question-and-answer capabilities but also controls the status and execution of smart home devices through the smart home module. The data collection and interaction modules construct different question sets based on the push method. Real-time push questions require the addition of scenario tags and hardware trigger conditions for corresponding question push. A fragmented interactive method randomly pushes question sets based on the smart home device status information stored in the memory. By pushing smart home status question-and-answer questions within a scenario to the terminal device, the user simply selects "yes / no." The terminal device's memory stores the user-selected question and answer questions for the current scenario, and after executing a reply or control operation, a basic Chinese character corpus for that scenario is obtained, serving as the data foundation for self-learning. The cloud device can not only learn a language-based question-and-answer model, but also associate the conversation with the smart home scenario by adding hardware-related tags. The reply content can be classified into two types by a support vector machine, achieving different goals, thereby optimizing the user experience and increasing user engagement. The scenario-based scenario response module collects the status parameters of smart home devices in the smart home scenario in real time through the smart home module, such as water heater water temperature, air quality and humidity, refrigerator internal temperature, curtain and light switch status, etc. It can not only broadcast the status through voice, but also perform a series of control operations based on the input voice, such as heating the water heater, turning on or off the air conditioner or fresh air system, turning lights and curtains on and off, etc. The control method provided in the embodiment of the application can be run on a terminal device with a smart screen. The terminal device library belongs to a data acquisition system. The entire data acquisition system includes at least one terminal device, a cloud device and multiple smart home devices.

[0237] Compared with related technologies, the data collection method provided by the embodiments of the present application has at least the following advantages: it not only has basic question-and-answer capabilities, but also controls the status and execution operations of smart home devices through the smart home module; the data collection mode proposed in the embodiments of the present application performs data collection in a semi-automatic manner when data is insufficient. By pushing smart home status question-and-answer test questions within the scenario to the terminal device, the user only needs to make a "yes / no" selection. The terminal device's memory saves the question-and-answer test questions selected by the user in the current scenario and performs reply or control operations to obtain the basic Chinese character corpus in the scenario as the data foundation for self-learning. The embodiments of the present application propose a scenario-based scenario response module. Inquiry-type intents respond accordingly by matching data center slot-value pairs with the status parameters of smart home devices. Imperative intents issue instructions through the smart home platform to enable smart home devices to perform related operations.

[0238] In another embodiment of the present application, see Figure 8 , which shows a schematic diagram of the structure of a terminal device 80 provided in an embodiment of the present application. Figure 8 As shown, the terminal device 80 may include an acquisition unit 801, a first sending unit 802, a first receiving unit 803, and a corpus building unit 804, wherein:

[0239] An acquisition unit 801 is configured to acquire status parameters of a smart home device;

[0240] A first sending unit 802 is configured to send the state parameters of the smart home device to a cloud device;

[0241] A first receiving unit 803 is configured to receive a first type of question sent by the cloud device, wherein the first type of question is associated with a target scenario, and the target scenario is determined by the cloud device according to the state parameter of the smart home device;

[0242] A corpus building unit 804 is configured to push the first type of questions to the user and collect the user's answer sentences to build a first corpus;

[0243] The first sending unit 802 is further configured to send the first corpus to the cloud device, so that the cloud device trains and updates the question-answering model according to the first corpus.

[0244] In some embodiments, the first receiving unit 803 is further configured to receive a second type of question sent by the cloud device; wherein the second type of question has no correlation with the target scenario and is randomly sent by the cloud device;

[0245] The corpus building unit 804 is further configured to push the second type of questions to the user and collect the user's answer sentences to build a second corpus;

[0246] The first sending unit 802 is further configured to send the second corpus to the cloud device.

[0247] In some embodiments, the corpus establishment unit 804 is specifically configured to form a first dialogue response sequence based on the first type of questions and the corresponding reply statements; and to mark the first dialogue response sequence with the scene label of the target scene to obtain the first corpus.

[0248] In some embodiments, the corpus creation unit 804 is specifically configured to compose a second dialogue response sequence based on the second type of questions and the corresponding answer statements; and obtain the second corpus based on the second dialogue response sequence.

[0249] In some embodiments, the corpus creation unit 804 is specifically configured to determine the user's reply statement from two preset options; wherein the two preset options include "OK" and "No need".

[0250] In some embodiments, the corpus creation unit 804 is further configured to convert the reply sentence into text information;

[0251] The first sending unit 802 is further configured to send the text message to the cloud device;

[0252] The first receiving unit 803 is further configured to receive a reply statement or control instruction returned by the cloud device;

[0253] The corpus creation unit 804 is further configured to push the reply statement to the user; or send the control instruction to the smart home device, wherein the control instruction is used to instruct the smart home device to control its own status.

[0254] In some embodiments, the first receiving unit 803 is further configured to receive voice data of the user;

[0255] The corpus creation unit 804 is further configured to convert the voice data into text information;

[0256] The first sending unit 802 is further configured to send the status parameters of the smart home device and the text information to the cloud device, and receive a reply statement or control instruction returned by the cloud device;

[0257] The corpus creation unit 804 is further configured to push the reply statement to the user; or send the control instruction to the smart home device, wherein the control instruction is used to instruct the smart home device to control its own status.

[0258] In some embodiments, the acquisition unit 801 is further configured to collect status parameters of the smart home device;

[0259] The corpus creation unit 804 is further configured to push the status parameters of the smart home device in a preset manner; wherein the preset manner includes a voice playback manner and / or a screen display manner.

[0260] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0261] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0262] Therefore, this embodiment provides a computer storage medium, which stores a computer program. When the computer program is executed by the first processor, it implements the smart home linkage question and answer data collection method described in any one of the aforementioned embodiments applied to the terminal device.

[0263] Based on the above-mentioned composition of a terminal device 80 and computer storage medium, see Figure 9 , which shows a schematic diagram of the specific hardware structure of a terminal device 80 provided in an embodiment of the present application. Figure 9 As shown, it may include: a first communication interface 901, a first memory 902 and a first processor 903; each component is coupled together via a first bus system 904. It is understood that the first bus system 904 is used to achieve connection and communication between these components. In addition to the data bus, the first bus system 904 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 9 Various buses are labeled as the first bus system 904. Among them, the first communication interface 901 is used to receive and send signals during the process of sending and receiving information between other external network elements;

[0264] A first memory 902 is used to store computer programs that can be run on the first processor 903;

[0265] The first processor 903 is configured to, when running the computer program, execute:

[0266] Get the status parameters of smart home devices;

[0267] Sending the state parameters of the smart home device to a cloud device, and receiving a first type of question sent by the cloud device, wherein the first type of question is associated with a target scenario, and the target scenario is determined by the cloud device based on the state parameters of the smart home device;

[0268] Pushing the first type of questions to users and collecting the users' answer statements to establish a first corpus;

[0269] The first corpus is sent to the cloud device, so that the cloud device trains and updates the question-answering model based on the first corpus.

[0270] It is understood that the first memory 902 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 902 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0271] The first processor 903 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 903. The above-mentioned first processor 903 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 902 , and the first processor 903 reads the information in the first memory 902 and completes the steps of the above method in combination with its hardware.

[0272] It is understood that the embodiments described herein may be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or a combination thereof.

[0273] For software implementation, the techniques described herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0274] Optionally, as another embodiment, the first processor 903 is further configured to execute any one of the data collection methods described in the aforementioned embodiments applied to the terminal device when running the computer program.

[0275] For the terminal device 80, since the first corpus is established by the first type of questions associated with the target scenario and the corresponding user's reply statements, the cloud device can update the question-answering model based on the first corpus, so that when the training data is insufficient, the corpus obtained from the daily conversation process can also be used as training data for updating the question-answering model. Therefore, the updated question-answering model not only has basic question-answering capabilities, but can also achieve more accurate interactive question-answering in combination with the scenario, thereby increasing user stickiness and achieving the purpose of accurate question-answering in intelligent scenarios.

[0276] In another embodiment of the present application, see Figure 10 , which shows a schematic diagram of the structure of a cloud device 100 provided in an embodiment of the present application. Figure 10 As shown, the cloud device 100 may include a second receiving unit 1001, a determining unit 1002, a second sending unit 1003 and a training unit 1004, wherein:

[0277] The second receiving unit 1001 is configured to receive the status parameters of the smart home device sent by the terminal device;

[0278] A determining unit 1002 is configured to determine a target scenario according to the state parameters of the smart home device, and determine a first type of question from a first preset question set according to the target scenario;

[0279] The second sending unit 1003 is configured to send the first type of question to the terminal device;

[0280] The second receiving unit 1001 is further configured to receive the first corpus returned by the terminal device;

[0281] The training unit 1004 is configured to train and update the question-answering model based on the first corpus.

[0282] In some embodiments, the determining unit 1002 is further configured to randomly select a second type of question from a second preset question set during a preset fragmentation time period;

[0283] The second sending unit 1003 is further configured to send the second type of question to the terminal device;

[0284] The second receiving unit 1001 is further configured to receive a second corpus returned by the terminal device;

[0285] The training unit 1004 is further configured to train and update the question-answering model based on the second corpus.

[0286] In some embodiments, the first preset question set is different from the second preset question set, and the determination unit 1002 is further configured to determine at least one question to be pushed in real time, and mark the at least one question to be pushed in real time with a scene tag and a hardware trigger condition, respectively, to construct the first preset question set; or, set a fragmented interaction mode according to the status parameters of the smart home device, generate at least one question to be randomly pushed, and construct the second preset question set based on the at least one question to be randomly pushed.

[0287] In some embodiments, the training unit 1004 is further configured to perform text feature extraction on at least one dialogue response sequence in the first corpus and / or the second corpus to generate a word vector feature matrix corresponding to the at least one dialogue response sequence; and if the dialogue response sequence has an association with the state parameter of the smart home device, mark the word vector feature matrix corresponding to the dialogue response sequence with a hardware label; and use the at least one dialogue response sequence and the respective corresponding word vector feature matrices to train and update the preset model to obtain the question-answering model.

[0288] In some embodiments, the second receiving unit 1001 is further configured to receive a text message sent by the terminal device;

[0289] The training unit 1004 is further configured to process the text information using the question-answering model to generate a reply statement or a control instruction;

[0290] The second sending unit 1003 is further configured to send the reply statement or the control instruction to the terminal device.

[0291] In some embodiments, the training unit 1004 is specifically configured to perform semantic intent analysis on the text information through the question-answering model to obtain an analysis result; and if the analysis result is an interrogative intent, determine the candidate reply sentence corresponding to the text information, and determine the reply sentence from the candidate reply sentences; and if the analysis result is an imperative intent, determine the hardware tag corresponding to the text information, and generate the control instruction based on the hardware tag.

[0292] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0293] Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, and this computer software product is stored in a storage medium. Therefore, this embodiment also provides a computer storage medium, applied to the cloud device 100, which stores a computer program. When the computer program is executed by the first processor, it implements the smart home linkage question and answer data collection method described in any of the aforementioned embodiments.

[0294] Based on the above-mentioned composition of the cloud device 100 and the computer storage medium, see Figure 11 , which shows a specific hardware structure diagram of a cloud device 100 provided in an embodiment of the present application. Figure 11 As shown, it may include: a second communication interface 1101, a second memory 1102, and a second processor 1103; each component is coupled together via a second bus system 1104. It is understood that the second bus system 1104 is used to implement connection and communication between these components. In addition to the data bus, the second bus system 1104 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 11 Various buses are labeled as the second bus system 1104. Among them, the second communication interface 1101 is used to receive and send signals during the process of sending and receiving information between other external network elements;

[0295] The second memory 1102 is used to store computer programs that can be run on the second processor 1103;

[0296] The second processor 1103 is configured to, when running the computer program, execute:

[0297] Receive status parameters of smart home devices sent by terminal devices;

[0298] Determining a target scenario according to the state parameters of the smart home device, and determining a first type of question from a first preset question set according to the target scenario;

[0299] Sending the first type of questions to the terminal device;

[0300] Receive the first corpus returned by the terminal device, and train and update the question-answering model based on the first corpus.

[0301] It should be noted that the specific functional implementation of the second communication interface 1101, the second memory 1102, the second processor 1103 and the second bus system 1104 is consistent with the description of the aforementioned first communication interface 901, the first memory 902, the first processor 903 and the first bus system 904, and will not be repeated in the embodiments of this application.

[0302] Optionally, as another embodiment, the second processor 1103 is further configured to execute any of the data collection methods described in the aforementioned embodiments applied to the cloud device when running the computer program. For the cloud device 100, since the first corpus is associated with the target scenario, the updated question-answering model not only has basic question-answering capabilities, but can also achieve more accurate interactive question-answering in combination with the scenario, thereby increasing user stickiness and achieving the purpose of accurate question-answering in intelligent scenarios.

[0303] Based on the above terminal device 80 and cloud device 100, see Figure 12 , which shows a schematic diagram of the composition structure of another question-answering data collection system 50 provided in an embodiment of the present application. Figure 12 As shown, the question-and-answer data collection system 50 includes at least the terminal device 80, cloud device 100 and smart home device described in any one of the aforementioned embodiments.

[0304] In some embodiments, the number of terminal devices 80 and smart home devices is at least one.

[0305] For the question-and-answer data collection system 50, since the terminal device 80 establishes a first corpus through the first type of questions associated with the target scenario and the corresponding user's reply statements, the cloud device 100 can update the question-and-answer model based on the first corpus, so that when the training data is insufficient, the corpus obtained from the daily conversation process can also be used as training data for updating the question-and-answer model. Therefore, the updated question-and-answer model not only has basic question-and-answer capabilities, but can also achieve more accurate interactive question-and-answer in combination with the scenario, thereby increasing user stickiness, achieving the purpose of accurate question-and-answer in intelligent scenarios, and improving the robustness and robustness of the data collection system 50.

[0306] The above description is merely a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application.

[0307] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0308] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0309] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0310] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0311] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0312] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for collecting question-answer data for smart home linkage, characterized in that: Applied to a terminal device, the method includes: Get the status parameters of smart home devices; Sending the state parameters of the smart home device to a cloud device, and receiving a first type of question sent by the cloud device, wherein the first type of question is associated with a target scenario, and the target scenario is determined by the cloud device based on the state parameters of the smart home device; Pushing the first type of questions to users and collecting the users' reply statements, and establishing a first corpus based on the first type of questions and the users' reply statements; The first corpus is sent to the cloud device so that the cloud device trains and updates the question-answering model based on the first corpus; the question-answering model is used to provide a reply statement to the terminal device based on the user's reply statement, and to analyze the status parameters of the smart home device to determine the target scenario and the corresponding first type of question.

2. The method according to claim 1, characterized in that The method further comprises: Receiving a second type of question sent by the cloud device; wherein the second type of question is unrelated to the target scenario and is randomly sent by the cloud device; Pushing the second type of questions to the user and collecting the user's answer sentences to establish a second corpus; The second corpus is sent to the cloud device.

3. The method according to claim 1, characterized in that The establishing of the first corpus comprises: composing a first dialogue response sequence based on the first type of questions and corresponding answer statements; The first dialogue response sequence is labeled with a scene label of the target scene to obtain the first corpus.

4. The method according to claim 2, characterized in that The establishing of the second corpus comprises: forming a second dialogue response sequence based on the second type of questions and corresponding answer statements; The second corpus is obtained according to the second dialogue response sequence.

5. The method according to claim 1 or 2, characterized in that The collecting of the user's reply statement includes: Determine the user's reply statement from two preset options; wherein the two preset options include "OK" and "No need".

6. The method according to claim 1 or 2, characterized in that After collecting the user's reply statement, the method further includes: Converting the reply statement into text information; Sending the text message to the cloud device and receiving a reply statement or control instruction returned by the cloud device; Push the reply statement to the user; or send the control instruction to the smart home device, wherein the control instruction is used to instruct the smart home device to control its own status.

7. The method according to claim 1 or 2, characterized in that The method further comprises: Receiving voice data from the user and converting the voice data into text information; Sending the status parameters of the smart home device and the text information to the cloud device, and receiving a reply statement or control instruction returned by the cloud device; Push the reply statement to the user; or send the control instruction to the smart home device, wherein the control instruction is used to instruct the smart home device to control its own status.

8. The method according to claim 1, characterized in that Before obtaining the status parameters of the smart home devices, the method further includes: collecting the status parameters of the smart home devices and pushing the status parameters of the smart home devices in a preset manner; The preset mode includes a voice playback mode and / or a screen display mode.

9. A method for collecting question-answer data for smart home linkage, characterized in that: Applied to a cloud device, the method includes: Receive status parameters of smart home devices sent by terminal devices; Determining a target scenario according to the state parameters of the smart home device, and determining a first type of question from a first preset question set according to the target scenario; Sending the first type of questions to the terminal device; Receive a first corpus returned by the terminal device, and train and update a question-answering model based on the first corpus; the first corpus is established by the terminal device based on the first type of questions and the reply statements replied by the user; the question-answering model is used to provide a reply statement to the terminal device based on the user's reply statement, and analyze the status parameters of the smart home device to determine the target scenario and the corresponding first type of questions.

10. The method according to claim 9, characterized in that The method further comprises: In the preset fragmented time period, randomly select the second type of questions from the second preset question set; sending the second type of questions to the terminal device; Receive a second corpus returned by the terminal device, and train and update the question-answering model based on the second corpus.

11. The method according to claim 10, characterized in that The first preset question set is different from the second preset question set, and the method further includes: Determine at least one question to be pushed in real time, and mark the at least one question to be pushed in real time with a scenario tag and a hardware trigger condition, to construct the first preset question set; or, A fragmented interaction mode is set according to the state parameters of the smart home device, at least one question to be randomly pushed is generated, and the second preset question set is constructed according to the at least one question to be randomly pushed.

12. The method according to claim 10, characterized in that The method further comprises: performing text feature extraction on at least one dialogue response sequence in the first corpus and / or the second corpus to generate a word vector feature matrix corresponding to the at least one dialogue response sequence; If the dialogue response sequence is associated with the state parameter of the smart home device, marking a hardware label on the word vector feature matrix corresponding to the dialogue response sequence; The preset model is trained and updated using the at least one dialogue response sequence and the respective corresponding word vector feature matrices to obtain the question-answering model.

13. The method according to claim 9, characterized in that The method further comprises: receiving a text message sent by the terminal device; Processing the text information using the question-answer model to generate a reply statement or control instruction; The reply statement or the control instruction is sent to the terminal device.

14. The method according to claim 13, wherein: The processing of the text information by using the question-answer model to generate a reply statement or a control instruction includes: Performing semantic intent analysis on the text information using the question-answering model to obtain an analysis result; If the analysis result is an inquiry intent, determining candidate reply sentences corresponding to the text information, and determining the reply sentence from the candidate reply sentences; If the analysis result is an imperative intent, the hardware tag corresponding to the text information is determined, and the control instruction is generated according to the hardware tag.

15. A terminal device, characterized in that: The terminal device includes an acquisition unit, a first sending unit, a first receiving unit, and a corpus building unit, wherein: The acquiring unit is configured to acquire status parameters of the smart home device; The first sending unit is configured to send the status parameters of the smart home device to a cloud device; The first receiving unit is configured to receive a first type of question sent by the cloud device, wherein the first type of question is associated with a target scenario, and the target scenario is determined by the cloud device according to the state parameter of the smart home device; The corpus building unit is configured to push the first type of questions to the user and collect the user's reply sentences, and build a first corpus based on the first type of questions and the reply sentences replied by the user; The first sending unit is further configured to send the first corpus to the cloud device so that the cloud device trains and updates the question-answering model based on the first corpus; the question-answering model is used to provide a reply statement to the terminal device based on the user's reply statement, and to analyze the status parameters of the smart home device to determine the target scenario and the corresponding first type of question.

16. A cloud device, characterized in that: The cloud device includes a second receiving unit, a determining unit, a second sending unit and a training unit, wherein: The second receiving unit is configured to receive the status parameters of the smart home device sent by the terminal device; The determining unit is configured to determine a target scenario according to the state parameters of the smart home device, and determine a first type of question from a first preset question set according to the target scenario; The second sending unit is configured to send the first type of questions to the terminal device; The second receiving unit is further configured to receive the first corpus returned by the terminal device; The training unit is configured to train and update the question-answering model based on the first corpus; the first corpus is established by the terminal device based on the first type of questions and the reply statements replied by the user; the question-answering model is used to provide reply statements to the terminal device based on the user's reply statements, and to analyze the status parameters of the smart home device to determine the target scenario and the corresponding first type of questions.

17. A terminal device, characterized in that: The terminal device includes a first memory and a first processor, wherein: The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the question-and-answer data collection method for smart home linkage as described in any one of claims 1 to 8 when running the computer program.

18. A cloud device, characterized in that: The cloud device includes a second memory and a second processor, wherein: The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the smart home linkage question and answer data collection method according to any one of claims 9 to 14 when running the computer program.

19. A computer storage medium, characterized in that The computer storage medium stores a computer program, which, when executed by a first processor, implements the question-and-answer data collection method for smart home linkage as described in any one of claims 1 to 8, or, when executed by a second processor, implements the question-and-answer data collection method for smart home linkage as described in any one of claims 9 to 14.

20. A question-answer data collection system, characterized in that: The question-and-answer data collection system includes the terminal device according to claim 15 or 17, the cloud device according to claim 16 or 18, and a smart home device.

Images