Intelligent equipment man-machine interaction method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an intelligent equipment man-machine interaction method, which comprises the following steps: collecting interaction data of a user, responding to a first operation instruction of the interaction data and responding to a first feedback result of the user, classifying the interaction data, identifying the interaction data by using a preset model, and generating a second operation instruction according to an identification result. The second operation instruction is evaluated, a target operation instruction is obtained when a preset condition is met, the model parameters are adjusted by the user according to a second feedback result after the target operation instruction is responded, and execution is returned. Through a training data set formed by the interaction data missed by the user and the first operation instruction, obtaining a second operation instruction generated by identifying the training data set by the preset model, and performing parameter adjustment on the preset model in combination with a second feedback result obtained by evaluating the second operation instruction; the intelligent mode of the intelligent equipment such as the intelligent television is optimized by continuously updating the identification of the television on the interaction command, so that the response accuracy of the intelligent equipment is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, computer equipment and storage medium for human-computer interaction of an intelligent device. Background Art

[0002] With the development of artificial intelligence technology, users can use voice commands, gesture operations, etc. to perform various operations such as channel switching, playback control, and content search, which improves the convenience and naturalness of interaction.

[0003] At present, the operating logic of the operating system and user UI of smart devices such as smart TVs has become complicated, the user's learning cost has become high, and it has become difficult to learn to operate TV products. Traditional TV AI voice control still responds in a command-based manner, and its ability to recognize and transform the user's speaking intentions is insufficient. It is often the case that users want to control the TV but cannot understand their intentions. At the same time, the TV itself lacks the self-learning ability of semantic understanding, which causes the same problem to recur repeatedly and is difficult to solve.

[0004] Therefore, how to automatically and accurately identify user feedback to improve the response accuracy of smart devices has become an urgent problem to be solved. Summary of the invention

[0005] Based on this, the present invention provides a method, apparatus, computer device and storage medium for human-computer interaction of an intelligent device to solve the problem of how to automatically and accurately identify user feedback to improve the response accuracy of the intelligent device.

[0006] In a first aspect, an embodiment of the present invention provides a human-computer interaction method for an intelligent device, comprising: After the smart device is started, the user's interaction data, the first operation instruction in response to the interaction data, and the first feedback result of the user after the response are collected, and the interaction data are classified according to the first feedback result to obtain hit interaction data and miss interaction data, and the hit interaction data and the corresponding first operation instruction form a hit data set, and the miss interaction data and the corresponding first operation instruction form a training data set; For any interaction data in the training data set, use a preset model to identify the interaction data to obtain N recognition results, and generate a second operation instruction for each recognition result, where N is an integer greater than zero; Using the hit data set, evaluating each second operation instruction to obtain a corresponding evaluation result, determining the second operation instruction whose evaluation result meets a preset condition as a target operation instruction, and obtaining a second feedback result of the user after responding to the target operation instruction; According to the second feedback result, the parameters of the preset model are adjusted, and the first operation instruction for collecting the user's interaction data, responding to the interaction data, and the first feedback result of the user after the response are returned to execute until the smart device is turned off.

[0007] In a second aspect, an embodiment of the present invention provides a device for human-computer interaction of an intelligent device, comprising: A data classification module is used to collect user interaction data, a first operation instruction in response to the interaction data, and a first feedback result of the user after the response, and classify the interaction data according to the first feedback result to obtain hit interaction data and miss interaction data, and form a hit data set with the hit interaction data and the corresponding first operation instruction, and form a training data set with the miss interaction data and the corresponding first operation instruction; An instruction generation module, for identifying any interaction data in the training data set using a preset model, obtaining N recognition results, and generating a second operation instruction for each recognition result, where N is an integer greater than zero; An evaluation module, configured to evaluate each second operation instruction using the hit data set to obtain a corresponding evaluation result, determine that the second operation instruction whose evaluation result meets a preset condition is a target operation instruction, and obtain a second feedback result of the user after responding to the target operation instruction; The adjustment module is used to adjust the parameters of the preset model, return to execute the first operation instruction of collecting the user's interaction data, responding to the interaction data, and the first feedback result of the user after the response, until the smart device is turned off.

[0008] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the intelligent device human-computer interaction method described in the first aspect when executing the computer program.

[0009] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the human-computer interaction method of an intelligent device described in the first aspect above is implemented.

[0010] Compared with the prior art, the present invention has the following beneficial effects: after the smart device is started, the user's interaction data, the first operation instruction responding to the interaction data, and the first feedback result of the user after the response are collected; according to the first feedback result, the interaction data are classified to obtain hit interaction data and miss interaction data; the hit interaction data and the corresponding first operation instruction form a hit data set; the miss interaction data and the corresponding first operation instruction form a training data set; for any interaction data in the training data set, the preset model is used to identify the interaction data to obtain N recognition results; a second operation instruction is generated for each recognition result; each second operation instruction is evaluated using the hit data set to obtain a corresponding evaluation result; the second operation instruction whose evaluation result meets the preset conditions is determined to be the target operation instruction; the second feedback result of the user after responding to the target operation instruction is obtained; according to the second feedback result, the preset model is parameter adjusted; and the execution of collecting the user's interaction data, the first operation instruction responding to the interaction data, and the first feedback result of the user after the response is returned until the smart device is turned off. A training data set is formed by the user's missed interaction data and the first operation instruction, and a second operation instruction generated by the preset model for identifying the training data set is obtained. Combined with a second feedback result obtained by evaluating the second operation instruction, the parameters of the preset model are adjusted to continuously update the TV's recognition of interaction commands to optimize the intelligent mode of smart devices such as smart TVs, thereby improving the response accuracy of smart devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0012] Figure 1 This is a schematic diagram of an application environment of a human-computer interaction method for an intelligent device provided in Embodiment 1 of the present invention; Figure 2 It is a flowchart of a human-computer interaction method of an intelligent device provided in Embodiment 2 of the present invention; Figure 3 It is a flowchart of a human-computer interaction method of an intelligent device provided in Embodiment 3 of the present invention; Figure 4 It is a flowchart of a human-computer interaction method of an intelligent device provided by Embodiment 4 of the present invention; Figure 5 It is a flowchart of a human-computer interaction method of an intelligent device provided in Embodiment 5 of the present invention; Figure 6It is a flowchart of a human-computer interaction method of an intelligent device provided in Embodiment 6 of the present invention; Figure 7 This is a schematic diagram of a network structure of a human-computer interaction method for an intelligent device provided in a fourth embodiment of the present invention; Figure 8 It is a structural schematic diagram of a human-computer interaction device for an intelligent device provided by Embodiment 7 of the present invention; Fig. 9 It is a structural diagram of a computer device provided in Embodiment 8 of the present invention. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0014] like Figure 1 As shown, it is a schematic diagram of the application environment of a method for human-computer interaction of an intelligent device provided in the first embodiment of the present invention, wherein the client and the server are connected to communicate, and the user can provide the server with conditions, requirements and operation instructions for human-computer interaction of the intelligent device by operating the client, and the server is used to execute the method for human-computer interaction of the intelligent device of the present invention according to the relevant content sent by the client. Among them, the client includes but is not limited to various computer devices such as personal computers, laptops, smart phones, tablet computers and portable wearable devices. The computer device corresponding to the server can be implemented with an independent server or a server cluster composed of multiple servers.

[0015] like Figure 2 FIG. 1 is a flow chart of a method for human-computer interaction of an intelligent device provided in Embodiment 2 of the present invention, wherein the method for human-computer interaction of an intelligent device is applied in Figure 1 The intelligent device human-computer interaction method may include the following steps: Step S201, after the smart device is started, collect the user's interaction data, the first operation instruction in response to the interaction data, and the first feedback result of the user after the response, classify the interaction data according to the first feedback result, obtain hit interaction data and miss interaction data, form a hit data set with the hit interaction data and the corresponding first operation instruction, and form a training data set with the miss interaction data and the corresponding first operation instruction.

[0016] When a user turns on a smart TV, the system starts and enters an initialization state, ready to receive and process user interaction data, including but not limited to voice commands, remote control operations, gesture recognition, and touch screen input.

[0017] Among them, voice recognition may refer to the commands issued by the user through the voice assistant, remote control operation may refer to the operations performed by the user through the buttons or touch screen of the remote control, gesture recognition may refer to the gesture operations performed by the user through the camera or gesture recognition device, and touch screen input may refer to the interactive operations performed by the user through the touch screen of the smart TV.

[0018] The collection methods include, but are not limited to, voice recognition module, remote control interface, camera module, and touch screen sensor.

[0019] Among them, the voice recognition module may refer to capturing the user's voice commands through a microphone array and converting them into text form, the remote control interface may refer to receiving the user's remote control operations through infrared or Bluetooth, the camera module may refer to capturing the user's gestures through a camera and performing gesture recognition, and the touch screen sensor may refer to recording the user's touch operations through the touch screen sensor.

[0020] The smart TV's response to the user's interactive data can be parsed into a first operation instruction, and the smart TV performs a corresponding operation, such as opening an application, switching channels, adjusting the volume, etc. The user operation results will be reflected in the user interface, such as displaying a new channel, starting an application, etc.

[0021] The user's first feedback result includes, but is not limited to, voice feedback, operation feedback, and display feedback.

[0022] Among them, voice feedback can refer to users expressing satisfaction or dissatisfaction through voice, such as "right" or "this is not what I want", operation feedback can refer to users expressing dissatisfaction through further operations, such as immediately switching back to the original channel, closing the application, etc., and explicit feedback can refer to the smart TV providing a feedback button or rating system, and users explicitly express satisfaction or dissatisfaction by clicking or rating. Data in which users are satisfied with the response of the smart TV is classified as hit data, and data in which users are dissatisfied with the response of the smart TV is classified as miss data.

[0023] Optionally, after forming a training data set by combining the missed interaction data and the corresponding first operation instruction in step S201, the smart device human-computer interaction method may further include the following steps: According to the training data set, use the cleaning rules to clean the data to obtain the cleaned training data set; Based on the cleaned training data set, use standardization specifications to unify the data and obtain a training data set in a unified format; For any interaction data in a training data set of a unified format, a preset model is used to identify the interaction data to obtain N recognition results, and a second operation instruction is generated for each recognition result, where N is an integer greater than zero.

[0024] Among them, the cleaning rules are defined as removing invalid data, deduplication, formatting, and noise processing. For example, deleting incomplete, erroneous or irrelevant data, deleting duplicate interactive data, converting data formats, ensuring the consistency of all data, and removing background noise, misidentification and other interfering data.

[0025] The specific steps of data cleaning are: check whether each interactive data contains user input, the first operation instruction and the missed identification, delete the records with missing key information, use hash table or set data structure to record the unique identification of each interactive data, use speech recognition technology to filter background noise, remove misrecognized words through text preprocessing technology, check whether the new record already exists in the set, and delete it if it does. For example, delete the records with empty user input or the records with the first operation instruction in the wrong format, delete the duplicate records with the user input and the first operation instruction being exactly the same, use noise filter to remove background sound in user input, and remove invalid words through spelling check and dictionary matching.

[0026] By performing data cleaning on the training data set and unifying the cleaned data set, a training data set in a unified format is obtained, and then the interactive data in the training data set is subsequently identified to obtain N recognition results, and a second operation instruction is generated for the recognition results, thereby providing a unified data format for subsequent processing of the training data set.

[0027] Step S202: for any interaction data in the training data set, use a preset model to identify the interaction data to obtain N recognition results, and generate a second operation instruction for each recognition result, where N is an integer greater than zero.

[0028] Among them, a piece of missed interaction data is selected from the training data set generated in step S201, and this piece of data includes the original input of the user (such as voice command, remote control operation, etc.) and the corresponding first operation instruction, and the selected interaction data is preprocessed to adapt to the input requirements of the preset model. For example, the voice is converted into text, noise is removed, the text format is standardized, the operation code is converted into an understandable operation type, the gesture data is converted into a standardized feature vector, and the touch event sequence is converted into a standardized input format. The preset model generates N recognition results according to the input data. These recognition results are multiple operation instructions that the model believes the user may intend, and each recognition result is converted into a specific second operation instruction. These second operation instructions will be used for subsequent evaluation and optimization.

[0029] Optionally, the step of using a preset model to identify the interaction data in step S202 to obtain N identification results may include the following steps: According to any interaction data, feature extraction is performed to obtain interaction features; According to the interaction characteristics, a pre-trained model is used to identify the user's interaction intention, N intention recognition results are obtained, and a second operation instruction is generated for each intention recognition result.

[0030] Among them, you can select some commonly used pre-trained language models, load the pre-trained models and their related parameters from the model library, input user input data into the pre-trained language model, and obtain the output features from the model. Usually, the hidden state of the last layer is used as the feature. These feature vectors contain rich semantic information and can be used for subsequent intent recognition and operation instruction generation.

[0031] Among them, the pre-trained model can select some commonly used intent recognition models, such as transformer-based models or other machine learning models. The pre-trained model and its related parameters are loaded into the model library to ensure that the model can run normally. The interactive features extracted from the language model are input into the intent recognition model, and the N most likely intent recognition results and their corresponding probabilities are obtained from the model output. A mapping table from intent labels to operation instructions is constructed to facilitate the conversion of intent labels into specific operation instructions. Corresponding operation instructions are generated for each intent label to obtain N second operation instructions.

[0032] Step S203, using the hit data set, evaluates each second operation instruction to obtain a corresponding evaluation result, determines that the second operation instruction whose evaluation result meets the preset conditions is the target operation instruction, and obtains the second feedback result of the user after responding to the target operation instruction.

[0033] Among them, the hit data set includes the hit interaction data generated in step S201 and its corresponding first operation instruction. These data have been verified to be correctly parsed and executed and are the "correct answer" of the model. Each interaction data in the hit data set and its corresponding first operation instruction are compared with the second operation instruction generated in step S202 to evaluate the accuracy of the second operation instruction and the percentage of the second operation instruction matching the first operation instruction in some aspects (for example, the command type matches but the specific parameters are inconsistent). Whether the user is satisfied with the execution result of the second operation instruction can be judged through the user's subsequent operations or explicit feedback. For each second operation instruction, an evaluation result is generated, which can be a score or a classification (such as correct, partially correct, or wrong).

[0034] Among them, the preset condition can set the threshold of the evaluation result according to the specific business needs and system performance. For example, the accuracy threshold is above 80%, the partial matching threshold is above 60%, and the user satisfaction threshold is above 90%. The second operation instruction whose evaluation result meets the preset condition is selected as the target operation instruction. For example, if the preset condition is that the accuracy is greater than 80%, then only the second operation instruction with an accuracy greater than 80% will be selected.

[0035] Step S204, according to the second feedback result, the parameters of the preset model are adjusted, and the first operation instruction for collecting the user's interaction data, responding to the interaction data, and responding to the first feedback result of the user are returned to execute until the smart device is turned off.

[0036] Among them, the second feedback result can be the feedback result of the user on the target operation instruction obtained in step S203, which can be positive feedback or negative feedback. Positive feedback indicates that the target operation instruction is correct or the user is satisfied, and this part of the model parameters does not need to be adjusted. Negative feedback indicates that the target operation instruction is wrong or the user is not satisfied, and the model parameters need to be adjusted. The positive feedback and negative feedback data are incrementally added to the training set of the model, and online learning is performed to continuously update the model parameters. The positive feedback data can increase the model's weight for correct operation instructions, and the negative feedback data can reduce the model's weight for erroneous operation instructions, or increase the training intensity for correct operation instructions. For models based on deep learning, the back propagation algorithm is used to adjust the model parameters according to the negative feedback results, the loss function of the model is calculated according to the negative feedback results, and the gradient descent method or other optimization algorithms are used to update the model parameters to reduce the loss.

[0037] After the preset model parameters are adjusted, return to step S201 to continue collecting the user's interaction data, the first operation instruction in response to the interaction data, and the first feedback result of the user after the response. Through continuous user feedback and model adjustment, the interactive experience of the smart device will be gradually optimized to improve user satisfaction.

[0038] In an embodiment of the present application, after the smart device is started, the user's interaction data, the first operation instruction responding to the interaction data, and the first feedback result of the user after the response are collected, and according to the first feedback result, the interaction data are classified to obtain hit interaction data and miss interaction data, the hit interaction data and the corresponding first operation instruction form a hit data set, and the miss interaction data and the corresponding first operation instruction form a training data set, for any interaction data in the training data set, the interaction data are identified using a preset model to obtain N recognition results, a second operation instruction is generated for each recognition result, each second operation instruction is evaluated using the hit data set to obtain a corresponding evaluation result, and the second operation instruction whose evaluation result meets the preset conditions is determined to be the target operation instruction, the second feedback result of the user after responding to the target operation instruction is obtained, the preset model is parameter adjusted according to the second feedback result, and the execution of collecting the user's interaction data, the first operation instruction responding to the interaction data, and the first feedback result of the user after the response is returned until the smart device is turned off. A training data set is formed by the user's missed interaction data and the first operation instruction, and a second operation instruction generated by the preset model for identifying the training data set is obtained. Combined with a second feedback result obtained by evaluating the second operation instruction, the parameters of the preset model are adjusted to continuously update the TV's recognition of interaction commands to optimize the TV's intelligent mode.

[0039] After the basic model is trained, when the user turns on the TV next time and operates it again, the TV control model will quickly predict the user's operating intention through the acquired user interaction data, and use voice to actively ask the user whether he wants to perform a certain operation. When the user confirms the intention, a reward hit is completed. If the user refuses, it is considered that the judgment is wrong. After multiple rounds of collecting feedback data and training, the basic model can actively predict the user's intention and actively help the user complete the interaction with the TV, without the user having to go through multiple trial and error each time to complete it.

[0040] For example, the user wants to adjust the brightness of the TV, but the TV voice can only accept instructions to increase or decrease the brightness of the TV. After collecting the user's interaction data and learning enhancement, the user only needs to say "it's too dark" according to his original habit, and the smart device will actively ask and successfully complete the control of the TV brightness.

[0041] like Figure 3 As shown, it is a flowchart of a method for human-computer interaction of an intelligent device provided in Embodiment 3 of the present invention. In step S203, each second operation instruction is evaluated using the hit data set to obtain a corresponding evaluation result, which may include the following steps: Step S301, obtaining a preset reward supervision model, screening the evaluation results according to the reward supervision model, and obtaining corresponding optimized evaluation results.

[0042] Among them, the reward supervision model is a model trained with historical data and user feedback, which is used to evaluate and screen operation instructions. The model can assign a reward value to each operation instruction based on multiple indicators (such as accuracy, user satisfaction, operation execution time, etc.). The model types that can be selected include reinforcement learning models, supervised learning models (such as classifiers or regression models), etc.

[0043] The command text issued by the user (for example, "open the news channel"), the correct operation instruction parsed by the first operation instruction model (for example, open_channel (news channel)), the second operation instruction refers to multiple candidate operation instructions generated by the model (for example, open_channel (news channel), open_app (news application), open_app (weather forecast)), and the evaluation results of each second operation instruction (for example, accuracy, partial matching rate, user satisfaction, etc.).

[0044] Each second operation instruction generated in step S203 and its evaluation result are input into the reward supervision model. The reward supervision model calculates the reward value of each second operation instruction based on the input data, sets a threshold based on the reward value, and selects the second operation instruction with a reward value higher than the threshold as the preferred second operation instruction.

[0045] Step S302: determine that the second operation instruction whose evaluation result after optimization satisfies the preset condition is the target instruction, and obtain the second feedback result of the user after responding to the target operation instruction.

[0046] Among them, the preset conditions refer to the standards for selecting the optimal operation instructions. These conditions can be set according to specific application scenarios and business needs. The final target operation instruction is selected from the second operation instructions after optimization according to the preset conditions. For example, the second operation instruction with the highest reward value is selected as the target operation instruction. If the second operation instruction and evaluation results are generated multiple times, the average reward value of multiple rounds of evaluation results can be taken to select the operation instruction with the highest average reward value. Considering multiple indicators (such as accuracy, user satisfaction, execution time, etc.), select the operation instruction with the best comprehensive indicators. According to the user's preferences and historical behaviors, select the operation instruction that best suits the user's habits.

[0047] The target operation instruction is transmitted to the smart device, and the smart device executes the instruction and records the execution result of the target operation instruction, including operation content, execution time, execution success or failure, etc. After the target operation instruction is executed, the user's second feedback result is collected.

[0048] In this embodiment, the evaluation results are screened through the reward supervision model to obtain the optimized evaluation results, and the second operation instructions that meet the preset conditions after the optimized evaluation results are determined, thereby obtaining the second feedback results of the user. Therefore, the smart device can effectively select the optimal target operation instruction from the optimized second operation instructions, and collect the user's feedback results after execution.

[0049] like Figure 4 As shown, it is a flowchart of a human-computer interaction method of an intelligent device provided in Embodiment 4 of the present invention. In step S204, adjusting parameters of a preset model according to the second feedback result may include the following steps: Step S401, obtain a reward index of a preset reward supervision model, perform error calculation based on the second feedback result and the reward index, and obtain a reward error loss.

[0050] Among them, the reward indicator is used to evaluate the match between the second operation instruction generated by the model and the operation instruction expected by the user. Common reward indicators include user satisfaction score, operation success rate, positive and negative evaluation of user feedback, etc. The specific definition and parameters of the reward indicator are loaded from the preset reward supervision model. Assuming that the reward indicator is a user satisfaction score ranging from 0 to 5, other reward indicators can be defined, such as operation success rate (0 or 1), positive and negative evaluation of user feedback (1 represents positive feedback, -1 represents negative feedback).

[0051] The reward supervision model is used to predict the expected reward value of each second operation instruction, extract the actual reward value in the second feedback result, which can be a user satisfaction score, a Boolean value of whether the operation is successful or not, user behavior, etc. The model and its related parameters are loaded, and the reward supervision model is used to predict the expected reward value according to the second operation instruction and user input, and the error between the actual reward value and the expected reward value is calculated. For example, the error is calculated using methods such as mean square error and absolute error.

[0052] Step S402, according to the reward error loss, the parameters of the preset model are adjusted, and the execution returns to collect the user's interaction data, respond to the first operation instruction of the interaction data, and respond to the first feedback result of the user until the smart device is turned off.

[0053] Among them, according to the calculated reward error loss, the parameters of the preset model are adjusted, the purpose is to make the second operation instruction generated by the model closer to the user's expectations in the future, so as to improve user satisfaction. By calculating the gradient of the error, the model parameters are adjusted to minimize the error, for example, the gradient descent method, only part of the data is used in each update to speed up the training speed. When the smart device is in operation, the above process is continuously performed. When the user issues an instruction to shut down the device, or the device runs out of power, etc., the cycle stops.

[0054] In this embodiment, by setting up the reward index of the reward model and combining the second feedback result to calculate the reward error loss, the preset model is parameterized according to the error loss, and the first operation instruction for collecting the user's interaction data, responding to the interaction data, and the first feedback result of the user after the response are returned until the smart device is turned off, thereby achieving continuous optimization of the model.

[0055] For example, Figure 7 For example, Figure 7 A network structure diagram of a smart device human-computer interaction method provided in Example 4 of the present application, wherein, after the smart device is started, user interaction data is collected, a first operation instruction of the user interaction data and a first feedback result of the user to the smart device are responded to, the user interaction data is classified into hit interaction data and miss interaction data according to the user's first feedback result, and the miss interaction data and its corresponding first operation instruction form a training data set.

[0056] The training data set is serialized and preprocessed, and the preprocessed training data set is transmitted to the basic model for data recognition, and a second operation instruction is generated, and the second operation instruction is transmitted to the data processing model, and the second operation instruction is evaluated according to the hit data set, and the second operation instruction that meets the evaluation result is determined as the target operation instruction, and the target operation instruction is processed by the data processing model and transmitted to the reward supervision model, and the target instruction is screened by the reward supervision model, and the corresponding optimized evaluation result is obtained, and the optimized evaluation result and the data in the device knowledge base are transmitted to the data processing model and transmitted to the user, so that the basic model can be executed in a loop until the smart device is turned off. According to the preset reward threshold, when the target operation instruction meets the hit data set, the reward value is increased until the preset reward threshold is reached, thereby cyclically executing the adjustment of the reward supervision model.

[0057] like Figure 5 As shown, it is a flowchart of a human-computer interaction method of an intelligent device provided in Embodiment 5 of the present invention. The user interaction data in step S201 includes behavior data and voice data. In step S201, the first operation instruction of the response interaction data and the first feedback result of the user after the response are classified according to the first feedback result to obtain the hit interaction data and the miss interaction data, which may include the following steps: Step S501, responding according to the behavior operation instruction of the behavior data to obtain the user's behavior feedback result.

[0058] Step S502, responding according to the voice operation instruction of the voice data to obtain the user's voice feedback result.

[0059] Step S503, classifying the interaction data according to the behavior feedback result and the voice feedback result to obtain hit interaction data and miss interaction data.

[0060] Among them, specific behavioral features (such as buttons, gestures, etc.) and operation intentions (such as opening news channels, playing music, etc.) are extracted from the user's behavioral data. Behavioral data is a structured data. Corresponding operation instructions are generated based on the extracted features and intentions to ensure that smart devices can understand and execute them. Smart devices respond to operations based on the generated operation instructions and collect user behavior feedback results (such as click confirmation, gesture response, etc.). The behavior feedback result is usually a Boolean value, indicating whether the user has confirmed the execution result of the operation instruction.

[0061] Specific voice features (such as voice-to-text text) and operation intentions (such as opening a news channel, playing music, etc.) are extracted from the user's voice data. Voice data is a structured data that contains voice-to-text text and intentions. Corresponding operation instructions are generated based on the extracted features and intentions to ensure that smart devices can understand and execute them. Smart devices respond based on the generated operation instructions and collect user voice feedback results, such as voice confirmation, expression feedback, etc. The voice feedback result is usually a Boolean value, indicating whether the user has confirmed the execution result of the operation instruction.

[0062] First, two lists are initialized to store hit interaction data and miss interaction data respectively. According to the behavior feedback results and voice feedback results, the interaction data are classified into the hit interaction data list and the miss interaction data list respectively, and the classified hit interaction data and miss interaction data are output to facilitate subsequent analysis and improvement.

[0063] In this embodiment, the interaction data includes behavior data and voice data. By classifying the behavior feedback results and voice feedback results, the hit interaction data and the miss interaction data are obtained. This provides a basis for subsequent intention recognition and operation instruction generation. like Figure 6 As shown, it is a flowchart of a human-computer interaction method for an intelligent device provided in Embodiment 6 of the present invention. In step S203, each second operation instruction is evaluated to obtain a corresponding evaluation result, and the second operation instruction whose evaluation result meets the preset condition is determined as the target operation instruction, which may include the following steps: Step S601, sorting the second operation instructions in a reordering manner according to the hit data set to obtain a priority instruction sequence.

[0064] Step S602: Determine the second operation instruction in the priority instruction sequence that meets the preset condition as the target operation instruction.

[0065] Among them, before sorting the second operation instructions, features are first extracted from the hit interaction data. These features will be used as the input of the re-sorting model. The features can be the user's behavior data, voice data, etc. converted into a format that the model can process. The extracted features are input into the re-sorting model to obtain the probability score of each second operation instruction. The second operation instructions are sorted according to the probability score to obtain a priority instruction sequence.

[0066] Define preset conditions based on specific needs. Common conditions include selecting the instruction with the highest probability, selecting the most common instruction in the user's historical behavior, etc. Evaluate the instructions in the priority instruction sequence one by one, and determine the instructions that meet the preset conditions as the target operation instructions. If there are no instructions that meet the conditions, return None or other default values.

[0067] In this embodiment, the hit data set is sorted to obtain a priority instruction sequence, and then the second operation instruction that satisfies the preset condition in the priority instruction sequence is determined as the target operation instruction. In this way, the accuracy of the system and the user experience are improved by combining user feedback and new interaction data.

[0068] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0069] like Figure 8 As shown, it is a schematic diagram of a human-machine interaction device for an intelligent device provided in Embodiment 7 of the present invention, and the human-machine interaction device for an intelligent device corresponds one-to-one to the human-machine interaction method for an intelligent device in the above-mentioned embodiment. The human-machine interaction device for an intelligent device includes a data classification module 81, an instruction generation module 82, an evaluation module 83 and an adjustment module 84. The functional modules are described in detail as follows: The data classification module 81 is used to collect the user's interaction data, the first operation instruction in response to the interaction data, and the first feedback result of the user after the response, and classify the interaction data according to the first feedback result to obtain the hit interaction data and the miss interaction data, and form a hit data set with the hit interaction data and the corresponding first operation instruction, and form a training data set with the miss interaction data and the corresponding first operation instruction; The instruction generation module 82 is used to identify any interaction data in the training data set using a preset model to obtain N recognition results, and generate a second operation instruction for each recognition result, where N is an integer greater than zero; An evaluation module 83 is used to evaluate each second operation instruction using the hit data set to obtain a corresponding evaluation result, determine the second operation instruction whose evaluation result meets the preset condition as a target operation instruction, and obtain a second feedback result of the user after responding to the target operation instruction; The adjustment module 84 is used to adjust the parameters of the preset model, return to execute the first operation instruction of collecting the user's interaction data, responding to the interaction data, and the first feedback result of the user after the response, until the smart device is turned off.

[0070] Optionally, the evaluation module 83 includes: A screening unit is used to obtain a preset reward supervision model, screen the evaluation results according to the reward supervision model, and obtain the corresponding optimized evaluation results; The result acquisition unit is used to determine that the second operation instruction whose evaluation result after optimization satisfies the preset condition is a target instruction, and acquire the second feedback result of the user after responding to the target operation instruction.

[0071] Optionally, the intelligent device human-computer interaction apparatus further includes: A data cleaning module is used to clean the data according to the training data set using cleaning rules to obtain a cleaned training data set; A unification module is used to unify the data according to the cleaned training data set using standardized specifications to obtain a training data set in a unified format; The interactive data recognition module is used to use a preset model to recognize any interactive data in a training data set of a unified format, obtain N recognition results, and generate a second operation instruction for each recognition result, where N is an integer greater than zero.

[0072] Optionally, the adjustment module 84 includes: An error calculation unit, used to obtain a reward index of a preset reward supervision model, perform error calculation according to the second feedback result and the reward index, and obtain a reward error loss; The parameter adjustment unit is used to adjust the parameters of the preset model according to the reward error loss, return to execute the first operation instruction of collecting the user's interaction data, responding to the interaction data, and the first feedback result of the user after the response, until the smart device is turned off.

[0073] Optionally, the instruction generating module 82 includes: A feature extraction unit, used for extracting features according to any interaction data to obtain interaction features; The intention recognition unit is used to recognize the user's interaction intention based on the interaction characteristics using a pre-trained model, obtain N intention recognition results, and generate a second operation instruction for each intention recognition result.

[0074] Optionally, the data classification module 81 further includes: A behavior feedback unit responds to the behavior operation instructions of the behavior data to obtain the user's behavior feedback results; A voice feedback unit responds to the voice operation instruction of the voice data to obtain the user's voice feedback result; The interaction data classification unit is used to classify the interaction data according to the behavior feedback results and the voice feedback results to obtain the hit interaction data and the miss interaction data.

[0075] Optionally, the evaluation module 83 further includes: A priority sorting unit, used to sort the second operation instructions in a re-sorting manner according to the hit data set to obtain a priority instruction sequence; The determination unit is used to determine that the second operation instruction in the priority instruction sequence that meets the preset condition is the target operation instruction.

[0076] For the specific definition of the intelligent device human-computer interaction device, please refer to the definition of the intelligent device human-computer interaction method above, which will not be repeated here. Each module in the above-mentioned intelligent device human-computer interaction device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0077] like Fig. 9 As shown, it is a schematic diagram of the structure of a computer device provided in Embodiment 8 of the present invention. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a human-computer interaction method of an intelligent device is implemented.

[0078] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the human-computer interaction method of the intelligent device in the above embodiment is implemented, for example Figures 2 to 7 Alternatively, when the processor executes the computer program, the functions of each module / unit in the embodiment of the human-computer interaction device of the intelligent device are realized, for example Figure 8 The functions of the data classification module 81, instruction generation module 82, evaluation module 83 and adjustment module 84 are not described here in detail to avoid repetition.

[0079] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the human-computer interaction method of the smart device in the above embodiment is implemented. Figures 2 to 7 Alternatively, when the computer program is executed by the processor, the functions of each module / unit in the embodiment of the above-mentioned intelligent device human-computer interaction device are realized, for example Figure 8 The functions of the data classification module 81, instruction generation module 82, evaluation module 83 and adjustment module 84 are not described here in detail to avoid repetition. The computer-readable storage medium may be non-volatile or volatile.

[0080] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0081] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0082] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A human-computer interaction method for an intelligent device, characterized in that: include: After the smart device is started, the user's interaction data, the first operation instruction in response to the interaction data, and the first feedback result of the user after the response are collected, and the interaction data are classified according to the first feedback result to obtain hit interaction data and miss interaction data, and the hit interaction data and the corresponding first operation instruction form a hit data set, and the miss interaction data and the corresponding first operation instruction form a training data set; For any interaction data in the training data set, use a preset model to identify the interaction data to obtain N recognition results, and generate a second operation instruction for each recognition result, where N is an integer greater than zero; Using the hit data set, evaluating each second operation instruction to obtain a corresponding evaluation result, determining the second operation instruction whose evaluation result meets a preset condition as a target operation instruction, and obtaining a second feedback result of the user after responding to the target operation instruction; According to the second feedback result, the parameters of the preset model are adjusted, and the first operation instruction for collecting the user's interaction data, responding to the interaction data, and the first feedback result of the user after the response are returned to execute until the smart device is turned off.

2. The intelligent device human-computer interaction method according to claim 1, characterized in that: The using the hit data set to evaluate each second operation instruction to obtain a corresponding evaluation result includes: Obtaining a preset reward supervision model, screening the evaluation results according to the reward supervision model, and obtaining corresponding optimized evaluation results; A second operation instruction in which the evaluation result after optimization satisfies a preset condition is determined as a target instruction, and a second feedback result of the user after responding to the target operation instruction is obtained.

3. The intelligent device human-computer interaction method according to claim 2, characterized in that: The step of adjusting parameters of the preset model according to the second feedback result includes: Obtaining a reward indicator of a preset reward supervision model, performing error calculation according to the second feedback result and the reward indicator, and obtaining a reward error loss; According to the reward error loss, the parameters of the preset model are adjusted, and the collection of user interaction data, the first operation instruction responding to the interaction data, and the first feedback result of the user after the response are returned to execute until the smart device is turned off.

4. The intelligent device human-computer interaction method according to claim 1, characterized in that: After forming a training data set with the missed interaction data and the corresponding first operation instruction, the method further includes: According to the training data set, using a cleaning rule to perform data cleaning to obtain a cleaned training data set; According to the cleaned training data set, the data is unified using a standardized specification to obtain a training data set in a unified format; For any interaction data in the training data set in the unified format, the interaction data is identified using a preset model to obtain N identification results, and a second operation instruction is generated for each identification result, where N is an integer greater than zero.

5. The intelligent device human-computer interaction method according to claim 1, characterized in that: The interactive data is identified using a preset model to obtain N identification results, including: Extracting features based on any of the interaction data to obtain interaction features; According to the interaction features, a pre-trained model is used to identify the user's interaction intention, N intention recognition results are obtained, and a second operation instruction is generated for each intention recognition result.

6. The intelligent device human-computer interaction method according to claim 1, characterized in that: The user's interaction data includes behavior data and voice data; The first operation instruction in response to the interaction data and the first feedback result of the user after the response, and classifying the interaction data according to the first feedback result to obtain hit interaction data and miss interaction data, including: Responding to the behavior operation instruction of the behavior data to obtain the user's behavior feedback result; Responding to the voice operation instruction of the voice data to obtain a voice feedback result from the user; The interaction data is classified according to the behavior feedback result and the voice feedback result to obtain hit interaction data and miss interaction data.

7. The intelligent device human-computer interaction method according to any one of claims 1 to 6, characterized in that: The step of evaluating each second operation instruction to obtain a corresponding evaluation result, and determining the second operation instruction whose evaluation result meets a preset condition as a target operation instruction, comprises: According to the hit data set, the second operation instructions are sorted in a reordering manner to obtain a priority instruction sequence; A second operation instruction that satisfies a preset condition in the priority instruction sequence is determined as a target operation instruction.

8. A device for human-computer interaction of intelligent devices, characterized in that: include: A data classification module is used to collect user interaction data, a first operation instruction in response to the interaction data, and a first feedback result of the user after the response, and classify the interaction data according to the first feedback result to obtain hit interaction data and miss interaction data, and form a hit data set with the hit interaction data and the corresponding first operation instruction, and form a training data set with the miss interaction data and the corresponding first operation instruction; An instruction generation module, for identifying any interaction data in the training data set using a preset model, obtaining N recognition results, and generating a second operation instruction for each recognition result, where N is an integer greater than zero; An evaluation module, configured to evaluate each second operation instruction using the hit data set to obtain a corresponding evaluation result, determine that the second operation instruction whose evaluation result meets a preset condition is a target operation instruction, and obtain a second feedback result of the user after responding to the target operation instruction; The adjustment module is used to adjust the parameters of the preset model, return to execute the first operation instruction of collecting the user's interaction data, responding to the interaction data, and the first feedback result of the user after the response, until the smart device is turned off.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the human-computer interaction method of the intelligent device as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the human-computer interaction method of an intelligent device as claimed in any one of claims 1 to 7 is implemented.

Images