Intelligent device human-computer interaction method and device, computer device, and storage medium

Abstract

The present application relates to the technical field of artificial intelligence, and in particular to a human-computer interaction method for intelligent devices, which comprises the following steps: collecting interaction data of a user, a first operation instruction in response to the interaction data, and a first feedback result of the user after the response; classifying the interaction data; identifying the interaction data using a preset model; generating a second operation instruction based on the identification result; evaluating the second operation instruction; determining a target operation instruction if the preset condition is met; adjusting model parameters based on a second feedback result of the user after responding to the target operation instruction; and returning to execution. The training data set is formed by the user's missed interaction data and the first operation instruction, the second operation instruction generated by the preset model in response to the training data set is obtained, and the second feedback result obtained by evaluating the second operation instruction is combined to adjust the parameters of the preset model, so as to continuously update the identification of the television on the interaction command and optimize the intelligent mode of the intelligent television and other intelligent devices, thereby improving the response accuracy of the intelligent devices.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to an intelligent device human-computer interaction method and device, a computer device, and a storage medium. BACKGROUND

[0002] With the development of artificial intelligence technology, users can use voice commands, gestures, and other methods to perform channel switching, playback control, content search, and other operations, improving the convenience and naturalness of interaction.

[0003] Currently, the operating system and user UI operation logic of intelligent devices such as smart TVs are becoming more complex, and the user's learning cost is increasing, making it difficult to learn to control the TV product. Traditional TV AI voice control is still a command-based response, and the user's speech intent recognition and interpretation ability is not enough, often resulting in the user wanting to control the TV but failing to understand the intent. At the same time, the TV itself lacks semantic understanding and self-learning ability, leading to repeated problems that are difficult to solve.

[0004] Therefore, how to automatically and accurately identify user feedback to improve the response accuracy of intelligent devices has become a problem to be solved. SUMMARY

[0005] Therefore, the present application provides an intelligent device human-computer interaction method and device, a computer device, and a storage medium to solve the problem of how to automatically and accurately identify user feedback to improve the response accuracy of intelligent devices.

[0006] In a first aspect, an embodiment of the present application provides an intelligent device human-computer interaction method, comprising:

[0007] After the intelligent device is started, the user's interaction data, the first operation instruction in response to the interaction data, and the first feedback result of the user after the response are collected, the interaction data is classified according to the first feedback result, the hit interaction data and the missed interaction data are obtained, the hit interaction data and the corresponding first operation instruction form a hit data set, and the missed interaction data and the corresponding first operation instruction form a training data set;

[0008] For any interaction data in the training data set, a preset model is used to identify the interaction data, N identification results are obtained, a second operation instruction is generated for each identification result, and N is an integer greater than zero;

[0009] Each second operation instruction is evaluated using the hit data set to obtain a corresponding evaluation result, the second operation instruction that meets the preset condition is determined as the target operation instruction, and the second feedback result of the user in response to the target operation instruction is obtained.

[0010] According to the second feedback result, parameters of the preset model are adjusted, and collection of the interaction data of the user, the first operation instruction in response to the interaction data, and the first feedback result of the user after the response are performed until the smart device is closed.

[0011] In a second aspect, an embodiment of the present application provides a device for human-computer interaction of a smart device, comprising:

[0012] A data classification module is configured to collect the interaction data of the user, the first operation instruction in response to the interaction data, and the first feedback result of the user after the response, classify the interaction data according to the first feedback result, obtain the hit interaction data and the miss interaction data, form a hit data set by combining the hit interaction data and the corresponding first operation instruction, and form a training data set by combining the miss interaction data and the corresponding first operation instruction.

[0013] An instruction generation module is configured to use a preset model to identify any interaction data in the training data set, obtain N identification results, generate a second operation instruction for each identification result, and N is an integer greater than zero.

[0014] An evaluation module is configured to use the hit data set to evaluate each second operation instruction, obtain a corresponding evaluation result, determine a second operation instruction satisfying a preset condition as a target operation instruction, and obtain a second feedback result of the user after the response to the target operation instruction.

[0015] An adjustment module is configured to adjust parameters of the preset model, and return to perform collection of the interaction data of the user, the first operation instruction in response to the interaction data, and the first feedback result of the user after the response until the smart device is closed.

[0016] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method for human-computer interaction of a smart device in the first aspect when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executable on a processor to implement the method for human-computer interaction of a smart device in the first aspect.

[0018] Compared with the prior art, the present application has the beneficial effects that: by collecting the interaction data of the user after the intelligent device is started, the first operation instruction in response to the interaction data and the first feedback result of the user after the response, the interaction data is classified according to the first feedback result, the hit interaction data and the missed interaction data are obtained, the hit interaction data and the corresponding first operation instruction form a hit data set, the missed interaction data and the corresponding first operation instruction form a training data set, for any interaction data in the training data set, the interaction data is identified using a preset model, N identification results are obtained, a second operation instruction is generated for each identification result, each second operation instruction is evaluated using the hit data set, the corresponding evaluation result is obtained, the second operation instruction that meets the preset condition is determined as the target operation instruction, the second feedback result of the user after responding to the target operation instruction is obtained, the preset model is parameter adjusted according to the second feedback result, and the collection of the interaction data of the user, the first operation instruction in response to the interaction data and the first feedback result of the user after the response are returned to be executed until the intelligent device is closed. Through the training data set formed by the missed interaction data of the user and the first operation instruction, the second operation instruction generated by the preset model in response to the training data set is obtained, the second feedback result obtained by evaluating the second operation instruction is combined, and the preset model is parameter adjusted, so as to continuously update the identification of the interaction command of the television to optimize the intelligent mode of the intelligent television and other intelligent devices, thereby improving the response accuracy of the intelligent device. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0020] Figure 1 is an application environment schematic diagram of a smart device human-computer interaction method provided by an embodiment of the present application;

[0021] Figure 2 is a flow schematic diagram of a smart device human-computer interaction method provided by an embodiment of the present application;

[0022] Figure 3 is a flow schematic diagram of a smart device human-computer interaction method provided by an embodiment of the present application;

[0023] Figure 4 is a flow schematic diagram of a smart device human-computer interaction method provided by an embodiment of the present application;

[0024] Figure 5is a flow diagram of a smart device human-computer interaction method provided by the fifth embodiment of the present application;

[0025] Figure 6 is a flow diagram of a smart device human-computer interaction method provided by the sixth embodiment of the present application;

[0026] Figure 7 is a network structure diagram of a smart device human-computer interaction method provided by the fourth embodiment of the present application;

[0027] Figure 8 is a structure diagram of a smart device human-computer interaction device provided by the seventh embodiment of the present application;

[0028] Figure 9 is a structure diagram of a computer device provided by the eighth embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0030] As shown in Figure 1 , it is an application environment diagram of a smart device human-computer interaction method provided by the first embodiment of the present application, wherein the client and the server are connected to communicate, the user can provide conditions, demands and operation instructions for the smart device human-computer interaction to the server by operating the client, and the server is used to execute the smart device human-computer interaction method of the present application according to the related content sent by the client. The client includes but is not limited to various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices and other computer devices. The computer device corresponding to the server can be realized by an independent server or a server cluster composed of multiple servers.

[0031] As shown in Figure 2 , it is a flow diagram of a smart device human-computer interaction method provided by the second embodiment of the present application, wherein the smart device human-computer interaction method is applied to the server in Figure 1 . The smart device human-computer interaction method can include the following steps:

[0032] Step S201, after the smart device is started, the interactive data of the user, the first operation instruction in response to the interactive data and the first feedback result of the user after response are collected, the interactive data is classified according to the first feedback result, the hit interactive data and the missed interactive data are obtained, the hit interactive data and the corresponding first operation instruction form a hit data set, and the missed interactive data and the corresponding first operation instruction form a training data set.

[0033] When the user opens the smart TV, the system will start and enter the initialization state, and be ready to receive and process the interactive data of the user, which includes but is not limited to voice command, remote control operation, gesture recognition and touch screen input.

[0034] The voice recognition can refer to the instruction issued by the user through the voice assistant, the remote control operation can refer to the operation performed by the user through the buttons or touch screen of the remote control, the gesture recognition can refer to the gesture operation performed by the user through the camera or gesture recognition device, and the touch screen input can refer to the interactive operation performed by the user through the touch screen of the smart TV.

[0035] The collection method includes but is not limited to voice recognition module, remote control interface, camera module and touch screen sensor.

[0036] The voice recognition module can refer to capturing the voice instruction of the user through the microphone array and converting it into text form, the remote control interface can refer to receiving the remote control operation of the user through infrared or Bluetooth, the camera module can refer to capturing the gesture of the user through the camera and performing gesture recognition, and the touch screen sensor can refer to recording the touch operation of the user through the touch screen sensor.

[0037] The response of the smart TV to the interactive data of the user can be parsed as a first operation instruction, the smart TV performs corresponding operations, for example, opening a certain application program, switching channels, adjusting the volume, etc., and the user operation result will be reflected on the user interface, such as displaying a new channel, application startup, etc.

[0038] The first feedback result of the user includes but is not limited to voice feedback, operation feedback and display feedback.

[0039] The voice feedback can refer to the user expressing satisfaction or dissatisfaction through voice, such as "right" or "this is not what I want", the operation feedback can refer to the user expressing dissatisfaction through further operation, such as immediately switching back to the original channel, closing the application, etc., and the explicit feedback can refer to the smart TV providing a feedback button or a scoring system, and the user expressing satisfaction or dissatisfaction by clicking or scoring. The data classification of the user's response to the smart TV as hit data, and the data classification of the user's response to the smart TV as missed data.

[0040] Optionally, after the step of forming the training data set by the interaction data that is not hit and the corresponding first operation instruction in step S201, the intelligent device human-computer interaction method can further include the following steps:

[0041] According to the training data set, data cleaning is performed using a cleaning rule to obtain a cleaned training data set.

[0042] According to the cleaned training data set, data unification is performed using a standardization specification to obtain a training data set in a unified format.

[0043] For any interaction data in the training data set in a unified format, a preset model is used to identify the interaction data to obtain N identification results, and a second operation instruction is generated for each identification result, N being an integer greater than zero.

[0044] Among them, the definition of the cleaning rule is to remove invalid data, remove duplicates, format, and noise processing. For example, delete incomplete, incorrect or irrelevant data, delete duplicate interaction data, convert data format, ensure consistency of all data, and remove background noise, misidentification and other interference data.

[0045] Among them, the specific steps of data cleaning are to check whether each interaction data contains user input, first operation instruction and identification of not hit, delete records missing key information, use hash table or set data structure to record the unique identifier of each interaction data, use speech recognition technology to filter background noise, remove misrecognized words through text preprocessing technology, check whether the new record exists in the set, and delete if it exists. For example, delete records with empty user input or records with incorrect format of first operation instruction, delete duplicate records with identical user input and first operation instruction, use noise filter to remove background sound in user input, and remove invalid words through spelling check and dictionary matching.

[0046] By performing data cleaning on the training data set and performing data unification on the cleaned data set, a training data set in a unified format is obtained, and then the interaction data in the training data set is identified to obtain N identification results, and a second operation instruction is generated for the identification result, thereby providing a unified data format for subsequent processing of the training data set.

[0047] Step S202, for any interaction data in the training data set, a preset model is used to identify the interaction data to obtain N identification results, and a second operation instruction is generated for each identification result, N being an integer greater than zero.

[0048] The interaction data that does not match is selected from the training data set generated in step S201, and the interaction data includes the original input of the user (such as a voice command, a remote control operation, etc.) and the corresponding first operation instruction. The selected interaction data is preprocessed to adapt to the input requirements of the preset model. For example, the voice is converted into text, noise is removed, the text format is standardized, the operation code is converted into an understandable operation type, the gesture data is converted into a standardized feature vector, and the touch event sequence is converted into a standardized input format. The preset model generates N recognition results according to the input data, and the recognition results are multiple operation instructions that the model considers to be the user's possible intentions. Each recognition result is converted into a specific second operation instruction, and the second operation instruction will be used for subsequent evaluation and optimization.

[0049] Optionally, the identification of the interaction data in step S202 using the preset model to obtain N recognition results can include the following steps:

[0050] According to any interaction data, feature extraction is performed to obtain interaction features;

[0051] According to the interaction features, a pre-trained model is used to identify the user's interaction intention to obtain N intention recognition results, and a second operation instruction is generated for each intention recognition result.

[0052] Among them, some commonly used pre-trained language models can be selected, the pre-trained language model and its related parameters are loaded from the model library, the user input data is input into the pre-trained language model, the output features of the model are obtained, and the last layer of hidden state is usually used as the feature. These feature vectors contain rich semantic information and can be used for subsequent intention recognition and operation instruction generation.

[0053] Among them, the pre-trained model can select some commonly used intention recognition models, such as a transformer model (Transformer) or other machine learning models. The pre-trained model and its related parameters are loaded from the model library to ensure that the model can operate normally. The interaction features extracted from the language model are input into the intention recognition model, and N most possible intention recognition results and their corresponding probabilities are obtained from the model output. A mapping table from the intention label to the operation instruction is constructed to facilitate the conversion of the intention label to the specific operation instruction. Each intention label generates a corresponding operation instruction to obtain N second operation instructions.

[0054] Step S203, using the hit data set, evaluating each second operation instruction to obtain the corresponding evaluation result, determining the second operation instruction that satisfies the preset condition as the target operation instruction, and obtaining the second feedback result of the user after responding to the target operation instruction.

[0055] The hit data set contains the interaction data of the hits generated in step S201 and the corresponding first operation instructions. These data have been verified as correct parsing and execution, which are the "correct answers" of the model. Each piece of interaction data in the hit data set and its corresponding first operation instruction is compared with the second operation instruction generated in step S202 to evaluate the accuracy of the second operation instruction. The percentage of the second operation instruction matching the first operation instruction in some aspects (for example, the command type matches but the specific parameters are inconsistent), and whether the user is satisfied with the execution result of the second operation instruction can be judged by the user's subsequent operation or explicit feedback. For each second operation instruction, an evaluation result is generated, which can be a score or a classification (such as correct, partially correct, and incorrect).

[0056] The preset condition can be set according to the specific business requirements and system performance to set the threshold of the evaluation result. For example, the accuracy threshold is more than 80%, the partial matching rate threshold is more than 60%, and the user satisfaction threshold is more than 90%. The second operation instruction that meets the preset condition of the evaluation result is selected as the target operation instruction. For example, if the preset condition is that the accuracy is greater than 80%, only the second operation instruction with an accuracy greater than 80% will be selected.

[0057] In step S204, the second feedback result is used to adjust the parameters of the preset model, and the collection of the user's interaction data, the first operation instruction in response to the interaction data, and the first feedback result of the user after the response are returned until the smart device is turned off.

[0058] The second feedback result can be obtained from the feedback result of the user to the target operation instruction in step S203, which can be positive feedback or negative feedback. Positive feedback indicates that the target operation instruction is correct or the user is satisfied, and the parameters of the model do not need to be adjusted. Negative feedback indicates that the target operation instruction is incorrect or the user is not satisfied, and the model needs to be adjusted. The data of positive feedback and negative feedback are incrementally added to the training set of the model for online learning, and the model parameters are updated continuously. The positive feedback data can increase the weight of the correct operation instruction in the model, and the negative feedback data can reduce the weight of the incorrect operation instruction in the model, or increase the training intensity of the correct operation instruction. For a deep learning-based model, the model parameters are adjusted using the back propagation algorithm according to the negative feedback result, the loss function of the model is calculated according to the negative feedback result, and the model parameters are updated using the gradient descent method or other optimization algorithms to reduce the loss.

[0059] After adjusting the parameters of the preset model, return to step S201 to continue collecting the user's interaction data, the first operation instruction in response to the interaction data, and the first feedback result of the user after the response. Through continuous user feedback and model adjustment, the interaction experience of the smart device will be gradually optimized, and the user satisfaction will be improved.

[0060] The embodiment of the application collects the interaction data of the user, the first operation instruction in response to the interaction data and the first feedback result of the user after the response, classifies the interaction data according to the first feedback result, obtains the hit interaction data and the miss interaction data, forms a hit data set by combining the hit interaction data and the corresponding first operation instruction, forms a training data set by combining the miss interaction data and the corresponding first operation instruction, identifies the interaction data using a preset model for any interaction data in the training data set, obtains N identification results, generates a second operation instruction for each identification result, evaluates each second operation instruction using the hit data set, obtains the corresponding evaluation result, determines the second operation instruction satisfying the preset condition as the target operation instruction, obtains the second feedback result of the user after responding to the target operation instruction, adjusts the parameters of the preset model according to the second feedback result, returns to collect the interaction data of the user, the first operation instruction in response to the interaction data and the first feedback result of the user after the response, and stops until the smart device is closed. The training data set is formed by the miss interaction data of the user and the first operation instruction, the second operation instruction generated by the preset model in response to the identification of the training data set, the second feedback result obtained by evaluating the second operation instruction, and the parameters of the preset model are adjusted to continuously update the identification of the interaction command of the television to optimize the intelligent mode of the television.

[0061] After the training of the basic model, when the user next time turns on the television and re-operates the television, the television control model quickly predicts the operation intention of the user through the obtained user interaction data, and calls the voice to actively ask the user whether it is intended to perform a certain operation. After the user confirms the intention, a reward hit is completed. If the user refuses, it is considered that the judgment is wrong. After a plurality of rounds of collection of feedback data and training, the basic model can actively predict the intention of the user and actively help the user complete the interaction with the television, without the user needing to try and error for a plurality of times each time.

[0062] For example, the intention of the user is to adjust the brightness of the television, but the voice of the television can only accept the instruction type speech of the user for increasing or decreasing the brightness of the television. After collecting the interaction data of the user and learning and enhancing, the user only needs to say that it is too dark according to the original habit, and the smart device will actively ask and successfully complete the control of the brightness of the television.

[0063] As shown in FIG. 3, it is a flowchart of a method for human-computer interaction of a smart device according to an embodiment of the application. The method comprises the following steps. Figure 3

[0064] ​Step S301, a preset reward supervision model is obtained, and the evaluation results are filtered according to the reward supervision model to obtain corresponding preferred evaluation results.

[0065] The reward supervision model is a model trained by historical data and user feedback, used for evaluating and filtering operation instructions. The model can assign a reward value to each operation instruction according to multiple indicators such as accuracy, user satisfaction, operation execution time, etc. The model type can be selected from reinforcement learning models, supervised learning models (such as classifiers or regression models), etc.

[0066] The instruction text issued by the user (for example, "open news channel"), the correct operation instruction parsed by the first operation instruction model (for example, open_channel (news channel)), the multiple candidate operation instructions generated by the second operation instruction model (for example, open_channel (news channel), open_app (news application), open_app (weather forecast)), and the evaluation results of each second operation instruction (for example, accuracy, partial matching rate, user satisfaction, etc.).

[0067] Each second operation instruction and its evaluation result generated in step S203 are input into the reward supervision model. The reward supervision model calculates the reward value of each second operation instruction according to the input data, sets a threshold according to the reward value, and selects the second operation instruction with a reward value higher than the threshold as the preferred second operation instruction.

[0068] Step S302, determine the second operation instruction whose preferred evaluation result meets the preset condition as the target instruction, and obtain the second feedback result of the user after responding to the target operation instruction.

[0069] The preset condition refers to the standard for selecting the optimal operation instruction. These conditions can be set according to specific application scenarios and business requirements. The final target operation instruction is selected from the preferred second operation instruction according to the preset condition. For example, the second operation instruction with the highest reward value is selected as the target operation instruction. If multiple second operation instructions and evaluation results are generated, the average reward value of multiple rounds of evaluation results can be taken, and the operation instruction with the highest average reward value is selected. Multiple indicators (such as accuracy, user satisfaction, execution time, etc.) are considered, and the operation instruction with the best comprehensive indicator is selected. The operation instruction that best meets the user's habits is selected according to the user's preferences and historical behavior.

[0070] The target operation instruction is transmitted to the smart device, the smart device executes the instruction, records the execution result of the target operation instruction, including operation content, execution time, execution success or failure, etc. After the execution of the target operation instruction, the second feedback result of the user is collected.

[0071] In this embodiment, the evaluation results are filtered by the reward supervision model to obtain the filtered evaluation results, and a second operation instruction that meets a preset condition is determined from the filtered evaluation results, so as to obtain the second feedback result of the user. Therefore, the intelligent device can effectively select the optimal target operation instruction from the filtered second operation instructions and execute the target operation instruction to collect the feedback result of the user.

[0072] As shown in Figure 4 FIG. 4 is a flowchart of a method for human-computer interaction of an intelligent device according to an embodiment of the present application. In step S204, the parameters of the preset model are adjusted according to the second feedback result, which can include the following steps:

[0073] In step S401, the reward index of the preset reward supervision model is obtained, and the error is calculated according to the second feedback result and the reward index to obtain the reward error loss.

[0074] The reward index is used to evaluate the matching degree between the second operation instruction generated by the model and the operation instruction expected by the user. Common reward indexes include user satisfaction score, operation success rate, and positive and negative evaluations of the user feedback. The specific definition and parameters of the reward index are loaded from the preset reward supervision model. Assuming that the reward index is a user satisfaction score ranging from 0 to 5, other reward indexes can be defined, such as operation success rate (0 or 1) and positive and negative evaluations of the user feedback (1 for positive feedback and -1 for negative feedback).

[0075] The reward supervision model is used to predict the expected reward value of each second operation instruction, and the actual reward value in the second feedback result is extracted. The second feedback result can be a user satisfaction score, a Boolean value indicating whether the operation is successful, or user behavior. The model and its related parameters are loaded, and the expected reward value is predicted according to the second operation instruction and the user input using the reward supervision model. The error between the actual reward value and the expected reward value is calculated. For example, the error is calculated using methods such as mean square error and absolute error.

[0076] In step S402, the parameters of the preset model are adjusted according to the reward error loss, and the process of collecting the interaction data of the user, the first operation instruction in response to the interaction data, and the first feedback result of the user after responding is returned until the intelligent device is turned off.

[0077] The parameters of the preset model are adjusted according to the calculated reward error loss, and the purpose is to make the second operation instruction generated by the model closer to the expectation of the user in the future, thereby improving the user satisfaction. The gradient of the error is calculated, and the model parameters are adjusted to minimize the error, for example, the gradient descent method, which uses only part of the data each time to speed up the training. When the intelligent device is in a running state, the above process is continuously performed, and when the user issues a command to turn off the device or the device runs out of power, etc., the cycle is stopped.

[0078] In the embodiment, the reward error loss is calculated by setting the reward index of the reward model and combining the second feedback result, the preset model is adjusted according to the error loss, and the above-mentioned collection of user interaction data, the first operation instruction in response to the interaction data and the first feedback result of the user after the response are returned until the smart device is closed. Thus, the continuous optimization of the model is realized.

[0079] For example, the reward error loss is calculated by setting the reward index of the reward model and combining the second feedback result, the preset model is adjusted according to the error loss, and the above-mentioned collection of user interaction data, the first operation instruction in response to the interaction data and the first feedback result of the user after the response are returned until the smart device is closed. Thus, the continuous optimization of the model is realized. Figure 7 For example, the reward error loss is calculated by setting the reward index of the reward model and combining the second feedback result, the preset model is adjusted according to the error loss, and the above-mentioned collection of user interaction data, the first operation instruction in response to the interaction data and the first feedback result of the user after the response are returned until the smart device is closed. Thus, the continuous optimization of the model is realized. Figure 7 As shown in FIG. 1, it is a network structure schematic diagram of a smart device human-computer interaction method provided by the embodiment four of the present application, wherein after the smart device is started, the interaction data of the user is collected, the first operation instruction in response to the user interaction data and the first feedback result of the user to the smart device are responded, the interaction data of the user is classified as the hit interaction data and the miss interaction data according to the first feedback result of the user, and the miss interaction data and the corresponding first operation instruction form a training data set.

[0080] The training data set is serialized and preprocessed, the preprocessed training data set is transmitted to the basic model for data recognition, and the second operation instruction is generated, the second operation instruction is transmitted to the data processing model, the second operation instruction is evaluated according to the hit data set, the second operation instruction meeting the evaluation result is determined as the target operation instruction, the target operation instruction is processed by the data processing model and transmitted to the reward supervision model, the target instruction is filtered by the reward supervision model, and the corresponding preferred evaluation result is obtained, the preferred evaluation result and the data in the device knowledge base are transmitted to the data processing model and transmitted to the user, so that the basic model can be executed in a loop until the smart device is closed, the reward value is increased when the target operation instruction meets the hit data set according to the preset reward threshold, and the adjustment of the reward supervision model is executed in a loop until the preset reward threshold is reached.

[0081] As shown in FIG. 1, it is a network structure schematic diagram of a smart device human-computer interaction method provided by the embodiment four of the present application, wherein after the smart device is started, the interaction data of the user is collected, the first operation instruction in response to the user interaction data and the first feedback result of the user to the smart device are responded, the interaction data of the user is classified as the hit interaction data and the miss interaction data according to the first feedback result of the user, and the miss interaction data and the corresponding first operation instruction form a training data set. Figure 5 As shown in FIG. 1, it is a network structure schematic diagram of a smart device human-computer interaction method provided by the embodiment four of the present application, wherein after the smart device is started, the interaction data of the user is collected, the first operation instruction in response to the user interaction data and the first feedback result of the user to the smart device are responded, the interaction data of the user is classified as the hit interaction data and the miss interaction data according to the first feedback result of the user, and the miss interaction data and the corresponding first operation instruction form a training data set.

[0082] Step S501, the behavior operation instruction of the behavior data is responded to obtain the behavior feedback result of the user.

[0083] Step S502, the voice operation instruction of the voice data is responded to obtain the voice feedback result of the user.

[0084] Step S503: Based on the behavioral feedback results and voice feedback results, classify the interaction data to obtain the matched interaction data and the unmatched interaction data.

[0085] Specifically, user behavior data is used to extract specific behavioral features (such as buttons, gestures, etc.) and operational intentions (such as opening news channels, playing music, etc.). Behavioral data is structured data. Based on the extracted features and intentions, corresponding operation instructions are generated to ensure that smart devices can understand and execute them. Smart devices respond to the generated operation instructions and collect user behavior feedback results (such as clicking to confirm, gesture response, etc.). The behavior feedback result is usually a Boolean value, indicating whether the user has confirmed the execution result of the operation instruction.

[0086] Specific voice features (such as speech-to-text) and operational intentions (such as opening a news channel or playing music) are extracted from the user's voice data. The voice data is structured data, which includes speech-to-text and intentions. Based on the extracted features and intentions, corresponding operation instructions are generated to ensure that the smart device can understand and execute them. The smart device responds to the generated operation instructions and collects the user's voice feedback results, such as voice confirmation and facial expression feedback. The voice feedback result is usually a Boolean value, indicating whether the user has confirmed the execution result of the operation instruction.

[0087] First, initialize two lists to store the matched and unmatched interaction data, respectively. Based on the behavioral and verbal feedback results, classify the interaction data into the matched and unmatched interaction data lists, and output the classified matched and unmatched interaction data for subsequent analysis and improvement.

[0088] In this embodiment, the interaction data includes behavioral data and voice data. By classifying the behavioral feedback results and voice feedback results, the matching interaction data and the non-matching interaction data are obtained. This provides a foundation for subsequent intent recognition and operation command generation.

[0089] like Figure 6 The diagram shown is a flowchart of a human-computer interaction method for an intelligent device according to Embodiment Six of the present invention. In step S203, each second operation instruction is evaluated to obtain a corresponding evaluation result. The second operation instruction whose evaluation result meets preset conditions is determined as the target operation instruction. This process may include the following steps:

[0090] Step S601: Based on the hit dataset, the second operation instructions are sorted using a reordering method to obtain the priority instruction sequence.

[0091] Step S602: Determine the second operation instruction in the priority instruction sequence that meets the preset conditions as the target operation instruction.

[0092] Before sorting the second operation instructions, features are extracted from the hit interaction data. These features will be used as input to the reordering model. The features can be user behavior data, voice data, etc., converted into a format that the model can process. The extracted features are input into the reordering model to obtain the probability score of each second operation instruction. The second operation instructions are sorted according to the probability scores to obtain the priority instruction sequence.

[0093] Define preset conditions according to specific needs. Common conditions include selecting the instruction with the highest probability, selecting the instruction most frequently seen in the user's historical behavior, etc. Evaluate the instructions in the priority instruction sequence one by one, and determine the instruction that meets the preset conditions as the target operation instruction. If there is no instruction that meets the conditions, return None or other default values.

[0094] In this embodiment, a priority instruction sequence is obtained by sorting the hit dataset, and then the second operation instruction in the priority instruction sequence that meets preset conditions is determined as the target operation instruction. This, combined with user feedback and new interaction data, improves the system's accuracy and user experience.

[0095] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0096] like Figure 8 The diagram shown is a schematic of a human-computer interaction device for intelligent devices according to Embodiment 7 of the present invention. This human-computer interaction device corresponds one-to-one with the human-computer interaction methods for intelligent devices in the above embodiments. The human-computer interaction device includes a data classification module 81, an instruction generation module 82, an evaluation module 83, and an adjustment module 84. Detailed descriptions of each functional module are as follows:

[0097] The data classification module 81 is used to collect user interaction data, the first operation command in response to the interaction data, and the first feedback result of the user after the response. Based on the first feedback result, the interaction data is classified to obtain the hit interaction data and the miss interaction data. The hit interaction data and the corresponding first operation command form a hit dataset, and the miss interaction data and the corresponding first operation command form a training dataset.

[0098] The instruction generation module 82 is used to identify any interactive data in the training dataset using a preset model, obtain N recognition results, and generate a second operation instruction for each recognition result, where N is an integer greater than zero.

[0099] The evaluation module 83 is configured to evaluate each second operation instruction using the hit data set to obtain a corresponding evaluation result, determine a second operation instruction whose evaluation result meets a preset condition as a target operation instruction, and obtain a second feedback result of the user after responding to the target operation instruction.

[0100] The adjustment module 84 is configured to perform parameter adjustment on the preset model, and return to execute the collection of the interaction data of the user, the first operation instruction responding to the interaction data, and the first feedback result of the user after responding until the smart device is closed.

[0101] Optionally, the evaluation module 83 includes:

[0102] The screening unit is configured to obtain a preset reward supervision model, screen the evaluation result according to the reward supervision model, and obtain a corresponding preferred evaluation result.

[0103] The result obtaining unit is configured to determine a second operation instruction whose preferred evaluation result meets a preset condition as a target instruction, and obtain a second feedback result of the user after responding to the target operation instruction.

[0104] Optionally, the smart device human-computer interaction device further includes:

[0105] The data cleaning module is configured to perform data cleaning on the training data set using a cleaning rule to obtain a cleaned training data set.

[0106] The unification module is configured to perform data unification on the cleaned training data set using a standardization specification to obtain a training data set in a unified format.

[0107] The interaction data recognition module is configured to recognize any interaction data in the training data set in the unified format using a preset model to obtain N recognition results, generate a second operation instruction for each recognition result, and N is an integer greater than zero.

[0108] Optionally, the adjustment module 84 includes:

[0109] The error calculation unit is configured to obtain a reward index of a preset reward supervision model, perform error calculation according to the second feedback result and the reward index, and obtain a reward error loss.

[0110] The parameter adjustment unit is configured to perform parameter adjustment on the preset model according to the reward error loss, return to execute the collection of the interaction data of the user, the first operation instruction responding to the interaction data, and the first feedback result of the user after responding until the smart device is closed.

[0111] Optionally, the instruction generation module 82 includes:

[0112] The feature extraction unit is configured to perform feature extraction according to any interaction data to obtain interaction features.

[0113] The intention recognition unit is configured to perform user interaction intention recognition according to the interaction features by using a pre-trained model to obtain N intention recognition results, and generate a second operation instruction for each intention recognition result.

[0114] Optionally, the data classification module 81 further includes:

[0115] The behavior feedback unit responds to the behavior operation instruction of the behavior data to obtain a behavior feedback result of the user.

[0116] The voice feedback unit responds to the voice operation instruction of the voice data to obtain a voice feedback result of the user.

[0117] The interaction data classification unit classifies the interaction data according to the behavior feedback result and the voice feedback result to obtain hit interaction data and non-hit interaction data.

[0118] Optionally, the evaluation module 83 further includes:

[0119] The prioritization unit sorts the second operation instructions in a reordering manner according to the hit data set to obtain a priority instruction sequence.

[0120] The determination unit determines that a second operation instruction that meets a preset condition in the priority instruction sequence is a target operation instruction.

[0121] The specific limitations of the intelligent device human-computer interaction device can be referred to the limitations of the intelligent device human-computer interaction method in the foregoing, and will not be repeated here. Each module in the intelligent device human-computer interaction device can be realized by software, hardware, and a combination thereof in whole or in part. Each module can be embedded in or independent of a processor in a computer device in a hardware form, or can be stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform operations corresponding to each module.

[0122] As Figure 9As shown in the figure, it is a structural schematic diagram of a computer device provided in Embodiment Eight of the present application. The computer device comprises a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement an intelligent device human-computer interaction method.

[0123] In an embodiment, a computer device is provided, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the intelligent device human-computer interaction method in the above embodiments when executing the computer program, for example Figures 2 to 7 As shown in the figure, for the sake of brevity, the functions of the data classification module 81, the instruction generation module 82, the evaluation module 83 and the adjustment module 84 will not be repeated here. Alternatively, the processor implements the functions of each module / unit in this embodiment of the intelligent device human-computer interaction device, for example Figure 8 As shown in the figure, for the sake of brevity, the functions of the data classification module 81, the instruction generation module 82, the evaluation module 83 and the adjustment module 84 will not be repeated here. Alternatively, the processor implements the functions of each module / unit in this embodiment of the intelligent device human-computer interaction device, for example

[0124] In an embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the intelligent device human-computer interaction method in the above embodiments, for example Figures 2 to 7 As shown in the figure, for the sake of brevity, the functions of the data classification module 81, the instruction generation module 82, the evaluation module 83 and the adjustment module 84 will not be repeated here. Alternatively, the computer program is executed by the processor to implement the functions of each module / unit in this embodiment of the intelligent device human-computer interaction device, for example Figure 8 As shown in the figure, for the sake of brevity, the functions of the data classification module 81, the instruction generation module 82, the evaluation module 83 and the adjustment module 84 will not be repeated here. Alternatively, the computer program is executed by the processor to implement the functions of each module / unit in this embodiment of the intelligent device human-computer interaction device, for example

[0125] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiments of each method can be included. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0126] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0127] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for human-computer interaction of intelligent devices, characterized in that, The method comprises the following steps: After the intelligent device is started, the interaction data of a user is collected, a first operation instruction is responded to the interaction data, and a first feedback result of the user after the response is obtained; the interaction data is classified according to the first feedback result, and the hit interaction data and the missed interaction data are obtained; the hit interaction data and the corresponding first operation instruction form a hit data set, and the missed interaction data and the corresponding first operation instruction form a training data set; For any interaction data in the training data set, a preset model is used to identify the interaction data, N identification results are obtained, and a second operation instruction is generated for each identification result, N being an integer greater than zero; The hit data set is used to evaluate each second operation instruction, and the corresponding evaluation result is obtained; the second operation instruction whose evaluation result meets a preset condition is determined as a target operation instruction, and a second feedback result of the user after responding to the target operation instruction is obtained; According to the second feedback result, the parameters of the preset model are adjusted, and the collection of the interaction data of the user, the first operation instruction responded to the interaction data and the first feedback result of the user after the response are returned until the intelligent device is closed.

2. The intelligent device human interaction method of claim 1, wherein, The use of the hit data set to evaluate each second operation instruction to obtain the corresponding evaluation result comprises: A preset reward supervision model is obtained, the evaluation result is filtered according to the reward supervision model, and the corresponding preferred evaluation result is obtained; The second operation instruction whose preferred evaluation result meets the preset condition is determined as the target instruction, and the second feedback result of the user after responding to the target operation instruction is obtained.

3. The intelligent device human interaction method of claim 2, wherein, The parameter adjustment of the preset model according to the second feedback result comprises: The reward index of the preset reward supervision model is obtained, the error calculation is performed according to the second feedback result and the reward index, and the reward error loss is obtained; According to the reward error loss, the parameters of the preset model are adjusted, and the collection of the interaction data of the user, the first operation instruction responded to the interaction data and the first feedback result of the user after the response are returned until the intelligent device is closed.

4. The intelligent device human interaction method of claim 1, wherein, After the missed interaction data and the corresponding first operation instruction form the training data set, the following steps are further included: According to the training data set, data cleaning is performed using a cleaning rule to obtain a cleaned training data set; According to the cleaned training data set, data unification is performed using a standardization specification to obtain a unified format training data set; For any interaction data in the unified format training data set, a preset model is used to identify the interaction data, N identification results are obtained, and a second operation instruction is generated for each identification result, N being an integer greater than zero.

5. The intelligent device human interaction method of claim 1, wherein, The use of the preset model to identify the interaction data to obtain N identification results comprises: According to the any interaction data, feature extraction is performed to obtain interaction features; According to the interaction features, a pre-training model is used to identify the interaction intention of the user to obtain N intention identification results, and a second operation instruction is generated for each intention identification result.

6. The intelligent device human interaction method of claim 1, wherein, The interaction data of the user includes behavior data and voice data; According to the first feedback result, the interaction data is classified to obtain hit interaction data and miss interaction data, including: According to the behavior operation instruction of the behavior data, a behavior feedback result of the user is obtained; According to the voice operation instruction of the voice data, a voice feedback result of the user is obtained; According to the behavior feedback result and the voice feedback result, the interaction data is classified to obtain hit interaction data and miss interaction data. 7.The intelligent device human-computer interaction method according to any one of claims 1 to 6, characterized in that, The evaluation result of the second operation instruction is determined as a target operation instruction when the evaluation result meets a preset condition, including: The second operation instruction is sorted in a reordering manner according to the hit data set to obtain a priority instruction sequence; The second operation instruction that meets the preset condition of the priority instruction sequence is determined as a target operation instruction.

8. An apparatus for human-computer interaction of intelligent devices, characterized in that, Including: The data classification module is configured to collect interaction data of a user, a first operation instruction in response to the interaction data, and a first feedback result of the user after the response, classify the interaction data according to the first feedback result, obtain hit interaction data and miss interaction data, form a hit data set by combining the hit interaction data and the corresponding first operation instruction, form a training data set by combining the miss interaction data and the corresponding first operation instruction, and identify the interaction data in the training data set using a preset model to obtain N identification results, generate a second operation instruction for each identification result, and N is an integer greater than zero. The evaluation module is configured to evaluate each second operation instruction using the hit data set to obtain a corresponding evaluation result, determine a second operation instruction whose evaluation result meets a preset condition as a target operation instruction, and obtain a second feedback result of the user in response to the target operation instruction. The adjustment module is configured to adjust parameters of the preset model and return to collect the interaction data of the user, the first operation instruction in response to the interaction data, and the first feedback result of the user after the response until the smart device is closed. The processor executes the computer program to implement the smart device human-computer interaction method of any one of claims 1 to 7.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The computer program is executed by the processor to implement the smart device human-computer interaction method of any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. ​

Images