Interaction intention determination method and device

Abstract

The invention provides an interaction intention determination method and device, and relates to the technical field of artificial intelligence. The method comprises the following specific steps: obtaining training data, and training a pre-trained neural network model according to the training data; obtaining current environmental parameters, and inputting the environmental parameters into the neural network model for reasoning operation to generate interaction intention information; and generating an operation instruction according to the interaction intention information, and controlling the intelligent equipment according to the operation instruction. According to the invention, by predicting the interaction intention information of the user, intelligent control of the intelligent device is realized, and the use experience of the user is improved.

Description

Technical Field

[0001] The present disclosure relates to a method and device for determining interaction intention, and in particular to the field of artificial intelligence. Background Art

[0002] As technologies such as 5G, artificial intelligence, the Internet of Things, and big data gradually mature and become popular, smart home, as a relatively mature field for the application of the Internet of Things, has continuously enriched its product categories, accelerated the expansion of the market segment, and gradually matured the industrial ecological pattern and industrial chain. Ecological participants in China's smart home field can be divided into roles such as underlying components, smart devices, ecological platforms, application scenarios, and security solutions. Among them, underlying component manufacturers include companies in the fields of chips, modules, screens, sensors, etc. Smart device manufacturers include companies in the fields of smart home appliances, smart routers, smart speakers, smart door locks, smart gateways, and smart lighting. Ecological platform manufacturers include home appliance companies, operators, mobile phone manufacturers, Internet platforms, etc. Application scenario manufacturers include companies in the fields of smart security, residential real estate, long-term and short-term rentals / hotels, and interior decoration. Security manufacturers include companies in the fields of security chips / modules and security services.

[0003] In the related art, smart devices cannot accurately understand the user's intention, resulting in the inability to perform corresponding operations when the user needs them, thereby reducing the user's experience. Summary of the invention

[0004] The present disclosure provides a method and device for determining interaction intention, so as to at least solve the problem in the related art that smart devices cannot accurately understand the user's intention. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a method for determining an interaction intention is provided, including:

[0006] Acquire training data, and train the pre-trained neural network model according to the training data;

[0007] Acquiring current environmental parameters, and inputting the environmental parameters into the neural network model for reasoning operation to generate interaction intention information;

[0008] Generate an operation instruction according to the interaction intention information, and control the smart device according to the operation instruction.

[0009] Optionally, the step of obtaining the current environmental parameters specifically includes:

[0010] The environmental sensor is dispatched to perform detection to obtain the environmental parameter, wherein the environmental sensor includes at least one of the following: a temperature sensor, a brightness sensor, a sound sensor, a humidity sensor, and a video monitoring sensor.

[0011] Optionally, the environmental parameter includes at least one of the following:

[0012] Ambient temperature;

[0013] Ambient brightness;

[0014] Ambient volume;

[0015] Ambient humidity;

[0016] Current time;

[0017] Video monitoring data.

[0018] Optionally, the interaction intention includes an operation intention and a decision intention, and the step of generating an operation instruction according to the interaction intention information specifically includes:

[0019] Determine a smart device that matches the operation intention, and obtain the operation type supported by the smart device;

[0020] A target operation type is selected from the operation types according to the operation intention, and the operation instruction is generated according to the target operation type.

[0021] Optionally, controlling the smart device according to the operation instruction includes:

[0022] In response to the decision intention being to agree to execute, and the video monitoring data containing the user, executing the operation instruction;

[0023] In response to the decision intention being a decision after inquiry, the operation instruction is executed according to the user decision.

[0024] Optionally, executing the operation instruction according to the user's decision includes:

[0025] generating an inquiry message according to the operation instruction, and sending the inquiry message;

[0026] Acquire the user's voice command, and execute the operation command according to the voice command.

[0027] Optionally, executing the operation instruction according to the voice instruction includes:

[0028] Converting the voice command into a text command and determining a user command corresponding to the text command;

[0029] In response to the user instruction indicating execution, executing the operation instruction;

[0030] In response to the user instruction indicating not to execute, the execution of the operation instruction is canceled.

[0031] Optionally, the method further includes:

[0032] The current neural network model is trained according to the environmental parameters and the voice command to update the neural network model.

[0033] Optionally, the acquired training data includes: environmental parameter training data, and interaction training data between the user and the smart device.

[0034] Optionally, the interactive training data specifically includes:

[0035] Voice command data;

[0036] Operation data, wherein the operation data includes operation data of the user on the smart device;

[0037] Device status data.

[0038] Optionally, the training of the pre-trained neural network model according to the training data includes:

[0039] Determining an interaction intention annotation according to the interaction training data, and annotating the environmental parameter training data according to the interaction intention annotation to generate a training data set;

[0040] Inputting the environmental parameter training data in the training data set into the neural network model to generate interaction intention training data;

[0041] The neural network model is adjusted with the goal of reducing the loss function between the interaction intent annotation and the interaction intent training data.

[0042] According to a second aspect of an embodiment of the present disclosure, there is provided an interaction intention determination device, including:

[0043] A training module, used to obtain training data and train the pre-trained neural network model according to the training data;

[0044] An interaction intention prediction module is used to obtain current environmental parameters and input the environmental parameters into the neural network model for reasoning operation to generate interaction intention information;

[0045] The interaction module is used to generate an operation instruction according to the interaction intention information and control the smart device according to the operation instruction.

[0046] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0047] processor;

[0048] a memory for storing instructions executable by the processor;

[0049] The processor is configured to execute the instructions to implement the method as described in any one of the first aspects.

[0050] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute a method as described in any one of the first aspects.

[0051] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0052] By predicting the user's interaction intention information, intelligent control of smart devices is achieved. When using smart devices, users only need to give simple answers or even no operation. This lowers the operating threshold of smart devices, and almost no learning is required when using them, which reduces the difficulty of getting started.

[0053] The neural network model is used to understand user needs more accurately, thereby improving the accuracy of user intent recognition and avoiding inaccurate recognition problems.

[0054] The current neural network model is trained according to the environmental parameters and voice commands generated by the user during use to update the neural network model, better learn the user's usage habits, and improve the user experience.

[0055] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0057] Figure 1 The present invention is a flowchart of a method for determining interaction intention according to an exemplary embodiment.

[0058] Figure 2 The present invention is a flowchart of a method for determining interaction intention according to an exemplary embodiment.

[0059] Figure 3 The present invention is a flowchart of a method for determining interaction intention according to an exemplary embodiment.

[0060] Figure 4 The present invention is a flowchart of a method for determining interaction intention according to an exemplary embodiment.

[0061] Figure 5 The present invention is a flowchart of a method for determining interaction intention according to an exemplary embodiment.

[0062] Figure 6 The present invention is a flowchart of a method for determining interaction intention according to an exemplary embodiment.

[0063] Figure 7 The invention is a block diagram of a device for determining interaction intention according to an exemplary embodiment.

[0064] Figure 8 It is a block diagram of a device according to an exemplary embodiment.

[0065] Fig. 9 It is a block diagram of a device according to an exemplary embodiment. DETAILED DESCRIPTION

[0066] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0067] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0068] It should be noted that the user information involved in this disclosure (including but not limited to user device information, user personal information, etc.) is all information authorized by the user or fully authorized by all parties.

[0069] As technologies such as 5G, artificial intelligence, the Internet of Things, and big data gradually mature and become popular, smart home, as a relatively mature field for the application of the Internet of Things, has continuously enriched its product categories, accelerated the expansion of the market segment, and gradually matured the industrial ecological pattern and industrial chain. Ecological participants in China's smart home field can be divided into roles such as underlying components, smart devices, ecological platforms, application scenarios, and security solutions. Among them, underlying component manufacturers include companies in the fields of chips, modules, screens, sensors, etc. Smart device manufacturers include companies in the fields of smart home appliances, smart routers, smart speakers, smart door locks, smart gateways, and smart lighting. Ecological platform manufacturers include home appliance companies, operators, mobile phone manufacturers, Internet platforms, etc. Application scenario manufacturers include companies in the fields of smart security, residential real estate, long-term and short-term rentals / hotels, and interior decoration. Security manufacturers include companies in the fields of security chips / modules and security services.

[0070] The smart home industry still faces many problems and challenges in its development. On the one hand, from the user (end consumer / smart home user) side, smart home needs to continue to provide users with core value points such as ease of use, convenience, and intelligence; on the other hand, brand manufacturers still need to continue to explore cross-brand interconnection business scenarios and improve the interconnection experience. At present, the main problems and challenges faced by the smart home industry include poor user experience, inability to interconnect across brands, insufficient content service resources, and unclear business models.

[0071] Figure 1 is a flow chart of a method for determining an interaction intention according to an exemplary embodiment. Figure 1 As shown, the method comprises the following steps:

[0072] Step 101, obtaining training data, and training the pre-trained neural network model according to the training data,

[0073] In this embodiment, a neural network model is used to analyze the user's interaction intention. In order to speed up the training process, the neural network model is trained by the method of pre-training + fine-tuning. In the related art, when the neural network is trained, it is generally based on the back propagation (BP) algorithm, first randomly initializes the parameters in the network, and then uses the optimization algorithm such as stochastic gradient descent (SGD) to continuously optimize the model parameters. The idea of ​​pre-training is that the model parameters are no longer randomly initialized, but pre-trained through some tasks to obtain a set of model parameters, and then use this set of parameters to initialize the model, and then further train it.

[0074] In a possible embodiment, in the intent recognition task, the neural network model is first pre-trained using a large amount of unlabeled training data, so that the neural network learns the commonalities in the unlabeled training data, and the features learned by the neural network are more universal. After the pre-training is completed, a small amount of training data is labeled accordingly for a specific training goal to generate labeled training data, and the pre-trained neural network model is trained using a small amount of labeled training data to adjust the parameters of each layer. In a possible embodiment, the training goal is to identify the interaction intent, and the annotations added to the small amount of training data are the corresponding user intent.

[0075] Step 102, obtaining current environmental parameters, and inputting the environmental parameters into the neural network model for reasoning operation to generate interaction intention information;

[0076] In this embodiment, the environment around the smart device is analyzed to determine the user's intention, that is, the operation that the user wants the smart device to perform at this time. The current environmental parameters are obtained through a variety of sensors, and the environmental parameters are input into the neural network model. After the encoder extracts high-dimensional features, the decoder performs dimensionality reduction operations on the high-dimensional features to generate interaction intention information and determine the user's intention.

[0077] Step 103: Generate an operation instruction according to the interaction intention information, and control the smart device according to the operation instruction.

[0078] In this embodiment, the operation that the user may want to perform is determined based on the interaction intention information, and whether the user needs to confirm before performing the operation. The corresponding operation instruction is generated based on the interaction intention information to control the smart device to perform the corresponding operation.

[0079] In a possible embodiment, the smart devices all communicate with the central control module through wireless signals or wired networks, receive control instructions sent by the central control module to perform corresponding operations, and all smart devices can be controlled by the central control module. The user experience is convenient and fast, avoiding the problem of low efficiency of operating each device individually.

[0080] Optionally, the smart device includes but is not limited to at least one of the following: home equipment, vehicle-mounted equipment, and office equipment.

[0081] In the home usage scenario, home appliances include but are not limited to at least one of the following: smart fans, smart desk lamps, smart speakers, smart washing machines, smart curtains, smart microwave ovens, and smart air conditioners.

[0082] In the in-vehicle usage scenario, the in-vehicle equipment includes but is not limited to at least one of the following: smart sunroof, smart car air conditioner, and smart car TV.

[0083] In office usage scenarios, office equipment includes but is not limited to at least one of the following: a smart printer, a smart coffee machine, a smart projector, and a smart shredder.

[0084] Optional, Figure 1 The step 102 of obtaining the current environmental parameters specifically includes:

[0085] The environmental sensor is dispatched to perform detection to obtain the environmental parameter, wherein the environmental sensor includes at least one of the following: a temperature sensor, a brightness sensor, a sound sensor, a humidity sensor, and a video monitoring sensor.

[0086] Optionally, the environmental parameters include at least one of the following: ambient temperature; ambient brightness; ambient volume; ambient humidity; current time; and video monitoring data.

[0087] In this embodiment, environmental parameters are obtained through sensors to determine what kind of environment the smart device is in. In addition to collecting sensor data, the time when these environmental parameters are collected is also recorded to facilitate matching the environmental parameters with the interaction intention information at the same time. This facilitates the subsequent use of these data as training data for further updating and training of the neural network model.

[0088] Optionally, the ambient temperature reflects the temperature of the environment in which the smart device is located. When the user is in the current environment, when the ambient temperature is high, the temperature may need to be lowered through the smart device, and when the ambient temperature is low, the temperature may need to be increased through the smart device; the ambient brightness reflects the brightness of the environment in which the smart device is located. When the user is in the current environment, when the ambient brightness is high or low, the ambient light may need to be adjusted through smart devices (such as smart curtains, smart desk lamps) to ensure appropriate brightness; the ambient volume reflects the volume of the environment in which the smart device is located. When the user is in the current environment, when the ambient volume is high or low, the ambient volume may need to be adjusted through smart devices (such as smart speakers) to ensure appropriate humidity.

[0089] Figure 2 is a flow chart of a method for determining an interaction intention according to an exemplary embodiment. Figure 2 As shown, Figure 1 Step 103 in includes the following steps.

[0090] Step 201, determining a smart device that matches the operation intention, and obtaining an operation type supported by the smart device;

[0091] In this embodiment, the interaction intention includes operation intention and decision intention. The operation intention is the operation that the user may want to perform on the smart device. The decision intention reflects the user's habits, that is, whether the operation corresponding to the operation intention requires the user to confirm before execution. If the user neural network model determines that the user often performs this operation under the current environmental parameters, the generated decision intention will instruct to directly execute the operation; if the user neural network model determines that the user may not perform this operation under the current environmental parameters, the generated decision intention will instruct to inquire the user to determine whether to perform the operation.

[0092] All controlled smart devices and the operation types they support are stored in the database. After the operation intention is determined, the corresponding smart device is retrieved from the database and its corresponding operation type is obtained.

[0093] In a possible embodiment, the smart device and its corresponding operation type include but are not limited to:

[0094] Smart desk lamp: adjust color temperature, adjust brightness, turn on, turn off, timed start, timed shutdown;

[0095] Smart car air conditioner or smart air conditioner: adjust temperature, adjust working mode (such as cooling, heating, ventilation), turn on, turn off, timed start, timed shutdown;

[0096] Smart fan: adjust gear, shake head, turn on and off;

[0097] Smart speakers: adjust volume, adjust playback mode, power on and off;

[0098] Smart washing machine: turn on, turn off, adjust working mode, adjust the duration of working mode;

[0099] Smart microwave oven, smart rice cooker and smart electric kettle: turn on, turn off, adjust temperature, adjust working mode, timer start and timer shutdown;

[0100] Smart car TV: turn on, turn off, adjust volume, play video, pause video.

[0101] Smart car aromatherapy: turn on the aromatherapy, turn off the aromatherapy, adjust the aromatherapy smell.

[0102] Step 202: Select a target operation type from the operation types according to the operation intention, and generate the operation instruction according to the target operation type.

[0103] In this embodiment, a mapping relationship table is generated in advance based on the mapping relationship between the operation intention and the operation type and stored. The mapping relationship table is searched according to the operation intention, and the matching operation type is used as the target operation type.

[0104] In a possible embodiment, the operation intention is to lower the temperature, and the corresponding target operation types include but are not limited to: adjusting the smart air conditioner to cooling mode, lowering the temperature of the smart air conditioner, turning on the smart fan gear, increasing the smart fan gear, adjusting the smart car air conditioner to cooling mode, and lowering the temperature of the smart car air conditioner.

[0105] In a possible embodiment, the operation intention is to reduce the brightness, and the corresponding target operation types include but are not limited to: closing the smart curtains and reducing the brightness of the smart desk lamp.

[0106] In a possible embodiment, the operation intention is to increase the humidity, and the corresponding target operation type includes but is not limited to: turning on the smart humidifier and turning off the dehumidifier.

[0107] In a possible embodiment, the operation intention is to reduce the humidity, and the corresponding target operation type includes but is not limited to: turning off the smart humidifier and turning on the dehumidifier.

[0108] In a possible embodiment, the operation intention is to turn on the aromatherapy, and the corresponding target operation type includes turning on the smart car aromatherapy.

[0109] Figure 3 is a flow chart of a method for determining an interaction intention according to an exemplary embodiment. Figure 3 As shown, Figure 1 Step 103 in the method further includes the following steps.

[0110] Step 301, in response to the decision intention being to agree to execute, and the video monitoring data containing the user, executing the operation instruction;

[0111] In this embodiment, when the decision intention is to agree to execute, it means that the neural network believes that the user can directly execute this operation without consent. At this time, the video monitoring data is analyzed to confirm whether the user is in the current environment. If the user is included in the video monitoring data, the operation instruction can be directly executed.

[0112] Step 302, in response to the decision intention being a decision after inquiry, executing the operation instruction according to the user decision.

[0113] In this embodiment, the decision intention is to make a decision after inquiry, which means that the neural network believes that the operation cannot be performed directly and needs to decide whether to execute the operation instruction based on the user's opinion.

[0114] In a possible embodiment, the decision intention is the probability that the user agrees to execute the current operation instruction. A certain probability threshold is preset. When the probability is greater than or equal to the probability threshold, it is determined that the user is likely to agree to execute the operation instruction, and the decision intention is to agree to execute; when the probability is less than the probability threshold, it is determined that the user may not agree to execute the operation, the user is questioned, the user's decision is obtained, and the operation instruction is executed according to the user's decision.

[0115] Figure 4 is a flow chart of a method for determining an interaction intention according to an exemplary embodiment. Figure 4 As shown, Figure 3 Step 302 in includes the following steps.

[0116] Step 401, generating an inquiry message according to the operation instruction, and sending the inquiry message;

[0117] In this embodiment, an inquiry message corresponding to the operation instruction is generated, and the user is notified of the inquiry message, asking the user whether he is sure to execute the corresponding operation instruction.

[0118] In a possible embodiment, the inquiry message is a voice message, and the inquiry message is broadcasted through a speaker to remind the user and determine the user's decision.

[0119] In a possible embodiment, the inquiry message is a text message, and the inquiry message is broadcasted through a smart display device (such as a smart TV, a smart display) to remind the user and determine the user's decision.

[0120] In a possible embodiment, the operation intention is to reduce the brightness, and the corresponding operation instruction is to close the smart curtains, reduce the brightness of the smart desk lamp, or turn off the smart desk lamp, and any of the following voice messages is generated: "Do you want to close the curtains?" or "Do you want to lower the brightness of the desk lamp?" or "Do you want to turn off the brightness of the desk lamp?" and the voice message is played through the speaker.

[0121] In a possible embodiment, the operation intention is to lower the brightness, and the corresponding operation instruction is to close the smart curtains, lower the brightness of the smart desk lamp or turn off the smart desk lamp. The generated inquiry message is a text message, including any one of the following: "Do you want to close the curtains?" or "Do you want to lower the brightness of the desk lamp?" or "Do you want to turn off the desk lamp?", and the text message is displayed on the TV screen, or the text message is displayed on the user's mobile phone screen through the APP.

[0122] In a possible embodiment, the operation intention is to close the vehicle sunroof, the corresponding operation instruction is to close the sunroof, and the generated inquiry message is a voice message, including any one of the following: "Do you want to close the sunroof?", and the voice message is broadcast through the vehicle speaker.

[0123] Step 402: Acquire the user's voice command, and execute the operation command according to the voice command.

[0124] In this embodiment, after receiving the inquiry message, the user can provide feedback through voice. The user's voice instructions are collected through a sound sensor, and the voice instructions are converted into text and analyzed to determine the user's decision to determine whether to execute the operation instruction.

[0125] In a possible embodiment, the operation intention is to reduce the brightness. After the speaker announces "Do you want to turn off the desk lamp?", the collected user's voice command content is a message with certain semantics, such as "Yes", "Execute", "OK", "Turn off". Then the operation instruction can be executed to turn off the smart desk lamp.

[0126] In a possible embodiment, the operation intention is to reduce the brightness, and after the speaker announces "Do you want to close the curtains?", the collected user's voice command content is a message with certain semantics, such as "No", "Not Executed", "No Way", "Not Closed". Then the operation instruction is not executed.

[0127] Figure 5 is a flow chart of a method for determining an interaction intention according to an exemplary embodiment. Figure 5 As shown, Figure 4 Step 402 in includes the following steps.

[0128] Step 501, converting the voice command into a text command, and determining a user command corresponding to the text command;

[0129] Step 502A, in response to the user instruction, executing the operation instruction;

[0130] Step 502B: in response to the user instruction indicating not to execute, cancel the execution of the operation instruction.

[0131] In this embodiment, the pre-trained semantic analysis model is used to convert the voice command into a text command, and the semantics of the text command is analyzed. Since the user only needs to decide whether to execute or not execute the operation command, the voice analysis model only needs to analyze whether the semantics of the text command is positive or negative. The user command finally generated is any one of the following: execute, do not execute.

[0132] Optionally, the method further includes:

[0133] The current neural network model is trained according to the environmental parameters and the voice command to update the neural network model.

[0134] In this embodiment, each time the neural network determines the interaction intention data based on the current environmental parameters and executes the operation instructions based on the interaction intention data, the environmental parameters and the voice instructions issued by the user are recorded, and the current neural network model is further trained based on these data so that the neural network model can better learn the user's usage habits of smart devices.

[0135] In a possible embodiment, the neural network learns that the user usually controls the smart desk lamp to turn off the light at 10 o'clock. Then, at 10 o'clock every day, an operation intention to lower the brightness (the target operation type is to turn off the light) and a decision intention for decision after inquiry will be generated, and a voice message will be sent to ask the user whether to turn off the light. After 100 days, the user agrees to turn off the light 99 times. Then on the 101st day, at 10 o'clock, an operation intention to lower the brightness and a decision intention to agree to execute will be generated, and the light will be turned off directly without sending a voice message to ask the user whether to turn off the light. On the 150th day, after turning off the light at 10 o'clock, the user turns the light on again. Then on the 151st day, at 10 o'clock, an operation intention to lower the brightness and a decision intention for decision after inquiry will be generated, and a voice message will be sent to ask the user whether to turn off the light, instead of turning off the light directly.

[0136] In a possible embodiment, the neural network analyzes the environmental parameters and obtains that the current ambient temperature is 30°C, then generates an operation intention to lower the temperature (the target operation type is to turn on the smart air conditioner, adjust the smart air conditioner to cooling mode, and adjust the air conditioner temperature to 26°C) and a decision intention to make a decision after inquiry, and sends a voice message to ask the user "Do you want to turn on the air conditioner and adjust it to cooling mode, the temperature is 26°C", after 30 times of this scenario, the user agrees to turn on the air conditioner and adjust it to cooling mode each time. Then, when this scenario appears for the 31st time, an operation intention to lower the temperature and a decision intention to agree to execute will be generated, and the smart air conditioner will be directly turned on and adjusted to cooling mode without sending a voice message to ask the user, and the temperature is 26°C. When this scenario appears for the 50th time, after turning on the smart air conditioner and adjusting it to cooling mode, the user performs any of the following actions: turning off the air conditioner, adjusting the air conditioner temperature to 27°C, and adjusting the air conditioner to ventilation mode. Then when this scenario occurs for the 51st time, a voice message will be sent again to ask the user whether to turn on the air conditioner and adjust it to cooling mode and the temperature to 26°C, or whether to turn on the air conditioner and adjust it to cooling mode and the temperature to 27°C, or whether to turn on the air conditioner and adjust it to ventilation mode, instead of turning on the air conditioner directly.

[0137] Optionally, the acquired training data includes: environmental parameter training data, and interaction training data between the user and the smart device.

[0138] Optionally, the interactive training data specifically includes:

[0139] Voice command data;

[0140] Operation data, wherein the operation data includes operation data of the user on the smart device;

[0141] Device status data.

[0142] In this embodiment, by combining the voice command data and the corresponding time, features can be extracted to analyze which operations the user is more inclined to perform at which time point, such as turning off the lights at 10 pm, turning on the TV at 8 am, etc. Based on the voice command data and environmental parameter training data at the same time point or time period, it can be analyzed in which environments the user is more inclined to perform what operations, such as turning on the fan when the temperature is high, lowering the light brightness when the ambient brightness is high, and closing the window when the ambient volume is high.

[0143] The operation data includes the user's operation data on the smart device, including but not limited to: operations performed by the user directly pressing buttons on the device, operations performed by the user through the device remote control, and operations performed remotely by the user through a mobile phone.

[0144] Equipment status data reflects the operating status of the equipment. Equipment status data includes but is not limited to:

[0145] Smart fan: gear data (first gear, second gear, third gear), shaking head (left and right, up and down), switch.

[0146] Smart speakers: volume data, switch.

[0147] Smart washing machine: switch, working mode (mixed wash, quick wash, energy-saving wash, wool wash, cotton and linen wash, etc.), duration of working mode (such as 30 minutes, 45 minutes, etc.).

[0148] Smart air conditioner: temperature data (such as 25℃, 28℃, etc.), working mode data (such as cooling, heating, ventilation), power on and off data, timing data (such as scheduled startup time, scheduled shutdown time).

[0149] Figure 6 is a flow chart of a method for determining an interaction intention according to an exemplary embodiment. Figure 6 As shown, Figure 1 Step 101 in includes the following steps.

[0150] Step 601 : determining interaction intention annotations according to the interaction training data, and annotating the environmental parameter training data according to the interaction intention annotations to generate a training data set.

[0151] Optionally, the interaction intention labeling includes operation intention labeling and decision intention labeling.

[0152] Step 602: input the environmental parameter training data in the training data set into the neural network model to generate interaction intention training data.

[0153] In this embodiment, after the environmental parameter training data in the training data set is input into the neural network model, the high-dimensional features are extracted by the encoder, and then the high-dimensional features are reduced in dimension by the decoder to generate training interaction intention training data. The interaction intention training data is the operation that the user wants to perform, including operation intention and decision intention.

[0154] Step 603, adjusting the neural network model with the goal of reducing the loss function between the interaction intent annotation and the interaction intent training data.

[0155] In this embodiment, the goal of training is to make the interaction intent training data generated by the model as close to the interaction intent annotation as possible, so the standard for judging whether the model meets the goal is the gap between the operation intention and decision intention in the interaction intent training data and the annotated operation intention annotation and decision intention annotation. The loss function between the interaction intent annotation and the interaction intention training data is calculated, and the model is iterated with the goal of reducing the value of the loss function. After a certain round of training, the final neural network model is obtained.

[0156] Through the solution in this embodiment, the following effects can be achieved:

[0157] 1. Lowering the threshold for operating smart home devices: Compared with the prior art, this embodiment enables users to be more passive when using smart devices. Users only need to answer questions or even do nothing, and the threshold for family members to use smart devices is almost zero.

[0158] 2. Cultivate the way and habit of users to use smart home devices: The technical solution of the present invention gradually cultivates the way and habit of users to use smart home devices through interaction with users, thereby improving users' acceptance of smart home devices.

[0159] 3. Improve the utilization rate of smart home devices: The technical solution of the present invention improves the interaction mode, making users more willing to use smart devices, thereby improving the utilization rate of smart home devices.

[0160] 4. Understand user needs more accurately: The technical solution of the present invention utilizes a large prediction model and a small private domain model to more accurately understand what the user wants to express and improve the accuracy of speech recognition.

[0161] 5. Learn user habits and train user private domain models: The technical solution of the present invention can train user private domain models according to user usage habits, so that smart home devices can better meet user needs.

[0162] 6. Predict user needs based on the current scenario: The technical solution of the present invention can predict the content that the user wants to operate based on the current scenario, making smart home devices more intelligent.

[0163] 7. Improve the quality of life: The technical solution of the present invention makes smart home devices more intelligent, which can better meet user needs and improve the quality of life of users.

[0164] 8. Reduce energy consumption: The technical solution of the present invention helps to reduce energy consumption and achieve green environmental protection by intelligently adjusting the use of household appliances.

[0165] 9. Enhanced home security: The technical solution of the present invention can realize home security monitoring through smart home devices and improve home security.

[0166] 10. Improve the lives of the elderly and disabled: The technical solution of the present invention makes smart home devices easier to use, which helps to improve the quality of life of the elderly and disabled.

[0167] Figure 7 is a block diagram of a device for determining an interaction intention according to an exemplary embodiment. Figure 7 , the device 700 comprises:

[0168] A training module 710, used to obtain training data and train the pre-trained neural network model according to the training data;

[0169] The interaction intention prediction module 720 is used to obtain current environmental parameters and input the environmental parameters into the neural network model for reasoning operation to generate interaction intention information;

[0170] The interaction module 730 is used to generate an operation instruction according to the interaction intention information, and control the smart device according to the operation instruction.

[0171] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0172] Figure 8 8 is a block diagram of an apparatus 800 according to an exemplary embodiment. For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a tablet device, a personal digital assistant, etc.

[0173] Reference Figure 8 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0174] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0175] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0176] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0177] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0178] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0179] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0180] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0181] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0182] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.

[0183] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the device 800 to complete the above method. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0184] Fig. 9 900 is a block diagram of a device 900 according to an exemplary embodiment. For example, the device 900 may be provided as a server. Fig. 9 , the apparatus 900 includes a processing component 922, which further includes one or more processors, and a memory resource represented by a memory 932 for storing instructions, such as an application, that can be executed by the processing component 922. The application stored in the memory 932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 922 is configured to execute the instructions to perform the above method.

[0185] The device 900 may also include a power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and an input / output (I / O) interface 958. The device 900 may operate based on an operating system stored in the memory 932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0186] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0187] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for determining interaction intention, characterized in that: include: Acquire training data, and train the pre-trained neural network model according to the training data; Acquiring current environmental parameters, and inputting the environmental parameters into the neural network model for reasoning operation to generate interaction intention information; Generate an operation instruction according to the interaction intention information, and control the smart device according to the operation instruction.

2. The method according to any one of claims 1, characterized in that The step of obtaining the current environmental parameters specifically includes: The environmental sensor is dispatched to perform detection to obtain the environmental parameter, wherein the environmental sensor includes at least one of the following: a temperature sensor, a brightness sensor, a sound sensor, a humidity sensor, and a video monitoring sensor.

3. The method according to claim 2, characterized in that The environmental parameters include at least one of the following: Ambient temperature; Ambient brightness; Ambient volume; Ambient humidity; Current time; Video monitoring data.

4. The method according to claim 1, characterized in that: The interaction intention includes an operation intention and a decision intention, and the step of generating an operation instruction according to the interaction intention information specifically includes: Determine a smart device that matches the operation intention, and obtain the operation type supported by the smart device; A target operation type is selected from the operation types according to the operation intention, and the operation instruction is generated according to the target operation type.

5. The method according to claim 4, characterized in that The controlling the smart device according to the operation instruction comprises: In response to the decision intention being to agree to execute, and the video monitoring data containing the user, executing the operation instruction; In response to the decision intention being a decision after inquiry, the operation instruction is executed according to the user decision.

6. The method according to claim 5, characterized in that The executing the operation instruction according to the user's decision includes: generating an inquiry message according to the operation instruction, and sending the inquiry message; Acquire the user's voice command, and execute the operation command according to the voice command.

7. The method according to claim 6, characterized in that The executing the operation instruction according to the voice instruction comprises: Converting the voice command into a text command and determining a user command corresponding to the text command; In response to the user instruction indicating execution, executing the operation instruction; In response to the user instruction indicating not to execute, the execution of the operation instruction is canceled.

8. The method according to claim 7, characterized in that The method further comprises: The current neural network model is trained according to the environmental parameters and the voice command to update the neural network model.

9. The method according to any one of claims 1 to 8, characterized in that The acquired training data includes: environmental parameter training data and interaction training data between the user and the smart device.

10. The method according to claim 9, characterized in that The interactive training data specifically includes: Voice command data; Operation data, wherein the operation data includes operation data of the user on the smart device; Device status data.

11. The method according to claim 10, characterized in that The training of the pre-trained neural network model according to the training data includes: Determining an interaction intention annotation according to the interaction training data, and annotating the environmental parameter training data according to the interaction intention annotation to generate a training data set; Inputting the environmental parameter training data in the training data set into the neural network model to generate interaction intention training data; The neural network model is adjusted with the goal of reducing the loss function between the interaction intent annotation and the interaction intent training data.

12. An interactive intention determination device, characterized in that: include: A training module, used to obtain training data and train the pre-trained neural network model according to the training data; An interaction intention prediction module, used to obtain current environmental parameters, input the environmental parameters into the neural network model for reasoning operation, so as to generate interaction intention information; The interaction module is used to generate an operation instruction according to the interaction intention information and control the smart device according to the operation instruction.

13. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 11.

Images