Determination Method and Device of Control Instruction, Storage Medium and Electronic Device

Abstract

The present application discloses a method and device for determining control instructions, a storage medium, and an electronic device, relating to the technical field of smart homes. The method for determining control instructions includes: acquiring an image of a target object through an image acquisition device to obtain a target image; obtaining a first operation record of the target object on a first group of devices within a first preset time; determining the behavior preference of the target object; determining the target intention of the target object according to the target image, the first operation record, and the behavior preference; and determining a group of control instructions according to the target intention, where the group of control instructions is used to control a second group of devices to perform corresponding operations. By adopting the above technical solution, the problem that the device cannot perform corresponding device control according to the user's behavior, resulting in a low degree of intelligence of the device, is solved.

Description

Technical Field

[0001] The present application relates to the technical field of smart homes, and in particular, to a method and device for determining control instructions, a storage medium, and an electronic device. Background Art

[0002] With the rapid development of society, people's demand for household appliances is shifting from basic functional requirements to intelligent requirements. However, existing household appliances still remain at the level of manual active control, that is, users issue control instructions to household appliances through terminals or voice interactions. For example, when a user is about to go out, the user needs to perform a series of electrical appliance shutdown operations through a terminal or voice interaction, which is cumbersome and not intelligent enough.

[0003] In view of the problem in the related art that devices cannot perform corresponding device control according to the user's behavior, resulting in a low level of device intelligence, no effective solution has been proposed yet.

[0004] Therefore, it is necessary to improve the related art to overcome the defects in the related art. Summary of the Invention

[0005] Embodiments of the present invention provide a method and device for determining control instructions, a storage medium, and an electronic device, so as to at least solve the problem that devices cannot perform corresponding device control according to the user's behavior, resulting in a low level of device intelligence.

[0006] According to an aspect of the embodiments of the present invention, a method for determining control instructions is provided, including: performing image acquisition on a target object through an image acquisition device to obtain a target image; obtaining a first operation record of the target object on a first group of devices within a first preset time; determining the behavior preference of the target object; determining the target intention of the target object according to the target image, the first operation record, and the behavior preference; and determining a group of control instructions according to the target intention, where the group of control instructions is used to control a second group of devices to perform corresponding operations.

[0007] In an exemplary embodiment, determining the behavior preference of the target object includes: obtaining the behavior preference set by the target object on a target application, where the target application is set to allow control of devices in the first group of devices; or obtaining a second operation record of the target object on the first group of devices within a second preset time, and determining the behavior preference of the target object according to the second operation record, where the duration corresponding to the second preset time is greater than the duration corresponding to the first preset time.

[0008] In an exemplary embodiment, determining the target intention of the target object according to the target image, the first operation record, and the behavior preference includes: performing feature processing on the target image, the first operation record, and the behavior preference and then inputting them into an intention recognition model to obtain a plurality of intentions and probabilities corresponding to the plurality of intentions; determining the intention with the highest probability among the plurality of intentions as the target intention of the target object.

[0009] In an exemplary embodiment, the intention recognition model at least includes a first module, a second module, a third module, and a fourth module; the first module is configured to obtain a first intermediate result through the position of the target object and the action of the target object, the second module is configured to obtain a second intermediate result according to the first operation record, the third module is configured to determine a third intermediate result through the behavior preference, and the fourth module is configured to determine the plurality of intentions and probabilities corresponding to the plurality of intentions through the first intermediate result, the second intermediate result, and the third intermediate result, and the position of the target object and the action of the target object are obtained by performing image recognition on the target image.

[0010] In an exemplary embodiment, after determining the target intention of the target object according to the target image, the first operation record, and the behavior preference, the method further includes: instructing the target object to determine whether the target intention is correct; in the case where the target object determines that the target intention is incorrect, obtaining the true intention fed back by the target object, and saving the target data and the correspondence between the target data and the true intention as a piece of training data to a target list, where the target data includes the target image, the first operation record, and the behavior preference; in the case where the number of training data in the target list is greater than a preset threshold, retraining the intention recognition model according to the training data in the target list; determining a set of control instructions according to the target intention, including: in the case where the target object determines that the target intention is correct, determining a set of control instructions according to the target intention.

[0011] In an exemplary embodiment, determining a set of control instructions according to the target intention includes: determining a set of control instructions corresponding to the target intention according to a preset correspondence between intentions and control instructions; or determining a set of control instructions according to environmental information, the behavior preference of the target object, and the target intention.

[0012] In an exemplary embodiment, a set of control instructions is determined according to the environmental information, the behavior preferences of the target object, and the target intention, including: determining a second set of devices according to the behavior preferences of the target object and the target intention; determining the operating parameters of each device in the second set of devices according to the environmental information and the behavior preferences, and determining the set of control instructions according to the operating parameters of each device.

[0013] According to another aspect of the embodiments of the present invention, there is also provided a device for determining control instructions, including: an acquisition module, configured to perform image acquisition on a target object through an image acquisition device to obtain a target image; and acquire a first operation record of the target object on a first set of devices within a first preset time; and determine the behavior preferences of the target object; a first determination module, configured to determine the target intention of the target object according to the target image, the first operation record, and the behavior preferences; a second determination module, configured to determine a set of control instructions according to the target intention, where the set of control instructions is used to control a second set of devices to perform corresponding operations.

[0014] According to still another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, in which a computer program is stored, where the computer program is configured to execute the above-mentioned method for determining control instructions when running.

[0015] According to still another aspect of the embodiments of the present invention, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the above-mentioned processor executes the above-mentioned method for determining control instructions through the computer program.

[0016] Through the present invention, the target intention of the target object is determined according to the target image obtained by performing image acquisition on the target object through the image acquisition device, the first operation record of the target object on the first set of devices within the first preset time, and the behavior preferences of the target object, and then a set of control instructions for controlling the second set of devices is determined according to the target intention. Since the intention of the user can be predicted and determined in advance, and then the device can be controlled according to the intention of the user, the problem that the device cannot perform corresponding device control according to the behavior of the user, resulting in a low degree of intelligence of the device, is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0018] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a schematic diagram of the hardware environment of a method for determining a control instruction according to an embodiment of the present application;

[0020] Figure 2 It is a flowchart (I) of a method for determining a control instruction according to an embodiment of the present invention;

[0021] Figure 3 It is a flowchart (II) of a method for determining a control instruction according to an embodiment of the present invention;

[0022] Figure 4 It is a block diagram (I) of a device for determining a control instruction according to an embodiment of the present invention;

[0023] Figure 5 It is a block diagram (II) of a device for determining a control instruction according to an embodiment of the present invention. Detailed implementation manners

[0024] In order to enable those skilled in the art of the present technology to better understand the solutions of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0026] According to one aspect of the embodiments of the present application, a method for determining a control instruction is provided. The method for determining the control instruction is widely applied to the whole-house intelligent digital control application scenarios such as Smart Home, smart home, intelligent household device ecosystem, IntelligenceHouse ecosystem, etc. Optionally, in this embodiment, the above method for determining the control instruction can be applied to, for example, Figure 1 the hardware environment composed of the terminal device 102 and the server 104 as shown. As Figure 1 shown, the server 104 is connected to the terminal device 102 through a network, and can be used to provide services (such as application services, etc.) for the terminal or the client installed on the terminal. A database can be set on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data operation services for the server 104.

[0027] The above network can include but is not limited to at least one of the following: wired network, wireless network. The above wired network can include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network. The above wireless network can include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 is not limited to a PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing device, smart dishwasher, smart projection device, smart TV, smart clothes hanger, smart curtain, smart audio and video, smart socket, smart speaker, smart sound box, smart fresh air device, smart kitchen and bathroom device, smart bathroom device, smart floor sweeping robot, smart window cleaning robot, smart mopping robot, smart air purification device, smart steam box, smart microwave oven, smart kitchen treasure, smart purifier, smart water dispenser, smart door lock, etc.

[0028] To solve the above problems, in this embodiment, a method for determining a control instruction is provided, including but not limited to being applied in a cloud server. Figure 2 It is a flowchart (one) of the method for determining the control instruction according to the embodiments of the present invention. The process includes the following steps:

[0029] Step S202, perform image acquisition on the target object through an image acquisition device to obtain a target image; and obtain the first operation record of the target object on the first group of devices within the first preset time; and determine the behavior preference of the target object.

[0030] It should be noted that the image acquisition device can be one or more. In the case where there are multiple image acquisition devices, the target image can be acquired by multiple image acquisition devices, and then the images acquired by multiple devices can be stitched together to obtain the target image. The image acquisition device includes devices with image acquisition functions, such as refrigerators, TVs, washing machines, door locks, etc. with image acquisition functions.

[0031] It should be noted that the first group of devices can be understood as all the devices included in the home.

[0032] In an exemplary embodiment, the first preset time can be within 10 minutes. The specific value can be set by the target object (i.e., the user using the device), or can be adjusted accordingly by the cloud server according to the result of intention prediction.

[0033] In an exemplary embodiment, determining the behavior preference of the target object can be achieved by one of the following methods:

[0034] Method 1: Obtain the behavior preference set by the target object on the target application, where the target application is set to allow controlling the devices in the first group of devices;

[0035] It should be noted that when the applications corresponding to the devices in the first group of devices for device control are different, that is, in the case of multiple applications, the behavior preferences set by the target object on multiple applications can be obtained. Then, the behavior preference of the target object can be comprehensively determined.

[0036] In this embodiment, obtaining the behavior preference of the target object by Method 1 can make the obtained behavior preference of the target object more accurate.

[0037] Method 2: Obtain the second operation record of the target object on the first group of devices within the second preset time, and determine the behavior preference of the target object according to the second operation record, where the duration corresponding to the second preset time is greater than the duration corresponding to the first preset time.

[0038] It should be noted that the behavior preference of the target object includes but is not limited to the operations performed by the target object on the device before and after doing something, the preferences and portraits of the target object, etc.

[0039] It should be noted that the second preset time can be within 1 month or within 2 months.

[0040] The above Method 2 can also determine the behavior preference of the target object in the case where the target object has no behavior preference set on the target application, avoiding the low prediction accuracy caused by the lack of the behavior preference of the target object when predicting the intention of the target object.

[0041] In an exemplary embodiment, the behavior preference of the target object can also be jointly determined by the behavior preference set by the target object on the target application and the second operation record of the target object on the first group of devices within the second preset time.

[0042] Step S204, determine the target intention of the target object according to the target image, the first operation record, and the behavior preference;

[0043] It should be noted that, compared with predicting the intention by using only one of the target image, the first operation record, and the behavior preference, predicting the intention jointly by using the target image, the first operation record, and the behavior preference can improve the accuracy of the prediction.

[0044] In an exemplary embodiment, the above step S204 can be implemented in the following manner: perform feature processing on the target image, the first operation record, and the behavior preference and then input them into an intention recognition model to obtain multiple intentions and probabilities corresponding to the multiple intentions; determine the intention with the highest probability among the multiple intentions as the target intention of the target object. Optionally, the above intention recognition model is a deep learning model, and the intention recognition model is pre-trained according to training data.

[0045] It should be noted that the intentions include but are not limited to: going out, working, meeting guests, entertaining, etc. Specifically, it can also be predicted that the target intention of the target object is to go out for half an hour, etc.

[0046] In this embodiment, predicting the intention of the target object by means of the intention recognition model can more conveniently predict the intention and make the predicted intention more accurate.

[0047] In an exemplary embodiment, the intention recognition model at least includes a first module, a second module, a third module, and a fourth module; the first module is used to obtain a first intermediate result through the position of the target object and the action of the target object, the second module is used to obtain a second intermediate result according to the first operation record, the third module is used to determine a third intermediate result through the behavior preference, and the fourth module is used to determine the multiple intentions and probabilities corresponding to the multiple intentions through the first intermediate result, the second intermediate result, and the third intermediate result, and the position of the target object and the action of the target object are obtained by performing image recognition on the target image.

[0048] It should be noted that the first module can first perform image recognition on the target image collected by the image acquisition device, and then determine the position of the target object and the action of the target object (for example, determine that the target object is currently watching TV in the living room, cooking in the kitchen, etc.).

[0049] In an exemplary embodiment, it is assumed that an image acquisition device acquires a first image, where the first image is used to indicate that the target object is putting on shoes at the door at this time, and the obtained first operation record is used to indicate that the user started the washing machine to wash clothes 10 minutes ago. The behavior preference of the user indicates that the target object often goes out for about half an hour after starting the washing machine. Then, through prediction by an intention recognition model, it is determined that the target intention of the target object is to go out for half an hour.

[0050] In this embodiment, when constructing the intention recognition model, different modules process and predict different data inputs and outputs, and comprehensively process the processing and prediction results of different modules, which can make the prediction efficiency of the intention recognition model higher and the prediction results more accurate.

[0051] In an exemplary embodiment, after performing the above step S204, the following steps S11 - S14 are further included:

[0052] Step S11: Instruct the target object to determine whether the target intention is correct;

[0053] Specifically, the determined target intention can be sent to the mobile terminal corresponding to the target object, so that the target intention is displayed on the mobile terminal, instructing the target object to determine whether the target intention is correct.

[0054] It should be noted that the mobile terminal includes but is not limited to: mobile phones, watches, etc.

[0055] Step S12: In the case where the target object determines that the target intention is incorrect, obtain the true intention feedback by the target object, and save the target data and the corresponding relationship between the target data and the true intention as a piece of training data in the target list, where the target data includes the target image, the first operation record, and the behavior preference;

[0056] It should be noted that if the target object determines through the mobile terminal that the recognized intention is incorrect, the target object can feedback the true intention to the cloud server through the mobile terminal.

[0057] In an exemplary embodiment, after obtaining the true intention of the target object, the second set of control instructions can be determined according to the true intention, and then the third set of devices corresponding to the second set of control instructions can be controlled to perform corresponding operations.

[0058] Step S13: In the case where the number of training data in the target list is greater than the preset threshold, retrain the intention recognition model according to the training data in the target list;

[0059] It should be noted that by iteratively optimizing the intent recognition model according to the feedback of the target object, the accuracy of the intent recognition model in predicting the intent of the target object is further improved.

[0060] Step S14: When the target object determines that the target intent is correct, determine a set of control instructions according to the target intent.

[0061] Optionally, only when the target intent of the target object recognized by the intent recognition model is correct, step S206 is executed.

[0062] Step S206, determine a set of control instructions according to the target intent, where the set of control instructions is used to control a second set of devices to perform corresponding operations;

[0063] It should be noted that the devices in the second set of devices are part of the devices in the family where the target object is located. When the first set of devices is all the devices in the family, the first set of devices includes the second set of devices.

[0064] In an exemplary embodiment, the above step S206 can be implemented by the following method three or method four:

[0065] Method three: Determine a set of control instructions corresponding to the target intent according to the preset correspondence between the intent and the control instructions;

[0066] It should be noted that the preset correspondence between the intent and the control instructions can be reflected by a control instruction table, and the correspondence between the intent and the control instructions in the control instruction table can be customized by the target object.

[0067] That is to say, the target object can customize different control instructions according to different intents. For example, when the intent is to go out for a long time, the control instruction is to turn off all the devices at home. When the intent is to go out for half an hour, the control instruction is to turn off the lighting appliances at home, etc.

[0068] In this embodiment, the user can customize different control instructions according to different intents, which can make the control instructions fully meet the actual needs of the user, thereby improving the user experience.

[0069] Method four: Determine a set of control instructions according to the environmental information, the behavior preferences of the target object, and the target intent.

[0070] The above method four can avoid the situation where the control instructions cannot be determined when the user does not customize the control instructions according to the intent, improving the user experience; at the same time, determining the control instructions according to the environmental information, the behavior preferences of the target object, and the target intent can also make the predicted control instructions more accurate.

[0071] In an exemplary embodiment, the above-mentioned fourth method can be implemented as follows: determining a second set of devices according to the behavior preferences of the target object and the target intention; determining the operating parameters of each device in the second set of devices according to the environmental information and the behavior preferences, and determining a set of control instructions according to the operating parameters of each device.

[0072] It should be noted that since the behavior preferences of the target object include the device operations performed by the target object after doing a certain thing. For example, the device operations performed after going out and coming home. Thus, a second set of devices to be controlled can be determined according to the behavior preferences and the target intention of the target object. For example, after the user goes out and comes home, the user will turn on the air conditioner, turn on the lighting, use the water heater to boil water, and so on. Thus, the second set of devices includes an air conditioner, a lighting lamp, and a water heater.

[0073] It should be noted that after determining the second set of devices, the control instructions for each device in the second set of devices can be jointly determined according to the current environmental information and the user's behavior preferences. For example, taking the air conditioner as an example, the user's behavior preferences indicate that the user often turns on the air conditioner at 28 degrees. However, since the environmental temperature is very hot at this time, the air conditioner temperature can be set to 25 degrees according to the environmental temperature. Another example is that when the user comes home, the user often adjusts the lighting lamp to the brightest gear. However, since the environmental information indicates that the current indoor and outdoor ambient light is relatively sufficient at this time, the brightness of the lighting lamp can be set to be relatively dim.

[0074] It should be noted that in this embodiment, first determining the devices to be controlled according to the behavior preferences and the target intention, and then determining the operating parameters of the devices according to the environmental information and the behavior preferences can make the final control instructions more in line with the actual needs of the user, improving the user experience.

[0075] Through the above steps, determining the target intention of the target object according to the target image obtained by the image acquisition device for image acquisition of the target object, the first operation record of the target object on the first set of devices within the first preset time, and the behavior preferences of the target object. Then, determining a set of control instructions for controlling the second set of devices according to the target intention. Since the user's intention can be predicted in advance, and then the devices can be controlled according to the user's intention, the problem that the devices cannot perform corresponding device control according to the user's behavior, resulting in a low degree of intelligence of the devices, is solved.

[0076] Obviously, the above-described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. To better understand the above method, the following combines embodiments to illustrate the above process, but is not used to limit the technical solutions of the embodiments of the present invention. Specifically:

[0077] In an alternative embodiment, Figure 3 is a flowchart (II) of a method for determining a control instruction according to an embodiment of the present invention. Specifically, it includes the following steps:

[0078] Step S301: Upload IoT data of the user's home scene (image data collected by the camera, usage records of household appliances (such as air conditioners, refrigerators));

[0079] Specifically, uploading IoT data of the user's home scene includes the location of the user's home identified by the home camera (living room, kitchen, etc.), the user's action sequence (standing, walking, sitting, etc.), and the operation records of the user's use of household appliances, such as switch operations, adjustment operations, etc. Household appliances include all household appliance units that have been device-bound, such as televisions, air conditioners, refrigerators, etc.

[0080] Step S302: Structure the raw data in the cloud and generate multi-modal features;

[0081] Specifically, the raw data obtained in the above step S301 is cleaned, analyzed and processed to produce structured feature dimension data.

[0082] Step S303: Build a user home scene intention classification system and label the user's intention through the image data collected by the camera;

[0083] It should be noted that it is necessary to classify the intentions of the user's home scene and build an intention classification system, such as classifying them into major intentions such as sleeping, taking a bath, cooking, dining, exercising, entertaining, working, receiving guests, and going out. Then, split the secondary sub-intentions under each major intention. For example, entertainment is split into watching TV, playing games, etc. Identify the user's current intention through the image data collected by the home camera. For example, if the user puts on shoes and opens the door to leave the house, it can be considered that the user's current intention is to go out.

[0084] Step S304: Build a deep learning bi-directional-LSTM model and train the user intention recognition model;

[0085] It should be noted that based on the characteristics of the user's intention (the characteristics before the user's intention occurs) and labels (the intention classification of the user), this dataset can be used as training data to train the user intention recognition model. The model structure can adopt the deep learning Bi-directional-LSTM model. The input is various behavior sequences of the user and statistical features related to the user portrait, and the output is the probability distribution vector of the user's intention.

[0086] Step S305: Predict the user's home intention, push the intention to the user through the mobile app or other smart wearable devices, and wait for the user's confirmation feedback;

[0087] That is to say, after obtaining the trained user intention recognition model, the user intention can be predicted in real time through the features calculated from the data reported by the user in the cloud, and the user intention can be pushed to the user through the mobile phone APP or other intelligent wearable devices, and the feedback on whether the user confirms the intention can be obtained. If the user does not give feedback, it is defaulted that the intention recognition is incorrect.

[0088] Step S306: If the user confirms the intention, make adjustment actions for various household electrical appliances connected to the home intelligently according to the user intention, and then inform the user of what operations have been done, and obtain the feedback from the user on the device operations;

[0089] It should be noted that if the user confirms the intention, for the corresponding intention, combined with the environment where the user is located at that time and the user's historical behavior habit preferences, etc., the bound household appliances are personalized for relevant operations and configurations. For example, if the user confirms the intention of traveling for a short time, and the user has always turned on the air conditioner for cooling before, then the household appliances other than the air conditioner, such as the lights, will be turned off. After the adjustment, the user is informed of the intelligent operations performed through the mobile app or other intelligent devices again. The user can feedback which operations are satisfactory and which need to be improved. If the user does not give feedback, it is defaulted that the operations are satisfactory.

[0090] Step S307: Collect the feedback from the user on the intention and the device operations, report it to the cloud, form a data closed loop, and perform iterative optimization on the intention model and the corresponding intention operations of the user.

[0091] That is to say, a complete user feedback mechanism can be established to obtain the feedback from the user, and then perform iterative optimization on the user intention recognition model and the corresponding intention operations of the user, form a data closed loop, and improve the user experience.

[0092] The embodiments of the present invention can record user habits, user behavior patterns, user preferences, etc. through multi-modal data. Furthermore, the artificial intelligence deep learning model is used to deeply identify the user intention in combination with the multi-modal data, predict the user intention when the user does not explicitly inform, and then perform intelligent configuration of the corresponding scenario for the predicted user intention.

[0093] That is to say, in the home scenario, the embodiments of the present invention make full use of the Internet of Things technology to fully mine the multi-modal feature data of the user's home behavior, understand the user's needs more intelligently, perform fine-grained prediction on various intentions in the user's home scenario, and perform personalized and intelligent home scenario configuration for the user's corresponding intentions to improve the user's home experience.

[0094] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0095] In this embodiment, a device for determining control instructions is also provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated. As used hereinafter, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0096] Figure 4 is a structural block diagram (one) of the device for determining control instructions according to an embodiment of the present invention. This device includes:

[0097] An acquisition module 42, configured to collect an image of a target object through an image acquisition device to obtain a target image; and acquire a first operation record of the target object on a first group of devices within a first preset time; and determine the behavior preference of the target object;

[0098] A first determination module 44, configured to determine the target intention of the target object according to the target image, the first operation record, and the behavior preference;

[0099] A second determination module 46, configured to determine a group of control instructions according to the target intention, where the group of control instructions is used to control a second group of devices to perform corresponding operations.

[0100] Through the above device, the target intention of the target object is determined according to the target image collected by the image acquisition device for the target object, the first operation record of the target object on the first group of devices within the first preset time, and the behavior preference of the target object. Furthermore, a group of control instructions for controlling the second group of devices is determined according to the target intention. Since the intention of the user can be predicted and determined in advance, and then the device can be controlled according to the intention of the user, the problem that the device cannot perform corresponding device control according to the behavior of the user, resulting in a low degree of intelligence of the device, is solved.

[0101] In an exemplary embodiment, the acquisition module 42 is further configured to acquire the behavior preferences set by the target object on the target application, where the target application is set to allow controlling the devices in the first group of devices; or acquire the second operation record of the target object on the first group of devices within a second preset time, and determine the behavior preferences of the target object according to the second operation record, where the duration corresponding to the second preset time is greater than the duration corresponding to the first preset time.

[0102] In an exemplary embodiment, the first determination module 44 is configured to perform feature processing on the target image, the first operation record, and the behavior preferences and then input them into an intent recognition model to obtain multiple intents and probabilities corresponding to the multiple intents; and determine the intent with the highest probability among the multiple intents as the target intent of the target object.

[0103] In an exemplary embodiment, the intent recognition model at least includes a first module, a second module, a third module, and a fourth module; the first module is configured to obtain a first intermediate result through the position of the target object and the action of the target object, the second module is configured to obtain a second intermediate result according to the first operation record, the third module is configured to determine a third intermediate result through the behavior preferences, and the fourth module is configured to determine the multiple intents and probabilities corresponding to the multiple intents through the first intermediate result, the second intermediate result, and the third intermediate result, and the position of the target object and the action of the target object are obtained by performing image recognition on the target image.

[0104] Figure 5 It is a structural block diagram (two) of a control instruction determination device according to an embodiment of the present invention. The device includes: a processing module 48, configured to, after determining the target intent of the target object according to the target image, the first operation record, and the behavior preferences, instruct the target object to determine whether the target intent is correct; in the case where the target object determines that the target intent is incorrect, acquire the true intent fed back by the target object, and save the target data and the correspondence between the target data and the true intent as a piece of training data in a target list, where the target data includes the target image, the first operation record, and the behavior preferences; in the case where the number of training data in the target list is greater than a preset threshold, retrain the intent recognition model according to the training data in the target list; the second determination module is further configured to, in the case where the target object determines that the target intent is correct, determine a group of control instructions according to the target intent.

[0105] In an exemplary embodiment, the second determination module 46 is further configured to determine a set of control instructions corresponding to the target intent according to a preset correspondence between intents and control instructions; or determine a set of control instructions according to environmental information, the behavior preferences of the target object, and the target intent.

[0106] In an exemplary embodiment, the second determination module 46 is further configured to determine a second set of devices according to the behavior preferences of the target object and the target intent; determine the operating parameters of each device in the second set of devices according to the environmental information and the behavior preferences, and determine the set of control instructions according to the operating parameters of each device.

[0107] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and wherein the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0108] Optionally, in this embodiment, the above storage medium may be configured to store a computer program for executing the following steps:

[0109] S1, perform image acquisition on the target object through an image acquisition device to obtain a target image; and obtain a first operation record of the target object on a first set of devices within a first preset time; and determine the behavior preferences of the target object;

[0110] S2, determine the target intent of the target object according to the target image, the first operation record, and the behavior preferences;

[0111] S3, determine a set of control instructions according to the target intent, where the set of control instructions is used to control a second set of devices to perform corresponding operations.

[0112] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disc that can store computer programs.

[0113] The specific examples in this embodiment may refer to the examples described in the above embodiments and exemplary embodiments, and will not be repeated here.

[0114] An embodiment of the present invention further provides an electronic device, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0115] Optionally, in this embodiment, the above-mentioned processor may be configured to execute the following steps through a computer program:

[0116] S1, perform image acquisition on the target object through an image acquisition device to obtain a target image; and acquire a first operation record of the target object on a first set of devices within a first preset time; and determine the behavior preference of the target object;

[0117] S2, determine the target intention of the target object according to the target image, the first operation record, and the behavior preference;

[0118] S3, determine a set of control instructions according to the target intention, where the set of control instructions is used to control a second set of devices to perform corresponding operations.

[0119] In an exemplary embodiment, the above-mentioned electronic device may further include a transmission device and an input / output device, where the transmission device is connected to the above-mentioned processor, and the input / output device is connected to the above-mentioned processor.

[0120] Specific examples in this embodiment may refer to the examples described in the above-mentioned embodiments and exemplary embodiments, and will not be elaborated here.

[0121] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. They can be implemented by program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order than here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module to implement. In this way, the present invention is not limited to any specific combination of hardware and software.

[0122] The above are only the preferred embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method for determining a control instruction, characterized in that Including: Performing image acquisition on a target object through an image acquisition device to obtain a target image; And obtaining a first operation record of the target object on a first set of devices within a first preset time; And determining the behavior preference of the target object; Determining the target intention of the target object according to the target image, the first operation record and the behavior preference; Determining a set of control instructions according to the target intention, wherein the set of control instructions is used to control a second set of devices to perform corresponding operations; Wherein, determining the target intention of the target object according to the target image, the first operation record and the behavior preference includes: Performing feature processing on the target image, the first operation record and the behavior preference and inputting them into an intention recognition model to obtain multiple intentions and probabilities corresponding to the multiple intentions; Determining the intention with the highest probability among the multiple intentions as the target intention of the target object; Wherein, the intention recognition model at least includes a first module, a second module, a third module, and a fourth module; the first module is used to obtain a first intermediate result through the position and action of the target object, the second module is used to obtain a second intermediate result according to the first operation record, the third module is used to determine a third intermediate result through the behavior preference, and the fourth module is used to determine the multiple intentions and probabilities corresponding to the multiple intentions through the first intermediate result, the second intermediate result and the third intermediate result, and the position and action of the target object are obtained by performing image recognition on the target image.

2. The method according to claim 1, characterized in that, Determining the behavior preference of the target object includes: Obtaining the behavior preference set by the target object on a target application, wherein the target application is set to allow controlling the devices in the first set of devices; or Obtaining a second operation record of the target object on the first set of devices within a second preset time and determining the behavior preference of the target object according to the second operation record, wherein the duration corresponding to the second preset time is greater than the duration corresponding to the first preset time.

3. The method according to claim 1, wherein After determining the target intention of the target object according to the target image, the first operation record and the behavior preference, the method further includes: Instructing the target object to determine whether the target intention is correct; In the case where the target object determines that the target intention is incorrect, obtaining the true intention fed back by the target object and saving the target data and the corresponding relationship between the target data and the true intention as a piece of training data in a target list, wherein the target data includes the target image, the first operation record, and the behavior preference; In the case where the number of training data in the target list is greater than a preset threshold, retraining the intention recognition model according to the training data in the target list; Determining a set of control instructions according to the target intention includes: In the case where the target object determines that the target intention is correct, determining a set of control instructions according to the target intention.

4. The method according to claim 1, wherein Determine a set of control instructions according to the target intention, including: Determine a set of control instructions corresponding to the target intention according to a preset correspondence between intentions and control instructions; or Determine a set of control instructions according to environmental information, the behavior preferences of the target object, and the target intention.

5. The method according to claim 4, wherein Determine a set of control instructions according to the environmental information, the behavior preferences of the target object, and the target intention, including: Determine a second set of devices according to the behavior preferences of the target object and the target intention; Determine the operating parameters of each device in the second set of devices according to the environmental information and the behavior preferences, and determine the set of control instructions according to the operating parameters of each device.

6. A determining device for control instructions, characterized in that, Including: An acquisition module for acquiring an image of a target object through an image acquisition device to obtain a target image; And acquire the first operation record of the target object on the first set of devices within a first preset time; And Determine the behavior preferences of the target object; A first determination module for determining the target intention of the target object according to the target image, the first operation record, and the behavior preferences; A second determination module for determining a set of control instructions according to the target intention, wherein the set of control instructions is used to control a second set of devices to perform corresponding operations; Wherein, the first determination module is configured to perform feature processing on the target image, the first operation record, and the behavior preferences and then input them into an intention recognition model to obtain multiple intentions and probabilities corresponding to the multiple intentions; determine the intention with the highest probability among the multiple intentions as the target intention of the target object; Wherein, the intention recognition model at least includes a first module, a second module, a third module, and a fourth module; the first module is used to obtain a first intermediate result through the position of the target object and the action of the target object, the second module is used to obtain a second intermediate result according to the first operation record, the third module is used to determine a third intermediate result through the behavior preferences, and the fourth module is used to determine the multiple intentions and the probabilities corresponding to the multiple intentions through the first intermediate result, the second intermediate result, and the third intermediate result, and the position of the target object and the action of the target object are obtained through image recognition of the target image.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 5 when running.

8. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 5 through the computer program.

Images