Control method, device, storage medium and electronic device for household appliances

Abstract

The present application discloses a control method, device, storage medium and electronic device for household appliances, and relates to the field of smart home technology. The control method for household appliances includes: obtaining N device operation logs; generating a heterogeneous graph based on the N device operation logs, wherein the heterogeneous graph includes multiple operation jump nodes between M household appliances and connection relationships between the multiple operation jump nodes; determining the jump probability distribution between the target operation jump node and other operation jump nodes in the heterogeneous graph to determine the jump node associated with the target operation jump node; and determining the target operation instruction of the target household appliance according to the jump node associated with the target operation jump node.

Description

Technical Field

[0001] The present application relates to the field of equipment, and more specifically, to a control method, device, storage medium and electronic device for household electrical appliances. Background Art

[0002] At present, the development of "smart home" is in full swing, but the traditional "single-point intelligence" or "stand-alone intelligence" has gradually failed to bring better experience to users, and the demand for "network-device linkage" is becoming increasingly strong. For example, when a user wants to take a bath, intelligently recommending a bathing scene to the user (the scene is the linkage operation of several network devices, where the bathing scene can be <turn on the water heater, set the temperature to 39℃, turn on the bathroom heater, turn on the ventilation fan>) can greatly improve the user experience.

[0003] However, scene generation is difficult, involving multiple network devices and multiple functions. The possible combinations are in the hundreds of millions, but not all combinations are meaningful. It is basically impossible to screen them manually. The scenes predefined by humans are relatively monotonous and it is difficult to take into account the personalized needs of users.

[0004] At present, the control of home appliances is still based on "single-point intelligence", that is, focusing on the intelligence of a single network device. Control instructions are generated based on isomorphic graphs, ignoring the properties of nodes and edges in the graph. It is not possible to accurately generate control instructions for interconnection between home appliances. Summary of the invention

[0005] Embodiments of the present invention provide a control method, device, storage medium and electronic device for household electrical appliances, so as to at least solve the problem in the related art that control instructions for interconnection between household electrical appliances cannot be accurately generated.

[0006] According to one embodiment of the present invention, a control method for household appliances is provided, comprising: obtaining N device operation logs, wherein each of the above-mentioned device operation logs includes multiple operation jump instructions executed on M household appliances, and the above-mentioned operation jump instructions are used to indicate a jump from a first operation control of the above-mentioned household appliances to other operation controls, and the above-mentioned M and the above-mentioned N are natural numbers greater than or equal to 1; generating a heterogeneous graph based on the N above-mentioned device operation logs, wherein the above-mentioned heterogeneous graph includes multiple operation jump nodes between the M above-mentioned household appliances and connection relationships between the multiple above-mentioned operation jump nodes; determining the jump probability distribution between the target operation jump node and the other above-mentioned operation jump nodes in the above-mentioned heterogeneous graph to determine the jump node associated with the above-mentioned target operation jump node; and determining the target operation instruction of the target household appliance according to the jump node associated with the above-mentioned target operation jump node.

[0007] According to one embodiment of the present invention, a control device for household appliances is provided, comprising: a first acquisition module, used to acquire N device operation logs, wherein each of the above-mentioned device operation logs includes multiple operation jump instructions executed on M household appliances, and the above-mentioned operation jump instructions are used to indicate a jump from a first operation control of the above-mentioned household appliances to other operation controls, and the above-mentioned M and the above-mentioned N are natural numbers greater than or equal to 1; a first generation module, used to generate a heterogeneous graph based on the N above-mentioned device operation logs, wherein the above-mentioned heterogeneous graph includes multiple operation jump nodes between the M above-mentioned household appliances and connection relationships between the multiple above-mentioned operation jump nodes; a first determination module, used to determine the jump probability distribution between the target operation jump node and the other above-mentioned operation jump nodes in the above-mentioned heterogeneous graph, so as to determine the jump node associated with the above-mentioned target operation jump node; a second determination module, used to determine the target operation instruction of the target household appliance according to the jump node associated with the above-mentioned target operation jump node.

[0008] In an exemplary embodiment, the above-mentioned first acquisition module includes: a first acquisition unit, used to obtain each of the above-mentioned device operation logs, and obtain N of the above-mentioned device operation logs, wherein the following operations are performed to obtain each of the above-mentioned device operation logs: obtain the operation instructions for controlling the above-mentioned household appliances in a preset area, and obtain multiple operation instructions; determine at least one of the following information as the above-mentioned device operation log: multiple of the above-mentioned operation instructions, time information corresponding to multiple of the above-mentioned operation instructions, location information corresponding to multiple of the above-mentioned operation instructions, and environmental information corresponding to multiple of the above-mentioned operation instructions.

[0009] In an exemplary embodiment, the above-mentioned first generation module includes: a first processing unit, used to deduplicate the N above-mentioned device operation logs to obtain K above-mentioned device operation logs, wherein the above-mentioned K is a natural number less than or equal to the above-mentioned M; a first determination unit, used to determine the operation category of each of the above-mentioned device operation logs in the K above-mentioned device operation logs; a second determination unit, used to determine the operation jump node corresponding to each of the above-mentioned operation jump instructions in the K above-mentioned device operation logs, and obtain multiple above-mentioned operation jump nodes; a third determination unit, used to determine the connection relationship between the above-mentioned multiple operation jump nodes, and obtain multiple connection edges; a first generation unit, used to generate the above-mentioned heterogeneous graph based on the above-mentioned operation category, the above-mentioned multiple operation jump nodes and the above-mentioned multiple connection edges.

[0010] In an exemplary embodiment, the above-mentioned device also includes: a third determination module, which is used to determine the operation category corresponding to each of the above-mentioned operation jump nodes after generating a heterogeneous graph based on the N above-mentioned device operation logs; and a first response module, which is used to respond to the operation instructions acting on the above-mentioned household appliance device using the above-mentioned heterogeneous graph based on the operation category corresponding to each of the above-mentioned operation jump nodes.

[0011] In an exemplary embodiment, the first determination module includes: a first receiving unit, used to receive an operation instruction acting on the target home appliance, wherein the operation instruction includes the target operation jump node; a fourth determination unit, used to determine the environmental information in a preset area corresponding to the operation instruction, and the characteristic information triggering the operation instruction, wherein the target home appliance is set in the preset area; a fifth determination unit, used to determine the jump probability distribution between the target operation jump node and the adjacent operation jump nodes using the heterogeneous graph, wherein the adjacent operation jump nodes are included in the other operation jump nodes; a sixth determination unit, used to determine the jump node associated with the target operation jump node based on the jump probability distribution, the environmental information and the characteristic information.

[0012] In an exemplary embodiment, the fifth determination unit includes: a first determination subunit, used to determine multiple operation jump nodes associated with the target operation jump node; a second determination subunit, used to determine the connection relationship between the multiple associated operation jump nodes and the target operation jump node in the heterogeneous graph, and determine multiple connection relationships; a third determination subunit, used to determine the sum of weights of the multiple connection relationships; and a first processing subunit, used to normalize the sum of the weights to determine the jump probability distribution.

[0013] In an exemplary embodiment, the fifth determination unit includes: a fourth determination sub-unit, used to determine multiple operation jump nodes associated with the target operation jump node; a fifth determination sub-unit, used to determine multiple connection edges between the target operation jump node and the associated multiple operation jump nodes as sample data; and a first establishment sub-unit, used to establish a network model using the sample data, so as to output the jump probability distribution using the network model.

[0014] In an exemplary embodiment, the apparatus further includes: a first correction module for correcting the target operation instruction of the target home appliance according to the jump node associated with the target operation jump node, and then correcting the target operation instruction according to the device information of the target home appliance.

[0015] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0016] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0017] Through the present invention, by obtaining N device operation logs, each device operation log includes multiple operation jump instructions executed on M home appliances, the operation jump instructions are used to indicate jumping from the first operation control of the home appliance to other operation controls, and M and N are both natural numbers greater than or equal to 1; a heterogeneous graph is generated based on the N device operation logs, wherein the heterogeneous graph includes multiple operation jump nodes between M home appliances and the connection relationship between multiple operation jump nodes; the jump probability distribution between the target operation jump node and other operation jump nodes is determined in the heterogeneous graph to determine the jump node associated with the target operation jump node; the target operation instruction of the target home appliance is determined according to the jump node associated with the target operation jump node. Since in the above method, a heterogeneous graph of the operation log is constructed, the attributes of the edges between the multiple operation jump nodes and the attributes of the multiple operation jump nodes can be obtained. By combining the attributes of the edges and nodes and the jump probability distribution between the target operation jump node and other operation jump nodes, the control instructions of the home appliance can be generated more realistically. While ensuring the search speed, the accuracy and diversity of the search control instructions are also improved. Therefore, the problem that the control instructions for interconnection between household appliances cannot be accurately generated in the related art can be solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 is a schematic diagram of a hardware environment of a method for controlling a household appliance according to an embodiment of the present application;

[0021] Figure 2 A control method for a household appliance according to an embodiment of the present invention;

[0022] Figure 3 is a schematic diagram of nodes and edges according to an embodiment of the present invention;

[0023] Figure 4is a schematic diagram of probability distribution according to an embodiment of the present invention;

[0024] Figure 5 is a structural block diagram of a control device for a household appliance according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] According to one aspect of an embodiment of the present application, a method for controlling a home appliance is provided. The method for controlling a home appliance is widely used in smart home, smart home, smart home device ecosystem, smart home ecosystem and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the method for controlling a home appliance can be applied to Figure 1 In the hardware environment composed of the terminal device 102 and the server 104 shown in FIG. Figure 1 As shown, the server 104 is connected to the terminal device 102 via a network, and can be used to provide services (such as application services, etc.) for the terminal or a client installed on the terminal. A database can be set on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.

[0028] The network may include but is not limited to at least one of the following: wired network, wireless network. The wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the wireless network may include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 may be but is not limited to a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing device, a smart dishwasher, a smart projection device, a smart TV, a smart clothes drying rack, a smart curtain, a smart audio and video, a smart socket, a smart speaker, a smart fresh air device, a smart kitchen and bathroom device, a smart bathroom device, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purification device, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock, etc.

[0029] This embodiment provides a control method for a household appliance. Figure 2 A control method for a household appliance according to an embodiment of the present invention is as follows. Figure 2 As shown, the process includes the following steps:

[0030] Step S202, obtaining N device operation logs, wherein each device operation log includes a plurality of operation jump instructions executed on the M home appliances, the operation jump instructions are used to indicate a jump from a first operation control on the home appliance to another operation control, and M and N are both natural numbers greater than or equal to 1;

[0031] Step S204, generating a heterogeneous graph based on the N device operation logs, wherein the heterogeneous graph includes a plurality of operation jump nodes between the M home appliances and connection relationships between the plurality of operation jump nodes;

[0032] Step S206, determining the jump probability distribution between the target operation jump node and other operation jump nodes in the heterogeneous graph to determine the jump node associated with the target operation jump node;

[0033] Step S208: determining the target operation instruction of the target home appliance according to the jump node associated with the target operation jump node.

[0034] Optionally, the values ​​of N and M can be flexibly set based on the actual reference scenario or composition requirements. For example, more than 5 million device operation logs are obtained, and each device operation log is an operation transfer of a home appliance in a family. For example, based on the family as a unit, the control operation from turning on the air conditioner to turning off the air conditioner is recorded. In addition, a device operation log includes contextual features such as the time, location, weather, etc. from the from operation to the to operation, as well as some family portrait features.

[0035] The execution subject of the above steps may be a terminal, a server, a specific processor provided in the terminal or the server, or a processor or processing device provided relatively independently from the terminal or the server, but is not limited thereto.

[0036] Through the above steps, by obtaining N device operation logs, wherein each device operation log includes multiple operation jump instructions executed on M home appliances, the operation jump instructions are used to indicate jumping from the first operation control of the home appliance to other operation controls, and M and N are both natural numbers greater than or equal to 1; generating a heterogeneous graph based on the N device operation logs, wherein the heterogeneous graph includes multiple operation jump nodes between M home appliances and the connection relationship between multiple operation jump nodes; determining the jump probability distribution between the target operation jump node and other operation jump nodes in the heterogeneous graph to determine the jump node associated with the target operation jump node; determining the target operation instruction of the target home appliance according to the jump node associated with the target operation jump node. Since in the above method, a heterogeneous graph of operation logs is constructed, the attributes of the edges between multiple operation jump nodes and the attributes of multiple operation jump nodes can be obtained. Combining the attributes of the edges and nodes and the jump probability distribution between the target operation jump node and other operation jump nodes, the control instructions of the home appliance can be generated more realistically. While ensuring the search speed, the accuracy and diversity of the search control instructions are also improved. Therefore, the problem that the control instructions for interconnection between household appliances cannot be accurately generated in the related art can be solved.

[0037] In an exemplary embodiment, obtaining N device operation logs includes:

[0038] S1, obtain each device operation log to obtain N device operation logs, wherein the following operations are performed to obtain each device operation log:

[0039] Obtaining operation instructions for controlling home appliances in a preset area, and obtaining multiple operation instructions;

[0040] At least one of the following information is determined as the device operation log: multiple operation instructions, time information corresponding to the multiple operation instructions, location information corresponding to the multiple operation instructions, and environment information corresponding to the multiple operation instructions.

[0041] Optionally, the preset area can be a local area network. For example, taking a family as a unit, obtain operations such as "turn on the air conditioner-turn off the air conditioner" that occur in a family, record the time when each operation instruction is issued in "turn on the air conditioner-turn off the air conditioner", the location of the air conditioner, and weather information (for example, sunny or rainy), etc. By obtaining each device operation log in a family unit, a heterogeneous graph can be accurately generated in combination with the contextual features of the device operation log.

[0042] In an exemplary embodiment, generating a heterogeneous graph based on N device operation logs includes:

[0043] S1, perform deduplication processing on N device operation logs to obtain K device operation logs, where K is a natural number less than or equal to M;

[0044] S2, determining the operation category of each device operation log in the K device operation logs;

[0045] S3, determining the operation jump node corresponding to each operation jump instruction in the K device operation logs, and obtaining multiple operation jump nodes;

[0046] S4, determining the connection relationship between the multiple operation jump nodes to obtain multiple connection edges;

[0047] S5, generates a heterogeneous graph based on operation categories, multiple operation jump nodes, and multiple connection edges.

[0048] Optionally, the value of K can be flexibly set based on actual reference scenarios or composition requirements. For example, duplicate logs are removed from 5 million device operation logs to obtain 3 million device operation logs.

[0049] Optionally, the deduplicated device operation logs may be set in a collection, so that log information corresponding to the device operation logs (eg, the type of the log, the time and location of acquisition, etc.) is recorded in the collection.

[0050] In an exemplary embodiment, after generating a heterogeneous graph based on N device operation logs, the method further includes:

[0051] S1, determine the operation category corresponding to each operation jump node;

[0052] S2, based on the operation category corresponding to each operation jump node, a heterogeneous graph is used to respond to the operation instructions acting on the home appliance.

[0053] Optionally, the operation categories corresponding to different operation jump nodes may be the same or different. For example, the operation category of the instruction to turn on the air conditioner is "turn on". Adjusting the temperature of the refrigerator to 4 degrees belongs to the "temperature adjustment" category. Select the operation jump node corresponding to the user's operation instruction from the heterogeneous graph. For example, the operation jump node corresponding to the air conditioner can be <turn on the air conditioner, lower the air conditioner temperature, and increase the air conditioner wind speed>.

[0054] In an exemplary embodiment, determining the jump probability distribution between a target operation jump node and other operation jump nodes in a heterogeneous graph to determine a jump node associated with the target operation jump node includes:

[0055] S1, receiving an operation instruction acting on a target home appliance, wherein the operation instruction includes a target operation jump node;

[0056] S2, determining environmental information in a preset area corresponding to the operation instruction and characteristic information triggering the operation instruction, wherein the target home appliance is set in the preset area;

[0057] S3, determining a jump probability distribution between a target operation jump node and an adjacent operation jump node using a heterogeneous graph, wherein the adjacent operation jump node is included in other operation jump nodes;

[0058] S4, determining a jump node associated with the target operation jump node based on the jump probability distribution, the environment information and the feature information.

[0059] Optionally, the operation instruction can be in the form of voice or touch screen. For example, the user issues the instruction of "turn on the water heater", and searches the heterogeneous graph for the operation jump nodes adjacent to "turn on the water heater", such as "air conditioning temperature modulation 28 degrees", "turn on the ventilation equipment", etc. And calculate the jump probability between "air conditioning temperature modulation 28 degrees" and "turn on the ventilation equipment". When the jump probability between "air conditioning temperature modulation 28 degrees" and "turn on the ventilation equipment" is greater than 70%, it is determined that "air conditioning temperature modulation 28 degrees" and "turn on the ventilation equipment" are the jump nodes associated with "turn on the water heater", and the air conditioning and ventilation equipment are turned on according to the control instructions corresponding to the associated jump nodes.

[0060] In an exemplary embodiment, determining the jump probability distribution between a target operation jump node and adjacent operation jump nodes using a heterogeneous graph includes:

[0061] S1, determining multiple operation jump nodes associated with a target operation jump node;

[0062] S2, determining connection relationships between multiple associated operation jump nodes and a target operation jump node in the heterogeneous graph, and determining multiple connection relationships;

[0063] S3, determine the sum of weights of multiple connection relationships;

[0064] S4, the sum of normalized weights to determine the jump probability distribution.

[0065] Optionally, multiple connection relationships can be lines between operation jump nodes in a heterogeneous graph. They constitute the edges of the heterogeneous graph. For example, a transfer from operation i to operation j is an edge, and an edge is a directed graph. If there are multiple transfer operations from node i to node j, there will be multiple edges, and there will be homogeneous edges among the multiple edges. Aggregation can be performed based on edge attributes, and edges with the same attributes can be aggregated together.

[0066] Optionally, in this embodiment, without considering user or family feature information and context features, all edge weights from node i to node j may be added together and then normalized to determine a probability distribution.

[0067] In an exemplary embodiment, determining the jump probability distribution between a target operation jump node and adjacent operation jump nodes using a heterogeneous graph includes:

[0068] S1, determining multiple operation jump nodes associated with a target operation jump node;

[0069] S2, determining multiple connection edges between the target operation jump node and multiple associated operation jump nodes as sample data;

[0070] S3, using the sample data to establish a network model, so as to output the jump probability distribution using the network model.

[0071] Optionally, each connection edge in the heterogeneous graph can be used as a sample. For example, in a heterogeneous graph, node i to node j can produce T samples. After obtaining the training samples, xgboost or LSTM time series modeling can be used to learn the probability of node i to any of its adjacent nodes.

[0072] In an exemplary embodiment, after determining the target operation instruction of the target home appliance according to the jump node associated with the target operation jump node, the method further includes:

[0073] S1, modifying the target operation instruction according to the device information of the target home appliance.

[0074] Optionally, the revised rules include but are not limited to: the sub-function of the home appliance must be turned on after the home appliance itself is turned on; the time difference between turning on and off the same home appliance must be greater than a predetermined threshold; the output operation instructions do not include pause-type operation instructions.

[0075] The present invention will be described below in conjunction with specific embodiments:

[0076] This embodiment uses the control of home appliances as an example to illustrate. In this embodiment, the operation (action) of the home appliance is a specific operation of the network device function, such as "turn on the air conditioner", "increase the air conditioner speed", "pause the TV", and the network device function is used to represent the control instruction of the home appliance. In this embodiment, the network device function can generate the use scenario of the home appliance. For example, when taking a bath, a bathing scene is recommended to the user (the scene is the linkage operation of several network devices, and the bathing scene here can be <turn on the water heater, set the temperature to 39°C, turn on the bathroom heater, and turn on the ventilation fan>).

[0077] A scene is used to represent an operation sequence (i.e., associated operation instructions), for example, <turn on the air conditioner, lower the air conditioner temperature, and increase the air conditioner speed>.

[0078] This embodiment uses multiple operation logs to generate a heterogeneous graph, where the heterogeneous graph G = (V, E), V is used to represent a set of nodes, V = {v1, v2, ..., v i , ...}, 1≤i≤|V|, E is used to represent the set of edges E={e1,e2,...,e j , ...}, 1≤j≤|E|. If there is a mapping Φ:V→O and a mapping Ψ:E→R, O and R are used to represent the type set of nodes and edges respectively, if |O|+|R|>2, the graph G is a heterogeneous graph. In this embodiment, action is used to represent a node in a heterogeneous graph, and the transfer relationship between actions is the edge of the heterogeneous graph.

[0079] The main implementation process of this embodiment includes:

[0080] Step 1: Collect and clean the historical operation logs of all users;

[0081] Step 2, build a heterogeneous graph based on the operation log;

[0082] Step 3: Based on the heterogeneous graph, use the beamsearch strategy to search and generate several scenarios (operation instructions associated with home appliances);

[0083] Step 4: Modify the output scenario according to the established rules.

[0084] In this embodiment, collecting and cleaning the historical operation logs of all users specifically includes: obtaining more than 5 million records, each record is an operation transfer of a family, for example, from turning on the air conditioner to turning off the air conditioner, where the family is used as the unit. In addition, a record will contain contextual features such as the time, location, weather, etc. of the from operation to the to operation, as well as some family portrait features.

[0085] In this embodiment, constructing a heterogeneous graph specifically includes:

[0086] 1) Define nodes (corresponding to the operation jump nodes mentioned above): Based on the full data obtained in Step 1, all actions are deduplicated to obtain an action set, which constitutes all the nodes v of the heterogeneous graph. i ∈V, and define each node v i Category c i , c i ∈C, for example, “turning on the air conditioner” belongs to the “cooling” category, and “turning on the oven” belongs to the “cooking” category.

[0087] 2) Define the edge (corresponding to the connecting edge in the above): a transfer from operation i to operation j is an edge, and e ij To represent (here is a directed graph). If there are multiple transfer operations between node i and node j, there will be multiple edges, and there will be homogeneous edges among the multiple edges. We can aggregate them according to the edge attributes and aggregate the edges with the same attributes together.

[0088] The schematic diagram of nodes and edges is as follows Figure 3 Optionally, use represents the edge set from i to j, where For the tth edge from node i to j, use the vector To represent E ij The property of the tth edge of To represent E ij The weight of the t-th edge, the actual meaning of the weight is the number of times operation i transfers to operation j within the statistical range.

[0089] 3) Define a heterogeneous graph: After obtaining all nodes and edges, a heterogeneous graph G = (V, E) can be constructed;

[0090] 4) Define metapath: metapath is a man-made walking rule used to constrain the type of nodes selected during walking, M = [m1, m2, ..., m m ], m comes from the node type set C. For example, if M = [cooling, cooling, light], the first two positions in the generated scene operation sequence must be nodes of the "cooling" class, and the third must be a node of the "light" class. This is similar to the idea of ​​metapath2vec, but does not require metapath to be symmetrical.

[0091] In this embodiment, based on heterogeneous graphs, a beamsearch strategy (beamsearch strategy is used to select better results) is used to search and produce several scenarios; searching based on beamsearch is a trade-off between search complexity and search speed. If the beam constraint of the search is not performed, the complexity will increase exponentially with each subsequent expansion, which is equivalent to a full permutation. If depth is the depth, n is the number of nodes, and k is the search bandwidth, the complexity is O(n depth ), using beamseach to constrain the search bandwidth can reduce the complexity to O(depth·k·n). If a greedy strategy is used for searching, it is equivalent to beam=1, that is, the search only produces one sequence.

[0092] When doing a beamsearch search, each time you expand one level backward, you need to refer to the probability distribution of the current node to its adjacent nodes. It also needs to consider the user or family features and context features that trigger the scene generation, which are represented by q. ij are all n-order vectors, so the probability distribution to be learned is P(node j |node i , q).

[0093] In this embodiment, the probability distribution can be determined in the following two ways:

[0094] 1) Using generative-model:

[0095] Based on the heterogeneous graph in Step 2, if user or family characteristics and context characteristics are not considered, the probability distribution is obtained by adding up the weights of all edges from node i to node j and then normalizing them, that is, If user / family characteristics and contextual characteristics are considered, similarity is the similarity calculation function.

[0096] 2) Using the discriminative model:

[0097] Each edge of the heterogeneous graph in Step 2 can be used as a sample, the action corresponding to node i and the attribute of the edge from i to j are used as independent variables x, and the action of node j is used as dependent variable y. More formally speaking: in a heterogeneous graph, node i to node j can produce T samples, and each sample s (t) ={x (t) ,y (t)}, y (t) =action j , where t∈[1,T]. After obtaining the training samples, xgboost or LSTM time series modeling can be used to learn the probability P(node ​​i) of connecting to any of its adjacent nodes. j |node i ,p)=model(nide i , p).

[0098] In practical applications, it is found that heterogeneous graphs are highly sparse, and the discriminant method is more practical.

[0099] In addition, when selecting the next node, not only the probability distribution of the current node to its adjacent nodes is considered, but also the nodes in the previous step, such as Figure 4 As shown:

[0100] t->v->z, the weight is 0 because there is no path from v to z;

[0101] t->v->t, the weight is 0 because backtracking is not allowed;

[0102] t->v->x3, the weight is w(t,v)+w(v,x3);

[0103] t->v->x1, the weight is w(t,v)+w(v,x1)+α·w(t,x1). Since there is also a path from t->x1, the weight of this part is also taken into account and controlled by a coefficient α. α>0. The larger the value, the more inclined it is to wander locally.

[0104] In this embodiment, when walking, walking can be performed based on a defined metapath, and a node that does not meet the requirements of the metapath will not be selected as the next node.

[0105] In this embodiment, the output scene is modified according to the established rules. The specific revision rules include: the device sub-function must be turned on after the device itself is turned on; the time difference between the opening and closing of the same device must be greater than the established threshold; the output scene does not contain pause operations.

[0106] In summary, in the field of intelligent network-device linkage, this embodiment proposes to abstract the actual network-device linkage problem into a heterogeneous graph for modeling, define the attributes of each network device and each transfer operation, and combine user characteristics and contextual characteristics during reasoning, which is more in line with reality; it can realize the due role of beamsearch in scene generation, while ensuring the search speed, improving the accuracy and diversity of search results; based on machine learning methods, user characteristics and contextual characteristics are considered to predict the user's next action. It is more in line with reality, making the output results more personalized and improving user experience; better algorithm performance, even if it is a depth-first search, guarantees the results while controlling the complexity, making it better in engineering and easier to use.

[0107] In this embodiment, an image processing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0108] Figure 5 is a structural block diagram of a control device for household electrical appliances according to an embodiment of the present invention. Figure 5 As shown, the device comprises:

[0109] A first acquisition module 52 is used to acquire N device operation logs, wherein each device operation log includes a plurality of operation jump instructions executed on M household electrical appliances, the operation jump instructions are used to indicate a jump from a first operation control on the household electrical appliance to another operation control, and M and N are both natural numbers greater than or equal to 1;

[0110] A first generating module 54 is used to generate a heterogeneous graph based on the N device operation logs, wherein the heterogeneous graph includes a plurality of operation jump nodes between the M home appliances and connection relationships between the plurality of operation jump nodes;

[0111] A first determination module 56, configured to determine a jump probability distribution between a target operation jump node and other operation jump nodes in the heterogeneous graph, so as to determine a jump node associated with the target operation jump node;

[0112] The second determining module 58 is configured to determine a target operation instruction of a target household appliance according to a jump node associated with the target operation jump node.

[0113] In an exemplary embodiment, the first acquisition module includes:

[0114] The first acquisition unit is used to acquire each of the above-mentioned device operation logs to obtain N of the above-mentioned device operation logs, wherein the following operations are performed to acquire each of the above-mentioned device operation logs: acquiring the operation instructions for controlling the above-mentioned household electrical appliances in a preset area to obtain multiple operation instructions; determining at least one of the following information as the above-mentioned device operation log: multiple of the above-mentioned operation instructions, time information corresponding to the multiple of the above-mentioned operation instructions, location information corresponding to the multiple of the above-mentioned operation instructions, and environmental information corresponding to the multiple of the above-mentioned operation instructions.

[0115] In an exemplary embodiment, the first generating module includes:

[0116] A first processing unit is used to perform deduplication processing on the N device operation logs to obtain K device operation logs, where K is a natural number less than or equal to M;

[0117] A first determining unit is used to determine the operation category of each of the K device operation logs;

[0118] A second determining unit is used to determine the operation jump node corresponding to each of the operation jump instructions in the K device operation logs to obtain a plurality of the operation jump nodes;

[0119] A third determining unit is used to determine the connection relationship between the plurality of the above-mentioned operation jump nodes to obtain a plurality of connection edges;

[0120] The first generating unit is used to generate the heterogeneous graph based on the operation category, the multiple operation jump nodes and the multiple connection edges.

[0121] In an exemplary embodiment, the above device further comprises:

[0122] A third determination module is used to determine the operation category corresponding to each of the above-mentioned operation jump nodes after generating a heterogeneous graph based on the N above-mentioned device operation logs;

[0123] The first response module is used to respond to the operation instruction acting on the above-mentioned home appliance by using the above-mentioned heterogeneous graph based on the operation category corresponding to each of the above-mentioned operation jump nodes.

[0124] In an exemplary embodiment, the first determining module includes:

[0125] A first receiving unit, configured to receive an operation instruction acting on the target household appliance, wherein the operation instruction includes the target operation jump node;

[0126] a fourth determining unit, configured to determine environmental information in a preset area corresponding to the operation instruction and feature information triggering the operation instruction, wherein the target home appliance is arranged in the preset area;

[0127] A fifth determining unit, configured to determine, by using the heterogeneous graph, a jump probability distribution between the target operation jump node and an adjacent operation jump node, wherein the adjacent operation jump node is included in the other operation jump nodes;

[0128] The sixth determining unit is used to determine the jump node associated with the target operation jump node based on the jump probability distribution, the environment information and the feature information.

[0129] In an exemplary embodiment, the fifth determining unit includes:

[0130] A first determining subunit, configured to determine a plurality of operation jump nodes associated with the target operation jump node;

[0131] A second determining subunit is used to determine the connection relationship between the associated multiple operation jump nodes and the target operation jump node in the heterogeneous graph, and determine multiple connection relationships;

[0132] A third determining subunit is used to determine the sum of weights of the plurality of connection relationships mentioned above;

[0133] The first processing subunit is used to normalize the sum of the weights to determine the jump probability distribution.

[0134] In an exemplary embodiment, the fifth determining unit includes:

[0135] A fourth determining subunit, used to determine a plurality of operation jump nodes associated with the target operation jump node;

[0136] A fifth determining subunit, configured to determine a plurality of connection edges between the target operation jump node and the associated plurality of operation jump nodes as sample data;

[0137] The first establishing subunit is used to establish a network model using the sample data, so as to output the jump probability distribution using the network model.

[0138] In an exemplary embodiment, the apparatus further comprises:

[0139] The first correction module is used to correct the target operation instruction of the target home appliance according to the device information of the target home appliance after determining the target operation instruction of the target home appliance according to the jump node associated with the target operation jump node.

[0140] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0141] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0142] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for controlling a household appliance, characterized in that: include: Acquire N device operation logs, wherein each of the device operation logs includes a plurality of operation jump instructions executed on M household electrical appliances, the operation jump instructions are used to indicate a jump from a first operation control on the household electrical appliance to another operation control, and both M and N are natural numbers greater than or equal to 1; generating a heterogeneous graph based on the N device operation logs, wherein the heterogeneous graph includes a plurality of operation jump nodes between the M home appliances and connection relationships between the plurality of operation jump nodes; Determining the jump probability distribution between the target operation jump node and the other operation jump nodes in the heterogeneous graph to determine the jump node associated with the target operation jump node; Determine a target operation instruction of a target home appliance according to a jump node associated with the target operation jump node; The generating of a heterogeneous graph based on N device operation logs includes: performing deduplication processing on the N device operation logs to obtain K device operation logs, wherein K is a natural number less than or equal to M; determining an operation category of each of the K device operation logs; determining an operation jump node corresponding to each operation jump instruction in the K device operation logs to obtain a plurality of operation jump nodes; determining a connection relationship between the plurality of operation jump nodes to obtain a plurality of connection edges; generating the heterogeneous graph based on the operation category, the plurality of operation jump nodes and the plurality of connection edges.

2. The method according to claim 1, characterized in that Get N device operation logs, including: Obtain each of the device operation logs to obtain N of the device operation logs, wherein obtaining each of the device operation logs involves performing the following operations: Acquire an operation instruction for controlling the household appliance in a preset area, and obtain multiple operation instructions; At least one of the following information is determined as the device operation log: a plurality of the operation instructions, time information corresponding to the plurality of the operation instructions, location information corresponding to the plurality of the operation instructions, and environment information corresponding to the plurality of the operation instructions.

3. The method according to claim 1, characterized in that: After generating a heterogeneous graph based on the N device operation logs, the method further includes: Determine the operation category corresponding to each of the operation jump nodes; Based on the operation category corresponding to each of the operation jump nodes, the heterogeneous graph is used to respond to the operation instruction acting on the home appliance.

4. The method according to claim 1, characterized in that: Determining the jump probability distribution between the target operation jump node and the other operation jump nodes in the heterogeneous graph to determine the jump node associated with the target operation jump node includes: Receiving an operation instruction acting on the target home appliance, wherein the operation instruction includes the target operation jump node; Determine environmental information in a preset area corresponding to the operation instruction and characteristic information triggering the operation instruction, wherein the target home appliance is set in the preset area; Determining the jump probability distribution between the target operation jump node and the adjacent operation jump nodes by using the heterogeneous graph, wherein the adjacent operation jump nodes are included in other operation jump nodes; Based on the jump probability distribution, the environment information and the feature information, a jump node associated with the target operation jump node is determined.

5. The method according to claim 4, characterized in that Determining the jump probability distribution between the target operation jump node and the adjacent operation jump nodes by using the heterogeneous graph includes: Determining a plurality of operation jump nodes associated with the target operation jump node; Determine, in the heterogeneous graph, connection relationships between the associated multiple operation jump nodes and the target operation jump node, and determine multiple connection relationships; Determining the sum of weights of a plurality of the connection relationships; The sum of the weights is normalized to determine the jump probability distribution.

6. The method according to claim 4, characterized in that Determining the jump probability distribution between the target operation jump node and the adjacent operation jump nodes by using the heterogeneous graph includes: Determining a plurality of operation jump nodes associated with the target operation jump node; Determine a plurality of connection edges between the target operation jump node and a plurality of associated operation jump nodes as sample data; A network model is established using the sample data, so as to output the jump probability distribution using the network model.

7. The method according to claim 1, characterized in that After determining the target operation instruction of the target home appliance according to the jump node associated with the target operation jump node, the method further includes: The target operation instruction is modified according to the device information of the target home appliance.

8. A control device for household electrical appliances, characterized in that: include: A first acquisition module is used to acquire N device operation logs, wherein each of the device operation logs includes a plurality of operation jump instructions executed on M household electrical appliances, the operation jump instructions are used to indicate a jump from a first operation control on the household electrical appliance to other operation controls, and both M and N are natural numbers greater than or equal to 1; A first generating module is used to generate a heterogeneous graph based on the N device operation logs, wherein the heterogeneous graph includes a plurality of operation jump nodes between the M home appliances and connection relationships between the plurality of operation jump nodes; A first determination module is used to determine the jump probability distribution between the target operation jump node and other operation jump nodes in the heterogeneous graph to determine the jump node associated with the target operation jump node; A second determining module, configured to determine a target operation instruction of a target home appliance according to a jump node associated with the target operation jump node; The first generating module comprises: a first processing unit, used for performing deduplication processing on N device operation logs to obtain K device operation logs, wherein K is a natural number less than or equal to M; a first determining unit, used for determining the operation category of each of the K device operation logs; a second determining unit, used for determining the operation jump node corresponding to each operation jump instruction in the K device operation logs to obtain a plurality of operation jump nodes; a third determining unit, used for determining the connection relationship between the plurality of operation jump nodes to obtain a plurality of connection edges; and generating the heterogeneous graph based on the operation category, the plurality of operation jump nodes and the plurality of connection edges.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images