Washing control method and device, washing machine and storage medium

Abstract

Embodiments of the present application relate to a washing control method and device, a washing machine and a storage medium. The method comprises: when a washing task is received, obtaining a current washing parameter set of the washing task, and obtaining a historical washing parameter set of at least one historical washing task; performing reinforcement learning on the current washing parameter set and the at least one historical washing parameter set by using a trained reinforcement learning model to obtain a target washing strategy; and controlling the washing machine to perform the washing task according to the target washing strategy. Thus, the target of water saving, power saving, time saving and washing can be achieved in the washing process.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of smart home, and in particular to a washing control method and device, a washing machine and a storage medium. BACKGROUND

[0002] At present, most of the household appliances adopt the traditional control strategy, which is difficult to achieve the purpose of energy saving and emission reduction.

[0003] Among them, the washing machine market share is large, which provides great convenience for people to wash clothes. However, most of the existing washing control strategies are rule-based control methods based on the experience of designers, and the washing control strategy is generally fixed and cannot be changed by considering the washing habits of different users, which has problems of long washing time, power consumption and water consumption. SUMMARY

[0004] In view of this, in order to solve the technical problems that the existing washing control strategy cannot change by considering the washing habits of different users, and has problems of long washing time, power consumption and water consumption, embodiments of the present application provide a washing control method and device, a washing machine and a storage medium.

[0005] In a first aspect, the embodiments of the present application provide a washing control method, which comprises:

[0006] When a washing task is received, a current washing parameter set of the washing task is obtained, and a historical washing parameter set of at least one historical washing task is obtained;

[0007] The current washing parameter set and at least one historical washing parameter set are subjected to reinforcement learning by using a trained reinforcement learning model to obtain a target washing strategy;

[0008] The washing machine is controlled to perform the washing task according to the target washing strategy.

[0009] In a possible implementation, the obtaining of the historical washing parameter set of at least one historical washing task comprises:

[0010] The historical washing parameter set of a historical washing task recently performed by the washing machine is obtained, and / or the historical washing parameter set of a historical washing task performed by the washing machine within a recent set time period is obtained.

[0011] In a possible implementation, the reinforcement learning of the current washing parameter set and at least one historical washing parameter set by using the trained reinforcement learning model to obtain a target washing strategy comprises:

[0012] A first reward value corresponding to each historical washing parameter set is determined.

[0013] inputting the current washing parameter set, at least one of the historical washing parameter sets, and at least one of the first reward values into the trained reinforcement learning model to obtain a first washing strategy;

[0014] performing simulation processing by using the first washing strategy, and obtaining the target washing strategy according to a simulation processing result.

[0015] In one possible implementation, the determining of the first reward value corresponding to each of the historical washing parameter sets comprises:

[0016] for each of the historical washing parameter sets, determining a set of actual result parameters of the washing machine when performing a corresponding historical washing task;

[0017] determining the first reward value corresponding to the historical washing parameter set according to the set of actual result parameters.

[0018] In one possible implementation, the set of actual result parameters comprises one or more of the following parameters: total water consumption, total power consumption, total duration, and washing degree.

[0019] The determining of the first reward value corresponding to the historical washing parameter set according to the set of actual result parameters comprises:

[0020] substituting each parameter in the set of actual result parameters into a preset reward function to obtain the first reward value corresponding to the historical washing parameter set.

[0021] In one possible implementation, the washing degree of each of the historical washing tasks is determined by the following method:

[0022] in a case where the historical washing task has been performed, outputting a prompt message through a visual interface, the prompt message being used to prompt a user to evaluate the washing degree of the historical washing task;

[0023] receiving the washing degree input by the user according to the prompt message, and determining the washing degree input by the user as the washing degree of the historical washing task.

[0024] In one possible implementation, the performing of the simulation processing by using the first washing strategy and the obtaining of the target washing strategy according to the simulation processing result comprise:

[0025] performing iterative processing by using the following steps until a preset iteration stop condition is met:

[0026] controlling a virtual washing machine to perform simulation running according to the current first washing strategy to obtain a set of actual result parameters;

[0027] determining a second reward value corresponding to the current first washing strategy according to the actual result parameter set;

[0028] inputting the second reward value and the current washing parameter set into the reinforcement learning model to obtain a first washing strategy of this iteration;

[0029] After the iteration is completed, the first washing strategy obtained in the last iteration is determined as the target washing strategy.

[0030] In a possible implementation, the iteration stopping condition comprises that the number of iterations reaches a preset number threshold, or the sum of the second reward value obtained in the current iteration process and the second reward value obtained in a historical iteration process reaches a preset reward value threshold.

[0031] In a second aspect, an embodiment of the present application provides a washing control device, which comprises:

[0032] an acquisition module, configured to acquire a current washing parameter set of a washing task when the washing task is received, and acquire a historical washing parameter set of at least one historical washing task;

[0033] a learning module, configured to perform reinforcement learning on the current washing parameter set and at least one historical washing parameter set by using a trained reinforcement learning model to obtain a target washing strategy;

[0034] an execution module, configured to control a washing machine to perform a washing task according to the target washing strategy.

[0035] In a possible implementation, the acquisition module is specifically configured to:

[0036] acquire a historical washing parameter set of a historical washing task performed by the washing machine last time, and / or acquire a historical washing parameter set of a historical washing task performed by the washing machine in a recent preset time period.

[0037] In a possible implementation, the learning module comprises:

[0038] a determination sub-module, configured to determine a first reward value corresponding to each historical washing parameter set;

[0039] a training sub-module, configured to input the current washing parameter set, at least one historical washing parameter set, and at least one first reward value into the trained reinforcement learning model to obtain a first washing strategy;

[0040] a processing sub-module, configured to perform simulation processing by using the first washing strategy, and obtain the target washing strategy according to a simulation processing result.

[0041] In a possible implementation, the determining sub-module comprises:

[0042] The first determining unit is configured to determine, for each of the historical washing parameter set, a set of actual result parameters of the washing machine when performing a corresponding historical washing task.

[0043] The second determining unit is configured to determine, according to the set of actual result parameters, a first reward value corresponding to the historical washing parameter set.

[0044] In a possible implementation, the set of actual result parameters comprises one or more of the following parameters: total water consumption, total power consumption, total duration, and washing degree.

[0045] The second determining unit is specifically configured to:

[0046] substitute each parameter in the set of actual result parameters into a preset reward function to obtain the first reward value corresponding to the historical washing parameter set.

[0047] In a possible implementation, the washing degree of each historical washing task is determined by the following method:

[0048] In a case where the historical washing task has been performed, a visual interface is used to output a prompt message, and the prompt message is used to prompt a user to evaluate the washing degree of the historical washing task.

[0049] The washing degree input by the user according to the prompt message is determined as the washing degree of the historical washing task.

[0050] In a possible implementation, the processing sub-module is specifically configured to:

[0051] iteratively processes by using the following steps until a preset iteration stop condition is met:

[0052] controlling a virtual washing machine to perform simulation operation according to the current first washing strategy to obtain a set of actual result parameters;

[0053] determining, according to the set of actual result parameters, a second reward value corresponding to the current first washing strategy;

[0054] inputting the second reward value and the current washing parameter set into the reinforcement learning model to obtain a first washing strategy of this iteration;

[0055] After the iteration is completed, the first washing strategy obtained in the last iteration is determined as the target washing strategy.

[0056] In a possible implementation, the iteration stopping condition comprises: the number of iterations reaches a preset number threshold, or the sum of the second reward value obtained in the current iteration process and the second reward value obtained in a historical iteration process reaches a preset reward value threshold.

[0057] In a third aspect, an embodiment of the present application provides a washing machine, comprising a processor and a memory, the processor is used to execute a washing control program stored in the memory to implement the washing control method in any one of the first aspect.

[0058] In a fourth aspect, an embodiment of the present application provides a storage medium, the storage medium stores one or more programs, the one or more programs can be executed by one or more processors to implement the washing control method in any one of the first aspect.

[0059] The technical solution provided by the embodiment of the present application comprises the following steps: when a washing task is received, a current washing parameter set of the washing task is obtained, and a historical washing parameter set of at least one historical washing task is obtained; a trained reinforcement learning model is used to perform reinforcement learning on the current washing parameter set and the at least one historical washing parameter set to obtain a target washing strategy; and the washing machine is controlled to perform the washing task according to the target washing strategy. This technical solution inputs the current washing parameter set and the at least one historical washing parameter set into the trained reinforcement learning model to perform reinforcement learning. Since the historical washing parameter set can reflect the historical washing habits of the user, the washing habits of the user can be better explored, and the washing strategy can be continuously improved through reinforcement learning on the premise of fully understanding the washing habits of different users, so that an optimal target washing strategy can be formed, and the goals of water saving, power saving, time saving and washing are achieved in the washing process. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 An embodiment flowchart of a washing control method provided by the embodiment of the present application is provided.

[0061] Figure 2 An embodiment flowchart of another washing control method provided by the embodiment of the present application is provided.

[0062] Figure 3 An embodiment flowchart of still another washing control method provided by the embodiment of the present application is provided.

[0063] Figure 4 A reinforcement learning schematic diagram provided by the embodiment of the present application is provided.

[0064] Figure 5 A reinforcement learning Actor-Critic algorithm network schematic diagram provided by the embodiment of the present application is provided.

[0065] Figure 6 A reinforcement learning process schematic diagram provided for an embodiment of the present application;

[0066] Figure 7 An embodiment flow chart of another washing control method provided for an embodiment of the present application;

[0067] Figure 8 An embodiment block diagram of a washing control device provided for an embodiment of the present application;

[0068] Figure 9 A structure schematic diagram of a washing machine provided for an embodiment of the present application. DETAILED DESCRIPTION

[0069] To make the objectives, technical solutions and advantages of embodiments of the present application clearer, the following will be combined with the accompanying drawings for the embodiments of the present application to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0070] The following will be further explained and described the washing control method provided by the present application with specific embodiments combined with the accompanying drawings, and the embodiments do not constitute a limitation on the embodiments of the present application.

[0071] Reference is made to Figure 1 An embodiment flow chart of a washing control method provided for an embodiment of the present application. As shown in Figure 1 The flow chart can include the following steps:

[0072] Step 101, when receiving a washing task, obtaining a current washing parameter set of the washing task and obtaining a historical washing parameter set of at least one historical washing task.

[0073] The current washing parameter set includes the washing parameters initially set for the washing task. The washing parameters include but are not limited to: the time to start washing, the washing mode (such as standard washing mode, fast washing mode, strong washing mode, etc.), the washing time length, the water consumption, etc.

[0074] The historical washing parameter set includes the washing parameters initially set for the washing task in the completed washing task. Similarly, the washing parameters include but are not limited to: the time to start washing, the washing mode (such as standard washing mode, fast washing mode, strong washing mode, etc.), the washing time length, the water consumption, etc.

[0075] In actual life, the user can set the washing parameters corresponding to the washing task in advance when performing the washing task. Based on this, the execution subject of the embodiment of the present application can generate the current washing parameter set corresponding to the washing task according to the washing parameters set by the user in advance when receiving the washing task.

[0076] In an embodiment, the washing machine stores the current washing parameter set corresponding to the washing task in the preset database every time the washing task is completed. Based on this, the execution subject of the embodiment of the present application can directly obtain the historical washing parameter set of the historical washing task from the above-mentioned database when obtaining the historical washing parameter set of the at least one historical washing task.

[0077] In an embodiment, the execution subject of the embodiment of the present application can obtain the historical washing parameter set of the historical washing task performed by the washing machine last time.

[0078] In another embodiment, the execution subject of the embodiment of the present application can obtain the historical washing parameter set of the historical washing task performed by the washing machine in a recent set time period. For example, the historical washing parameter set of the historical washing task performed by the washing machine in the last week is obtained.

[0079] In still another embodiment, in the case that two or more historical washing tasks are performed by the washing machine in a recent set period, the execution subject of the embodiment of the present application can obtain the historical washing parameter set of the historical washing task performed by the washing machine last time and the historical washing parameter set of the historical washing task performed by the washing machine in a recent set time period. For example, in the case that two historical washing tasks are performed by the washing machine in the last week, the historical washing parameter set corresponding to the historical washing task performed by the washing machine last time and the historical washing task performed in the last week is obtained.

[0080] Step 102, performing reinforcement learning on the current washing parameter set and the at least one historical washing parameter set by using the trained reinforcement learning model to obtain a target washing strategy.

[0081] The reinforcement learning model can be used to perform reinforcement learning on the input current washing parameter set and the at least one historical washing parameter set to generate a target washing strategy.

[0082] The target washing strategy refers to the parameters actually controlling each stage of the washing process when the washing task is completed. The stages include but are not limited to soaking, rinsing, dehydration, etc., and the parameters include but are not limited to time, speed, temperature, water consumption, and power consumption, etc.

[0083] In the embodiment of the present application, the number of family members and the number of clothes of a certain user are generally constant, and the types and quantities of clothes to be washed are different. Therefore, by inputting the current washing parameter set and at least one historical washing parameter set into the trained reinforcement learning model for reinforcement learning, the user's laundry habits can be better explored, and the optimal target washing strategy can be formed.

[0084] As to how to use the trained reinforcement learning model to perform reinforcement learning on the current washing parameter set and at least one historical washing parameter set to obtain the target washing strategy, the following flowchart can be used for illustration, which will not be described in detail here. Figure 2

[0085] Step 103, controlling the washing machine to perform a washing task according to the target washing strategy.

[0086] In an embodiment, after obtaining the target washing strategy of the current washing task, the execution subject of the embodiment of the present application can control the washing machine to perform a washing task according to the target washing strategy.

[0087] For example, assuming that the execution subject of the embodiment of the present application obtains the current washing parameter set corresponding to the current washing task as {start washing time: 2022-01-02 12:00; washing mode: fast washing mode; washing time: 15 minutes; water consumption: 50L (liter)}, and the historical washing parameter set corresponding to the last washing task as {start washing time: 2022-01-01 16:00; washing mode: fast washing mode; washing time: 15 minutes; water consumption: 60L (liter)}.

[0088] Continuing to assume that the trained reinforcement learning model is used to perform reinforcement learning on the above-mentioned current washing parameter set and historical washing parameter set, the parameters of each stage corresponding to the obtained target washing strategy are: {washing: {time: 4 minutes; speed: 800 rpm; temperature: 25℃; water consumption: 15L}, rinsing: {time: 8 minutes; speed: 800 rpm; temperature: 30℃; water consumption: 30L}, and dehydration: {time: 1 minute; speed: 1000 rpm; temperature: 30℃; water consumption: 0L}}.

[0089] Then, the washing machine can be controlled to perform a washing task according to the above-mentioned target washing strategy. As can be seen from the above, the parameters corresponding to the above-mentioned target washing strategy are less than the washing parameters set by the user, whether in terms of washing time or water consumption.

[0090] ​The technical scheme provided by the embodiment of the present application comprises the following steps: when a washing task is received, a current washing parameter set of the washing task is acquired, and a historical washing parameter set of at least one historical washing task is acquired; the current washing parameter set and the historical washing parameter set are subjected to reinforcement learning by using a trained reinforcement learning model, so as to obtain a target washing strategy; and the washing machine is controlled to execute the washing task according to the target washing strategy. According to the technical scheme, the current washing parameter set and the historical washing parameter set are input into the trained reinforcement learning model to perform reinforcement learning. Since the historical washing parameter set can reflect the historical washing habits of the user, the washing habits of the user can be better explored, and the washing strategy can be continuously improved by reinforcement learning on the premise of fully understanding the washing habits of different users, so that an optimal target washing strategy can be formed, and the goals of water saving, power saving, time saving and washing in the washing process can be achieved.

[0091] In addition, the current washing parameter set can also include the type of detergent, the material and weight of clothes and the like set by the user, so that the type of detergent, the material and weight of clothes and the like can be comprehensively considered to form an optimal target washing strategy, and the goals of water saving, power saving, time saving and washing in the washing process can be achieved.

[0092] Referring to Figure 2 An embodiment flowchart of another washing control method provided by the embodiment of the present application is shown. Figure 2 The flowchart shown in Figure 1 Based on the flowchart shown in Figure 2 The flowchart shown in

[0093] Step 201, determining a first reward value corresponding to each historical washing parameter set.

[0094] The first reward value can be obtained according to the actual result parameter obtained when the historical washing task corresponding to the historical washing parameter set is executed.

[0095] Specifically, for each historical washing parameter set, an actual result parameter set when the washing machine executes the corresponding historical washing task is determined, and the first reward value corresponding to the historical washing parameter set is determined according to the actual result parameter set.

[0096] Optionally, the set of actual result parameters can include, but is not limited to, the following parameters: total water consumption, total power consumption, total time length, and washing degree, etc. Then, each parameter included in the set of actual result parameters can be substituted into a preset reward function to obtain a first reward value corresponding to the set of historical washing parameters. The reward function can be seen from the following formula (I):

[0097] R = a * (-Power) + b * (-Water) + c * (-Time) + d * (Washability) Formula (I)

[0098] wherein R is the first reward value corresponding to the set of historical washing parameters, Power is the total power consumption, a is the weight corresponding to Power, Water is the total water consumption, b is the weight corresponding to Water, Time is the total time length, c is the weight corresponding to Time, Washability is the washing degree, and d is the weight corresponding to Washability.

[0099] wherein the four parameters Power, Water, Time, and Washability can be normalized, that is, the values of the four parameters are changed to decimals between 0 and 1 by calculation. Since Power, Water, and Time need to be minimized, the three parameters are required to be negative numbers in advance.

[0100] Optionally, the washing degree of each historical washing task can be determined in the following manner: in the case that the historical washing task has been completed, a prompt message can be output through a visual interface, which is used to prompt the user to evaluate the washing degree of the historical washing task. For example, the user can determine whether there is a task to be evaluated when performing the washing task, and if there is a task to be evaluated, the user can evaluate the task to be evaluated.

[0101] Then, the washing degree input by the user according to the prompt message can be received, and the washing degree input by the user is determined as the washing degree of the historical washing task. For example, the prompt message can set three dimensions: clean, general, and not clean, and the corresponding scores are 1.0, 0.8, and 0.6, respectively. Then, if the user evaluates the washing degree of the historical washing task as clean, the corresponding score of 1.0 can be obtained according to the washing degree input by the user, and the score of 1.0 is determined as the washing degree of the historical washing task.

[0102] Step 202, inputting the set of current washing parameters, at least one set of historical washing parameters, and at least one first reward value into the trained reinforcement learning model to obtain a first washing strategy.

[0103] In step 203, simulation processing is performed using the first washing strategy, and a target washing strategy is obtained according to a simulation processing result.

[0104] The following describes steps 202 and 203:

[0105] The above reinforcement learning model can perform reinforcement learning on the input current washing parameter set, at least one historical washing parameter set, and at least one first reward value, so as to obtain an initial training washing strategy (hereinafter referred to as a first washing strategy for convenience of description).

[0106] The above simulation processing refers to simulation running of a virtual washing machine according to the first washing strategy.

[0107] In the embodiment of the present application, the first washing strategy can be further adjusted according to a simulation running result obtained by simulation running of the virtual washing machine according to the first washing strategy, and finally a target washing strategy is obtained.

[0108] As to how to perform simulation processing using the first washing strategy and obtain the target washing strategy according to the simulation processing result, the following Figure 5 will be described, which will not be described in detail here.

[0109] The technical solution provided by the embodiment of the present application comprises the following steps: determining a first reward value corresponding to each historical washing parameter set, inputting a current washing parameter set, at least one historical washing parameter set, and at least one first reward value into a trained reinforcement learning model to obtain a first washing strategy, performing simulation processing using the first washing strategy, and obtaining a target washing strategy according to a simulation processing result. This technical solution can better explore the user's washing rule and the washing effect of the washing rule corresponding to the washing strategy according to the historical washing parameter set and the reward value corresponding to the historical washing parameter set by inputting the current washing parameter set, the historical washing parameter set, and the first reward value corresponding to the historical washing parameter set into the reinforcement learning model for reinforcement learning, so as to obtain the first washing strategy. Then, simulation processing of the first washing strategy can simulate the running result of the first washing strategy and constantly improve the first washing strategy, so as to obtain an optimal target washing strategy, and the water saving, power saving, time saving, and washing target in the washing process are achieved.

[0110] Referring to Figure 3 , an embodiment flowchart of another washing control method provided by the embodiment of the present application is shown. Figure 3 The following describes how to perform simulation processing using the first washing strategy and obtain the target washing strategy according to the simulation processing result on the basis of the flowchart shown in Figure 2 . As shown in Figure 3 , the flowchart can comprise the following steps:

[0111] Step 301, control the virtual washing machine to perform simulation operation according to the current first washing strategy, and obtain an actual result parameter set.

[0112] The actual result parameter set can include actual result parameters obtained after the virtual washing machine performs simulation operation. The actual result parameter set can include, but is not limited to, the following parameters: total water consumption, total power consumption, total time length, and washing degree.

[0113] The virtual washing machine can be a virtual washing machine constructed by simulation software (for example, Matlab).

[0114] As an optional implementation manner, since the simulation software generally consumes more resources (which can include computing resources and storage space), the simulation software can be placed on a remote cloud server, and the washing machine interacts with the remote virtual machine through network communication, such as WiFi (wireless network communication technology), 4G (the 4th generation mobile communication technology), and the like.

[0115] As another optional implementation manner, a program with simulation function can be installed in the washing machine as a virtual washing machine.

[0116] Step 302, determine a second reward value corresponding to the current first washing strategy according to the actual result parameter set.

[0117] The detailed process of determining the second reward value corresponding to the current first washing strategy according to the actual result parameter set can refer to the description in step 201, which will not be repeated here.

[0118] Step 303, input the second reward value and the current washing parameter set into the reinforcement learning model, and obtain the first washing strategy of this iteration.

[0119] In an embodiment, referring to Figure 4 , a reinforcement learning schematic diagram provided by an embodiment of the present application. As shown in Figure 4 , after receiving St and Rt of the washing machine and related environment input, the reinforcement learning intelligent agent (that is, the reinforcement learning model) can generate corresponding At.

[0120] Wherein, St refers to the state S generated by the environment at t moment, S mainly refers to the washing task, including washing time, washing mode, user set washing time, set water quantity and the like, that is, the current washing parameter set in the embodiment of the application; Rt refers to the reward value corresponding to the completion of the current washing task, that is, the second reward value in the embodiment of the application; At refers to the washing control strategy generated by the reinforcement learning agent, that is, the first washing strategy in the embodiment of the application; Correspondingly, St+1 is the washing parameter set corresponding to the next washing task, and Rt+1 is the reward value corresponding to the next washing task.

[0121] Wherein, the above-mentioned reinforcement learning agent refers to a program for generating a washing control strategy by using a reinforcement learning algorithm, and the carrier thereof is a reinforcement learning intelligent chip, which can be installed in the washing machine. The above-mentioned washing machine and related environment refer to the washing machine, user, electricity meter, water meter and the like related to washing, which can be collectively referred to as the environment.

[0122] Optionally, the above-mentioned reinforcement learning algorithm can refer to Figure 5 As shown in Figure 5 , it is a reinforcement learning Actor (actioner)-Critic (commenter) algorithm network diagram provided by the embodiment of the application. Wherein, Agent (agent) is the above-mentioned reinforcement learning agent; Washer (washing machine) is the washing machine; WasherSimulator (washing machine simulator) is a virtual washing machine constructed by simulation software (for example, Matlab); π(a|s) is the probability of generating action a under the current state s, which can be called strategy; Actor network is the network used to generate strategy in the deep reinforcement learning algorithm; Crititc network is the network used to generate value function to evaluate the good and bad of Actor strategy in the deep reinforcement learning algorithm; ▽wJ(w) is the gradient of strategy π, which updates the Actor network in the form of minimize (gradient descent); L (φ) is the loss function of the value function generated by the Critic network, which updates the Critic network in the form of backpropagation (backpropagation).

[0123] Step 304, determine whether the preset iteration stop condition is met at present, if yes, execute step 305; if not, execute step 301 by taking the first washing strategy obtained in this iteration as the current first washing strategy.

[0124] Step 305, after the iteration is ended, the first washing strategy obtained in the last iteration is determined as the target washing strategy.

[0125] The following uniformly describes step 304 and step 305:

[0126] The preset iteration stopping condition can include that the iteration number reaches a preset number threshold (for example, 1000 times), or the sum of the second reward value obtained in the current iteration process and the second reward value obtained in the historical iteration process reaches a preset reward value threshold.

[0127] In an embodiment, after completing the current iteration process, it can be determined whether the preset iteration stopping condition is met at present. If the preset iteration stopping condition is met at present, the current iteration is ended, and the first washing strategy obtained in the last iteration is determined as the target washing strategy; if the preset iteration stopping condition is not met at present, the first washing strategy obtained in the current iteration process is taken as the current first washing strategy, and the step of returning to perform control on the virtual washing machine to simulate running according to the current first washing strategy to obtain the actual result parameter set is performed until the preset iteration stopping condition is met.

[0128] The technical scheme provided by the embodiment of the present application determines the second reward value corresponding to the current first washing strategy according to the actual result parameter set, inputs the second reward value and the current washing parameter set into the reinforcement learning model to obtain the first washing strategy in the current iteration, and in the case that it is determined that the preset iteration stopping condition is met at present, the iteration is ended, and the first washing strategy obtained in the last iteration is determined as the target washing strategy. This technical scheme further simulates running according to the current first washing strategy, and iteratively processes the first washing strategy according to the simulation running result, and the optimal target washing strategy can be obtained by continuously adjusting the first washing strategy, so that the target of water saving, power saving, time saving and washing is achieved in the washing process.

[0129] Referring to Figure 6 A reinforcement learning process schematic diagram provided by the embodiment of the present application is shown. Figure 6 The process shown takes the historical washing tasks including the previous washing task and the previous week washing task as an example to illustrate the washing control method provided by the embodiment of the present application. As shown in Figure 6 As shown in the figure, the process can include the following contents:

[0130] Firstly, the execution subject of the embodiment of the present application obtains the washing parameter set related to the current washing task, the previous washing task and the previous week washing task through the environment (washing machine, user, electric meter, water meter, etc. related to washing) where the washing task is located. The washing parameter set can include but is not limited to the mode (clothes material / weight, etc.) set by the user, the washing time set by the user, the water amount set by the user, etc.

[0131] Afterwards, the above washing parameter set is input into the agent (i.e., the reinforcement learning model), which can generate an action A (i.e., a target washing strategy) according to the above washing parameter set. The action A can include, but is not limited to, the following washing stages: soaking, rinsing, dehydration, etc. The parameters of each stage can include, but are not limited to, the time length, the rotation speed, the temperature, the water consumption, and the power consumption, etc.

[0132] Afterwards, the washing machine is controlled to run the washing task according to the above target washing strategy, and the reward R corresponding to the washing task can be obtained. The reward R can be calculated by the total time length, the total water consumption, the total power consumption, and the washing degree of the current washing task. The washing degree can be input by the user through the control panel or the mobile phone APP.

[0133] Finally, the reinforcement learning of the next state can be performed.

[0134] Referring to Figure 7 Another embodiment flowchart of a washing control method provided by the embodiment of the present application is provided. As shown in the figure, Figure 7 The flowchart can include the following stages: a washing task construction stage, a washing strategy generation stage, and a result evaluation and storage stage.

[0135] Firstly, the washing task construction stage: after the washing machine is powered on and water is supplied, it is in a standby state. When the user puts in the clothes, selects the mode, sets the time length and water consumption, a washing task T is constructed. Before starting the washing, if the washing degree of the last washing task has not been evaluated, the user needs to evaluate it before starting the current washing task. The deep reinforcement learning chip connected to the main control board in the washing machine obtains the current washing task T and the last washing task T1, the last week's washing task T7. If T1 or T7 does not exist, the current task is used instead. The reason for introducing the last washing task and the last week's washing task is that for a certain user, the number of family members and the amount of clothes are basically unchanged, and the behavior pattern of washing clothes has certain regularity, such as washing clothes every night, and the types and amounts of clothes washed on weekdays and weekends are different. Therefore, considering the last washing task and the last week's washing task can better explore the user's washing habits and be more conducive to forming the optimal target washing strategy.

[0136] Then, the washing strategy generation stage: after the agent obtains the washing tasks T, T1 and T7, it uses Figure 5The illustrated deep reinforcement learning Actor-Critic algorithm (or other deep reinforcement learning algorithms such as DQN / A3C / PPO / SAC, etc.) interacts with the virtual simulation WasherSimulator. Since Actor-Critic uses neural networks for policy updating and iteration, multiple interactions are required. When the number of interactions reaches the upper limit, such as 10,000 times, or the cumulative reward G(t) = ∑R(t) reaches a certain threshold M (obtained through experiments), the final policy π* is obtained. The policy, which controls some parameters in each stage of the washing process, such as the length of the soaking, rinsing, and dehydration stages, speed, temperature, water consumption, and electricity consumption, is sent to the real washing machine Washer as the final control instruction. Washer completes the current washing process according to the instruction.

[0137] It should be noted that since the simulation program WasherSimulator generally consumes resources (computing resources and storage space), it can be placed on a remote cloud server, and the washing machine interacts with the remote WasherSimulator through network communication methods such as WiFi, 4G, etc. Alternatively, for high-end washing machines, a program with simulation function can be developed as WasherSimulator.

[0138] Finally, the result evaluation and storage stage: after the washing task is completed, the total water consumption, electricity consumption, and time length can be easily obtained, but the cleaning degree is difficult to evaluate. Therefore, this scheme proposes to set a cleaning evaluation button on the control panel of the washing machine (or through a mobile phone App) to obtain the real evaluation of the washing task. Thereafter, the agent combines the state S generated by the washing task, the action A represented by the generated washing strategy, and the reward R into an experience and stores it in the reinforcement learning chip for next use. The reward function R is shown in formula (I) above.

[0139] Figure 8 An embodiment block diagram of a washing control device provided by an embodiment of the present application is shown in FIG. 1. Figure 8 As shown in the figure, the device comprises:

[0140] The acquisition module 81 is configured to acquire a current washing parameter set of the washing task and acquire a historical washing parameter set of at least one historical washing task when the washing task is received.

[0141] The learning module 82 is configured to perform reinforcement learning on the current washing parameter set and at least one historical washing parameter set by using the trained reinforcement learning model to obtain a target washing strategy.

[0142] The execution module 83 is configured to control the washing machine to perform the washing task according to the target washing strategy.

[0143] In a possible implementation, the acquisition module 81 is specifically configured to:

[0144] acquire a set of historical washing parameters of a historical washing task that is most recently executed by the washing machine, and / or acquire a set of historical washing parameters of a historical washing task that is executed within a set time period and most recently set by the washing machine.

[0145] In a possible implementation, the learning module 82 includes:

[0146] a determination sub-module configured to determine a first reward value corresponding to each of the set of historical washing parameters;

[0147] a training sub-module configured to input the set of current washing parameters, at least one of the set of historical washing parameters, and at least one of the first reward values into the trained reinforcement learning model to obtain a first washing strategy;

[0148] a processing sub-module configured to perform simulation processing by using the first washing strategy, and obtain the target washing strategy according to a simulation processing result.

[0149] In a possible implementation, the determination sub-module includes:

[0150] a first determination unit configured to determine, for each of the set of historical washing parameters, a set of actual result parameters of the washing machine when performing a corresponding historical washing task;

[0151] a second determination unit configured to determine, according to the set of actual result parameters, a first reward value corresponding to the set of historical washing parameters.

[0152] In a possible implementation, the set of actual result parameters includes one or more of the following parameters: total water consumption, total power consumption, total duration, and washing degree.

[0153] The second determination unit is specifically configured to:

[0154] substitute each parameter included in the set of actual result parameters into a preset reward function to obtain the first reward value corresponding to the set of historical washing parameters.

[0155] In a possible implementation, the washing degree of each of the historical washing tasks is determined by the following method:

[0156] In a case where the historical washing task has been executed, a visual interface is used to output a prompt message, and the prompt message is used to prompt a user to evaluate the washing degree of the historical washing task.

[0157] receive a washing degree input by a user according to the prompt message, and determine the washing degree input by the user as the washing degree of the historical washing task.

[0158] In a possible implementation, the processing submodule is specifically configured to:

[0159] iteratively processing by using the following steps until a preset iteration stop condition is met:

[0160] controlling the virtual washing machine to perform simulation operation according to the current first washing strategy, to obtain an actual result parameter set;

[0161] determining a second reward value corresponding to the current first washing strategy according to the actual result parameter set;

[0162] inputting the second reward value and the current washing parameter set into the reinforcement learning model to obtain the first washing strategy of this iteration;

[0163] After the iteration is completed, the first washing strategy obtained in the last iteration is determined as the target washing strategy.

[0164] In a possible implementation, the iteration stop condition includes that the number of iterations reaches a preset number threshold, or the sum of the second reward value obtained in the current iteration process and the second reward value obtained in a historical iteration process reaches a preset reward value threshold.

[0165] Figure 9 a structural schematic diagram of a washing machine provided in an embodiment of the present application, Figure 9 The electronic device 900 shown includes at least one processor 901, a memory 902, at least one network interface 904, and a user interface 903. The various components in the electronic device 900 are coupled together by a bus system 905. It can be understood that the bus system 905 is used to realize the connection and communication between the components. In addition to including a data bus, the bus system 905 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, all the buses are marked as the bus system 905 in the Figure 9

[0166] The user interface 903 can include a display, a keyboard, or a clicking device (for example, a mouse, a trackball, a touchpad, or a touch screen, etc.).

[0167] ​It is to be appreciated that the memory 902 in the embodiments of the present application can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 902 described herein is intended to include, without being limited to, these and any other suitable types of memory.

[0168] In some embodiments, the memory 902 stores the following elements, executable units or data structures, or a subset of them, or an extended set of them: an operating system 9021 and application programs 9022.

[0169] Among them, the operating system 9021 contains various system programs, such as framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application programs 9022 contain various application programs, such as Media Player, Browser, etc., for implementing various application services. The programs for implementing the method embodiments of the present application can be contained in the application programs 9022.

[0170] In the embodiments of the present application, by calling the programs or instructions stored in the memory 902, specifically, the programs or instructions stored in the application programs 9022, the processor 901 is used to execute the method steps provided by each method embodiment, for example, including:

[0171] When receiving the washing task, a current washing parameter set of the washing task is acquired, and a historical washing parameter set of at least one historical washing task is acquired;

[0172] The current washing parameter set and the at least one historical washing parameter set are subjected to reinforcement learning by using the trained reinforcement learning model, to obtain a target washing strategy;

[0173] The washing machine is controlled to perform the washing task according to the target washing strategy.

[0174] The method disclosed in the embodiments of the present application can be applied to the processor 901 or implemented by the processor 901. The processor 901 can be an integrated circuit chip having a signal processing capability. In the implementation process, the steps of the above method can be completed by hardware integrated logic circuits or software form instructions in the processor 901. The processor 901 described above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software units in the decoding processor for execution. The software unit can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory 902, and the processor 901 reads the information in the memory 902, and combines the hardware to complete the steps of the above method.

[0175] It can be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing units can be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSP Devices, DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof.

[0176] For software implementation, the techniques described herein can be implemented with a processing unit that executes program code that includes functions described herein. The software code can be stored in memory and executed by a processor. Memory can be implemented within the processor or external to the processor.

[0177] The washing machine provided by the embodiments can be a washing machine as shown in Figure 9 , can perform all steps of the washing control method as Figures 1-7 , and thus achieve the technical effects of the washing control method as shown in Figures 1-7 . For details, please refer to the relevant description, which will not be repeated here for brevity. Figures 1-7

[0178] The embodiments of the present application also provide a storage medium (computer readable storage medium). The storage medium herein stores one or more programs. Wherein, the storage medium can include volatile memory, such as random access memory; the storage medium can also include non-volatile memory, such as read only memory, flash memory, hard disk or solid state disk; the storage medium can also include a combination of the above kinds of memory.

[0179] When the one or more programs in the storage medium can be executed by one or more processors to implement the washing control method described above, which is executed on the washing machine device side.

[0180] The processor is configured to execute the washing control program stored in the memory to implement the following steps of the washing control method executed on the washing control device side:

[0181] Upon receiving a washing task, a current washing parameter set of the washing task is obtained, and a historical washing parameter set of at least one historical washing task is obtained.

[0182] ​perform reinforcement learning on the current set of washing parameters and at least one of the historical sets of washing parameters using the trained reinforcement learning model to obtain a target washing strategy;

[0183] controlling the washing machine to perform a washing task according to the target washing strategy.

[0184] Those skilled in the art will further appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or both. The disclosure is not limited to any particular hardware or software configuration; and the specific configuration will depend on the specific application and design constraints imposed on the particular implementation. Skilled persons can use various methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present disclosure.

[0185] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), non-volatile memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0186] The above detailed description has further explained the purpose, technical solutions and advantages of the present disclosure. It should be understood that the above description is only a specific implementation of the present disclosure and is not intended to limit the protection scope of the present disclosure. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A washing control method, characterized in that, The method includes: Upon receiving a washing task, the current washing parameter set of the washing task is obtained, as well as the historical washing parameter set of at least one historical washing task is obtained. The target washing strategy is obtained by performing reinforcement learning on the current washing parameter set and at least one of the historical washing parameter sets using a trained reinforcement learning model. Control the washing machine to perform the washing task according to the target washing strategy; The step of using a trained reinforcement learning model to perform reinforcement learning on the current washing parameter set and at least one of the historical washing parameter sets to obtain a target washing strategy includes: determining a first reward value corresponding to each of the historical washing parameter sets; inputting the current washing parameter set, at least one of the historical washing parameter sets, and at least one of the first reward values ​​into the trained reinforcement learning model to obtain a first washing strategy; performing simulation processing using the first washing strategy, and obtaining the target washing strategy based on the simulation processing results. The step of determining the first reward value corresponding to each of the historical washing parameter sets includes: for each historical washing parameter set, determining the actual result parameter set when the washing machine performs the corresponding historical washing task; and determining the first reward value corresponding to the historical washing parameter set based on the actual result parameter set. The actual result parameter set includes one or more of the following parameters: total water consumption, total electricity consumption, total duration, and washing cleanliness; determining the first reward value corresponding to the historical washing parameter set based on the actual result parameter set includes: Substitute each parameter in the actual result parameter set into a preset reward function to obtain the first reward value corresponding to the historical washing parameter set.

2. The method according to claim 1, characterized in that, The process of obtaining a set of historical washing parameters for at least one historical washing task includes: Obtain the set of historical washing parameters for the most recent historical washing task executed by the washing machine, and / or obtain the set of historical washing parameters for the historical washing tasks executed by the washing machine within the most recent set time period.

3. The method according to claim 1, characterized in that, The cleanliness level of each historical washing task was determined in the following way: When the historical washing task has been completed, a prompt message is output through a visual interface, which is used to prompt the user to evaluate the cleaning level of the historical washing task. The system receives the cleaning level input by the user based on the prompt message and determines the cleaning level input by the user as the cleaning level of the historical washing task.

4. The method according to claim 1, characterized in that, The step of performing simulation processing using the first washing strategy and obtaining the target washing strategy based on the simulation results includes: The following steps are used to perform iterative processing until the preset iteration stopping condition is met: The virtual washing machine is controlled to run in simulation according to the current first washing strategy, and the actual result parameter set is obtained. The second reward value corresponding to the current first washing strategy is determined based on the set of actual result parameters. The second reward value and the current set of washing parameters are input into the reinforcement learning model to obtain the first washing strategy for this iteration. After the iteration is completed, the first washing strategy obtained in the last iteration is determined as the target washing strategy.

5. The method according to claim 4, characterized in that, The iteration stopping conditions include: the number of iterations reaches a preset threshold, or the sum of the second reward value obtained in the current iteration and the second reward value obtained in the previous iteration reaches a preset reward value threshold.

6. A washing control device, characterized in that, The device includes: The acquisition module is used to acquire the current set of washing parameters for the washing task and the historical set of washing parameters for at least one historical washing task when a washing task is received. The learning module is used to perform reinforcement learning on the current washing parameter set and at least one of the historical washing parameter sets using a trained reinforcement learning model to obtain the target washing strategy. The execution module is used to control the washing machine to perform washing tasks according to the target washing strategy; The step of using a trained reinforcement learning model to perform reinforcement learning on the current washing parameter set and at least one of the historical washing parameter sets to obtain a target washing strategy includes: determining a first reward value corresponding to each of the historical washing parameter sets; inputting the current washing parameter set, at least one of the historical washing parameter sets, and at least one of the first reward values ​​into the trained reinforcement learning model to obtain a first washing strategy; performing simulation processing using the first washing strategy, and obtaining the target washing strategy based on the simulation processing results. The step of determining the first reward value corresponding to each of the historical washing parameter sets includes: for each historical washing parameter set, determining the actual result parameter set when the washing machine performs the corresponding historical washing task; and determining the first reward value corresponding to the historical washing parameter set based on the actual result parameter set. The actual result parameter set includes one or more of the following parameters: total water consumption, total electricity consumption, total duration, and washing cleanliness; determining the first reward value corresponding to the historical washing parameter set based on the actual result parameter set includes: Substitute each parameter in the actual result parameter set into a preset reward function to obtain the first reward value corresponding to the historical washing parameter set.

7. A washing machine, characterized in that, include: A processor and a memory, the processor being configured to execute a washing control program stored in the memory to implement the washing control method according to any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the washing control method according to any one of claims 1 to 5.

Images