Training method and application method of smart home device and control model thereof

Abstract

The present application relates to the technical field of smart home devices, and discloses a training method and application method of a smart home device and its control model. The training method includes: responding to task instructions to obtain the current sensor data of the smart home device; based on the sensor data, the control model controls the smart home device to perform corresponding actions; the control model performs behavioral learning according to the performance feedback of the smart home device in performing the action, so as to automatically train the control model until the control model is guided to control the smart home device to complete the task corresponding to the task instruction and achieve the task goal corresponding to the task. The present application increases the autonomous decision-making ability of the smart home device by automatically training the control model, so that the control model can control the smart home device end-to-end to complete complex tasks.

Description

Technical Field

[0001] The present invention relates to the field of smart home devices, and in particular to a training method and an application method of a smart home device and a control model thereof. Background Art

[0002] In the field of smart home, manually written algorithms and rules are usually used to control smart home devices so that smart home devices can complete work tasks. However, the existing manually written algorithms and rules cannot flexibly adapt to complex and changing environments, making the smart home devices not very intelligent. Summary of the invention

[0003] In view of this, the present application proposes a training method and an application method for a smart home device and its control model, which automatically trains the model to effectively increase the autonomous decision-making ability of the smart home device, so that the smart home device can complete complex tasks autonomously and flexibly adapt to complex and changing environments.

[0004] In a first aspect, an embodiment of the present application provides a training method for a control model of a smart home device, comprising:

[0005] Respond to task instructions to obtain the current sensor data of smart home devices;

[0006] Based on the self-sensor data, the control model controls the smart home device to perform corresponding actions;

[0007] The control model performs behavioral learning based on the performance feedback of the smart home device executing the action, so as to automatically train the control model until the control model is guided to control the smart home device to complete the task corresponding to the task instruction and achieve the task goal corresponding to the task.

[0008] In some embodiments, based on the sensor data, the control model controls the smart home device to perform a corresponding action, including:

[0009] The control model determines the executable action corresponding to the current self-sensor data according to the pre-constructed correspondence between the self-sensor data and the executable action, so that the control model drives the actuator of the smart home device to execute the executable action.

[0010] In some embodiments, the control model performs behavior learning based on the performance feedback of the smart home device executing the action, including:

[0011] The performance feedback of the smart home device executing the action is obtained by using a behavior evaluation rule or a behavior evaluation model and sent to the control model so that the control model can perform behavior learning.

[0012] In some embodiments, the using of a behavior evaluation rule or a behavior evaluation model to obtain performance feedback of the smart home device executing the action and sending the feedback to the control model includes:

[0013] According to the self-sensor data and / or external sensor data updated after the smart home device executes the action, the performance of executing the action is evaluated using the behavior evaluation rule or the behavior evaluation model, and the performance feedback is sent to the control model.

[0014] In some embodiments, the using the behavior evaluation rule or the behavior evaluation model to evaluate the performance of executing the action and sending the performance feedback to the control model includes:

[0015] If the performance of the action is an abnormal performance included in the behavior evaluation rule or the behavior evaluation model, sending negative feedback and behavior correction information corresponding to the abnormal performance to the control model;

[0016] Accordingly, the controlling model is made to perform behavior learning, including:

[0017] The control model adjusts the action being executed or to be executed by the smart home device according to the behavior correction information, and updates the corresponding relationship between the self sensor data and the executable action.

[0018] In some embodiments, the behavior evaluation rules pre-construct a correspondence between the abnormal performance, negative feedback, and the behavior correction information corresponding to the abnormal performance; sending the negative feedback and the behavior correction information corresponding to the abnormal performance to the control model includes: sending the negative feedback and the behavior correction information corresponding to the abnormal performance to the control model according to the correspondence in the behavior evaluation rules.

[0019] In some embodiments, the behavior evaluation model is a large language model, and the sending of negative feedback and the behavior correction information corresponding to the abnormal performance to the control model includes: the large language model outputs semantic text information to the control model based on the input abnormal performance, and the semantic text information includes the negative feedback and the behavior correction information corresponding to the abnormal performance.

[0020] In some embodiments, the using the behavior evaluation rule or the behavior evaluation model to evaluate the performance of executing the action and sending the performance feedback to the control model includes:

[0021] If the performance of the action is a good performance included in the behavior evaluation rule or the behavior evaluation model, sending positive feedback to the control model;

[0022] Accordingly, the controlling model is made to perform behavior learning, including:

[0023] The control model records the action as a reference for subsequent action execution of the task, and updates the corresponding relationship between the self-sensor data and the executable action.

[0024] In some embodiments, the step of guiding the smart home device to complete the task corresponding to the task instruction and achieve the task goal corresponding to the task includes:

[0025] After completing this task, if the task target corresponding to the task is not achieved, the smart home device is controlled to move to the starting position of the task, and the process returns to the step of obtaining the current sensor data of the smart home device until the task target is achieved.

[0026] In some embodiments, the task objective is determined according to the execution configuration requirements of the task;

[0027] The execution configuration requirements include: one or more combinations of task execution time lower than a preset time, task execution coverage higher than a preset threshold, and the number of abnormal performance occurrences lower than a preset number.

[0028] In a second aspect, an embodiment of the present application provides an application method of a control model of a smart home device, including:

[0029] In response to a task instruction issued by a user, the control model controls the smart home device to perform a task corresponding to the task instruction based on the task instruction;

[0030] After the task is performed, receiving user feedback information, so that the control model adjusts the action or the combination of actions when the task is performed next time according to the user feedback information;

[0031] Wherein, the control model is obtained by training based on the training method of the control model of the smart home device.

[0032] In some embodiments, the control model adjusts the action or combination of actions when performing the task next time according to the user feedback information, including:

[0033] Inputting the user feedback information into a large language model;

[0034] If the user feedback information includes negative feedback, the large language model outputs semantic text information to the control model; the semantic text information includes negative feedback and behavior correction information;

[0035] The control model adjusts the action or the combination of actions when the task is performed next time according to the negative feedback and the behavior correction information.

[0036] In a third aspect, an embodiment of the present application provides a smart home device, including:

[0037] An actuator is used to execute corresponding actions;

[0038] a memory storing a computer program;

[0039] A controller is used to execute the computer program to implement the training method or application method of the control model of the smart home device.

[0040] In a fourth aspect, an embodiment of the present application proposes a computer-readable storage medium storing a computer program. When the computer program is executed on a processor, the training method or application method of the control model of the smart home device is implemented.

[0041] The embodiments of the present application have the following advantages:

[0042] The training method of the control model of the smart home device proposed in the embodiment of the present application includes: the smart home device responds after receiving the task instruction, first obtains the current sensor data of itself, and based on the sensor data of itself, the control model controls the smart home device to perform the corresponding action in an end-to-end manner; then, the control model performs behavior learning according to the performance feedback of the smart home device in performing the action, so as to perform automatic training, until the control model is guided to control the smart home device to complete the task corresponding to the task instruction and achieve the task goal corresponding to the task. This training method automatically trains the control model with the goal of completing the task and achieving the task goal as the guide, so that the smart home device will not perform the task according to fixed algorithms and rules, effectively increasing the autonomous decision-making ability of the smart home device, so that the smart home device can autonomously complete complex tasks through the control model, and flexibly adapt to complex and changing environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solution of the present invention, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope of protection of the present invention. In each of the drawings, similar components are numbered similarly.

[0044] Figure 1A schematic diagram of the structure of a smart home device according to an embodiment of the present application is shown;

[0045] Figure 2 A first flow chart showing a method for training a control model of a smart home device according to an embodiment of the present application is shown;

[0046] Figure 3 A second flow chart showing a method for training a control model of a smart home device according to an embodiment of the present application;

[0047] Figure 4 A third flow chart showing a method for training a control model of a smart home device according to an embodiment of the present application;

[0048] Figure 5 A first flow chart showing an application method of a control model for a smart home device according to an embodiment of the present application;

[0049] Figure 6 A structural schematic diagram of a training device for a control model of a smart home device according to an embodiment of the present application is shown;

[0050] Figure 7 A structural schematic diagram of an application device of a control model of a smart home device in an embodiment of the present application is shown.

[0051] Description of main component symbols:

[0052] 10-smart home device; 11-processor; 12-memory; 13-actuator; 14-sensing unit;

[0053] 100 - training device for control model; 110 - response module; 120 - execution module; 130 - model training module;

[0054] 200 - application device of control model; 210 - task execution module; 220 - feedback adjustment module. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0056] The components of the embodiments of the present invention generally described and shown in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0057] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present invention, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or adding the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0058] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0059] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present invention belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present invention.

[0060] Example 1

[0061] Figure 1 Shown is a structural schematic diagram of a smart home device 10 proposed in an embodiment of the present application.

[0062] Exemplarily, the smart home device 10 includes a processor 11, a memory 12 and an actuator 13, wherein the actuator 13 is used to execute corresponding actions; the memory 12 stores a computer program, and the processor 11 runs the computer program, so that the smart home device 10 executes the training method or application method of the control model of the following embodiment.

[0063] Among them, the processor 11 can be an integrated circuit chip with signal processing capabilities. The processor 11 can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or at least one of other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0064] The memory 12 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory 12 is used to store a computer program, and the processor 11 may execute the computer program accordingly after receiving an execution instruction.

[0065] Among them, the actuator 13 is mainly used to enable the smart home device 10 to perform corresponding actions or tasks. For example, for a sweeper, its actuator 13 includes a cleaning mechanism and a walking mechanism. For example, the cleaning mechanism may include but is not limited to a spray washing component, a dirt removal component, etc., and the walking mechanism may include but is not limited to side wheels, track wheels, tracks, rollers, etc. For another example, for a food delivery robot, its actuator 13 includes a lifting component, or a dual robotic arm, etc. It can be understood that the specific structure of the actuator 13 can be determined according to the actual needs of the smart home device 10, and no further description is given here.

[0066] Furthermore, in order to realize the intelligence of the smart home device 10, the smart home device 10 also includes a sensing unit 14 arranged on the smart home device 10, and the sensing unit 14 is used to transmit necessary internal and external state information to the smart home device 10, so that the smart home device 10 performs corresponding actions and completes the corresponding tasks.

[0067] For example, if divided according to the sensing range, the sensing unit 14 includes an internal sensing unit and an external sensing unit. Among them, the internal sensing unit is mainly used for the smart home device 10 to collect the status information of the device itself, such as an encoder, a motor current sensor, an inertial measurement unit (IMU), etc. The external sensing unit is mainly used for the smart home device 10 to collect the environment information in which it is located, such as a ranging sensor, an ultrasonic sensor, a visual device, etc. It can be understood that for different smart home devices 10, the sensing unit 14 can be adaptively set according to the actual function, which is not limited here.

[0068] It should be understood that the above-mentioned smart home device 10 may include, but is not limited to, a sweeping machine (also called a cleaning robot), a mopping robot driven by a sweeping machine (i.e., a sweeping and mopping robot), a food delivery robot, an autonomously driven carrying robot, a companion robot, etc. It is understood that the existence forms of the various robots listed above are not limited, for example, they may be wheeled robots, bipedal humanoid robots, or multi-legged robots, etc.

[0069] Example 2

[0070] In order to improve the intelligence level of the smart home device 10 in the above embodiment, the present application proposes to increase the autonomous decision-making ability of the smart home device 10 by automatically training the control model in the smart home device 10, so that the control model can control the smart home device 10 end-to-end to complete complex tasks.

[0071] It should be understood that the control model is a machine learning model that can run in the processor 11. By automatically training it to learn the mapping relationship from input to output, the control model can have end-to-end control capabilities. There is no limitation here on which models can be used to construct the machine learning model.

[0072] Among them, end-to-end means that the control model can directly control the actuator 13 according to the input of the perception unit 14, and no step-by-step processing is required in the middle. For the above control model, on the one hand, since the control model can directly generate control output from the perception input, the processing time is reduced, thereby improving the response speed. On the other hand, since the intermediate step-by-step processing is reduced, the system design is also simplified.

[0073] The following is a detailed description of the training method of the control model of the smart home device 10 proposed in the embodiment of the present application.

[0074] Figure 2 The figure shows a flow chart of a method for training a control model of a smart home device 10 according to an embodiment of the present application.

[0075] Exemplarily, the training method of the control model of the smart home device 10 includes steps S110-S130.

[0076] S110, responding to the task instruction to obtain the current sensor data of the smart home device.

[0077] Among them, the task instruction can be sent to the smart home device 10 by an external object (such as a user) through a terminal device (such as a mobile phone, a tablet, a smart speaker, etc.) or a button on the smart home device 10.

[0078] Different smart home devices 10 often have different task instructions. For example, if the smart home device 10 is a sweeper, the task instructions may include, but are not limited to, cleaning the entire room, cleaning along the edges, or cleaning a designated area; if the smart home device 10 is a sweeping and mopping robot, the task instructions may include, but are not limited to, finding and moving household items.

[0079] Exemplarily, when the smart home device 10 receives the task instruction, it responds to the task instruction. First, the smart home device 10 obtains the current state of the device and / or the current environment information through the sensing unit 14.

[0080] For example, taking a sweeping robot as an example, it can collect visual images of the current environment through its own visual sensors (such as monocular or binocular cameras, etc.), collect whether there are obstacles in front through its own collision sensors (such as ultrasonic, infrared sensors, etc.), collect point cloud data of obstacles in the current environment through its own lidar, collect current data of each motor in the device through its own current sensor, and collect device posture information through its own posture sensor (such as IMU, etc.).

[0081] S120, based on the sensor data, the control model controls the smart home device to perform corresponding actions.

[0082] Among them, the self sensor is the sensor of the sensing unit 14 of the smart home device 10.

[0083] Exemplarily, after acquiring its own sensor data, the control model in the smart home device 10 will determine the executable action corresponding to the current own sensor data according to the pre-built correspondence between the own sensor data and the executable action, so that the control model drives the actuator 13 of the smart home device 10 to execute the above executable action. In other words, the control model determines the current corresponding executable action and tries it by querying the above correspondence according to the current sensor data.

[0084] For example, the control model can determine whether there is a wall, table, chair, etc. in front based on the data from the visual sensor or lidar, and then control the smart home device 10 to try to perform one of the avoidance actions such as deceleration, rotation, and retreat. Of course, it can also be a combination of the above-mentioned multiple actions.

[0085] It can be understood that regarding the correspondence between the sensor data itself and the executable actions, for example, in one embodiment, the control model can determine the corresponding device state or environmental state through the collected sensor data, and determine which actions or action combinations can be executed based on the corresponding device state or environmental state, thereby constructing an input-output relationship between the corresponding sensor data and a single executable action or between the permutations and combinations of executable actions.

[0086] It is understandable that for the smart home device 10, the executable actions are often different in different situations. For example, for a sweeper, the executable actions may include forward, backward, left turn, right turn, stop, deceleration, acceleration, etc. For a food delivery robot, the executable actions may include grabbing, moving up, moving down, translating, turning, etc.

[0087] S130, the control model performs behavior learning according to the performance feedback of the smart home device executing the action, so as to automatically train the control model until the control model is guided to control the smart home device to complete the task corresponding to the task instruction and achieve the task goal corresponding to the task.

[0088] It can be understood that in the present application, since the control model can only control the smart home device to perform corresponding actions when receiving the task instruction, it cannot control the smart home device to complete the task corresponding to the task instruction. Therefore, it is necessary to complete the task and achieve the task goal as the guide, and to automatically train the control model so that the trained control model can control the smart home device 10 to complete the task autonomously and achieve the task goal. There is no need to control the smart home device 10 to complete the task through artificially pre-written algorithms and rules, thereby improving the autonomous decision-making ability of the smart home device 10.

[0089] Exemplarily, after the smart home device 10 completes an action or is performing an action, performance feedback of the smart home device 10 performing the action is obtained, and then the performance feedback is sent to the control model so that the control model can perform behavioral learning, thereby realizing automatic training of the control model, that is, the control model adjusts its behavior after obtaining the performance feedback, so as to autonomously learn how to better complete the assigned task.

[0090] It should be understood that the action in the execution action described in step S130 is an executable action determined by step S120, and therefore, it may be a single action or multiple arranged continuous actions (such as back-rotate-forward, etc.). Based on this, there is no limitation on the timing of evaluating and feeding back the performance feedback of the execution of the action to the control model. For example, the behavior evaluation may be performed while the smart home device 10 is executing the action, and the feedback may be given to the control model in real time so that the control model immediately adjusts the behavior; or the behavior evaluation may be performed after the smart home device 10 has completed the execution of the action, and then the feedback may be given to the control model so that the control model adjusts the next behavior, etc.

[0091] In this application, the task objectives corresponding to each task may be different, which can be determined by the user according to the execution configuration requirements of the current task. For example, the execution configuration requirements include but are not limited to: the task execution time is less than the preset time, the task execution coverage rate is higher than the preset threshold (such as the cleaning area is not less than the preset area threshold, etc.), the number of abnormal performance occurrences is lower than the preset number of times, etc. One or more combinations thereof.

[0092] The implementation process of automatic training of the control model in step S130, for example, in one embodiment, Figure 3 As shown, it includes the following sub-steps:

[0093] S131, using a behavior evaluation rule or a behavior evaluation model to obtain performance feedback of the smart home device executing the action and sending it to the control model.

[0094] Exemplarily, when using behavior evaluation rules or behavior evaluation models for evaluation, the rationality of the smart home device 10 executing the action can be evaluated in combination with the updated self-sensor data and / or external sensor data obtained after the smart home device 10 executes the action, and then the corresponding evaluation feedback information is sent to the control model.

[0095] The external sensor is relative to the internal sensor, and refers to a sensor arranged outside the smart home device 10, such as a sensor arranged in a home area or a test environment, and usually has a certain distance from the smart home device 10. For example, the external sensor can be arranged on a wall, table, or chair in the home area or test environment where the smart home device 10 is located.

[0096] For example, external sensors include, but are not limited to, multiple fixed or steerable following cameras, depth cameras, inductive sensors (such as infrared sensors, photoelectric sensors, ultrasonic sensors), etc., which are arranged in the space around the smart home device 10 (such as above, left side, right side, etc.).

[0097] It should be noted that both the external sensor and the smart home device 10 have a wireless communication module, which may be a Bluetooth module or a WiFi module. The smart home device 10 may obtain external sensor data through wireless communication; the smart home device 10 may obtain external sensor data in real time, or may obtain external sensor data while the smart home device 10 is performing an action or after performing an action.

[0098] It is understandable that as the smart home device 10 performs an action, the data collected by its own sensors may also change accordingly. Therefore, it can be judged in combination with the updated self-sensor data and the existing external sensors. For example, these sensor data can be used to detect whether the smart home device 10 has collided, has an unreasonable posture, is stuck by an obstacle, or takes longer than a preset time to complete a task.

[0099] In one embodiment, the above-mentioned behavior evaluation rules can be obtained by a pre-manually written algorithm, which may include: which own sensor data and / or external sensor data are customized good performances, and the corresponding relationship between good performances and positive feedback; which own sensor data and / or external sensor data are customized abnormal performances, and the corresponding relationship between abnormal performances and negative feedback. Furthermore, for abnormal performances, it can also include behavior correction information corresponding to the abnormal performance. In other words, the behavior evaluation rules also include the correspondence between abnormal performance, negative feedback, and behavior correction information corresponding to the abnormal performance.

[0100] In another embodiment, the above-mentioned behavior evaluation model can be a pre-trained large language model (LLM), which analyzes the input self-sensor data and / or external sensor data information and outputs corresponding text information. It can be understood that the text information can be positive feedback sent to the control model, or negative feedback sent to the control model, and behavior correction information corresponding to the negative feedback, etc. For example, the text message of negative feedback sent by a directional control model may be: "Turn now, it's too far away from the table leg, and the sweeper hasn't swept a lot of places yet!" Correspondingly, the behavior correction information is that when the table leg is detected by its own sensor data, it is necessary to continue moving forward for a distance before rotating, etc., or the text message of negative feedback may be: "The sweeper hit the table leg on the right side. Next time you see a table leg, remember to brake and change direction in advance to avoid collision!" Correspondingly, the behavior correction information is that when the table leg is detected by its own sensor data, it is necessary to rotate and change the direction of travel in advance; for another example, the text message of positive feedback sent by a directional control model may be: "Great! The sweeper did not collide with any obstacles this time, nor was it too far away from the obstacles to cause missed sweeping."

[0101] It can be understood that the above-mentioned behavior evaluation rules or behavior evaluation models can be run outside the smart home device 10 (such as a cloud server, etc.), or can be run on the processor 11 inside the smart home device 10, depending on the main control computing power of the smart home device 10. For example, when the internal main control computing power of the smart home device 10 is insufficient, it can be run outside the smart home device 10 to speed up the model training process.

[0102] It should be noted that the behavior evaluation rules or behavior evaluation models can be adjusted according to the task objectives, so that the control model can perform behavior learning for different task objectives through the behavior evaluation rules or behavior evaluation models, so that the control module can control the smart home device 10 to achieve different task objectives.

[0103] The above-mentioned behavior correction information is used to guide the smart home device 10 to adjust its actions when abnormal performance occurs, so as to guide the control model to control the smart home device 10 to complete the task and achieve the task goal.

[0104] For example, when the smart home device 10 is a sweeping robot, if the abnormal performance is that the sweeping robot collides with a table leg, the behavior correction information may be: when the table leg obstacle is detected, the sweeping robot is controlled to reduce the moving speed, and the sweeping robot is controlled to turn at a preset distance from the table leg.

[0105] Alternatively, if the abnormal behavior is turning at a distance from the table legs, the behavior correction information may be: when moving to a preset distance from the table legs, control the sweeping machine to turn.

[0106] Alternatively, if the abnormal performance is that the time taken by the smart home device 10 to complete the task exceeds the preset time, the behavior correction information may be: adjusting the moving speed of the smart home device 10 according to the time taken to complete the task.

[0107] Therefore, for step S130, if the performance of the action is a good performance included in the above-mentioned behavior evaluation rules or behavior evaluation models, steps S132 and S133 are executed. On the contrary, if the performance of the action is an abnormal performance included in the behavior evaluation rules or behavior evaluation models, steps S134 and S135 are executed.

[0108] S132: If the performance of the action is good, positive feedback is sent to the control model.

[0109] Among them, good behavior can be customized according to the self-sensor data and / or external sensor data, for example, including but not limited to, including, according to the self-sensor data and / or external sensor data, detecting that the smart home device 10 has not collided, has a reasonable walking path, has a reasonable posture, has completed the task within a preset time, etc. If the smart home device 10 is a sweeper, the good performance may also include no missed sweeps, etc.

[0110] S133, the control model records the action as a reference for subsequent action execution of the task, and updates the correspondence between its own sensor data and executable actions.

[0111] It can be understood that if the performance of the action executed by the smart home device 10 is good, the control model will record it, so that when the same situation is encountered when the same task is executed next time, that is, the same self-sensor data is detected, it will be executed according to this action, that is, the behavior of this time is learned. Optionally, the correspondence between the pre-stored self-sensor data and the executable action can be updated according to the currently executed action, that is, the control model is updated.

[0112] It should be noted that before updating the control model, one self-sensor data may correspond to multiple executable actions or action combinations. After updating the control model according to positive feedback, the self-sensor data detected this time corresponds to this action or action combination.

[0113] S134, if the performance of the action is abnormal, negative feedback and behavior correction information corresponding to the abnormal performance are sent to the control model.

[0114] Among them, abnormal performance can also be customized according to the self-sensor data and / or external sensor data, for example, including but not limited to, detecting that the smart home device 10 has collided, has an unreasonable posture, is stuck by an obstacle, takes longer than a preset time to complete a task, etc. according to the self-sensor data and / or external sensor data. If the smart home device 10 is a sweeper, the abnormal performance may also include missed sweeping, etc.

[0115] S135, the control model adjusts the action being executed or to be executed by the smart home device according to the behavior correction information, and updates the correspondence between its own sensor data and executable actions.

[0116] It should be noted that after the correspondence between the pre-stored self-sensor data and the executable actions is updated, that is, after the control model is updated, when the same situation is encountered during the subsequent execution of the same task, that is, the same self-sensor data is detected, the adjusted action will be executed.

[0117] It can be understood that each time the smart home device 10 performs an action, the control model is guided by completing the task and achieving the task goal, and behavioral learning is performed based on the performance feedback obtained each time the action is executed.

[0118] S136, after completing the task, determine whether the task goal corresponding to the task is achieved. If the task goal is achieved, execute step S138 to stop model training. As an optional solution, if the task goal is not achieved, execute step S137.

[0119] S137, if the task target corresponding to the task is not achieved, the smart home device 10 is controlled to move to the starting position of the task execution, and the process returns to the step of obtaining the current sensor data of the smart home device 10, that is, the next round of task execution and automatic training of the control model is continued until the task target is achieved.

[0120] It should be noted that when the next round of task execution and automatic training of the control model is carried out, the control model used is the control model after the previous round of model training.

[0121] S138: If the task goal corresponding to the task is achieved, stop model training.

[0122] In other words, if the corresponding task goal is not achieved after executing a task once, it needs to be executed again. Each execution will use the correspondence between its own sensor data and executable actions recorded and updated in the previous execution to iteratively learn, that is, the updated control model, until the task goal is finally completed.

[0123] It can be understood that the above-mentioned model training process can be performed in various environments so that the control model can adapt to different environments.

[0124] The training method of the control model proposed in the embodiment of the present application is guided by the task goal, and realizes automatic training of the control model by outputting positive feedback, negative feedback and behavior correction information corresponding to the negative feedback to the control model, so that the control model can control the smart home device 10 to complete complex tasks autonomously based on the task goal, and at the same time can adapt to the complex and changeable home environment, making the smart home device 10 more intelligent.

[0125] In another embodiment, if a complex task is received, the complex task can be further decomposed, and then each decomposed task is executed in sequence and the decomposed task is learned, and finally the autonomous learning of the complex task is completed.

[0126] After responding to the task instruction in step S110, and before the control model controls the smart home device to perform the corresponding action in step S120, such as Figure 4 As shown, the training method of the control model of the smart home device 10 includes the following steps:

[0127] S210, decomposing the global task to obtain a plurality of decomposed tasks; wherein each decomposed task corresponds to a decomposed control model.

[0128] S220, each decomposed control model performs behavior learning according to the execution order of the decomposed tasks until the learning of all decomposed control models is completed.

[0129] In one implementation, exemplary, according to the execution order of all decomposed tasks, when each decomposed control model performs behavior learning, the current self-sensor data of the smart home device 10 can be obtained, so that each decomposed control model controls the smart home device 10 to perform corresponding actions based on the current self-sensor data. Then, each decomposed control model will perform behavior learning based on the performance feedback of the smart home device 10 performing the action, until the smart home device 10 is guided to complete the corresponding decomposed task and achieve the decomposed task goal corresponding to the decomposed task.

[0130] Accordingly, in step S120, based on the sensor data, the control model controls the smart home device 10 to perform corresponding actions, including:

[0131] S230, based on its own sensor data, the control model controls the cleaning robot to sequentially combine the decomposed tasks and execute actions corresponding to the decomposed tasks according to each decomposed control model.

[0132] After obtaining each decomposed control model, the smart home device 10 starts to execute the training method of the control model of the global task. Exemplarily, the control model will control the cleaning robot to sequentially combine the decomposed tasks involved according to each decomposed control model, and then execute the actions corresponding to each decomposed task, so that the subsequent control model can learn the behavior according to the performance feedback of the cleaning robot executing the actions corresponding to each decomposed task, so that the control model can control the cleaning robot to complete the global task in the optimal sequential combination manner.

[0133] In this embodiment, the global task is decomposed, and then the above-mentioned behavioral autonomous learning guided by the task objectives of the decomposed task is adopted for each decomposed task to obtain various decomposed control models; finally, the control model uses these decomposed control models to guide the cleaning robot to complete the execution of complex tasks, so that the smart home device 10 can cope with more complex tasks through the control model.

[0134] Example 3

[0135] Figure 5The flowchart of the application method of the control model of the smart home device 10 according to the embodiment of the present application is shown. Exemplarily, the application method of the control model of the smart home device 10 includes:

[0136] S310, in response to a task instruction issued by a user, the control model controls the smart home device 10 to perform the task corresponding to the task instruction based on the task instruction.

[0137] Among them, the control model in this embodiment is obtained by training based on the training method of the control model of the smart home device 10 in the above-mentioned embodiment 2, so it will not be repeated here.

[0138] S320, after the task is executed, receiving user feedback information, so that the control model adjusts the action or the combination of actions when the task is executed next time according to the user feedback information.

[0139] In one embodiment, the user feedback information can be input into the large language model for semantic analysis and corresponding feedback information can be obtained. If the user feedback information includes negative feedback, for example, "the cleaning time needs to be further shortened this time", the large language model outputs semantic text information to the control model; wherein the semantic text information includes negative feedback and behavior correction information.

[0140] Furthermore, the control model adjusts the action or combination of actions when performing the task next time based on the negative feedback and behavior correction information.

[0141] Alternatively, in another case, if the user feedback information includes positive feedback, for example, "the number of collisions during this cleaning is small", the control model will also record all the behaviors executed this time as a reference for the next action when performing the same task.

[0142] It can be understood that during actual use, the control model will be updated according to the user's feedback. For example, if the smart home device 10 is a sweeper, after the sweeper completes the cleaning task, the user can rate the sweeper's performance in this task through the application APP on the terminal device, and input relevant suggestions, questions, affirmations, etc. Therefore, the behavior evaluation model can use the user's feedback information so that the control model will adjust the action of the next execution of the same task according to the user's suggestions, so that the smart home device 10 has the ability to continuously learn.

[0143] Example 4

[0144] Figure 6 Shown is a structural schematic diagram of a training device 100 for a control model of a smart home device 10 according to an embodiment of the present application.

[0145] Exemplarily, the training device 100 of the control model of the smart home device 10 includes:

[0146] The response module 110 is used to respond to the task instruction to obtain the current sensor data of the smart home device 10.

[0147] The action execution module 120 is used to control the smart home device 10 to perform corresponding actions based on the self sensor data and the control model.

[0148] The model training module 130 is used for the control model to perform behavioral learning based on the performance feedback of the smart home device 10 executing the action, so as to automatically train the control model until the control model is guided to control the smart home device 10 to complete the task corresponding to the task instruction and achieve the task goal corresponding to the task.

[0149] It can be understood that the device of this embodiment corresponds to the training method of the control model of the smart home device 10 in the above-mentioned embodiment 2, and the optional items in the above-mentioned embodiment are also applicable to this embodiment, so they will not be repeated here.

[0150] Example 5

[0151] Figure 7 Shown is a structural schematic diagram of an application device 200 of a control model of a smart home device 10 according to an embodiment of the present application.

[0152] Exemplarily, the application device 200 of the control model of the smart home device 10 includes:

[0153] The task execution module 210 is used to respond to the task instruction issued by the user. The control model controls the smart home device 10 to execute the task corresponding to the task instruction based on the task instruction.

[0154] The control model is obtained by training based on the training method of the control model of the smart home device 10 described in the second embodiment.

[0155] The feedback adjustment module 220 is used to receive user feedback information after the task is executed, so that the control model adjusts the action or the combination of actions when the task is executed next time according to the user feedback information.

[0156] It can be understood that the device of this embodiment corresponds to the application method of the control model of the smart home device 10 in the above-mentioned embodiment 3, and the optional items in the above-mentioned embodiment are also applicable to this embodiment, so they will not be described repeatedly here.

[0157] Example 6

[0158] The embodiment of the present application also proposes a computer-readable storage medium storing a computer program. When the computer program is executed on the processor 11, the training method of the control model of the smart home device 10 in the above-mentioned embodiment 2 or the application method of the control model of the smart home device 10 in the above-mentioned embodiment 3 is implemented.

[0159] For example, the computer-readable storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0160] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow chart, and the combination of boxes in the structure diagram and / or the flow chart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0161] In addition, the functional modules or units in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0162] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0163] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A training method for a control model of a smart home device, characterized in that: Respond to task instructions to obtain the current sensor data of smart home devices; The control model determines the executable action corresponding to the current self-sensor data according to the pre-constructed correspondence between the self-sensor data and the executable action, so that the control model drives the actuator of the smart home device to execute the executable action; After the smart home device performs the action, the updated self-sensor data and external sensor data are input into a large language model, and the performance of performing the action is evaluated by using the large language model, so as to automatically train the control model; If the performance of the action is an abnormal performance included in the large language model, negative feedback and behavior correction information corresponding to the abnormal performance are sent to the control model; the control model adjusts the action being executed or to be executed by the smart home device according to the behavior correction information, and updates the corresponding relationship between the self-sensor data and the executable action; If the performance of the action is a good performance included in the large language model, positive feedback is sent to the control model; the control model records the action as a reference for subsequent actions when executing the task, and updates the corresponding relationship between the self-sensor data and the executable action; The external sensor is arranged in the home area or test environment where the smart home device is located, including a plurality of fixed or steerable camera devices and / or inductive sensors arranged in the space around the smart home device; the smart home device obtains the external sensor data through wireless communication; After completing this task, if the task target corresponding to the task is not achieved, the smart home device is controlled to move to the starting position of the task, and the process returns to the step of obtaining the current sensor data of the smart home device until the task target is achieved.

2. The method according to claim 1, characterized in that The sending of negative feedback and behavior correction information corresponding to the abnormal performance to the control model includes: The large language model outputs semantic text information to the control model based on the input abnormal performance, where the semantic text information includes the negative feedback and behavior correction information corresponding to the abnormal performance.

3. The method according to any one of claims 1 to 2, characterized in that The task objective is determined according to the execution configuration requirements of the task; The execution configuration requirements include: one or more combinations of task execution time lower than a preset time, task execution coverage higher than a preset threshold, and the number of abnormal performance occurrences lower than a preset number.

4. An application method of a control model for a smart home device, characterized in that: include: In response to a task instruction issued by a user, the control model controls the smart home device to perform a task corresponding to the task instruction based on the task instruction; After the task is performed, receiving user feedback information, so that the control model adjusts the action or the combination of actions when the task is performed next time according to the user feedback information; The control model is obtained by training based on the training method of the control model of the smart home device described in any one of claims 1 to 3.

5. The method according to claim 4, characterized in that The control model adjusts the action or the combination of actions when the task is executed next time according to the user feedback information, including: Inputting the user feedback information into a large language model; If the user feedback information includes negative feedback, the large language model outputs semantic text information to the control model; the semantic text information includes negative feedback and behavior correction information; The control model adjusts the action or the combination of actions when the task is performed next time according to the negative feedback and the behavior correction information.

6. A smart home device, characterized in that: include: An actuator is used to execute corresponding actions; a memory storing a computer program; A controller, configured to execute the computer program to implement the method according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that: The device stores a computer program, which, when executed on a processor, implements the method according to any one of claims 1 to 5.

Images