Device control method and apparatus based on personalized action recognition, and intelligent air conditioner

Abstract

The application relates to the technical field of smart homes, and discloses a device control method based on individualized action recognition. The device control method comprises the following steps: determining a current action instruction model corresponding to current user identity information, and then determining a plurality of candidate control instructions corresponding to a current user action image / video and first probabilities corresponding to the candidate control instructions; determining current user historical use information corresponding to the current user identity information; adjusting the first probabilities corresponding to the plurality of candidate control instructions according to current environment parameters and the current user historical use information, so as to obtain the plurality of candidate control instructions and second probabilities corresponding to the candidate control instructions; and determining a candidate control instruction with the maximum second probability in the plurality of candidate control instructions as a current control instruction corresponding to the current user action image / video, and controlling a device. The device control method can obtain a more accurate current control instruction. The application further discloses a household appliance control device based on individualized action recognition and a smart air conditioner.

Description

Technical Field

[0001] The present application relates to the field of smart home technology, for example, to a device control method and apparatus based on personalized motion recognition and a smart air conditioner. Background Art

[0002] Currently, air conditioners are primarily controlled through remote controls, mobile phone apps, and voice control. Each method has its own limitations: remote controls and mobile phones require the user to touch the remote or phone to operate them, while voice control can easily disturb other family members during nighttime rest. To address this, sophisticated image recognition technology can be used to identify user gestures, retrieve the corresponding control commands, and use these commands to control the air conditioner.

[0003] During the gesture recognition process, manufacturers usually define basic gestures with high differentiation. Users can control the air conditioner based on these basic gestures. For example, control commands can be determined by identifying gestures with high differentiation, such as the number of extended fingers, finger directions, and fist clenching.

[0004] During the implementation of the embodiments of the present application, it was found that at least the following problems exist in the related art:

[0005] Some gestures may be similar or identical to daily user movements, such as clenching a fist or extending a palm when stretching. Other gestures may also be similar or identical to movements in everyday life, such as pointing with a finger when educating a baby. To distinguish these situations, users often customize gestures to replace these familiar gestures. However, image recognition technology requires a large number of training samples, while personalized gestures entered by users often require fewer training samples. This can lead to underfitting of image recognition algorithms, which can reduce recognition accuracy and cause the air conditioner to malfunction. Summary of the Invention

[0006] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0007] The embodiments of the present application provide a device control method, apparatus, and smart air conditioner based on personalized motion recognition, so as to improve the accuracy of user gesture recognition and reduce the occurrence of malfunction of the air conditioner when the user adopts personalized motion instructions.

[0008] In some embodiments, a device control method based on personalized action recognition includes: obtaining current user identity information; determining a current action instruction model corresponding to the current user identity information based on the correspondence between the user identity information and the action instruction model; wherein the action instruction model includes a correspondence between actions and instructions; obtaining a current user action image / video, and determining multiple candidate control instructions corresponding to the current user action image / video and their corresponding first probabilities based on the current action instruction model; determining the current user historical usage information corresponding to the current user identity information based on the correspondence between the user identity information and historical usage information; wherein the historical usage information records the correspondence between environmental parameters and control instructions; obtaining current environmental parameters, and adjusting the first probabilities corresponding to multiple candidate control instructions based on the current environmental parameters and the current user historical usage information to obtain multiple candidate control instructions and their corresponding second probabilities; determining the candidate control instruction with the largest second probability among the multiple candidate control instructions as the current control instruction corresponding to the current user action image / video; and controlling the device according to the current control instruction.

[0009] Optionally, based on the current action instruction model, multiple candidate control instructions corresponding to the current user action image / video and their corresponding first probabilities are determined, including: based on the image recognition model, multiple action recognition results corresponding to the current user action image / video and their corresponding first probabilities are determined; based on the current action instruction model, multiple action recognition results and their first probabilities are mapped to multiple candidate control instructions and their corresponding first probabilities.

[0010] Optionally, based on the current environmental parameters and the current user's historical usage information, the first probabilities corresponding to multiple candidate control instructions are adjusted to obtain multiple candidate control instructions and their corresponding second probabilities, including: determining multiple historical control instructions corresponding to the current environmental parameters in the current user's historical usage information; adjusting the first probabilities corresponding to multiple candidate control instructions based on the single instruction occurrence frequency of each historical control instruction in the multiple historical control instructions to obtain multiple candidate control instructions and their corresponding second probabilities; wherein, the higher the single instruction occurrence frequency of the historical control instruction, the greater the adjustment range of the first probability of the candidate control instruction corresponding to the historical control instruction.

[0011] Optionally, according to the frequency of occurrence of a single instruction of each historical control instruction among multiple historical control instructions, the first probabilities corresponding to multiple candidate control instructions are adjusted to obtain multiple candidate control instructions and their corresponding second probabilities, including: calculating the total frequency of all control instructions according to the frequency of occurrence of a single instruction of each historical control instruction; calculating the proportion of the frequency of occurrence of a single instruction of each historical control instruction in the total frequency; for a first candidate control instruction that is the same as the historical control instruction among multiple candidate control instructions, adjusting the first probability corresponding to the first candidate control instruction according to the proportion corresponding to the first candidate control instruction to obtain the second probability corresponding to the first candidate control instruction; for a second candidate control instruction that is different from the historical control instruction among multiple candidate control instructions, reducing the first probability corresponding to the second candidate control instruction according to the total frequency to obtain the second probability corresponding to the second candidate control instruction.

[0012] Optionally, for a first candidate control instruction that is the same as a historical control instruction among multiple candidate control instructions, the first probability corresponding to the first candidate control instruction is adjusted according to the proportion corresponding to the first candidate control instruction to obtain a second probability corresponding to the first candidate control instruction, including: determining the second probability corresponding to the first candidate control instruction according to the product of the proportion corresponding to the first candidate control instruction and the first probability corresponding to the first candidate control instruction; or, calculating the product of the proportion corresponding to the first candidate control instruction and the first probability corresponding to the first candidate control instruction, and determining the second probability corresponding to the first candidate control instruction according to the sum of the product and the first probability corresponding to the first candidate control instruction.

[0013] Optionally, the correspondence between actions and instructions is determined by: obtaining user action images / videos, identifying the user action images / videos to obtain specific actions; in response to user input, obtaining specific instructions input by the user; establishing and storing the correspondence between the specific actions and the specific instructions.

[0014] Optionally, in the case that the current user identity information is legal identity information, the current action instruction model corresponding to the current user identity information is determined according to the correspondence between the user identity information and the action instruction model.

[0015] In some embodiments, the device control apparatus based on personalized action recognition includes an acquisition module, a first determination module, a second determination module, a third determination module, an adjustment module, a fourth determination module and a control module; the acquisition module is used to obtain the current user identity information; the first determination module is used to determine the current action instruction model corresponding to the current user identity information based on the correspondence between the user identity information and the action instruction model; wherein the action instruction model includes the correspondence between actions and instructions; the second determination module is used to obtain the current user action image / video, and determine multiple candidate control instructions corresponding to the current user action image / video and their corresponding first probabilities based on the current action instruction model; the third determination module is used to determine the current user identity information based on the correspondence between the user identity information and the action instruction model; wherein the action instruction model includes the correspondence between actions and instructions; the second determination module is used to obtain the current user action image / video, and determine multiple candidate control instructions corresponding to the current user action image / video and their corresponding first probabilities based on the current action instruction model; The determination module is used to determine the current user's historical usage information corresponding to the current user's identity information based on the correspondence between the user identity information and the historical usage information; wherein the historical usage information records the correspondence between the environmental parameters and the control instructions; the adjustment module is used to obtain the current environmental parameters, and adjust the first probabilities corresponding to multiple candidate control instructions based on the current environmental parameters and the current user's historical usage information to obtain multiple candidate control instructions and their corresponding second probabilities; the fourth determination module is used to determine the candidate control instruction with the highest second probability among the multiple candidate control instructions as the current control instruction corresponding to the current user action image / video; the control module is used to control the device according to the current control instruction.

[0016] In some embodiments, a device control apparatus based on personalized motion recognition includes a processor and a memory storing program instructions, and the processor is configured to execute the device control method based on personalized motion recognition provided by the aforementioned embodiment when executing the program instructions.

[0017] In some embodiments, the smart air conditioner includes the device control device based on personalized motion recognition provided by the aforementioned embodiments.

[0018] The device control method, apparatus, and smart air conditioner based on personalized motion recognition provided by the embodiments of the present application can achieve the following technical effects:

[0019] The current action instruction model corresponding to the current user identity information can be a user-defined personalized correspondence between actions and instructions. Based on the current action instruction model, the candidate control instructions corresponding to the current user action image / recognition and their corresponding first probabilities can be determined. Since the current instruction action model corresponds to the current user identity information, it has personalized characteristics, which leads to the disadvantage of low accuracy of multiple candidate control instructions and their corresponding first probabilities determined by image recognition technology; then the current user historical usage information corresponding to the current user identity information is determined, and the correspondence between the environmental parameters and control instructions in the current user historical usage information is used to adjust the first probability corresponding to the candidate control instructions, that is, based on the user's usage habits, multiple candidate instructions and their corresponding first probabilities are adjusted to obtain candidate control instructions and their corresponding second probabilities that are more in line with user habits; finally, the current control instruction is determined based on the candidate control instructions and their corresponding second probabilities adjusted according to user habits. The current control instruction determined in this way is more accurate, and controlling the device with such a current control instruction can reduce the phenomenon of device malfunction.

[0020] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] One or more embodiments are exemplarily described by corresponding drawings. These exemplary descriptions and drawings do not limit the embodiments. Elements with the same reference numerals in the drawings are considered similar elements, and wherein:

[0022] Figure 1 Schematic diagram of an implementation environment of a device control method based on personalized motion recognition provided by an embodiment of the present application;

[0023] Figure 2 This is a flow chart of a device control solution based on personalized action recognition provided by an embodiment of the present application;

[0024] Figure 3 This is a schematic diagram of a process for adjusting first probabilities corresponding to multiple candidate control instructions provided by an embodiment of the present application;

[0025] Figure 4 This is a schematic diagram of a process for adjusting first probabilities corresponding to multiple candidate instructions provided in an embodiment of the present application;

[0026] Figure 5 is a schematic diagram of a device control apparatus based on personalized action recognition provided by an embodiment of the present application;

[0027] Figure 6This is a schematic diagram of a device control apparatus based on personalized motion recognition provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to be able to understand the features and technical contents of the embodiments of the present application in more detail, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference only and are not used to limit the embodiments of the present application. In the following technical description, for the sake of convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0029] In the description and claims of the embodiments of the present application and the accompanying drawings, the terms "first," "second," and the like are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate for the purposes of describing the embodiments of the present application. In addition, the terms "including," "having," and any variations thereof are intended to cover non-exclusive inclusions.

[0030] Unless otherwise stated, the term "plurality" means more than two.

[0031] In the embodiments of the present application, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B means: A or B.

[0032] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0033] In the embodiment of the present application, the device refers to a device that has the ability to adjust indoor air parameters, including but not limited to smart air conditioners, smart fresh air fans, smart air purifiers, smart humidifiers, and smart dehumidifiers.

[0034] For ease of explanation, the embodiment of the present application only takes an air conditioner as an example to exemplify the device control method based on personalized motion recognition, and does not limit the specific type of device.

[0035] Figure 1 This is a schematic diagram of an implementation environment of a device control method based on personalized action recognition provided in an embodiment of the present application.

[0036] Combine Figure 1As shown, the image acquisition device 11 can obtain the user image / video, and then send the user image / video to the smart air conditioner 12, which parses the control instruction corresponding to the user image / video, and then the smart air conditioner 12 operates according to the control instruction.

[0037] Figure 1 The image acquisition device 11 is illustrated as an example in which the image acquisition device 11 can be the image acquisition device provided by the smart air conditioner 12, and does not constitute a substantial limitation on the setting location and form of the image acquisition device 11; in the specific scenario of implementing the device control method based on personalized motion recognition, the image acquisition device 11 can also be an intelligent monitoring device provided in the smart home system.

[0038] Figure 1 In the example, the image acquisition device 11 sends the user image / video to the smart air conditioner 12, and the smart air conditioner 12 parses the control instructions corresponding to the user image / video for explanation, without limiting the specific execution subject of the control instructions corresponding to the parsing of the user image / video; in the specific scenario of implementing the device control method based on personalized action recognition, the image recognition process can also be completed at the image acquisition device 11 end to parse the control instructions corresponding to the user image / video, or the image acquisition device 11 or the smart air conditioner 12 can upload the user image / video to the server, and after parsing the control instructions, the server returns the control instructions to the image acquisition device 11 or the smart air conditioner 12.

[0039] In an embodiment of the present application, the current user identity information corresponds to the current action instruction model and the current user's historical usage information, respectively. This current user's historical usage information is used to adjust the candidate control instructions and their first probabilities determined based on the current action instruction model, thereby providing targeted results. Furthermore, the current user's historical usage information is related to the user's usage habits. That is, in the present application, based on user habits, the candidate control instructions and their corresponding first probabilities determined based on image recognition technology and the current action instruction model are fine-tuned and corrected to obtain relatively accurate candidate control instructions and their corresponding second probabilities. Finally, the air conditioner is controlled based on these candidate control instructions and their corresponding second probabilities to reduce the risk of false air conditioner operation.

[0040] Figure 2 This is a flow chart of a device control solution based on personalized action recognition provided in an embodiment of the present application.

[0041] Combine Figure 2 As shown, the device control method based on personalized action recognition includes:

[0042] S201: Obtain current user identity information.

[0043] The user identity information may be represented by a digital number or a character string.

[0044] The user's facial image information can be identified through existing image recognition technology to obtain the user identity information corresponding to the user's facial image information.

[0045] S202: Determine a current action instruction model corresponding to the current user identity information according to the correspondence between the user identity information and the action instruction model.

[0046] The action instruction model includes the corresponding relationship between actions and instructions. In specific applications, the action instruction model can exist in the form of a one-to-one corresponding data table.

[0047] The correspondence between user identity information and action instruction models can be pre-stored in a database. After obtaining the current user identity information, the current action instruction model corresponding to the current user identity information can be obtained by querying the database.

[0048] That is, before executing the process of the device control method based on personalized motion recognition, the correspondence between the user identity information and the motion instruction model, as well as the correspondence between the motion and the instruction in the motion instruction model, needs to be determined and stored.

[0049] Furthermore, the correspondence between actions and commands is determined by: obtaining user action images / videos, recognizing the user action images / videos to obtain specific actions; responding to user input, obtaining the specific commands input by the user; and establishing and storing the correspondence between the specific actions and specific commands. In this way, the correspondence between actions and commands, i.e., the action command model, is obtained.

[0050] In actual applications, the action instruction models corresponding to different user identity information can be stored in different address spaces. First, the user identity information is identified, and the address space of the action instruction model is located based on the user identity information. The above process of determining the correspondence between actions and instructions is performed to obtain the action instruction model corresponding to the user identity information.

[0051] In the above scheme, the current user identity information is used to determine the current action instruction model. Before determining the action instruction model, the authority (legitimacy) of the current user identity information can also be verified. For example, after obtaining the current user identity information, it is determined whether the current user identity information is legitimate. If the current user identity information is illegitimate, the execution of the device control method is stopped. If the current user identity information is legitimate, the current action instruction model corresponding to the current user identity information is determined based on the correspondence between the user identity information and the action instruction model.

[0052] S203: Obtain a current user action image / video, and determine, based on a current action instruction model, a plurality of candidate control instructions corresponding to the current user action image / video and their corresponding first probabilities.

[0053] A candidate control instruction is a control instruction that may correspond to the current user action image / video. A candidate control instruction corresponds one-to-one to a first probability, which indicates the likelihood that the current user action image / video is a candidate control instruction. The greater the first probability corresponding to a candidate control instruction, the greater the probability that the candidate control instruction is the control instruction corresponding to the current user action image / video.

[0054] Specifically, according to the current action instruction model, determining multiple candidate control instructions corresponding to the current user action image / video and their corresponding first probabilities may include: according to the image recognition model, determining multiple action recognition results corresponding to the current user action image / video and their corresponding first probabilities; according to the current action instruction model, mapping the multiple action recognition results and their first probabilities to multiple candidate control instructions and their corresponding first probabilities.

[0055] The image recognition model here refers to an image recognition model in the prior art, whose input is an image or video, and which outputs the recognition result of the image or video. In the embodiment of the present application, the input of the image recognition model is the current user action image / video, and the output of the image recognition model is the action recognition result corresponding to the current user action image / video and its corresponding first probability.

[0056] The current action instruction model includes the correspondence between actions and instructions. According to the current action instruction model, the action recognition results can be mapped to control instructions (such as control instructions for smart air conditioners). For example, by querying the action recognition results in the current action instruction model, the control instructions corresponding to the action recognition results can be obtained.

[0057] S204: Determine the current user's historical usage information corresponding to the current user's identity information based on the correspondence between the user identity information and the historical usage information.

[0058] The historical usage information records the corresponding relationship between environmental parameters and control instructions.

[0059] The corresponding relationship between the user identity information and the historical usage information can be stored in a database. After obtaining the current user identity information, the current user historical usage information corresponding to the current user identity information can be obtained by querying the current user identity information in the database.

[0060] Specifically, the above-mentioned environmental parameters correspond to the functions of the equipment. For example, the function of the smart air conditioner is temperature regulation, and the environmental parameter may be the indoor temperature; the function of the smart fresh air fan is to adjust the freshness of the indoor air, and the environmental parameter may be the indoor carbon dioxide concentration; the function of the smart air purifier is to purify the air, and the environmental parameter may be the indoor inhalable particle concentration, or the environmental parameter may be the volatile organic compound (VOCs) concentration; the function of the smart humidifier and the smart dehumidifier is to adjust the humidity, and the environmental parameter may be the outdoor ambient humidity.

[0061] The above-mentioned control instructions refer to control instructions received by the device in response to user operations, for example, control instructions to increase or decrease the set indoor temperature, control instructions to increase or decrease the set fresh air volume, and control instructions to increase or decrease the set indoor humidity.

[0062] The correspondence between the above environmental parameters and control instructions refers to the control instructions received by the device in response to user operations under the environmental parameters. The control instructions can be sent by the user based on the current action instruction model, sent by the user through an application (Application, APP), or sent by the user through a remote control.

[0063] For example, when the indoor temperature is 24° C., a control instruction to lower the set temperature is received. At this time, a correspondence between 24° C. and the control instruction to lower the set temperature is established and stored.

[0064] In actual applications, when the device receives a control instruction in response to a user operation, the environmental parameters at that time are obtained, and a one-to-one correspondence between the environmental parameters and the control instruction is established and stored.

[0065] S205 : Obtain current environment parameters, and adjust first probabilities corresponding to multiple candidate control instructions according to the current environment parameters and the current user's historical usage information to obtain multiple candidate control instructions and their corresponding second probabilities.

[0066] The adjusting of the first probabilities corresponding to the plurality of candidate control instructions includes increasing the first probabilities corresponding to the plurality of candidate control instructions and decreasing the first probabilities corresponding to the plurality of candidate control instructions.

[0067] S206 : Determine the candidate control instruction with the second highest probability among the multiple candidate control instructions as the current control instruction corresponding to the current user action image / video.

[0068] Among multiple candidate control instructions, each candidate control instruction corresponds one-to-one to a second probability. The largest second probability can be selected, and then the candidate control instruction corresponding to the largest second probability can be determined. At this time, the candidate control instruction corresponding to the largest second probability can be determined as the current control instruction corresponding to the current user action image / video.

[0069] S207: Control the device according to the current control instruction.

[0070] Controlling a device based on the current control command causes the device to execute the action corresponding to the current control command. For example, if the control command is to increase the airflow speed of the smart air conditioner, the smart air conditioner will increase the indoor fan speed; if the control command is to increase the indoor set temperature, the smart air conditioner will increase the current set temperature by a preset value (such as 1°C or 2°C).

[0071] Different devices and different current control instructions will cause the device to perform different actions. We will not go into details here. Just make sure that the device performs the action corresponding to the current control instruction.

[0072] The current action instruction model corresponding to the current user identity information can be a user-defined personalized correspondence between actions and instructions. Based on the current action instruction model, the candidate control instructions corresponding to the current user action image / recognition and their corresponding first probabilities can be determined. Since the current instruction action model corresponds to the current user identity information, it has personalized characteristics, which leads to the disadvantage of low accuracy of multiple candidate control instructions and their corresponding first probabilities determined by image recognition technology; then the current user historical usage information corresponding to the current user identity information is determined, and the correspondence between the environmental parameters and control instructions in the current user historical usage information is used to adjust the first probability corresponding to the candidate control instructions, that is, based on the user's usage habits, multiple candidate instructions and their corresponding first probabilities are adjusted to obtain candidate control instructions and their corresponding second probabilities that are more in line with user habits; finally, the current control instruction is determined based on the candidate control instructions and their corresponding second probabilities adjusted according to user habits. The current control instruction determined in this way is more accurate, and controlling the device with such a current control instruction can reduce the phenomenon of device malfunction.

[0073] Figure 3 This is a schematic diagram of a process for adjusting the first probabilities corresponding to multiple candidate control instructions provided in an embodiment of the present application.

[0074] Combine Figure 3 As shown, according to the current environment parameters and the current user's historical usage information, the first probabilities corresponding to the multiple candidate control instructions are adjusted to obtain multiple candidate control instructions and their corresponding second probabilities, including:

[0075] S301: Determine a plurality of historical control instructions corresponding to current environment parameters in the current user's historical usage information.

[0076] For any device, the current user's historical usage information includes multiple environmental parameters. Taking the device as a smart air conditioner as an example, the multiple environmental parameters can be multiple indoor temperatures, and each environmental parameter corresponds to multiple historical control instructions; due to different needs, users will perform different operations on the device under the same environmental parameters, that is, the device will receive different control instructions. Taking the device as a smart air conditioner as an example, when the environmental parameter is 23°C, if the user has just finished exercising, the user will usually lower the set temperature (the device receives a control instruction to lower the set temperature); if the user needs to rest, the user will usually increase the set temperature (the device receives a control instruction to increase the set temperature).

[0077] S302 : Adjust first probabilities corresponding to multiple candidate control instructions according to the single instruction occurrence frequency of each historical control instruction in the multiple historical control instructions to obtain multiple candidate control instructions and their corresponding second probabilities.

[0078] The higher the frequency of occurrence of a single instruction of a historical control instruction is, the greater the adjustment range of the first probability of the candidate control instruction corresponding to the historical control instruction is.

[0079] In the above steps, the first probabilities corresponding to the plurality of candidate instructions are adjusted.

[0080] Figure 4 This is a schematic diagram of a process for adjusting the first probabilities corresponding to multiple candidate instructions provided in an embodiment of the present application.

[0081] Combine Figure 4 As shown, according to the single instruction occurrence frequency of each historical control instruction in multiple historical control instructions, the first probabilities corresponding to the multiple candidate control instructions are adjusted to obtain multiple candidate control instructions and their corresponding second probabilities, including:

[0082] S401: Calculate the total frequency of all control instructions according to the frequency of occurrence of each single instruction of each historical control instruction.

[0083] For example, the historical control instructions corresponding to an environment parameter are: instruction A, instruction B, instruction A, instruction A and instruction B. The single instruction occurrence frequency of instruction A is 3 times, the single instruction occurrence frequency of instruction B is 2 times, and the total frequency of all control instructions is 5 times.

[0084] S402: Calculate the proportion of the single instruction occurrence frequency of each historical control instruction in the total frequency.

[0085] For example, the historical control instructions corresponding to an environment parameter are: instruction A, instruction B, instruction A, instruction A and instruction B. The frequency of occurrence of single instruction A in the total frequency is 3 / 5, and the frequency of occurrence of single instruction B in the total frequency is 2 / 5.

[0086] S403 . For a first candidate control instruction that is the same as a historical control instruction among the multiple candidate control instructions, adjust a first probability corresponding to the first candidate control instruction according to a proportion corresponding to the first candidate control instruction to obtain a second probability corresponding to the first candidate control instruction.

[0087] Moreover, the greater the proportion corresponding to the first candidate control instruction is, the greater the adjustment range of the first probability corresponding to the first candidate control instruction is.

[0088] Optionally, for a first candidate control instruction that is the same as a historical control instruction among multiple candidate control instructions, the first probability corresponding to the first candidate control instruction is adjusted according to the proportion corresponding to the first candidate control instruction to obtain a second probability corresponding to the first candidate control instruction, including: determining the second probability corresponding to the first candidate control instruction according to the product of the proportion corresponding to the first candidate control instruction and the first probability corresponding to the first candidate control instruction; or, calculating the product of the proportion corresponding to the first candidate control instruction and the first probability corresponding to the first candidate control instruction, and determining the second probability corresponding to the first candidate control instruction according to the sum of the product and the first probability corresponding to the first candidate control instruction.

[0089] For example, the product of the proportion corresponding to the first candidate control instruction and the first probability corresponding to the first candidate control instruction can be determined as the second probability corresponding to the first candidate control instruction; or, due to the influence of some practical factors, the product of the proportion corresponding to the first candidate control instruction and the first probability corresponding to the first candidate control instruction is adjusted to reduce the influence of these practical factors, and the adjusted product is determined as the second probability corresponding to the first candidate control instruction.

[0090] The product of the proportion corresponding to the first candidate control instruction and the first probability corresponding to the first candidate control instruction can be calculated, and the sum of the product and the first probability corresponding to the first candidate control instruction can be determined as the second probability corresponding to the first candidate control instruction; or, the product of the proportion corresponding to the first candidate control instruction and the first probability corresponding to the first candidate control instruction and the sum of the product and the first probability corresponding to the first candidate control instruction can be calculated, and the sum can be adjusted due to the influence of some actual factors to reduce the influence of these actual factors, and the adjusted sum can be determined as the second probability corresponding to the first candidate control instruction.

[0091] S404 . For a second candidate control instruction that is different from the historical control instruction among the multiple candidate control instructions, reduce the first probability corresponding to the second candidate control instruction according to the total frequency to obtain a second probability corresponding to the second candidate control instruction.

[0092] The greater the total frequency, the greater the reduction in the first probability corresponding to the second candidate instruction.

[0093] In the above process, multiple candidate control instructions are divided into two categories: first candidate control instructions and second candidate control instructions, and the first probability corresponding to the second candidate control instruction is reduced to further reduce the impact of the second candidate control instruction on the determination of the current control instruction, thereby improving the accuracy of the determined current control instruction.

[0094] Furthermore, the minimum decrease in the first probability corresponding to the second candidate control instruction is greater than or equal to the maximum decrease in the first probability corresponding to the first candidate control instruction, so as to further improve the accuracy of the determined current control instruction and reduce the occurrence of device malfunction.

[0095] Figure 5 Schematic diagram of a device control apparatus based on personalized action recognition provided by an embodiment of the present application. The device control apparatus based on personalized action recognition can be implemented in the form of software, hardware, or a combination of the two.

[0096] Combine Figure 5 As shown, the device control apparatus based on personalized action recognition includes an obtaining module 51, a first determining module 52, a second determining module 53, a third determining module 54, an adjusting module 55, a fourth determining module 56 and a control module 57;

[0097] The acquisition module 51 is used to obtain the current user identity information;

[0098] The first determining module 52 is used to determine the current action instruction model corresponding to the current user identity information according to the correspondence between the user identity information and the action instruction model; wherein the action instruction model includes the correspondence between the action and the instruction;

[0099] The second determination module 53 is used to obtain the current user action image / video and determine a plurality of candidate control instructions corresponding to the current user action image / video and their corresponding first probabilities based on the current action instruction model;

[0100] The third determining module 54 is used to determine the current user's historical usage information corresponding to the current user's identity information based on the correspondence between the user identity information and the historical usage information; wherein the historical usage information records the correspondence between the environmental parameters and the control instructions;

[0101] The adjustment module 55 is used to obtain current environment parameters and adjust the first probabilities corresponding to the multiple candidate control instructions based on the current environment parameters and the current user's historical usage information to obtain multiple candidate control instructions and their corresponding second probabilities;

[0102] The fourth determining module 56 is configured to determine the candidate control instruction with the second highest probability among the multiple candidate control instructions as the current control instruction corresponding to the current user action image / video;

[0103] The control module 57 is used to control the device according to the current control instruction.

[0104] Optionally, the second determination module 53 includes a first determination unit and a mapping unit, the first determination unit being used to determine, based on the image recognition model, multiple action recognition results and their corresponding first probabilities corresponding to the current user action image / video; the mapping unit being used to map, based on the current action instruction model, the multiple action recognition results and their first probabilities into multiple candidate control instructions and their corresponding first probabilities.

[0105] Optionally, the adjustment module 55 includes a second determination unit and an adjustment unit, the second determination unit is used to determine multiple historical control instructions corresponding to the current environmental parameters in the current user's historical usage information; the adjustment unit is used to adjust the first probabilities corresponding to multiple candidate control instructions according to the single instruction occurrence frequency of each historical control instruction in the multiple historical control instructions, and obtain multiple candidate control instructions and their corresponding second probabilities; wherein, the higher the single instruction occurrence frequency of the historical control instruction, the greater the adjustment range of the first probability of the candidate control instruction corresponding to the historical control instruction.

[0106] Optionally, the adjustment unit is specifically used to calculate the total frequency of all control instructions based on the single instruction occurrence frequency of each historical control instruction; calculate the proportion of the single instruction occurrence frequency of each historical control instruction in the total frequency; for a first candidate control instruction that is the same as the historical control instruction among multiple candidate control instructions, adjust the first probability corresponding to the first candidate control instruction according to the proportion corresponding to the first candidate control instruction to obtain a second probability corresponding to the first candidate control instruction; for a second candidate control instruction that is different from the historical control instruction among multiple candidate control instructions, reduce the first probability corresponding to the second candidate control instruction according to the total frequency to obtain a second probability corresponding to the second candidate control instruction.

[0107] Optionally, for a first candidate control instruction that is the same as a historical control instruction among multiple candidate control instructions, the first probability corresponding to the first candidate control instruction is adjusted according to the proportion corresponding to the first candidate control instruction to obtain a second probability corresponding to the first candidate control instruction, including: determining the second probability corresponding to the first candidate control instruction according to the product of the proportion corresponding to the first candidate control instruction and the first probability corresponding to the first candidate control instruction; or, calculating the product of the proportion corresponding to the first candidate control instruction and the first probability corresponding to the first candidate control instruction, and determining the second probability corresponding to the first candidate control instruction according to the sum of the product and the first probability corresponding to the first candidate control instruction.

[0108] Optionally, the correspondence between actions and instructions is determined by: obtaining user action images / videos, identifying user action images / videos to obtain specific actions; responding to user input, obtaining specific instructions input by the user; establishing and storing the correspondence between specific actions and specific instructions.

[0109] Optionally, the first determining module 52 is specifically configured to determine a current action instruction model corresponding to the current user identity information according to a correspondence between the user identity information and the action instruction model when the current user identity information is legal identity information.

[0110] In some embodiments, a device control apparatus based on personalized motion recognition includes a processor and a memory storing program instructions. The processor is configured to execute the device control method based on personalized motion recognition provided by the aforementioned embodiment when executing the program instructions.

[0111] Figure 6 Schematic diagram of a device control device based on personalized action recognition provided by an embodiment of the present application. Figure 6 As shown, the device control device based on personalized action recognition includes:

[0112] The processor 61 and memory 62 may also include a communication interface 63 and a bus 64. The processor 61, communication interface 63, and memory 62 may communicate with each other via the bus 64. The communication interface 63 may be used for information transmission. The processor 61 may call the logic instructions in the memory 62 to execute the device control method based on personalized motion recognition provided in the aforementioned embodiment.

[0113] In addition, the logic instructions in the memory 62 can be implemented in the form of software functional units and stored in a computer-readable storage medium when sold or used as an independent product.

[0114] Memory 62, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of the present application. Processor 61 executes the software programs, instructions, and modules stored in memory 62 to perform functional applications and data processing, thereby implementing the methods in the above-mentioned method embodiments.

[0115] The memory 62 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated based on the use of the terminal device. Furthermore, the memory 62 may include high-speed random access memory and non-volatile memory.

[0116] An embodiment of the present application provides a smart air conditioner, comprising the device control device based on personalized motion recognition provided by the aforementioned embodiment.

[0117] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the device control method based on personalized motion recognition provided by the aforementioned embodiment.

[0118] An embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the device control method based on personalized action recognition provided by the aforementioned embodiment.

[0119] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0120] The technical solutions of the embodiments of the present application can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of the embodiments of the present application. The aforementioned storage medium can be a non-transitory storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program code, or a transient storage medium.

[0121] The above description and accompanying drawings sufficiently illustrate the embodiments of the present application to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, process, and other changes. The embodiments represent only possible variations. Unless expressly required, individual components and functions are optional, and the order of operations may vary. Portions and features of some embodiments may be included in or substituted for portions and features of other embodiments. Furthermore, the terms used in this application are intended only to describe the embodiments and are not intended to limit the claims. As used in the embodiments and in the claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. In addition, when used in this application, the terms "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. In the absence of further limitations, the phrase "comprising a..." does not preclude the presence of other identical elements in the process, method, or apparatus comprising the elements. In this document, each embodiment may focus on the differences from other embodiments, and similar portions between the embodiments may refer to each other. For methods, products, etc. disclosed in the embodiments, if they correspond to the method portion disclosed in the embodiments, the relevant portions may refer to the description of the method portion.

[0122] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application. Technicians can clearly understand that for the convenience and brevity of description, the specific working process of the above-described systems, devices and units can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0123] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units can be merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, the functional units in the embodiments of the present application may be integrated into a processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

[0124] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the system, method and computer program product according to the embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or code, and a part of a module, program segment or code comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a special hardware-based system that performs the function or action of the specification, or can be implemented by a combination of special hardware and computer instructions.

Claims

1. A device control method based on personalized motion recognition, characterized in that: include: Get the current user's identity information; Determining a current action instruction model corresponding to the current user identity information according to the correspondence between the user identity information and the action instruction model; wherein the action instruction model includes a correspondence between actions and instructions; Obtaining a current user action image / video, and determining, based on the current action instruction model, a plurality of candidate control instructions corresponding to the current user action image / video and their corresponding first probabilities; Determining the current user's historical usage information corresponding to the current user's identity information based on the correspondence between the user's identity information and the historical usage information; wherein the historical usage information records the correspondence between the environmental parameters and the control instructions; Obtaining current environment parameters, and adjusting first probabilities corresponding to a plurality of candidate control instructions based on the current environment parameters and the current user's historical usage information, to obtain a plurality of candidate control instructions and their corresponding second probabilities; specifically comprising: determining a plurality of historical control instructions corresponding to the current environment parameters in the current user's historical usage information; adjusting the first probabilities corresponding to the plurality of candidate control instructions based on a single instruction occurrence frequency of each of the plurality of historical control instructions, to obtain a plurality of candidate control instructions and their corresponding second probabilities; wherein, a higher single instruction occurrence frequency of a historical control instruction, a greater adjustment amplitude is applied to the first probabilities of the candidate control instructions corresponding to the historical control instruction; Determining the candidate control instruction with the second highest probability among the multiple candidate control instructions as the current control instruction corresponding to the current user action image / video; The device is controlled according to the current control instruction.

2. The device control method according to claim 1, wherein: Determining, based on the current action instruction model, a plurality of candidate control instructions corresponding to the current user action image / video and their corresponding first probabilities, including: Determine, based on the image recognition model, a plurality of action recognition results corresponding to the current user action image / video and their corresponding first probabilities; According to the current action instruction model, a plurality of action recognition results and their first probabilities are mapped into a plurality of candidate control instructions and their corresponding first probabilities.

3. The device control method according to claim 1, wherein: Adjusting the first probabilities corresponding to the plurality of candidate control instructions according to the single instruction occurrence frequency of each historical control instruction in the plurality of historical control instructions to obtain the plurality of candidate control instructions and their corresponding second probabilities includes: According to the frequency of occurrence of each historical control instruction, the total frequency of all control instructions is calculated; Calculate the proportion of the single instruction occurrence frequency of each historical control instruction in the total frequency; For a first candidate control instruction that is the same as a historical control instruction among multiple candidate control instructions, adjusting a first probability corresponding to the first candidate control instruction according to a proportion corresponding to the first candidate control instruction to obtain a second probability corresponding to the first candidate control instruction; For a second candidate control instruction different from the historical control instruction among the multiple candidate control instructions, the first probability corresponding to the second candidate control instruction is reduced according to the total frequency to obtain a second probability corresponding to the second candidate control instruction.

4. The device control method according to claim 3, characterized in that: For a first candidate control instruction that is the same as a historical control instruction among multiple candidate control instructions, adjusting a first probability corresponding to the first candidate control instruction according to a proportion corresponding to the first candidate control instruction to obtain a second probability corresponding to the first candidate control instruction includes: determining a second probability corresponding to the first candidate control instruction according to a product of the proportion corresponding to the first candidate control instruction and the first probability corresponding to the first candidate control instruction; Alternatively, the product of the proportion corresponding to the first candidate control instruction and the first probability corresponding to the first candidate control instruction is calculated, and the second probability corresponding to the first candidate control instruction is determined according to the sum of the product and the first probability corresponding to the first candidate control instruction.

5. The device control method according to any one of claims 1 to 4, characterized in that: The correspondence between actions and instructions is determined as follows: Obtaining a user action image / video, and identifying the user action image / video to obtain a specific action; In response to user input, obtaining a specific instruction input by the user; A correspondence between the specific action and the specific instruction is established and stored.

6. The device control method according to any one of claims 1 to 4, characterized in that: In the case that the current user identity information is legal identity information, the current action instruction model corresponding to the current user identity information is determined according to the correspondence between the user identity information and the action instruction model.

7. A device control device based on personalized action recognition, characterized in that: include: The acquisition module is used to obtain the current user identity information; A first determining module, configured to determine a current action instruction model corresponding to the current user identity information based on a correspondence between the user identity information and the action instruction model; wherein the action instruction model includes a correspondence between actions and instructions; a second determination module, configured to obtain a current user action image / video, and determine, based on the current action instruction model, a plurality of candidate control instructions corresponding to the current user action image / video and their corresponding first probabilities; A third determining module is configured to determine the current user's historical usage information corresponding to the current user's identity information based on the correspondence between the user identity information and the historical usage information; wherein the historical usage information records the correspondence between the environmental parameters and the control instructions; An adjustment module is configured to obtain current environment parameters and, based on the current environment parameters and the current user's historical usage information, adjust first probabilities corresponding to a plurality of candidate control instructions to obtain a plurality of candidate control instructions and their corresponding second probabilities. The adjustment module specifically comprises: determining, from the current user's historical usage information, a plurality of historical control instructions corresponding to the current environment parameters; adjusting the first probabilities corresponding to the plurality of candidate control instructions based on a single instruction occurrence frequency of each of the plurality of historical control instructions to obtain a plurality of candidate control instructions and their corresponding second probabilities. The higher the single instruction occurrence frequency of a historical control instruction, the greater the adjustment amplitude of the first probability of the candidate control instruction corresponding to the historical control instruction. a fourth determining module, configured to determine the candidate control instruction with the second highest probability among the multiple candidate control instructions as the current control instruction corresponding to the current user action image / video; A control module is used to control the device according to the current control instruction.

8. A device control apparatus based on personalized motion recognition, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the device control method based on personalized motion recognition according to any one of claims 1 to 6 when executing the program instructions.

9. An intelligent air conditioner, characterized in that: It includes the device control device based on personalized action recognition as described in claim 7 or 8.

Images