Air conditioner control method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an air conditioner control method and device, electronic equipment and a storage medium. The air conditioner control method comprises the steps that user state data, environment data and air conditioner output parameters are obtained; feature extraction is carried out based on the user state data and the environment data, and user hidden preference features are determined; fusing the user state data and the user recessive preference features, and determining state space features; based on the air conditioner output parameters, behavior space characteristics are determined; performing inverse reinforcement learning on the user state data, the behavior space features and the state space features, and determining a reward function; air conditioner control parameters are determined by combining the behavior space characteristics, the state space characteristics and the reward function; and controlling the air conditioner based on the air conditioner control parameters. According to the embodiment of the invention, personalized, accurate and comfortable temperature control of a user can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart home control, and in particular to an air conditioner control method, an air conditioner control device, an electronic device and a computer readable storage medium. BACKGROUND

[0002] At present, in the air conditioner temperature control scheme, the overall room temperature is usually obtained by a temperature sensor, and the temperature of the entire room is uniformly adjusted to realize indoor environment control. Since only single temperature sensor data is relied on and only one target temperature is controlled, the air conditioner cannot accurately and comfortably control the temperature for the user. The existing control method cannot meet the personalized and accurate comfort needs of the user. SUMMARY

[0003] In view of the above problems, the present application embodiments are proposed to provide an air conditioner control method, an air conditioner control device, an electronic device and a computer readable storage medium which overcome the above problems or at least partially solve the above problems.

[0004] To solve the above problems, in a first aspect of the present application, the present application embodiments disclose an air conditioner control method, comprising:

[0005] obtaining user state data, environment data and air conditioner output parameters;

[0006] performing feature extraction based on the user state data and the environment data to determine user implicit preference features;

[0007] fusing the user state data and the user implicit preference features to determine state space features;

[0008] determining action space features based on the air conditioner output parameters;

[0009] performing inverse reinforcement learning on the user state data, the action space features and the state space features to determine a reward function;

[0010] determining air conditioner control parameters in combination with the action space features, the state space features and the reward function;

[0011] controlling the air conditioner based on the air conditioner control parameters.

[0012] Optionally, the step of performing feature extraction based on the user state data and the environment data to determine user implicit preference features comprises:

[0013] performing short-term instant feature extraction on the user state data and the environment data to determine short-term preference features;

[0014] extracting short-term habit features from the user state data and the environment data, to determine short-term preference features;

[0015] extracting long-term trend features from the user state data and the environment data, to determine long-term preference features;

[0016] combining the short-term preference features, the medium-term preference features and the long-term preference features to determine implicit preference features of the user.

[0017] Optionally, the step of combining the short-term preference features, the medium-term preference features and the long-term preference features to determine implicit preference features of the user comprises:

[0018] performing weighted calculation on the short-term preference features, the medium-term preference features and the long-term preference features to determine implicit preference features of the user.

[0019] Optionally, the step of fusing the user state data and the implicit preference features of the user to determine state space features comprises:

[0020] performing multi-modal fusion on the user state data and the implicit preference features of the user to determine a context embedding vector;

[0021] combining the context embedding vector to determine state space features.

[0022] Optionally, the step of determining a behavior space feature based on the air conditioner output parameter comprises:

[0023] determining a control behavior feature based on the air conditioner output parameter;

[0024] combining the control behavior feature to determine a behavior space feature.

[0025] Optionally, the step of performing inverse reinforcement learning on the user state data, the behavior space feature and the state space feature to determine a reward function comprises:

[0026] determining an initial function based on the user state data;

[0027] performing multi-objective fusion on the behavior space feature and the state space feature to determine a correction parameter;

[0028] combining the initial function and the correction parameter to determine a reward function.

[0029] Optionally, the step of performing inverse reinforcement learning on the user state data, the behavior space feature and the state space feature to determine a reward function further comprises:

[0030] The reward function is adversarially trained based on the user state data, the action space feature and the state space feature to update the reward function.

[0031] Optionally, the step of determining the air conditioner control parameter by combining the action space feature, the state space feature and the reward function comprises:

[0032] In the state space feature, an action sequence matching a change trend of the state space feature within a preset time length and the reward function being at a target value is determined.

[0033] An air conditioner control parameter is determined based on the action sequence.

[0034] Optionally, the method further comprises:

[0035] The action space feature, the state space feature, the reward function and the air conditioner control parameter are displayed.

[0036] In a second aspect of the present application, embodiments of the present application disclose an air conditioner control device, comprising:

[0037] An acquisition module is configured to acquire user state data, environment data and air conditioner output parameters.

[0038] A first feature extraction module is configured to perform feature extraction based on the user state data and the environment data to determine a user implicit preference feature.

[0039] A fusion module is configured to fuse the user state data and the user implicit preference feature to determine a state space feature.

[0040] A second feature extraction module is configured to perform feature extraction based on the air conditioner output parameters to determine an action space feature.

[0041] An inverse reinforcement learning module is configured to perform inverse reinforcement learning on the user state data, the action space feature and the state space feature to determine a reward function.

[0042] A decision module is configured to determine an air conditioner control parameter by combining the action space feature, the state space feature and the reward function.

[0043] A control module is configured to control an air conditioner based on the air conditioner control parameter.

[0044] In a third aspect of the present application, embodiments of the present application disclose an electronic device, comprising a processor, a memory and a computer program stored on the memory and capable of running on the processor, wherein the computer program is executed by the processor to implement the steps of the air conditioner control method as described above.

[0045] In a fourth aspect of the present application, the embodiment of the present application discloses a computer readable storage medium, a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to implement the steps of the air conditioner control method.

[0046] The embodiment of the present application has the following advantages:

[0047] The embodiment of the present application obtains user state data, environment data and air conditioner output parameters; performs feature extraction based on the user state data and the environment data, determines user implicit preference features; fuses the user state data and the user implicit preference features, determines state space features; determines action space features based on the air conditioner output parameters; performs inverse reinforcement learning on the user state data, the action space features and the state space features, determines a reward function; determines air conditioner control parameters in combination with the action space features, the state space features and the reward function; and controls the air conditioner based on the air conditioner control parameters. By sensing the user state data and the environment data, and analyzing the user behavior and demand based on the same, the state of the user can be determined, and the user state and the output action of the air conditioner are processed as a consideration factor of the control decision, which can provide more accurate and personalized comfortable experience for the user. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is a step flow chart of an air conditioner control method embodiment of the present application;

[0049] Figure 2 is a step flow chart of another air conditioner control method embodiment of the present application;

[0050] Figure 3 is an execution schematic diagram of an air conditioner control method example of the present application;

[0051] Figure 4 is a structure block diagram of an air conditioner control device embodiment of the present application;

[0052] Figure 5 is a structure block diagram of a washing machine provided by the embodiment of the present application;

[0053] Figure 6 is a structure block diagram of a storage medium provided by the embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the above objectives, features and advantages of the present application more apparent and understandable, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0055] Reference Figure 1, a step flow chart of an embodiment of an air conditioner control method of the present application is shown, which can specifically include the following steps:

[0056] Step 101, obtaining user state data, environment data and air conditioner output parameters;

[0057] The user state data can be obtained through various sensors. The environment data such as real-time environment state or future environment state of a certain time length of the environment can be obtained through sensors or network databases. The air conditioner output parameters can be obtained from the running log, performance parameters and other data of the air conditioner system. The user state data is used to represent the physiological state, psychological state and position state of the user. The environment data is used to represent the environment state of the air conditioner output area. The air conditioner output parameters are used to represent the output behavior of the air conditioner system.

[0058] Step 102, feature extraction based on the user state data and the environment data to determine the user implicit preference feature;

[0059] The user state data and the environment data can be jointly extracted to identify the user implicit preference feature. The user implicit preference feature is used to represent the user's needs without instructions.

[0060] Step 103, fusion of the user state data and the user implicit preference feature to determine the state space feature;

[0061] The user state data and the user implicit preference feature are fused to establish the mapping relationship between the user state data and the user implicit preference feature, and to determine the state space feature. The state space feature is used to represent the user behavior and demand.

[0062] Step 104, determining the action space feature based on the air conditioner output parameters;

[0063] The behavior feature of the air conditioner output parameters can be extracted to determine the corresponding air conditioner output behavior in different states, and the action space feature is determined based on the air conditioner output behavior.

[0064] Step 105, inverse reinforcement learning of the user state data, the action space feature and the state space feature to determine the reward function;

[0065] The user state data, the action space feature and the state space feature can be used for inverse reinforcement learning to analyze the corresponding user behavior demand and air conditioner output behavior in different user states, and to determine the reward function for target optimization.

[0066] In step 106, the air conditioner control parameter is determined by combining the action space feature, the state space feature and the reward function.

[0067] The output target of the air conditioner is calculated by combining the action space feature and the state space feature, and the air conditioner control parameter is determined by optimizing based on the reward function.

[0068] In step 107, the air conditioner is controlled based on the air conditioner control parameter.

[0069] The air conditioner is controlled based on the air conditioner control parameter, so as to achieve the temperature control effect.

[0070] In the embodiment of the application, the user state data, the environment data and the air conditioner output parameter are obtained, the user implicit preference feature is determined based on the user state data and the environment data, the state space feature is determined by fusing the user state data and the user implicit preference feature, the action space feature is determined based on the air conditioner output parameter, the reward function is determined by inverse reinforcement learning on the user state data, the action space feature and the state space feature, the air conditioner control parameter is determined by combining the action space feature, the state space feature and the reward function, and the air conditioner is controlled based on the air conditioner control parameter. By sensing the user state data and the environment data, the user behavior and demand are analyzed based on the user state data and the environment data, the state of the user is determined, the user state and the output action of the air conditioner are taken as the consideration factors of the control decision, and the user can be provided with more accurate and personalized comfortable experience.

[0071] Referring to Figure 2 FIG. 4 shows a step flowchart of another air conditioner control method embodiment of the application, and the air conditioner control method can specifically include the following steps:

[0072] In step 201, the user state data, the environment data and the air conditioner output parameter are obtained.

[0073] The user state data, the environment data and the air conditioner output parameter can be obtained through various sensors, devices and databases.

[0074] The precise position and activity state (sitting, sleeping, light activity) of the user in the room are perceived, such as by using Wi-Fi RTT / CSI (Wi-Fi round trip time), Bluetooth AA / AD (Bluetooth positioning), or ultrasonic technology. The collected data will be directly used to construct the space context and activity state features of the user, provide key parameters for the definition of the state space, and be used to judge the influence of the activity intensity of the user on the comfort. The body surface temperature distribution of the user is monitored in real time by using a thermal imaging camera. The data will serve as a direct indicator of physiological comfort, can directly quantify the user's sense of cold and heat, and participate in the construction of the user's physiological comfort dimension in the state space. The breathing rate, sleep posture, and quality of the user are detected by using a millimeter wave radar. The data mainly serve the sleep scenario and serve as a reference for the physiological state of the user in the state space. Weather forecasts and air quality indexes (AQI) are obtained, and current and future predicted data are provided to provide time dynamic information. Pollen indexes and specific pollutant indexes provide references for specific needs of the user. Micro-expression and posture data collected by the camera are used to judge the instantaneous comfort feeling of the user that is not explicitly expressed. The emotional tendency in the voice instruction is used to judge the satisfaction of the user to the current adjustment.

[0075] Environmental data such as intelligent home device linkage data, such as state data of curtains, lights, purifiers, and the like, and schedule information, provide more rich behavior correlation information, and at the same time support strategy generation.

[0076] In step 202, feature extraction is performed based on the user state data and the environment data, and a user implicit preference feature is determined.

[0077] The information correlation in the user state data and the environment data can be subjected to feature extraction, and a user implicit preference feature is determined.

[0078] In an optional embodiment of the present application, the step of performing feature extraction based on the user state data and the environment data to determine a user implicit preference feature includes:

[0079] In sub-step S2021, short-term instantaneous feature extraction is performed on the user state data and the environment data, and a short-term preference feature is determined.

[0080] The user state data and the environment data can be subjected to short-term instantaneous feature extraction in real time in a short term, and a short-term preference feature is determined. The short-term preference feature is used to represent the instantaneous behavior feature of the user.

[0081] For example, the operation time lag and amplitude analysis, the analysis result is quantified as the intensity of the user's immediate modification of the current automatic adjustment strategy, as the user's direct feedback on the system behavior under the current state, which can modify the mapping relationship of the reward function corresponding to the state and behavior. Continuously fine-tune the behavior mode, and the mode analysis result reflects the user's subtle preference for the current parameters, which is quantified as a feedback signal, which can more finely adjust the reward function.

[0082] The sub-step S2022 extracts the medium-term habit features from the user state data and the environment data to determine the medium-term preference features.

[0083] The user state data and the environment data can be extracted for medium-term habit features in a certain time length to determine the medium-term preference features. The medium-term preference features are used to represent the habit features of the user.

[0084] For example, only shadow comparison, based on the difference comparison with the previous data, the difference quantization result represents the deviation degree of the user's long-term behavior and the initial strategy of the system, as the feedback of the user's stable preference, used to update the overall trend of the reward function. The repeated behavior pattern mining in a specific situation can extract the pattern as the user's preferred behavior in a specific situation (state).

[0085] The sub-step S2023 extracts the long-term trend features from the user state data and the environment data to determine the long-term preference features.

[0086] The user state data and the environment data can be extracted for long-term trend features in a long time to determine the long-term trend features. The long-term trend features are used to represent the behavior trend features of the user.

[0087] For example, the comfort zone drift detection result quantifies the change amount of the user's preference over time, which can dynamically adjust the basic parameters of the reward function to adapt to the evolution of the user's long-term preference. The personalized trade-off point analysis of energy consumption and comfort degree can determine the user's preference weight in multi-objective optimization, which can adjust the weight coefficients of each target (comfort, energy saving, etc.) in the reward function.

[0088] Among them, the observation time length corresponding to the long-term trend features, the medium-term habit features, and the short-term immediate features decreases in turn, that is, the long-term trend features are the longest, the medium-term habit features are the second, and the short-term immediate features are the shortest.

[0089] The sub-step S2024 determines the user's implicit preference features by combining the short-term preference features, the medium-term preference features, and the long-term preference features.

[0090] The short-term preference feature, the medium-term preference feature and the long-term preference feature can be combined to determine the implicit preference feature of the user, which can be used as the core basis of the learning reward function. The high confidence of the implicit preference feature of the user will have a higher weight in the reward function update.

[0091] Specifically, the step of combining the short-term preference feature, the medium-term preference feature and the long-term preference feature to determine the implicit preference feature of the user includes: performing weighted calculation on the short-term preference feature, the medium-term preference feature and the long-term preference feature to determine the implicit preference feature of the user.

[0092] The weight value corresponding to the short-term preference feature, the weight value corresponding to the medium-term preference feature and the weight value corresponding to the long-term preference feature can be determined respectively, and then the weight value corresponding to the short-term preference feature, the weight value corresponding to the medium-term preference feature and the weight value corresponding to the long-term preference feature are weighted with the short-term preference feature, the medium-term preference feature and the long-term preference feature respectively to determine the implicit preference feature of the user.

[0093] In step 203, the user state data and the implicit preference feature of the user are fused to determine the state space feature.

[0094] The user state data and the implicit preference feature of the user can be fused to cross-correlate the implicit preference feature of the user and the user state data to determine the state space feature.

[0095] In an optional embodiment of the present application, the step of fusing the user state data and the implicit preference feature of the user to determine the state space feature includes: performing multi-modal fusion on the user state data and the implicit preference feature of the user to determine a context embedding vector; and combining the context embedding vector to determine the state space feature.

[0096] The multi-modal data of the user state data and the implicit preference feature of the user are fused and compressed into a context embedding vector through a self-encoder technology, and the context embedding vector represents the comprehensive context (including environmental, physiological, behavioral, time and other dimensions) in which the current user is located. The context embedding vector is combined to determine the state space feature. The state space feature represents the specific preference of the user in a certain context.

[0097] In step 204, the space feature of the action is determined based on the output parameter of the air conditioner.

[0098] The actual air conditioner output behavior action is determined based on the output parameter of the air conditioner, and the corresponding space feature of the action is extracted. The space feature of the action represents the output behavior action of the air conditioner.

[0099] In an optional embodiment of the present application, the step of determining the space feature based on the air conditioner output parameter comprises: determining a control behavior feature based on the air conditioner output parameter; and combining the control behavior feature to determine the space feature.

[0100] The control behavior features corresponding to different air conditioner output parameters can be determined, the high-level behavior target of the control behavior feature is associated with the low-level behavior of the control behavior feature, and the mapping relationship of different control targets and corresponding air conditioner output behavior actions is determined. Therefore, the low-level action can be selected under the guidance of the high-level target in the subsequent process. The space feature is determined by combining these control behavior features.

[0101] In step 205, inverse reinforcement learning is performed on the user state data, the space feature and the state space feature to determine a reward function.

[0102] The user state data, the space feature and the state space feature can be used as input parameters for inverse reinforcement learning to determine the corresponding reward function. The core objective of the inverse reinforcement learning is to infer the potential reward function from the observed user state data, the space feature and the state space feature, and then to explain or simulate the expert decision logic. For example, the reward function is determined by maximizing the likelihood function (such as maximum entropy) of the expert trajectory or minimizing the difference (such as apprentice learning) between the strategy and the expert behavior.

[0103] In an optional embodiment of the present application, the step of performing inverse reinforcement learning on the user state data, the space feature and the state space feature to determine a reward function comprises:

[0104] In sub-step S2051, an initial function is determined based on the user state data.

[0105] Firstly, an initial reward function corresponding to the user state data, i.e., an initial function, can be determined. The initial reward function is the starting point of the cold start phase, and its parameters will be continuously iteratively optimized in the subsequent feedback. For example, the reward function corresponding to the user portrait in the user state data is determined as the initial function.

[0106] In sub-step S2052, a correction parameter is determined by performing multi-objective fusion on the space feature and the state space feature.

[0107] Then, the space feature and the state space feature are used for multi-objective fusion to extract the preference weight of the user in multiple dimensions such as comfort, energy saving and quietness, and a vectorized reward function or a Pareto frontier is learned. The reward function clearly defines the "user satisfaction" that can be obtained by taking a certain behavior in the space feature in the state space feature, thereby determining the correction parameter.

[0108] In step S2053, the reward function is determined by combining the initial function and the correction parameter.

[0109] The initial function is iteratively optimized based on the correction parameter to determine the reward function.

[0110] In an optional embodiment of the present application, the step of performing inverse reinforcement learning on the user state data, the space feature and the state feature to determine the reward function comprises:

[0111] In step S2054, the reward function is updated by performing adversarial training on the user state data, the space feature and the state feature.

[0112] The mapping relationship between the user operation mode in the user state data, the space feature and the state feature can be used to learn a more robust reward function through adversarial training, reduce the influence of noise data, and make the reward function more accurately reflect the user's potential preferences, thereby updating the reward function.

[0113] In addition, non-action reward can also be introduced to update the reward function, so that the reward function can be more accurate. The state of long-time non-intervention of the user is quantified as positive feedback by feedback and input into the multi-objective fusion process, so that the reward function gives positive incentive to this implicit satisfaction state.

[0114] In step 206, the air conditioner control parameter is determined by combining the space feature, the state feature and the reward function.

[0115] The space feature, the state feature and the reward function can be combined, and the control parameter when the three are optimal can be used as the air conditioner control parameter.

[0116] In an optional embodiment of the present application, the step of determining the air conditioner control parameter by combining the space feature, the state feature and the reward function comprises: determining a behavior sequence in the state feature that matches the change trend in the state feature within a preset time length and has the reward function at a target value; and determining the air conditioner control parameter based on the behavior sequence.

[0117] The behavior sequence can be determined by determining the behavior in the state feature that matches the change trend in the state feature within a preset time length and has the reward function at a target value, and then combining these behaviors into a behavior sequence, and further converting the behavior sequence into an air conditioner control parameter.

[0118] The environment trend prediction data in the state space feature can be used in combination with the learned environment dynamic model to search for a sequence of actions in the action space feature that maximizes the cumulative reward (calculated based on the reward function) over a future period of time. This enables the policy generation to not only consider the current state, but also to anticipate future changes, and to output the optimal immediate action for the current state. When a significant change in user preference is detected (such as a comfort zone drift), the new feedback data and the current state space feature-action space feature can be used to quickly adjust the parameters of the policy generation to adapt to the new preference pattern, and to output an updated control policy. In addition, the policy generation is performed in the action space feature, but is subject to safety constraints to ensure that the output action (air conditioner adjustment parameter) is within a safe range. Based on the uncertainty of the current state (from the representation in the state space feature) and the confidence of the feedback, the ratio of exploration and exploitation is dynamically adjusted to efficiently learn new user preferences while ensuring the current comfort level.

[0119] Step 207, controlling the air conditioner based on the air conditioner control parameters;

[0120] These air conditioner control parameters are used to specifically control the operation of the air conditioner, thereby achieving the corresponding temperature control effect.

[0121] Step 208, displaying the action space feature, the state space feature, the reward function, and the air conditioner control parameters.

[0122] To enable the user to understand the control process, explain the rationality of the action to the user, enhance the user's trust, and reduce unnecessary intervention, thereby reducing invalid feedback. The action space feature, the state space feature, the reward function, and the air conditioner control parameters can be displayed.

[0123] In addition, explicit feedback from the user (such as "great" and "too cold") can also be directly collected as high-confidence feedback signals, combined with other implicit feedback, and then affect the update of the reward function by the inverse reinforcement learning, and finally guide the policy generation module to output a policy that is more in line with the user's new feedback. Optimal strategies under different situations can also be refined into typical patterns and stored in a pattern library. When the user switches modes, it is equivalent to directly specifying a high-level behavior goal, and the policy generation module generates a corresponding strategy in the action space feature based on this goal.

[0124] Further, to improve deployment efficiency and privacy protection, edge computing and cloud collaboration can be used for processing. The edge computing node can be responsible for real-time data processing of the data acquisition module, part of the real-time analysis of feedback quantization, real-time construction of state space features, and rapid execution of optimal actions, to ensure low-latency response. The cloud can be responsible for complex long-term trend mining, deep model training and reward function optimization, and anonymous group intelligence analysis of large-scale user data, to provide better initial models and update parameters for the edge. The privacy and security of data collection and flow can be ensured, and all the above processing can be efficiently coordinated under compliance, to ensure that user data is securely and effectively supported for model training and inference between local and cloud.

[0125] The embodiment of the present application breaks through the limitations of traditional air conditioners based on overall environment regulation, and incorporates the spatial position and physiological state of individual users into the regulation decision factors, to achieve more refined regional control. It can provide more accurate and personalized comfortable experience for users, avoid wasting energy in unoccupied areas, and improve energy utilization efficiency. A multi-objective inverse reinforcement learning (M-IRL) is used to construct a vectorized reward function, to learn the Pareto frontier between different objectives such as comfort, energy saving, and noise, and dynamically adjust the weights of each objective according to long-term user feedback, to achieve multi-dimensional dynamic balance. The traditional single-objective optimization method is changed, the multi-objective relationship is quantified through a mathematical model, so that the air conditioner can intelligently weigh and dynamically optimize multiple demands. While meeting the user's comfort, it effectively reduces energy consumption and controls noise, maximizes comprehensive benefits, and adapts to different users' personalized trade-off preferences. The "black box" characteristics of the intelligent air conditioner control model are broken, the complex decision logic is converted into a language understandable by users, and users are given the initiative to optimize strategies. It improves users' understanding and trust in intelligent regulation functions, reduces misoperation, and at the same time realizes continuous optimization of the model through user participation, enhancing system adaptability.

[0126] To enable those skilled in the art to clearly understand the implementation process of the embodiments of the present application, with reference to Figure 3 , the following is described with one example:

[0127] Step 1: Data acquisition

[0128] Environmental data:

[0129] Indoor: Wi-Fi RTT positioning shows that the user has lain on the bed (fixed position), millimeter wave radar detects that the user's breathing rate is stable (initially entering light sleep), thermal imaging shows that the user's body temperature is 34℃ (slightly high), the current indoor temperature is 26℃, and the humidity is 60%.

[0130] External: API (Application Program Interface) obtains that the outdoor temperature will drop by 5℃ in the next 2 hours (night cooling), AQI is good but pollen index is moderate (user has authorized allergy history).

[0131] Behavioral data:

[0132] Smart home linkage: Curtains closed, lights off, air purifier running at low speed (linkage data).

[0133] Calendar shows user wakes up at 7 am the next day (predicts sleep duration of approximately 8 hours).

[0134] Step 2: Feedback extraction

[0135] Short-term immediate feedback: User does not operate air conditioner 10 minutes after lying down (no intervention, time lag ∞), considered as preliminary acceptance of the current initial adjustment (quantified as "neutral to positive feedback", confidence 80%).

[0136] Medium-term habit feedback: Historical data shows that the user unconsciously adjusts the wind speed to the lowest level between 2-4 am in the past week (repeated behavior pattern), quantified as "sleep mid-term preference for low wind speed" (weight 0.7).

[0137] Long-term trend feedback: Data from the past month shows that the user's sleep comfort zone decreases with seasonal temperature, from "27℃→26℃" (comfort zone drift), quantified as "temperature preference down 1℃" (long-term feedback coefficient 0.6).

[0138] Feedback fusion: Integrate short-term no intervention, medium-term low wind speed preference, and long-term temperature down trend to generate quantified feedback vector: "Temperature adjustment requirement: -1℃ (medium confidence), wind speed adjustment requirement: down 1 gear (high confidence)".

[0139] Step 3: Space definition

[0140] State space characteristics (S):

[0141] Compressed into a low-dimensional context vector by self-encoder: "(indoor temperature 26℃, user light sleep state, body surface temperature slightly high, external temperature decreases after 2 hours, pollen index moderate, remaining sleep duration 7.5 hours)" - integrates environmental, physiological, and time dynamic characteristics.

[0142] Action space characteristics (A):

[0143] High-level goal: "Maintain deep sleep comfort + low noise energy saving + allergy prevention".

[0144] Low-level behavior primitives: Executable parameter combinations (such as "temperature -1℃ + wind speed down 1 gear + new air volume reduced by half (anti-pollen)" "temperature unchanged + wind speed down 2 gears + dehumidification 5%" etc.).

[0145] Step 4: Inverse reinforcement learning

[0146] Based on the quantified feedback in Step 2 and the spatial feature definition in Step 3, the reward function is updated:

[0147] In the current state S, assign a high reward to "wind speed down 1 level" (as the intermediate feedback shows that the user prefers low wind speed for sleep);

[0148] Assign a medium reward to "temperature -1℃" (long-term trend supports temperature reduction, but needs to balance the natural body temperature drop at night);

[0149] Assign an additional reward to "fresh air volume halved" (combined with pollen index, match the user's allergic constitution needs).

[0150] The final learned personalized reward function: R(S, A) = 0.3 × temperature adjustment reward + 0.5 × wind speed adjustment reward + 0.2 × anti-allergy reward.

[0151] Step 5: Strategy generation

[0152] Model predictive control (MPC) combined with external temperature drop prediction, predicts that "if not adjusted, the indoor temperature may drop to 24℃ with the outdoor temperature in 2 hours, and the user is likely to be woken up by the cold".

[0153] Search for "the action sequence with the highest cumulative reward in the future 2 hours" in the action space A:

[0154] First phase (0-1 hour): Execute "temperature 25℃ (down 1℃) + wind speed lowest level (down 1 level) + fresh air volume halved" (match high weight items of reward function);

[0155] Second phase (1-2 hours): Adjust to 24.5℃ in advance (avoid sudden drop with outdoor temperature), maintain low wind speed (prevent noise).

[0156] Because the current strategy is stable (the user has no significant preference changes), use "utilization" as the main method, do not increase the exploration ratio.

[0157] Step 6: Execution and feedback

[0158] Execution: The air conditioner adjusts according to the first phase action of the strategy, and at the same time, a brief explanation is pushed through the APP: "It is detected that you have entered light sleep, the body temperature is slightly high, and the temperature will drop at night. The temperature has been lowered by 1℃ and the wind speed has been reduced (to prevent noise), and the fresh air has been reduced (for pollen protection)".

[0159] Feedback loop: The user did not intervene overnight ("satisfied silence"), and the millimeter wave radar showed that he entered deep sleep in the second half of the night (physiological feedback verified effective). The system takes this result as high-confidence feedback, inputs it into the feedback module in Step 2, and further strengthens the reward weight of "sleeping medium low wind speed + temperature gentle reduction", optimizing the strategy for the next day's sleep adjustment.

[0160] It should be noted that, for the method embodiments, for the sake of simple description, they are all described as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited to the action sequence described, because according to the embodiments of the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily necessary for the embodiments of the present application.

[0161] Referring to Figure 4 , a structural block diagram of an air conditioner control embodiment of the present application is shown, and the air conditioner control device can specifically include the following modules:

[0162] The acquisition module 401 is configured to acquire doorway user state data, environment data and air conditioner output parameters.

[0163] The first feature extraction module 402 is configured to perform feature extraction based on the user state data and the environment data to determine user implicit preference features.

[0164] The fusion module 403 is configured to fuse the user state data and the user implicit preference features to determine state space features.

[0165] The second feature extraction module 404 is configured to perform feature extraction based on the air conditioner output parameters to determine action space features.

[0166] The inverse reinforcement learning module 405 is configured to perform inverse reinforcement learning on the user state data, the action space features and the state space features to determine a reward function.

[0167] The decision module 406 is configured to combine the action space features, the state space features and the reward function to determine air conditioner control parameters.

[0168] The control module 407 is configured to control the air conditioner based on the air conditioner control parameters.

[0169] In an optional embodiment of the present application, the first feature extraction module 402 includes:

[0170] The short-term extraction submodule is configured to perform short-term and instant feature extraction on the user state data and the environment data to determine short-term preference features.

[0171] The medium-term extraction submodule is configured to perform medium-term habit feature extraction on the user state data and the environment data to determine medium-term preference features.

[0172] The long-term extraction submodule is configured to perform long-term trend feature extraction on the user state data and the environment data to determine long-term preference features.

[0173] The first combination sub-module is configured to combine the short-term preference feature, the medium-term preference feature and the long-term preference feature to determine the implicit preference feature of the user.

[0174] In an optional embodiment of the present application, the first combination sub-module comprises:

[0175] The weighting unit is configured to weight and calculate the short-term preference feature, the medium-term preference feature and the long-term preference feature to determine the implicit preference feature of the user.

[0176] In an optional embodiment of the present application, the fusion module 403 comprises:

[0177] The multi-modal fusion sub-module is configured to perform multi-modal fusion on the user state data and the implicit preference feature of the user to determine a context embedding vector.

[0178] The first combination sub-module is configured to combine the context embedding vector to determine a state space feature.

[0179] In an optional embodiment of the present application, the second feature extraction module 404 comprises:

[0180] The control behavior extraction sub-module is configured to determine a control behavior feature based on the air conditioner output parameter.

[0181] The second combination sub-module is configured to combine the control behavior feature to determine a behavior space feature.

[0182] In an optional embodiment of the present application, the inverse reinforcement learning module 405 comprises:

[0183] An initial function is determined based on the user state data.

[0184] The multi-objective fusion sub-module is configured to perform multi-objective fusion on the behavior space feature and the state space feature to determine a correction parameter.

[0185] The second combination sub-module is configured to combine the initial function and the correction parameter to determine a reward function.

[0186] In an optional embodiment of the present application, the inverse reinforcement learning module 405 further comprises:

[0187] The update sub-module is configured to perform adversarial training on the reward function based on the user state data, the behavior space feature and the state space feature to update the reward function.

[0188] In an optional embodiment of the present application, the decision module 406 comprises:

[0189] a behavior sequence determination sub-module configured to determine, in the state space feature, a behavior sequence that matches a variation trend of the state space feature within a preset time length and has a target value of the reward function;

[0190] a decision sub-module configured to determine an air conditioner control parameter based on the behavior sequence.

[0191] In an optional embodiment of the present application, the device further comprises:

[0192] a display module configured to display the behavior space feature, the state space feature, the reward function and the air conditioner control parameter.

[0193] In the embodiments of the present application, user state data, environment data and air conditioner output parameters are acquired; feature extraction is performed based on the user state data and the environment data to determine user implicit preference features; the user state data and the user implicit preference features are fused to determine state space features; the behavior space features are determined based on the air conditioner output parameters; inverse reinforcement learning is performed on the user state data, the behavior space features and the state space features to determine reward functions; the air conditioner control parameters are determined in combination of the behavior space features, the state space features and the reward functions; and the air conditioner is controlled based on the air conditioner control parameters. By sensing the user state data and the environment data and analyzing the user behavior and demand based thereon, the user state can be determined, and the user state and the output action of the air conditioner are taken as the consideration factors for processing the control decision, so that more accurate and personalized comfortable experience can be provided for the user.

[0194] For the device embodiment, the description is relatively simple because it is basically similar to the method embodiment, and the related parts can be referred to the part of the description of the method embodiment.

[0195] With reference to Figure 5 The embodiments of the present application further provide a washing machine, which comprises:

[0196] a processor 501 and a storage medium 502, where the storage medium 502 stores a computer program executable by the processor 501, and when the washing machine control is running, the processor 501 executes the computer program to implement the air conditioner control method according to any one of the embodiments of the present application.

[0197] The air conditioner control method comprises:

[0198] acquiring user state data, environment data and air conditioner output parameters;

[0199] performing feature extraction based on the user state data and the environment data to determine user implicit preference features;

[0200] fusing the user state data and the user implicit preference feature, determining a state space feature;

[0201] determining a space space feature based on the air conditioner output parameter;

[0202] performing inverse reinforcement learning on the user state data, the space space feature and the state space feature, determining a reward function;

[0203] determining an air conditioner control parameter in combination with the space space feature, the state space feature and the reward function;

[0204] controlling an air conditioner based on the air conditioner control parameter.

[0205] Optionally, the step of determining a user implicit preference feature based on the user state data and the environment data includes:

[0206] performing short-term instant feature extraction on the user state data and the environment data, determining a short-term preference feature;

[0207] performing medium-term habit feature extraction on the user state data and the environment data, determining a medium-term preference feature;

[0208] performing long-term trend feature extraction on the user state data and the environment data, determining a long-term preference feature;

[0209] determining a user implicit preference feature in combination with the short-term preference feature, the medium-term preference feature and the long-term preference feature.

[0210] Optionally, the step of determining a user implicit preference feature in combination with the short-term preference feature, the medium-term preference feature and the long-term preference feature includes:

[0211] performing weighted calculation on the short-term preference feature, the medium-term preference feature and the long-term preference feature, determining a user implicit preference feature.

[0212] Optionally, the step of fusing the user state data and the user implicit preference feature, determining a state space feature includes:

[0213] performing multi-modal fusion on the user state data and the user implicit preference feature, determining a context embedding vector;

[0214] combining the context embedding vector to determine a state space feature.

[0215] Optionally, the step of determining a space space feature based on the air conditioner output parameter includes:

[0216] determine a control behavior feature based on the air conditioner output parameter;

[0217] combine the control behavior feature to determine a space behavior feature.

[0218] Optionally, the step of determining a reward function based on the user state data, the space behavior feature and the state space feature by inverse reinforcement learning comprises:

[0219] determine an initial function based on the user state data;

[0220] perform multi-objective fusion on the space behavior feature and the state space feature to determine a correction parameter;

[0221] combine the initial function and the correction parameter to determine the reward function.

[0222] Optionally, the step of determining a reward function based on the user state data, the space behavior feature and the state space feature by inverse reinforcement learning further comprises:

[0223] perform adversarial training on the reward function based on the user state data, the space behavior feature and the state space feature to update the reward function.

[0224] Optionally, the step of determining an air conditioner control parameter based on the space behavior feature, the state space feature and the reward function comprises:

[0225] in the state space feature, determine a behavior sequence that matches a change trend of the state space feature within a preset time length and has the reward function at a target value;

[0226] determine an air conditioner control parameter based on the behavior sequence.

[0227] Optionally, the method further comprises:

[0228] display the space behavior feature, the state space feature, the reward function and the air conditioner control parameter.

[0229] The embodiment of the present application acquires user state data, environment data and air conditioner output parameters; performs feature extraction based on the user state data and the environment data, determines user implicit preference features; fuses the user state data and the user implicit preference features, determines state space features; determines action space features based on the air conditioner output parameters; performs inverse reinforcement learning on the user state data, the action space features and the state space features, determines a reward function; determines air conditioner control parameters in combination with the action space features, the state space features and the reward function; and controls the air conditioner based on the air conditioner control parameters. By perceiving user state data and environment data, and analyzing user behavior and demand based thereon, the state of the user can be determined, and the state of the user and the output action of the air conditioner are processed as consideration factors of control decisions, which can provide the user with more accurate and personalized comfortable experience.

[0230] The memory can include a random access memory (RAM) and can also include a non-volatile memory such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0231] The aforementioned processor can be a general-purpose processor including a central processing unit (CPU), a network processor (NP), etc., and can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0232] Reference Figure 6 The embodiment of the present application also provides a computer readable storage medium 601, and the storage medium 601 stores a computer program. When the computer program is run by a processor, the air conditioner control method is executed as any one of the embodiments of the present application.

[0233] The air conditioner control method comprises:

[0234] Acquiring user state data, environment data and air conditioner output parameters;

[0235] feature extraction based on the user state data and the environment data, to determine a user implicit preference feature;

[0236] fusing the user state data and the user implicit preference feature, to determine a state space feature;

[0237] based on the air conditioner output parameter, to determine a line space feature;

[0238] inverse reinforcement learning on the user state data, the line space feature and the state space feature, to determine a reward function;

[0239] determining an air conditioner control parameter in combination with the line space feature, the state space feature and the reward function;

[0240] controlling an air conditioner based on the air conditioner control parameter.

[0241] Optionally, the step of feature extraction based on the user state data and the environment data, to determine a user implicit preference feature, comprises:

[0242] short-term instant feature extraction on the user state data and the environment data, to determine a short-term preference feature;

[0243] medium-term habit feature extraction on the user state data and the environment data, to determine a medium-term preference feature;

[0244] long-term trend feature extraction on the user state data and the environment data, to determine a long-term preference feature;

[0245] determining a user implicit preference feature in combination with the short-term preference feature, the medium-term preference feature and the long-term preference feature.

[0246] Optionally, the step of determining a user implicit preference feature in combination with the short-term preference feature, the medium-term preference feature and the long-term preference feature, comprises:

[0247] weighted calculation on the short-term preference feature, the medium-term preference feature and the long-term preference feature, to determine a user implicit preference feature.

[0248] Optionally, the step of fusing the user state data and the user implicit preference feature, to determine a state space feature, comprises:

[0249] multimodal fusion on the user state data and the user implicit preference feature, to determine a context embedding vector;

[0250] combining the context embedding vector to determine a state space feature.

[0251] Optionally, the step of determining the space feature based on the air conditioner output parameter comprises:

[0252] determining a control behavior feature based on the air conditioner output parameter;

[0253] combining the control behavior feature to determine the space feature.

[0254] Optionally, the step of determining the reward function based on the user state data, the space feature and the state space feature through inverse reinforcement learning comprises:

[0255] determining an initial function based on the user state data;

[0256] performing multi-objective fusion on the space feature and the state space feature to determine a correction parameter;

[0257] combining the initial function and the correction parameter to determine the reward function.

[0258] Optionally, the step of determining the reward function based on the user state data, the space feature and the state space feature through inverse reinforcement learning further comprises:

[0259] performing adversarial training on the reward function based on the user state data, the space feature and the state space feature to update the reward function.

[0260] Optionally, the step of determining the air conditioner control parameter based on the space feature, the state space feature and the reward function comprises:

[0261] determining a behavior sequence in the state space feature that matches a change trend of the state space feature within a preset time length and has the reward function at a target value;

[0262] determining the air conditioner control parameter based on the behavior sequence.

[0263] Optionally, the method further comprises:

[0264] displaying the space feature, the state space feature, the reward function and the air conditioner control parameter.

[0265] The embodiment of the present application acquires user state data, environment data and air conditioner output parameters; performs feature extraction based on the user state data and the environment data, determines user implicit preference features; fuses the user state data and the user implicit preference features, determines state space features; determines action space features based on the air conditioner output parameters; performs inverse reinforcement learning on the user state data, the action space features and the state space features, determines a reward function; determines air conditioner control parameters in combination with the action space features, the state space features and the reward function; and controls the air conditioner based on the air conditioner control parameters. By perceiving user state data and environment data, and analyzing user behavior and demand based thereon, the state of the user can be determined, and the state of the user and the output action of the air conditioner are processed as consideration factors of control decisions, which can provide the user with more accurate and personalized comfortable experience.

[0266] Each of the embodiments in the present specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts between the embodiments can be referred to each other.

[0267] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, device or computer program product. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0268] The embodiments of the present application are described with reference to flowcharts and / or block diagrams of the method, terminal device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal device produce a device for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The device for implementing the functions specified in one or more flows and / or blocks Figure 1 The device for implementing the functions specified in one or more flows and / or blocks

[0269] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or multiple blocks.

[0270] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or multiple blocks.

[0271] Although preferred embodiments of the present application have been described, those skilled in the art will be able to make additional changes and modifications to these embodiments once they have the benefit of the foregoing description. Accordingly, it is intended to embrace all such changes and modifications in the scope of the embodiments of the present application.

[0272] Finally, it should be noted that the terms "first", "second", and the like, herein do not necessarily denote any order, quantity, or importance, but rather are used to distinguish one entity from another, and are more especially used to distinguish an entity broadly described herein from another entity broadly described herein. Also, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0273] The above provides a kind of air conditioner control method, an air conditioner control device, an electronic equipment and a computer readable storage medium provided by the present application, detailed introduction is carried out, specific examples are applied in this paper to the principle and implementation mode of the present application are described, the above example is only for helping understanding the method of the present application and its core idea;For the general technical personnel of the field, according to the idea of the present application, there will be changes in specific implementation mode and application range, and the above description should not be understood as the limitation of the present application.

Claims

1. An air conditioner control method characterized by comprising: The method comprises: acquiring user state data, environment data and air conditioner output parameters; performing feature extraction based on the user state data and the environment data to determine user implicit preference features; fusing the user state data and the user implicit preference features to determine state space features; determining action space features based on the air conditioner output parameters; performing inverse reinforcement learning on the user state data, the action space features and the state space features to determine a reward function; determining air conditioner control parameters in combination with the action space features, the state space features and the reward function; controlling an air conditioner based on the air conditioner control parameters.

2. The method of claim 1, wherein, The step of performing feature extraction based on the user state data and the environment data to determine user implicit preference features comprises: performing short-term instant feature extraction on the user state data and the environment data to determine short-term preference features; performing medium-term habit feature extraction on the user state data and the environment data to determine medium-term preference features; performing long-term trend feature extraction on the user state data and the environment data to determine long-term preference features; determining user implicit preference features in combination with the short-term preference features, the medium-term preference features and the long-term preference features.

3. The method of claim 1, wherein, The step of determining user implicit preference features in combination with the short-term preference features, the medium-term preference features and the long-term preference features comprises: performing weighted calculation on the short-term preference features, the medium-term preference features and the long-term preference features to determine user implicit preference features.

4. The method of claim 1, wherein, The step of fusing the user state data and the user implicit preference features to determine state space features comprises: performing multi-modal fusion on the user state data and the user implicit preference features to determine a context embedding vector; combining the context embedding vector to determine state space features.

5. The method of claim 1, wherein, The step of determining action space features based on the air conditioner output parameters comprises: determining control behavior features based on the air conditioner output parameters; combining the control behavior features to determine action space features.

6. The method of claim 1, wherein, The step of performing inverse reinforcement learning on the user state data, the action space features and the state space features to determine a reward function comprises: determining an initial function based on the user state data; performing multi-objective fusion on the action space features and the state space features to determine a correction parameter; determining a reward function in combination with the initial function and the correction parameter.

7. The method of claim 6, wherein, The step of performing inverse reinforcement learning on the user state data, the action space features and the state space features to determine a reward function further comprises: performing adversarial training on the reward function based on the user state data, the action space features and the state space features to update the reward function.

8. The method of claim 1, wherein, The step of determining air conditioner control parameters in combination with the action space features, the state space features and the reward function comprises: determining, in the state space features, a behavior sequence that matches a change trend of the state space features within a preset time length and has a target value of the reward function; determining air conditioner control parameters based on the behavior sequence.

9. The method of claim 1, wherein, The method further comprises: The state space feature, the reward function and the air conditioner control parameter are displayed.

10. An air conditioner control device characterized by comprising: The method comprises the following steps: An acquisition module is configured to acquire doorway user state data, environment data and air conditioner output parameters. A first feature extraction module is configured to extract features based on the user state data and the environment data to determine user implicit preference features. A fusion module is configured to fuse the user state data and the user implicit preference features to determine a state space feature. A second feature extraction module is configured to extract features based on the air conditioner output parameters to determine a room space feature. An inverse reinforcement learning module is configured to perform inverse reinforcement learning on the user state data, the room space feature and the state space feature to determine a reward function. A decision module is configured to determine an air conditioner control parameter in combination with the room space feature, the state space feature and the reward function. A control module is configured to control an air conditioner based on the air conditioner control parameter.

11. An electronic device, comprising: The computer program is stored on the computer readable storage medium and is executed by the processor to implement the steps of the air conditioner control method according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium and is executed by the processor to implement the steps of the air conditioner control method according to any one of claims 1-9.

Images