Multi-household-appliance linkage control method and system

Abstract

The invention provides a multi-household appliance linkage control method and system. The method comprises the following steps: arranging an intelligent monitoring device to collect indoor and outdoor environment data; carrying out preprocessing and feature extraction on the calibrated and calibrated data; establishing a relation model between the environment parameters and household appliance control by adopting a regression model according to the comprehensive feature vector; processing the data acquired by the security and protection sensor and extracting features; a specific multi-household-appliance control strategy is generated according to the self-adaptive control strategy and the security comprehensive feature vector, the generation of the multi-household-appliance control strategy adopts a hybrid model, a deep reinforcement learning optimization control strategy is used, the optimization control strategy is applied to specific intelligent household appliances, and real-time monitoring is carried out; and designing a user interface to check and control the state of the household appliance. The household appliance linkage control system not only overcomes many defects in the prior art, but also provides an efficient, intelligent, energy-saving and safe household appliance linkage control system, and brings more comfortable and convenient life experience to users.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-station linkage control, and in particular relates to a multi-station linkage control method and system. Background Art

[0002] With the continuous improvement of modern living standards, household appliances are becoming increasingly common in families. Traditional home appliance control methods mostly rely on the independent operation of a single device, requiring users to manually turn each appliance on and off, which is both cumbersome and inefficient. Although smart home systems have become increasingly popular in recent years, and centralized control of home appliances through mobile phones or voice assistants has become a trend, current smart home systems still have many shortcomings and cannot fully meet users' needs for convenience, intelligence, energy saving, and security.

[0003] Existing smart home appliance systems mostly focus on single-scenario control and lack a global, interconnected control mechanism. For example, some systems can control air conditioners, lights, televisions, and other devices through voice assistants. However, the control of these devices is often isolated and lacks interconnectivity, making it impossible to automatically adjust to environmental changes and user behavior. This requires users to constantly manually adjust each device, making it difficult to achieve true intelligent control. At the same time, existing systems also perform poorly in terms of energy conservation, unable to adaptively adjust based on real-time environmental data, resulting in excessive energy consumption. Furthermore, existing security systems primarily focus on monitoring and alarm functions. While they can issue alarms when anomalies are detected, they lack interconnected control with home appliances and cannot provide comprehensive security protection.

[0004] These shortcomings of the prior art are mainly manifested in the following aspects:

[0005] 1. Independent control: Most home appliances operate independently, lack linkage, and cannot automatically adjust according to user behavior and environmental changes, and have a low level of intelligence.

[0006] 2. Insufficient energy saving: Unable to make adaptive adjustments based on real-time environmental data, resulting in high energy consumption and difficulty in achieving energy-saving optimization.

[0007] 3. Single security: The security system mainly focuses on monitoring and alarm functions, lacks linkage control with home appliances, and cannot provide comprehensive security protection.

[0008] 4. Poor user experience: Users are required to frequently perform manual operations or manage through multiple interfaces, which is cumbersome and leads to poor user experience. Summary of the Invention

[0009] The purpose of this invention is to design an intelligent energy-saving home appliance linkage control system based on environmental perception and home security. Through intelligent monitoring devices, reinforcement learning algorithms and multimodal data fusion technology, intelligent linkage control of multiple home appliances is realized, which significantly improves the intelligence level of home appliances, energy-saving effects and home safety.

[0010] In order to achieve the above object, a first aspect of the present invention provides a multi-device linkage control method, the method comprising the following steps:

[0011] S1. Deploy intelligent monitoring devices to collect indoor and outdoor environmental data, perform real-time monitoring of home entrances and important areas, calibrate and calibrate the collected and monitored data, convert the calibrated and calibrated data into digital signals, transmit them to a central processing unit, and store them; wherein the intelligent monitoring devices include environmental sensors and security sensors, the environmental sensors including temperature sensors, humidity sensors, light sensors, and air quality sensors, and the security sensors including cameras, door and window sensors, and smoke detectors;

[0012] S2. Preprocess and extract features from the calibrated and calibrated data, fuse the extracted feature vectors to obtain a comprehensive feature vector, use a long short-term memory network to build a time series model to analyze the comprehensive feature vector, and use an autoencoder based on reconstruction error to detect anomalies based on the analysis results. The analysis results are then transmitted to the cloud server.

[0013] S3. Using a regression model based on the comprehensive feature vector, a relationship model between environmental parameters and home appliance control is established. Based on the relationship model, a reinforcement learning algorithm is used to generate an adaptive control strategy so that the home appliance can perform adaptive adjustments based on real-time environmental data.

[0014] S4. Process the data collected by the security sensors and perform feature extraction. Fuse the feature vectors of the security sensors to obtain a comprehensive security feature vector. Use a long short-term memory network to perform time series modeling on the fused comprehensive security feature vector to obtain time series features under normal conditions. Use an autoencoder to perform anomaly detection based on the time series features under normal conditions. When an anomaly is detected, trigger an alarm and record the abnormal data and timestamp.

[0015] S5. Generate a specific multi-appliance control strategy based on the adaptive control strategy and the comprehensive security feature vector. Generate the multi-appliance control strategy using a hybrid model. The hybrid model combines a rule-based control strategy with a deep reinforcement learning-optimized control strategy, executes the control strategy based on predefined rules, optimizes the control strategy using deep reinforcement learning, applies the optimized control strategy to specific smart home appliances and monitors them in real time, and then dynamically adjusts the control strategy based on feedback data and real-time monitoring results.

[0016] S6. Design a user interface to view and control the status of home appliances, and collect user operations and feedback information in real time, and store it in a database. Perform sentiment analysis on user operations and feedback information, extract user emotional tendencies, build a user behavior model based on user emotional tendencies, perform feature extraction on user behavior to obtain cluster analysis, identify different types of user behavior patterns, perform cluster analysis based on user behavior characteristics, identify different types of user behavior patterns, generate personalized control strategies for each user based on the user behavior model, and further combine user feedback data to optimize the existing control strategies.

[0017] Furthermore, the calibration and verification include:

[0018] The temperature sensor is calibrated using a standard substance with a known temperature, multiple calibration points are obtained, and the calibration coefficient ktemp and the offset btemp are calculated:

[0019] {(Vtemp1,T1),(Vtemp2,T2),…(Vtempn,Tn)}

[0020] Where Vtempn represents the raw output voltage of the nth temperature sensor, and Tn represents the actual temperature of the nth temperature sensor. The calibration coefficient and offset are solved using the linear regression method, as shown below:

[0021]

[0022] Where Ti represents the temperature of the i-th temperature sensor, and Vtempi represents the original output voltage value of the i-th temperature sensor;

[0023] The humidity sensor calibration is performed using a known humidity environment to obtain calibration points and calculate the calibration coefficient khum and bias bhum, which are expressed as follows:

[0024] {(Vhum1,H1),(Vhum2,H2),…(Vhumn,Hn)}

[0025] Where Vhumn represents the original output voltage value of the nth humidity sensor, and Hn represents the actual humidity of the nth humidity sensor;

[0026] The calibration coefficient and offset are solved using the linear regression method, which is expressed as follows:

[0027]

[0028] Where, Hi represents the actual humidity of the i-th humidity sensor, and Vhumi represents the original output voltage value of the i-th humidity sensor;

[0029] The light sensor is calibrated using a standard light source of known light intensity to obtain calibration points and calculate the calibration coefficient klight and bias blight:

[0030] {(Vlight1,L1),(Vlight2,L2),…(Vlightn,Ln)}

[0031] Where Vlightn represents the original output voltage value of the nth light sensor, and Hn represents the actual light intensity of the nth light sensor;

[0032] The calibration coefficient and offset are solved using the linear regression method, which is expressed as follows:

[0033]

[0034] Where Li represents the actual light intensity of the i-th light sensor, and Vlighti represents the original output voltage value of the i-th light sensor;

[0035] The air quality sensor is calibrated using a known air quality environment to obtain calibration points, and the calibration coefficient kair and bias bair are calculated, which are expressed as follows:

[0036] {(Vair1,A1),(Vair2,A2),…(Vairn,An)}

[0037] Wherein, Vairn represents the original output voltage value of the nth air quality sensor, and An represents the actual air quality index of the nth air quality sensor;

[0038] The calibration coefficient and offset are solved using the linear regression method, which is expressed as follows:

[0039]

[0040] Wherein, Ai represents the actual air quality index of the i-th air quality sensor, and Vairi represents the raw output voltage value of the i-th air quality sensor.

[0041] Furthermore, in S2, for the data of the environmental sensor, the normalized value is directly used as the feature; for the camera data, the convolutional neural network is used to extract the image features; for the binary data of the door and window sensor, the original value is directly used as the feature.

[0042] Furthermore, the attention mechanism is used to weight the fusion of each modality feature, and the feature vectors of different sensors are fused to obtain the fused feature vector F fused (t), is expressed as follows:

[0043]

[0044] Among them, F i (t) represents the feature vector of the i-th sensor at time t, α i (t) represents the attention weight of the i-th modal feature at time t, which is calculated by the following formula:

[0045]

[0046] e i (t) = w T tanh(W i ·F i (t)+b i )

[0047] Among them, e i (t) represents the attention score of the i-th modality feature, α i (t) represents the attention weight of the i-th modality feature, w represents the global weight vector, W i represents the weight matrix of the i-th modal feature, b i Represents the bias vector of the i-th modal feature;

[0048] For the fused feature vector F fused (t), long short-term memory network is used for time series modeling, which is expressed as follows:

[0049] h(t) fused =LSTM(F fused (t),h(t-1) fused )

[0050] Among them, h(t) represents the hidden state of LSTM at time t, F fused (t) represents the fused feature vector, h(t-1) fused represents the hidden state of LSTM at time t-1;

[0051] The autoencoder decoder based on reconstruction error is used to detect anomalies in the fused time series data, as shown below:

[0052]

[0053] The anomaly score is the reconstruction error and is expressed as follows:

[0054]

[0055] Where h(t) fused represents the hidden state of LSTM at time t, represents the hidden state reconstructed by the autoencoder, E(t) fused Represents the reconstruction error, when E(t) fusedWhen the preset threshold is exceeded, the data at the current time t is considered to be abnormal.

[0056] Furthermore, the S3 specifically includes:

[0057] S301: Use a regression model to establish a relationship model between environmental parameters and home appliance control, as shown below:

[0058]

[0059] Among them, S(t) represents the state vector of the home appliance at time t, β0 represents the intercept term of the regression model, and β i represents the coefficient of the regression model, F fused,i (t) represents the i-th element of the fused feature vector, ∈ represents the error term;

[0060] S302, design reinforcement learning, define the state space S, action space A and reward function R(s,a), where the state space S is composed of the fusion feature vector F fused (t) constitutes a high-dimensional space; the action space A includes the control instruction set of the home appliance; the reward function R(s,a) is used to evaluate the effect of each action in a specific state;

[0061] S303, using the Q-learning algorithm to perform policy iteration and optimize the control strategy, as shown below:

[0062]

[0063] Among them, Q(s,a) represents the Q value function, E represents the expectation operator, γ represents the discount factor, R(s t+k ,a t+k ) means in state s t+k Next take action a t+k Instant rewards;

[0064] S304. Update the Q-value function according to the Q-learning algorithm, which is expressed as follows:

[0065]

[0066] Among them, α represents the learning rate, s′ represents the new state reached after executing action a, and max a′ Q(s′,a′) represents the maximum Q value under the new state s′;

[0067] S305. Determine the optimal strategy π based on the optimized Q value function * (s), which is as follows:

[0068]

[0069] Among them, π * (s) represents the optimal strategy in state s, argmax a Represents the action a that maximizes the Q-value function, and applies the optimal strategy to the control of home appliances to achieve adaptive adjustment.

[0070] Furthermore, in the above S4, the feature vectors of different sensors are fused by weighted averaging method to obtain the security comprehensive feature vector F security (t), is expressed as follows:

[0071] F security (t) = w I ·f I (t)+w door ·f door (t)+w smoke ·f smoke (t)

[0072] Among them, F security (t) represents the fused security feature vector, w I Represents the weight of the image feature vector, w door Represents the weight of the door and window sensor feature vector, w smoke represents the weight of the smoke detector feature vector, f I (t) represents the image feature vector, f door (t) represents the door and window sensor feature vector, f smoke (t) represents the smoke detector feature vector.

[0073] Furthermore, the improved long short-term memory network is used to analyze the security comprehensive feature vector F security (t) Perform time series modeling and output the hidden state h at the current moment security (t), is expressed as follows:

[0074] h security (t)=LSTM(F security (t),h security (t-1))

[0075] Among them, h security (t) represents the hidden state of the LSTM network at time t, F security (t) represents the fused feature vector, h security (t-1) represents the hidden state of the LSTM network at time t-1;

[0076] The hidden state h at the current moment is obtained by using the autoencoder based on reconstruction error security (t) performs anomaly detection, which is expressed as follows:

[0077]

[0078] The anomaly score is the reconstruction error:

[0079]

[0080] Among them, h security (t) represents the hidden state of the LSTM network at time t, represents the hidden state reconstructed by the autoencoder; E security (t) represents the security reconstruction error, when E security When (t) exceeds the preset threshold, the data at the current time t is considered to be abnormal.

[0081] Furthermore, the update formula of the improved long short-term memory network is expressed as follows:

[0082] i t =σ(W i ·[h security (t-1),F security (t)]+b i )

[0083] f t =σ(W f ·[h security (t-1),F security (t)]+b f )

[0084] o t =σ(W o ·[h security (t-1),F security (t)]+b o )

[0085]

[0086] h(t)=o t tanh(c t )

[0087] Among them, i t represents the input gate; f t represents the forget gate; o t represents the output gate; represents the candidate memory unit; c t Represents a memory unit; W i ,W f ,W o ,W c represents the weight matrix; b i ,b f ,b o ,b crepresents the bias vector; σ represents the Sigmoid function.

[0088] Furthermore, the control strategy optimized using deep reinforcement learning is expressed as follows:

[0089] Assuming that the environment state is E(t), the control strategy is C(t), and the reward function is R(t), the Q value update formula is expressed as follows:

[0090]

[0091] Among them, Q(s t ,a t ) means in state s t Next take action a t Q value; α represents the learning rate; R(t) represents the reward value; γ represents the discount factor; max a， Q(s t+1 ,a′) means in state s t+1 The Q value of taking the optimal action under s t Indicates the current state; a t Indicates the current action; s t+1 represents the next state; a′ represents the next action;

[0092] Apply the generated control strategy to specific smart home appliances, as shown below:

[0093] Assume that the control strategy C(t) contains the control instruction I i (t), where i represents a specific device, and the control instruction application formula is as follows:

[0094] I i (t)=DeviceControl(C(t),D i )

[0095] Among them, I i (t) represents the control instruction of the i-th device, C(t) represents the control strategy, and D i Indicates the status of the i-th device, and DeviceControl represents the device control function.

[0096] In a second aspect of the present invention, a multi-device linkage control system is provided, the system comprising:

[0097] A data acquisition module, configured to deploy intelligent monitoring devices to collect indoor and outdoor environmental data, perform real-time monitoring of home entrances and important areas, calibrate and calibrate the collected and monitored data, convert the calibrated and calibrated data into digital signals, transmit them to a central processing unit, and store them; wherein the intelligent monitoring devices include environmental sensors and security sensors, the environmental sensors including temperature sensors, humidity sensors, light sensors, and air quality sensors, and the security sensors including cameras, door and window sensors, and smoke detectors;

[0098] The comprehensive feature extraction module is used to preprocess and extract features from the calibrated and calibrated data, fuse the extracted feature vectors to obtain a comprehensive feature vector, use a long short-term memory network to build a time series model to analyze the comprehensive feature vector, and use an autoencoder based on reconstruction error to detect anomalies based on the analysis results. The analysis results are then transmitted to the cloud server.

[0099] A relationship modeling module is used to establish a relationship model between environmental parameters and home appliance control using a regression model based on the comprehensive feature vector, and to generate an adaptive control strategy based on the relationship model using a reinforcement learning algorithm to enable home appliances to perform adaptive adjustments based on real-time environmental data;

[0100] The security feature extraction module is used to process and extract features from the data collected by the security sensors, fuse the feature vectors of the security sensors to obtain a comprehensive security feature vector, and use a long short-term memory network to perform time series modeling on the fused comprehensive security feature vector to obtain the time series features under normal conditions. Based on the time series features under normal conditions, an autoencoder is used to detect anomalies. When an anomaly is detected, an alarm is triggered and the abnormal data and timestamp are recorded;

[0101] a strategy adjustment module for generating a specific multi-appliance control strategy based on the adaptive control strategy and the comprehensive security feature vector, wherein the generation of the multi-appliance control strategy adopts a hybrid model that combines a rule-based control strategy with a control strategy optimized by deep reinforcement learning, executes the control strategy based on predefined rules, optimizes the control strategy using deep reinforcement learning, applies the optimized control strategy to specific smart home appliances and monitors them in real time, and then dynamically adjusts the control strategy based on feedback data and real-time monitoring results;

[0102] The interactive feedback module is used to design a user interface to view and control the status of home appliances, and collect user operations and feedback information in real time and store it in a database. It performs sentiment analysis on user operations and feedback information, extracts user emotional tendencies, builds a user behavior model based on user emotional tendencies, extracts features of user behavior to obtain cluster analysis, identifies different types of user behavior patterns, performs cluster analysis based on user behavior characteristics, identifies different types of user behavior patterns, generates personalized control strategies for each user based on the user behavior model, and further combines user feedback data to optimize the existing control strategies.

[0103] The beneficial technical effects of the present invention are at least as follows:

[0104] (1) This invention uses an intelligent monitoring device to collect indoor and outdoor environmental parameters (such as temperature, humidity, light intensity, air quality, etc.) in real time, and uses reinforcement learning algorithms (such as DQN and PPO) for analysis and adjustment. It automatically adjusts the operating status of home appliances according to environmental changes, achieving energy-saving optimization. This overcomes the shortcomings of existing home appliances in operating independently and lacking linkage, realizes intelligent adaptive adjustment, and improves the user experience.

[0105] (2) The present invention uses intelligent monitoring devices (such as cameras, door and window sensors, smoke detectors, etc.) to monitor the home security status in real time, uses image processing and intelligent analysis technology to detect abnormal situations, and triggers the linkage control of home appliances. This solves the problem of existing security systems lacking linkage control with home appliances, improves the level of home security protection, and provides comprehensive security protection.

[0106] (3) The present invention combines environmental parameters and security data, uses deep learning algorithms (such as Transformer and Attention mechanisms) to perform multimodal data fusion and intelligent analysis, extract key features, and provide comprehensive intelligent control.

[0107] Problem Solving: Overcame the limitations of existing technologies in data analysis, achieved more precise control of home appliances, and improved the intelligence level of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] The present invention is further described with reference to the accompanying drawings. However, the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative effort.

[0109] Figure 1 This is a flow chart of a multi-device linkage control method according to the present invention.

[0110] Figure 2 This is a framework diagram of a multi-station electric linkage control system according to the present invention. DETAILED DESCRIPTION

[0111] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0112] In one or more embodiments, Figure 1 As shown, a multi-device linkage control method is disclosed, which includes the following steps S1-S6:

[0113] S1. Arrange intelligent monitoring devices to collect indoor and outdoor environmental data, and conduct real-time monitoring of home entrances and important areas, calibrate and calibrate the collected data and monitoring data, convert the calibrated and calibrated data into digital signals, transmit them to the central processing unit and store them; wherein, the intelligent monitoring device includes environmental sensors and security sensors, the environmental sensors include temperature sensors, humidity sensors, light sensors and air quality sensors, and the security sensors include cameras, door and window sensors and smoke detectors.

[0114] Specifically, various intelligent monitoring devices are deployed throughout the home, including environmental sensors (for temperature, humidity, light, air quality, etc.), cameras, door and window sensors, and smoke detectors. These devices collect real-time data on indoor and outdoor environmental parameters and home security status, and transmit this data to a central processing unit. This establishes a comprehensive, real-time data collection foundation, ensuring that subsequent analysis and decision-making are based on accurate and up-to-date information.

[0115] Among them, the temperature sensor is installed in different locations indoors and outdoors to detect the ambient temperature. The relationship between the output voltage Vtemp of the temperature sensor and the temperature T is linearly converted using the calibration coefficient ktemp:

[0116] T=ktemp·Vtemp+btemp (1)

[0117] Among them, Vtemp represents the original output voltage value of the temperature sensor, ktemp represents the calibration coefficient of the temperature sensor, btemp represents the calibration offset of the temperature sensor, and T represents the actual temperature

[0118] The humidity sensor is installed indoors to detect the ambient humidity. The relationship between the humidity sensor's output voltage Vhum and humidity H is linearly converted using the calibration coefficient khum:

[0119] H=khum·Vhum+bhum (2)

[0120] Where Vhum represents the raw output voltage of the humidity sensor; khum represents the calibration coefficient of the humidity sensor; bhum represents the calibration offset of the humidity sensor; and H represents the actual humidity.

[0121] The light sensor is installed near the window to detect the light intensity. The relationship between the output voltage Vlight of the light sensor and the light intensity L is linearly converted using the calibration coefficient klight:

[0122] L=klight·Vlight+blight (3)

[0123] Among them, Vlight represents the raw output voltage value of the light sensor, klight represents the calibration coefficient of the light sensor, blight represents the calibration bias of the light sensor, and L represents the actual light intensity.

[0124] Air quality sensors are installed indoors to detect particulate matter and CO2 concentrations in the air. The relationship between the output voltage Vair of the air quality sensor and the air quality index A is linearly converted using the calibration coefficient kair:

[0125] A=kair·Vair+bair (4)

[0126] Where Vair represents the raw output voltage value of the air quality sensor, kair represents the calibration coefficient of the air quality sensor, bair represents the calibration offset of the air quality sensor, and A represents the actual air quality index.

[0127] Cameras are installed at the entrance and important areas of the home to monitor video images in real time.

[0128] Door and window sensors are installed on doors and windows to detect the open and close status. The output signal Sdoor of the door and window sensor is a binary signal (open = 1, close = 0). Sdoor: The output signal of the door and window sensor (open = 1, close = 0)

[0129] Smoke detectors are installed in fire-prone areas such as kitchens and living rooms to detect smoke concentrations. The relationship between the smoke detector's output voltage Vsmoke and the smoke concentration Csmoke is linearly converted using the calibration coefficient ksmoke:

[0130] Csmoke=ksmoke·Vsmoke+bsmoke (5)

[0131] Where Vsmoke represents the original output voltage value of the smoke detector, ksmoke represents the calibration coefficient of the smoke detector, bsmoke represents the calibration offset of the smoke detector, and Csmoke represents the actual smoke concentration.

[0132] Furthermore, all sensors are connected to the central processing unit (CPU) via wired or wireless means (such as ZigBee, Wi-Fi) to transmit data in real time. At the same time, the data collection frequency Fsensor of each sensor is set, such as the temperature sensor collects data once per minute, the humidity sensor collects data once every 10 minutes, the light sensor collects data once every second, etc. The specific frequency is set according to the sensor type and application scenario. Fsensor represents the data collection frequency of the sensor

[0133] Transmit sensor data in a standard format (such as JSON). Each piece of data includes the sensor ID, timestamp, and measurement value:

[0134] {

[0135] "sensor_id":"temp_sensor_01",

[0136] "timestamp":1625270400,

[0137] "value":25.3

[0138] }

[0139] Wherein, sensor_id: unique identifier of the sensor, timestamp: timestamp of data collection (UNIX time), value: measured value of the sensor.

[0140] Furthermore, the temperature sensor is calibrated using a known temperature standard material (such as ice-water mixture at 0°C, boiling water at 100°C), multiple calibration points are obtained, and the calibration coefficient ktemp and offset btemp are calculated:

[0141] {(Vtemp1,T1),(Vtemp2,T2),…}(6)

[0142] Use linear regression to solve for the calibration coefficients and bias:

[0143]

[0144] The humidity sensor is calibrated using a known humidity environment (such as a saturated salt solution with 75% RH), obtaining the calibration points and calculating the calibration coefficient khum and bias bhum:

[0145] {(Vhum1,H1),(Vhum2,H2),…}(8)

[0146] Use linear regression to solve for the calibration coefficients and bias:

[0147]

[0148] The light sensor is calibrated using a standard light source of known light intensity, obtaining the calibration points and calculating the calibration coefficient klight and the bias blight:

[0149] {(Vlight1,L1),(Vlight2,L2),…}(10)

[0150] Use linear regression to solve for the calibration coefficients and bias:

[0151]

[0152] Air quality sensor calibration uses a known air quality environment for calibration (such as standard gas concentration), obtains calibration points, and calculates the calibration coefficient kair and bias bair:

[0153] {(Vair1,A1),(Vair2,A2),…}(12)

[0154] Use linear regression to solve for the calibration coefficients and bias:

[0155]

[0156] Furthermore, the sensor's analog signal is converted to a digital signal via an A / D converter and transmitted to the central processing unit. Each piece of data is timestamped to ensure temporal consistency. The timestamp format is UNIX time (seconds since January 1, 1970). The data is stored in a local database with the following table structure:

[0157] CREATE TABLE sensor_data(

[0158] idINTPRIMARYKEYAUTO_INCREMENT,

[0159] sensor_idVARCHAR(50),

[0160] timestampBIGINT,

[0161] valueFLOAT );

[0163] After data storage is completed, the central processing unit sends the data to the cloud server for further analysis and processing.

[0164] S2. Preprocess and extract features from the calibrated and calibrated data, fuse the extracted feature vectors to obtain a comprehensive feature vector, use a long short-term memory network to build a time series model to analyze the comprehensive feature vector, and use an autoencoder based on reconstruction error to detect anomalies based on the analysis results. At the same time, transmit the analysis results to the cloud server.

[0165] Specifically, collect data from all sensors deployed in step 1, including temperature sensors, humidity sensors, light sensors, air quality sensors, cameras, door and window sensors, and smoke detectors. Each data point contains a sensor ID, a timestamp, and a measurement value. To ensure data consistency and comparability, data synchronization is required. Assume that all data is synchronized to a minute-by-minute granularity. Use linear interpolation to synchronize low-frequency data to minute-by-minute time points.

[0166] The data levels and units of different sensors are different, so the data needs to be normalized. Assume that the original data is x i (t), the normalized data is x i ′(t), the normalization formula is as follows:

[0167]

[0168] Among them, x i (t): the raw data of the i-th sensor at time t, x i ′(t): Normalized data of the i-th sensor at time t, μ i : the mean of the i-th sensor data, σ i : The standard deviation of the i-th sensor data.

[0169] Furthermore, for the data of environmental sensors (temperature, humidity, light, air quality, smoke concentration), the normalized values ​​are directly used as features.

[0170] For camera data, convolutional neural network (CNN) is used to extract image features. Assuming that the image data is I(t), the feature vector f is extracted by CNN. I (t):

[0171] f I (t)=CNN(I(t)) (15)

[0172] Where, I(t): image data of the camera at time t, f I (t): image feature vector.

[0173] For the binary data of door and window sensors, the original values ​​are directly used as features.

[0174] Furthermore, the feature vectors of different sensors are fused to obtain the comprehensive feature vector F(t). In order to consider the importance difference of each modality data, the attention mechanism is used to weightedly fuse the features of each modality:

[0175]

[0176] Among them, F i (t): Feature vector of the i-th sensor at time t, α i (t): The attention weight of the i-th modal feature at time t, calculated by the following formula:

[0177]

[0178] e i (t) = w T tanh(W i ·F i (t)+b i ) (18)

[0179] Among them, e i (t): attention score of the i-th modality feature, α i (t): attention weight of the i-th modality feature, w: global weight vector, W i : The weight matrix of the i-th modal feature, b i : Bias vector of the i-th modal feature.

[0180] Furthermore, for the fused feature vector F fused (t), a long short-term memory network (LSTM) is used for time series modeling to capture the temporal dependency of the data. The input of LSTM is the fused feature vector at the current moment, , which is expressed as follows:

[0181] h(t) fused =LSTM(F fused (t),h(t-1) fused )(19)

[0182] Among them, h(t) represents the hidden state of LSTM at time t, F fused (t) represents the fused feature vector, h(t-1) fused represents the hidden state of LSTM at time t-1;

[0183] The autoencoder decoder based on reconstruction error is used to detect anomalies in the fused time series data, as shown below:

[0184]

[0185] The anomaly score is the reconstruction error and is expressed as follows:

[0186]

[0187] Where h(t) fused represents the hidden state of LSTM at time t, represents the hidden state reconstructed by the autoencoder, E(t) fused Represents the reconstruction error, when E(t) fused When the data exceeds the preset threshold, it is considered that the data at the current time t is abnormal and preprocessing (such as interpolation or deletion) is performed.

[0188] Furthermore, the fused time series feature data is stored in the database for subsequent intelligent control and analysis. The database table structure is as follows:

[0189] CREATETABLE fused_sensor_data(

[0190] idINTPRIMARYKEYAUTO_INCREMENT,

[0191] timestampBIGINT,

[0192] fused_featuresBLOB );

[0194] The fused feature data is transmitted to the cloud server through a secure channel for further processing and analysis.

[0195] S3. Using a regression model based on the comprehensive feature vector, a relationship model between environmental parameters and home appliance control is established. Based on the relationship model, a reinforcement learning algorithm is used to generate an adaptive control strategy so that the home appliance can perform adaptive adjustments based on real-time environmental data. Specifically, the strategy includes:

[0196] S301, through multimodal data fusion and preprocessing steps, obtain the fused time series feature vector F fused (t), these feature vectors contain the data of each environmental sensor. A regression model is used to establish the relationship between environmental parameters and home appliance control. Assume that the state vector of the home appliance is S(t), which is expressed as follows:

[0197]

[0198] Among them, S(t) represents the state vector of the home appliance at time t, β0 represents the intercept term of the regression model, and β i represents the coefficient of the regression model, F fused,i (t) represents the i-th element of the fused feature vector, ∈ represents the error term;

[0199] S302, design reinforcement learning, define the state space S, action space A and reward function R(s,a), where the state space S is composed of the fusion feature vector F fused (t); the action space A includes a set of control instructions for home appliances, such as adjusting the air conditioner temperature and the light brightness; the reward function R(s,a) is used to evaluate the effect of each action in a specific state. The reward function can be defined based on indicators such as energy saving and user comfort.

[0200] S303. Use the Q-learning algorithm to perform policy iteration and optimize the control strategy. Define the Q-value function Q(s,a), which represents the expected cumulative reward of taking action a in state s, as follows:

[0201]

[0202] Among them, Q(s,a) represents the Q value function, E represents the expectation operator, γ represents the discount factor, R(s t+k ,a t+k ) means in state s t+k Next take action a t+k Instant rewards;

[0203] S304. Update the Q-value function according to the Q-learning algorithm, which is expressed as follows:

[0204]

[0205] Among them, α represents the learning rate, s′ represents the new state reached after executing action a, and max a′ Q(s′,a′) represents the maximum Q value under the new state s′;

[0206] S305. Determine the optimal strategy π based on the optimized Q value function * (s), which is as follows:

[0207]

[0208] Among them, π * (s) represents the optimal strategy in state s, argmax a Represents the action a that maximizes the Q-value function, and applies the optimal strategy to the control of home appliances to achieve adaptive adjustment.

[0209] S4. Process the data collected by the security sensor and perform feature extraction, fuse the feature vectors of the security sensor to obtain a comprehensive security feature vector, use the long short-term memory network to perform time series modeling on the fused comprehensive security feature vector to obtain the time series features under the normal state, use the autoencoder to perform anomaly detection based on the time series features under the normal state, trigger an alarm when an anomaly is detected, and record the abnormal data and timestamp.

[0210] Specifically, collect data from the cameras, door and window sensors, and smoke detectors deployed in step 1. Each data point contains the sensor ID, timestamp, and measurement value. Assume that the camera data is I(t) and the door and window sensor data is S door (t), the smoke detector data is C smoke (t).

[0211] For camera data, an improved convolutional neural network (CNN) is used for video image processing and feature extraction. Assuming that the image data of the video frame at time t is I(t), the feature vector f is extracted by CNN. I (t):

[0212] f I (t)=CNN(I(t)) (26)

[0213] Where, I(t): image data of the camera at time t, f I (t): image feature vector.

[0214] Among them, the CNN model uses multiple layers of convolutional layers, pooling layers, and fully connected layers. The specific architecture is as follows:

[0215] Input layer: image I(t);

[0216] Convolutional layer 1: uses 32 convolution kernels of size 3×3 and activation function ReLU;

[0217] Pooling layer 1: Maximum pooling, using a 2×2 window for maximum pooling;

[0218] Convolutional layer 2: uses 64 convolution kernels of size 3×3 and activation function ReLU;

[0219] Pooling layer 2: Again use a 2×2 window for maximum pooling;

[0220] Fully connected layer: 128 neurons, activation function ReLU;

[0221] Output layer: feature vector f I (t), the size is 128.

[0222] Furthermore, for the data of the door and window sensors, the original binary data is directly used as features. Assume that the state of the door and window sensors at time t is S door (t), then the eigenvector is:

[0223] f door (t) = S door (t)(27)

[0224] S door (t): The state of the door and window sensor at time t (open = 1, close = 0)

[0225] For the smoke detector data, the original concentration data is used directly as the feature. Assume that the smoke concentration of the smoke detector at time t is C smoke (t), then the eigenvector is:

[0226] f smoke (t) = C smoke (t)(28)

[0227] C smoke (t): smoke concentration of the smoke detector at time t

[0228] Furthermore, the feature vectors of different sensors are fused to obtain the comprehensive feature vector F security (t). The specific fusion process uses the weighted average method to ensure that the importance difference of each modality data is reasonably reflected. Assume that the fusion weight is w I ,w door ,w smoke , then the fusion feature vector formula is:

[0229] F security (t) = w I ·f I (t)+w door ·f door (t)+w smoke ·f smoke (t)(29)

[0230] Among them, F security (t): fused security feature vector; w I : weight of image feature vector; w door : weight of the door and window sensor feature vector; w smoke : weight of the smoke detector feature vector; f I (t): image feature vector; f door (t): door and window sensor feature vector; f smoke (t): smoke detector feature vector.

[0231] Furthermore, the improved long short-term memory network is used to analyze the security comprehensive feature vector F security (t) Perform time series modeling and output the hidden state h at the current moment security (t), is expressed as follows:

[0232] h security (t)=LSTM(F security (t),h security (t-1))(30)

[0233] Among them, h security (t) represents the hidden state of the LSTM network at time t, F security (t) represents the fused feature vector, h security (t-1) represents the hidden state of the LSTM network at time t-1; the update formula of the improved LSTM network is as follows:

[0234]

[0235] Among them, i t represents the input gate; f t represents the forget gate; o t represents the output gate; represents the candidate memory unit; c t Represents a memory unit; W i ,W f ,W o ,W c represents the weight matrix; b i ,b f ,b o ,b c represents the bias vector; σ represents the Sigmoid function.

[0236] The hidden state h at the current moment is obtained by using the autoencoder based on reconstruction error security (t) performs anomaly detection, which is expressed as follows:

[0237]

[0238] The anomaly score is the reconstruction error:

[0239]

[0240] Among them, h security (t) represents the hidden state of the LSTM network at time t, represents the hidden state reconstructed by the autoencoder; E security (t) represents the security reconstruction error, when E security When (t) exceeds the preset threshold, the data at the current time t is considered to be abnormal.

[0241] Furthermore, when an anomaly is detected, the system triggers an alarm and records the anomaly data and timestamp. The alarm information includes the anomaly type (such as door and window anomalies, smoke anomalies, video anomalies) and handling suggestions (such as activating an alarm, notifying the user, turning off the gas, etc.).

[0242] S5. Generate a specific multi-appliance control strategy based on the adaptive control strategy and the comprehensive security feature vector. The generation of the multi-appliance control strategy adopts a hybrid model. The hybrid model combines the rule-based control strategy and the deep reinforcement learning optimized control strategy, executes the basic control strategy based on predefined rules, uses deep reinforcement learning to optimize the control strategy, applies the optimized control strategy to specific smart home appliances and monitors them in real time, and then dynamically adjusts the control strategy based on feedback data and real-time monitoring results.

[0243] Furthermore, a specific control strategy is generated based on the environmental perception and adaptive control strategy generated in step 3 and the home security status determined by anomaly detection in step 4.

[0244] The control strategy generation adopts a hybrid model that combines the rule-based control strategy and the reinforcement learning optimization control strategy. Assuming that the current environment state is E(t), the security state is A(t), and the control strategy is C(t), the control strategy generation formula is:

[0245] C(t)=α·Policy rule (E(t),A(t))+(1-α)·Policy RL (E(t),A(t)) (34)

[0246] Among them, C(t): control strategy, α: strategy mixing weight, Policy rule : Rule-based control strategy, Policy RL : control strategy optimized by reinforcement learning, E(t): current environment state, A(t): current security state.

[0247] Execute basic control strategies based on predefined rules. For example, when an abnormality is detected in a door or window, an alarm signal is immediately issued and the user is notified. The specific rule-based control strategies are as follows:

[0248] if A(t)=door_open then C(t)=send_alert (35)

[0249] Among them, A(t): current security status, C(t): control strategy, door_open: abnormal state of door and window, send_alert: send alarm signal.

[0250] Furthermore, deep reinforcement learning (such as DQN, DeepQ-Network) is used to optimize control strategies, especially for decision-making in complex scenarios. Assuming that the environment state is E(t), the control strategy is C(t), and the reward function is R(t), the Q value update formula is:

[0251]

[0252] Among them, Q(s t ,a t ): In state s t Next take action a t Q value, α: learning rate, R(t): reward value, γ: discount factor, max a′ Q(s t+1 ,a′): in state s ++1 The Q value of taking the optimal action under t : Current state, a t : Current action, s t+1 : next state, a′: next action.

[0253] Furthermore, the generated control strategy is applied to specific smart home appliances. Assume that the control strategy C(t) contains the control instruction I i (t), where i represents a specific device, then the control instruction application formula is:

[0254] I i (t)=DeviceControl(C(t),D i ) (37)

[0255] Among them, I i (t): control instruction of the i-th device, C(t): control strategy, D i : The status of the i-th device, DeviceControl: device control function.

[0256] Monitor the device status in real time after executing the control strategy and collect feedback data. Assume that the device status feedback is D i (t), the feedback data collection formula is:

[0257] F i (t) = CollectFeedback(D i (t)) (38)

[0258] Among them, F i (t): feedback data of the i-th device, D i (t): the state of the i-th device at time t, CollectFeedback: feedback data collection function.

[0259] According to the feedback data and real-time monitoring results, the control strategy is dynamically adjusted. Assume that the feedback data is F i (t), the adjusted control strategy is C′(t), and the adaptive strategy adjustment formula is:

[0260] C′(t)=AdjustPolicy(F i (t),C(t)) (39)

[0261] Among them, C′(t): the adjusted control strategy, F i (t): feedback data, C(t): initial control strategy, AdjustPolicy: strategy adjustment function.

[0262] S6. Design a user interface to view and control the status of home appliances, and collect user operations and feedback information in real time, and store it in a database. Perform sentiment analysis on user operations and feedback information, extract user emotional tendencies, build a user behavior model based on user emotional tendencies, perform feature extraction on user behavior to obtain cluster analysis, identify different types of user behavior patterns, perform cluster analysis based on user behavior characteristics, identify different types of user behavior patterns, generate personalized control strategies for each user based on the user behavior model, and further combine user feedback data to optimize the existing control strategies.

[0263] Specifically, a user-friendly interface should be designed so that users can easily view and control the status of home appliances. The user interface includes functional modules such as device control, status monitoring, alarm information, and feedback recording. The interface is accessed through a variety of devices (such as mobile phones, tablets, and computers). The user interface adopts a responsive design, automatically adapting to different screen sizes, and graphically displays device status and control options. Users can easily control the device by dragging, clicking, and other operations. The user interface should intuitively display the status, historical data, and alarm information of home appliances through charts and graphs. For example, use a line chart to show temperature changes and a bar chart to show energy consumption.

[0264] Furthermore, real-time user feedback is collected: user operations and feedback information are collected in real time and stored in the database. Assume that the user feedback data is U(t), including user operation records O(t) and user feedback information F u (t), then the user feedback data collection formula is:

[0265] U(t)={O(t),F u (t)} (40)

[0266] Among them, U(t): user feedback data, O(t): user operation record, F u (t): User feedback information.

[0267] Furthermore, the user feedback information F u (t) Perform sentiment analysis to extract the user’s sentiment tendency. Assume that the sentiment score is S u (t), calculated using the sentiment analysis model:

[0268] S u (t) = SentimentAnalysis(F u (t)) (41)

[0269] Among them, S u (t): Sentiment score SentimentAnalysis: sentiment analysis function.

[0270] Furthermore, we use user feedback data to build a user behavior model. Assume that the user behavior feature vector is B u (t), by analyzing the user operation record O(t) and feedback information F u (t), extract user behavior characteristics. The specific formula is:

[0271] B u (t)=[O(t),S u (t),FeatureExtraction(F u (t))] (42)

[0272] Among them, B u (t): user behavior feature vector, FeatureExtraction: feature extraction function.

[0273] Furthermore, cluster analysis is performed on user behavior characteristics to identify different types of user behavior patterns. Assume there are k types of user behavior patterns, and each behavior pattern is represented by a central vector μ i Indicates that the clustering algorithm is k-means clustering:

[0274]

[0275] Among them, μ i : the center vector of the i-th behavior mode, The set of user behavior features belonging to category i.

[0276] Furthermore, based on the user behavior model, a personalized control strategy is generated for each user. Assume that the personalized control strategy is C u (t), combined with user behavior characteristics B u (t) and the environmental state E(t) to generate a personalized control strategy. The specific formula is:

[0277]

[0278] Among them, C u (t): personalized control strategy, w i : The weight of the i-th measure, M i : The specific strategy function of the i-th control measure, B u (t): user behavior feature vector, E(t): current environment state.

[0279] Furthermore, in the process of generating personalized strategies, the weights of various control strategies are dynamically adjusted by combining user behavior characteristics and real-time environmental status. Assume that the control strategy includes n specific measures, and the weight of each measure is w i , then the control strategy is:

[0280]

[0281] Among them, w i : The weight of the i-th measure, M i : The specific strategy function of the i-th control measure.

[0282] Furthermore, based on user feedback data, the existing control strategy is optimized to make it more in line with user needs. Assume that the optimized control strategy is C u ′(t), combined with user feedback data U(t) and the existing control strategy C u (t), optimize the control strategy. The specific formula is:

[0283] C u ′(t)=OptimizePolicy(U(t),C u (t)) (46)

[0284] Among them, C u ′(t): optimized control strategy, OptimizePolicy: policy optimization function, U(t): user feedback data, C u (t): Personalized control strategy.

[0285] It is understandable that reinforcement learning algorithms (such as Proximal Policy Optimization, PPO) are used to optimize the control strategy. Define the reward function R u (t), update the control strategy based on user feedback data and system performance:

[0286]

[0287] Among them, R u (t): reward function, α: learning rate, gradient.

[0288] Furthermore, a continuous learning system is implemented to continuously learn from user feedback and system execution results, and gradually optimize the control strategy. Assume that the learning process is L(t), including feedback data U(t) and execution results E r (t), through the learning function Learning(U(t),E r (t)), update the control strategy. The specific formula is:

[0289] C t+1 =Learning(U(t),E r (t),C t ) (48)

[0290] Among them, C t+1 : updated control strategy, Learning: continuous learning function, U(t): user feedback data, E r (t): execution effect, C t : Current control strategy.

[0291] Furthermore, online learning and offline training are combined to ensure the adaptability and stability of the control strategy in a real-time environment. Online learning is used to adjust the control strategy in real time, while offline training is used to optimize the strategy during non-operating hours.

[0292] In one or more embodiments, Figure 2 As shown, a multi-device linkage control system is disclosed, the system comprising:

[0293] The data acquisition module 101 is configured to deploy intelligent monitoring devices to collect indoor and outdoor environmental data, perform real-time monitoring of home entrances and important areas, calibrate and calibrate the collected and monitored data, convert the calibrated and calibrated data into digital signals, transmit them to the central processing unit, and store them. The intelligent monitoring devices include environmental sensors and security sensors. The environmental sensors include temperature sensors, humidity sensors, light sensors, and air quality sensors. The security sensors include cameras, door and window sensors, and smoke detectors.

[0294] The comprehensive feature extraction module 102 is used to preprocess and extract features from the calibrated and calibrated data, fuse the extracted feature vectors to obtain a comprehensive feature vector, use a long short-term memory network to build a time series model to analyze the comprehensive feature vector, use a reconstruction error-based autoencoder to perform anomaly detection based on the analysis results, and transmit the analysis results to the cloud server;

[0295] A relationship modeling module 103 is configured to establish a relationship model between environmental parameters and home appliance control using a regression model based on the comprehensive feature vector, and to generate an adaptive control strategy based on the relationship model using a reinforcement learning algorithm to enable the home appliance to perform adaptive adjustments based on real-time environmental data;

[0296] The security feature extraction module 104 is used to process and extract features from the data collected by the security sensors, fuse the feature vectors of the security sensors to obtain a comprehensive security feature vector, use a long short-term memory network to perform time series modeling on the fused comprehensive security feature vector to obtain time series features under normal conditions, use an autoencoder to perform anomaly detection based on the time series features under normal conditions, trigger an alarm when an anomaly is detected, and record the anomaly data and timestamp;

[0297] a strategy adjustment module 105 for generating a specific multi-appliance control strategy based on the adaptive control strategy and the comprehensive security feature vector. The multi-appliance control strategy is generated using a hybrid model that combines a rule-based control strategy with a control strategy optimized by deep reinforcement learning. The hybrid model executes the basic control strategy based on predefined rules, optimizes the control strategy using deep reinforcement learning, applies the optimized control strategy to specific smart home appliances, and monitors them in real time. The control strategy is then dynamically adjusted based on feedback data and real-time monitoring results.

[0298] The interactive feedback module 106 is used to design a user interface for viewing and controlling the status of home appliances, and to collect user operation and feedback information in real time and store it in a database. It performs sentiment analysis on the user's operation and feedback information, extracts the user's emotional tendencies, builds a user behavior model based on the user's emotional tendencies, extracts features of the user behavior to obtain cluster analysis, identifies different types of user behavior patterns, performs cluster analysis based on user behavior features, identifies different types of user behavior patterns, generates personalized control strategies for each user based on the user behavior model, and further combines user feedback data to optimize the existing control strategies.

[0299] It is worth noting that the specific workflow of a multi-device linkage control system provided by the embodiment of the present invention is the same as the workflow of a multi-device linkage control method described in the above embodiment, and will not be described in detail here.

[0300] Compared with the prior art, the embodiment of the present invention provides a multi-family electrical linkage control system, which arranges intelligent monitoring devices to collect indoor and outdoor environmental data, and monitors home entrances and important areas in real time, calibrates and calibrates the collected data and monitoring data, converts the calibrated and calibrated data into digital signals, transmits them to the central processing unit and stores them; wherein, the intelligent monitoring device includes environmental sensors and security sensors, the environmental sensors include temperature sensors, humidity sensors, light sensors and air quality sensors, and the security sensors include cameras, door and window sensors and smoke detectors; the calibrated and calibrated data are preprocessed and feature extracted , the extracted feature vectors are fused to obtain a comprehensive feature vector, and a long short-term memory network is used to build a time series model to analyze the comprehensive feature vector. According to the analysis results, an autoencoder based on reconstruction error is used for anomaly detection, and the analysis results are transmitted to the cloud server at the same time; based on the comprehensive feature vector, a regression model is used to establish a relationship model between environmental parameters and home appliance control, and a reinforcement learning algorithm is used based on the relationship model to generate an adaptive control strategy so that home appliances can be adaptively adjusted according to real-time environmental data; the data collected by the security sensor is processed and feature extracted, and the feature vectors of the security sensor are fused to obtain a comprehensive security feature vector, and the method is used The long short-term memory network performs time series modeling on the fused security comprehensive feature vector to obtain the time series features under normal conditions. The autoencoder is used to detect anomalies based on the time series features under normal conditions. When an anomaly is detected, an alarm is triggered and the abnormal data and timestamp are recorded. A specific multi-home appliance control strategy is generated based on the adaptive control strategy and the security comprehensive feature vector. The generation of the multi-home appliance control strategy adopts a hybrid model. The hybrid model combines the rule-based control strategy and the deep reinforcement learning optimized control strategy. The basic control strategy is executed based on predefined rules, and the control strategy is optimized using deep reinforcement learning. The optimized control strategy is applied to specific smart home appliances and then Conduct real-time monitoring, and then dynamically adjust the control strategy based on feedback data and real-time monitoring results; design a user interface to view and control the status of home appliances, and collect user operations and feedback information in real time, and store it in the database, perform sentiment analysis on user operations and feedback information, extract user emotional tendencies, build a user behavior model based on user emotional tendencies, perform feature extraction on user behavior to obtain cluster analysis, identify different types of user behavior patterns, perform cluster analysis based on user behavior characteristics, identify different types of user behavior patterns, generate personalized control strategies for each user based on the user behavior model, and further combine user feedback data to optimize the existing control strategies.

[0301] An embodiment of the present invention further provides a multi-device linkage control device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the steps in the embodiment of the multi-device linkage control method described above are implemented, for example: Figure 1 or, the processor implements the functions of the modules in the above-mentioned system embodiments when executing the computer program.

[0302] For example, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the multi-device linkage control device.

[0303] The multi-device linkage control device can be a computing device such as a desktop computer, laptop, PDA, or cloud server. The multi-device linkage control device can include, but is not limited to, a processor and memory. Those skilled in the art will appreciate that the multi-device linkage control device can also include input / output devices, network access devices, buses, and the like.

[0304] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The processor serves as the control center of the multi-device linkage control device, connecting various parts of the multi-device linkage control device using various interfaces and lines.

[0305] The memory can be used to store the computer programs and / or modules, and the processor implements the various functions of the multi-device linkage control device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the operation of the air-conditioning controller, etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMedaaCard, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0306] Wherein, if the module integrated into a multi-device linkage control device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0307] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0308] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A multi-device linkage control method, characterized in that: The method comprises the following steps: S1. Arrange intelligent monitoring devices to collect indoor and outdoor environmental data, and monitor home entrances and important areas in real time, calibrate and calibrate the collected data and monitoring data, convert the calibrated and calibrated data into digital signals, transmit them to the central processing unit and store them; wherein the intelligent monitoring device includes environmental sensors and security sensors, the environmental sensors include temperature sensors, humidity sensors, light sensors and air quality sensors, and the security sensors include cameras, door and window sensors and smoke detectors; S2. Preprocess and extract features from the calibrated and calibrated data, fuse the extracted feature vectors to obtain a comprehensive feature vector, use a long short-term memory network to build a time series model to analyze the comprehensive feature vector, use an autoencoder based on reconstruction error to perform anomaly detection based on the analysis results, and transmit the analysis results to the cloud server; S3. Using a regression model based on the comprehensive feature vector to establish a relationship model between environmental parameters and home appliance control, and using a reinforcement learning algorithm based on the relationship model to generate an adaptive control strategy so that the home appliance can be adaptively adjusted according to real-time environmental data; S4, processing the data collected by the security sensor and extracting features, fusing the feature vectors of the security sensor to obtain a comprehensive security feature vector, using a long short-term memory network to perform time series modeling on the fused comprehensive security feature vector to obtain time series features under normal conditions, using an autoencoder to perform anomaly detection based on the time series features under normal conditions, triggering an alarm when an anomaly is detected and recording the abnormal data and timestamp; S5. Generate a specific multi-home appliance control strategy based on the adaptive control strategy and the security comprehensive feature vector, wherein the generation of the multi-home appliance control strategy adopts a hybrid model, wherein the hybrid model combines the rule-based control strategy and the deep reinforcement learning optimized control strategy, executes the control strategy based on predefined rules, optimizes the control strategy using deep reinforcement learning, applies the optimized control strategy to specific smart home appliances and performs real-time monitoring, and then dynamically adjusts the control strategy based on feedback data and real-time monitoring results; S6. Design a user interface to view and control the status of home appliances, and collect user operation and feedback information in real time, and store it in a database. Perform sentiment analysis on user operation and feedback information, extract user emotional tendencies, build a user behavior model based on user emotional tendencies, extract user behavior features to obtain cluster analysis, identify different types of user behavior patterns, perform cluster analysis based on user behavior features, identify different types of user behavior patterns, generate personalized control strategies for each user based on the user behavior model, and further combine user feedback data to optimize existing control strategies.

2. A multi-device linkage control method according to claim 1, characterized in that: The calibration and verification include: The temperature sensor is calibrated using a known temperature standard substance, multiple calibration points are obtained, and the calibration coefficient ktemp and the offset btemp are calculated: {(Vtemp1,T1),(Vtemp2,T2),…(Vtempn,Tn)} Wherein, Vtempn represents the original output voltage value of the nth temperature sensor, and Tn represents the actual temperature of the nth temperature sensor. The calibration coefficient and the offset are solved by the linear regression method, as shown below: Wherein, Ti represents the temperature of the i-th temperature sensor, and Vtempi represents the original output voltage value of the i-th temperature sensor; The humidity sensor calibration is performed using a known humidity environment to obtain calibration points, and the calibration coefficient khum and bias bhum are calculated, as shown below: {(Vhum1,H1),(Vhum2,H2),…(Vhumn,Hn)} Wherein, Vhumn represents the original output voltage value of the nth humidity sensor, and Hn represents the actual humidity of the nth humidity sensor; The calibration coefficient and offset are solved by linear regression method, which are expressed as follows: Wherein, Hi represents the actual humidity of the i-th humidity sensor, and Vhumi represents the original output voltage value of the i-th humidity sensor; The light sensor is calibrated using a standard light source with known light intensity, the calibration point is obtained, and the calibration coefficient klight and the offset blight are calculated: {(Vlight1,L1),(Vlight2,L2),…(Vlightn,Ln)} Wherein, Vlightn represents the original output voltage value of the nth light sensor, and Hn represents the actual light intensity of the nth light sensor; The calibration coefficient and offset are solved by linear regression method, which are expressed as follows: Where Li represents the actual light intensity of the i-th light sensor, and Vlighti represents the original output voltage value of the i-th light sensor; The air quality sensor is calibrated using a known air quality environment to obtain calibration points, and the calibration coefficient kair and the offset bair are calculated, which are expressed as follows: {(Vair1,A1),(Vair2,A2),…(Vairn,An)} Wherein, Vairn represents the original output voltage value of the nth air quality sensor, and An represents the actual air quality index of the nth air quality sensor; The calibration coefficient and offset are solved by linear regression method, which are expressed as follows: Wherein, Ai represents the actual air quality index of the i-th air quality sensor, and Vairi represents the original output voltage value of the i-th air quality sensor.

3. A multi-device linkage control method according to claim 1, characterized in that: In S2, for the data of the environmental sensor, the normalized value is directly used as the feature; for the camera data, a convolutional neural network is used to extract image features; for the binary data of the door and window sensor, the original value is directly used as the feature.

4. A multi-device linkage control method according to claim 3, characterized in that: The attention mechanism is used to weight the fusion of each modal feature, and the feature vectors of different sensors are fused to obtain the fused feature vector F fused (t), expressed as follows: Among them, F i (t) represents the feature vector of the i-th sensor at time t, α i (t) represents the attention weight of the i-th modal feature at time t, which is calculated by the following formula: e i (t)=w T ·tanh(W i ·F i (t)+b i ) Among them, e i (t) represents the attention score of the i-th modality feature, α i (t) represents the attention weight of the i-th modality feature, w represents the global weight vector, W i represents the weight matrix of the i-th modal feature, b i Represents the bias vector of the i-th modal feature; For the fused feature vector F fused (t), long short-term memory network is used for time series modeling, which is expressed as follows: h(t) fused =LSTM(F fused (t),h(t-1) fused ) Among them, h(t) represents the hidden state of LSTM at time t, F fused (t) represents the fused feature vector, h(t-1) fused represents the hidden state of LSTM at time t-1; The autoencoder Decoder based on reconstruction error is used to detect anomalies on the fused time series data, as shown below: The anomaly score is the reconstruction error and is expressed as follows: Among them, h(t) fused represents the hidden state of LSTM at time t, represents the hidden state reconstructed by the autoencoder, E(t) fused Represents the reconstruction error, when E(t) fused When the preset threshold is exceeded, the data at the current time t is considered abnormal.

5. A multi-device linkage control method according to claim 4, characterized in that: The S3 specifically includes: S301, using a regression model to establish a relationship model between environmental parameters and home appliance control, expressed as follows: Among them, S(t) represents the state vector of the household appliance at time t, β0 represents the intercept term of the regression model, and β i represents the coefficient of the regression model, F fused,i (t) represents the i-th element of the fused feature vector, ∈ represents the error term; S302, design reinforcement learning, define the state space S, action space A and reward function R(s,a), where the state space S is composed of the fused feature vector F fused (t) constitutes a high-dimensional space; the action space A includes a set of control instructions for household appliances; the reward function R(s,a) is used to evaluate the effect of each action in a specific state; S303, using Q-learning algorithm to perform strategy iteration and optimize control strategy, as shown below: Among them, Q(s,a) represents the Q value function, E represents the expectation operator, γ represents the discount factor, and R(s t+k ,a t+k ) means in state s t+k Take action a t+k Instant rewards; S304. According to the Q-learning algorithm, the Q value function is updated, which is expressed as follows: Among them, α represents the learning rate, s′ represents the new state reached after executing action a, and max a′ Q(s′,a′) represents the maximum Q value under the new state s′; S305. Determine the optimal strategy π based on the optimized Q value function * (s), which is represented as follows: Among them, π * (s) represents the optimal strategy in state s, argmax a It represents the action a that maximizes the Q-value function, and applies the optimal strategy to the control of household appliances to achieve adaptive regulation.

6. A multi-device linkage control method according to claim 1, characterized in that: In S4, the feature vectors of different sensors are fused by weighted average method to obtain the security comprehensive feature vector F security (t), expressed as follows: F security (t)=w I ·f I (t)+w door ·f door (t)+w smoke ·f smoke (t) Among them, F security (t) represents the fused security feature vector, w I Represents the weight of the image feature vector, w door represents the weight of the door and window sensor feature vector, w smoke represents the weight of the smoke detector feature vector, f I (t) represents the image feature vector, f door (t) represents the door and window sensor feature vector, f smoke (t) represents the smoke detector feature vector.

7. A multi-device linkage control method according to claim 6, characterized in that: The improved long short-term memory network is used to analyze the security comprehensive feature vector F security (t) Perform time series modeling and output the hidden state h at the current moment security (t), expressed as follows: h security (t)=LSTM(F security (t),h security (t-1)) Among them, h security (t) represents the hidden state of the LSTM network at time t, F security (t) represents the fused feature vector, h security (t-1) represents the hidden state of the LSTM network at time t-1; The hidden state h at the current moment is obtained by using the autoencoder based on the reconstruction error security (t) performs anomaly detection, which is expressed as follows: The anomaly score is the reconstruction error: Among them, h security (t) represents the hidden state of the LSTM network at time t, represents the hidden state reconstructed by the autoencoder; E security (t) represents the security reconstruction error, when E security When (t) exceeds the preset threshold, the data at the current time t is considered to be abnormal.

8. A multi-device linkage control method according to claim 7, characterized in that: The update formula of the improved long short-term memory network is expressed as follows: i t =σ(W i ·[h security (t-1),F security (t)]+b i ) f t =σ(W f ·[h security (t-1),F security (t)]+b f ) o t =σ(W o ·[h security (t-1),F security (t)]+b o ) h(t)=o t λtanh(c t ) Among them, i t represents the input gate; f t represents the forget gate; o t represents the output gate; represents the candidate memory unit; c t Represents a memory unit; W i ,W f ,W o ,W c represents the weight matrix; b i ,b f ,b o ,b c represents the bias vector; σ represents the Sigmoid function.

9. A multi-device linkage control method according to claim 1, characterized in that: The control strategy optimized by deep reinforcement learning is expressed as follows: Assuming that the environment state is E(t), the control strategy is C(t), and the reward function is R(t), the Q value update formula is as follows: Among them, Q(s t ,a t ) means in state s t Take action a t Q value; α represents the learning rate; R(t) represents the reward value; γ represents the discount factor; max a′ Q(s t+1 ,a′) means in state s t+1 The Q value of taking the best action; s t Indicates the current state; a t Indicates the current action; s t+1 represents the next state; a′ represents the next action; Apply the generated control strategy to specific smart home appliances, as shown below: Assume that the control strategy C(t) contains the control instruction I i (t), where i represents a specific device, and the control instruction application formula is as follows: I i (t)=DeviceControl(C(t),D i ) Among them, I i (t) represents the control instruction of the ith device, C(t) represents the control strategy, and D i Indicates the status of the i-th device, and DeviceControl represents the device control function.

10. A multi-device linkage control system, characterized in that: The system comprises: A data acquisition module is used to arrange intelligent monitoring devices to collect indoor and outdoor environmental data, and to monitor home entrances and important areas in real time, to calibrate and calibrate the collected data and monitoring data, and to convert the calibrated and calibrated data into digital signals for transmission to a central processing unit for storage; wherein the intelligent monitoring device includes environmental sensors and security sensors, the environmental sensors include temperature sensors, humidity sensors, light sensors and air quality sensors, and the security sensors include cameras, door and window sensors and smoke detectors; The comprehensive feature extraction module is used to preprocess and extract features from the calibrated and calibrated data, fuse the extracted feature vectors to obtain a comprehensive feature vector, use a long short-term memory network to build a time series model to analyze the comprehensive feature vector, use an autoencoder based on reconstruction error to perform anomaly detection based on the analysis results, and transmit the analysis results to the cloud server; A relationship modeling module is used to establish a relationship model between environmental parameters and home appliance control using a regression model based on the comprehensive feature vector, and to generate an adaptive control strategy using a reinforcement learning algorithm based on the relationship model so that the home appliance can be adaptively adjusted according to real-time environmental data; The security feature extraction module is used to process the data collected by the security sensor and extract features, fuse the feature vectors of the security sensor to obtain a comprehensive security feature vector, use the long short-term memory network to perform time series modeling on the fused comprehensive security feature vector, obtain the time series features under normal conditions, use the autoencoder to perform anomaly detection based on the time series features under normal conditions, and trigger an alarm when an anomaly is detected and record the abnormal data and timestamp; A strategy adjustment module, for generating a specific multi-home appliance control strategy based on an adaptive control strategy and a security comprehensive feature vector, wherein the generation of the multi-home appliance control strategy adopts a hybrid model, wherein the hybrid model combines a rule-based control strategy with a deep reinforcement learning optimized control strategy, executes the control strategy based on predefined rules, optimizes the control strategy using deep reinforcement learning, applies the optimized control strategy to specific smart home appliances and performs real-time monitoring, and then dynamically adjusts the control strategy based on feedback data and real-time monitoring results; The interactive feedback module is used to design a user interface to view and control the status of home appliances, and to collect user operations and feedback information in real time and store it in a database. It performs sentiment analysis on user operations and feedback information, extracts user emotional tendencies, builds a user behavior model based on user emotional tendencies, extracts features from user behaviors to obtain cluster analysis, identifies different types of user behavior patterns, performs cluster analysis based on user behavior features, identifies different types of user behavior patterns, generates personalized control strategies for each user based on the user behavior model, and further combines user feedback data to optimize existing control strategies.

Images