Smart home automation control system and control method thereof

Abstract

The invention provides a smart home automation control system and a control method thereof, and relates to the technical field of smart home, and the method comprises the steps: enabling home equipment to access a smart home network, converting a communication protocol of the home equipment into a unified protocol, and monitoring the working state of the home equipment; receiving a user instruction, converting the user instruction into a corresponding control instruction, and performing multi-mode interaction control on the home equipment according to the control instruction; presetting a scene mode, when a preset condition is satisfied, automatically triggering the corresponding scene mode, and adjusting the working state of the corresponding home equipment; security processing is carried out on the transmission data and the user data, the equipment state of the home equipment is monitored, and when the equipment state is abnormal, an alarm device is triggered; the indoor environment parameters are monitored to generate the monitoring result, the working state of the home equipment is adjusted according to the monitoring result and the preset energy-saving strategy, and the preset energy-saving strategy is optimized according to the use habit of the user, so that the equipment compatibility and safety can be improved, and the diversification of the control method is ensured.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to a smart home automation control system and a control method thereof. Background Art

[0002] With the rapid development of the Internet of Things, cloud computing and artificial intelligence technologies, smart home control systems have become an indispensable part of modern families, greatly improving the convenience, comfort and safety of home life. Traditional home appliances are gradually turning to intelligence and networking to meet the growing needs of users. Although significant progress has been made in the field of smart homes, there are still some problems in the existing technologies that limit its wider application and further improvement of user experience.

[0003] In the prior art, there are many brands of smart home devices on the market, and the standards are different, which makes it difficult to achieve seamless connection and collaborative work between devices of different brands and types. This incompatibility between devices not only increases the difficulty of user selection, but also limits the overall function and efficiency of the smart home system. Secondly, most systems rely on mobile phone applications or voice assistants for control, lack diversified control methods, and cannot meet the personalized needs of different user groups. In addition, this single control method may not provide a reliable control experience in certain specific scenarios, such as when the network is unstable or the device fails. At the same time, the complex setting process and unintuitive user interface often confuse users and reduce the ease of use of the system. In addition, the existing control system may leak transmission data and user information, resulting in poor system security, which directly affects the user's trust in and willingness to use the smart home system. Although the existing smart home control system has played an important role in improving the quality of home life, problems such as poor device compatibility, single control method, poor system security, and poor user experience still restrict its further development.

[0004] Therefore, it is necessary to provide a smart home automation control system and a control method thereof to solve the above technical problems. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a smart home automation control system and a control method thereof for solving the problems of poor compatibility of existing smart home control system equipment, single control method, poor security and poor user experience.

[0006] The present invention provides a smart home automation control system, comprising:

[0007] A device interconnection module is used to connect home devices to the smart home network, convert the communication protocol of the home devices to a unified protocol, and monitor the working status of the home devices;

[0008] A multimodal interactive control module, used to receive the user's voice instructions, gestures and touch operations and convert them into corresponding control instructions, and perform multimodal interactive control on the home appliance according to the control instructions;

[0009] An automatic scene triggering module is used to preset a scene mode. When a preset condition is met, the corresponding scene mode is automatically triggered and the working state of the corresponding home appliance is adjusted;

[0010] A security protection module, used for processing transmission data and user data securely, and monitoring the device status of the household device, and triggering an alarm device when the device status is abnormal;

[0011] The energy-saving and environmental protection module is used to monitor indoor environmental parameters to generate corresponding monitoring results, adjust the working status of the home appliances according to the monitoring results and preset energy-saving strategies, and optimize the preset energy-saving strategies according to user usage habits.

[0012] Preferably, the device interconnection module includes:

[0013] A device communication unit, used for wirelessly connecting the household devices;

[0014] A protocol conversion unit, used to convert the communication protocols of different household devices into the unified protocol recognizable by the system;

[0015] The device management unit is used to manage the home devices and monitor the working status of the home devices.

[0016] Preferably, the multimodal interactive control module is used to: receive the user's voice instructions, gesture actions and touch operations according to the following interactive control algorithm and convert them into corresponding control instructions, and perform multimodal interactive control on the home appliance according to the control instructions:

[0017] The user's voice command is collected by a microphone and converted into a corresponding voice control command, and the voice control of the home appliance is completed according to the voice control command, including the following operations:

[0018] The collected voice command is divided into multiple frames, features of each frame are extracted to obtain feature sequences of all frames, the feature sequences and the voice command are modeled to obtain a corresponding relationship between the feature sequences and the voice command, and a hidden state sequence, i.e., the voice control command, is determined according to the state transition probability, the observation probability, and the initial state probability;

[0019] The expression of the state transition probability is:

[0020] a ij =P(qt+1 =S j |q t =S i )

[0021] In the formula, a ij Represents the hidden state S i Transfer to hidden state S j The state transition probability; P(|) represents the probability symbol; q t+1 represents the hidden state variable at time t+1; q t represents the hidden state variable at time t; S j , S i Indicates hidden state;

[0022] The expression of the observation probability is:

[0023] b j (k) = P(O t =V k |q t =S j )

[0024] Where b j (k) represents the hidden state S j Generate observations V k The observation probability; P(|) represents the probability symbol; O t represents the observed variable at time t; q t represents the hidden state variable at time t; V k represents the observed value; S j Indicates hidden state;

[0025] The expression of the initial state probability is:

[0026] π i =P(q1=S i )

[0027] In the formula, π i Indicates that it is in the hidden state S at the initial moment i The initial state probability; P(|) represents the probability symbol; q1 represents the hidden state variable at the initial moment; S i Indicates hidden state;

[0028] Determine the hidden state sequence, i.e., the voice control instruction, according to the state transition probability, the observation probability and the initial state probability;

[0029] Recognizing the user's gestures through a camera and converting them into corresponding action control instructions, and completing the action control of the home appliance according to the action control instructions, including the following operations:

[0030] The camera is used to recognize the user's gesture to generate an input image, and feature extraction is performed on the input image to obtain output image features. The expression of the output image features is:

[0031] O(u,v)=(I*K)(u,v)=∑ m ∑ n I(u+m)(v+n)K(m,n)

[0032] Where, O represents the output image feature; u and v represent the coordinates of the output image feature; I represents the input image; K represents the convolution kernel; m and n represent the convolution kernel size;

[0033] Obtaining the action control instruction according to the output image feature, and completing the action control of the home appliance according to the action control instruction;

[0034] The user's touch operation on the touch interface is processed by the following positioning formula, and the voice command and gesture action are recognized and processed accordingly:

[0035]

[0036] In the formula, L represents the objective function; Q represents the total length of the sequence; q represents the current position index in the sequence; c represents the size of the upper and lower position windows; w q represents the touch point at the current position index q; w q+d represents the touch point at the d position index after the current position index q; P(|) represents the probability symbol; P(w q+d |w q ) represents the touch point with predicted position index q+d given the current position index q; Σ represents the summation symbol; log represents the logarithmic symbol;

[0037] By receiving the user's voice instructions, gesture actions and touch operations and converting them into corresponding control instructions, the home appliance is subjected to multimodal interactive control according to the control instructions.

[0038] Preferably, the automatic scene triggering module includes:

[0039] A setting unit, used for presetting a plurality of the scene modes according to user requirements;

[0040] A trigger unit, used to automatically trigger the corresponding scene mode when the preset condition is met;

[0041] The adjustment unit is used to adjust the working state of the home appliance after the corresponding scene mode is automatically triggered.

[0042] Preferably, the preset conditions include: the current time is a preset trigger time, the current position is a preset trigger position, and the working state of the home appliance is a preset trigger state.

[0043] Preferably, the security protection module includes:

[0044] A data encryption unit, used for encrypting the transmission data;

[0045] A privacy protection unit, used for anonymizing the user data;

[0046] The monitoring and alarm unit is used to monitor the device status of the household device and trigger an alarm device when the device status is abnormal.

[0047] Preferably, the energy-saving and environmental protection module includes:

[0048] A monitoring unit, configured to monitor the indoor environmental parameters through sensors and generate corresponding monitoring results, wherein the indoor environmental parameters include temperature parameters, humidity parameters and light intensity parameters;

[0049] An adjusting unit, used for obtaining the preset energy-saving strategy, and adjusting the working state of the household equipment according to the monitoring result and the preset energy-saving strategy;

[0050] An optimization unit is used to optimize the preset energy-saving strategy according to the user's usage habits.

[0051] The present invention provides a smart home automation control method, comprising:

[0052] Connecting home appliances to a smart home network, converting the communication protocols of the home appliances to a unified protocol, and monitoring the working status of the home appliances;

[0053] Receive the user's voice commands, gestures and touch operations and convert them into corresponding control commands, and perform multimodal interactive control on the home appliance according to the control commands;

[0054] Preset scene mode, when the preset conditions are met, the corresponding scene mode is automatically triggered and the working state of the corresponding home appliance is adjusted;

[0055] Process the transmission data and user data securely, monitor the device status of the home device, and trigger an alarm device when the device status is abnormal;

[0056] Monitor indoor environmental parameters to generate corresponding monitoring results, adjust the working state of the home appliances according to the monitoring results and preset energy-saving strategies, and optimize the preset energy-saving strategies according to user usage habits.

[0057] Compared with the related art, the intelligent home automation control system and control method provided by the present invention have the following beneficial effects:

[0058] The present invention connects home appliances to a smart home network through a device interconnection module, a multimodal interactive control module, an automatic scene triggering module, a security protection module, and an energy-saving and environmental protection module, and converts the communication protocol of the home appliances into a unified protocol, while monitoring the working status of the home appliances; receives user voice commands, gestures, and touch operations and converts them into corresponding control commands, and performs multimodal interactive control on the home appliances according to the control commands; presets scene modes, and automatically triggers corresponding scene modes when preset conditions are met, and adjusts the working status of corresponding home appliances; performs secure processing on transmission data and user data, and monitors the device status of home appliances, and triggers an alarm device when the device status is abnormal; monitors indoor environmental parameters to generate corresponding monitoring results, adjusts the working status of home appliances according to the monitoring results and preset energy-saving strategies, and optimizes the preset energy-saving strategies according to user usage habits, thereby improving the compatibility of home appliances, ensuring the security of the control system, and preventing the leakage of transmission data and user information. The multiple control methods used at the same time improve the efficiency and accuracy of the home appliance control system, and the automation level is high, and the user experience effect is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A system block diagram of an intelligent home automation control system of the present invention;

[0060] Figure 2 It is a structural block diagram of a device interconnection module of a smart home automation control system of the present invention;

[0061] Figure 3 It is a structural block diagram of an automatic scene triggering module of an intelligent home automation control system of the present invention;

[0062] Figure 4 It is a structural block diagram of a security protection module of a smart home automation control system of the present invention;

[0063] Figure 5 It is a structural block diagram of an energy-saving and environmental protection module of an intelligent home automation control system of the present invention;

[0064] Figure 6 The present invention is a flowchart of a smart home automation control method.

[0065] Description of reference numerals:

[0066] 1. Equipment interconnection module; 11. Equipment communication unit; 12. Protocol conversion unit; 13. Equipment management unit; 2. Multimodal interaction control module; 3. Automatic scene trigger module; 31. Setting unit; 32. Trigger unit; 33. Adjustment unit; 4. Security protection module; 41. Data encryption unit; 42. Privacy protection unit; 43. Monitoring and alarm unit; 5. Energy-saving and environmental protection module; 51. Monitoring unit; 52. Adjustment unit; 53. Optimization unit. DETAILED DESCRIPTION

[0067] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.

[0068] Embodiment 1

[0069] like Figure 1 As shown, a smart home automation control system comprises:

[0070] Device interconnection module 1, used to connect home devices to the smart home network, convert the communication protocol of the home devices into a unified protocol, and monitor the working status of the home devices;

[0071] The multimodal interactive control module 2 is used to receive the user's voice instructions, gestures and touch operations and convert them into corresponding control instructions, and perform multimodal interactive control on the home appliance according to the control instructions;

[0072] The automatic scene trigger module 3 is used to preset the scene mode, and when the preset conditions are met, the corresponding scene mode is automatically triggered and the working state of the corresponding home appliance is adjusted;

[0073] The security protection module 4 is used to process the transmission data and user data securely and monitor the device status of the household device. When the device status is abnormal, an alarm device is triggered;

[0074] The energy-saving and environmental protection module 5 is used to monitor indoor environmental parameters to generate corresponding monitoring results, adjust the working state of the household appliances according to the monitoring results and preset energy-saving strategies, and optimize the preset energy-saving strategies according to user usage habits.

[0075] In practical applications, the device interconnection module 1 uses wireless communication technologies, such as Wi-Fi, Zigbee, and Bluetooth, to achieve interconnection between home appliances. Users can easily remotely control home appliances through mobile devices such as mobile phones and tablets. In addition, the smart home automation control system of the present invention supports multiple communication protocols, thereby ensuring that home appliances of different brands and types can be seamlessly connected and work together.

[0076] Secondly, since the system has built-in voice recognition technology, users can control home appliances with simple voice commands, such as turning on and off smart TVs, smart air conditioners, and adjusting the volume. In addition, the multimodal interactive control module supports the recognition and understanding of complex voice commands, which can improve the convenience of user operation. In addition to voice control, the system also supports gesture control and touch control interaction. Users can interact with home appliances in a variety of ways, which can provide users with a more convenient operating experience. In addition, the system also integrates a smart home cloud platform, through which users can remotely control home appliances and set scenes.

[0077] In addition, users can preset multiple automated scene modes through the automatic scene trigger module, such as home mode, movie mode, and leaving mode. In the corresponding scene mode, home appliances can automatically adjust to the appropriate working state. In this way, users can flexibly set and adjust scene modes according to the different needs of family members, thereby improving the quality of life.

[0078] It should be noted that the system uses advanced encryption technology to ensure the secure transmission of data. At the same time, the security protection module can anonymously process user voice commands, thereby effectively protecting user privacy. In addition, the system also has abnormal detection and alarm functions. When abnormal situations such as illegal intrusion and equipment failure occur, it can automatically alarm and send reminder information to the user's mobile phone.

[0079] Furthermore, the system can automatically adjust the working status of home appliances according to the indoor and outdoor environment and user needs. For example, by sensing the indoor temperature, the smart air conditioner can automatically adjust to the appropriate temperature, thereby achieving the effect of energy saving and emission reduction. At the same time, it can automatically adjust according to user usage habits to reduce energy consumption.

[0080] In the specific implementation process, Figure 2 As shown, the device interconnection module 1 includes a device communication unit 11, a protocol conversion unit 12 and a device management unit 13:

[0081] A device communication unit 11, used for wirelessly connecting the household devices;

[0082] A protocol conversion unit 12, used to convert the communication protocols of different household devices into the unified protocol recognizable by the system;

[0083] The device management unit 13 is used to manage the home devices and monitor the working status of the home devices.

[0084] It is understandable that the device communication unit 11 can be a communication device such as Wi-Fi, Zigbe and Bluetooth, and the home appliances can access the smart home network through the built-in device communication unit 11, and wireless connection can be achieved between home appliances.

[0085] In practical applications, the protocol conversion unit 12 can convert the communication protocols of different devices into a unified protocol that can be recognized by the system, thereby ensuring the compatibility and smooth communication between home appliances. The device management unit can complete the management work such as registration, authentication and status monitoring of the corresponding home appliances, thereby ensuring the normal access and use of the corresponding home appliances.

[0086] The multimodal interactive control module 2 is used to receive the user's voice instructions, gestures and touch operations according to the following interactive control algorithm and convert them into corresponding control instructions, and perform multimodal interactive control on the home appliance according to the control instructions:

[0087] The user's voice command is collected by a microphone and converted into a corresponding voice control command, and the voice control of the home appliance is completed according to the voice control command, including the following operations:

[0088] The collected voice command is divided into multiple frames, features of each frame are extracted to obtain feature sequences of all frames, the feature sequences and the voice command are modeled to obtain a corresponding relationship between the feature sequences and the voice command, and a hidden state sequence, i.e., the voice control command, is determined according to the state transition probability, the observation probability, and the initial state probability;

[0089] The expression of the state transition probability is:

[0090] a ij =P(q t+1 =S j |q t =S i )

[0091] In the formula, a ij Represents the hidden state S i Transfer to hidden state S j The state transition probability; P(|) represents the probability symbol; q t+1 represents the hidden state variable at time t+1; q t represents the hidden state variable at time t; S j , S i Indicates hidden state;

[0092] For example, when the light is turned on and off by voice control, its hidden state (i.e., voice control command) includes two states: light on state and light off state. According to the calculation formula of state transition probability, the corresponding state transition probability matrix can be obtained: For this matrix, a 11 =0.8 means that when the current hidden state is the light-on state, the probability that the next hidden state will still be the light-on state is 0.8; a 12 =0.2 means that when the current hidden state is the light-on state, the probability that the next hidden state will become the light-off state is 0.2; a 21 =0.3 means that when the current hidden state is the light-off state, the probability that the next hidden state will become the light-on state is 0.3; a 22 =0.7 means that when the current hidden state is the light-off state, the probability that the next hidden state is still the light-off state is 0.7;

[0093] The expression of the observation probability is:

[0094] b j (k) = P(O t =V k |q t =S j )

[0095] Where b j (k) represents the hidden state S j Generate observations V k The observation probability; P(|) represents the probability symbol; O t represents the observed variable at time t; q t represents the hidden state variable at time t; V k represents the observed value; S j Indicates hidden state;

[0096] For example, according to the calculation formula of observation probability, the corresponding observation probability matrix is ​​obtained: For this matrix, b 11 =0.9 means that when the current hidden state is the light-on state, the probability of observing the light-on state is 0.9; b 12 = 0.1 means that when the current hidden state is the light-on state, the probability of observing the light-off state is 0.1; b 21 =0.2 means that when the current hidden state is the light-off state, the probability of observing the light-on state is 0.2; b 22 =0.8 means that when the current hidden state is the light-off state, the probability of observing the light-off state is 0.8;

[0097] The expression of the initial state probability is:

[0098] π i =P(q1=S i )

[0099] In the formula, π iIndicates that it is in the hidden state S at the initial moment i The initial state probability; P(|) represents the probability symbol; q1 represents the hidden state variable at the initial moment; S i Indicates hidden state;

[0100] For example, according to the calculation formula of the initial state probability, the corresponding initial state probability vector is obtained: For this matrix, c1=0.6 means that the probability that the system is in the light-on state at the initial moment is 0.6; c2=0.4 means that the probability that the system is in the light-off state at the initial moment is 0.4;

[0101] Determine the hidden state sequence, i.e., the voice control instruction, according to the state transition probability, the observation probability and the initial state probability;

[0102] Recognizing the user's gestures through a camera and converting them into corresponding action control instructions, and completing the action control of the home appliance according to the action control instructions, including the following operations:

[0103] The camera is used to recognize the user's gesture to generate an input image, and feature extraction is performed on the input image to obtain output image features. The expression of the output image features is:

[0104] O(u,v)=(I*K)(u,v)=∑ m ∑ n I(u+m)(v+n)K(m,n)

[0105] Where, O represents the output image feature; u and v represent the coordinates of the output image feature; I represents the input image; K represents the convolution kernel; m and n represent the convolution kernel size;

[0106] Obtaining the action control instruction according to the output image feature, and completing the action control of the home appliance according to the action control instruction;

[0107] The user's touch operation on the touch interface is processed by the following positioning formula, and the voice command and gesture action are recognized and processed accordingly:

[0108]

[0109] In the formula, L represents the objective function; Q represents the total length of the sequence; q represents the current position index in the sequence; c represents the size of the upper and lower position windows; w q represents the touch point at the current position index q; w q+d represents the touch point at the d position index after the current position index q; P(|) represents the probability symbol; P(w q+d |w q) represents the touch point with predicted position index q+d given the current position index q; Σ represents the summation symbol; log represents the logarithmic symbol;

[0110] In this embodiment, it is assumed that a 3×3 input image and a convolution kernel size of 2 are used. The input image is a grayscale image, that is, each pixel has only one brightness value. Convolution Kernel According to the calculation formula of the output image features, the output image features can be obtained

[0111] By receiving the user's voice instructions, gesture actions and touch operations and converting them into corresponding control instructions, the home appliance is subjected to multimodal interactive control according to the control instructions.

[0112] It should be noted that, based on the interactive control algorithm, the convolutional neural network algorithm and the touch positioning algorithm, the present invention can collect the user's voice instructions through a microphone, and convert them into corresponding voice control instructions, and send the voice control instructions to the corresponding home appliance, so that the home appliance can complete the operation corresponding to the user's voice instruction according to the voice control instruction, and realize the voice control of the home appliance. At the same time, the user's gestures can be recognized by the camera, and converted into corresponding action control instructions, and the action control instructions are sent to the corresponding home appliance, so that the home appliance can complete the operation corresponding to the user's gestures according to the action control instructions, and realize the action control of the home appliance. In addition, the present invention provides a physical control interface of the touch interface, and the user can complete the corresponding touch operation through the touch interface, thereby realizing intelligent human-computer interaction.

[0113] In actual applications, users can interact with the smart home system through a variety of control methods such as voice commands, gestures and touch operations. After receiving the user's voice commands, gestures and touch operations, they can be converted into corresponding control instructions, and home appliances can perform corresponding operations according to these control instructions, thereby realizing multi-modal interactive control of home appliances.

[0114] like Figure 3 As shown, the automatic scene triggering module 3 includes a setting unit 31, a triggering unit 32 and an adjusting unit 33:

[0115] A setting unit 31, used for presetting a plurality of scene modes according to user requirements;

[0116] A trigger unit 32, configured to automatically trigger the corresponding scene mode when the preset condition is met;

[0117] The adjustment unit 33 is used to adjust the working state of the home appliance after the corresponding scene mode is automatically triggered.

[0118] The preset conditions include: the current time is a preset trigger time, the current position is a preset trigger position, and the working state of the home appliance is a preset trigger state.

[0119] In actual applications, the setting unit 31 can be used to preset multiple automation scene modes according to user needs, such as home mode, away mode, and movie watching mode.

[0120] It can be understood that the preset trigger time refers to the time preset in advance by the staff to trigger the corresponding scene mode, the preset trigger position refers to the position preset in advance by the staff to trigger the corresponding scene mode, and the preset trigger state refers to the working state of the home appliance that triggers the corresponding scene mode preset in advance by the staff. If the current time is the preset trigger time, the current position is the preset trigger position, and the working state of the home appliance is the preset trigger state, the preset conditions are met, and the corresponding scene mode can be automatically triggered at this time. After determining the triggered scene mode, the working state of the home appliance can be adjusted accordingly.

[0121] In the above manner, users can preset multiple automation scene modes through mobile devices or cloud platforms. Among them, the trigger unit 32 can monitor in real time whether the preset conditions are met. When the preset conditions are met, the trigger unit 33 automatically triggers the corresponding scene mode, and then the adjustment unit 33 can automatically adjust the working state of the home appliance, so that multiple control methods can be used at the same time, which improves the efficiency and accuracy of the home appliance control system, and has a high degree of automation and a better user experience.

[0122] like Figure 4 As shown, the security protection module 4 includes a data encryption unit 41, a privacy protection unit 42 and a monitoring alarm unit 43:

[0123] A data encryption unit 41, used for encrypting the transmission data;

[0124] A privacy protection unit 42, used for anonymizing the user data;

[0125] The monitoring alarm unit 43 is used to monitor the device status of the household device and trigger an alarm device when the device status is abnormal.

[0126] It is understandable that the data encryption unit 41 can encrypt the transmission data, thereby ensuring the security of the data during transmission. The privacy protection unit 42 can anonymize the user data, thereby preventing the user's personal information from being leaked. The monitoring alarm unit 43 can monitor the device status of the home appliance in real time. Once an abnormal situation is found, such as illegal intrusion and device failure, the alarm device can be immediately triggered and a reminder message can be sent to the user.

[0127] Through the above method, the security of the control system can be guaranteed, the leakage of transmission data and user information can be prevented, and the efficiency and accuracy of the home appliance control system can be improved.

[0128] like Figure 5 As shown, the energy-saving and environmental protection module 5 includes a monitoring unit 51, a regulating unit 52 and an optimizing unit 53:

[0129] A monitoring unit 51, configured to monitor the indoor environmental parameters through sensors and generate corresponding monitoring results, wherein the indoor environmental parameters include temperature parameters, humidity parameters and light intensity parameters;

[0130] An adjusting unit 52, configured to obtain the preset energy-saving strategy, and adjust the working state of the household device according to the monitoring result and the preset energy-saving strategy;

[0131] The optimization unit 53 is used to optimize the preset energy-saving strategy according to the user's usage habits.

[0132] In practical applications, the monitoring unit 51 can monitor indoor environmental parameters in real time through sensors and generate corresponding monitoring results, wherein the indoor environmental parameters include temperature parameters, humidity parameters and light intensity parameters.

[0133] It is understandable that the preset energy-saving strategy refers to the usage strategy of the corresponding household equipment with the least energy consumption preset in advance by the staff. After obtaining the preset energy-saving strategy, the adjustment unit 52 can adjust the working state of the household equipment according to the monitoring results and the preset energy-saving strategy, such as automatically adjusting the air-conditioning temperature and turning off unnecessary lighting, etc., and the optimization unit 53 can perform adaptive learning by recording the user's usage habits and continuously optimize the preset energy-saving strategy.

[0134] Through the above method, indoor environmental parameters such as temperature, humidity and light intensity can be monitored in real time, the working status of corresponding home appliances can be automatically adjusted according to the preset energy-saving strategy, and the preset energy-saving strategy can be optimized accordingly according to user usage habits, thereby improving the automation degree and work efficiency of the control system and ensuring good energy-saving effects.

[0135] Embodiment 2

[0136] like Figure 6 As shown, a smart home automation control method includes:

[0137] S1, connecting the home appliances to the smart home network, converting the communication protocols of the home appliances to a unified protocol, and monitoring the working status of the home appliances;

[0138] S2, receiving the user's voice command, gesture action and touch operation and converting them into corresponding control instructions, and performing multimodal interactive control on the home appliance according to the control instructions;

[0139] S3, preset scene mode, when the preset conditions are met, the corresponding scene mode is automatically triggered and the working state of the corresponding home appliance is adjusted;

[0140] S4, securely processing the transmission data and user data, and monitoring the device status of the household device, and triggering an alarm device when the device status is abnormal;

[0141] S5, monitor indoor environmental parameters to generate corresponding monitoring results, adjust the working state of the home appliances according to the monitoring results and the preset energy-saving strategy, and optimize the preset energy-saving strategy according to the user's usage habits.

[0142] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0143] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0144] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

Claims

1. A smart home automation control system, It is characterized in that include: A device interconnection module is used to connect home devices to the smart home network, convert the communication protocol of the home devices to a unified protocol, and monitor the working status of the home devices; A multimodal interactive control module, used to receive the user's voice instructions, gestures and touch operations and convert them into corresponding control instructions, and perform multimodal interactive control on the home appliance according to the control instructions; An automatic scene triggering module is used to preset a scene mode. When a preset condition is met, the corresponding scene mode is automatically triggered and the working state of the corresponding home appliance is adjusted; A security protection module, used for processing transmission data and user data securely, and monitoring the device status of the household device, and triggering an alarm device when the device status is abnormal; The energy-saving and environmental protection module is used to monitor indoor environmental parameters to generate corresponding monitoring results, adjust the working status of the home appliances according to the monitoring results and preset energy-saving strategies, and optimize the preset energy-saving strategies according to user usage habits.

2. A smart home automation control system according to claim 1, It is characterized in that The device interconnection module comprises: A device communication unit, used for wirelessly connecting the household devices; A protocol conversion unit, used to convert the communication protocols of different household devices into the unified protocol recognizable by the system; The device management unit is used to manage the home devices and monitor the working status of the home devices.

3. The smart home automation control system according to claim 1, It is characterized in that The multimodal interactive control module is used to: receive the user's voice instructions, gesture actions and touch operations according to the following interactive control algorithm and convert them into corresponding control instructions, and perform multimodal interactive control on the home appliance according to the control instructions: The user's voice command is collected by a microphone and converted into a corresponding voice control command, and the voice control of the home appliance is completed according to the voice control command, including the following operations: The collected voice command is divided into multiple frames, features of each frame are extracted to obtain feature sequences of all frames, the feature sequences and the voice command are modeled to obtain a corresponding relationship between the feature sequences and the voice command, and a hidden state sequence, i.e., the voice control command, is determined according to the state transition probability, the observation probability, and the initial state probability; The expression of the state transition probability is: a ij =P(q t+1 =S j |q t =S i ) In the formula, a ij Represents the hidden state S i Transfer to hidden state S j The state transition probability; P(|) represents the probability symbol; q t+1 represents the hidden state variable at time t+1; q t represents the hidden state variable at time t; S j , S i Indicates hidden state; The expression of the observation probability is: b j (k)=P(O t =V k |q t =S j ) Where b j (k) represents the hidden state S j Generate observations V k The observation probability; P(|) represents the probability symbol; O t represents the observed variable at time t; q t represents the hidden state variable at time t; V k represents the observed value; S j Indicates hidden state; The expression of the initial state probability is: π i =P(q 1 =S i ) In the formula, π i Indicates that it is in the hidden state S at the initial moment i The initial state probability; P(|) represents the probability symbol; q 1 represents the hidden state variable at the initial moment; S i Indicates hidden state; Determine the hidden state sequence, i.e., the voice control instruction, according to the state transition probability, the observation probability and the initial state probability; Recognizing the user's gestures through a camera and converting them into corresponding action control instructions, and completing the action control of the home appliance according to the action control instructions, including the following operations: The camera is used to recognize the user's gesture to generate an input image, and feature extraction is performed on the input image to obtain output image features. The expression of the output image features is: o(u,v)=(I*K)(u,v)=∑ m ∑ n I(u+m)(v+n)K(m,n) Where O represents the output image feature; u and v represent the coordinates of the output image features; I represents the input image; K represents the convolution kernel; m and n represent the convolution kernel size; Obtaining the action control instruction according to the output image feature, and completing the action control of the home appliance according to the action control instruction; The user's touch operation on the touch interface is processed by the following positioning formula, and the voice command and gesture action are recognized and processed accordingly: In the formula, L represents the objective function; Q represents the total length of the sequence; q represents the current position index in the sequence; c represents the size of the upper and lower position windows; w q represents the touch point at the current position index q; w q+d represents the touch point at the d position index after the current position index q; P(|) represents the probability symbol; P(w q+d |w q ) represents a touch point with a predicted position index q+d given a current position index q; ∑ represents a summation symbol; log represents a logarithmic symbol; By receiving the user's voice instructions, gesture actions and touch operations and converting them into corresponding control instructions, the home appliance is subjected to multimodal interactive control according to the control instructions.

4. The smart home automation control system according to claim 1, It is characterized in that The automatic scene triggering module comprises: A setting unit, used for presetting a plurality of the scene modes according to user requirements; A trigger unit, used to automatically trigger the corresponding scene mode when the preset condition is met; The adjustment unit is used to adjust the working state of the home appliance after the corresponding scene mode is automatically triggered.

5. A smart home automation control system according to claim 4, It is characterized in that The preset conditions include: the current time is a preset trigger time, the current position is a preset trigger position, and the working state of the home appliance is a preset trigger state.

6. The smart home automation control system according to claim 1, It is characterized in that The security protection module comprises: A data encryption unit, used for encrypting the transmission data; A privacy protection unit, used for anonymizing the user data; The monitoring and alarm unit is used to monitor the device status of the household device and trigger an alarm device when the device status is abnormal.

7. The smart home automation control system according to claim 1, It is characterized in that The energy-saving and environmental protection module comprises: A monitoring unit, configured to monitor the indoor environmental parameters through sensors and generate corresponding monitoring results, wherein the indoor environmental parameters include temperature parameters, humidity parameters and light intensity parameters; An adjusting unit, used for obtaining the preset energy-saving strategy, and adjusting the working state of the household equipment according to the monitoring result and the preset energy-saving strategy; An optimization unit is used to optimize the preset energy-saving strategy according to the user's usage habits.

8. A smart home automation control method, applied to a smart home automation control system as claimed in any one of claims 1 to 7, wherein the control method include: Connecting home appliances to a smart home network, converting the communication protocols of the home appliances to a unified protocol, and monitoring the working status of the home appliances; Receive the user's voice commands, gestures and touch operations and convert them into corresponding control commands, and perform multimodal interactive control on the home appliance according to the control commands; Preset scene mode, when the preset conditions are met, the corresponding scene mode is automatically triggered and the working state of the corresponding home appliance is adjusted; Process the transmission data and user data securely, monitor the device status of the home device, and trigger an alarm device when the device status is abnormal; Monitor indoor environmental parameters to generate corresponding monitoring results, adjust the working state of the home appliances according to the monitoring results and preset energy-saving strategies, and optimize the preset energy-saving strategies according to user usage habits.

Images