Equipment control method and device of smart home equipment, electronic equipment and medium

Abstract

The invention discloses an equipment control method and device for smart home equipment, electronic equipment and a medium, and belongs to the technical field of equipment control. Comprising the steps of obtaining one or more pieces of environment data in a process that one or more pieces of home equipment operate according to a first scene mode; generating an environment feature vector; inputting the environment feature vector into a preset scene classifier, and outputting a prediction scene corresponding to the environment feature vector and a prediction probability of the prediction scene; when it is judged that the first scene mode is triggered to be switched into a second scene mode corresponding to the prediction scene according to the prediction scene and the prediction probability, an equipment control sub-module corresponding to the second scene mode is loaded in a local database, and the equipment control sub-module and a cloud training meta-model share the same convolutional backbone structure; performing fine tuning on the equipment control sub-model based on the environment feature vector in the second scene mode; and in the second scene mode, controlling one or more pieces of home equipment by adopting the finely-adjusted equipment control sub-model.

Description

Technical Field

[0001] This application belongs to the field of device control technology, specifically relating to a device control method for a smart home device, a device control apparatus for a smart home device, an electronic device, and a readable storage medium. Background Technology

[0002] In existing technologies, smart home systems typically achieve device interaction through predefined rule sets or a single AI (Artificial Intelligence) model. Because rule engines rely on static condition judgments (such as "turn on the air conditioner if the temperature is >28℃"), the system cannot dynamically adjust its strategy when environmental parameters change abruptly or user behavior is abnormal, leading to inaccurate device responses (such as accidentally turning on lights at night). Furthermore, since different scenarios require independently trained dedicated models (such as sleep mode models and entertainment mode models), data needs to be collected and retrained when switching models, resulting in wasted computing resources and increased response latency. Summary of the Invention

[0003] The purpose of this application is to provide a device control method, apparatus, electronic device, and medium for smart home devices, which can solve the problems of wasted computing resources and increased response latency caused by switching models in different scenarios.

[0004] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a device control method for a smart home device, the method comprising: During the operation of one or more home devices in the first scene mode, one or more environmental data are acquired; Generate an environmental feature vector based on the one or more environmental data; The environmental feature vector is input into a preset scene classifier, and the predicted scene corresponding to the environmental feature vector and the predicted probability of the predicted scene are output. When the first scene mode is switched to the second scene mode corresponding to the predicted scene based on the predicted scene and the predicted probability, the device control sub-model corresponding to the second scene mode is loaded in the local database. The device control sub-module shares the same convolutional backbone structure with the meta-model trained in the cloud. The device control sub-model is fine-tuned based on the environmental feature vectors in the second scenario mode; In the second scenario mode, a finely tuned device control sub-model is used to control one or more home devices.

[0005] Optionally, the step of determining to switch the first scene mode to the second scene mode corresponding to the predicted scene based on the predicted scene and the predicted probability includes: If, within N consecutive detection periods, the predicted scene does not match the first scene mode and the predicted probability is greater than the probability threshold, it is determined that the first scene mode is switched to the second scene mode corresponding to the predicted scene, where N is a positive integer greater than or equal to 1.

[0006] Optionally, the fine-tuning of the device control sub-model based on the environmental feature vector under the second scene mode includes: Determine the environmental feature vector and the device control command in the second scene mode; The fully connected layer of the device control sub-model is fine-tuned using the environmental feature vector and the device control command as training data.

[0007] Optionally, the step of using a finely tuned device control sub-model to control one or more home devices in the second scenario mode includes: In the second scenario mode, a real-time feature vector generated based on one or more environmental data is obtained; The real-time feature vector is input into the device control sub-model, and one or more device control commands corresponding to the real-time feature vector are output. Device control of one or more home appliances in accordance with the one or more device control instructions.

[0008] Optionally, the device control of one or more home appliances according to the one or more device control instructions includes: The probability of the target scene corresponding to the second scene mode is determined based on the second scene mode and the scene classifier. Obtain current time information and device priority rules based on scenario and time settings; The priority information of each home device is determined based on the target scenario probability, the current time information, and the priority rules. Device control of one or more home appliances is performed according to the priority information and the one or more control commands.

[0009] Optionally, the device control of one or more home appliances according to the priority information and the one or more control commands includes: Identify target home appliances whose priority information is greater than a preset priority threshold; Determine the target control command corresponding to the target home appliance from the one or more control commands; The target home appliance is controlled according to the target control command.

[0010] Optionally, during the operation of one or more home devices in the first scene mode, acquiring one or more environmental data includes: During the operation of one or more home devices in the first scene mode, light data in the target environment is acquired through a light sensor; And / or, during the operation of one or more home devices in accordance with the first scene mode, the motion data of the target person in the target environment is acquired by infrared sensors; And / or, during the operation of one or more home devices in a first scene mode, sound data in the target environment is acquired through a sound sensor.

[0011] Optionally, generating an environmental feature vector based on the one or more environmental data includes: Construct an original feature vector based on the one or more environmental data; Determine the weight data corresponding to the one or more environmental data based on the first scenario mode; The weighted feature vector is determined based on the original feature vector and the weight data; The weighted feature vector is normalized to obtain the environmental feature vector.

[0012] Optionally, constructing the original feature vector based on the one or more environmental data includes: Determine the alignment period corresponding to the one or more environmental data; Align the one or more environmental data according to the alignment period to obtain aligned data; The original feature vector is constructed based on the one or more aligned data.

[0013] Secondly, embodiments of this application provide a device control apparatus for a smart home device, the apparatus comprising: The environmental data acquisition module is used to acquire one or more environmental data during the operation of one or more home devices in the first scene mode; An environmental feature vector generation module is used to generate an environmental feature vector based on the one or more environmental data. The scene prediction module is used to input the environmental feature vector into a preset scene classifier and output the predicted scene corresponding to the environmental feature vector and the prediction probability of the predicted scene. The scene mode switching module is used to load the device control sub-model corresponding to the second scene mode in the local database when the first scene mode is switched to the second scene mode corresponding to the predicted scene based on the predicted scene and the predicted probability. The device control sub-module shares the same convolutional backbone structure with the meta-model trained in the cloud. The sub-model fine-tuning module is used to fine-tune the device control sub-model based on the environmental feature vectors in the second scenario mode; The device control module is used to control one or more home devices in the second scenario mode using a finely tuned device control sub-model.

[0014] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0015] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0016] In this embodiment, during the operation of one or more home devices in a first scene mode, an environmental feature vector is generated based on the acquired environmental data. Then, the scene and probability can be predicted based on the environmental feature vector. When the scene mode is switched according to the predicted scene and the predicted probability, the device control sub-model is loaded and fine-tuned to realize device control. Since the device control sub-model in this embodiment is a model that shares the same convolutional backbone structure with the cloud meta-model, there are few parameters that need to be fine-tuned. Therefore, when the scene mode is switched and the corresponding model needs to be replaced, the computing resources can be reduced and the response latency can be reduced. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a device control method for a smart home device according to an embodiment of this application; Figure 2a This is a flowchart illustrating a device control method for a smart home device according to an embodiment of this application; Figure 2b This is a flowchart illustrating a device control method according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a device control device for a smart home device according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0020] Current smart home systems typically employ fixed rule engines or single AI models to achieve device linkage. Because rule engines rely on static condition judgments (e.g., "turn on the air conditioner if the temperature > 28℃"), the system cannot dynamically adjust its strategy when environmental parameters change abruptly or user behavior becomes abnormal, leading to inaccurate device responses (e.g., accidentally turning on lights at night). For example, a scene linkage automatic generation system based on the FP-Tree algorithm has been proposed, generating strong association rules by mining frequent itemsets from device usage records. However, the following problems exist: Poor scene generalization ability: Fixed rules are difficult to adapt to dynamic environmental changes (such as sudden changes in lighting or abnormal user behavior), resulting in an increased false trigger rate (actual error rate > 15%). High model switching costs: Different scenarios require independent training of dedicated models, and data needs to be recollected during migration, increasing computational resource consumption by more than 30%.

[0021] In practical applications, the ReactorQL rule engine can also be used to achieve device linkage, and rules can be optimized through a feedback mechanism. However, the following problems exist: Lack of multimodal perception: Relying solely on device status data and ignoring the correlation between environmental parameters (temperature, humidity, and light) and user behavior, scene recognition accuracy is less than 80%; Insufficient real-time performance: Rule matching requires multiple cloud-to-device data transmissions, with an average latency of 200ms, which cannot meet the needs of emergency scenarios such as security.

[0022] Furthermore, since different scenarios require independent training of dedicated models (such as sleep mode model and entertainment mode model), data needs to be collected and retrained when switching models, resulting in wasted computing resources and increased response latency.

[0023] In this embodiment, a device control sub-model that shares the same convolutional backbone structure as the meta-model trained in the cloud can be stored in a local database. Furthermore, multimodal environmental data can be fused to identify home scenes in real time, and transfer learning can be used to achieve rapid switching of sub-model parameters, thus solving the dual problems of poor scene adaptability and high switching costs.

[0024] The embodiments provided in this application are described below with reference to the accompanying drawings and specific examples and application scenarios. Figure 1 A detailed explanation will be provided.

[0025] Reference Figure 1 The diagram illustrates a flowchart of a device control method for a smart home device according to an embodiment of this application, which may specifically include the following steps: Step S101: During the operation of one or more home devices in the first scene mode, acquire one or more environmental data. In practical applications, home appliance control systems can be configured with different scene modes for different scenarios. Under each scene mode, the corresponding device control model can be run to control one or more home appliances. In this embodiment, control can be based on sleep scenarios, home scenarios, security scenarios, etc., and can be performed according to the characteristics of each scenario.

[0026] To achieve precise control of home appliances, when one or more home appliances are running in the first scene mode, one or more environmental data can be obtained from sensors in the monitoring area used to collect environmental data. This environmental data can be data in different modalities, which can display the environmental status of the current monitoring area from various aspects, and thus accurately reflect the current real-time scene.

[0027] In one embodiment of this application, during the operation of one or more home devices in a first scene mode, illumination data in the target environment can be acquired by a light sensor; and / or, during the operation of one or more home devices in a first scene mode, motion data of a target person in the target environment can be acquired by an infrared sensor; and / or, during the operation of one or more home devices in a first scene mode, sound data in the target environment can be acquired by a sound sensor.

[0028] In addition to the sensors mentioned above, other sensors can be introduced to achieve comprehensive collection of environmental data.

[0029] Step S102: Generate an environmental feature vector based on the one or more environmental data; After obtaining multimodal environmental data, feature analysis can be performed on the multimodal environmental data to obtain an environmental feature vector that integrates the multimodal data.

[0030] In one embodiment of this application, generating an environmental feature vector based on the one or more environmental data includes: constructing an original feature vector based on the one or more environmental data; determining weight data corresponding to the one or more environmental data according to the first scene mode; determining a weighted feature vector according to the original feature vector and the weight data; and normalizing the weighted feature vector to obtain the environmental feature vector.

[0031] After obtaining one or more environmental data, an original feature vector can be obtained by fusing the data. In practical applications, different weight data can be set based on different scenarios. When the current scenario is the first scenario mode, the weight data corresponding to each type of environmental data is determined according to the first scenario mode. Then, the original feature vector is weighted according to the weight data to determine the weighted feature vector. The weighted feature vector can then be normalized to obtain the environmental feature vector, so that data in different intervals can be converted into the same dimension, which facilitates subsequent calculations.

[0032] In one embodiment of this application, constructing an original feature vector based on the one or more environmental data includes: determining the alignment period corresponding to the one or more environmental data; aligning the one or more environmental data according to the alignment period to obtain aligned data; and constructing an original feature vector based on the one or more aligned data.

[0033] In this embodiment, the alignment period can be set according to the actual scenario. Specifically, the alignment period can be set according to the upload cycle of multiple environmental data. For example, the local gateway receives data packets uploaded by each sensor every 200ms. Since the sampling rate of each sensor is different, the system maintains a short-term data buffer (such as 1 second depth), so 200ms can be determined as the alignment period.

[0034] In practical applications, different types of environmental data have different collection periods. When different types of environmental data exist, in order to determine the environmental data of different dimensions at the same time, it is necessary to perform time alignment processing on the collected environmental data of different dimensions. Specifically, after determining the alignment period, one or more environmental data points can be identified as corresponding data within that period, i.e., aligned data. After aligning the data, an environmental feature vector can be constructed based on the aligned data. For example, within an alignment period, a set of aligned data is obtained: environmental data 1, environmental data 2, and environmental data 3. The corresponding environmental feature vector can be [environmental data 1, environmental data 2, and environmental data 3].

[0035] In the embodiments of this application, spatiotemporal alignment can solve the problem of time sequence mismatch in heterogeneous sensor data. Specifically, a Kalman filter algorithm can be used for data synchronization and prediction. For example, for a human infrared signal with only one effective value every 500ms, the system uses the predicted value of the Kalman filter to fill in the gaps in the middle moments, forming a continuous and smooth estimation sequence P_est(t), which is aligned with the high-frequency illumination data L(t) on the time axis.

[0036] The following example illustrates the process of generating environmental feature vectors: A. Construction of original features: Within each 200ms decision cycle of the local gateway, the system extracts spatiotemporally aligned sensor data from the cache and constructs the original feature vector:

[0037] in: L(t) ∈ [0, 65535]Lux, from the BH1750 sensor; Pest(t) ∈ [0, 1] is the probability of a human presence predicted by the Kalman filter (0 indicates no one is there, and 1 indicates a high degree of certainty that someone is there). D(t) ∈ [30, 120] dB, which is the intensity of the sound event extracted by the MEMS microphone via Short-Time Energy (STE).

[0038] Example data: In the case of a late-night anomaly, L(t) = 5, Pest(t) = 0.95, D(t) = 90, therefore...

[0039] B. Dynamic weight allocation: The system maintains a scene-weight mapping table and dynamically adjusts the weights of each dimension based on the current scene mode. The weight calculation formula is as follows:

[0040] in, For indicator functions (such as) = 1 (current suspected security scenario), the coefficient is determined by offline calibration. Typical values ​​are shown in the table below:

[0041] Example: When the scene classifier outputs P(Ssecurity) = 0.85 > 0.75, the system determines that it enters the "security" mode, so the security weights are: Wl = 0.1, Wp = 0.8, Wd = 0.9.

[0042] C. Weighting and Normalization: The weighted eigenvector is calculated as follows:

[0043] Then, Min-Max normalization is performed to the ([0,1]) interval to adapt to the neural network input:

[0044] The preset ranges are: L ∈ [0, 1000], P ∈ [0,1], D ∈ [30,100].

[0045] Example calculation:

[0046] After normalization:

[0047] Final input vector: .

[0048] This application introduces a multi-sensor spatiotemporal fusion mechanism. This mechanism can align the time-series data of an illumination sensor (100ms / time) and a human infrared sensor (500ms / time) using Kalman filtering; and can dynamically allocate sensor weights according to the urgency of the scene (human infrared weight Wp=0.8 in security mode). This solves the problem of frequency mismatch in heterogeneous sensor data and reduces noise interference by 40%.

[0049] Step S103: Input the environmental feature vector into a preset scene classifier, and output the predicted scene corresponding to the environmental feature vector and the predicted probability of the predicted scene. In the equipment control system, a scene classifier can also be set up. This scene classifier can predict the current scene based on the input environmental feature vector, output the prediction probability corresponding to each scene, and thus determine the predicted scene and its corresponding prediction probability. The scene classifier is a pre-trained classification model. For example, the scene classifier is a lightweight one-dimensional convolutional neural network (1D-CNN).

[0050] In one embodiment of this application, when the predicted scene does not match the first scene mode within N consecutive detection cycles, and the predicted probability is greater than a probability threshold, it is determined that the first scene mode is switched to the second scene mode corresponding to the predicted scene, where N is a positive integer greater than or equal to 1. Determining this through consecutive detection cycles can prevent jitter and make the data more accurate.

[0051] The following examples illustrate scene recognition and model switching decisions: (1) Scene classification (analysis and prediction): The scene classifier is a lightweight one-dimensional convolutional neural network (1D-CNN).

[0052] A. Lightweight 1D-CNN architecture: The scene classifier uses a 3-layer 1D-CNN, with the following structure: Conv1D layer: 8 convolutional kernels, kernel size = 3, stride = 1, ReLU activation; MaxPooling1D: Pooling size = 2; Flatten + Fully Connected Layer: Output Dimension = 3 (corresponding to 3 scenarios); Softmax output: probability distribution P = [p1, p2, p3].

[0053] The forward propagation calculation process is as follows:

[0054] Example: Input Enorm = [0.0005, 0.76, 0.729], after kernel sliding (assuming kernel parameters are trained), output logits vector z = [-2.99, -2.30, 1.74], then:

[0055] B. Scene switching decision logic: The system adopts a threshold decision mechanism with a hysteresis zone: if max(P) > 0.75 and the corresponding scene ≠ the current scene; and the scene meets the condition (anti-jitter) for two consecutive cycles; then the model switching is triggered.

[0056] For example, if the current mode is "sleep", the output of the new cycle is P(Ssecurity) = 0.85>0.75, and the next cycle is still 0.88, the switching condition is met, and the security sub-model is triggered to load.

[0057] Step S104: When the first scene mode is switched to the second scene mode corresponding to the predicted scene according to the predicted scene and the predicted probability, the device control sub-model corresponding to the second scene mode is loaded in the local database. The device control sub-module shares the same convolutional backbone structure with the meta-model trained in the cloud. In practical applications, the meta-model in the cloud can be aligned with the sub-model structure on the local end. That is, all sub-models (such as Msleep, Mhome, Msecurity) share the same convolutional backbone structure (i.e., the first two Conv1D+Pooling layers) with the general meta-model trained in the cloud. Only the structure of the fully connected layer (FC) can be differentiated.

[0058] When the scene mode is switched to the target scene, perform the following operations: (1) Load the structure of the pre-stored sub-model Mi from the local SSD; (2) Share the meta-model weight Wshared = { from the cloud. , , , Copy to the corresponding layer in Mi; set `requires_grad = False` (PyTorch semantics) for these layers, i.e., freeze the parameter and prevent it from being updated in subsequent fine-tuning.

[0059] Step S105: Fine-tune the device control sub-model based on the environmental feature vector in the second scene mode; After invoking the device control sub-model, alignment can be fine-tuned to suit the current scenario.

[0060] In one embodiment of this application, an environmental feature vector and a device control command under the second scene mode can be determined; the fully connected layer of the device control sub-model can be fine-tuned using the environmental feature vector and the device control command as training data.

[0061] For example, in this embodiment of the application, only the fully connected layer of the sub-model may be considered. Fine-tune for 1–2 epochs, using the current alignment feature Enorm as training samples, with labels being the one-hot encoding of the current scene (e.g., security is [0,0,1]).

[0062] Optimizer: Adam, learning rate η = 0.001; Loss function: Weighted cross-entropy

[0063] Where wk = 1.5 if k is a security class, otherwise wk = 1.0; Training samples: Mini-batches (batch size=5) are constructed using the Enorm from the most recent 5 epochs.

[0064] Example: With Enorm = [0.0005, 0.76, 0.729] as input, label y = [0,0,1], initial output y' = [0.1, 0.2, 0.7], loss: L = -1.5 · log(0.7) = 0.535 After one backpropagation, Wfc is updated, making the output approach [0,0,1].

[0065] In one embodiment of this application, after fine-tuning the device control sub-model, the fine-tuned device control sub-model can be verified to determine the confidence level of the device control sub-model. When the confidence level is greater than a preset threshold, it is determined that the fine-tuning of the device control sub-model is finished, and step S1016 is executed.

[0066] For example, after fine-tuning, the system performs forward inference using the latest feature vector to verify that the output command meets expectations (such as outputting "Start camera"). If the confidence level is >0.9, the model is confirmed to be successfully adapted, and the system enters the execution phase.

[0067] After fine-tuning, the model achieves a 92% confidence level for the same input-output control vector [0.1, 0.05, 0.92], corresponding to the "security action", thus meeting the execution conditions.

[0068] Step S106: In the second scenario mode, the finely tuned device control sub-model is used to control one or more home devices.

[0069] In this embodiment of the application, by fine-tuning the local device sub-model, the computational load and latency can be reduced while quickly switching to the corresponding device sub-model in the current scene, and then using the device sub-model to control the home devices.

[0070] In this embodiment, a three-layer architecture comprising a scene classifier, a sub-model library, and a transfer adaptation layer can be established. An environmental feature vector is generated through multi-sensor fusion (lighting + human infrared + sound), input to the scene classifier, and outputs a scene label Si (e.g., S1 = sleep mode). Based on Si, a pre-trained model Mi is called from the sub-model library. The transfer adaptation layer inherits shared parameters from the meta-model, and only the output layer weights Wo are fine-tuned.

[0071] This invention marks the first time a meta-learning framework has been introduced into the field of home control, enabling "training once and reuse in multiple scenarios." By designing a transfer adaptation layer, the cost of model switching and retraining can be reduced by 90% (from 10 minutes to 1 minute). Furthermore, in this embodiment, the scene classifier's scene recognition accuracy is improved to 95%, with a response latency of <50ms.

[0072] The implementation of this application has the following advantages: (1) Precise scene adaptation: Through real-time environmental perception and meta-model switching, the control accuracy in complex scenes is increased from 82% to 96%; (2) Resource efficiency optimization: Transfer learning reduces repetitive model training, and cloud computing load is reduced by 70%; (3) Emergency response guarantee: Local edge devices perform model switching, and the response speed of security scenarios is increased by 5 times (200ms→40ms).

[0073] In this embodiment, during the operation of one or more home devices in a first scene mode, an environmental feature vector is generated based on the acquired environmental data. Then, the scene and probability can be predicted based on the environmental feature vector. When the scene mode is switched according to the predicted scene and the predicted probability, the device control sub-model is loaded and fine-tuned to realize device control. Since the device control sub-model in this embodiment is a model that shares the same convolutional backbone structure with the cloud meta-model, there are fewer parameters that need to be fine-tuned. Therefore, when the scene mode is switched and the corresponding model needs to be replaced, the computing resources are reduced and the latency is reduced.

[0074] Reference Figure 2a The diagram illustrates a flowchart of another smart home device control method according to an embodiment of this application, which may specifically include the following steps: Step S201: During the operation of one or more home devices in the first scene mode, acquire one or more environmental data. Step S202: Generate an environmental feature vector based on the one or more environmental data; Step S203: Input the environmental feature vector into a preset scene classifier, and output the predicted scene corresponding to the environmental feature vector and the predicted probability of the predicted scene. Step S204: When the first scene mode is switched to the second scene mode corresponding to the predicted scene according to the predicted scene and the predicted probability, the device control sub-model corresponding to the second scene mode is loaded in the local database. The device control sub-module shares the same convolutional backbone structure with the meta-model trained in the cloud. Step S205: Fine-tune the device control sub-model based on the environmental feature vector in the second scene mode; Step S206: In the second scene mode, obtain a real-time feature vector generated based on one or more environmental data; Step S207: Input the real-time feature vector into the device control sub-model and output one or more device control commands corresponding to the real-time feature vector; The device control sub-model can output corresponding device control commands based on the input real-time feature vector, so as to facilitate the control of home appliances.

[0075] Step S208: Control one or more home appliances according to the one or more device control instructions.

[0076] In one embodiment of this application, the device control of one or more home devices according to the one or more device control instructions includes: determining the target scene probability corresponding to the second scene mode based on the second scene mode and the scene classifier; obtaining current time information and device priority rules set based on scene and time; determining priority information of each home device according to the target scene probability, the current time information and the priority rules; and controlling the one or more home devices according to the priority information and the one or more control instructions.

[0077] In practical applications, different home appliances can be assigned different priorities, and these priorities can vary across different time periods and scenarios. Device priority rules can be set based on factors such as scenario and time. Then, a scenario classifier can be used to determine the probability of a target scenario. Finally, the target scenario probability, along with the current time, is matched against the device priority rules to obtain the correct priority for each home appliance. Device control can then be implemented based on these priorities.

[0078] In one embodiment of this application, the device control of one or more home devices according to the priority information and the one or more control instructions includes: determining a target home device whose priority information is greater than a preset priority threshold; determining a target control instruction corresponding to the target home device from the one or more control instructions; and controlling the target home device according to the target control instruction.

[0079] In practical applications, the device control and dynamic priority execution process is as follows: (1) Command generation: The fine-tuned model M_security directly outputs control commands (such as: [curtains closed, main light turned off, camera started, alarm on standby]) based on the real-time feature vector E_t.

[0080] The output layer of each scene sub-model uses a multi-dimensional control head activated by Sigmoid, directly mapping the normalized feature vector Enorm to the activation probability of each device action (such as camera activation, alarm push, etc.). When the output value of an action is greater than or equal to a threshold (such as 0.6), the corresponding instruction is triggered. For example, in an abnormal event at night, the input features are inferred by the security sub-model to output [0.86, 0.89, 0.29, 0.79, 0.83]. Based on this, the system executes "activate camera, send alarm, lock door, activate sound and light alarm", while "turn on light" is automatically ignored because its confidence level (0.29) is lower than the threshold, thus realizing a clear and quantifiable mapping from features to instructions.

[0081] (2) Dynamic Priority Management (Calculation): Fuzzy logic algorithms are used to resolve device instruction conflicts. The system defines a priority weight Q (between 0 and 1) for each device type. This weight is not fixed but is dynamically calculated based on the scenario and time. For example: Q_camera = f(Security_Probability, Time_of_Day) At 2 a.m. and when the security probability is high, Q_camera can be calculated to be 0.9.

[0082] The weight of the lighting equipment, Q_light, may be only 0.4 at this time.

[0083] The system ultimately executes the instruction set with the highest weighted priority, ensuring that critical tasks are executed first.

[0084] The specific technical solution of the fuzzy logic algorithm is as follows: This system deploys a lightweight fuzzy inference engine at the local decision-making level to dynamically coordinate resource conflicts or behavioral contradictions among multiple control commands (such as "turn on the lights" versus "covert monitoring"). Its detailed implementation includes the following three steps: 1. Input variable definition and fuzzification: Select two key input variables: Scene security confidence S ∈ [0,1]: that is, the probability of the current security mode output by the scene classifier (e.g., P(Ssecurity) = 0.85); Local time factor T ∈ [0,24]: converted to "nighttime" membership, through membership function Blur the image.

[0085] Each variable is divided into three fuzzy sets: S: {Low (L), Medium (M), High (H)}; T: {Non-Night (NIGHT)}.

[0086] 2. Fuzzy rule base (If-Then rules): Predefine several expert rules, for example: Rule 1: IF S is high AND T is nighttime THEN. Camera priority = 0.95, lighting priority = 0.3; Rule 2: If S is low and Tis is not nighttime, then lighting priority = 0.8, camera priority = 0.4; Rule 3: If S is high, the alarm push priority = 0.95 (regardless of time).

[0087] 3. Deblurring and Priority Output: The weighted average defuzzification method is used to aggregate the results of the activation rules into the final priority weight Q ∈ [0,1] for each device. For example, at 2 AM, when S=0.85, Rule 1 is highly activated, and the following calculation is performed: Qcamera = 0.95, Qlight = 0.3, Qalert = 0.95.

[0088] In practical applications, the system only executes Q. The instructions are set to 0.6 and ordered by Q value, thereby automatically suppressing low-priority actions in conflict scenarios (such as "turning on the lights may reveal the location").

[0089] This fuzzy logic module is rule-driven and has low computational overhead (<5ms), enabling it to run in real time in applications and ensuring that critical security commands are executed with priority.

[0090] Reference Figure 2b The diagram shows a flowchart of a device control method in an embodiment of this application. The process can adopt a three-layer collaborative architecture of "edge-local-cloud" to achieve the best balance between efficiency and intelligence.

[0091] The "edge-local-cloud" three-layer collaborative architecture can include an edge perception layer, a local decision-making layer, and a cloud training and maintenance layer.

[0092] (1) Edge Sensing Layer: Composed of STM32F7 microcontroller nodes distributed throughout the rooms. Each node is equipped with a sensor array, including a BH1750 light sensor (sampling period 100ms, accuracy 1 Lux), an HC-SR501 human infrared sensor (sampling period 500ms, detection distance 7 meters), and a MEMS microphone (for sound event detection, sampling rate 16kHz). This layer is responsible for raw data acquisition, preliminary filtering (such as using median filtering to eliminate false alarms from infrared sensors), and packaging.

[0093] (2) Local Decision Layer: The core is a Raspberry Pi 4B (4GB memory) as a home local gateway. It has built-in our developed dynamic metamodel scheduling software, which includes a lightweight scene classifier, a pre-trained sub-model library (stored on a local SSD), and a migration adaptation module. This layer receives data from all edge nodes, performs fusion analysis, scene recognition, model switching, and control command generation. All decision-making processes are completed locally, ensuring low latency.

[0094] (3) Cloud-based training and maintenance layer: The meta-model training platform is deployed using Alibaba Cloud ECS (g6e instance). It regularly collects anonymized environment data from customers and uses greater computing power to retrain and optimize the general meta-model (shared feature extraction layer). The trained new model version is distributed to each home gateway through a secure link to update its sub-model library. At the same time, blockchain services (such as Alibaba Cloud BaaS) are used to store the file hash value of each model version to ensure the integrity and immutability of the model during transmission and storage.

[0095] The equipment control process based on the above structure is as follows: (1) The light sensor in the edge layer collects real-time data streams, the human infrared sensor performs motion detection, and the sound sensor detects the ambient decibel level. The collected data is then uploaded to the microcontroller according to a preset cycle. Multi-sensor fusion is then performed to generate an environmental feature vector.

[0096] (2) In the local gateway, the environmental feature vector can be sent to the scene classifier to output the scene label and the corresponding probability. When the scene is inconsistent with the currently executed first scene mode and the probability is greater than the probability threshold, a pre-trained model can be loaded in the sub-model library. The models stored in the sub-model library are trained and generated by the cloud and then distributed. Then, the output layer weights are fine-tuned through the transfer adaptation layer, and then the device control engine generates device control commands. The device control values ​​can be used to control smart lights, door and window sensors, and smart speakers and other home devices.

[0097] (3) The cloud may include blockchain and meta-model training platform. The interaction between the two can generate device control sub-models and send them to the local gateway.

[0098] Example scenario: Dynamic switching from "sleep mode" to "security mode" at night.

[0099] Initial state: Time is 02:00:00 AM. The system is currently running the M_sleep model. Environmental data is stable: Illumination L=5 Lux, human infrared P=0, sound D=35 dB. All lights are off, door and window magnetic sensors are enabled, and the camera is in low-power sleep mode.

[0100] Event triggered: T+0.0s: The magnetic sensor of the balcony door and window detects abnormal vibration and sends an interrupt signal to the gateway.

[0101] T+0.2s: The system collects data in the new decision cycle. The light sensor reading remains 5 Lux; the human infrared sensor outputs P=1 due to motion detection; the microphone captures the sound of breaking glass, the sound event detection algorithm identifies the sudden noise, and the D peak rises to 90 dB. The feature vector becomes E_t = [5, 1, 90].

[0102] Processing and decision-making process: Spatiotemporal fusion: The Kalman filter module outputs a high-confidence P_est = 0.95 based on the historical data of the infrared sensor and the new value P=1.

[0103] Scene prediction: The weighted feature vector E_t is input into the scene classifier. The CNN model calculates the probability distribution: P(S_sleep)=0.05, P(S_home)=0.10, P(S_security)=0.85. If the probability of S_security > 0.75 and is different from the current mode, a switch is triggered.

[0104] (3) Model migration and updating: Load the M_security model from the library.

[0105] Copy and freeze the convolution weights W_shared of the metamodel.

[0106] Using E_t = [5, 0.95, 90] as samples, fine-tune the fully connected layer W_fc. The learning rate is 0.001, and training lasts for one epoch. A weighted loss function w_i = 1.5 accelerates the model's convergence to anomalous features.

[0107] (4) Command generation and execution: The fine-tuned M_security immediately processes the current feature and outputs the following commands: {Start the camera and record: priority 0.9, send an alarm push to the user's mobile phone: priority 0.95, turn on the living room lights: priority 0.6}.

[0108] (5) Dynamic priority decision: The fuzzy logic calculator determines that "send alarm information" has the highest priority (0.95) and executes it immediately; at the same time, "start camera" (0.9) is executed; while the "turn on lights" command is ignored in this cycle because it has a lower priority and is slightly contrary to the security intention.

[0109] Final result: The entire process, from sensor detection of an anomaly to receiving the alarm information on the mobile phone, took less than 400ms. This far exceeds traditional solutions (which require uploading data to the cloud for rule matching, taking more than 2000ms), providing users with crucial reaction time and effectively preventing property loss.

[0110] It should be noted that the device control method for smart home devices provided in this application embodiment can be executed by a device control device for smart home devices, or a control module within the device control device for executing the device control method for loading smart home devices. This application embodiment uses the execution of the device control method for loading smart home devices by a device control device for smart home devices as an example to illustrate the device control method for smart home devices provided in this application embodiment.

[0111] Reference Figure 3 This document illustrates a flowchart of a device control apparatus for a smart home device according to an embodiment of this application, which may specifically include the following modules: The environmental data acquisition module 301 is used to acquire one or more environmental data during the operation of one or more home devices in the first scene mode; The environmental feature vector generation module 302 is used to generate an environmental feature vector based on the one or more environmental data. The scene prediction module 303 is used to input the environmental feature vector into a preset scene classifier and output the predicted scene corresponding to the environmental feature vector and the prediction probability of the predicted scene. The scene mode switching module 304 is used to load the device control sub-model corresponding to the second scene mode in the local database when the first scene mode is switched to the second scene mode corresponding to the predicted scene according to the predicted scene and the predicted probability. The device control sub-module shares the same convolutional backbone structure with the meta-model trained in the cloud. The sub-model fine-tuning module 305 is used to fine-tune the device control sub-model based on the environmental feature vector in the second scene mode; The device control module 306 is used to control one or more home devices in the second scenario mode using a finely tuned device control sub-model.

[0112] In one embodiment of this application, the scene mode switching module 304 may include: The trigger determination submodule is used to determine and trigger the first scene mode to switch to the second scene mode corresponding to the predicted scene when the predicted scene does not match the first scene mode within N consecutive detection periods and the predicted probability is greater than the probability threshold, where N is a positive integer greater than or equal to 1.

[0113] In one embodiment of this application, the sub-model fine-tuning module 305 may include: The device control command determination submodule is used to determine the environmental feature vector and the device control command in the second scene mode; The fine-tuning submodule is used to fine-tune the fully connected layer of the device control sub-model using the environmental feature vector and the device control command as training data.

[0114] In one embodiment of this application, the device control module 306 may include: The real-time feature vector acquisition submodule is used to acquire real-time feature vectors generated based on one or more environmental data in the second scenario mode. The device control command generation submodule is used to input the real-time feature vector into the device control sub-model and output one or more device control commands corresponding to the real-time feature vector. The device control submodule is used to control one or more home appliances in accordance with the one or more device control instructions.

[0115] In one embodiment of this application, the device control submodule may include: A target scene probability determination unit is used to determine the target scene probability corresponding to the second scene mode based on the second scene mode and the scene classifier. The time and priority rule determination unit is used to obtain current time information and device priority rules based on scenario and time settings; The priority information determination unit is used to determine the priority information of each home device based on the target scene probability, the current time information, and the priority rules. A device control unit is used to control one or more home appliances according to the priority information and the one or more control commands.

[0116] In one embodiment of this application, the device control unit may include: A target home appliance determination subunit is used to determine target home appliances whose priority information is greater than a preset priority threshold. The target control instruction is determined from the one or more control instructions to identify the target control instruction corresponding to the target home appliance; The device control submodule is used to control the target home device according to the target control command.

[0117] In one embodiment of this application, the environmental data acquisition module 301 may include: The illumination data acquisition submodule is used to acquire illumination data in the target environment through a light sensor during the operation of one or more home devices in the first scene mode. The data acquisition submodule is used to acquire motion data of target people in the target environment through infrared sensors during the operation of one or more home devices in the first scene mode. The sound data acquisition submodule is used to acquire sound data in the target environment through sound sensors during the operation of one or more home devices in the first scene mode.

[0118] In one embodiment of this application, the environmental feature vector generation module 302 may include: The original feature vector generation submodule is used to construct an original feature vector based on the one or more environmental data. The weight data determination submodule is used to determine the weight data corresponding to the one or more environmental data based on the first scenario mode. The weighted feature vector determination submodule is used to determine the weighted feature vector based on the original feature vector and the weight data; The environmental feature vector determination submodule is used to normalize the weighted feature vector to obtain the environmental feature vector.

[0119] In one embodiment of this application, the original feature vector determination submodule may include: Alignment period determination unit, used to determine the alignment period corresponding to the one or more environmental data; An alignment data determination unit is used to align one or more environmental data according to the alignment period to obtain aligned data; An environmental feature vector construction unit is used to construct an original feature vector based on the one or more aligned data.

[0120] In this embodiment, during the operation of one or more home devices in a first scene mode, an environmental feature vector is generated based on the acquired environmental data. Then, the scene and probability can be predicted based on the environmental feature vector. When the scene mode is switched according to the predicted scene and the predicted probability, the device control sub-model is loaded and fine-tuned to realize device control. Since the device control sub-model in this embodiment is a model that shares the same convolutional backbone structure with the cloud meta-model, there are fewer parameters that need to be fine-tuned. Therefore, when the scene mode is switched and the corresponding model needs to be replaced, the computing resources are reduced and the latency is reduced.

[0121] The device control device for smart home devices in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0122] The device control device for smart home devices in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0123] The device control device for smart home devices provided in this application embodiment can achieve… Figures 1 to 3 The various processes implemented by the device control device of the smart home device in the method embodiment will not be described again here to avoid repetition.

[0124] Optionally, this application embodiment also provides an electronic device, including a processor 1010, a memory 1009, and a program or instructions stored in the memory 1009 and executable on the processor 1010. When the program or instructions are executed by the processor 1010, they implement the various processes of the above-described smart home device control method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0125] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0126] Figure 4 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application. The electronic device 1000 includes, but is not limited to, components such as: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010. The memory 1009 includes applications and an operating system; the user input unit 1007 may include a touch panel 10071 and other input devices 100072; the input unit 1004 may include an image processor 10041 and a microphone 10042; and the display unit 1006 may include a display panel 10061.

[0127] Those skilled in the art will understand that the electronic device 1000 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1010 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here. This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described smart home device control method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0128] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0129] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0130] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0131] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A device control method for a smart home device, characterized in that, The method includes: During the operation of one or more home devices in the first scene mode, one or more environmental data are acquired; Generate an environmental feature vector based on the one or more environmental data; The environmental feature vector is input into a preset scene classifier, and the predicted scene corresponding to the environmental feature vector and the predicted probability of the predicted scene are output. When the first scene mode is switched to the second scene mode corresponding to the predicted scene based on the predicted scene and the predicted probability, the device control sub-model corresponding to the second scene mode is loaded in the local database. The device control sub-module shares the same convolutional backbone structure with the meta-model trained in the cloud. The device control sub-model is fine-tuned based on the environmental feature vectors in the second scenario mode; In the second scenario mode, a finely tuned device control sub-model is used to control one or more home devices.

2. The method according to claim 1, characterized in that, The step of determining whether to switch the first scene mode to the second scene mode corresponding to the predicted scene based on the predicted scene and the predicted probability includes: If, within N consecutive detection periods, the predicted scene does not match the first scene mode and the predicted probability is greater than the probability threshold, it is determined that the first scene mode is switched to the second scene mode corresponding to the predicted scene, where N is a positive integer greater than or equal to 1.

3. The method according to claim 1, characterized in that, The fine-tuning of the device control sub-model based on the environmental feature vector under the second scene mode includes: Determine the environmental feature vector and the device control command in the second scene mode; The fully connected layer of the device control sub-model is fine-tuned using the environmental feature vector and the device control command as training data.

4. The method according to claim 1, 2, or 3, characterized in that, The method of using a finely tuned device control sub-model to control one or more home devices in the second scenario mode includes: In the second scenario mode, a real-time feature vector generated based on one or more environmental data is obtained; The real-time feature vector is input into the device control sub-model, and one or more device control commands corresponding to the real-time feature vector are output. Device control of one or more home appliances in accordance with the one or more device control instructions.

5. The method according to claim 4, characterized in that, The device control of one or more home appliances according to the one or more device control instructions includes: The probability of the target scene corresponding to the second scene mode is determined based on the second scene mode and the scene classifier. Obtain current time information and device priority rules based on scenario and time settings; The priority information of each home device is determined based on the target scenario probability, the current time information, and the priority rules. Device control of one or more home appliances is performed according to the priority information and the one or more control commands.

6. The method according to claim 5, characterized in that, The device control of one or more home appliances according to the priority information and the one or more control commands includes: Identify target home appliances whose priority information is greater than a preset priority threshold; Determine the target control command corresponding to the target home appliance from the one or more control commands; The target home appliance is controlled according to the target control command.

7. The method according to claim 1, 2, or 3, characterized in that, During the operation of one or more home devices in a first scene mode, one or more environmental data are acquired, including: During the operation of one or more home devices in the first scene mode, light data in the target environment is acquired through a light sensor; And / or, during the operation of one or more home devices in accordance with the first scene mode, the motion data of the target person in the target environment is acquired by infrared sensors; And / or, during the operation of one or more home devices in a first scene mode, sound data in the target environment is acquired through a sound sensor.

8. The method according to claim 1, 2, or 3, characterized in that, The generation of environmental feature vectors based on the one or more environmental data includes: Construct an original feature vector based on the one or more environmental data; Determine the weight data corresponding to the one or more environmental data based on the first scenario mode; The weighted feature vector is determined based on the original feature vector and the weight data; The weighted feature vector is normalized to obtain the environmental feature vector.

9. The method according to claim 8, characterized in that, The construction of the original feature vector based on the one or more environmental data includes: Determine the alignment period corresponding to the one or more environmental data; Align the one or more environmental data according to the alignment period to obtain aligned data; The original feature vector is constructed based on the one or more aligned data.

10. A device control device for a smart home device, characterized in that, The device includes: The environmental data acquisition module is used to acquire one or more environmental data during the operation of one or more home devices in the first scene mode; An environmental feature vector generation module is used to generate an environmental feature vector based on the one or more environmental data. The scene prediction module is used to input the environmental feature vector into a preset scene classifier and output the predicted scene corresponding to the environmental feature vector and the prediction probability of the predicted scene. The scene mode switching module is used to load the device control sub-model corresponding to the second scene mode in the local database when the first scene mode is switched to the second scene mode corresponding to the predicted scene based on the predicted scene and the predicted probability. The device control sub-module shares the same convolutional backbone structure with the meta-model trained in the cloud. The sub-model fine-tuning module is used to fine-tune the device control sub-model based on the environmental feature vectors in the second scenario mode; The device control module is used to control one or more home devices in the second scenario mode using a finely tuned device control sub-model.

11. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein when the program or instructions are executed by the processor, they implement the steps of the device control method for the smart home device as described in claims 1-9.

12. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the device control method for smart home devices as described in claims 1-9.

Images