Regulation and control method of intelligent equipment, intelligent gateway system, equipment, medium and product

Abstract

The embodiment of the invention provides a regulation and control method of intelligent equipment, an intelligent gateway system, equipment, a medium and a product. The method is applied to an intelligent gateway system, the intelligent gateway system is in communication connection with a plurality of intelligent devices based on a plurality of communication protocols through a network communication module, and the method comprises the following steps: receiving environment sensing data collected by a multi-modal sensor and device data sent by the plurality of intelligent devices, the device data comprising operation data and network states; determining a scene type based on the environment perception data, the operation data and the network state; determining a target intelligent device from the plurality of intelligent devices based on the scene type and the device data; and generating a parameter regulation and control strategy of the target intelligent equipment based on the target parameter corresponding to the scene type so as to regulate and control the target intelligent equipment. According to the method, unified access of cross-protocol equipment is realized, regulation and control of personalized intelligent equipment are realized, and the adaptation capability of the intelligent equipment to different scenes is improved.

Description

Technical Field

[0001] This application relates to the field of Internet of Things (IoT) technology, and in particular to a method for controlling smart devices, a smart gateway system, devices, media, and products. Background Technology

[0002] With the development of sensing, networking, and cloud computing technologies, the Internet of Things (IoT) has flourished and been widely applied. In homes, multiple smart home devices exist, such as smart air conditioners, smart door locks, Bluetooth speakers, and smart water meters. In farmland, irrigation equipment and sensing sensors such as soil moisture sensors and light sensors are deployed. Within communities, smart devices such as cameras and smart access control systems are deployed. In these scenarios, a smart gateway system can manage all the smart devices within a given management domain.

[0003] Smart devices involve multiple communication protocols, but current smart gateway systems only support one or two mainstream communication protocols (such as only Wi-Fi and Bluetooth), and do not integrate protocols suitable for low-power, wide-connectivity scenarios such as ZigBee and LoRa. This narrow protocol coverage forces users to purchase multiple gateways, creating "device silos." Furthermore, current technologies adjust device parameters based on a single sensing dimension, resulting in insufficient state awareness and inaccurate parameter control.

[0004] Therefore, we continue to develop an intelligent gateway system that supports multiple protocols and integrates multiple sensing dimensions to achieve accurate control of various intelligent devices under the management domain. Summary of the Invention

[0005] This application provides a method for controlling smart devices, a smart gateway system, devices, media, and products, which realizes unified access for cross-protocol devices, enables personalized control of smart devices, and improves the adaptability of smart devices to different scenarios.

[0006] In a first aspect, embodiments of this application provide a method for controlling smart devices, applied to a smart gateway system. The smart gateway system communicates with multiple smart devices via a network communication module based on various communication protocols. The method includes:

[0007] The system receives environmental perception data collected by the multimodal sensor and device data sent by the multiple smart devices, wherein the device data includes: operating data and network status;

[0008] Based on the environmental perception data, operational data, and network status, the scenario type is determined;

[0009] Based on the scenario type and the device data, a target smart device is determined from the plurality of smart devices;

[0010] Based on the target parameters corresponding to the scenario type, a parameter control strategy for the target intelligent device is generated to control the target intelligent device.

[0011] In one possible implementation, the target parameters include: target operating parameters and target network parameters. The steps of determining a scenario type based on the environmental perception data, operating data, and network status; determining a target intelligent device from the plurality of intelligent devices based on the scenario type and the device data; and generating a parameter control strategy for the target intelligent device based on the target parameters corresponding to the scenario type include:

[0012] The environmental perception data, operational data, and network status are input into a pre-trained scene recognition model to determine the scene type;

[0013] Based on the scenario type, determine the target operating parameters and target network parameters;

[0014] Extract the current operating parameters and current network parameters of the multiple smart devices corresponding to the scene type from the device data of the multiple smart devices;

[0015] The target operating parameters and target network parameters are compared with the current operating parameters and current network parameters of each smart device corresponding to the scene type. Based on the comparison results, the target smart device is determined from the multiple smart devices corresponding to the scene type.

[0016] Based on the current operating parameters and current network parameters of the target intelligent device, as well as the target operating parameters and target network parameters, a control strategy for the target intelligent device is generated.

[0017] In one possible implementation, the scene recognition model includes a classification model or a time series analysis model. The step of inputting the environmental perception data, operational data, and network status into a pre-trained scene recognition model to determine the scene type includes:

[0018] Feature extraction and fusion are performed on the environmental perception data, operational data, and network status to determine the first fused feature;

[0019] When the scene recognition model is a classification model, the probability that the first fusion feature belongs to multiple preset scene labels is determined, and the type corresponding to the preset scene label with the highest probability is determined as the scene type;

[0020] When the scene recognition model is a time series analysis model, the user's current behavior is determined based on the first fusion feature, and the scene type corresponding to the current behavior is determined according to the pre-stored behavior-scene type mapping relationship.

[0021] In one possible implementation, the above-mentioned determination of target operating parameters and target network parameters in conjunction with scenario type includes:

[0022] From the pre-stored semantic feature set, the semantic features of the scene type are determined, and the semantic features are fused with the first fusion feature to obtain the second fusion feature. The semantic feature set includes semantic features of multiple preset scene types. The semantic features of each preset scene type are obtained by inputting the scene description text of the preset scene type into a large language model.

[0023] The second fused feature is input into the device parameter prediction model to obtain the target operating parameters and target network parameters. The device parameter prediction model is trained based on historical environmental perception data, historical device data and corresponding device parameters.

[0024] In one possible implementation, the network parameters include at least one of: Wi-Fi frequency band, communication power, or bandwidth.

[0025] In one possible implementation, the network communication module supports multiple communication protocols, including at least two of Wi-Fi, ZigBee, Bluetooth, and LoRa protocols.

[0026] In one possible implementation, the method further includes:

[0027] The target operating parameters and target network parameters of the target intelligent device are fed back to the user terminal or cloud server for the user to view or adjust.

[0028] Receive user feedback to adjust operating parameters and network parameters;

[0029] Based on the adjusted operating parameters, adjusted network parameters, current operating parameters, and current network parameters, a control strategy for the target intelligent device is generated.

[0030] In one possible implementation, the method further includes:

[0031] The aforementioned control strategy and network parameter control strategy are executed to regulate the operating parameters and network parameters of the target intelligent device.

[0032] Secondly, embodiments of this application provide an intelligent gateway system, including: a network communication module and a main control chip;

[0033] The network communication module is used to establish communication connections with multiple smart devices based on various communication protocols, and to receive environmental perception data collected by the multimodal sensor and device data sent by the multiple smart devices. The device data includes: operating data and network status.

[0034] The main control chip includes a processing module and a generation module;

[0035] The processing module is used to determine the scene type based on the environmental perception data, operational data, and network status.

[0036] The processing module is further configured to determine the target smart device from the plurality of smart devices based on the scene type and the device data;

[0037] The generation module is used to generate a parameter control strategy for the target intelligent device based on the target parameters corresponding to the scene type, so as to control the target intelligent device.

[0038] In one possible implementation, the processing module is further configured to input the environmental perception data, operational data, and network status into a pre-trained scene recognition model to determine the scene type; combine the scene type to determine the target operational parameters and target network parameters; and extract the current operational parameters and current network parameters of the multiple smart devices corresponding to the scene type from the device data of the multiple smart devices.

[0039] The processing module is further configured to compare the target operating parameters and target network parameters with the current operating parameters and current network parameters of each smart device corresponding to the scene type, and based on the comparison results, determine the target smart device from among the multiple smart devices corresponding to the scene type;

[0040] The generation module is also used to generate a control strategy for the target intelligent device based on the current operating parameters and current network parameters of the target intelligent device, as well as the target operating parameters and target network parameters.

[0041] In one possible implementation, the scene recognition model includes: a classification model or a time series analysis model, and a processing module, further configured to extract and fuse features from the environmental perception data, operational data, and network status to determine a first fused feature; if the scene recognition model is a classification model, determining the probability that the first fused feature belongs to multiple preset scene labels, and determining the type corresponding to the preset scene label with the highest probability as the scene type; if the scene recognition model is a time series analysis model, determining the user's current behavior based on the first fused feature, and determining the scene type corresponding to the current behavior according to a pre-stored mapping relationship between behavior and scene type.

[0042] In one possible implementation, the processing module is further configured to determine the semantic features of the scene type from a pre-stored set of semantic features, and fuse the semantic features with the first fusion feature to obtain a second fusion feature. The set of semantic features includes semantic features of multiple preset scene types, and the semantic features of each preset scene type are obtained by inputting the scene description text of the preset scene type into a large language model.

[0043] The processing module is also used to input the second fused feature into the device parameter prediction model to obtain the target operating parameters and the target network parameters. The device parameter prediction model is trained based on historical environmental perception data, historical device data and corresponding device parameters.

[0044] In one possible implementation, the network parameters include at least one of the following: Wi-Fi frequency band, communication power, or bandwidth.

[0045] In one possible implementation, the network communication module supports multiple communication protocols, including at least two of Wi-Fi, ZigBee, Bluetooth, and LoRa protocols.

[0046] In one possible implementation, the main control chip also includes an interaction module;

[0047] The interaction module is used to feed back the target operating parameters and target network parameters of the target smart device to the user terminal or cloud server for the user to view or adjust; and to receive user feedback to adjust the operating parameters and network parameters.

[0048] The generation module is also used to generate a control strategy for the target intelligent device based on the adjusted operating parameters, adjusted network parameters, current operating parameters, and current network parameters.

[0049] In one possible implementation, the main control chip also includes an execution module;

[0050] The execution module is used to execute the control strategy and network parameter control strategy to control the operating parameters and network parameters of the target smart device.

[0051] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0052] The memory stores computer-executed instructions;

[0053] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0054] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0055] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0056] The intelligent device control method, intelligent gateway system, device, medium, and product provided in this application receive environmental perception data collected by multimodal sensors and device data sent by multiple intelligent devices. Based on the environmental perception data, operational data, and network status, the scenario type is determined. Based on the scenario type and device data, a target intelligent device is identified from among the multiple intelligent devices. Based on the target parameters corresponding to the scenario type, a parameter control strategy for the target intelligent device is generated for control. Multi-protocol compatibility is achieved through an intelligent gateway system supporting multiple communication protocols, solving the problem of device silos and enabling unified access for devices across protocols. Multimodal perception of the current physical environment by multimodal sensors solves the problem of inaccurate device parameter control based on a single perception dimension. Personalized intelligent device control is achieved by combining scenario types, improving the adaptability of intelligent devices to different scenarios, thereby realizing the control of the target intelligent device. Attached Figure Description

[0057] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0058] Figure 1 A schematic diagram illustrating a scenario for a smart device control method provided in an embodiment of this application;

[0059] Figure 2 A flowchart illustrating a method for controlling a smart device provided in this application embodiment. Figure 1 ;

[0060] Figure 3 A flowchart illustrating a method for controlling a smart device provided in this application embodiment. Figure 2 ;

[0061] Figure 4 This is a schematic diagram of the structure of an intelligent gateway system provided in an embodiment of this application;

[0062] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0063] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0064] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0065] "Multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0066] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, products, or apparatus.

[0067] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0068] With the development of sensing technology, network technology, and cloud computing technology, the Internet of Things (IoT) technology has flourished and been widely applied.

[0069] Figure 1This is a schematic diagram illustrating a scenario for a smart device control method provided in an embodiment of this application. The smart device control method can be applied to, for example... Figure 1 In the smart home scenario shown, multiple smart devices exist within the home. These smart home devices may include, for example,... Figure 1 The smart air conditioner 11, smart door lock 12, television 13, smart curtains 14, etc., are shown. These smart devices communicate with the network communication module of the smart gateway system 15 based on various communication protocols. Furthermore, this method for controlling smart devices can also be applied to smart farmland equipped with irrigation equipment and sensing sensors such as soil moisture sensors and light sensors, and to smart communities equipped with cameras, smart access control systems, and other smart devices. In these scenarios, the smart gateway system can manage various smart devices within a management domain, such as adjusting device operating parameters and controlling device activation and deactivation.

[0070] Smart devices involve multiple communication protocols. (Continue to refer to...) Figure 1 In existing technologies, smart curtains typically communicate with smart gateway systems via Zigbee, smart door locks typically communicate with smart gateway systems via Zigbee or Bluetooth, smart air conditioners and televisions typically communicate with smart gateway systems via Wi-Fi, and smart water meters typically communicate with smart gateway systems via the LoRa protocol.

[0071] Based on the scenarios described above, it is evident that current smart gateways only support one or two mainstream communication protocols (e.g., only Wi-Fi and Bluetooth), and do not integrate protocols such as ZigBee and LoRa, which are suitable for low-power, wide-connectivity scenarios. This narrow protocol coverage necessitates users purchasing multiple gateways, creating "device silos." Furthermore, current technologies rely on a single sensing dimension to adjust the device parameters of smart devices, resulting in insufficient state awareness and inaccurate parameter control.

[0072] This application provides a method for controlling intelligent devices. By integrating multiple communication protocols, collecting environmental perception data, device operation data, and network status, and combining them with specific scenario types, the method enables parameter control of the target intelligent device. This solves the problems of narrow protocol coverage, insufficient status awareness, and inaccurate device parameter control in existing technologies.

[0073] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0074] Figure 2A flowchart illustrating a method for controlling a smart device provided in this application embodiment. Figure 1 This method is applied to a smart gateway system, which communicates with multiple smart devices through a network communication module based on various communication protocols. The executing entity of this method can be, for example, a... Figure 1 The intelligent gateway system 15 shown is as follows: Figure 2 As shown, the method includes:

[0075] S201. Receive environmental perception data collected by multimodal sensors and device data sent by multiple smart devices. The device data includes: operation data and network status.

[0076] The intelligent gateway system includes a main control chip, as well as multimodal sensors and network communication modules connected to the main control chip.

[0077] Multimodal sensors include various sensing sensors, such as temperature sensors, humidity sensors, light sensors, and motion sensors. These sensing sensors can be integrated into the smart device's own sensors or be independently configured. They can also include both built-in sensors and independently configured sensors. For example, a smart device's built-in sensing sensor could be a temperature and humidity sensor from a smart air conditioner, while independently configured sensors could be human body sensors, light sensors, etc.

[0078] Environmental sensing data refers to quantitative data obtained by measuring the state of various environmental elements in the environment through multimodal sensors, such as temperature, humidity, light intensity, and wind speed.

[0079] Device data refers to quantitative information that reflects the status, performance, and operation of a smart device. Examples include operational data and network status.

[0080] Operational data refers to the working data of the smart device, such as whether the device is on, in standby mode, or off, as well as runtime, compressor current frequency, fan real-time speed, and power consumption. Network status indicates the communication quality between the smart device and the network communication module. This can be measured, for example, by response speed, signal strength, and transmission speed.

[0081] The intelligent gateway system communicates with multiple intelligent devices through a network communication module based on various communication protocols. Multimodal sensors collect environmental perception data, and the multimodal sensors and multiple intelligent devices send environmental perception data, operation data, and network status according to a specified period or in real time.

[0082] By enabling multi-protocol compatibility through an intelligent gateway system that supports multiple communication protocols, the problem of device silos is solved. By using multimodal sensors to perceive the current physical environment in a multimodal manner, the problem of inaccurate device parameter control based on a single perception dimension is solved, providing a basis for subsequent control and realizing the linkage cognition of "environment-device-network" to determine multi-dimensional data and achieve intelligent control of intelligent devices.

[0083] S202. Determine the scenario type based on environmental perception data, operational data, and network status;

[0084] In smart home scenarios, scenarios could include home theater scenarios, high-load office scenarios, nighttime energy-saving scenarios, and smart temperature control scenarios. In smart farmland scenarios, scenarios could include smart irrigation scenarios, smart greenhouse scenarios, and precision fertilization scenarios. In smart communities scenarios, scenarios could include smart waste management and seamless access control.

[0085] The main control chip of the intelligent gateway system can determine the current scenario type from multiple preset scenarios based on environmental perception data, operational data, and network status, and through rule matching using predefined rules. It can also determine training and validation sets using historical environmental perception data, operational data, network status, and labeled preset scenario tags. A classification model is trained using the training set and validated using the validation set until training is complete, resulting in a scenario recognition model. This model then predicts the scenario type. Furthermore, it can predict the user's subsequent behavior by determining the user's current behavior to identify the current scenario type.

[0086] For example, when the main living room light is detected to be off, the TV is on, and a user is present in front of the TV, the scene type might be a home theater scene. In this scenario, the smart curtains can be closed to provide the user with a better viewing experience.

[0087] For example, if high load is detected in the CPU usage, memory usage, network throughput, or number of connections of a router or computer, the scenario type might be a high-load office scenario. In this scenario, the hardware configuration of the room where the office equipment is located can be adjusted to modify network parameters and provide users with a better network experience.

[0088] Understandably, even if the environmental perception data collected by multimodal sensors is the same, the control of smart devices will differ under different scenario types. For example, for the same ambient temperature, the opening and closing status of smart curtains will differ in a home scenario and an away scenario. For instance, if the collected environmental perception data indicates sunny weather, in a home scenario, the light-transmitting curtains will be closed to prevent direct sunlight from shining on the user, while in an away scenario, the light-transmitting curtains can be opened to maintain air circulation.

[0089] By combining different scenarios, personalized control of smart devices can be achieved, improving the adaptability of smart devices to different scenarios.

[0090] S203. Based on the scenario type and device data, determine the target smart device from multiple smart devices;

[0091] Among them, target intelligent devices refer to devices whose parameters need to be adjusted to adapt to the device operation requirements of the current scenario type.

[0092] The target smart device can be determined from multiple smart devices by comparing the target parameters of the scene type with the current device parameters of the device.

[0093] S204. Based on the target parameters corresponding to the scenario type, generate a parameter control strategy for the target intelligent device to control the target intelligent device.

[0094] The current device parameters are adjusted to target parameters to generate a parameter adjustment strategy for the target intelligent device, thereby enabling the control of the target intelligent device.

[0095] In one possible implementation, the network communication module supports multiple communication protocols, including at least two of the following: Wi-Fi, ZigBee, Bluetooth, and LoRa protocols.

[0096] The network communication module supports multiple communication protocols, achieving multi-protocol compatibility and solving the problem of device silos. It enables the coexistence of multiple protocol devices through a single smart gateway system, improving the adaptability of the smart gateway system in various application scenarios such as smart homes, smart agriculture, and smart communities.

[0097] This application provides a method for controlling intelligent devices. It receives environmental perception data collected by multimodal sensors and device data sent by multiple intelligent devices. Based on the environmental perception data, operational data, and network status, it determines the scene type. Based on the scene type and device data, it identifies a target intelligent device from among the multiple intelligent devices. Based on the target parameters corresponding to the scene type, it generates a parameter control strategy for the target intelligent device to control it. By using an intelligent gateway system supporting multiple communication protocols, it achieves multi-protocol compatibility, solving the problem of device silos and enabling unified access for devices across protocols. Through multimodal sensing of the current physical environment using multimodal sensors, it solves the problem of inaccurate device parameter control based on a single sensing dimension. Combined with scene type, it achieves personalized control of intelligent devices, improving the adaptability of intelligent devices to different scenes, thereby realizing the control of the target intelligent device.

[0098] Figure 3A flowchart illustrating a method for controlling a smart device provided in this application embodiment. Figure 2 In this embodiment, the target parameters include: target operating parameters and target network parameters. Figure 2 Based on the embodiments, a method for controlling a smart device is described in detail, such as... Figure 3 As shown, the method includes:

[0099] S301: Receives environmental perception data collected by multimodal sensors and device data sent by multiple smart devices. The device data includes: operating data and network status.

[0100] Step S301 is similar to step S201, and will not be described again here.

[0101] S302. Extract and fuse features from environmental perception data, operational data, and network status to determine the first fused feature;

[0102] Features can be extracted from environmental perception data, operational data, and network status using methods such as Principal Component Analysis (PCA) and one-hot coding. After feature extraction, feature fusion can be achieved through feature concatenation, weighted fusion, attention mechanisms, and Transformer models.

[0103] Preferably, before feature extraction and fusion, the data can be preprocessed by data cleaning, deduplication, etc., and feature extraction and fusion can be performed based on the preprocessed data.

[0104] S303. Determine the probability that the first fusion feature belongs to multiple preset scene labels, and determine the type corresponding to the preset scene label with the highest probability as the scene type;

[0105] When the scene recognition model is a classification model, the first fused feature is input into the pre-trained scene recognition model, and the model determines the scene type by the type corresponding to the preset scene label with the highest probability. The scene recognition model is determined by using historical environmental perception data, operational data, network status, and labeled preset scene labels to establish training and validation sets. The classification model is trained using the training set and validated using the validation set.

[0106] S304. Based on the first fusion feature, determine the user's current behavior, and determine the scene type corresponding to the current behavior according to the pre-stored behavior-scene type mapping relationship;

[0107] Since user activity is a sequence that changes over time, time series analysis models can be used to determine the user's current behavior, predict subsequent behavior, and determine the current scenario type based on pre-stored mappings between behavior and scenario type. These behavioral sequence patterns can be learned using models such as Long Short-Term Memory (LSTM) networks.

[0108] For example, if a human body sensor detects user activity and the curtains open in the morning, it can be determined that the current behavior corresponds to the scenario of waking up.

[0109] S305. From the pre-stored semantic feature set, determine the semantic features of the scene type, and fuse the semantic features with the first fusion feature to obtain the second fusion feature;

[0110] The semantic feature set includes semantic features for multiple preset scene types. The semantic features for each preset scene type are obtained by inputting the scene description text of the preset scene type into the large language model.

[0111] From a set of semantic features that includes semantic features of multiple preset scene types, the semantic features of the current scene type are obtained. The semantic features and the first fusion feature are then fused together using attention mechanisms, feature concatenation, and other methods to obtain the second fusion feature.

[0112] S306. Input the second fusion feature into the device parameter prediction model to obtain the target operating parameters and target network parameters;

[0113] The equipment parameter prediction model is trained based on historical environmental perception data, historical equipment data, and corresponding equipment parameters.

[0114] The second fused feature is input into a device parameter prediction model trained based on historical environmental perception data, historical device data, and corresponding device parameters to obtain the target operating parameters and target network parameters. The model can be a deep learning model, a random forest model, etc.

[0115] S307. Extract the current operating parameters and current network parameters of multiple smart devices corresponding to the scene type from the device data of multiple smart devices;

[0116] S308. Compare the target operating parameters and target network parameters with the current operating parameters and current network parameters of each smart device corresponding to the scene type. Based on the comparison results, determine the target smart device from among the multiple smart devices corresponding to the scene type.

[0117] Retrieve a list of preset smart devices for the current scene type. For any preset smart device in the list, compare its target operating parameters and target network parameters with its current operating parameters and current network parameters to obtain the comparison results. If the comparison results show that the target operating parameters and / or target network parameters are inconsistent with the current operating parameters and / or current network parameters, then the preset smart device is selected as the target smart device.

[0118] Understandably, if the target cooling temperature of the air conditioner is 26℃, and the current cooling temperature of the air conditioner is 28℃, then the air conditioner will be considered as the target smart device.

[0119] S309. Based on the current operating parameters and current network parameters of the target intelligent device, as well as the target operating parameters and target network parameters, generate a control strategy for the target intelligent device.

[0120] The current operating parameters and network parameters of the target intelligent device are adjusted to the target operating parameters and target network parameters to generate a parameter adjustment strategy for the target intelligent device, so as to control the target intelligent device.

[0121] In one possible implementation, the network parameters include at least one of the following: Wi-Fi frequency band, communication power, or bandwidth.

[0122] When the network signal quality of the smart device being used by the user is poor, the system dynamically switches the Wi-Fi frequency band, communication power, or bandwidth to achieve real-time allocation of network resources to adapt to the current scenario.

[0123] For example, Wi-Fi bands include 2.4GHz and 5GHz. The 2.4GHz band has strong wall penetration but is slower, while the 5GHz band has weaker wall penetration but is faster. Wi-Fi, Bluetooth, and Zigbee all operate on this band, and using a fixed Wi-Fi band can lead to signal congestion. Therefore, when adjusting the network parameters of smart devices, you can adjust the network parameters of the TV to the 5GHz band to obtain a clear video stream, and adjust the Wi-Fi band of the smart bulb to the 2.4GHz band.

[0124] In one possible implementation, the method further includes:

[0125] Feedback the target operating parameters and target network parameters of the target intelligent device to the user terminal or cloud server for the user to view or adjust; receive user feedback on adjusting operating parameters and network parameters; and generate a control strategy for the target intelligent device based on the adjusted operating parameters, adjusted network parameters, current operating parameters, and current network parameters.

[0126] After generating the control strategy for the target smart device, the target operating parameters and target network parameters of the target smart device in the control strategy are fed back to the user terminal or cloud server for the user to view or adjust. After the user modifies and saves the target operating parameters and target network parameters, the system receives the user's feedback on the adjusted operating parameters and adjusted network parameters. After receiving the adjusted operating parameters and adjusted network parameters, the system adjusts the current operating parameters and current network parameters of the target smart device according to the adjusted operating parameters and adjusted network parameters, generating a parameter control strategy for the target smart device, thereby controlling the target smart device.

[0127] Optionally, the user terminal can receive abnormal alerts.

[0128] Optionally, the method may also include periodically optimizing the equipment parameter prediction model.

[0129] By adjusting operating parameters and network parameters based on user feedback, corresponding historical environmental perception data and historical equipment data are labeled, and the model parameters of the equipment parameter prediction model are optimized using the labeled historical environmental perception data and historical equipment data.

[0130] By continuously optimizing the model parameters of the equipment parameter prediction model, the target operating parameters and target network parameters predicted by the model can meet the user's preferences.

[0131] Through cloud servers and user terminals, local-cloud collaborative management is supported. The cloud is responsible for data storage and in-depth analysis, while the user terminal serves as the entry point for remote user interaction, forming a closed loop of "perception-analysis-operation".

[0132] In one possible implementation, the main control chip of the smart gateway system may also include a storage module for data storage. The stored data may also include device parameters corresponding to user-defined preference modes. Therefore, adjusting operating parameters and network parameters may also be done using device parameters corresponding to preference modes.

[0133] In one possible implementation, the method further includes: executing a control strategy and a network parameter control strategy to control the operating parameters and network parameters of the target smart device.

[0134] The intelligent gateway system distributes control policies and network parameter control policies to the target intelligent devices to control their operating parameters and network parameters.

[0135] This application provides a method for controlling a smart device. It receives environmental perception data collected by a multimodal sensor, as well as operational data and network status sent by multiple smart devices. Features are extracted and fused from the environmental perception data, operational data, and network status to determine a first fused feature. The probability of the first fused feature belonging to multiple preset scene labels is then determined. The type corresponding to the preset scene label with the highest probability is identified as the scene type. Based on the first fused feature, the user's current behavior is determined. According to a pre-stored mapping relationship between behavior and scene type, the scene type corresponding to the current behavior is determined. Semantic features of the scene type are determined from a pre-stored set of semantic features. The semantic features and the first fused feature are fused to obtain a second fused feature. The second fused feature is then input into a device parameter prediction model. In this process, target operating parameters and target network parameters are obtained. Current operating parameters and current network parameters of multiple smart devices corresponding to a specific scenario type are extracted from device data of multiple smart devices. The target operating parameters and target network parameters are compared with the current operating parameters and current network parameters of each smart device corresponding to the scenario type. Based on the comparison results, the target smart device is determined from the multiple smart devices corresponding to the scenario type. A control strategy for the target smart device is generated based on its current operating parameters and current network parameters, as well as the target operating parameters and target network parameters. Combined with the scenario type, personalized control of the smart device is achieved, improving the adaptability of smart devices to different scenarios. This ensures that the generated control strategy meets the actual needs of the current scenario type, thus improving the user experience.

[0136] Figure 4 This is a schematic diagram of the structure of an intelligent gateway system provided in an embodiment of this application, as shown below. Figure 4 As shown, the intelligent gateway system 40 provided in this embodiment includes: a network communication module 401 and a main control chip 402;

[0137] The network communication module 401 is used to communicate with multiple smart devices based on multiple communication protocols, and to receive environmental perception data collected by multimodal sensors and device data sent by multiple smart devices. The device data includes: operation data and network status.

[0138] The main control chip 402 includes a processing module 4021 and a generation module 4022;

[0139] Processing module 4021 is used to determine the scene type based on environmental perception data, operational data, and network status;

[0140] The processing module 4021 is also used to determine the target smart device from multiple smart devices based on the scene type and device data;

[0141] The generation module 4022 is used to generate a parameter control strategy for the target smart device based on the target parameters corresponding to the scene type, so as to control the target smart device.

[0142] In one possible implementation, the processing module 4021 is further configured to input environmental perception data, operational data, and network status into a pre-trained scene recognition model to determine the scene type; combine the scene type to determine the target operational parameters and target network parameters; and extract the current operational parameters and current network parameters of multiple smart devices corresponding to the scene type from the device data of multiple smart devices.

[0143] The processing module 4021 is also used to compare the target operating parameters and target network parameters with the current operating parameters and current network parameters of each smart device corresponding to the scene type, and based on the comparison results, determine the target smart device from multiple smart devices corresponding to the scene type;

[0144] The generation module 4022 is also used to generate a control strategy for the target intelligent device based on the current operating parameters and current network parameters of the target intelligent device, as well as the target operating parameters and target network parameters.

[0145] In one possible implementation, the scene recognition model includes: a classification model or a time series analysis model. The processing module 4021 is further used to extract and fuse features from environmental perception data, operational data, and network status to determine a first fused feature. If the scene recognition model is a classification model, it determines the probability that the first fused feature belongs to multiple preset scene labels, and determines the type corresponding to the preset scene label with the highest probability as the scene type. If the scene recognition model is a time series analysis model, it determines the user's current behavior based on the first fused feature, and determines the scene type corresponding to the current behavior according to the pre-stored mapping relationship between behavior and scene type.

[0146] In one possible implementation, the processing module 4021 is further configured to determine the semantic features of the scene type from the pre-stored semantic feature set, and fuse the semantic features with the first fusion feature to obtain the second fusion feature. The semantic feature set includes semantic features of multiple preset scene types, and the semantic features of each preset scene type are obtained by inputting the scene description text of the preset scene type into the large language model.

[0147] The processing module 4021 is also used to input the second fused feature into the device parameter prediction model to obtain the target operating parameters and the target network parameters. The device parameter prediction model is trained based on historical environmental perception data, historical device data and corresponding device parameters.

[0148] In one possible implementation, the network parameters include at least one of the following: Wi-Fi frequency band, communication power, or bandwidth.

[0149] In one possible implementation, the network communication module 401 supports multiple communication protocols including at least two of Wi-Fi, ZigBee, Bluetooth, and LoRa protocols.

[0150] In one possible implementation, the main control chip 402 also includes an interaction module 4023;

[0151] The interaction module 4023 is used to feed back the target operating parameters and target network parameters of the target smart device to the user terminal or cloud server for the user to view or adjust; and to receive user feedback to adjust the operating parameters and network parameters.

[0152] The generation module 4022 is also used to generate a control strategy for the target intelligent device based on the adjustment of operating parameters, the adjustment of network parameters, the current operating parameters, and the current network parameters.

[0153] In one possible implementation, the main control chip 402 further includes an execution module 4024;

[0154] The execution module 4024 is used to execute the control strategy and network parameter control strategy to control the operating parameters and network parameters of the target smart device.

[0155] The intelligent gateway system provided in this embodiment can execute the methods provided in the above method embodiments. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0156] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus.

[0157] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0158] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0159] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0160] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0161] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0162] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0163] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0164] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0165] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0166] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0167] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0168] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0169] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0170] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0171] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for controlling an intelligent device, characterized in that, The method is applied to an intelligent gateway system, the intelligent gateway system is connected with a plurality of intelligent devices based on a plurality of communication protocols through a network communication module, and the method comprises the following steps: Receiving environment perception data collected by a multi-modal sensor and device data sent by the plurality of intelligent devices, wherein the device data comprises running data and network status; Determining a scene type based on the environment perception data, the running data and the network status; Determining a target intelligent device from the plurality of intelligent devices based on the scene type and the device data; Generating a parameter control strategy of the target intelligent device based on a target parameter corresponding to the scene type, so as to control the target intelligent device.

2. The method of claim 1, wherein, The target parameter comprises a target running parameter and a target network parameter, and the method of determining the scene type based on the environment perception data, the running data and the network status, determining the target intelligent device from the plurality of intelligent devices based on the scene type and the device data, and generating the parameter control strategy of the target intelligent device based on the target parameter corresponding to the scene type comprises the following steps: Inputting the environment perception data, the running data and the network status into a pre-trained scene recognition model to determine the scene type; Determining the target running parameter and the target network parameter in combination with the scene type; Extracting the current running parameter and the current network parameter of the plurality of intelligent devices corresponding to the scene type from the device data of the plurality of intelligent devices; Comparing the target running parameter and the target network parameter with the current running parameter and the current network parameter of each intelligent device corresponding to the scene type, and determining the target intelligent device from the plurality of intelligent devices corresponding to the scene type based on a comparison result; Generating the control strategy of the target intelligent device based on the current running parameter and the current network parameter of the target intelligent device and the target running parameter and the target network parameter.

3. The method of claim 2, wherein the method further comprises: The scene recognition model comprises a classification model or a time series analysis model, and the method of inputting the environment perception data, the running data and the network status into the pre-trained scene recognition model to determine the scene type comprises the following steps: Extracting and fusing features of the environment perception data, the running data and the network status to determine first fused features; In the case that the scene recognition model is a classification model, determining a probability that the first fused features belong to a plurality of preset scene labels, and determining a type corresponding to a preset scene label with the largest probability as the scene type; In the case that the scene recognition model is a time series analysis model, determining a current behavior of a user based on the first fused features, and determining a scene type corresponding to the current behavior according to a pre-stored mapping relationship between behaviors and scene types.

4. The method of claim 3, wherein the method further comprises: The method of determining the target running parameter and the target network parameter in combination with the scene type comprises the following steps: From a pre-stored semantic feature set, determine the semantic feature of the scene type, fuse the semantic feature and the first fusion feature to obtain a second fusion feature, the semantic feature set includes semantic features of multiple preset scene types, and each semantic feature of the preset scene type is obtained by inputting a scene description text of the preset scene type into a large language model; Input the second fusion feature into a device parameter prediction model to obtain target running parameters and target network parameters, and the device parameter prediction model is trained based on historical environment perception data, historical device data and corresponding device parameters.

5. The method of claim 2, wherein the method further comprises: The network parameters include at least one of Wi-Fi frequency band, communication power or bandwidth. 6.The method of claim 1, wherein, The network communication module supports multiple communication protocols, including at least two of Wi-Fi, ZigBee, Bluetooth and LoRa protocols. 7.The method of claim 2, wherein, The method further comprises: Feedback the target running parameters and target network parameters of the target smart device to a user terminal or a cloud server for the user to view or adjust; Receive the user feedback adjustment running parameters and adjustment network parameters; Based on the adjustment running parameters, adjustment network parameters, current running parameters and current network parameters, generate a control strategy of the target smart device. 8.The method of claim 1 or 6, wherein, The method further comprises: Execute the control strategy and network parameter control strategy to control the running parameters and network parameters of the target smart device.

9. An intelligent gateway system, characterized by It includes: A network communication module and a master control chip; The network communication module is used for communication connection with multiple smart devices based on multiple communication protocols, receives environment perception data collected by the multi-modal sensor and device data sent by the multiple smart devices, and the device data includes running data and network status; The master control chip includes a processing module and a generation module; The processing module is used for determining a scene type based on the environment perception data, running data and network status; The processing module is also used for determining a target smart device from the multiple smart devices based on the scene type and the device data; The generation module is used for generating a parameter control strategy of the target smart device based on the target parameters corresponding to the scene type, to control the target smart device.

10. An electronic device, comprising: It includes: A memory and a processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method of any one of claims 1-8.

11. A computer readable storage medium characterized by, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method of any one of claims 1-8.

12. A computer program product, characterised in that, It includes a computer program, which is executed by the processor to implement the method of any one of claims 1-8.

Images