Smart home equipment operation monitoring method and device, equipment and storage medium

Abstract

The invention relates to the technical field of smart home, and discloses a smart home equipment operation monitoring method and device, equipment and a storage medium, and the method comprises the steps: responding to a monitoring instruction, and collecting the operation data of all smart home equipment in a preset region in real time; wherein the operation data comprises basic operation information of the equipment and connection information between the equipment; processing the operation data, and inputting the processed operation data into a home equipment monitoring model to obtain an abnormality judgment result; wherein the home equipment monitoring model is obtained by training a hybrid model composed of a graph neural network and a Transform based on historical operation data of the smart home equipment. According to the method and the device, the graph neural network and the Transform are combined, so that the accuracy and scene adaptability of abnormal prediction of the smart home equipment can be improved.

Description

Technical Field

[0001] The present application relates to the field of smart home technology, and in particular to a method, apparatus, device, and storage medium for monitoring the operation of smart home devices. Background Art

[0002] With the rapid development of the Internet of Things (IoT), smart home devices have become deeply integrated into people's daily lives. These devices continuously generate massive amounts of log data during operation, covering critical information such as device status, operation time, and execution duration. To ensure stable device operation and effectively identify potential faults, traditional methods rely on log analysis based on pre-set fixed rules on cloud platforms, such as "exceeding a threshold triggers an alarm." While these methods can, to a certain extent, assist users in identifying pre-defined abnormal conditions and implementing early warnings, as the functional complexity and product diversity of smart home devices continue to increase, these rule-based analysis methods are increasingly showing significant limitations. They can only identify pre-configured abnormal patterns and lack effective response mechanisms for unknown or complex abnormal scenarios. As a result, in real-world applications, potential issues may not be identified and addressed promptly, impacting system reliability and security. Summary of the Invention

[0003] In view of this, in order to solve at least one of the above technical problems, the embodiments of the present application provide a method, apparatus, device and storage medium for monitoring the operation of smart home devices.

[0004] In a first aspect, the present invention provides a method for monitoring the operation of a smart home device, comprising:

[0005] In response to monitoring instructions, real-time collection of operating data of all smart home devices in a preset area; wherein the operating data includes basic operating information of the devices and connection information between the devices;

[0006] The operation data is processed and input into a home appliance monitoring model to obtain an abnormality judgment result; wherein, the home appliance monitoring model is obtained by training a hybrid model composed of a graph neural network and a Transformer based on the historical operation data of the smart home device.

[0007] In an optional embodiment, the basic operation information of the device includes the device controlled log, the device operation status, the time when the device status change occurs, the device execution time, and the device online and offline information;

[0008] The connection information between the devices includes the network connection relationship between the devices, the message transmission relationship between the devices and the time series association relationship between the devices.

[0009] In an optional embodiment, the processing the operating data includes:

[0010] extracting time series data from basic operation information of the equipment;

[0011] A topological relationship between each smart home device is constructed based on the basic operation information of the device and the connection information between the devices.

[0012] In an optional embodiment, the abnormality determination result includes the abnormality type, abnormal device, abnormality occurrence time and abnormality severity;

[0013] After obtaining the abnormality judgment result, the method further includes:

[0014] According to the abnormality judgment result, corresponding processing suggestions are generated in combination with the abnormality processing knowledge base, and corresponding alarm instructions are output.

[0015] In an optional implementation, obtaining the household device monitoring model includes:

[0016] Preprocess the historical operation data of smart home devices to obtain historical time series data and historical topological relationships between various smart home devices;

[0017] Based on the historical time series data and the historical topological relationship, the hybrid model is trained in combination with a preset loss function to obtain the home appliance monitoring model.

[0018] In an optional embodiment, the training of the hybrid model based on the historical time series data and the historical topological relationship in combination with a preset loss function to obtain a home appliance monitoring model includes:

[0019] Input the historical time series data into the Transformer module to obtain time series features, and input the historical topological relationship into the graph neural network module to obtain device relationship features;

[0020] The time series features and the device relationship features are fused and input into a fully connected layer to obtain a model prediction result;

[0021] The loss between the model prediction result and the actual result is calculated by the loss function, and the parameters of the hybrid model are updated according to the loss until the updated hybrid model meets the preset cutoff condition, thereby obtaining the home appliance monitoring model.

[0022] In an optional embodiment, the loss function is:

[0023] Among them, y i is the true result of the sample, y′ iis the prediction result of the model; N is the number of training samples; L BEC is the loss between the predicted result and the true result; w(t i ) is the time decay weight of the i-th sample; t i is the generation time of the i-th sample.

[0024] In a second aspect, the present invention provides a smart home device operation monitoring device, comprising:

[0025] A collection module is used to collect the operating data of all smart home devices in a preset area in real time in response to monitoring instructions; wherein the operating data includes basic operating information of the devices and connection information between the devices;

[0026] An acquisition module is used to process the operating data and input it into a home appliance monitoring model to obtain an abnormality judgment result; wherein, the home appliance monitoring model is obtained by training a hybrid model composed of a graph neural network and a Transformer based on the historical operating data of the smart home device.

[0027] In a third aspect, the present invention provides a terminal device, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the smart home device operation monitoring method described in the aforementioned embodiment.

[0028] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed on a processor, the method for monitoring the operation of smart home devices described in the aforementioned embodiment is implemented.

[0029] The embodiments of this application have the following beneficial effects: In monitoring the operation of smart home devices, this application not only focuses on the operating data of the devices themselves, but also on the associations or dependencies between devices. Furthermore, a home device monitoring model is introduced, consisting of a graph neural network capable of capturing static dependencies between devices and a Transformer capable of capturing the dynamic patterns of device changes over time. This allows this embodiment to better adapt to diverse scenarios and unknown issues, and more accurately predict device anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 A first flow chart of the method for monitoring the operation of smart home devices according to an embodiment of the present application is shown;

[0032] Figure 2 A second flow chart of the method for monitoring the operation of smart home devices according to an embodiment of the present application is shown;

[0033] Figure 3 A third flow chart of the method for monitoring the operation of smart home devices according to an embodiment of the present application is shown;

[0034] Figure 4 A structural diagram of a smart home device operation monitoring device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0036] The components of the embodiments of the present application generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but rather merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0037] Hereinafter, the terms "including", "having" and their cognates used in various embodiments of the present application are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the aforementioned items, and should not be understood as excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the aforementioned items or adding the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the aforementioned items. In addition, the terms "first", "second", "third" and the like are only used to distinguish descriptions and should not be understood as indicating or implying relative importance.

[0038] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meaning as in the context of the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in the various embodiments of the present application.

[0039] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.

[0040] The following describes the method for monitoring the operation of smart home devices in conjunction with some specific embodiments.

[0041] Figure 1 A flow chart of a method for monitoring the operation of a smart home device according to an embodiment of the present application is shown. Exemplarily, the method for monitoring the operation of a smart home device includes the following steps S100 to S200:

[0042] S100, in response to the monitoring instruction, collecting the operating data of all smart home devices in the preset area in real time.

[0043] This embodiment is mainly used in the home field, and its preset area can be the entire home area (including the living room, bedroom, kitchen, living room, etc.), or it can be a separate room or an area composed of several rooms. For example, the preset area can be the living room.

[0044] Smart home devices include, but are not limited to, smart sockets, smart lights, refrigerators, washing machines, air conditioners, cameras, alarms, door locks, smart speakers, routers, gateways, and other connected home devices. As you can see, smart home devices encompass all aspects of family life, including appliances, security, lighting, health, environmental control, and networking equipment. Each device collects specific operational data, which serves as input to the home device monitoring model to enable intelligent monitoring and early warning.

[0045] In this step, after receiving the monitoring instruction, it is necessary to collect the operating information of each smart home device in the preset area in real time or at preset intervals. The operating information includes basic operating information of the device and connection information between devices.

[0046] The basic operation information of the device includes but is not limited to the device controlled log, device operation status, the time when the device status change occurs, the execution time of the device, and the device online and offline information.

[0047] For example, the device control log records the time, content, and execution results of external control commands received by the device. For example, a user adjusts the temperature of a smart air conditioner from 26°C to 24°C via a mobile app at 10:30 AM; or a user successfully unlocks a smart door lock using a fingerprint at 8:15 PM.

[0048] The device operating status refers to the current working status of the device, including whether it is turned on and the mode setting. For example, a smart washing machine is currently in "Wash" mode and has been running for 15 minutes with 10 minutes remaining; a smart air purifier is currently in "Auto" mode and the air quality index is "Good".

[0049] The device state change time records the specific time when the device state changes. For example, a smart light bulb switches from off to on at 6:00 PM, or a smart curtain switches from fully closed to half-open at 7:30 AM.

[0050] Device execution time refers to the time it takes for a device to complete a task. For example, a smart refrigerator takes 2 seconds to switch from freezer mode to refrigerator mode, while a router takes 15 seconds to restart its network service.

[0051] Device online and offline information refers to the connection status between the device and the cloud platform or local area network, including the online and offline times. For example, a smart camera may go offline due to a power outage at 2:00 AM and come back online at 2:10 AM after power is restored. A power outlet may go offline due to a Wi-Fi signal interruption at 9:00 AM and come back online at 9:05 AM after the signal is restored.

[0052] The connection information between devices includes, but is not limited to, the network connection relationship between devices, the message transmission relationship between devices, and the time series association relationship between devices.

[0053] For example, the network connection relationship between devices describes the connection method and topology established between devices through the network. For example, a smart router and a smart light bulb connect to the router via Wi-Fi, forming a point-to-point connection; a smart speaker and a smart TV connect via Bluetooth, enabling synchronized audio and video playback.

[0054] The message transmission relationship between devices records the content of messages sent between devices and their transmission logic. For example, when a smart door lock detects someone opening the door, it sends a "record video" command to the camera. When a smart air conditioner detects that the indoor temperature exceeds the set value, it sends a "start cooling" command to the air conditioner. The operating status of a refrigerator may be affected by the kitchen socket.

[0055] Time series relationships between devices describe the dependencies or collaborations between them over time. For example, smart curtains and smart lights automatically open at 7:30 a.m., while the light brightness gradually decreases to simulate natural sunlight. A smart water heater and a smart alarm clock start heating the water heater when the alarm goes off (6:00 a.m.), ensuring hot water is readily available upon waking.

[0056] S200: Process the operating data and input it into the home appliance monitoring model to obtain an abnormality judgment result.

[0057] Exemplarily, processing the operating data includes a preprocessing process and a feature extraction process.

[0058] Preprocessing involves performing noise removal, normalization, and time alignment on the collected operational data. Noise removal typically involves removing invalid or erroneous data points (such as missing timestamps and outliers). Normalization involves bringing data of different dimensions (such as temperature, current, and time consumption) into the same range. Time alignment ensures consistent timestamps across multiple devices for easier analysis.

[0059] Feature extraction involves extracting time series data from preprocessed basic device operation information and constructing topological relationships between smart home devices based on this basic device operation information and the connection information between devices. This topological relationship constructs a graph structure between the devices, with smart home devices as nodes and connections between devices as edges (edges represent connections or dependencies between devices). Each node also includes the basic device operation data for the corresponding device.

[0060] After the running data is processed, it is input into the pre-trained home device monitoring model to obtain the abnormal judgment results.

[0061] Among them, the home device monitoring model is obtained by training a hybrid model consisting of graph neural network and Transformer based on the historical operation data of smart home devices.

[0062] like Figure 2 As shown, the acquisition of the home appliance monitoring model includes steps S210 to S220:

[0063] S210 , preprocessing the historical operation data of the smart home devices to obtain historical time series data and historical topological relationships between the smart home devices.

[0064] Among them, when training the home appliance monitoring model, the historical operation data used is the same as the above-mentioned operation data, including basic operation information of the equipment and connection information between the equipment.

[0065] The purpose of preprocessing the historical operation data of smart home devices in this step is to extract time series features and topological relationship features from the original historical operation data.

[0066] Among them, time series data is to arrange historical operation data (such as equipment status changes, execution time, etc.) in chronological order to form a time series, and convert these data into a standardized time series format to facilitate subsequent input into the Transformer module to capture the dynamic change rules of the equipment.

[0067] The historical topology refers to a topological graph that analyzes the network connectivity and message passing relationships between devices. The relationships between devices are modeled as a graph, where each node represents a device and edges represent connections or dependencies between devices. For example, device A controls device B (such as a smart switch controlling a light bulb); devices C and D share the same network (such as Wi-Fi).

[0068] S220 , based on the time series data and the historical topological relationship, the hybrid model is trained in combination with a preset loss function to obtain a home device monitoring model.

[0069] Since the operating status of smart home devices is not only affected by their own dynamic changes, but also closely related to the topological relationship between devices. Therefore, this application uses a hybrid model (graph neural network + Transformer) to predict whether the device has an abnormality. Among them, the Transformer module is good at processing time series data and can capture the dynamic laws of device changes over time (such as temperature fluctuations, time-consuming changes, etc.); the graph neural network module is good at processing graph structure data and can capture the connections and dependencies between devices (such as message passing between devices, network connections, etc.). The hybrid model of this embodiment combines the two modules to capture the time series characteristics and topological relationship characteristics of the device at the same time, thereby more comprehensively describing the operating status of the device, and then more accurately predicting the abnormal situation of the device.

[0070] In one embodiment, Figure 3 As shown, based on the time series data and the historical topological relationship, the hybrid model is trained in combination with the preset loss function to obtain a home appliance monitoring model, including steps S221 to S223:

[0071] S221, input the historical time series data into the Transformer module to obtain time series features, and input the historical topological relationship into the graph neural network module to obtain device relationship features.

[0072] S222: The time series features and device relationship features are fused and input into the fully connected layer to obtain the model prediction results.

[0073] S223, calculating the loss between the model prediction result and the actual result through the loss function, and updating the parameters of the hybrid model according to the loss until the updated hybrid model meets the preset cutoff condition, thereby obtaining the home appliance monitoring model.

[0074] For example, historical time series data is fed into the Transformer module, which then extracts time series feature vectors from the device operation data to capture the state changes of the devices at different time points. Simultaneously, historical topological relationships are fed into the graph neural network module. Graph neural networks excel at processing graph-structured data and can learn the dependencies between devices. This module then extracts device relationship feature vectors to capture the collaborative working patterns between devices.

[0075] Then, a feature fusion method (such as splicing or weighted summation) is used to merge the two features, and the fused comprehensive feature vector is used as the input of the fully connected layer to map the comprehensive feature vector to the final prediction result, such as whether the device will have an abnormality.

[0076] Finally, the loss function is used to calculate the loss between the predicted result and the actual result, and the model parameters are continuously optimized according to the loss to make the model prediction result closer to the actual result.

[0077] Among them, the loss function can be: Among them, y i is the true result of the sample, y i ′ is the prediction result of the model; N is the number of training samples; L BEC is the loss between the predicted result and the true result; w(t i ) is the time decay weight of the i-th sample; t i is the generation time of the i-th sample.

[0078] It is understandable that the operating status of smart home devices may change over time, especially when user behavior patterns adjust (such as seasonal changes, changes in living habits, etc.). Traditional loss functions treat all historical data equally and may not fully reflect the importance of recent data. This embodiment introduces a time decay mechanism into the loss function, that is, assigning different weights to historical data (the closer the data is to the current data, the higher the weight), so that the model can pay more attention to the user's latest behavior patterns, thereby improving the accuracy and adaptability of the prediction.

[0079] Among them, the time decay function w(t i ) can be an exponential decay function, Where, t curis the current time, λ is the decay rate parameter, and λ is greater than 0.

[0080] In one embodiment, the abnormality judgment result includes the abnormality type (e.g., device overheating, connection interruption, etc.), abnormal device (clearly indicating which device has the problem), abnormality occurrence time (recording the specific time of abnormality occurrence) and abnormality severity (assessing the impact of the abnormality on device operation (such as minor, moderate or severe)).

[0081] After obtaining the abnormality judgment result, it also includes: generating corresponding processing suggestions based on the abnormality judgment result and combining it with the abnormality processing knowledge base, and outputting corresponding alarm instructions.

[0082] Exemplarily, after obtaining the abnormal judgment result, the system will provide the user with specific processing suggestions based on the abnormal judgment result and the pre-established abnormal handling knowledge base. Among them, the abnormal handling knowledge base is a database that stores a large number of known problems and their solutions. For example, if it is "device overheating", the knowledge base may suggest that the user check the radiator or reduce the device load; if it is "connection interruption", the knowledge base may suggest restarting the device or checking the network configuration. The processing suggestions can usually be: automated processing (for example, for simple problems (such as restarting the device), the system can directly perform the repair operation automatically) or manual processing (for complex problems (such as hardware failure), the system will prompt the user to contact a technician or operate manually). When performing manual processing, the system will give the corresponding processing method.

[0083] In addition to providing treatment suggestions, the system will also generate corresponding alarm instructions based on the severity and type of the anomaly and notify the user. The form of these alarm instructions can be diverse. For example, a minor anomaly can be pushed as a standard notification via the mobile app; a moderate anomaly can alert the user via text message or voice; and a severe anomaly can trigger an emergency alarm, which may include flashing lights, audible alarms, or a direct call to a preset emergency contact. The alarm instructions will also include, but are not limited to, a detailed description of the anomaly and corresponding treatment suggestions (if manual intervention is required, the next steps will be clearly stated).

[0084] Furthermore, if certain exceptions lack clear handling suggestions, or if the suggestions are incomplete, the system will mark these issues as "pending." Backend technicians will manually annotate these unresolved issues and add the final solutions to the exception handling knowledge base. This additional data will be used for subsequent model retraining, continuously improving the system's intelligence.

[0085] In addition, all exception information (including exception type, device, time, severity, and handling suggestions) is stored in a database to facilitate rapid retrieval and handling of similar issues. For example, if a similar exception occurs again in the future, the system can directly retrieve the corresponding solution from the database without reanalyzing it.

[0086] The home device monitoring model of this embodiment is composed of a graph neural network that can capture the static dependencies between devices and a Transformer that can capture the dynamic patterns of device changes over time. This allows it to simultaneously capture both the time series characteristics and topological relationship characteristics of the devices, thereby more comprehensively describing the device's operating status. Furthermore, by integrating multiple feature extraction methods, the hybrid model can demonstrate greater adaptability in different scenarios. This is particularly true in complex smart home environments, where the operating status of devices is often affected by multiple factors. The hybrid model can better generalize to unseen data, allowing for more accurate identification of device anomalies.

[0087] Figure 4 A schematic diagram of the structure of a smart home device operation monitoring device according to an embodiment of the present application is shown. Exemplarily, the smart home device operation monitoring device includes:

[0088] The collection module 100 is used to collect the operating data of all smart home devices in a preset area in real time in response to monitoring instructions.

[0089] The operation data includes basic operation information of the equipment and connection information between the equipment.

[0090] The acquisition module 200 is used to process the operation data and input it into the home appliance monitoring model to obtain an abnormality judgment result.

[0091] Among them, the home device monitoring model is obtained by training a hybrid model consisting of graph neural network and Transformer based on the historical operation data of smart home devices.

[0092] In one embodiment, the basic operation information of the device includes the device controlled log, the device operation status, the time when the device status change occurs, the device execution time, and the device online and offline information; the connection information between devices includes the network connection relationship between devices, the message transmission relationship between devices, and the time series association relationship between devices.

[0093] In one embodiment, when the acquisition module 200 processes the operation data, it specifically includes: extracting time series data from the basic operation information of the device; and constructing the topological relationship between each smart home device based on the basic operation information of the device and the connection information between the devices.

[0094] In one embodiment, the abnormality determination result includes the abnormality type, abnormal device, abnormality occurrence time, and abnormality severity.

[0095] The smart home device operation monitoring device also includes an alarm module, which is used to generate corresponding processing suggestions based on the abnormality judgment result and in combination with the abnormality processing knowledge base after obtaining the abnormality judgment result, and output a corresponding alarm indication.

[0096] In one embodiment, the smart home device operation monitoring device also includes a model acquisition module, which is used to train the model, specifically for preprocessing the historical operation data of the smart home devices to obtain historical time series data and historical topological relationships between each smart home device; then based on the time series data and historical topological relationships, combined with a preset loss function, the hybrid model is trained to obtain a home device monitoring model.

[0097] Among them, based on time series data and historical topological relationships, the hybrid model is trained in combination with a preset loss function to obtain a home appliance monitoring model, including: inputting historical time series data into the Transformer module to obtain time series features, and inputting historical topological relationships into the graph neural network module to obtain device relationship features; fusing the time series features and device relationship features and inputting them into the fully connected layer to obtain the model prediction results; calculating the loss between the model prediction results and the actual results through the loss function, and updating the parameters of the hybrid model according to the loss until the updated hybrid model meets the preset cutoff conditions, thereby obtaining the home appliance monitoring model.

[0098] Among them, the loss function is: Among them, y i is the true result of the sample, is the prediction result of the model.

[0099] It can be understood that the device of this embodiment corresponds to the smart home device operation monitoring method of the above embodiment, and the optional options in the above embodiment are also applicable to this embodiment, so they will not be repeated here.

[0100] The present application also provides a terminal device. Exemplarily, the terminal device includes a processor and a memory, wherein the memory stores a computer program, and the processor runs the computer program to enable the terminal device to execute the functions of each module in the above-mentioned smart home device operation monitoring method or the above-mentioned smart home device operation monitoring device.

[0101] Among them, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU) and a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or at least one of other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application.

[0102] The memory may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory is used to store a computer program, and the processor may execute the computer program accordingly after receiving an execution instruction.

[0103] The present application also provides a computer-readable storage medium for storing the computer program used in the terminal device. For example, the computer-readable storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0104] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or flowchart, and the combination of boxes in the structure diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0105] In addition, the functional modules or units in the various embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0106] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a smart phone, personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0107] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. A method for monitoring the operation of smart home devices, characterized in that: include: In response to monitoring instructions, real-time collection of operating data of all smart home devices in a preset area; wherein the operating data includes basic operating information of the devices and connection information between the devices; The operating data is processed and input into a home appliance monitoring model to obtain an abnormality judgment result; wherein, the home appliance monitoring model is obtained by training a hybrid model composed of a graph neural network and a Transformer based on the historical operating data of the smart home device.

2. The method for monitoring the operation of smart home devices according to claim 1, wherein: The basic operation information of the device includes the device controlled log, device operation status, device status change time, device execution time and device online and offline information; The connection information between the devices includes the network connection relationship between the devices, the message transmission relationship between the devices and the time series association relationship between the devices.

3. The method for monitoring the operation of smart home devices according to claim 1, wherein: The processing of the operating data includes: extracting time series data from basic operation information of the equipment; A topological relationship between each smart home device is constructed based on the basic operation information of the device and the connection information between the devices.

4. The method for monitoring the operation of smart home devices according to claim 1, wherein: The abnormality judgment result includes the abnormality type, abnormal device, abnormality occurrence time and abnormality severity; After obtaining the abnormality judgment result, the method further includes: According to the abnormality judgment result, corresponding processing suggestions are generated in combination with the abnormality processing knowledge base, and corresponding alarm instructions are output.

5. The method for monitoring the operation of smart home devices according to claim 1, wherein: The acquisition of the household equipment monitoring model includes: Preprocessing the historical operation data of the smart home devices to obtain historical time series data and historical topological relationships between the smart home devices; Based on the historical time series data and the historical topological relationship, the hybrid model is trained in combination with a preset loss function to obtain the home appliance monitoring model.

6. The method for monitoring the operation of smart home devices according to claim 5, characterized in that: The method of training the hybrid model based on the historical time series data and the historical topological relationship in combination with a preset loss function to obtain a home appliance monitoring model includes: Input the historical time series data into the Transformer module to obtain time series features, and input the historical topological relationship into the graph neural network module to obtain device relationship features; The time series features and the device relationship features are fused and input into a fully connected layer to obtain a model prediction result; The loss between the model prediction result and the actual result is calculated by the loss function, and the parameters of the hybrid model are updated according to the loss until the updated hybrid model meets the preset cutoff condition, thereby obtaining the home appliance monitoring model.

7. The method for monitoring the operation of smart home devices according to claim 5 or 6, characterized in that: The loss function is: Among them, y i is the true result of the sample, y′ i is the prediction result of the model; N is the number of training samples; L BEC is the loss between the predicted result and the true result; w(t i ) is the time decay weight of the i-th sample; t i is the generation time of the i-th sample.

8. A smart home equipment operation monitoring device, characterized in that: include: A collection module is used to collect the operating data of all smart home devices in a preset area in real time in response to monitoring instructions; wherein the operating data includes basic operating information of the devices and connection information between the devices; An acquisition module is used to process the operating data and input it into a home appliance monitoring model to obtain an abnormality judgment result; wherein, the home appliance monitoring model is obtained by training a hybrid model composed of a graph neural network and a Transformer based on the historical operating data of the smart home device.

9. A terminal device, characterized in that: The terminal device includes a processor and a memory, the memory stores a computer program, and the processor is used to execute the computer program to implement the smart home device operation monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that It stores a computer program, which, when executed on a processor, implements the smart home device operation monitoring method according to any one of claims 1 to 7.

Images