Intelligent house based on digital twinning

Abstract

The application discloses an intelligent home house based on digital twinning, and relates to the technical field of intelligent device monitoring, which comprises the following steps: collecting home device data by installing multi-modal sensors, extracting device basic features for preliminary classification of devices, constructing a home topology structure in combination with device connection relationships, performing device modeling on the basis of the characteristics of each device in the topology structure, and finally outputting device virtual features; a feature fusion model is established, local residual errors and global residual errors are extracted according to the device features, the feature fusion model is trained in combination with a loss function, and finally fused device features are outputted; and a multi-layer state monitoring network is constructed according to the fused device features to output device states. The application enhances the understanding ability of device behaviors and spatial layouts, realizes higher device state detection accuracy and early warning capability, and provides users with an intuitive home environment control experience.

Description

Technical Field

[0001] This invention relates to the field of intelligent device monitoring technology, and in particular to an intelligent home based on digital twins. Background Technology

[0002] With the rapid development of Internet of Things (IoT) technology, smart homes, as one of the important application scenarios of IoT, have gradually become a focus of research and industry. Traditional smart home systems are mostly based on single sensing modalities (such as temperature, humidity, motion sensors, etc.) for information collection, and use rule engines or simple data analysis to achieve intelligent adjustment of the home environment. This type of technology emphasizes the improvement of hardware sensing capabilities and has successfully achieved real-time monitoring and control of some environmental parameters, but it still has limitations in the comprehensive control of the status of home devices and anomaly detection. In recent years, with the rise of data-driven technologies, deep learning, edge computing, and big data analysis have been gradually introduced, greatly enriching the means of status sensing in smart homes. For example, methods based on multimodal data fusion combine multi-source information... Information has improved the accuracy and robustness of home environment perception, and promoted the development of automated monitoring and fault diagnosis technologies. However, such methods mostly focus on modeling single spatial environmental parameters, lacking a deep understanding and comprehensive modeling of device levels. They are unable to fully capture the complex interaction relationships between devices and the dynamic changes of multiple states within devices, resulting in limited monitoring accuracy and fault identification rate. Although many studies have attempted to combine sensor arrays for device state identification and proposed methods such as state classification based on deep neural networks, enhanced feature representation, and time series prediction, they still face many shortcomings in practical applications. The diversity and complex connection relationships of devices are not fully expressed, and the need to integrate heterogeneous and multi-source information cannot be met, affecting the in-depth promotion of home intelligence. Summary of the Invention

[0003] In view of the aforementioned existing problems, the present invention is proposed.

[0004] Therefore, the present invention provides a smart home based on digital twins, which solves the problem that the diversity and complex connection relationships of devices in the prior art are not fully expressed and cannot meet the needs of fusion of heterogeneous devices and multi-source information.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] In a first aspect, the present invention provides a smart home based on digital twins, comprising,

[0007] By installing multimodal sensors to collect data on home appliances, extracting basic features of the appliances for preliminary classification, and constructing a home topology based on the connection relationships of the appliances, the system models the characteristics of each appliance in the topology and finally outputs virtual features of the appliances.

[0008] Establish a feature fusion model, extract local and global residuals based on device features, train the feature fusion model with a loss function, output fused device features, and construct a multi-layer state monitoring network based on the fused device features to output device status.

[0009] The status of smart home devices is determined and displayed based on their status, and then synchronously stored in the smart database.

[0010] As a preferred embodiment of the smart home based on digital twins described in this invention, the following steps are taken: First, basic device features are extracted for preliminary device classification. Then, a home topology is constructed based on device connectivity. Next, device modeling is performed for the characteristics of each device in the topology, ultimately outputting virtual device feature vectors. Based on collected home device data, basic device features are extracted, and statistical methods are used to standardize these features to form device feature vectors. Finally, a clustering algorithm is used to perform preliminary device classification based on these feature vectors.

[0011] Establish a device adjacency matrix A based on device type and actual device connection relationships. If device i and device j are direct vectors, then... The actual connection relationship of the devices is 1, which includes pipeline connection and network connection;

[0012] Based on the device adjacency matrix, connecting edges are formed between adjacent home devices, and the actual connection relationships of the devices are simultaneously marked on the connecting edges;

[0013] The basic features of the devices are used as node features, and the home topology G is constructed by combining the connecting edges. The direction of the connecting edges is defined according to the direction of the connection relationship between the devices to form a directed topology graph.

[0014] Each home appliance is modeled individually based on its characteristics. For all smart home appliances, a heat conduction model is established using partial differential equations of heat conduction.

[0015] For some smart home devices, a vibration model is established using a spring-mass-damping model.

[0016] The home equipment model is optimized using a loss function and optimization algorithm, and the virtual features of the equipment are output based on the optimized home equipment model.

[0017] As a preferred embodiment of the smart home based on digital twins described in this invention, the following steps are described: establishing a feature fusion model, extracting local and global residuals based on device features, training the feature fusion model using a loss function, and outputting fused device feature indicators; and using a deep learning network to establish the feature fusion model, extracting local temperature partial differential residuals based on the device temperature field output by the device heat conduction model. ;

[0018] The global temperature residual is calculated by comprehensively considering the local temperature partial differential residuals. ;

[0019] Simultaneously extract global vibration residuals for some home appliances. ;

[0020] The mean squared error is calculated based on the feature fusion model, and then combined with the global residual to construct a multi-task loss function. ;

[0021] The gradient descent method is used to train the feature fusion model parameters based on the multi-task loss function, and the optimized feature fusion model is output. The actual vibration response and temperature field of the equipment are extracted from the equipment data collected by the sensor and the optimized feature fusion model is input and the fused equipment features are output.

[0022] As a preferred embodiment of the smart home based on digital twins described in this invention, the step of constructing a multi-layer state monitoring network to output device status based on fused device characteristics refers to constructing a multi-layer state monitoring network using a multi-layer perceptron network after obtaining the fused device characteristics. The multi-layer state monitoring network has two output layers. The fused device characteristics are input into the multi-layer state monitoring network to output device status. The upstream and downstream influencing devices are extracted in the directed topology graph. The influencing devices are analyzed based on the device status and marked in the directed topology graph.

[0023] As a preferred embodiment of the smart home based on digital twins described in this invention, the step of judging and displaying the smart home status based on the device status and synchronously storing it in the smart database refers to judging the smart home situation based on the device status. If the device status affects the smart home environment, the device is reported for repair and a maintenance report is generated and stored in the smart database.

[0024] As a preferred embodiment of the smart home based on digital twins described in this invention, the smart database synchronously collects and stores device data after obtaining a maintenance report, and records it according to timestamps as historical device data.

[0025] As a preferred embodiment of the smart home based on digital twins described in this invention, the step of collecting home device data by installing multimodal sensors refers to collecting data from home devices by installing multimodal sensors on the home devices and calibrating and synchronizing the sensor time using a high-precision synchronous clock, and preprocessing the collected home device data.

[0026] As a preferred embodiment of the smart home based on digital twins described in this invention, it includes:

[0027] The device modeling module is used to collect data on home devices by installing multimodal sensors, extract basic features of the devices for preliminary classification, and construct a home topology based on the device connection relationships. It then models the characteristics of each device in the topology and finally outputs virtual features of the devices.

[0028] The equipment analysis module is used to establish a feature fusion model, extract local and global residuals based on equipment features, train the feature fusion model with a loss function, and output fused equipment features. Based on the fused equipment features, a multi-layer status monitoring network is constructed to output the equipment status.

[0029] The display storage module is used to determine the status of smart home devices and display it, which is then synchronously stored in the smart database.

[0030] In a second aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the digital twin-based smart home as described in the first aspect of the present invention.

[0031] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein, when the computer program is executed by a processor, it implements any step of the digital twin-based smart home as described in the first aspect of the present invention.

[0032] The beneficial effects of this invention are as follows: By integrating rich device data collected by multimodal sensors, the system achieves accurate classification of home devices and structured expression of spatial relationships, enhancing the understanding of device behavior and spatial layout. Based on deep feature fusion technology, it effectively integrates multi-source information, highlighting minor anomalies and overall trends, thereby achieving higher accuracy in device status detection and early warning capabilities. At the same time, intelligent status judgment and dynamic display provide users with an intuitive experience of controlling their home environment. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart of a smart home room based on digital twins in Example 1.

[0035] Figure 2 This is a structural diagram of the smart home room based on digital twins in Example 1. Detailed Implementation

[0036] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0037] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0038] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0039] Example 1, referring to Figure 1 and Figure 2 This is the first embodiment of the present invention, which provides a smart home based on digital twins, including the following steps:

[0040] S1. Collect home appliance data by installing multimodal sensors, extract basic features of the appliances for preliminary classification, and construct a home appliance topology based on the connection relationships of the appliances. Model the appliances for each feature in the topology and finally output the virtual features of the appliances.

[0041] Specifically, collecting data from home appliances by installing multimodal sensors refers to collecting data such as vibration, temperature, and electrical parameters of home appliances by installing multimodal sensors, including vibration sensors, temperature sensors, and electrical parameter sensors, and calibrating and synchronizing the sensors using a high-precision synchronous clock, and then preprocessing the collected data.

[0042] Specifically, basic device features are extracted for preliminary device classification, and a home topology is constructed based on device connectivity. Device modeling is then performed for each device characteristic within the topology, ultimately outputting virtual device feature vectors. Based on collected home device data, basic device features are extracted, including rated power, current, voltage, thermal efficiency, operating frequency, vibration amplitude, and temperature fluctuation range. These basic features are then standardized using statistical methods (mean, variance, skewness, kurtosis) to form device feature vectors. Finally, a clustering algorithm is used to perform preliminary device classification based on these feature vectors.

[0043]

[0044] in For clustering loss, Let be the device feature vector of the i-th device. For the j-th type of equipment, K is the class center, and K is the total number of device types;

[0045] Establish a device adjacency matrix A based on device type and actual device connection relationships. If device i and device j are direct vectors, then... The actual connection relationship of the devices is 1, which includes pipeline connection and network connection;

[0046] Based on the device adjacency matrix, connection edges are formed between adjacent home devices. For different device connection relationships, different device parameters are calculated as connection edge weights. For devices with pipeline connections, the distance between devices is calculated as connection edge weights. For devices with network connections, the network bandwidth is calculated as connection edge weights. The actual device connection relationships are simultaneously marked on the connection edges.

[0047] The basic features of the devices are used as node features. The home topology G is constructed by combining the connecting edges. The direction of the connecting edges is defined according to the direction of the connection relationship between the devices to form a directed topology graph. For example, if device i and device j are connected by a pipe and the pipe flow direction is from device i to device j, then the direction of the connecting edge between device i and device j is also from device i to device j.

[0048] Each smart home device is modeled individually based on its characteristics. Since all smart home devices are powered on and therefore generate heat, a heat conduction model for each device is established using partial differential equations of heat conduction.

[0049]

[0050] in The temperature field at location x of a home appliance at time t is estimated by model simulation using temperature data at the corresponding location of the appliance. For the device resistance, For equipment current, The thermal diffusivity;

[0051] For some smart home devices with mechanical moving structures, vibrations will occur during operation. A vibration model of the device is established using a spring-mass-damping model:

[0052]

[0053] in External excitations to the equipment, such as eccentric vibrations or mechanical disturbances, Let m be the vibration response of the equipment at time t, c be the damping coefficient, and k be the spring stiffness.

[0054] The home appliance models (heat conduction and vibration models, with the vibration model only applicable to some appliances) are optimized using a loss function and an optimization algorithm. The optimization parameters are m, c, k, and... The system outputs virtual features of the devices based on the optimized home appliance model, including vibration response. and temperature field All feature parameters in the virtual features of the device are ideal parameters of the basic features of the device, which are obtained by simulating the ideal operating state through the device model.

[0055] Statistical methods (mean, variance, skewness, kurtosis) are used to normalize and standardize equipment characteristics, eliminating biases in units, measurement conditions, and scale. This provides a unified analytical foundation for multidimensional features, improves the accuracy and robustness of equipment feature representation, effectively filters outlier data interference, and prevents clustering results from deviating from the true categories. Multi-faceted descriptions of statistical parameters enhance the ability to differentiate equipment, especially skewness and kurtosis, which capture the asymmetry and peak-to-peak characteristics of equipment operation. This is particularly crucial for identifying abnormal or special operating conditions. The resulting equipment classification has higher reliability and consistency, facilitating subsequent intelligent diagnosis and maintenance. Equipment connection relationships (pipeline connections, network connections), distance, and bandwidth are introduced as edge weights. Combined with an adjacency matrix to define the home topology, a directed graph model is established. By differentiating edge weights, connection strength and energy transfer characteristics are dynamically reflected, providing a mathematical basis for diverse relationship analyses. Directional design achieves a natural expression of flow directions between equipment. This model facilitates the reflection of actual fluid and heat transfer mechanisms, providing reasonable boundary conditions for the simulation of heat conduction and vibration models. Utilizing the spatial relationship information that can be implemented using graphical models helps optimize energy consumption control, fault prediction, and fault propagation path analysis, thereby improving the overall intelligence level of the system. A multi-physics coupled model is established using partial differential equations (heat conduction) and a spring-damping model (vibration). Parametric optimization and simulation output virtual features accurately simulate the heat distribution and vibration response of equipment during actual operation, overcoming the problem of inaccurate parameter estimation in traditional models. By adjusting model parameters (such as resistance, current, and vibration parameters) through optimization algorithms, a high degree of consistency between the model and actual equipment operation is achieved, improving the representativeness and reliability of virtual features. The generated virtual features (vibration response and temperature field) provide richer and more detailed diagnostic indicators for equipment fault detection, performance monitoring, and intelligent diagnosis. Even in scenarios lacking on-site measurements, it can simulate real working conditions, providing an effective basis for equipment maintenance.

[0056] S2. Establish a feature fusion model, extract local and global residuals based on device features, train the feature fusion model using a loss function, output fused device features, and construct a multi-layer state monitoring network based on the fused device features to output device status.

[0057] Specifically, a feature fusion model is established, and local and global residuals are extracted based on equipment features. After training the feature fusion model with a loss function, the fused equipment feature index is output. A deep learning network is used to establish the feature fusion model. The model input is vibration response and temperature field, and the output is fused equipment features. Local temperature partial differential residuals are extracted based on the equipment temperature field output by the equipment heat conduction model. :

[0058]

[0059] in The operating current of the equipment is obtained through sensors, while the temperature field at this time is obtained by simulation of the equipment's heat conduction model. There is a residual between the temperature field and the sensor data.

[0060] The global temperature residual is calculated by comprehensively considering the local temperature partial differential residuals. :

[0061]

[0062] in For all spatial points of the equipment;

[0063] Simultaneously extract global vibration residuals for some home appliances. :

[0064]

[0065] in The external stimulus to the device is acquired through sensors. For ideal vibration response, it differs from the actual external excitation of the equipment. There are errors between them;

[0066] The mean squared error is calculated based on the feature fusion model, and then combined with the global residual to construct a multi-task loss function. For household appliances that only have a temperature field, the function is:

[0067]

[0068] For home appliances that possess temperature field and vibration response, the function is:

[0069]

[0070] The gradient descent method is used to train the feature fusion model parameters based on the multi-task loss function, and the optimized feature fusion model is output. The actual vibration response and temperature field of the equipment are extracted from the equipment data collected by the sensor and the optimized feature fusion model is input and the fused equipment features are output.

[0071] This study employs a deep learning network to fuse two multimodal data sources: vibration response and temperature field. This overcomes the limitations of traditional single-parameter methods, taking into account both static heat conduction characteristics and dynamic vibration information. This achieves an effective combination of multi-scale and multi-physics field information. The deep learning model adaptively adjusts the feature representation during the fusion process, solving the technical challenges of inconsistent scales across different data modalities and the difficulty of information fusion. The fused features exhibit excellent robustness in reflecting the true operating state of equipment, especially in complex environments and nonlinear conditions, effectively improving monitoring accuracy and even adapting to and recognizing some new types of equipment. This achieves an organic combination of complementary vibration and temperature information, effectively alleviating the bottleneck of single feature sources failing to capture complex anomalies. By rationally utilizing partial differential equations (PDEs) to describe point temperature gradient changes and monitoring the model's ability to fit local features, and incorporating PDE residuals into multimodal deep fusion, the model's sensitivity to minute local thermal anomalies is significantly improved, far exceeding the error monitoring methods that rely solely on global residual detection. This allows for the early detection of local thermal stress accumulation and potential hazards, providing support for preventative maintenance. By introducing external excitation signals from the equipment and combining them with vibration response deviations to establish global residuals, deviations from external mechanical disturbances can be captured. In real-world scenarios, these residuals can sensitively capture deviation signals, enhancing the sensitivity and reliability of mechanical disturbance monitoring. In terms of fault early warning, the response time is several times faster than traditional diagnostic methods, significantly improving operational safety and equipment lifespan.

[0072] Furthermore, based on the characteristics of the fused equipment, a multi-layer condition monitoring network is constructed to output the device status. After obtaining the fused characteristics of the equipment, a multi-layer perceptron network is used to construct the multi-layer condition monitoring network. The multi-layer condition monitoring network has two output layers, including fault classification and lifetime prediction. The fused equipment characteristics are input into the multi-layer condition monitoring network to output the device status. The upstream and downstream influencing devices are extracted in the directed topology graph. The influencing devices are analyzed based on the device status and marked in the directed topology graph.

[0073] The deep structure of a multilayer perceptron can automatically learn high-order feature interactions, capturing complex nonlinear changes in equipment better than traditional linear or single-layer models. For example, the combination of vibration and temperature can not only detect obvious anomalies but also identify subtle degradation features in advance, enabling earlier warning capabilities. The multi-task structure design can learn fault types and remaining equipment lifespan on the basis of shared features, enabling multi-angle and multi-level monitoring and early warning. In anomaly detection, it can not only identify specific fault categories but also provide equipment remaining lifespan predictions, significantly improving the flexibility and effectiveness of maintenance strategies. The introduction of a directed graph structure clarifies the order of influence and causal relationships between devices, enabling dynamic influence inference capabilities. Events are propagated along directed paths, allowing the influence chain to be traced after a fault occurs, achieving early warning and accurate fault source location, thereby reducing maintenance costs.

[0074] S3. Determine the status of smart home devices and display the status, and store it synchronously in the smart database;

[0075] Specifically, judging the status of smart home devices based on their status and displaying it, and storing it synchronously in the smart database means judging the status of smart home devices based on their status. If the status of a device affects the smart home environment (e.g., the device is damaged or its remaining lifespan is about to end), then the device is reported for repair and a maintenance report is generated and stored in the smart database.

[0076] Furthermore, after obtaining the maintenance report, the intelligent database synchronously collects and stores the equipment data, and records it according to the timestamp as historical equipment data. The intelligent database can call up all historical data when needed.

[0077] This embodiment also provides a smart home status monitoring system based on digital twins, including:

[0078] The device modeling module is used to collect data on home devices by installing multimodal sensors, extract basic features of the devices for preliminary classification, and construct a home topology based on the device connection relationships. It then models the characteristics of each device in the topology and finally outputs virtual features of the devices.

[0079] The equipment analysis module is used to establish a feature fusion model, extract local and global residuals based on equipment features, train the feature fusion model with a loss function, and output fused equipment features. Based on the fused equipment features, a multi-layer status monitoring network is constructed to output the equipment status.

[0080] The display storage module is used to determine the status of smart home devices and display it, which is then synchronously stored in the smart database.

[0081] This embodiment also provides a computer device applicable to smart homes based on digital twins, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the smart home based on digital twins as proposed in the above embodiment.

[0082] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0083] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements a smart home based on digital twins as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0084] In summary, by integrating rich device data collected by multimodal sensors, this invention enables the system to accurately classify home devices and structurally express their spatial relationships, enhancing the understanding of device behavior and spatial layout. Based on deep feature fusion technology, it effectively integrates multi-source information, highlighting minor anomalies and overall trends, thereby achieving higher accuracy in device status detection and early warning capabilities. At the same time, intelligent status judgment and dynamic display provide users with an intuitive experience of controlling their home environment.

[0085] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A smart home room based on digital twins, characterized in that: include, By installing multimodal sensors to collect data on home appliances, extracting basic features of the appliances for preliminary classification, and constructing a home topology based on the connection relationships of the appliances, the system models the characteristics of each appliance in the topology and finally outputs virtual features of the appliances. Establish a feature fusion model, extract local and global residuals based on device features, train the feature fusion model with a loss function, output fused device features, and construct a multi-layer state monitoring network based on the fused device features to output device status. The status of smart home devices is determined and displayed, and then synchronously stored in the smart database. The process involves extracting basic device features for preliminary device classification, constructing a home topology based on device connectivity, modeling each device characteristic within the topology, and finally outputting virtual device feature vectors. Based on collected home device data, basic device features are extracted, and statistical methods are used to standardize these features to form device feature vectors. Clustering algorithms are then used to perform preliminary device classification based on these feature vectors. Establish a device adjacency matrix A based on device type and actual device connection relationships. If device i and device j are direct vectors, then... The actual connection relationship of the devices is 1, which includes pipeline connection and network connection; Based on the device adjacency matrix, connecting edges are formed between adjacent home devices, and the actual connection relationships of the devices are simultaneously marked on the connecting edges; The basic features of the devices are used as node features, and the home topology G is constructed by combining the connecting edges. The direction of the connecting edges is defined according to the direction of the connection relationship between the devices to form a directed topology graph. Each home appliance is modeled individually based on its characteristics. For all smart home appliances, a heat conduction model is established using partial differential equations of heat conduction. For some smart home devices, a vibration model is established using a spring-mass-damping model. The home equipment model is optimized using a loss function and optimization algorithm, and the virtual features of the equipment are output based on the optimized home equipment model. The feature fusion model is established, and local and global residuals are extracted based on equipment features. After training the feature fusion model with a loss function, the fused equipment feature index is output. A deep learning network is used to establish the feature fusion model, and local temperature partial differential residuals are extracted based on the equipment temperature field output by the equipment heat conduction model. ; The global temperature residual is calculated based on the local temperature partial differential residual. ; Simultaneously extract global vibration residuals for some home appliances. ; The mean squared error is calculated based on the feature fusion model, and then combined with the global residual to construct a multi-task loss function. ; The gradient descent method is used to train the feature fusion model parameters based on the multi-task loss function, and the optimized feature fusion model is output. The actual vibration response and temperature field of the equipment are extracted from the equipment data collected by the sensor and the optimized feature fusion model is input and the fused equipment features are output.

2. The smart home based on digital twin as described in claim 1, characterized in that: The process of constructing a multi-layer state monitoring network based on the characteristics of fused devices to output device status refers to the process of constructing a multi-layer state monitoring network using a multi-layer perceptron network after obtaining the fused device characteristics. The multi-layer state monitoring network has two output layers. The fused device characteristics are input into the multi-layer state monitoring network to output device status. The upstream and downstream devices that affect the device are extracted in the directed topology graph. The affected devices are analyzed based on the device status and marked in the directed topology graph.

3. The smart home room based on digital twin as described in claim 2, characterized in that: The process of determining and displaying the smart home status based on device status and storing it synchronously in the smart database refers to determining the smart home situation based on device status. If the device status affects the smart home environment, a repair request will be processed and a maintenance report will be generated and stored in the smart database.

4. The smart home based on digital twin as described in claim 3, characterized in that: The intelligent database collects and stores device data synchronously after obtaining a maintenance report, and records it according to timestamps as historical device data.

5. The smart home room based on digital twin as described in claim 4, characterized in that: The process of collecting home appliance data by installing multimodal sensors refers to collecting data from home appliances by installing multimodal sensors and calibrating the sensors using a high-precision synchronous clock, and then preprocessing the collected data.

6. The smart home room based on digital twin as described in claim 5, characterized in that: include, The device modeling module is used to collect data on home devices by installing multimodal sensors, extract basic device features for preliminary device classification, construct a home topology based on device connection relationships, model the characteristics of each device in the topology, and finally output virtual device features. The equipment analysis module is used to establish a feature fusion model, extract local and global residuals based on equipment features, train the feature fusion model with a loss function, and output fused equipment features. Based on the fused equipment features, a multi-layer status monitoring network is constructed to output the equipment status. The display storage module is used to determine the status of smart home devices and display it, which is then synchronously stored in the smart database.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the smart home based on digital twins as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the smart home based on digital twins as described in any one of claims 1 to 6.

Images