Smart home service management and control method based on federated learning

Abstract

The application provides a smart home service management and control method based on federated learning, comprising the following steps: step 1: for the problem of smart home device heterogeneity, a smart home time sequence knowledge graph construction method is proposed, the specific device is decoupled from the upper management and control logic, the functions of the smart home device are uniformly managed in the form of service, and support for user-perceived environmental changes and service state time sequence data is provided; step 2: for the differentiated needs of different users in the smart home service, differentiated and personalized smart home service management and control are provided for the users on the basis of the time sequence knowledge graph.

Description

Technical Field

[0001] This invention relates to the field of smart home technology, and in particular to a smart home service management method based on federated learning. Background Technology

[0002] With the rapid development of IoT and AI technologies, smart devices are increasingly enriching people's lives, and smart homes are gradually becoming an indispensable part of the home experience. Smart homes provide diverse services through smart devices, bringing people a more convenient and comfortable living experience. The concept of smart homes was formally proposed by the National Association of Home Builders in 1984. At that time, smart home systems only served as a unified control portal for different services and smart devices in the home. The process of smart home service management involves collecting and analyzing various information about the service recipient's situation, then making decisions and implementing corresponding smart home services to meet the service recipient's needs for adjustments to sensitive environmental conditions, thereby providing users with a comfortable smart home service experience. With the continuous development of smart device technology and the continuous improvement of network communication infrastructure, users' requirements for smart home service management are also constantly increasing. Currently, major smart device manufacturers are committed to building smart home ecosystems for their own devices; smart home systems with only simple device control capabilities can no longer meet user needs. How to achieve more intelligent and personalized smart home service management, lower the threshold for using smart homes, and provide users with a more comfortable living experience has become a research hotspot in the smart home field.

[0003] Smart home service management is often scenario-based. For example, it might adjust temperature, humidity, and lighting based on the user's environment to provide a comfortable living experience. However, the massive number of heterogeneous smart home devices and the on-demand nature of smart home services pose significant challenges to implementing smart home service management. Firstly, different brands of smart devices lack standardized specifications. Each manufacturer focuses on building its own ecosystem, resulting in different data formats, communication protocols, and function call methods for smart home devices from different brands. Therefore, achieving interactive and collaborative management between different smart devices in scenarios involving different brands requires familiarity with each hardware structure and communication protocol, significantly increasing the difficulty of implementing smart home service management. Secondly, smart home service management must be combined with user-perceived contextual knowledge. To provide real-time, targeted smart home service management, a connection needs to be established between user contextual knowledge and smart home devices, making the implementation of smart home service management extremely complex.

[0004] Furthermore, during the use of smart home services, each user exhibits personalized and differentiated service needs due to differences in their lifestyle, application scenarios, and climate. These differences make it difficult for pre-set, fixed control rules to meet the personalized service needs of different smart home users and the varying smart home environments. Therefore, how to consider the differentiated needs of different users during the control process and achieve more intelligent and personalized control is also an important issue in the smart home field.

[0005] Existing smart home service management is often based on general programming languages ​​such as C / C++ / Java, and functions are called by directly calling the management interface provided by the corresponding device. Although this programming method can adapt well to different devices, it also has the problem of excessive programming overhead. When dealing with different smart devices, it is necessary to write code specifically for the structure and control interface of different smart devices in order to control the type of device and further complete the collaborative management. At the same time, the strong coupling between management code and device code makes it difficult to migrate the management logic of similar applications to other devices. In addition, most existing smart home device management methods rely on users to operate through terminal devices or pre-set rules to provide services [4]. Although the method of pre-setting rules has achieved a certain degree of intelligence, it is difficult to meet the service needs of different users and different smart home environments. If users manually set the rules, it will increase the threshold for users to use and reduce the user experience of smart home devices. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a smart home service management and control method based on federated learning, to build a personalized management and control model, and to provide users with intelligent and personalized smart home service management and control.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a smart home service management method based on federated learning, comprising the following steps:

[0008] Step 1: To address the heterogeneity issue of smart home devices, a method for constructing a smart home temporal knowledge graph is proposed. This method decouples specific devices from upper-level control logic, unifies the management of smart home device functions in the form of services, and provides support for temporal data on user-perceived environmental changes and service status.

[0009] Step 2: Based on the time-series knowledge graph, provide differentiated and personalized smart home service management for users with different needs in smart home services.

[0010] In a preferred embodiment, step 1 includes constructing a conceptual model of a smart home runtime knowledge graph; the conceptual model of the smart home runtime knowledge graph defines the core concepts of Location, User, Device, Service, and Context; wherein:

[0011] Location refers to a specific area within a smart home, defined as a tuple. <locname>The specific name of the region is represented by a single attribute, LocName.

[0012] In the conceptual model of the smart home knowledge graph, the user (User) represents a smart home service object, using tuples.<UsrName,LocName> This indicates the resident's name and current location information;

[0013] Context refers to the specific environmental type perceived by users within a home environment, expressed through tuples.<CtxType,LocName,CtxVal> Define the Context, which represents the specific type of environment that the user can perceive, the specific location of the environment, and the specific physical quantity of the environment.

[0014] A device refers to a smart device deployed in a smart home environment that provides services to users. Each device instance is represented by a tuple. <DevName,LocName,{Parm1,Parm2,...,Parm n The `DevName` and `LocName` attributes represent the name of the smart device and the name of the smart device's deployment location, respectively; the `Parm` attribute... i This means that the parameters and state of the smart device during operation will change as the state of the smart device changes;

[0015] A service is an abstraction of the functionality provided by a smart device; a service instance is defined as a 5-tuple.<DevName,LocName,CtxType,Effect,State,SvcVal> The DevName attribute represents the provider of the service, i.e., the name of the corresponding smart device; the LocName records the area affected by the service; the CtxType records the environmental state type affected by the service; the Effect attribute refers to the effect of the service on the environmental type specified by CtxType, specifically including four types of effects: (1) Monitor: indicates that the service has the function of monitoring the environmental state and can report the state value of the corresponding environmental state; (2) Increase: indicates that the service can increase the state value of the corresponding environmental state; (3) Reduce: indicates that the service can reduce the state value of the corresponding environmental state; (4) Assign: indicates that the service can maintain the corresponding environmental value at a specified level; the State attribute of the service instance represents the state of the service, including both on and off; the SvcVal attribute is the parameter value of the service operation, such as the specific setting value when setting the service;

[0016] The knowledge graph conceptual model for smart home runtime further defines the corresponding set of concept instances: U = {U1, U2, ..., U...} n Let L = {L1, L2, ..., L} represent the set of all user instances participating in smart home services; n } represents the set of location instances in different areas of a smart home environment; let D = {D1, D2, ..., D} n } represents the set of smart device instances in a smart home environment; let S = {S1, S2, ..., S...} n } represents the set of service instances provided by smart devices; is used to represent C = {C1, C2, ..., C}. n } represents the set of perceptible environmental instances in a smart home environment;

[0017] Define a set CSet containing all concept instances in a smart home scenario:

[0018] CSet = {U, L, S, C, D} # Formula (1)

[0019] The specific definitions of the relationships between the concepts are as follows:

[0020] Relation={Located tn, Sense, Provide, Monitor, Assign, Increase, Reduce}#Formula (2)

[0021] Define the rules for constructing relationships between concept instances. Based on these rules, a smart home knowledge graph can be automatically constructed. When two concept instances satisfy specific preconditions for constructing a relationship, the corresponding relationship instance between the two concept instances can be constructed. When the construction precondition is X... i .LocName = L j .LocName, the relationship is located at When the construction premise is U i ·LocName=C j • LocName, the relationship is perception When the construction premise is (S) i .Effec=Monitor)∧(S i .CtxType = C j .CtxType)∧(S i .LocName = C j .LocName), the relationship is for monitoring When the construction premise is (S) i .Effect=Assign)∧(S i .CtxType = C j .CType)∧(S i .LocName = C j .LocName), the relationship is set. When the construction premise is (S) i Effect = Increase) ∧ (S i .CtxType = C j .CtxType)∧(S i .LocName = C j .LocName), the relationship is to improve When the construction premise is (S) i .Effect=Reduce)∧(S i .CtxType = C j .CtxType)∧(S i .LocName = C j .LocName), the relationship is reduced

[0022] Based on the above construction rules, a series of relations are automatically constructed, and the set of these relations is denoted as RSet:

[0023]

[0024] Based on the definitions of the concept instance set and relation set mentioned above, the smart home knowledge graph is defined as all concept instances and their relations in a smart home scenario, denoted as KG:

[0025] KG =<CSet,RSet> #Formula(4)

[0026] Based on the above definitions, the construction steps for a smart home knowledge graph are as follows:

[0027] Step (1) Construct the corresponding set of location instances L based on the family layout;

[0028] Step (2) Construct a user instance set U based on resident information;

[0029] Step (3) Construct a set of device instances D based on the smart device deployment information, and construct a corresponding set of services S based on the functions they provide;

[0030] Step (4) Construct the environment instance set C based on the location instance set L;

[0031] Step (5) constructs the corresponding location relations for U, C, D, S and L;

[0032] Step (6) Based on the set of environment instances C and the set of service instances S, construct the corresponding influence relationship between the service and the environment, and update the corresponding state value;

[0033] Step (7) Construct the perceptual relationship between the user and the environment based on the user's location information;

[0034] The smart home knowledge graph uses a runtime model to acquire the location information of users and read and transform the status of smart devices. The corresponding runtime operations will be represented by RTModel in the following text.

[0035] In a preferred embodiment, the method for constructing a smart home temporal knowledge graph specifically includes first extending the state of concept instances to the time dimension and defining symbols. Represents a concept instance X i The state at time t, such as This indicates that a certain user U i The state at time t, Represents a certain environment C j The state at time t; then define the set of state instances of each type at time t;

[0036] The set specifically includes the set of position instance states at time t. Assuming that the area information definition in a smart home does not change after initialization, the location information is the same at any time, i.e., L t =L 0 ;

[0037] User instance state set at time t This includes the user's name and location information at time t, where the user's name is set to a fixed value;

[0038] Set of device instance states at time t This includes the device name, device deployment location, and device attribute parameters at time t, where the device attribute parameters change frequently at different times.

[0039] Service instance state set at time t This includes the device name, affected area, action type, and service status information corresponding to the service at time t;

[0040] Set of environment instance states at time t This includes the environment type at time t, the location of environment perception, and the specific physical quantities of the environment, where the environment type is set to a fixed value;

[0041] Based on the definitions of the sets of state instances of each type at time t, aggregating them together yields the set of all concept instance states at time t, CSet. t Defined as:

[0042] CSet t ={U t L t S t C t D t }#Formula (5)

[0043] Based on the relation construction rules described above, the relation instance set RSet at time t can also be constructed on the basis of the concept instance set. t ;

[0044]

[0045] Based on the above temporal definition, the state of the smart home knowledge graph at time t can be obtained through CSet. t and RSet t Definition, denoted as KG t :

[0046] KG t = <CSet t ,RSet t >#Formula(7)

[0047] Finally, the temporal knowledge graph TKG model is obtained, defined as:

[0048] TKG = <KG 1 KG 2 ... KG n >#Formula (8).

[0049] In a preferred embodiment, in step 1, the time-series knowledge graph model needs to persist its state in the database at certain time intervals during operation to record its historical state and corresponding changes; before persisting the data, a state update operation is performed to ensure the real-time nature of the data. The update steps before persisting the state of the time-series knowledge graph at time t include:

[0050] (i) Update the current user location state information U from the runtime model t ;

[0051] (ii) Update the current device instance state information D from the runtime model. t ;

[0052] (iii) Update the corresponding service instance status S based on the current device instance information. t ;

[0053] (iv) Update C based on Monitor type relationship t The CtxVal property corresponding to the environment instance;

[0054] Once the state update is complete, the state data can be stored as a CSet containing the concept instance state data with the current timestamp t. t Relational data RSet t Serialize to the corresponding database;

[0055] Based on the given smart home time-series knowledge graph, the following specific data acquisition operations are defined in the time-series knowledge graph, and the specific meanings of the operations are as follows:

[0056] (I) The operation represents retrieving the state of a concept instance at a specified time t and returning the result.

[0057] (II)Get CSet t The operation represents retrieving the state of all concept instances at time t and returning a set of concept instance states, CSet. t ;

[0058] (III) Get RSet t The operation represents constructing and returning the relations in the knowledge graph at time t, returning the relation set RSet. t ;

[0059] (IV) Get KG t The operation retrieves a snapshot of the knowledge graph at time t, including the set of concept instances and the set of relations, and returns the state of the entire knowledge graph KG. t ;

[0060] (V)GetAfter(X i The operation (t, k) represents retrieving a concept instance X from the time-series knowledge graph, starting from time t, with a time window length of k. i The time-series data is used to obtain the corresponding state sequences from the knowledge graph state sequences.

[0061] (VI)GetBefore(X i The operation (t, k) retrieves a conceptual instance X of a time window of length k up to time t. i Time-series data, extracting the state sequence at a given time point from the knowledge graph state sequence.

[0062] Some operations can be implemented based on a single time-series knowledge graph state slice. For example, operations (I) to (IV), (V), and (VI) rely on multiple consecutive knowledge graph state slices to obtain state data sequences. In particular, when the stored data does not contain data at the time point t specified for a given time-series operation, the data at the nearest time point no earlier than t will be returned. The smart home time-series knowledge graph model provides the knowledge graph KG at time t. t This includes the state set CSet of each concept instance at time t. t and the relationships between various concept instances RSet t And the sequence of state changes of concept instances within a specified time period.

[0063] In a preferred embodiment, in step 2, to obtain the changes in the environmental state around the user over a period of time in the smart home temporal knowledge graph model, the user's location information U is first obtained. i .LName, according to the Sense relation construction rules, satisfies condition C. j ·LName=U i The Context instance of LName represents the environment that the user can perceive, denoted as the user-sensitive environment. Secondly, C is obtained through time-series knowledge graphs. j Recent historical state data, which refers to changes in the user's surrounding environment over a period of time, is defined as:

[0064]

[0065] Where t is the current time and k is the time window length; based on the operation definition of the time series knowledge graph. Equivalent to environment state instance C j The state data sequence from time t back k time points, obtained through GetBefore(C j The operation (t, k) is obtained;

[0066] In the smart home temporal knowledge graph model, regulating the environmental state perceived by the user is equivalent to regulating the environmental C that is sensitive to the user. j Controlling the relevant functions of the smart devices that have an impact; in the temporal knowledge graph, the operation of a certain function of a smart device is abstracted as the manipulation of service instance objects, which can affect the environment C. j The service instance is the one that exists in the relationship. Service Instance S k The current state of the smart device is the service instance S. k Current status User operations on smart device functions can be represented as service S. k The state at the next moment The input to the service control rules is defined as the time-series data History(C) of the user's sensitive environmental state. j (t, k) and the current state of sensitive service instances The output is defined as user-sensitive service instance S. k State decision at the next moment

[0067] Based on the smart home time-series knowledge graph and the rule definition, the control of smart home devices is transformed into the control of specified types of services. To enable users to control the different functions of multiple smart home devices in their homes, many different rules need to work together; it is necessary to build exclusive control rules for each user based on their behavior history.

[0068] In a preferred embodiment, in step 2, the smart home service management model based on neural networks uses a multi-hidden-layer deep neural network (DNN) model to learn user habits and generate corresponding management rules.

[0069] It is necessary to collect user behavior data on smart devices at different times, locations, and scenarios. In the smart home temporal knowledge graph, this information is represented as user location information, the user's perceived environmental state, and the state of the smart devices. Based on this information, personalized management rules can be built for each user; this is achieved through the input layer L. in Three fully connected hidden layers H1, H2, H3 and an output layer L out The network is composed of several layers; the number of neurons in the input layer depends on the time window length k, i.e., the environmental state changes and device state data within each time window are used as input; the number of neurons in the three hidden layers are 32, 64, and 32, respectively; a fully connected structure is used between the layers; the Sigmoid activation function is used for nonlinear transformation; assuming L... in The input to each layer up to H3 is X i Then its corresponding output is L i :

[0070] L i =Sigmoid(X) i ·W i +B i )#Formula (10)

[0071] The Sigmoid function is defined as follows:

[0072]

[0073] The model uses the Adam optimization algorithm to update the gradient, and the loss function used is the mean squared error (MSE), which is defined as:

[0074]

[0075] The weight matrices W1, W2, W3, W obtained after model training out And bias matrices B1, B2, B3, B out The neural network is initialized accordingly; after multiple iterations, the weight matrices W1, W2, W3, and W4 can be obtained. out and bias vectors B1, B2, B3, B out The value is saved as the user's personalized model;

[0076] Supported by a temporal knowledge graph, we collect the time-series data of user-sensitive environmental states for the first k time points: History(C j Given the time series knowledge graph (t, k), obtain its corresponding CValue input model, obtain the decision result SValue regarding the service state, and output it to the time series knowledge graph. By leveraging its control capabilities, it can issue control commands to corresponding smart home devices, thereby enabling the management and control of corresponding device services and achieving a seamless and intelligent smart home user experience.

[0077] In a preferred embodiment, in step 2, the smart home service management method based on federated learning consists of a global server GS and N local households; for each Service type, a corresponding public prediction model M is established. i Each user uses the same model structure and trains their own local model based on their own data. Collaborative model training and user privacy protection are achieved by sharing model parameters rather than private data between the GS and local households. For newly joined households, there is initially no local data or very little local data, so directly training the model will not yield good results. Therefore, after the smart home temporal knowledge graph is established, the data pre-collection stage is first entered. In this stage, only user behavior data is collected, and the control functions of smart home devices are not enabled. After a small amount of data is collected, personalized model training is carried out based on the collected data.

[0078] During multi-family collaborative training, local families first download a public model from the global server as an initial model and train it using local data. After a fixed number of training rounds, the model communicates with the global server to exchange model parameters, repeating this process until the model converges, resulting in a usable local personalized model. At this point, control of smart home devices begins, entering the management and control phase. During the management and control phase, user behavior data is still collected, and after a certain amount of incremental data is collected, a background process is started to retrain the model to ensure that the model can adapt to possible changes in user habits. First, the existing public model is downloaded from GS, and then the model is updated using a small amount of locally collected data to learn user personalized preferences.

[0079] The specific process of training the model collaboratively between local households and global servers based on the FedAvg federated averaging algorithm is as follows:

[0080] Initialize the parameters M of the service management model for each type in the global server. G and client parameter list Φ;

[0081] Determine the current number of participating households, N;

[0082] Initialize the parameters M of the local service management model for each household. i ;

[0083] for communication episode=1, Max communication episodes do;

[0084] for client in available clients in parallel do

[0085] Perform home-based local model training

[0086] end

[0087] Each household uploads local model parameters M i = <W1,W2,W3,W out B1, B2, B3, Bo ut >To the global server;

[0088] Save the parameters of each family model to Φ = (M1, M2, ..., M n );

[0089] The new global model parameters are derived using the FedAvg algorithm:

[0090] Replace the old global model parameters with the new global model parameters: M G ←M′ G ;

[0091] Broadcast the new global model parameters to all households;

[0092] Update the local model parameters of each household to the global model parameters: M i ←M G ;

[0093] end

[0094] For new families that join midway, the existing global model parameters M are used first. G The model is initialized using its local model, and a certain amount of local data is collected before being added to the above collaborative training process.

[0095] Compared with the prior art, the present invention has the following advantages: The present invention only needs to provide basic contextual knowledge of the construction scenario, as well as the relationship between conceptual instances and smart devices, and collect a small amount of user behavior data to build a personalized control model, providing users with intelligent and personalized smart home service control. Attached Figure Description

[0096] Figure 1 This is a conceptual model of a smart home knowledge graph, as preferred by the present invention.

[0097] Figure 2 This is a time-series knowledge graph model, a preferred embodiment of the present invention.

[0098] Figure 3 The service management rules are those of a preferred embodiment of the present invention;

[0099] Figure 4 This is a preferred embodiment of the DNN network structure of the present invention;

[0100] Figure 5 This is the federated learning training process of a preferred embodiment of the present invention;

[0101] Figure 6 This is a prototype of a smart home scene according to a preferred embodiment of the present invention;

[0102] Figure 7 This is a smart home timing knowledge graph according to a preferred embodiment of the present invention;

[0103] Figure 8 A comparison of the accuracy of temperature-related service control in a preferred embodiment of the present invention;

[0104] Figure 9 The accuracy of humidity-related service management in the preferred embodiment of the present invention;

[0105] Figure 10 The accuracy change curve of adding new families to federal collaborative training in a preferred embodiment of the present invention;

[0106] Figure 11 The curve showing the change of the loss function when adding a new family to the federal collaborative training in a preferred embodiment of the present invention. Detailed Implementation

[0107] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0108] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0109] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations according to this application; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0110] A method for managing smart home services based on federated learning, referencing Figures 1 to 11 This method consists of two parts.

[0111] The first part addresses the heterogeneity issue of smart home devices by proposing a method for constructing a smart home temporal knowledge graph. This method decouples specific devices from upper-level control logic, unifies the management of smart home device functions in the form of services, and provides support for temporal data on user-perceived environmental changes and service status.

[0112] The second part addresses the differentiated needs of various users in smart home services by providing differentiated and personalized smart home service management based on a time-series knowledge graph. Both of these methods only require basic contextual knowledge of the constructed scenario, the relationships between conceptual instances and smart devices, and the collection of a small amount of user behavior data to build a personalized management model and provide users with intelligent and personalized smart home service management.

[0113] 1.1 Construction Method of Smart Home Temporal Knowledge Graph

[0114] 1.1.1 Conceptual Model of Knowledge Graph in Smart Home Operation

[0115] The conceptual model of the smart home temporal knowledge graph defines core concepts such as Location, User, Device, Service, and Context. Specific examples are defined in Table 1.

[0116] Table 1. Definition of Knowledge Graph Concepts and Examples

[0117]

[0118] These concepts, such as Figure 1 The nodes in the diagram represent the following:

[0119] • Location refers to a specific area within a smart home, such as the living room, kitchen, or bedroom, and is defined as a tuple. <locname>Use a single attribute, LocName, to represent the specific name of the region.

[0120] In the smart home knowledge graph conceptual model, a "User" represents a smart home service object, such as a specific household. This is represented using tuples.<UsrName,LocName> This indicates the resident's name and current location information.

[0121] • Context refers to specific environmental types that can be perceived by users within a home environment, such as brightness and temperature. This invention utilizes tuples...<CtxType,LocName,CtxVal> Define the Context, which represents the specific environment type that the user can perceive, the specific location of the environment, and the specific physical quantity of the environment.

[0122] • A device refers to a smart device deployed in a smart home environment that can provide services to users. Each device instance is represented by a tuple. <DevName,LocName,{Parm1,Parm2,...,Parm n The `DevName` and `LocName` attributes represent the smart device name and the deployment location name, respectively. The `Parm` attribute... i This indicates that the parameters and state of the smart device during operation will change as the state of the smart device changes.

[0123] A service is an abstraction of the functionality provided by a smart device. A service instance is defined as a 5-tuple.<DevName,LocName,CtxType,EffecL,State,SvcVal> The DevName attribute represents the provider of the service, i.e., the name of the corresponding smart device. LocName records the area affected by the service. CtxType records the environmental state type affected by the service. The Effect attribute refers to the effect of the service on the environmental type specified by CtxType, specifically including four types of effects: (1) Monitor: indicates that the service has the function of monitoring the environmental state and can report the state value of the corresponding environmental state; (2) Increase: indicates that the service can increase the state value of the corresponding environmental state; (3) Reduce: indicates that the service can reduce the state value of the corresponding environmental state; (4) Assign: indicates that the service can maintain the corresponding environmental value at a specified level. The State attribute of the service instance represents the state of the service, including both on and off. The SvcVal attribute is the parameter value of the service operation, such as the specific setting value when setting the service.

[0124] Based on the above definition of concept instances, the model further defines the corresponding set of concept instances to better represent contextual knowledge in smart homes, as shown in Table 2:

[0125] Table 2 Collection of Concept Examples

[0126]

[0127] Based on these sets of concept instances, define a set CSet of all concept instances in the smart home scenario:

[0128] CSet = {U, L, S, C, D} # Formula (1)

[0129] In addition to the knowledge graph instance set, the knowledge graph conceptual model also defines the relationships between the above concepts to describe the connections between entities in a smart home scenario. Figure 1 The arrow in the middle is represented by a labeled arrow. The specific definitions of the relationships between concepts are as follows:

[0130] Relation={Located tn, Sense, Provide, Monitor, Assign, Increase, Reduce}#Formula (2)

[0131] Based on the basic abstraction of smart home scenarios provided by the smart home knowledge graph conceptual model, this section introduces how to automatically construct a smart home knowledge graph to maintain synchronization between the smart home knowledge graph and the real world. Specifically, building on the concept definitions in the previous section, this section defines the rules for constructing relationships between concept instances. Based on these rules, the smart home knowledge graph can be automatically constructed. As shown in Table 3, when two concept instances satisfy specific preconditions for relationship construction, the corresponding relationship instance between the two concept instances can be constructed.

[0132] Table 3. Relationship Construction Rules of Smart Home Knowledge Graph

[0133]

[0134] Based on the above construction rules, a series of relations can be automatically constructed, and the set of these relations is denoted as RSet:

[0135]

[0136] Based on the definitions of the concept instance set and relation set mentioned above, the smart home knowledge graph is defined as all concept instances and their relations in a smart home scenario, denoted as KG:

[0137] KG =<CSet,RSet> #Formula(4-13)

[0138] Based on the above definitions, the construction steps for a smart home knowledge graph are as follows:

[0139] (1) Construct a set of corresponding location instances L based on the family layout;

[0140] (2) Construct a user instance set U based on resident information;

[0141] (3) Construct a set of device instances D based on the deployment information of smart devices, and construct a corresponding set of services S based on the functions they provide;

[0142] (4) Construct the environment instance set C based on the location instance set L;

[0143] (5) Construct the corresponding location relations for U, C, D, S and L;

[0144] (6) Based on the set of environment instances C and the set of service instances S, construct the corresponding influence relationship between the service and the environment, and update the corresponding state value;

[0145] (7) Construct the perception relationship between the user and the environment based on the user's location information.

[0146] Smart home knowledge graph through runtime model [24，71] This is to achieve the acquisition of user location information and the reading and operation transformation of smart device status. The corresponding runtime operations will be represented by RTModel. The smart home knowledge graph defines a series of operation transformation rules to maintain bidirectional synchronization between the conceptual model in the knowledge graph and the real world. Due to space limitations, this invention only provides a brief introduction to the relevant content and does not supplement this part. For details, please refer to the relevant literature.

[72] The operations on concept instances include three types: List, Get, and Set. The List operation lists the corresponding collection of instances and their properties; the Get operation retrieves the value of a specific property of a concept instance; and the Set operation sets the value of a specific property of an instance.

[0147] 1.1.2 Construction Method of Smart Home Temporal Knowledge Graph

[0148] To record historical changes in the smart home environment and users' historical behavior data, and to provide data support for building a smart home service management and control model later, this section introduces the time dimension into the smart home runtime knowledge graph, constructs a smart home temporal knowledge graph, and gives the definition and basic operation of the smart home temporal knowledge graph, providing a foundation for building a smart home management and control model later.

[0149] Figure 2 This invention showcases the temporal knowledge graph model proposed in this paper, which extends the traditional smart home knowledge graph on a temporal scale. While describing the concepts and relationships involved in personalized smart home solutions, the temporal knowledge graph, by recording the state information of the smart home knowledge graph at different points in time, represents the process of change in concept instances and relationships over a certain period. A temporal knowledge graph can be understood as a knowledge graph stream with time labels.

[73] That is, the knowledge graph at each time point T1, T2..., T within a given time period. n The sequence of state data, where the state data at each time point includes knowledge graph concept instances, relation instances, and a specific time label T at that time point. k .

[0150] To construct the aforementioned temporal knowledge graph, we first need to extend the state of concept instances to the time dimension and define symbols. Represents a concept instance X i The state at time t, such as This indicates that a certain user U i The state at time t, Represents a certain environment C j The state at time t. The set of state instances of each type at time t is then defined, as shown in Table 4.

[0151] Table 4. Set of state instances of each type at time t

[0152]

[0153] The specific meanings of each instance set in the time-series knowledge graph are as follows:

[0154] · Let L represent the set of location instance states at time t. Specifically, this invention assumes that the area information definition in a smart home remains unchanged after initialization; therefore, the location information at any given time is the same, i.e., L. t =L 0 .

[0155] · This represents the set of user instance states at time t, including the user's name and location information at time t, where the user name is set to a fixed value.

[0156] · This represents the set of device instance states at time t, including the device name, device deployment location, and device attribute parameters at time t. The device attribute parameters change frequently at different times.

[0157] · This represents the set of service instance states at time t, including information such as the device name, affected area, function type, and service status corresponding to the service at time t.

[0158] · This represents the set of environmental instance states at time t, including the environmental type, the location of environmental perception, and the specific physical quantities of the environment at time t, where the environmental type is set to a fixed value.

[0159] Based on the definitions of the sets of state instances of each type at time t, aggregating them together yields the set of all concept instance states at time t, CSet. t Defined as:

[0160] CSet t ={U t L t S t C t D t }#Formula (5)

[0161] Based on the relation construction rules described above, the relation instance set RSet at time t can also be constructed on the basis of the concept instance set. t .

[0162]

[0163] Based on the above temporal definition, the state of the smart home knowledge graph at time t can be obtained through CSet. t and RSet t Definition, denoted as KG t :

[0164] KG t = <CSet t ,RSet t >#Formula(7)

[0165] Finally, the Temporal Knowledge Graph (TKG) model is obtained, defined as follows:

[0166] TKG = <KG 1 KG 2 ... KG n >#Formula(8)

[0167] 1.1.3 Temporal Modeling Method for Smart Home Knowledge Graph

[0168] In the smart home knowledge graph model, the state of concept instances changes dynamically over time. Therefore, the temporal knowledge graph model needs to persist its state in the database at regular intervals during operation to record its historical state and corresponding changes. Before persisting the data, a state update operation is performed to ensure data real-time performance. The update steps before persisting the temporal knowledge graph state at time t include:

[0169] (1) Update the current user location status information U from the runtime model. t ;

[0170] (2) Update the current device instance state information D from the runtime model. t ;

[0171] (3) Update the corresponding service instance status S based on the current device instance information. t ;

[0172] (4) Update C based on Monitor type relationship t The CtxVal property of the corresponding environment instance.

[0173] Once the state update is complete, the state data can be stored as a CSet containing the concept instance state data with the current timestamp t. t Relational data RSet t Serialize to the corresponding database.

[0174] Based on the given smart home time-series knowledge graph, this invention also defines the following specific data acquisition operations within the time-series knowledge graph, as shown in Table 5. The specific meanings of these operations are as follows:

[0175] (1) The operation represents retrieving the state of a concept instance at a specified time t and returning the result.

[0176] (2) Get CSet t The operation represents retrieving the state of all concept instances at time t and returning a set of concept instance states, CSet. t ;

[0177] (3)GetRSet t The operation represents constructing and returning the relations in the knowledge graph at time t, returning the relation set RSet. t ;

[0178] (4) Get KG t The operation retrieves a snapshot of the knowledge graph at time t, including the set of concept instances and the set of relations, and returns the state of the entire knowledge graph KG. t ;

[0179] (5) GetAfter(X) i The operation (t, k) represents retrieving a concept instance x from the time-series knowledge graph, starting from time t, with a time window length of k. i The time-series data is used to obtain the corresponding state sequences from the knowledge graph state sequences.

[0180] (6) Get Before(X) i The operation (t, k) retrieves a conceptual instance X of a time window of length k up to time t. i Time-series data, extracting the state sequence at a given time point from the knowledge graph state sequence.

[0181] Table 5 Operational Rules for Time Series Knowledge Graph

[0182]

[0183] Some operations can be implemented based on a single knowledge graph state slice at a single time point. For example, operations (1) to (4), (5), and (6) rely on multiple consecutive knowledge graph state slices to obtain the state data sequence. Specifically, if the stored data does not contain data at the time point t specified by a given time-series operation, the data at the nearest time point no earlier than t will be returned. Based on the above definitions, the smart home time-series knowledge graph model can provide the knowledge graph KG at time t. t This includes the state set CSet of each concept instance at time t. t and the relationships between various concept instances RSet t And the sequence of state changes of concept instances within a specified time period.

[0184] 1.2 Smart Home Service Management Method Based on Federated Learning

[0185] 1.2.1 Smart Home Service Management Rules Based on Temporal Knowledge Graph

[0186] The intelligent and automated management and control of smart home devices is a crucial capability of smart home services. Smart home service management involves collecting, analyzing, making decisions, and implementing measures based on various information about the user's environment to meet their needs for adjustments to sensitive environmental conditions.

[0187] During the use of smart home devices, users' operational decisions are influenced by changes in the surrounding environment and the current status of the devices over a period of time. [34，36] Therefore, the management and control of smart home services for users should be based on changes in the user's surrounding environment and the current status of smart devices.

[30] The goal is to regulate the environment as perceived by the user to meet their service needs. Currently, smart home service management technology remains at the stage of control based on expert-preset rules. However, rigid logic control struggles to adapt to changing home environments and the personalized needs of different users, often requiring manual rule adjustments in practice. These cumbersome rule adjustments increase the complexity and learning cost of device use, significantly reducing the user experience. Furthermore, each person's control needs for smart devices differ, and generic rules often cannot fully meet everyone's requirements. Rules built for users must be tailored to their individual needs. To create intelligent and personalized management rules that align with user habits, learning user decision preferences from their past decision-making history is a viable solution.

[0188] In the smart home temporal knowledge graph model, to obtain the changes in the environmental state around a user over a period of time, the user's location information U must first be obtained. i .LName, according to the Sense relation construction rules, satisfies condition C. j .LName=U i The Context instance of .LName represents the environment that the user can perceive, denoted as the user-sensitive environment. Secondly, C is obtained through time-series knowledge graphs. j Recent historical state data, which refers to changes in the user's surrounding environment over a period of time, is defined as:

[0189]

[0190] Where t is the current time and k is the time window length. According to the definition of time-series knowledge graph operations, Equivalent to environment state instance C j The state data sequence from time t back k time points can be obtained using GetBefore(C j The operation is obtained by (t, k).

[0191] In the smart home temporal knowledge graph model, regulating the environmental state perceived by the user is equivalent to regulating the environmental C that is sensitive to the user. j The relevant functions of the smart devices that have an impact are controlled. In the temporal knowledge graph, the operation of a certain function of a smart device is abstracted as the manipulation of service instance objects, which can affect the environment C. j The service instance is the one that has a relationship. Service Instance S k The current state of the smart device is service instance S. k Current status User interactions with smart device functions can be represented as service S. k The state at the next moment Therefore, this invention defines the input to service control rules as the time-series data History(C) of the user's sensitive environmental state. j (t, k) and the current state of sensitive service instances The output is defined as user-sensitive service instance S. k Next time step state decision

[0192] Based on the smart home temporal knowledge graph and this rule definition, the control of smart home devices is transformed into the control of specified types of services. Controlling different functions of multiple smart home devices in a user's home requires the collaboration of many different rules. To provide personalized service control that aligns with user habits, it is necessary to build exclusive control rules for each user based on their behavior history.

[0193] 1.2.2 Neural Network-Based Smart Home Service Management and Control Model

[0194] Based on the preceding definitions, this invention proposes a neural network-based smart home service management model. It employs a multi-hidden-layer deep neural network (DNN) model to learn user habits and generate corresponding management rules. Compared to traditional methods that rely on manually setting fixed rules, this method better adapts to changes in user behavior and the environment, enabling dynamic adjustment and optimization of smart device functions. The neural network-based model can analyze and predict personalized user information through deep learning, thereby providing more intelligent and user-friendly smart home services.

[0195] To train this neural network model, it is necessary to collect user behavior data on smart devices used at different times, locations, and scenarios. In the smart home temporal knowledge graph, this information is represented as user location information, the user's perceived environmental state, and the state of the smart devices. Based on this information, personalized management rules can be constructed for each user. According to the definition above, its network structure is as follows: Figure 4 As shown. This network consists of an input layer L in Three fully connected hidden layers H1, H2, H3 and an output layer L out Composition. The number of neurons in the input layer depends on the time window length k, i.e., the environmental state changes and device state data within each time window are used as input. The number of neurons in the three hidden layers are 32, 64, and 32, respectively. A fully connected structure is used between each layer. This invention uses the Sigmoid activation function for nonlinear transformation. Assume L... in The input to each layer up to H3 is X i Then its corresponding output is L i :

[0196] L i =Sigmoid(X) i ·W i +B i )#Formula (10)

[0197] The Sigmoid function is defined as follows:

[0198]

[0199] The model uses the Adam optimization algorithm to update the gradient, and the loss function used is the mean squared error (MSE), which is defined as:

[0200]

[0201] The weight matrices W1, W2, W3, W obtained after model training out And bias matrices B1, B2, B3, B out The neural network is initialized accordingly. The main process of training the model is shown in Algorithm 1:

[0202] After multiple iterations, the weight matrices W1, W2, W3, and W can be obtained. out and bias vectors B1, B2, B3, B out The value is saved as the user's personalized model.

[0203] Supported by a temporal knowledge graph, we collect the time-series data of user-sensitive environmental states for the first k time points: History(C j Given the time series knowledge graph (t, k), obtain its corresponding CValue input model, obtain the decision result SValue regarding the service state, and output it to the time series knowledge graph. By leveraging its control capabilities, it can issue control commands to corresponding smart home devices, thereby enabling the management and control of corresponding device services and achieving a seamless and intelligent smart home user experience.

[0204]

[0205]

[0206] 1.2.3 Smart Home Service Management Method Based on Federated Learning

[0207] To address the challenges of personalized smart home models in multi-family environments, this invention further proposes a multi-family smart home service management method based on federated learning.

[0208] like Figure 5 As shown, this method consists of a global server GS and N local households. For each Service type, a corresponding common prediction model M is established. i Each user uses the same model structure and trains their own local model based on their own data.

[0209] By sharing model parameters between the smart home ecosystem (GS) and local households instead of private data, collaborative model training and user privacy protection were achieved. For newly added households, initially there is little or no local data, and directly training the model will not yield good results. Therefore, after the smart home temporal knowledge graph is established, the first step is a data pre-collection phase. In this phase, only user behavior data is collected, and the control functions of smart home devices are not enabled. After collecting a small amount of data, personalized model training can be performed based on the collected data.

[0210] In the multi-family collaborative training process, local families first download a public model from the global server as an initial model and train it using local data. After a fixed number of training rounds, the model communicates with the global server to exchange model parameters, repeating this process until the model converges, resulting in a usable local personalized model. At this point, control of smart home devices begins, entering the management phase. Although the user's personalized model has been built, users' personal habits may subtly change with seasons, moods, or changes in smart devices. Therefore, during the management phase, user behavior data is still collected, and after collecting a certain amount of incremental data, a background process is started to retrain the model to ensure that the model can adapt to possible changes in user habits. Therefore, this method first downloads the existing public model from GS, and then updates the model using a small amount of locally collected data to learn user personalized preferences and improve the accuracy of service management with limited data.

[0211] This method is based on the FedAvg algorithm. The specific process of local households and global servers cooperating to train the model is shown in Algorithm 2:

[0212]

[0213]

[0214] For new families that join midway, this method first uses the existing global model parameters M. G The model is initialized using its local model, and a certain amount of local data is collected before being incorporated into the collaborative training process described above. Based on this method, high accuracy can be achieved with only a small amount of local data, and the model can achieve convergence in a relatively short number of epochs.

[0215] 1.3 Experimental Evaluation

[0216] 1.3.1 Experimental Setup

[0217] This invention constructs a prototype system based on a real-world environment and evaluates the proposed method. The experimental prototype system considers five users from five households, each with different preferences. The experiment constructs three typical Context types for users: temperature, humidity, and brightness. Smart devices in the smart home scenario can provide Monitor, Increase, and Reduce services to the corresponding Context. The smart home device settings for each household are shown in Table 4. Based on the characteristics of the smart home temporal knowledge graph, the functional operations of different smart home devices are abstracted into service instances, enabling unified data reading, writing, and function invocation for the devices. The DNN network used in the experiment includes an input layer L... in An output layer L out The model consists of three hidden layers: L1, L2, and L3, all connected by a fully connected network. The number of neurons in the three hidden layers are 32, 64, and 32, respectively. The batch size during training is 32, and the Sigmoid activation function and Adam optimization algorithm are used.

[0218] Table 4: Smart Devices Owned by Each Household

[0219]

[0220]

[0221] After deploying five family models, each family first obtains the global model from the global server and updates the model using local data. In this experiment, after every 100 iterations of the local model, communication occurs between the local model and the global server to exchange model parameters. Once the local model converges, a personalized model tailored to the user's individual habits is obtained for service management. The data used in the experiment is simulation data generated for each family's users based on real-world data. Each family has different preferences, and the time interval between each data point is 15 minutes. The experimental code is written in Python 3.9 and runs on a personal computer with an Intel(R) Core(TM) i5-9500 CPU@3.00GHz, 16.0GB of RAM, and a Windows 10 operating system.

[0222] 1.3.2 Case Study of Control Measures

[0223] To verify how this method manages smart homes, this experiment uses Household 1 as an example to demonstrate the construction process of the temporal knowledge graph and the service management process. The smart home scenario of Household 1 is as follows: Figure 6 As shown, a household has a user U1 who owns smart air conditioners, smart humidifiers, and smart lights. In this scenario, this experiment constructs three types of Contexts: temperature, humidity, and brightness. Each smart device in the scenario can provide environmental adjustment services of types Monitor, Increase, and Reduce to its corresponding Context.

[0224] Step 1: Construct a knowledge graph based on scenario knowledge.

[0225] First, based on Figure 6 The smart home scenario prototype shown generates a smart home temporal knowledge graph instance containing 1 User instance, 1 Location instance, 3 Device instances, 3 Context instances, and 10 Service instances. Next, according to the relationship construction rules in Section 4.1.2, the corresponding relationships between the above relationship instances are constructed, including: 17 LocatedIn relationships, 10 Provide relationships, 3 Monitor relationships, 2 Reduce relationships, 3 Increase relationships, 1 Assign relationship, and 3 Sense relationships.

[0226] Step 2: Data pre-collection and model training.

[0227] After the smart home temporal knowledge graph is established, the first stage is data pre-collection. This stage only collects user behavior data; corresponding control functions are not yet enabled. It's important to note that data collection requires the user to be within the smart home service control area and be active; situations where the user is away or asleep are excluded. During this period, the temporal knowledge graph retrieves the state data of all current instances every fifteen minutes, generating a "snapshot" of the current knowledge graph to obtain environmental instances. Taking the state as an example, the data acquisition process is introduced: Execute "Get C 1_1 .CType→C 1_1 ".CType", "Get C" 1_1 .LName→C 1_1 ".LName" retrieves the environment state type and location information of the environment instance; execute That is, by obtaining the current CValue state value through the corresponding smart device, the instance state is obtained as C. 1_1 =<LName=bedroom,CType=Temperature,CValue=30℃> The instance state will be saved to the database along with the current timestamp. Specifically, relationships between concept instances can be generated based on relationship construction rules and therefore are not saved in the database.

[0228] Once 200 user behavior data points have been collected, a personalized model will be trained based on the collected data. During training, a public model is first downloaded from the global server as the initial model, and then trained using local data. After every 100 epochs of training, the model communicates with the global server to exchange model parameters, repeating this process until the model converges, resulting in a usable local personalized model. At this point, control of smart home devices will begin, entering the management and control phase. During the management and control phase, data will continue to be collected, and after a certain amount of incremental data is collected, a background process will be started to retrain the model to ensure that the model can adapt to possible changes in user habits.

[0229] Step 3: The process of controlling smart home devices using a local model:

[0230] Taking temperature environment status related service management as an example, user Jack (U1) enters bedroom (L1) at time t, and the temperature in L1 is relatively high. First, update Jack's (U1) location information by executing "Get U1.LName→RTModel(Get T1.location)", and construct a relation instance based on "U1.LName=L1.LName". According to "U1.LName=C 11 .LName constructs a relation instance User-sensitive environment instance is C 11 =<LName=bedroom,CType=Temperature,CValue=30℃> Secondly, based on relational instances Obtaining the ability to reduce C 1_1 Service instance S of status value 1_3 =<DName=D1,LName=bedroom,CType=Temperature,Effect=reduce,Status=off,SValue=0> C is obtained through the GetBefore operation of the time-series knowledge graph. 1_1 Temperature history data History(C) 1_1 ,t,6)=<27.1℃,27.6℃,27.6℃,28.2℃,28.2℃,28.5℃>, and compare it with S 1_3 The state is used as the input to the personalized model corresponding to U1 to obtain S. 1_3 The service configuration parameter SValue = 26, and S is set through a time-series knowledge graph. 1_3 Service parameters: Set S 1_3 .Status = on, Set S 1_3 SValue = 26. Finally, the time-series knowledge graph will analyze service S. 1_ 3. Changes in Svalue are mapped to the device. In terms of operation: Set D1.Key1 = on, Set D1.Key2 = 26 to complete the automatic control of smart device D1, i.e., smart air conditioner.

[0231] 1.3.3 Service Control Accuracy Analysis

[0232] This invention analyzes the service control accuracy of the proposed model and compares it with the following comparative methods:

[0233] 1) Rule-based management model: This method involves manually setting unified service control rules.

[0234] 2) Local-only DNN management model trained on local data: This method uses only a small amount of local data to train a local DNN model for service management. The main difference between this method and the method of this invention is the lack of a process of training the model in collaboration with other households; all model parameters are trained from local data.

[0235] 3) Global-only DNN management model based on federated global model: This method directly uses a shared DNN model trained based on other families in a federated collaboration. The main difference between this method and the method of this invention is that it lacks the process of updating the model using local data.

[0236] Table 5 Comparison of accuracy rates of four comparison methods in temperature-related service management

[0237]

[0238] Figure 8 The graph shows the accuracy of temperature-related service management across households with different preferences. As can be seen, the average accuracy of the management model based on preset rules is low, at only 33.88%, indicating that this method cannot meet users' service management needs. Furthermore, this method shows significant differences across households with different preferences, with the best and worst results differing by more than 15%. This is mainly because the method uses uniform rules and cannot adapt to the personalized needs of different households, although it may perform well for some. The average accuracy of the DNN management model trained on local data is 68.67%, primarily because the local model training relies on only a small amount of local data (200 records), which is insufficient for training the neural network model. The average accuracy of the DNN management model based on a federated global model is 68.66%, which is still relatively low. This is because although this model uses federated learning to learn some generalization ability from different households, it does not include local personalized data and cannot address specific user preferences, thus resulting in low accuracy.

[0239] In comparison, the method proposed in this invention achieves an average accuracy of 89.47%, and performs best among households with varying preferences. This method addresses the issue of insufficient data by using federated learning and updates the model with local data, selectively learning a management model that aligns with local user habits. Compared to other methods, it can meet the management needs of different households.

[0240] Table 6 Comparison of accuracy rates of four comparison methods in humidity-related service management

[0241]

[0242] Figure 9 The accuracy of humidity-related service management is demonstrated for households with different preferences. The average accuracy of the management model based on preset rules is slightly higher than that of temperature-related services at 47.63%, but it still lags significantly behind the method of this invention. Similar to the problem encountered in the temperature scenario, this method also shows significant differences among households with different preferences, thus it cannot meet the needs of different households in the humidity scenario. The average accuracy of the DNN management model trained on local data is 61.37%, which is about 20% lower than the method of this invention. The average accuracy of the DNN management model based on the federated global model is 74.23%, slightly higher than that of the DNN management model trained on local data, but it still lags behind the method of this invention by about 15%. The method of this invention achieves an average accuracy of 87.04% in humidity-related services, reaching the best performance across all households, with an accuracy difference of about 5% between different households, indicating that this method can meet the management needs of different households for different services.

[0243] 1.3.4 Model Convergence Rate Analysis

[0244] To evaluate the benefits of federated learning on model convergence speed for newly deployed households, this experiment simulates a scenario where newly deployed households join an existing federated cluster in a real-world application. In this experiment, households 1, 2, and 4 collaboratively train a shared model with communication intervals of one communication every 100 rounds. After three rounds of communication, the globally shared model reaches a basically usable state. At this point, household 5 completes data pre-collection and begins joining the federated collaborative training; its corresponding training accuracy change curve is shown in the figure. Figure 10 As shown in the figure, compared to the three families requiring approximately 400 rounds of training from scratch to achieve around 80% accuracy, the newly added family only needs about 70 rounds to achieve the same result, demonstrating a significant improvement in model convergence speed. This indicates that federated learning can quickly find the optimal strategy for newly added models and has the ability to scale to a larger number of families. Furthermore, the figure shows that when the model first joins the collaborative training, its accuracy is close to that of the globally shared model. Lacking the habitual information of family 5, its accuracy on family 5 is lower. However, after updating with local data, the model can achieve an accuracy of approximately 90%.

[0245] Figure 11 The diagram illustrates the convergence curve changes of the model with newly added households under temperature scenarios. The convergence speed of the model with newly added households is significantly faster compared to training from scratch, achieving a lower loss function value with fewer training epochs. This demonstrates that federated learning can significantly accelerate the convergence speed of the model during the training process of newly added households, which will greatly reduce the computational resources and communication overhead required when adding new households.

[0246] 2. Summary

[0247] With the continuous development of smart home technologies, more and more smart devices are appearing in people's daily lives. However, many heterogeneous devices and the services they provide often cannot communicate with each other, making it difficult for many smart home devices to work together and posing significant challenges to the management and control of smart home services. Furthermore, beyond basic smart home device management, providing users with services that better suit their habits without increasing their burden remains a difficult problem. To address these issues, this invention, after reviewing existing domestic and international work, found that current research still cannot adequately support intelligent and personalized smart home service management and control. To solve these problems, this invention applies knowledge graphs and federated learning to smart home service management and control, proposing a spatiotemporal data-driven method for smart home service management and control, providing users with an intelligent and personalized smart home service experience.< / locname> < / locname>

Claims

1. A smart home service management method based on federated learning, characterized in that, Includes the following steps: Step 1: To address the heterogeneity issue of smart home devices, a method for constructing a smart home temporal knowledge graph is proposed. This method decouples specific devices from upper-level control logic, unifies the management of smart home device functions in the form of services, and provides support for time-series data on user-perceived environmental changes and service status. Step 2: Based on the time-series knowledge graph, provide differentiated and personalized smart home service management for users with different needs in smart home services. In step 2, the smart home service management method based on federated learning consists of a global server GS and N local households; for each service type, a corresponding common prediction model is established. Each user uses the same model structure and trains their own local model based on their own data. Collaborative model training and user privacy protection are achieved by sharing model parameters rather than private data between the GS and local households. For newly joined households, there is initially no local data or very little local data, so directly training the model will not yield good results. Therefore, after the smart home temporal knowledge graph is established, the data pre-collection stage is first entered. In this stage, only user behavior data is collected, and the control functions of smart home devices are not enabled. After a small amount of data is collected, personalized model training is carried out based on the collected data. During multi-family collaborative training, local families first download a public model from the global server as an initial model and train it using local data. After a fixed number of training rounds, the model communicates with the global server to exchange model parameters, repeating this process until the model converges, resulting in a usable local personalized model. At this point, control of smart home devices begins, entering the management and control phase. During the management and control phase, user behavior data is still collected, and after a certain amount of incremental data is collected, a background process is started to retrain the model to ensure that the model can adapt to possible changes in user habits. First, the existing public model is downloaded from GS, and then the model is updated using a small amount of locally collected data to learn user personalized preferences. The specific process of training the model collaboratively between local households and global servers based on the FedAvg federated averaging algorithm is as follows: Initialize the parameters of the service management model for each type in the global server. and client parameter list ; Determine the current number of participating households, N; Initialize the parameters of the local service management model for each household. ; For communication round = 1, up to the maximum number of communication rounds, perform the following steps: For each client in the set of clients eligible for training, execute in parallel: Perform home-based local model training Each household uploads local model parameters To the global server; Save the parameters of each family model to ; The new global model parameters are derived using the FedAvg algorithm: ; Replace the old global model parameters with the new global model parameters: ; Broadcast the new global model parameters to all households; Update the local model parameters of each household to the global model parameters: ; Until the maximum number of communication rounds is reached or the model converges; For new families that join midway, the existing global model parameters are used first. The model is initialized using its local model, and a certain amount of local data is collected before being added to the above collaborative training process.

2. The smart home service management method based on federated learning according to claim 1, characterized in that, Step 1 includes constructing a conceptual model of a knowledge graph for smart home runtime; defining the location of the conceptual model of the knowledge graph for smart home runtime. ,user ,equipment ,Serve and environment Core concepts; among which: Location 𝐿𝑜𝑐𝑎𝑡𝑖𝑜𝑛 refers to a specific area in a smart home, defined as a tuple < > Using a single attribute Indicates the specific name of the area; user In the conceptual model of the smart home knowledge graph, smart home service objects are represented using tuples. This indicates the resident's name and current location information; environment This refers to the specific environmental type perceived by users in a home environment, expressed through tuples. To define , respectively represent the specific type of environment that the user can perceive, the specific location of the environment, and the specific physical quantity of the environment; equipment This refers to smart devices deployed in a smart home environment that provide services to users. Each device instance is represented by a tuple. It means that, among them The attributes represent the name of the smart device and the name of the deployment location of the smart device, respectively; This means that the parameters and state of the smart device during operation will change as the state of the smart device changes; Serve It is an abstraction of the functions provided by smart devices; service instances are defined as 5-tuples. , among which attributes This indicates the provider of the service, i.e., the name of the corresponding smart device; Record the area affected by this service; Record the types of environmental conditions affected by this service; Attributes refer to the service's... The effects produced by the specified environmental type include four types of effects: (1) monitoring (" "): indicates that the service has the function of monitoring the environmental status and can report the status value of the corresponding environmental status; (2) improve (" "): indicates that the service can improve the status value of the corresponding environment; (3) reduce (" "): indicates that the service can reduce the corresponding environmental state value; (4) set value (" "): This indicates that the service can maintain the corresponding environmental values ​​at a specified level; service instance The attribute indicates the status of the service, including both enabled and disabled; These are the parameter values ​​for the service operation, such as the specific settings when configuring the service; The knowledge graph conceptual model for smart home runtime further defines the corresponding set of concept instances: using Represents the set of all user instances participating in smart home services; Represents a set of location instances in different areas of a smart home environment; using Represents a collection of smart device instances in a smart home environment; using Represents a collection of service instances provided by smart devices; represented by Represents the set of perceptible environmental instances in a smart home environment; Define the set of all conceptual instances in a smart home scenario. : （1） The specific definitions of the relationships between the concepts are as follows: （2） Define the rules for constructing relationships between concept instances. Based on these rules, a smart home knowledge graph can be automatically constructed. When two concept instances meet specific preconditions for constructing relationships, the corresponding relationship instances can be constructed. When the construction premise is , The relationship is located at ( ); When the construction premise is , The relationship is perception ( ); When the construction premise is , The relationship is for monitoring ( ); When the construction premise is , The relationship is set to a value ( ); When the construction premise is , Relationship to improve ( ); When the construction premise is , The relationship is decreasing ( ); Based on the above construction rules, a series of relations are automatically constructed, and the set of these relations is denoted as . : （3） Based on the definitions of the concept instance set and relation set mentioned above, the smart home knowledge graph is defined as all concept instances and their relations in a smart home scenario, denoted as . : （4） Based on the above definitions, the construction steps for a smart home knowledge graph are as follows: Step (1) Construct a set of corresponding location instances based on the family layout. ; Step (2) Construct a set of user instances based on resident information ; Step (3) Construct a set of device instances based on the smart device deployment information. And build a corresponding service set based on the functions it provides. ; Step (4) Based on the set of location instances Construct a set of environment instances ; Step (5) is and Construct the corresponding location relation; Step (6) Based on the set of environment instances Service instance collection Construct the corresponding influence relationship between the service and the environment, and update the corresponding state values; Step (7) Construct the perceptual relationship between the user and the environment based on the user's location information; The smart home knowledge graph uses a runtime model to understand users. Location information acquisition and smart devices The state reading and operation transition will be discussed later. This is used to represent the corresponding runtime operation.

3. The smart home service management method based on federated learning according to claim 2, characterized in that, The method for constructing a smart home temporal knowledge graph specifically includes first extending the state of concept instances to the time dimension and defining symbols. Represents an instance of a concept exist The state at any given moment, such as This indicates that a certain user Here I am The state at any given moment, Indicates a certain environment exist The state at a given moment; then define A collection of state instances of each type at any given moment; The set specifically includes Time, location, instance, state set Assuming that the area information definition in a smart home does not change after initialization, the location information is the same at any given time, i.e. ; Time-based user instance state set Including users The name and location information at any given time, where the username is set to a fixed value; Time-based device instance state collection ,include The device name, deployment location, and attribute parameters at different times are recorded, and the attribute parameters of the device change frequently at different times. Time-based service instance state collection ,include The service provides the corresponding device name, affected area, function type, and service status information at any time. Time-based environment instance state collection ,include The environment type at any given time, the location of the environment perception, and the specific physical quantities of the environment, where the environment type is set to a fixed value; according to The collection of state instances of each type at any given time is defined; by aggregating these instances, we can obtain the desired result. The set of all concept instance states at any given moment Defined as: （5） Based on the aforementioned relational construction rules, it is also possible to construct based on the set of concept instances. Time-based relational instance set ; （6） Based on the above timing definition, The status of the smart home knowledge graph at any time can be obtained through and Defined as : （7） Finally, the temporal knowledge graph TKG model is obtained, defined as: （8）。 4. The smart home service management method based on federated learning according to claim 1, characterized in that, In step 1, the time-series knowledge graph model needs to persist its state in the database at regular intervals during operation to record its historical state and corresponding changes. Before persisting the data, a state update operation is performed to ensure the real-time nature of the data. The update steps before the state of a time-series knowledge graph is persisted include: (i) Update the current user location status information from the runtime model. ; (ii) Update the current device instance state information from the runtime model ; (iii) Update the status of the corresponding service instance based on the current device instance information. ; (iv) According to Type Relationship Update Corresponding environment instance property; Once the status update is complete, the status data can be stored, including the current timestamp. Concept Instance State Data and relational data Serialize to the corresponding database; Based on the given smart home time-series knowledge graph, the following specific data acquisition operations are defined in the time-series knowledge graph, and the specific meanings of the operations are as follows: (I) The operation represents retrieving a specific concept instance at a specified time. The status is returned. ; (II) Operation means to obtain Returns a set of concept instance states for each time point. ; (III) The operation represents constructing and returning. Relationships in a knowledge graph at any given time, return a set of relationships. ; (IV) Operation Acquisition A snapshot of the knowledge graph at any given moment, including the set of concept instances and the set of relations, returns the state of the entire knowledge graph. ; (V) Operations are derived from time-series knowledge graphs. From time t, the time window length is Conceptual Examples The time-series data is used to obtain the corresponding state sequences from the knowledge graph state sequences. (VI) Operation Acquisition Up to time, the time window length is Conceptual Examples Time-series data, extracting the state sequence at a given time point from the knowledge graph state sequence. ; Some operations can be implemented based on a single time point of knowledge graph state slice, such as operations (Ⅰ) to (Ⅳ), (Ⅴ) and (Ⅵ), which rely on multiple consecutive knowledge graph state slices to obtain state data sequences; in particular, when the stored data does not contain the time point specified by the given time sequence operation. When the data is returned, it will be no earlier than Data from the most recent time point; the smart home time-series knowledge graph model provides Knowledge graph of time ,include The set of states of each concept instance at any given moment and the relationships between various conceptual instances And the sequence of state changes of concept instances within a specified time period.

5. The smart home service management method based on federated learning according to claim 1, characterized in that, In step 2, to obtain the changes in the environmental state around the user over a period of time in the smart home temporal knowledge graph model, the user's location information must first be obtained. According to the Sense relation construction rules, the condition is met. The Context instance represents the environment that the user can perceive, denoted as the user-sensitive environment. ( Secondly, it obtains information through time-series knowledge graphs. Recent historical state data, which refers to changes in the user's surrounding environment over a period of time, is defined as: （9） Where t is the current time and k is the time window length; based on the operation definition of the time series knowledge graph. Equivalent to an instance of the environment state from Always moving forward A sequence of state data at each time point, through Obtained through operation; In the smart home temporal knowledge graph model, regulating the environmental state perceived by the user is equivalent to... The relevant functions of the smart devices that have an impact are controlled; in the temporal knowledge graph, the operation of a certain function of a smart device is abstracted as the manipulation of service instance objects, which can affect the environment. The service instance is the one that exists in the relationship. Service instances The current state of the smart device is the service instance. Current state ( User interactions with smart device functions can be represented as services. The state at the next moment ( The input to service control rules is defined as time-series data of user-sensitive environmental states. and the current state of sensitive service instances ( The output is defined as a user-sensitive service instance. State decision at the next moment ; Based on the smart home time-series knowledge graph and the rule definition, the control of smart home devices is transformed into the control of specified types of services. To enable users to control the different functions of multiple smart home devices in their homes, many different rules need to work together; it is necessary to build exclusive control rules for each user based on their behavior history.

6. The smart home service management method based on federated learning according to claim 1, characterized in that, In step 2, the smart home service management model based on neural networks uses a multi-hidden-layer deep neural network (DNN) model to learn user habits and generate corresponding management rules. It is necessary to collect user behavior data on smart devices at different times, locations, and scenarios. In the smart home temporal knowledge graph, this information is represented as user location information, the user's perceived environmental state, and the state of the smart devices. Based on this information, personalized management rules can be built for each user; this is achieved through the input layer. Three fully connected hidden layers , , and output layer The network is composed of layers; the number of neurons in the input layer depends on the time window length k, i.e., the environmental state changes and device state data within each time window are used as input; the number of neurons in the three hidden layers are 32, 64, and 32, respectively; a fully connected structure is used between the layers; The activation function performs a nonlinear transformation; assuming arrive The input for each layer is Then its corresponding output is : （10） The Sigmoid function is defined as follows: （11） The model uses the Adam optimization algorithm to update the gradient, and the loss function used is the mean squared error, which is defined as: （12） The weight matrix obtained after model training , , , and bias matrix , , , The neural network is initialized accordingly; after multiple iterations, the weight matrix can be obtained. , , , and bias vector , , , The value is saved as the user's personalized model; Supported by temporal knowledge graphs, the collection of previous... Time series data of user-sensitive environmental status at each time point Obtain the corresponding CValue input model to get the decision result regarding the service status. Output to the time-series knowledge graph and set the corresponding... By leveraging its control capabilities, it can issue control commands to corresponding smart home devices, thereby enabling the management and control of corresponding device services and achieving a seamless and intelligent smart home user experience.

Images