Local computing cloud interacting with public computing cloud

Abstract

A home computing cloud (HCC) may support one or more Internet of Things (IoT) devices using different connection protocols in a local environment. Typically, the HCC reduces the amount of data traffic sent to a public computing cloud (PCC) by processing the collected device data locally rather than by sending the device data to the PCC for processing. This approach reduces the amount of data traffic sent over the network, improves data privacy and helps maintain the desired quality of service level. To do this, the HCC may download an appropriate data analysis model from the PCC, train the model, execute the trained model to obtain predictive information from the collected IoT device data, and upload the trained model to the PCC. Alternatively, the HCC and the PCC may execute sub-models of the analysis model and exchange the outputs of these sub-models with each other.

Description

Technical Field

[0001] Aspects of the present disclosure relate to a local computing cloud that supports interaction with a public computing cloud. The local computing cloud may be located in a home and may support one or more Internet of Things (IoT) devices. Analysis models may be downloaded from the public computing cloud and executed locally. Reinforcement training may also be performed locally without external transmission of device data and user behavior information, thereby greatly reducing data traffic that may compromise data privacy. Background Art

[0002] Internet of Things (IoT) applications typically rely on remote and centralized servers to collect input data and generate certain actions based on current input as well as historical data. This approach typically requires IoT devices such as smart sensors, thermostats, and smart appliances to exchange data between themselves and remote servers such as public computing clouds. Using another approach, a gateway may be required to convert data from one connection protocol to another in order to send data from the terminal device to the server, for example, from ZigBee to WiFi. Large amounts of data transmission between the terminal device and the server mean expensive service costs. In addition, this may generate a large amount of data traffic in the network, which may result in additional network delays, data loss during transmission, or expensive maintenance costs in order to maintain the desired quality of service level. In addition, data security and privacy are an important concern when storing large amounts of personalized data in public computing clouds. Summary of the invention

[0003] A home computing system (which may be referred to as a "home computing cloud") integrates a communication gateway, a WiFi router, a cloud server, and a mass storage device to support one or more Internet of Things (IoT) devices in a local environment such as a home residence. Because the home computing cloud (HCC) processes the collected device data locally instead of sending the device data to a public computer cloud (system) for processing, the home computing cloud generally reduces the amount of data traffic sent to the public computing cloud (PCC). This approach improves network latency, reduces data loss during transmission, and helps maintain a desired quality of service level.

[0004] To do so, the HCC may download an appropriate data analysis model (which may be referred to as a "model" and selected from a plurality of data analysis models) from the PCC based on configuration information (e.g., the types of supported IoT devices). The HCC may then execute the model locally by obtaining device data from one or more IoT devices, applying some or all of the device data to the model, and obtaining prediction results from the model. The prediction results may then be applied to one or more of the supported IoT devices to affect the operation of the one or more IoT devices.

[0005] On the other hand, the HCC sends a subset of the device data to the PCC for further processing and receives decision information based on the subset of data. For example, the subset of the device data may represent one or more signal characteristics of a complex signal (e.g., a multimedia signal including speech, music, image, or video signals) that requires intensive processing that the HCC may not be able to support. In an exemplary method for facial recognition, the HCC may implement an image preprocessing layer and a feature extraction layer of an analysis model and send the resulting data to the PCC for analysis and decision making. The HCC applies the received results and other device data (corresponding to the model input) to the downloaded data analysis model.

[0006] On the other hand, the PCC executes the input processing layer of the prediction model and sends the corresponding output to the PCC. The PCC then executes all remaining hidden layers and sends the corresponding output of the final hidden layer back to the HCC. The HCC then executes the output layer. This approach generally eliminates sensitive information sent over the Internet and therefore enhances data privacy on the Internet.

[0007] On the other hand, the distribution of workload for executing the model can be based on the computer power of the HCC (such as sending raw data to the PCC for the entire process); the amount of data traffic (such as sending only feature data to the PCC to handle the remaining tasks); data privacy (such as sending mathematical transformation data within the model layer to the PCC for continued analysis); and the consistency of model parameters (such as an HCC execution layer with fixed parameters and a PCC executable layer with parameters that continuously change through intensive training).

[0008] On the other hand, the HCC may have sufficient computing resources to perform more complex tasks, such as training deep neural networks. The HCC may download appropriate templates for data analysis models from the PCC, train the models locally, and execute the trained models to obtain prediction information from the collected IoT device data.

[0009] On the other hand, both HCC and PCC can execute and train the same data analysis model (e.g., auxiliary training). However, the learning rates of HCC and PCC may be different (e.g., due to the greater computing power of PCC). Although HCC executes and trains local models based on IoT device data, HCC also sends device data to PCC. PCC executes and trains using the same device data and sends error measurements back to HCC. HCC compares the error measurements with the two clouds and continues training until the error measurement from HCC is below a threshold.

[0010] On the other hand, when the error measurement from PCC is continuously lower or substantially lower than that of HCC, HCC may decide to continue training using parameters from PCC.

[0011] On the other hand, if the error measurement from PCC reaches the threshold first, HCC may decide to use the model trained by PCC and stop training.

[0012] On the other hand, HCC can upload the trained model to PCC for archiving, sharing, or optimization.

[0013] On the other hand, the PCC can analyze all received models from other HCCs and optimize the new model. The PCC can distribute the new model to all HCCs.

[0014] On the other hand, the HCC can decide to use a new model entirely, use a new model with parameters from an existing model, or ignore the new model entirely. The decision can be based on a comparison with error measurements when executing different models with locally stored empirical data.

[0015] On the other hand, HCC can continue to run or train the local model, and PCC can train the new model in parallel with the new input data. HCC continuously sends the new input data to PCC to train the new model until the new model is accurate enough. HCC can then download the new model for use.

[0016] On the other hand, HCC can request PCC to continuously train new models using the parameters in HCC.

[0017] On the other hand, in training models, subjective weights can be applied when calculating errors based on application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The foregoing summary of the invention, as well as the following detailed description of exemplary embodiments of the invention, will be better understood when read in conjunction with the accompanying drawings, which are included by way of illustration and not as limitations of the invention as claimed.

[0019] Figure 1 A home environment is shown in which a home computing cloud (HCC) interacts with a public computing cloud (PCC) according to an embodiment.

[0020] Figure 2 An HCC without WiFi router capabilities is shown according to an embodiment.

[0021] Figure 3 An HCC with WiFi router capabilities is shown according to an embodiment.

[0022] Figure 4 Shown is a PCC interacting with multiple HCCs according to an embodiment.

[0023] Figure 5The HCC is shown interacting with the PCC and user applications according to an implementation scheme.

[0024] Figure 6 An HCC is shown according to an embodiment, wherein the HCC is executing an analytical model while the PCC is performing reinforcement training.

[0025] Figure 7 An HCC according to an embodiment is shown, wherein the HCC assigns all data analysis and intensive training tasks to the PCC.

[0026] Figure 8 A method for HCC to divide the analysis model into two sub-models according to an embodiment is shown. Part of the original model is executed at the HCC, and the remaining part of the model is executed at the PCC to reduce the calculation at the HCC, to reduce data traffic and to protect data privacy when sending data over the network.

[0027] Fig. 9 A method for HCC to perform reinforcement learning according to an implementation scheme is shown.

[0028] Fig.10 A method in which an HCC interacts with a PCC to perform assisted learning according to an embodiment is shown. DETAILED DESCRIPTION

[0029] "HCC" (home computing cloud) may not be limited to a home residence, and may support other types of entities, such as a business or a building. Therefore, "HCC" may be understood as a "local computing cloud". Moreover, "cloud" may be referred to as a computing system, etc.

[0030] According to aspects of the implementation scheme, the HCC integrates a communication gateway, a WiFi router, a cloud server, and a mass storage device to support one or more Internet of Things (IoT) devices in a local environment such as a home. Because the HCC processes the collected device data locally instead of sending the device data to a public computing cloud (PCC) for processing, the HCC generally reduces the amount of data traffic sent to the PCC. This approach improves network latency, reduces data loss during transmission, and helps maintain the desired level of quality of service. To do this, the HCC can download an appropriate data analysis model (which may be referred to as a "model") from the PCC based on configuration information (e.g., the type of IoT device supported). The HCC can then execute the model locally by obtaining device data from one or more IoT devices, applying some or all of the device data to the model, and obtaining prediction results from the model. The prediction results can then be applied to one or more of the supported IoT devices to affect the operation of one or more IoT devices.

[0031] The HCC may include one or more IoT devices located in the home. Embodiments support a variety of IoT devices, including but not limited to smart thermostats, appliances, lighting, security devices, etc.

[0032] The HCC may interact with the PCC to exchange information related to one or more IoT devices. The information may include data provided by one or more IoT devices (e.g., temperature measurements) and information indicating actions performed by one or more IoT devices (e.g., operating modes).

[0033] PCC (which may be referred to as a "public cloud") makes computing services provided by third-party providers available over the public Internet, making them available to anyone who wants to use or purchase them. Services may be free or sold on demand, thus allowing customers to pay only for usage in consumed CPU cycles, storage, or bandwidth.

[0034] In another aspect of the embodiment, algorithms can be used to train data analysis models locally. Reinforcement (machine) learning can also be added to provide machine learning capabilities to the HCC. By processing data locally, the user's privacy can be substantially improved by limiting the amount and type of data sent over the network and stored in the PCC.

[0035] In another aspect of the embodiment, the HCC (home computing system) executes both the data analysis model and reinforcement learning locally.

[0036] In another aspect of the embodiment, the data analysis model is divided into two sub-models. The first sub-model includes the input processing layer of the data analysis model and is executed by the home computing system (cloud). The second sub-model includes the hidden layer of the data analysis model and is executed by the public computing cloud. In this way, the original data is stored locally in the computing system, thereby protecting the privacy of the user.

[0037] In another aspect of the embodiment, the data analysis model is divided into three sub-models. The first sub-model and the third sub-model include an input layer and an output layer, respectively, and are executed by a home computing system (cloud). The second sub-model includes only a hidden layer and is executed by a public computing cloud. In this way, the user's privacy is further protected by maintaining the original input data and the predicted output locally.

[0038] In another aspect of the embodiment, assisted learning enables training in a public computing cloud while executing a data analysis model at a home computing system.

[0039] In another aspect of the embodiment, assisted learning enables parallel training in both the public computing cloud and the home computing cloud while executing the data analysis model at the home computing system.

[0040] Figure 1 A home environment is shown in which the HCC 101 interacts with the PCC 102 via the data channel 151 according to an embodiment.

[0041] An IoT device (not explicitly shown) may be an interconnected computing device (e.g., a smart thermostat or appliance) within a home that provides transmitted information and obtains received information via the HCC 101. The received information may indicate one or more actions that the IoT device should perform.

[0042] Although Figure 1 An operating environment spanning a home is depicted, but embodiments may span other local environments, such as a building or a commercial location.

[0043] When implementing an Internet of Things (IoT) system, data traffic capacity, data security, and data privacy are important considerations. By minimizing data traffic and carefully selecting the type of data to be sent on the data channel 151 between the PCC 102 and the application environment supported by the HCC 101, and by minimizing the amount and type of data stored inside the PCC 102, data exposure due to unauthorized access can be reduced. In addition, data traffic can be reduced, and the cost of using services provided by the PCC 102 can therefore be reduced. By storing data in the HCC 101 and performing data analysis and machine learning from the HCC 101, services can be maintained when an Internet connection is inaccessible. In addition, delays introduced from an Internet connection can be eliminated. However, since the PCC 102 typically provides computing power and software services that the HCC 101 may not provide, it may not be possible to completely circumvent the services provided by the PCC.

[0044] When deciding which data and functions to store locally in the HCC 101 and which data and services to allocate to the PCC 102, one approach is to store discrete time interval data (e.g., sensor data, manual settings, etc.) together with machine learning algorithms and data analysis models in the HCC 101, as will be discussed in further detail. The HCC 101 can continuously send supported IoT devices (e.g., such as the HCC 102) to the PCC 102 via the data channel 151. Figure 2 HCC 101 may also periodically send information about the training model (eg, parameters and error measurements of the analysis model) to public computing cloud 102.

[0045] PCC 102 can collect data from all available HCCs 101 and 401 (such as Figure 4Further discussion), and train a new pre-trained model, such as a model template, based on the collected data. The new pre-trained model can then be distributed back to each HCC 101 and 401. Alternatively, PCC 102 can notify HCC 101 and / or 401 that a new pre-trained model is available, wherein HCC 101 and / or 401 can decide whether to download the model via data channel 151 based on predetermined criteria.

[0046] As will be discussed in further detail, some or all of the data from the previous model can be directly applied to the model template. Alternatively, if no data is available, reinforcement learning can be applied using the model template with locally stored model data or completely new data.

[0047] As will be discussed in further detail, parallel training (machine learning in both the home and PCC) can be applied when performing reinforcement learning. Model parameters can be exchanged during training. The model adopted by the HCC can be selected based on error measurements.

[0048] PCC 102 can consistently update the machine learning algorithm to HCC 101.

[0049] For continuous time signals (eg, audio, image, and video data), the HCC 101 may stream the signal data to the PCC 102. When data analysis models are supported at the PCC 102, the results from the models may be returned to the HCC 101.

[0050] Alternatively, the analysis model can be divided into two parts (e.g., sub-models) and partially executed at the HCC and PCC. The data exchanged between the two clouds can be parameters in one or more layers of the analysis model. This approach can reduce the amount of data to be exchanged between the two clouds. In addition, privacy can be maintained relative to sending the original data stream.

[0051] Alternatively, the analysis model can be divided into three sub-models and partially executed at the HCC and PCC. In this way, the input processing layer and output layer of the analysis model are executed at the HCC, and the hidden layer is executed at the PCC. In this way, the original input data and predicted output that may contain private information about the device owner will be saved locally and will not be disclosed to the outside world.

[0052] Alternatively, the model can be trained in PCC 102, downloaded from PCC 102 to HCC 101, and executed locally by HCC 101. The decision to retrain a new model can be triggered by the owner (user). Examples include adding a new device for recognition in the model, adding a new rule in the model, etc.

[0053] Figure 2An HCC 201 is shown associated with a separate WiFi router 206 , interacting with a PCC 202 , according to an embodiment.

[0054] The interaction between local IoT devices 204-205 can be supported by a protocol gateway 210 and an IoT message converter 211 executed at HCC 201. IoT devices 204-205 communicate via the protocol gateway 210 through a corresponding protocol (e.g., Zigbee). The protocol gateway 210 passes the device message to the IoT message converter 211, which includes an IoT protocol message agent 208 (e.g., MQTT agent) or a COAP server (not explicitly shown) and an IoT protocol message bridge 209 (e.g., MQTT / Zigbee bridge). The message converter 211 bridges the IoT device message into an IoT protocol message (e.g., MQTT message). The MQTT message can be directed to other IoT devices connected to HCC 201, a rule engine, or PCC 202. As an example, a device message from a Zigbee device 205 can be sent to HCC 201 via a Zigbee gateway 210. Device data can be extracted from the device message and sent to an analysis model for processing. At the same time, device messages may be passed to the MQTT / Zigbee bridge 209 and the MQTT proxy 208 to reach the PCC 202 via the home WiFi router 206 .

[0055] WiFi devices that support MQTT clients (e.g., device 203) can also connect to the MQTT broker 208 within HCC201.

[0056] The device data collected by the HCC 201 may be stored in a mass data storage device (not explicitly shown), and thus the additional cost of sending the collected data back and forth with the PCC 201 is avoided.

[0057] Communication between the WiFi device 203 and the HCC 201 can occur through two different paths 251 or 252, depending on which WiFi access point the WiFi device 203 is connected to. For path 251, the MQTT message from the WiFi device 203 is routed from the home WiFi router 206 to the MQTT proxy 208, which can further direct the MQTT message to other IoT devices or the PCC 202 via the home WiFi router 206. Through path 252, the WiFi device 203 is directly connected to the HCC 201, which acts as a WiFi access point (AP) and can also connect to the home WiFi router 206 and then to the PCC 202.

[0058] A user application (app) 207 may interact with the HCC 201 and / or PCC 202 via the WiFi connection 253 .

[0059] Figure 3 An HCC 301 with WiFi router capabilities is shown according to an embodiment. Because the HCC 301 includes a WiFi router 306, all WiFi devices (e.g., device 303) can connect to the HCC to access user applications 307 and / or Internet services of the PCC 302. In addition, a mobile device 307 in close proximity can also connect to the HCC 301 for accessing Internet services.

[0060] Figure 4 The PCC 102 is shown interacting with multiple HCCs including HCC 101 and HCC 401 according to an embodiment. Thus, the PCC 102 can obtain data about data analysis models executed on multiple HCCs and can train mirror models executed on the PCC 102. The PCC 102 can then distribute the trained model to one or more HCCs so that the trained model can be executed locally.

[0061] Figure 5 The HCC 501 is shown interacting with the PCC 514 and the user application 512 according to an embodiment.

[0062] Similar to Figure 2 and Figure 3 , HCC 501 interacts with IoT devices 504 - 506 via communication server 507 , PCC 514 via cloud interface 503 and mobile device 512 .

[0063] HCC 501 includes a processing device 502, a cloud interface 503, a communication server 507, a memory device 509, and a storage device 511. In addition, HCC 501 may include an embedded WiFi router 508 having some embodiments (e.g., Figure 3 shown).

[0064] Processing device 502 controls the operation of HCC 501 by executing computer-readable instructions stored on memory device 509. For example, processing device 502 may execute computer-readable instructions to respectively perform Figures 6 to 10 The processes 600-1000 shown in . Embodiments may support various computer-readable media, which may be any available media accessed by the processing device 502 and include volatile and non-volatile media, removable and non-removable media. By way of example and not limitation, computer-readable media may include a combination of computer storage media and communication media.

[0065] Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computing device.

[0066] Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and includes any information delivery media. A modulated data signal is a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example and not limitation, communication media include wired media such as a wired network or direct wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.

[0067] The HCC 501 may execute the downloaded model 510 from the PCC 514 at the memory device 509. The machine learning model 510 may include a neural network model processing data from the IoT devices 504-506 as input, thereby generating one or more decision outputs from the model 510.

[0068] If there are any corrective actions to the predicted output from the model, the home computing cloud 501 can apply reinforcement learning to train the model.

[0069] Figure 6 A logic flow 600 for executing an analysis model locally is shown. When the HCC 101 is setting up or adding a new device, the HCC 101 sends system configuration information to the PCC 102 at box 601 and downloads the corresponding analysis model at box 602. The analysis model is implemented at box 603 and executed at box 604 based on the IoT device input 651 and the model parameters 650.

[0070] As an example, HCC 101 currently supports thermostats and presence sensors in a home. The thermostat learns that when a user is home from April to October, the operating mode should be set to cooling and the temperature should be set to 23°C, while from November to March, the operating mode should be set to heating and the temperature should be set to 25°C:

[0071] April to October: [0: Mode = Off | 1: Mode = Cool, Set_Temperature = 23] exists.

[0072] November to March: [0: Mode = Off | 1: Mode = Heating, Set_Temperature = 25] exists.

[0073] Continuing with the example, when the user adds smart curtains (new IoT device) to the ecosystem, HCC 101 sends configuration information (e.g., profile) about the thermostat, presence sensor, and smart curtains to PCC 102. When adding a new device (smart curtains), PCC 102 notifies HCC 101 that a new analysis model is available. Since the setting "Use new model template" is set to "Yes", HCC 101 downloads the new model template. For the new model, the original settings are applied. In addition, a new input parameter (IoT device input) "Degree of curtain opening (0% fully open to 100% fully closed)" is introduced. With the new model provided by PCC 102, the degree to which the curtains are opened has no effect on the set temperature:

[0074] April-October: Exists [0:Mode=Off|1:Mode=Cool, Set_Temperature=Curtain[0,100:23]].

[0075] November to March: exists [0:Mode=Off|1:Mode=Heating, Set_Temperature=Curtains[0,100:26]].

[0076] During execution, HCC 101 continuously receives input from IoT devices 651 and processes it through analytical model 604 to obtain prediction results. The prediction results are applied to the corresponding IoT devices at block 605.

[0077] The HCC 101 continuously monitors the IoT ecosystem to make any corrections to the prediction results at box 606. If any corrections are made, the HCC 101 provides the corrections T[n] 654 as feedback to the PCC 102 along with the corresponding IoT device input S[n] 652 and the prediction results R[n] 653 at box 607, where the result information may include R[n] 653 and T[n] 654. For example, the HCC 101 may conditionally initiate reinforcement learning at the PCC 102 and receive updated parameters from the PCC 102 in response. The HCC may then update the downloaded analytical model.

[0078] Continuing with the above example, in July, when the smart curtains are approximately half closed (e.g., curtains = 40% closed), the user changes the set temperature to 24°C:

[0079] 20200701:Exists[1:Mode=Cooling, Set_Temperature=Curtain[40:24]].

[0080] When the curtains are slightly closed (eg, curtains = 20% closed), the original set temperature remains unchanged (ie, 23° C.), and thus no message is received from the HCC 101 .

[0081] In this case, the HCC 101 continuously sends user corrections to the PCC 102 as long as they are available:

[0082] 20190706: exists [1:Mode=Cooling, Set_Temperature=Curtain[40:24]].

[0083] 20190707: exists [1:Mode=Cooling, Set_Temperature=Curtain[50:24]].

[0084] 20200713:Exists[1:Mode=Cooling, Set_Temperature=Curtain[65:24]].

[0085] 20200714: exists [1:Mode=Cooling, Set_Temperature=Curtain[45:24]].

[0086] 20200719: exists [1:Mode=Cooling, Set_Temperature=Curtain[60:24]].

[0087] 20200720:Exists[1:Mode=Cooling, Set_Temperature=Curtain[40:24]].

[0088] 20200721:Exists[1:Mode=Cooling, Set_Temperature=Curtain[50:24]].

[0089] At PCC 102, reinforcement learning is performed at block 621 to obtain a new set of model parameters (replacing model parameters 650). New parameters 655 are sent to HCC 101 at block 622 and used by the analytical model at block 608 for the next device input S[n+1].

[0090] Continuing with the above example, when the reinforcement training is completed, PCC 102 sends the new model parameters to HCC 101. HCC 101 then applies the new parameters in the model. The new parameters are:

[0091] April to October, there exists [0: Mode = Off | 1: Mode = Cool, Set_Temperature = Curtains [0, 40: 23 | 40, 100: 24]].

[0092] November to March, there exists [0: Mode = Off | 1: Mode = Heating, Set_Temperature = Curtains [0, 100: 26]].

[0093] Figure 7 A process 700 for processing multimedia signals is shown, where an analysis model is executed at a PCC 102. At block 701, the HCC 101 continuously streams source data 751 to the PCC 102. At block 721, the PCC 102 analyzes the data stream using a model with Z hidden layers to obtain a prediction result R 752. The result is then sent back to the HCC 101 at block 722, where the HCC 101 applies the prediction result to the IoT ecosystem at block 702.

[0094] The HCC 101 continuously monitors the IoT ecosystem to make any corrections to the prediction results at box 703. If there are any corrections, at box 704, the HCC 101 provides the corrections T753 to the PCC 102 as feedback.

[0095] At block 723, the PCC 102 performs reinforcement learning to obtain a new set of parameters W 754. The new parameters are then applied to the analysis model of the subsequent source data stream at block 721.

[0096] Figure 8 An analysis model with Z hidden layers 801 is shown, which can be divided into two sub-models 804, one sub-model with X hidden layers 802 (implemented at HCC 101) and the other sub-model with Y hidden layers 803 (implemented at PCC 102), where X+Y=Z. In application 800, a source data stream from one or more IoT devices is analyzed at HCC 101. The output of the xth hidden layer from hidden layer 802 is then sent to PCC 102 to continue analysis through hidden layer 803.

[0097] Using method 800, the amount of data sent from HCC 101 to PCC 102 is generally reduced. In addition, the privacy of user data can be protected by sending a transformed version of the data instead of the original data.

[0098] Additionally, the distribution of workload for executing models can be based on:

[0099] -Computer power of the home computing cloud. (For example, HCC 101 executes the first layer of the analytical model and then sends the output to PCC 102. PCC 102 then executes the remaining layers of the analytical model.

[0100] - The amount of data traffic. (For example, the HCC 101 executes the layers of the analytical model until a layer with the minimum number of output nodes is reached. The HCC will then send the output of that layer to the PCC 102. The PCC 102 then executes the remaining layers of the analytical model).

[0101] -Data privacy. For example, HCC 101 may execute layers of the analytical model until a layer position is reached where the output of the layer is completely independent of the source data. HCC 101 then sends the output of the layer to PCC 102. PCC 102 then executes the remaining layers of the analytical model.

[0102] -Consistency of model parameters. For example, HCC 101 executes the layers of the analysis model with fixed parameters. HCC 101 then sends the output of the last layer to PCC 102. PCC 102 then executes the remaining layers of the analysis model).

[0103] The capabilities of the HCC 101 may vary from those having a basic configuration that only allows execution of analytical models to more powerful capabilities equipped with more powerful hardware for training analytical models with multiple hidden layers.

[0104] In the case of further improving data privacy using another specific implementation, the analysis model can be divided into three sub-models. In this case, the input processing layer and the output layer are executed at the HCC 101, and some or all of the hidden layers are executed at the PCC 102. In this case, the original input data and the predicted output closely related to the user will be saved locally.

[0105] Fig. 9 A logic flow 900 is shown for performing reinforcement learning locally at the HCC 101. Reinforcement learning is performed at block 901. The HCC 101 optimizes the parameter set W of the analytical model by minimizing an error function using input data S[n] 951, predicted output R[n] 952, and correction T[n] 953 at the nth operation.

[0106] When training is complete, a new set of parameters W[n+1] 954 is provided to the analysis model at block 902 so that the analysis model can utilize them at block 903. When there are new device inputs S[n+1] 955 from one or more IoT devices, new prediction results R[n+1] 956 can be applied to the IoT ecosystem at block 904.

[0107] If there is a correction by the user at block 905 , then at block 906 , the reinforcement learning algorithm is repeated using the data (S, R, and T) from the [n+1]th example. Otherwise, the HCC 101 waits for input at block 907 .

[0108] For some applications, the training of the analytical model may be too demanding on the computer resources of the HCC 101. In such cases, the PCC 102 may be used to assist in reinforcement learning at the HCC 101.

[0109] Fig.10 A logic flow 1000 for assisted training (parallel training) is shown, where the HCC 101 performs training during a training sequence 1001 and the PCC 102 performs parallel training during a training sequence 1021 .

[0110] At box 1002, the HCC 101 performs reinforcement learning using the device input S[n] 1051 and the analytical model G. At box 1003, the predicted output O[m] 1053 from the analytical model G is compared with the correction T[n] 1052 from the user to calculate the error measurement value E[m] 1054. At box 1004, adjustments to the model parameters are determined based on the magnitude of the error value and the rate of change of the error value between iterations. Then, at box 1002, the analytical model G uses the new parameter set U[m+1] 1055 to calculate the new output O[m+1] using the same device input S[n]. The new error measurement value E[m+1] is then calculated by comparing T[n] with O[m+1]. Additional iterations can be performed until the desired error measurement value is obtained.

[0111] When reinforcement learning is performed at the HCC 101, a copy of the device input S[n] 1051 and the correction T[n] 1052 is sent to the PCC 102, for example, via the data channel 151. The PCC 102 performs a similar reinforcement learning process at box 1021 to assist in the model training at the HCC 1001. Correspondingly, at box 1022, the device input S[n] 1051 is performed by the analysis model P. At box 1023, the predicted output Q[k] 1073 from the analysis model P at box 1022 is compared with the correction T[n] 1052 from the user to calculate an error measurement value F[k] 1074. At box 1024, adjustments to the model parameters are based on the magnitude of the error value and the rate of change of the error value between iterations. The new parameter set V[k+1] 1075 is then used by analytical model P 1022 to calculate a new output Q[k+1] using the same plant input S[n], and a new error measure F[k+1] is calculated by comparing T[n] to Q[k+1], etc.

[0112] The PCC 102 may use the same algorithm (where model P is a copy of model G) to change model parameters U 1055 and V 1075. Alternatively, a different algorithm (where model P is not a copy of model G) may be used to adjust model parameters U 1055 and V 1075.

[0113] During auxiliary training, the error measures from the two learning models (G at box 1002 and P at box 1022) can be consistently compared. If there is a significant difference between the two error measures, both learning models G and P can select (switch to) the model parameter set that produces the lower error measure and continue training.

[0114] If either of the two models G and P meets the target error threshold, training may be terminated, with the set of model parameters that meets the error threshold being used by the analysis model at the HCC 101 .

[0115] At the end of reinforcement learning, HCC 101 may upload the trained model to PCC 102 for archiving, sharing, or optimization.

[0116] PCC 102 can analyze all models received from other HCCs and optimize new models from them. For some embodiments, new models can be trained when new default IoT devices are added. PCC 102 can distribute new models to all HCCs.

[0117] The HCC 101 may decide to use the new model provided by the PCC 102, use the new model with parameters from the original model, or ignore the new model altogether. When executing different models using locally stored empirical data, the decision may be based on a comparison with error measurements.

[0118] HCC 101 may decide to perform reinforcement learning at any time during operation, such as according to block 621 (e.g. Figure 6 As shown), frame 723 (as Figure 7 As shown), block 906 (as shown Fig. 9 ) or process 1000 (as shown Fig.10 shown).

[0119] In reinforcement learning, legacy IoT device data stored locally at the HCC 101 can be used to train the analysis model (e.g., continuous improvement of the original model). Alternatively, new IoT device data can be used to train the analysis model (e.g., a new analysis model with additional device types). Alternatively, a mixture of legacy IoT device data and new IoT device data can be used to execute the original model and train the new model in parallel.

[0120] During reinforcement learning, weights can be assigned when calculating error measures. For example, for object recognition, recognition errors might be assigned more weight than confidence level errors.

[0121] The various aspects described herein may be embodied as methods, devices, or computer executable instructions stored on one or more non-transitory and / or tangible computer-readable media. Therefore, these aspects may take the form of a complete hardware implementation, a complete software implementation (which may or may not include firmware) stored on one or more non-transitory and / or tangible computer-readable media, or an implementation combining software and hardware aspects. Any and / or all method steps described herein may be embodied in computer executable instructions stored on a computer-readable medium (such as a non-transitory and / or tangible computer-readable medium and / or a computer-readable storage medium). Additionally or alternatively, any and / or all method steps described herein may be embodied in computer-readable instructions stored in a memory and / or other non-transitory and / or tangible storage media of a device including one or more processors, so that when one or more processors execute computer-readable instructions, the device performs such method steps. In addition, various signals representing data or events described herein may be transmitted between a source and a destination in the form of optical waves and / or electromagnetic waves, which travel through a signal-conducting medium such as a metal wire, an optical fiber, and / or a wireless transmission medium (e.g., air and / or space).

[0122] Various aspects of the present disclosure have been described in terms of their illustrative embodiments. Many other embodiments, modifications, and variations within the scope and spirit of the appended claims will occur to those of ordinary skill in the art upon review of this disclosure. For example, those skilled in the art will appreciate that the steps illustrated in the illustrative figures may be performed in an order other than the order stated, and that one or more of the steps illustrated may be optional according to various aspects of the present disclosure.

Claims

1. A home computing system supporting at least one Internet of Things (IoT) device, the home computing system comprising: a communication gateway configured to connect to the at least one IoT device; a cloud interface configured to exchange information with a public computing cloud; a processor configured to execute computer executable instructions; a memory storing the computer-executable instructions that, when executed by the processor, cause the home computing system to: sending configuration data about the home computing system, wherein the configuration data describes the home computing system; downloading a downloaded data analysis model based on the configuration data from the public computing cloud via the cloud interface; obtaining device data from the at least one IoT device via the communication gateway; applying the device data to the downloaded data analysis model; Obtaining a first prediction result from the data analysis model based on the device data; determining a first error measurement value based on the first prediction result and correction information for the first prediction result; When the first error measurement exceeds a predetermined threshold, initiating reinforcement learning at the public computing cloud based on the device data, the first prediction result, and the correction information; In response to the sending, receiving modified model parameters from the public computing cloud; and The modified model parameters are applied to the downloaded data analysis model.

2. The home computing system of claim 1 , wherein the memory stores computer executable instructions that, when executed by the processor, further cause the home computing system to: When the first error measurement value does not exceed the predetermined threshold, the first prediction result is applied to the at least one IoT device.

3. The home computing system of claim 1 , wherein the memory stores computer executable instructions that, when executed by the processor, further cause the home computing system to: downloading new data analysis models from the public computing cloud when new IoT devices are added to the home computing system; Applying current model parameters to the new data analysis model; Initiating reinforcement learning of the new data analysis model; In response to the starting, obtaining new model parameters of the new data analysis model; and The new model parameters are reapplied to the new data analysis model.

4. The home computing system of claim 3, wherein the new model parameters are received from the public computing cloud.

5. The home computing system of claim 1 , wherein the memory stores computer executable instructions that, when executed by the processor, further cause the home computing system to: initiating reinforcement learning at the home computing system based on the device data, the first prediction result, and the first correction information; obtaining first updated model parameters from the reinforcement learning; and The first updated model parameters are applied to the downloaded data analysis model to obtain an updated data analysis model.

6. The home computing system of claim 5, wherein the memory stores computer executable instructions that, when executed by the processor, further cause the home computing system to: applying the device data to the updated data analysis model; obtaining a second prediction result from the updated data analysis model based on the device data; determining a second error measure based on the second prediction result and second correction information to the first prediction result; When the second error measure exceeds a desired error level, repeating the reinforcement learning; obtaining second updated model parameters from repeated reinforcement learning; and The second updated model parameters are reapplied to the downloaded data analysis model.

7. The home computing system of claim 6, wherein the memory stores computer executable instructions that, when executed by the processor, further cause the home computing system to: When computing resources of the home computing system are exceeded, cloud resources at the public computing cloud are requested to perform the reinforcement learning.

8. The home computing system of claim 5, wherein the memory stores computer executable instructions that, when executed by the processor, further cause the home computing system to: The updated data analysis model is uploaded to the public computing cloud.

9. The home computing system of claim 1 , wherein the memory stores computer executable instructions that, when executed by the processor, further cause the home computing system to: performing, by the processor at the home computing system, reinforcement learning of the downloaded data analysis model; obtaining a first set of model parameters from the reinforcement learning at the home computing system; initiating assisted learning at the public computing cloud based on the device data, wherein the assisted learning is performed in parallel with executing the downloaded data analysis model at the home computing system; Obtaining a second set of model parameters from the auxiliary learning at the public computing cloud; and The third error measure of the reinforcement learning is compared with the fourth error measure of the auxiliary learning.

10. The home computing system of claim 9, wherein the assisted learning utilizes a copy of the downloaded data analysis model.

11. The home computing system of claim 9, wherein the memory stores computer executable instructions that, when executed by the processor, further cause the home computing system to: selecting the first set of model parameters from the first set of model parameters and the second set of model parameters when the third error measure is less than the fourth error measure; and The first set of model parameters is applied to the downloaded data analysis model executed at the home computing system.

12. The home computing system of claim 9, wherein the memory stores computer executable instructions that, when executed by the processor, further cause the home computing system to: When the third error measure is greater than the fourth error measure, selecting the second set of model parameters from the first set of parameters and the second set of parameters; and The second set of model parameters is applied to the data analysis model executed at the home computing system.

13. The home computing system of claim 9, wherein the memory stores computer executable instructions that, when executed by the processor, further cause the home computing system to: When one of the third error measurement value and the fourth error measurement value is less than a target error threshold, the training of the downloaded data analysis model is terminated.

14. The home computing system of claim 13, wherein the memory stores computer executable instructions that, when executed by the processor, further cause the home computing system to: In response to the terminating, the downloaded data analysis model is uploaded to the public computing cloud.

15. A home computing system supporting at least one Internet of Things (IoT) device, the home computing system comprising: a communication gateway configured to connect to the at least one IoT device; a cloud interface configured to exchange information with a public computing cloud; a processor configured to execute computer executable instructions; a memory storing the computer-executable instructions that, when executed by the processor, cause the home computing system to: sending configuration data about the home computing system, wherein the configuration data describes the home computing system; downloading a downloaded data analysis model based on the configuration data from the public computing cloud via the cloud interface; obtaining device data from the at least one IoT device via the communication gateway; applying the device data to the downloaded data analysis model; Obtaining a first prediction result from the data analysis model based on the device data; downloading new data analysis models from the public computing cloud when new IoT devices are added to the home computing system; Applying current model parameters to the new data analysis model; Initiating reinforcement learning of the new data analysis model; In response to the starting, obtaining new model parameters of the new data analysis model; and The new model parameters are reapplied to the new data analysis model.

16. A home computing system supporting at least one Internet of Things (IoT) device, the home computing system comprising: a communication gateway configured to connect to the at least one IoT device; a cloud interface configured to exchange information with a public computing cloud; a processor configured to execute computer executable instructions; a memory storing the computer-executable instructions that, when executed by the processor, cause the home computing system to: sending configuration data about the home computing system, wherein the configuration data describes the home computing system; downloading a downloaded data analysis model based on the configuration data from the public computing cloud via the cloud interface; obtaining device data from the at least one IoT device via the communication gateway; applying the device data to the downloaded data analysis model; Obtaining a first prediction result from the downloaded data analysis model based on the device data; initiating reinforcement learning at the home computing system based on the device data, the first prediction result, and the first correction information; obtaining first updated model parameters from the reinforcement learning; and The first updated model parameters are applied to the downloaded data analysis model to obtain an updated data analysis model.

17. A home computing system supporting at least one Internet of Things (IoT) device, the home computing system comprising: a communication gateway configured to connect to the at least one IoT device; a cloud interface configured to exchange information with a public computing cloud; a processor configured to execute computer executable instructions; a memory storing the computer-executable instructions that, when executed by the processor, cause the home computing system to: sending configuration data about the home computing system, wherein the configuration data describes the home computing system; downloading a downloaded data analysis model based on the configuration data from the public computing cloud via the cloud interface; obtaining device data from the at least one IoT device via the communication gateway; applying the device data to the downloaded data analysis model; Obtaining a first prediction result from the downloaded data analysis model based on the device data; performing, by the processor at the home computing system, reinforcement learning of the downloaded data analysis model; obtaining a first set of model parameters from the reinforcement learning at the home computing system; initiating assisted learning at the public computing cloud based on the device data, wherein the assisted learning is performed in parallel with executing the downloaded data analysis model at the home computing system; Obtaining a second set of model parameters from the auxiliary learning at the public computing cloud; and The third error measure of the reinforcement learning is compared with the fourth error measure of the auxiliary learning.

Images