Enhanced data protection for data generation devices

Abstract

The present invention describes apparatus / methods for enhanced data protection by determining the required level of data protection for a data source, determining at least a first data-generating device, and assigning protection scheme settings and parameters to the data generated by at least the first device. In one example, a data stream from a data-generating device that is designated as confidential / protected (e.g., a device that could potentially leak confidential information) is protected from reconstruction from unprotected data streams from one or more other data-generating devices that could be used to provide useful input to a virtual sensor of that confidential / protected sensor.

Description

[Technical Field]

[0001] The present invention relates to a data protection system applicable to sensors or other data-generating devices located in a local network, such as a home network or home automation system, a personal network, a sensor network, or an augmented reality / virtual reality (AR / VR) or metaverse application. The sensors may also be connected by a long-range network such as a 3GPP-based network or another long-range network such as a LoRA network. [Background technology]

[0002] Sensors are increasingly being used in local networks such as personal networks, home networks, and home automation systems, as well as in locations such as commercial, industrial, and medical facilities. They can measure not only temperature and light conditions, but also personal data such as respiratory rate, occupancy, identity, blood pressure (mobile sensors), and all kinds of other medical data and exercise patterns. Modern home networking interoperability systems like Matter protect this data with access control mechanisms that determine who (or which device) can access it. However, once access is granted to a third party, the data becomes uncontrollable. This is undesirable for users, especially for sensitive data. Therefore, it can be important for users to be able to manage their personal data even after access has been granted.

[0003] Furthermore, sensitive sensor data (e.g., microphone data that listens to user dialogue) can be reconstructed from "non-sensitive" sensors, such as inertial sensors worn on the body or head (whose data is uploaded, for example, to control a VR rendering engine), and sensitive and non-sensitive sensors can operate individually or together as virtual sensors. For example, inertial sensors can be used to determine what a user is saying (e.g., using a password, chatting with a friend). Therefore, it is desirable to prevent sensitive information from the first ("important" or "confidential") sensor from being reconstructed from a "data breach" of data from one or more other ("less important" or "less sensitive") sensors. In this context, a "data breach" can be defined as data becoming unintentionally available to a third party. [Overview of the project] [Problems that the invention aims to solve]

[0004] The objective of this invention is to provide enhanced data protection for sensitive (sensor) data. [Means for solving the problem]

[0005] Thus, apparatus, method, and computer program product according to embodiments described in the claims are provided.

[0006] According to the first embodiment (targeting the protection management unit (controller) of an access control system), an apparatus is provided, and the apparatus is Determine the required level of protection for the target data. Determine at least one device capable of generating target data, The first device and metadata associated with the first device are registered as protective control information. Based on protection control information, the protection scheme settings and associated parameters are configured to be assigned to the target data generated by the first device.

[0007] According to a second aspect (which applies to a protection system implemented in the access control layer of an access control system), the system is A protection control unit including the apparatus of the first embodiment, An interface for setting the right of the second device to use target data generated by the first device in the protection management unit, and Having at least one first device, The first device is configured to protect collected data and share protected data by setting a protection scheme and using the parameters. The protection management unit is configured to receive requests for usage rights, as well as the setting of protection schemes and the parameters thereof. The protection management unit provides the second device with usage rights and the setting of the protection scheme.

[0008] In the embodiment, the protection management unit may be located alongside the first device or be a function of the first device. Therefore, if the first device includes the protection management unit function, the right of use and the setting of the protection scheme are provided by the first device.

[0009] According to a third aspect (which concerns the protection procedures of the access control layer), the method is: The steps include determining the required level of protection for the target data, A step of determining at least a first device capable of generating target data, A step of registering the first device and associated metadata as protective control information, The process includes the step of assigning the protection scheme settings and related parameters to the target data generated by the first device, based on the protection control information.

[0010] According to the fourth aspect, a computer program product is provided, which, when executed on a computer device, includes coding means for generating steps of the method of the third aspect.

[0011] Therefore, the proposed solution allows for the selective provision of individual configurations of data protection schemes for different sensors, other data generating devices, or data sources of target data, and can involve the (predetermined) assignment of data usage rights to target data generated from sensors, other data generating devices, or data sources within a system, network, or single entity, and prevents these usage rights from being circumvented by the use of virtual sensors that exploit data correlations.

[0012] Furthermore, an overall access control logic can be configured to maintain the correct level of control over the use of all (real and virtual) sensors within the controlled system.

[0013] According to the first option, which can be combined with any of the first to fourth embodiments, with respect to data generated by the first device, usage rights can be assigned to the second device, and usage rights, protection scheme settings, and parameters can be provided to the second device.

[0014] As a result, it may be determined or ordered that some target data needs to be treated as important or confidential. Importance / confidentiality may be context-dependent (e.g., always or only at specific times, all locations or only in specific locations, depending on or independent of the user's status or health), and therefore, enhanced protection against unauthorized or uncontrolled data access or virtual sensor re-engagement of privacy-sensitive data is necessary. This is achieved by assigning appropriate permissions (user rights) to users of the target data to use the raw, unprotected / protected data, and to other users to use the data resulting from protective actions performed on the target data.

[0015] According to a second option that can be combined with the first option or any of the first to fourth aspects, the protection scheme can be based on at least one of digital rights management, multiparty computing, homomorphic encryption, attribute-based encryption, blind source separation, and data obfuscation. Thereby, based on an appropriate protection scheme, the proposed solution can be flexibly used in various applications.

[0016] According to a third option that can be combined with the first or second option, or any of the first to fourth aspects, the second device can be placed outside the network domain where the first device is located. Thereby, the proposed extended protection scheme can be provided to (sensor) devices within a given network domain to protect the supply of data to an external receiver.

[0017] According to a fourth option that can be combined with any one of the first to third options or any one of the first to fourth aspects, one or more of the following actions can be performed (e.g., by a protection management unit of an access control system): · Require a device or application to provide security functions and / or requirements; · Require a user, application or service to provide input regarding target data; · Require a user to provide input regarding the protection level required for any of the individual devices or the aggregated devices; · Disclose data from one or more physical devices or one or more virtual devices protected according to the settings of the protection scheme; · Determine the protection level of a virtual device based at least on the settings of the protection scheme and / or requirements based at least on the protection level requirements of the underlying physical device and the protection level of the consuming application. Instruct the first device to use a specific type of protection so that the data generated by the first device is protected during transmission from the first device to the second device.

[0018] As a result, the proposed enhanced data protection system can adapt to various applications and environments.

[0019] According to a fifth option that can be combined with any one of the first to fourth options or any one of the first to fourth aspects, data from the first device is received, and the received data stream of the first device can be protected, where access control layer and protection to the second device can be achieved by a given protection option, and feedback from the second device regarding the success and / or failure of the application of a specific type of protection can be received. As a result, the raw data received from the first device can be uniformly protected in the access control layer of the access control system, so that the protected data can be transferred to the second device. The access control layer can be easily described as a function for interpreting the right to use and is often part of an access control system.

[0020] According to a sixth option that can be combined with any one of the first to fifth options or any one of the first to fourth aspects, the protection scheme is a multi-party computing (MPC) protection scheme, where the protection key is shared in a key sharing scheme. As a result, there is no need to provide the relevant protection key, and access to the plain text data can be prevented.

[0021] According to a seventh option that can be combined with any one of the first to fifth options or any one of the first to fourth aspects, the protection scheme can be a homomorphic encryption method, where the first device obtains a public key and the second device obtains an evaluation key, whereby the target data can be processed in the encrypted area.

[0022] According to one of the first to fifth options, or an eighth option which can be combined with one of the first to fourth embodiments, the protection scheme can be an attribute-based encryption scheme, where a first device obtains a public key derived from an access control attribute that may grant access to the data, and uses it to encrypt the generated data; a second device obtains a private key that depends on its own attributes, and the second device can access the data if its access control attribute matches the attributes of the public key used to encrypt the data.

[0023] According to a ninth option which can be any one of the first to fifth options or combined with any one of the first to fourth embodiments, the protection scheme can be based on a symmetric key, where the first device obtains a symmetric key to protect the generated data, and the second device, if permitted, obtains the same symmetric key and can access the data if permitted.

[0024] According to a tenth option, which can be one of the first to ninth options or combined with any of the first to fourth embodiments, the required level of protection for the target data can be determined based on the combined data protection requirements of multiple first devices. This allows the proposed protection scheme to be used for multiple devices or virtual sensors derived therefrom. In the embodiment, the required level of protection can correspond to the maximum protection level of the first device.

[0025] According to an eleventh option which can be one of the first to tenth options or combined with any of the first to fourth embodiments, the proposed protection system (e.g., a protection management unit) is controlled by a DRM system and can register information about which sensors on the network are designated to generate DRM-protected data, and based on that information can register an access control list corresponding to the designated sensors, the rights define the conditions for a second device to access the sensor data, and the protection management unit is configured to provide the designated sensors with the DRM protection key in order to protect the sensor data using the protection scheme and the DRM protection key, the rights and DRM protection key are received from the second device and provided to the second device. This allows the proposed enhanced protection scheme to be combined with a DRM system.

[0026] It will be understood that the apparatus, method and system and computer program products of the independent claims may have similar and / or identical embodiments, particularly as defined in the dependent claims.

[0027] As an alternative approach that can be combined with the approach described above, a device is also provided to mitigate data leakage from data generation devices, which is configured to set a privacy policy, calculate a data protection filter to filter data generated by data generation devices, and apply the data filter to data generated by data generation devices in accordance with the privacy policy.

[0028] In one embodiment, the apparatus is configured to compute a data filter using blind source isolation.

[0029] In one embodiment, the data filter calculation involves training a virtual sensor model to reproduce the data output of a data generating device using the data output (310) of one or more other data generating devices, using the virtual sensor model to evaluate the impact of at least some of the data outputs of one or more other data generating devices (10) on the reproduced data output of a confidential data generating device, and determining, based on the evaluation using the virtual sensor model, whether one or more other data generating devices include confidential data generating devices that leak confidential information (confidentiality leaking) with respect to the confidential data generating device.

[0030] In one embodiment, the device is configured to determine a sensitive data generating device that needs to be protected.

[0031] In one embodiment, the device is further configured to initiate a protective filtering process against a determined sensitive data generating device if at least one sensitive data leaking device has been determined.

[0032] In one embodiment, the device is further configured to retrain a virtual sensor model based on the data output of a determined confidential data leakage generating device.

[0033] In one embodiment, the device is configured to train a virtual sensor model using time-synced output data from one or more other data-generating devices that have been collected over a specific period and stored in a data repository.

[0034] In one embodiment, the virtual sensor model is a linear regression model or a convolutional or transformer-based neural network model, which is trained with mean squared error at the desired output.

[0035] In one embodiment, the device is configured to apply a virtual sensor performance metric to generate a value indicating how well the data output of a sensitive data generation device is reproduced by a virtual sensor model.

[0036] In one embodiment, the device is configured to use a virtual sensor model to adversarially train a sensor filter model that converts the output data of a determined confidential data leakage generating device into new data that matches the output data of a confidential data generating device, while the sensor filter model is updated during this training, but the virtual sensor model is not.

[0037] In one embodiment, the device includes an access rights update function that assigns data permission levels to data generation devices abstracted using metadata, a protection provision initialization function that determines when to start the protection provision system for a specified data generation device, and a protection provision usage function that determines when to filter confidentiality leak sensors using the protection provision system (42).

[0038] In one embodiment, the protection provision system includes a sensor filter model application function for processing the output data of a determined confidential data leakage generation device using a sensor filter model and generating filtered data represented as the output of a virtual sensor.

[0039] In one embodiment, the device is configured to acquire a sensor filter model by blind source isolation.

[0040] In one embodiment, the device is configured to determine when a sensor filter model is applied based on a policy determined by the user, device vendor, system vendor, or system operator, and / or based on metadata of a sensitive data generating device.

[0041] In one embodiment, the policy triggers the use of a sensor filter model based on at least one of time, time range, confidentiality leak metric, location, and context.

[0042] In one embodiment, the device is configured to perform one or more of the following: mark a determined sensitive data generation device as leaked in metadata associated with the sensitive data generation device; perform a determination of leaked data generation devices and mark the data generation device as a leaked device in metadata with respect to the sensitive data generation device; generate a sensor filter model using training data from a data repository and a virtual sensor model from a model cache; and represent each filtered leaked output data obtained from the sensor filter model as a new virtual sensor marked as not leaked with respect to the sensitive data generation device, and indicate this in the associated metadata.

[0043] Furthermore, a device is provided to mitigate data leakage from the data generation device, which is configured to receive a data protection filter from the aforementioned device, and which is configured to filter the data generated by the data generation device and apply the data filter to the data generated by the data generation device.

[0044] In one embodiment, the system includes an apparatus (as claimed in any of the above claims) and a plurality of data generating devices.

[0045] Furthermore, a method is provided for mitigating data leakage from data generating devices, the method for identifying a sensitive data generating device that needs protection, training a virtual sensor model to reproduce the data output of the sensitive data generating device using the data output of one or more other data generating devices, using the virtual sensor model to evaluate the impact of at least some of the data outputs of the one or more data generating devices on the reproduced data output of the sensitive data generating device, and determining, based on the evaluation using the virtual sensor model, whether the one or more other data generating devices include a sensitive data leakage device with respect to the sensitive data generating device.

[0046] Furthermore, a computer program product is also provided that includes coding means for generating the steps described above when executed on a computer device.

[0047] In this embodiment, there is a wearable device that is worn by a user and configured to receive messages from a second device, the messages including notification data, the wearable device having a processor, the processor being configured to use security functions to apply security policies to the notification data, the security functions being configured to prevent at least a portion of the notification data from being transmitted to a third device.

[0048] Furthermore, there is a wearable device that is worn by a user and configured to receive messages from a second device, the messages including notification data, the wearable device having a display and a processor, the processor being configured to use security functions to apply a security policy to the notification data, the security policy preventing at least a portion of the notification data from being displayed on the display.

[0049] The security policy is configured to selectively prohibit the transmission of at least some of the notification data based on its classification.

[0050] Furthermore, there is a wearable device configured to be worn by a user, having a sensor and a processor, wherein the sensor is configured to measure the user's physical parameters and output corresponding measurement data, and the processor is configured to use a security function to apply a security policy to the measurement data, the security function is configured to prevent at least a portion of the measurement data from being transmitted to a third device.

[0051] The wearable device is configured to receive messages from a second device, which in turn allows the second device to set security policies.

[0052] A wearable device according to any of the prior claims has secure storage, which is configured to store security information, and the security function is configured to use cryptographic keys as part of the enforcement of a security policy.

[0053] The security policy is based on detecting proximity between wearable devices and data input devices.

[0054] The data entry device is at least one of the following: a smartphone, tablet, laptop, or wireless keyboard.

[0055] The transmission prohibition is stopped upon receipt of a command from the data input device, or by an instruction from the data input device indicating that the data input device is not being used to input confidential data.

[0056] There is a device configured to set up a wearable device, and that setting up includes setting up the wearable device's privacy policy.

[0057] Setting a privacy policy includes providing security information to smart wearable devices and identifying the types of data to which the privacy policy applies.

[0058] The device has a privacy policy, the device has an application program that generates data, the privacy policy can be configured to restrict the provision of data to a selected wearable device or a selected application program, and the selected application program is configured to run on the wearable device.

[0059] Furthermore, this can be summarized as a method to restrict the transmission of information from a wearable device, including setting a privacy policy for the wearable device by sending a message to the wearable device, the message including instructions for setting a privacy policy, the privacy policy being configured to selectively prohibit the wearable device from performing an action, the action being to display or transmit information received by the wearable device.

[0060] The method includes setting a privacy policy for a wearable device by sending a message to the wearable device, the message including instructions for setting the privacy policy, the privacy policy being configured to selectively prohibit the wearable device from performing an action, the action being the transmission of information generated by the wearable device from the wearable device.

[0061] The aforementioned configuration is performed by a second device and includes the receipt of security information from the second device by the wearable device, the security information being used by the wearable device to enable the execution of the aforementioned action.

[0062] It should also be understood that preferred embodiments may be any combination of the dependent claims or the embodiments described above and their respective independent claims.

[0063] These and other embodiments are evident and will be described by referring to the embodiments described below. [Brief explanation of the drawing]

[0064] [Figure 1] A schematic diagram showing block diagrams of data protection systems according to various embodiments. [Figure 2] A schematic diagram illustrating the flow charts of data protection systems according to various embodiments. [Figure 3] A schematic diagram illustrating the training process for the virtual sensor model and the sensor filter model. [Figure 4] A schematic diagram showing a block diagram of a data protection system according to an embodiment. [Figure 5] This diagram schematically shows the process flow for data protection based on the system in Figure 4. [Figure 6] A schematic diagram showing the flow chart of the data protection process according to the embodiment. [Modes for carrying out the invention]

[0065] The embodiments will be described based on a sensor network system. Such a system may be a home networking system or a sensor system for metaverse applications. They may also be installed in commercial, industrial, or other public spaces (such as hospitals). The following description is written from the perspective of network deployment, but note that the same operation can be applied within a single device.

[0066] Through the following disclosures, “Home Network” is understood as a network containing sensors and actuators that facilitate a specific task (e.g., lighting or healthcare-related). This may include a Home Network Hub (such as a data distribution entity) that is responsible for managing the Home Network and allows multiple devices or nodes (such as sensors and actuators) to connect to the network. Furthermore, the term “Home” is understood to mean “Local,” which includes other settings such as offices, factories, and hospitals. The Home Network Hub may also be an entity responsible for directing secure data distribution, such as data originating from the network. The Home Network Hub may include or provide access to a router device for linking the Home Network to an external network (e.g., the Internet), and / or allow devices to be added to or removed from the network.

[0067] Furthermore, a “virtual sensor” or other “virtual device” is understood as a virtual entity that can be instantiated by a type of software that processes what a physical sensor or other device processes. It is trained to interpret relationships between various variables and monitor readings from various instruments. Virtual sensing technology is used to provide a viable and economical alternative to expensive or impractical physical measuring instruments. A virtual sensing system uses information obtained from other measurements or process parameters to calculate an estimate of the quantity under consideration. These virtual devices can use the data to collect information that cannot be measured by a single device. In this way, information that cannot be directly measured can be obtained.

[0068] Furthermore, "user" or "homeowner" is understood to refer to the person who owns the home network, and in most cases, the person who owns the data collected / stored / generated by the home network's sensors / actuators.

[0069] For the purposes of this application, “usage rights” can be understood to include the right to access, copy or transmit, and control other actions over the data.

[0070] Furthermore, "protection key" is understood to refer to cryptographic key material that enables the secure sharing / processing of data.

[0071] Furthermore, the term "metaverse" is understood to refer to a collection of interactive spaces where users can interact with each other through mutually recognizable virtual features (i.e., augmented reality (AR)), or where the space is entirely composed of virtual features (i.e., virtual reality (VR)). VR and AR are sometimes commonly referred to as "mixed reality" (MR). Such interactive spaces can also be persistent.

[0072] Furthermore, the term “data” is understood to refer to a known or agreed-upon representation of information that is stored, transferred, or processed. The information may, in particular, include one or more channels of audio, video, images, haptic, motion, or other forms of information or environmental / personal characteristics (e.g., temperature, heart rate) that may be synchronized and derived from sensors (e.g., microphones, cameras, motion detectors, etc.).

[0073] Furthermore, the term "registration" is understood to refer to the process of entering or recording an item in a list or directory.

[0074] Please note that only the blocks, components, and / or devices related to the proposed data distribution functionality are shown in the accompanying drawings. Other blocks have been omitted for brevity. Furthermore, blocks designated with the same reference number are intended to have the same or at least similar functionality, and therefore their functionality will not be described again later.

[0075] For example, Digital Rights Management (DRM) systems can control data throughout its lifecycle through encryption and data that associates rights describing the terms of use. DRM systems themselves are known for protecting valuable audiovisual content, particularly from music and film companies. However, using DRM systems to protect sensor data from home networks, for instance, is difficult. One reason for this is that the data generated by sensors is not specially created, corporate-distributed data. Rather, it is personal data related to the user, automatically generated (often without their knowledge), and provided on request by service providers in the background. This data can be extremely privacy-sensitive and therefore requires considerable care.

[0076] Over the past few years, the smart home market has grown significantly. In a market once dominated by tech-savvy early adopters, the World Economic Forum estimates that by 2022, more than 130 million households will own smart home devices. Matter is a smart home standard created in 2019 by Project Connected Home Over IP (Project Chip). It was officially launched in November 2022 and is maintained by the Connectivity Standards Alliance (CSA), formerly known as the Zigbee Alliance. The standard encourages interoperability between devices and platforms, allowing devices to operate offline without continuous access to the cloud or various cloud services.

[0077] Some modern home networking systems, such as Matter, use Access Control Lists (ACLs) to manage access to data at nodes and endpoints. They indicate what each device can do on other devices (nodes), such as reading or writing data. In essence, such systems control usage rights. Specifically, the access control function aims to ensure, through the interaction model, that only authorized nodes are allowed access to specific application layer functions exposed by the data model. Access control is the fundamental link between the secure channel and the interaction model. To implement access control policies, the administration generates and maintains a consistent, distributed configuration of ACLs across all nodes. Each node has an ACL containing access control entries that structure the policies. The access control cluster exposes a data model view of the node ACLs, enabling maintenance.

[0078] As a result, access control architectures must consider whether the data remains within a compliant group, is therefore protected, and resides within the owner's home. However, it may be assumed that certain sensor data is used to provide services to the owner, allowing the owner to have better control over which devices and / or entities inside and outside the home are accessing data generated by other devices. Such services may include health monitoring, presence detection, lighting control, heating, ventilation, and air conditioning (HVAC) control. It may also be assumed that certain devices may perform specific actions based on data input from other devices without actually accessing that data. Furthermore, it may be assumed that data generated at the user's location may be analyzed / processed in the cloud, which is desirable if the cloud does not have access to the data. Finally, it may be assumed that certain devices can only access data if their attributes or context permit it. In these situations, access control architectures may be insufficient.

[0079] Furthermore, data collected by sensors may leak information from other sensors. For example, accelerometer measurements may leak information collected by microphones. Thus, access control architectures that limit access rights to target devices without considering related equipment may be insufficient and fail to prevent information leakage from the target device.

[0080] Furthermore, data (e.g., data derived from measurements) may be presented to the user or application in an aggregated / processed form. For example, if there are multiple light sensors in a room, the user may be interested in knowing the aggregated light value of the room (e.g., average light intensity) rather than the light readings (intensity, color, etc.) of each individual light sensor. For example, given several physical sensors / actuators, it is possible to create virtual sensors / actuators, for example, a virtual actuator formed by multiple actuators, or a virtual sensor that provides sensing output based on measurements from one or more physical sensors. In such situations, an access control architecture that focuses solely on physical sensors / actuators may be insufficient.

[0081] Generally, home data is valuable and privacy-sensitive, and homeowners (or users) may want control over data usage (e.g., which devices / entities can access data generated for what purpose). They may want to ensure that data processors have restricted access to their data as they process it. They may want a device to be able to use data generated by other devices, but not be able to access the original plaintext data. They will want to prevent data leaks from confidentiality (conflict) sensors (such as virtual sensors / actuators). They may want data from virtual sensors / actuators to be exposed only to authorized entities.

[0082] This can be achieved, for example, by applying different techniques based on DRM, homomorphic encryption, multi-party computation, attribute-based encryption, blind source isolation (BSS), or data obfuscation (as will be described later in relation to the embodiments in Figures 3 to 6).

[0083] An example of using BSS is when a second device receives a configuration to perform blind source isolation on data sampled by the second device (which contains some information about the data sampled by the first device), where the BSS operation is intended to remove information related to the first device.

[0084] The second device could be a smartphone, and the first device could be another sensor. In another scenario, the first device is integrated into the same housing as the first device (such as a sensor within the smartphone), in which case the second device could be an app on the smartphone. The second device could also be a software program running remotely, for example, implemented in a cloud-based system.

[0085] The following embodiments address the adaptation of home network-generated data (e.g., sensor data) protection and access control architectures throughout the data lifetime.

[0086] Figure 1 schematically shows block diagrams of data protection systems according to various embodiments.

[0087] At least a first sensor (S1) 10 and a second sensor (S2) 12 are used to generate data, where the “first sensor” is understood as a data-generating device (e.g., within a home network) that can sense, for example, light, presence, sound, images, etc., and whose data requires protection (it may also leak data detected by the second sensor). Furthermore, the “second sensor” is understood as a data-generating device (e.g., within a home network) that can sense, for example, light, presence, sound, images, etc.

[0088] Furthermore, a receiving device (Rx) 50 is provided, which is understood to be a data consumption device both inside and outside the home network.

[0089] Furthermore, the protection management unit or manager (PA) 32 of the access control system (ACS) 30 is understood to refer to an entity (e.g., a processor) that provides data protection functions for processing / exchanging data generated / detected / processed by the home network or (smart) data generation devices (e.g., the first and second sensors 10, 12).

[0090] The protection management unit 32 is configured to communicate with, or access to or control, a protection control function or entity (PC) 40, and is understood to refer to a protection scheme or unit that can provide data protection when data needs to be shared with or processed by the receiving device 50.

[0091] Furthermore, the access control system 30 can be configured to provide a user interface (UI) 34 that allows the user to communicate with the protection management unit 32 to set up the necessary protection controls.

[0092] In one embodiment, the protection management unit 32 may include a user interface 34 used to set protection rights over sensor data.

[0093] The protection management unit 32 may be an entity operating in the access control layer of the protocol used by the access control system 30. The access control layer (e.g., within the device or within the home network) provides at least one of the following: data protection functions for managing data sources (e.g., sensors within the device or first and second sensors 10 within the home network), protection requirements for those data sources (data generating devices), and access rights for different receivers (e.g., receiver 50) to those data sources.

[0094] Furthermore, a user interface 34 can be provided so that the user can control the aspects of the access control layer and configure the necessary or desired protection controls.

[0095] In the embodiment, as will be described later in relation to the embodiment shown in Figures 3 to 6, data protection can be achieved by providing parameters to prevent leakage of data collected by the second sensor 12 through the first sensor 10.

[0096] In one embodiment, the protection management unit 32 may be located in a network device such as a network hub responsible for managing the home network. Alternatively, the protection management unit 32 may also be located in a smart device such as a smartphone, tablet, or AR / VR glasses. In such an embodiment, the user interface (UI) 34 is part of an access control UE, such as a network hub or smart device, and enables the setting and / or configuration of desired security protections.

[0097] In this embodiment, the protection management unit 32 and / or the protection control function or entity (PC) 40 may be a distributed entity, and its function may reside within a network device such as a network hub and / or a first sensor (S1) 10 and / or a second sensor (S2) and / or a receiving device (Rx) 50.

[0098] In this embodiment, the functions of the protection management unit 32 may overlap with, for example, the functions of an entity that manages access control. This entity may be an organization within a network that tracks the voluntary usage rights provided by a user.

[0099] Figure 2 schematically shows a flowchart of the data protection process according to various embodiments that can be implemented in the data protection system of Figure 1.

[0100] Please note that not all steps in Figure 2 are always necessary. Furthermore, some or all steps can be performed once or multiple times, and in different orders.

[0101] In step S201, the homeowner can, for example, indicate to the protection management unit 32 via the user interface 34 that the sensor data must be distributed (only) under a given protection control, such as an access control list (ACL) or data protection list (DPL), and that they have the right to use this sensor data. The protection control applied by the protection control unit 40 can be at least one of DRM, fully homomorphic encryption ((F)HE), multi-party computation (MPC), blind source isolation, and data obfuscation.

[0102] In step S202, the first sensor 10 can acquire (e.g., receive and / or generate) a data protection parameter (PP) which may include a protection key. The protection key may be received from the protection management unit 32 or generated by the sensor (e.g., the first and / or second sensors 10, 12).

[0103] In this embodiment, the protection key may be different from the group key used within the authorized group to keep the data secure.

[0104] In step S203, the first sensor 10 can apply data protection parameters and settings to the generated data.

[0105] If the receiver 50 (for example, a sensor / actuator within the home network or cloud service, or a sensor / actuator outside the home network) requires data generated by the sensor (for example, the first sensor 10 or the second sensor 12), the receiver 50 can send a data request (D-REQ) in step S204. Alternatively, the homeowner can subscribe to a service or add devices that require data sharing to the network.

[0106] In step S205, the protection management unit 32 can share / transmit the protection scheme (PS) for the requested data to the receiver 50.

[0107] In step S206, the receiver 50 may respond to the notified protection scheme (RESP). If the receiver 50 agrees to the intended “protected delivery”, it obtains the data protected by the protection key and, if applicable, the right to use it. If the receiver 50 wishes to access / process the data, it may request the necessary right to use and / or the associated protection key from the protection management unit 32. Some of these substeps may be performed simultaneously.

[0108] In step S207, the protection management unit 32 generates an access right (AR) that includes the relevant protection key (or multiple keys) and transmits it to the receiver 50. This information can be transmitted securely to the relevant sensors using, for example, a security infrastructure provided by an access control layer. Then, in step S208, the sensor (for example, the first sensor 10) retrieves the protected data (D PR ) and usage rights are provided to receiver 50, which may be outside the network. Alternatively, usage rights may be transferred separately by the protection management unit 32.

[0109] Finally, in step S209, the receiver 50 uses the protected data and associated protected key to perform, for example, decryption or processing (DCR) in order to access the protected data.

[0110] The following embodiments relate to the details and / or modifications of the protection management unit 32 and access control layer of the access control system 30 shown in Figure 1.

[0111] In this embodiment, the access control layer may be based on a computer program that runs on a device (e.g., a processor) and, for each device in the network, depends on the type of device (e.g., virtual / physical device, sensor / actuator), the type of data generated / used, the device components (e.g., the physical devices that make up the device), the device functions for protecting the data, the entities permitted to access the data generated by the device, and a data structure (e.g., a database) used to store at least one of the types of protection required when sharing device data (e.g., depending on the data-generating device in question).

[0112] In one embodiment, the access control system 30 can be configured to run the access control layer on a single master device (e.g., a home network hub). In this case, the master device can collect data from different devices (e.g., within the home network) and apply protection before exposing the data (e.g., outside the home network). Data collection can be performed securely by relying on security protocols used within the home network. This may also apply to a single smart device (such as a smartphone) rather than a home network, where the access control layer protects data from the smart device's sensors before it is exposed to applications, for example. If the access control layer runs on multiple devices (such as a home network hub or end devices within the home network), the master device can be responsible for orchestrating data protection, instructing end devices to apply specific protections, or configuring end devices to use specific protections.

[0113] In this embodiment, the access control layer of the access control system 30 can perform one or more of the following actions: • Request (the device) to provide security features (such as applicable protection technologies) and / or requirements; For example, requesting (the user / application / service / ...) to provide input regarding the necessary data in order to enable the application; • Request (e.g., the user) to provide input regarding the required level of protection for either individual devices or a collection of devices; • Expose data from one or more physical or virtual devices (e.g., sensors) that are protected according to the configured protection options; • Determine the protection level of the (virtual) device based on the protection level set at least on the protection level requirements of the underlying device (e.g., the (virtual) device component) and the protection level of the consuming application, and / or based on the requirements; • Instruct the device to use a specific type of protection to ensure that data generated by the device is protected during transmission from the device to the target device; • Receive data from a device (for example, in a home network where end-to-end protection can be based on link keys, providing end-to-end protection from the device to the access control layer) and protect the received data (stream) from the device (on behalf of the device) (protection from the access control layer to the target device is achieved by a given protection option); • Receive feedback from end devices regarding the success / failure of applying specific types of protection.

[0114] In DRM-based embodiments, a DRM protection scheme is used, and the protection key can be an encryption key (e.g., a symmetric key) used for protection (e.g., encryption or integrity protection). Encryption can be implemented by an encryption algorithm such as the Advanced Encryption Standard (AES) in a specific block mode or stream mode. For performance reasons, protection can be applied only to specific portions of the data. AES is based on a design principle known as a permutation permutation network and is efficient in both software and hardware. Unlike the earlier Data Encryption Standard (DES), AES does not use a Feistel network. AES is a variant of Rijndael, with a fixed block size of 128 bits and key sizes of 128, 192, or 256 bits. According to this embodiment, integrity protection can be achieved by a message integrity code. In this case, the associated protection key can be the protection key itself used for decryption and integrity verification. Other encryption schemes can also be used, such as Ascon, a family of authenticated encryption and hashing algorithms designed to be lightweight and easy to implement even with added countermeasures against side-channel attacks. A DRM system typically evaluates the right of use to determine whether a protective key can be used. This protective key is then paired with the right of use.

[0115] In (F)HE-based embodiments, an (F)HE protection scheme is used, where the protection key is a public key and the associated protection key can be an evaluation key. Homomorphic encryption is a form of encryption that allows computations to be performed on encrypted data without first decrypting it. The computation results are left in encrypted form and, upon decryption, produce the same output as if the operation had been performed on unencrypted data. Homomorphic encryption (HE) can be used for outsourced storage and computation that protects privacy. This allows data to be encrypted and outsourced for processing on devices or commercial cloud environments while remaining encrypted. For sensitive data such as medical information, HE can be used to remove privacy barriers that hinder data sharing or to enhance the security of existing services, enabling new services. For example, a predictive analytics service provider can compute encrypted data to mitigate privacy concerns. Fully homomorphic encryption (FHE) is an encryption scheme that allows analytical functions to be run directly on encrypted data while producing the same encrypted results as if the function had been run on plaintext. This allows evaluation of arbitrary circuits consisting of multiple types of gates of infinite depth and is the most powerful concept of homomorphic encryption.

[0116] In this embodiment, the sensor can obtain a public key, and the data processor (e.g., a first target device) can obtain an evaluation key, and the data is processed in an encrypted domain. The owner can hold a private key and share it with a second target device so that the second target device can decrypt the results provided by the first target device. The owner can hold / use the private key to retrieve the processed data.

[0117] As a further example, a sensor can obtain a public key and use it to protect data. A device receiving the data can obtain an evaluation key that allows it to make decisions based on the data without accessing the data itself. For example, a receiving device can determine whether the received data is greater than a threshold (a situation that can trigger an alarm).

[0118] Here are some examples of encryption schemes using homomorphic and fully homomorphic encryption. - Paillier cryptosystem: This is a public-key cryptography scheme that supports homomorphic addition of ciphertexts. Given two ciphertexts c1 and c2 that encrypt plaintexts m1 and m2 with the same public key, it is possible to compute the ciphertext c3 = c1 * c2 mod n^2 and encrypt m3 = m1 + m2 mod n (where n is part of the public key). The Paillier cryptosystem is based on the difficulty of the composite remainder determination problem and requires the selection of two large prime numbers p and q as parameters for generating the public and private keys. - BGV Cryptographic System: This is a lattice-based cryptographic scheme that is fully homomorphic, as it supports both homomorphic addition and multiplication of ciphertexts. The BGV cryptographic system uses a polynomial ring R_q = Z_q[x] / (x^n + 1) as the message space, where q is a large modulus and n is a power of 2. This scheme also performs message encryption and decryption using a smaller modulus t and noise distribution. The BGV cryptographic system allows arbitrary computation on encrypted data by using a bootstrap technique that updates the ciphertext after each homomorphic operation to reduce the noise level. In the BGV cryptographic system, several parameters such as moduls q and t, order n, noise distribution, and security parameters must be selected to ensure the correctness and security of the scheme. - CKKS Cryptographic System: This is another lattice-based encryption scheme that supports both homomorphic addition and multiplication of ciphertexts and is therefore fully homomorphic. The CKKS cryptographic system differs from the BGV cryptographic system in that it operates with approximations rather than exact arithmetic and can encrypt complex numbers rather than integers. The CKKS cryptographic system uses a polynomial ring R_q similar to the BGV cryptographic system, but uses different encoding and decryption procedures to map complex numbers to polynomials and vice versa. The CKKS cryptographic system also performs arbitrary calculations on encrypted data using bootstrap techniques, but some approximation errors occur in the process. The CKKS cryptographic system requires the selection of parameters similar to those for the BGV cryptographic system, but also requires specifying the precision and scale of the desired calculations.

[0119] The advantages and disadvantages of these schemes are as follows: - Paillier cryptographic system: The advantages of this scheme are its relative simplicity and efficiency, and its large message space. The disadvantage of this scheme is that it is not fully homomorphic, as it only supports homomorphic addition and not multiplication. Another drawback is that it depends on the difficulty of the number theory problem that could potentially be solved by a quantum computer. - BGV encryption system: The advantages of this scheme are that it is fully homomorphic and has a high level of security based on the difficulty of the lattice problem, which is considered quantum resistant. The disadvantages of this scheme are that it is more complex and less efficient than the Paillier encryption system and has a smaller message space. Another disadvantage is that bootstrap techniques are required to update the ciphertext, which increases computational overhead and noise. - CKKS cryptographic system: The advantages of this scheme are that it is fully homomorphic, can encrypt complex numbers, and is useful for applications such as machine learning and signal processing. The disadvantage of this scheme is that it is based on approximate arithmetic, so some errors can occur in the calculations. Another disadvantage is that bootstrap techniques are also required to update the ciphertext, which increases computational overhead and noise.

[0120] In MPC-based embodiments, an MPC protection scheme may be used, and the protection key may be shared, for example, in a key-sharing scheme, but the associated protection key is not provided. MPC is a subfield of cryptography, aiming to create a method in which parties jointly compute a function on inputs while keeping those inputs confidential. Unlike traditional cryptographic tasks where the cryptographic technique guarantees the security and integrity of communications and storage, and adversaries (those eavesdropping on senders and receivers) exist outside the participants' systems, the cryptographic technique in this model protects the privacy of the participants from each other.

[0121] In this embodiment, assume that an actuator (e.g., a robot) wants to navigate within a smart environment based on views from multiple smart cameras. In this case, the protection management unit 32 can configure the actuator using parameters and protection schemes (i.e., and MPC schemes). Sensors (such as cameras) sense and exchange protection data configured, for example, in an MPC scheme. The actuator can navigate using MPC-protected data shared by the cameras without accessing plaintext data.

[0122] The MPC scheme is a protocol that allows multiple parties to collaboratively compute a function on private inputs without exposing them to each other. MPC schemes can be classified into two main types: secret-sharing based and garbled circuit based. Secret-sharing based schemes divide each input into random assignments distributed among the parties and perform arithmetic operations on those assignments. Garbled circuit based schemes encode each input as a set of encrypted wires used to construct a Boolean circuit representing the function to be computed. The parties then exchange the encrypted wires and evaluate the circuit.

[0123] One example of a secret-sharing-based scheme is Shamir's secret-sharing scheme, which divides a secret into n assignments such that the secret can be reconstructed with any k assignments, but nothing is revealed with fewer than k assignments. Using this scheme, each sensor can share inputs with other sensors, and data from competing sensors can be protected by calculating a function on the assignment, such as the mean or median of the inputs, using a secure protocol. The results can then be reconstructed by any subset of the k sensors.

[0124] One example of a garbled circuit-based scheme is Yao's garbled circuit, which allows two parties to securely compute an arbitrary two-input function. One party, called the garbler, creates a garbled circuit by encrypting each wire of the circuit with a random key, and sends the circuit and the keys corresponding to its inputs to the other party, called the evaluator. The evaluator uses a technique called lost communication to retrieve the keys corresponding to its inputs from the garbler and evaluates the circuit by decrypting one output line for each gate. The results are then revealed to both parties.

[0125] An example of a scenario where a garbled circuit-based scheme can be used to protect data from conflicting sensors is when a user authenticates to a system using biometric data such as fingerprints or facial scans, but does not want to expose that biometric data to the system or other users. In this case, the user can act as a garbler, creating a garbled circuit that implements a matching function between the biometric data and a stored template. The system can act as an evaluator, using lost transmission to retrieve the key corresponding to the template and evaluate the circuit to determine whether the user is authorized. The result can then be sent back to the user.

[0126] More specific embodiments for protecting data from competing (adversarial) sensors against the target sensor are described below with reference to Figures 3 to 6, and at least one of the concepts described below can be applied.

[0127] If sensor outputs are correlated, training a virtual sensor allows the output of one sensor to be predicted from the outputs of other (correlated) sensors (see, for example, Andrea Brunello et al.: “Virtual Sensing and Sensors Selection for Efficient Temperature Monitoring in Indoor Environments”, Sensors 2021 (https: / / doi.org / 10.3390 / s21082728)). For example, motion sensor data from a VR headset acting as a virtual sensor can be used to infer data generated by the user's microphone (e.g., voice passwords) (see Cong Shi et al.: “Face-Mic: inferring live speech and speaker identity via subtle facial dynamics captured by AR / VR motion sensors”, MobiCom '21: Proceedings of the 27th Annual International Conference on Mobile Computing and Networking, October 2021 (DOI: 10.1145 / 3447993.3483272)).

[0128] Furthermore, a model input importance determination method for determining how much a particular feature (input) contributes to the neural network's decision has been proposed, for example, by Runjin Chen et al.: “Explaining Neural Networks Semantically and Quantitatively”, 18 December 2018. Knowledge encoded in a convolutional neural network (CNN) can be explained quantitatively and semantically, distilled into an explainable additive model, and this explainable model can be used to provide a quantitative explanation of CNN predictions. Typical bias interpretation problems of explainable models are analyzed, and prior losses are developed to guide the training of explainable additive models.

[0129] Furthermore, Generative Adversarial Networks (GANs) are a recent innovation in machine learning. GANs are generative models that create new data instances that resemble the training data. For example, a GAN can create an image that looks like a photograph of a human face, even though the face is not that of a real person. GANs achieve this level of realism by combining a generator that is trained to produce a target output with a discriminator that is trained to distinguish true data from the generator's output. The generator tries to fool the discriminator, and the discriminator tries not to be fooled.

[0130] For example, adversarial training used in GANs involves training a discriminator to train a model. This model is then used to train another network to output data that prevents the discriminator from making a correct decision on its output (for example, by propagating the gradient backward) (see Jason Brownlee: “A Gentle Introduction to Generative Adversarial Networks (GANs)” (https: / / machinelearningmastery.com / what-are-generative-adversarial-networks-gans / ) or Young-Bum Kim et al.: “Adversarial Adaptation of Synthetic or Stale Data”, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), July 2017 (DOI: 10.18653 / v1 / P17-1119)). This can be achieved, for example, by improving the output of a previously poorly performing generator, or by preventing the generator from outputting data that would help it make a correct decision.

[0131] Furthermore, least squares generative networks (LS-GANs) can be made to work with discriminators that have mean squares error loss (instead of using binary labels and cross-entropy loss) (see Xudong Mao et al.: “Least Squares Generative Adversarial Networks”, 2017 (DOI: 1611.04076v3)). While unsupervised training using GANs has proven highly successful, typical GANs assume that the discriminator is a classifier with a sigmoid cross-entropy loss function. However, this loss function can lead to the vanishing gradient problem during the training process. In machine learning, the vanishing gradient problem is encountered when training artificial neural networks using gradient-based learning methods and backpropagation. In such methods, during each iteration of training, each weight of the neural network receives an update proportional to the partial derivative of the error function with respect to the current weight. The problem is that, in some cases, the gradient vanishes and becomes small enough that changes in the weight values ​​are effectively prevented. In the worst case, the neural network may stop training altogether. One example of the cause of the problem is that conventional activation functions, such as the hyperbolic tangent function, have gradients within a range, and backpropagation calculates the gradients by the chain law. This has the effect of multiplying these small numbers by n to calculate the gradients of the initial layers of an n-layer network, and the gradients (error signals) decrease exponentially with n, so the initial layers are trained very slowly.

[0132] To overcome these problems, LS-GAN employs a least-squares loss function in its classifier. LSGAN has two advantages over regular GANs. First, LSGAN can generate higher-quality images than regular GANs. Second, LSGAN operates in a more stable manner during the training process.

[0133] A specific problem that arises in sensor networks stems from the fact that highly sensitive sensor data (such as microphones that listen to user interactions) can be reconstructed from "non-sensitive" sensors, such as inertial sensors worn on the body or head (whose data is uploaded, for example, to control a VR rendering engine), either individually or together as virtual sensors. Therefore, for example, inertial sensors could be used to determine what a user is saying, such as using a password or chatting with a friend. Thus, it is desirable to prevent sensitive information from "critical" sensors from being reconstructed from "data leaks" from data of other "less critical" sensors.

[0134] The following embodiments provide solutions to the specific problems described above by at least one of the following: Determine that certain sensor data should be treated as important or confidential (either always or only at specific points in time), thereby gaining protection against the leakage of personal information (e.g., virtual sensor reconstruction of privacy-sensitive data); In practice, it is determined that private and sensitive data (e.g., data that might be detected by the first sensor 10 in Figure 1) can be reconstructed from data from at least a second (irrelevant / insignificant) sensor (e.g., the second sensor 12 in Figure 1) using a virtual sensor; Identify which second sensor is necessary for the operation of such a virtual sensor; Because such virtual sensors could be built or used by malicious actors, their operation must be protected; Users of related data should be granted appropriate permissions to use the unprotected raw data, while other users should be granted appropriate permissions to use the data obtained as a result of the protection processes performed on that data.

[0135] The embodiment provides access control (AC) and data authorization processes to grant appropriate data authorization to abstracted sensors (actual and virtual) and provides data protection means to ensure that data streams from sensors designated as confidential / protected (those that may leak confidential information, etc.) are protected from reconstruction from unprotected data streams from one or more other sensors that may be used to provide useful input to at least one virtual sensor for said confidential / protected sensor.

[0136] Furthermore, embodiments can be configured to determine and provide appropriate data authorizations for actual and virtual sensors. For example, if a virtual device, such as a virtual sensor, is instantiated within a network, it is necessary to determine access permissions / data authorizations for that virtual sensor. If its output approximates that of a confidential / protected actual sensor, it should, in principle, be granted similar authorizations to that sensor (because it would be disclosing the confidentiality of that sensor).

[0137] In the embodiments described herein, confidentiality leak sensors and confidentiality / protection sensors are shown as actual sensors, but it should be noted that they themselves may be virtual sensors, and the proposed protective measures may also be applied to their outputs.

[0138] As initial access control, the access control system (e.g., access control system 30 in Figure 1) can, through some security settings, optionally instructed or confirmed by the user, configure or label a sensor as confidential / protected. This allows data rights to be established for that sensor (and virtual sensor), and for example, restricts which applications (e.g., software apps) can access that data. This can be set permanently, or the access control system can allow specific sensors to be marked as confidential / protected only at specific times or under specific circumstances. Labeling such as confidential / protected also enables virtual sensor protection for that sensor, for example, through the process described below. In the example, this approach can be implemented in the context of various operating systems (Android, iOS, etc.), such as when Android or iOS is configured to natively implement Matter.

[0139] In the embodiment, virtual sensor model generation can be applied, where, for each confidential / protected sensor, a proposed protection process initiated by the access control system can inspect all other sensors in the device / network (individually or collectively). The necessary information can be obtained from a database or central repository. The proposed protection process then determines to what extent the inspected sensors can reproduce data from the confidential / protected sensor. This can be achieved, for example, by constructing a predictive model (e.g., a virtual sensor that mimics the confidential / protected sensor) that attempts to reconstruct the confidential / protected sensor output from data supplied by (all) other local sensors (excluding the confidential / protected sensor). This model can be applied to individual sensors or combinations of sensors (e.g., receiving input from them).

[0140] In an embodiment, the construction of a virtual sensor model may be based on training with stored time-matched sensor data (e.g., from all sensors) collected over a specific period and stored in a data repository. In an embodiment, the virtual sensor model may be trained to attempt to reconstruct (as output) data from a selected sensitive / protected sensor from other sensor data (as input). The virtual sensor model may be, for example, a convolutional or transformer-based neural network model trained with mean squared error (MSE) for the desired output, or a very simple model such as a linear regression model.

[0141] The complexity of the virtual sensor model implementation can be adapted to the available computational conditions, as much of the benefits of this concept can be obtained from simpler models, even if more complex models are not necessarily better.

[0142] Edge artificial intelligence (Edge AI) is the implementation of artificial intelligence in an edge computing environment. This means that AI computations can be performed not in centralized cloud computing facilities or offsite data centers, but at the edge of a specific network, typically on the device where the data is created. Given the availability of edge AI chips, training relatively small neural networks is relatively quick and easy, and this also applies to the proposed filtering model. For example, training a virtual sensor model and a filtering model can be done in a single process. This can be based on a continuous online training process employed in either of the models.

[0143] In embodiments, individual other sensors that leak the confidentiality of the confidential / protected sensor can be determined by quantitative measurements with respect to the virtual sensor model, for example, by using model input importance determination to select a subset of sensors that contribute to the performance capabilities of the virtual sensor model. These sensors are marked as "confidentiality leaks" with respect to the confidential / protected sensor, and the virtual sensor model can optionally be retrained using only the outputs of these sensors as input. In embodiments, this concept can be applied to a user interface of an access control system (e.g., user interface 34 in Figure 1) or a privacy user interface to prevent privacy leaks from third sensors and / or to track potentially leaked data.

[0144] In the embodiment, a virtual sensor performance metric for evaluating sensor confidentiality leakage is obtained by analyzing a virtual sensor model to generate a value indicating the extent to which the confidential / protected sensor is mimicked / confidentially leaked by this virtual sensor model. The sensor performance metric can be based on a simple measurement of model performance using a test set (such as MSE), its output compared to the ground truth output, or the minimum MSE found in a sliding time data window, etc. Furthermore, or alternatively, the sensor performance metric can be a functional measure, such as extracting words from utterances into a microphone. Then, a speech recognition algorithm can be applied to the original protected microphone data and the virtual sensor data to compare word error rates. If the word error rate of the virtual sensor is low and close to the actual data, the microphone is confidentially leaked by the virtual sensor.

[0145] In the embodiment, the obtained sensor performance metric can be converted to a numerical value between, for example, 0 (e.g., no confidentiality leak) and 1 (e.g., complete confidentiality leak). If the merit value is sufficiently high, for example, exceeding a predefined threshold, there is a high risk that information from the protected sensor may be generated by a malicious party from other confidentiality leak sensors. In this case, a protection filtering process can be initiated for the relevant confidentiality / protection. Otherwise, no action may be taken on the confidentiality / protection sensor, and no protection measures may be employed. In the embodiment, the user interface and / or privacy UI of the access control system can be configured to configure these functions, where relevant information can be exchanged from the controller (e.g., the protection management unit 32 in Figure 1) to the devices (e.g., the first and / or second sensors 10, 12 in Figure 1).

[0146] Figure 3 schematically illustrates the training process for the virtual sensor model and the sensor filter model.

[0147] VR device 300 (e.g., VR headset or VR goggles) uses microphone data D M Microphone 312 generates a time series of data, and inertial sensor data D IS It includes an inertial sensor 310 that generates a time series of inertial sensor data D. IS This is the target of the virtual sensor model (VSM) 320 of microphone 312, and the model output data is the actual microphone data D M The reconstruction loss (RL) is determined by comparing it with, for example, applying the sensor performance metrics described above. Based on the comparison results, the metric-related gradients are backpropagated and the virtual sensor model 320 is updated.

[0148] Furthermore, inertial sensor data D IS This is supplied to the sensor filter model (SFM) 330, and the sensor filter model output data and inertial sensor data D ISThe reconstruction loss (RL) between them is determined, for example, by applying the above sensor performance metrics. Based on the comparison results, the metric-related gradients are backpropagated and the sensor filter model 330 is updated.

[0149] Furthermore, the sensor filter model output data is supplied to the virtual sensor model (VSM) 320 while it is not updated. In the output of the virtual sensor model 320, the adversarial reconstruction loss (ARL) is determined, for example, by applying the above sensor performance metrics. Based on the comparison results, the metric-related gradients are backpropagated to the sensor filter model 330 and the sensor filter model 330 is updated.

[0150] Therefore, the virtual sensor model 320 is used to adversarially train the sensor filter model 330, and the sensor filter model 330 converts the relevant set of data D of the (confidentiality-leaking) sensor into new data that maximally matches the original sensor data D IS while at the same time not being able to be used to reconstruct the sensor data D IS confidential / protected by the virtual sensor model 320. That is, the inertial sensor data D M is converted to minimize the distortion of the original inertial sensor data D IS while minimizing the performance degradation of the virtual sensor. IS

[0151] The sensor filter model 330 has the same number of outputs as the number of inputs (one for each confidentiality-leaking sensor), and thus can operate as a modified autoencoder (see Nagwa Elaraby et al.: "Large scale sensor data processing based on deep stacked autoencoder network", Journal of Theoretical and Applied Information Technology, November 2017 (DOI: 95(21):5907-5923)).

[0152] A sensor filter model, whose input is a set of sensitive leaked sensor data, can be trained to optimally reconstruct its input at its output and further prevent it from reconstructing the target sensitive sensor data, using a virtual sensor model 320 as a least-squares adversarial discriminator. This training uses the virtual sensor model acting as a discriminator that generates an adversarial loss (for example, by learning to output a value that does not match the expected output of a virtual sensor operating on an unmodified input) as the loss, and balances this with the least-squares reconstruction loss of the input of the sensor filter model 330 at its output.

[0153] To clarify, the two losses that are balanced during training are as follows: 1. Reconstruction loss by reproducing the original associated sensor data (as accurately as possible in the output), which is determined by using the MSE between the raw data of each sensor and the output of that sensor, for example, as the autoencoder loss. 2. Adversarial loss to prevent imitation of a confidential / protected sensor (e.g., microphone 312 in Figure 3 or first sensor 10 in Figure 1) by causing the virtual sensor to malfunction, for example by adding sufficient noise to its output or setting the output to zero, using a virtual sensor output that is different from the output when operating on raw data, and by backpropagating the gradient through the virtual sensor model 320 to the sensor filter model 320. In this case, the output of the sensor filter model is associated with the input of the virtual sensor model.

[0154] As already mentioned, the sensor filter model 320 is updated during this training, but the virtual sensor model is not.

[0155] Furthermore, the sensor filter model 320 can be applied to sensitive leak sensors (for example, the inertial sensor 310 in Figure 3 or the second sensor 12 in Figure 1) to generate new versions of their outputs, and it can be assured that the data from the protected sensors cannot be imitated by a malicious party when given these new filtered versions of the data.

[0156] In one embodiment, the sensor filter model 330 is implemented to run as close as possible to the sensor (e.g., inertial sensor 310, microphone 312) to ensure that it outputs data that cannot be reconstructed from the "confidential sensor" data.

[0157] In another embodiment, the sensor filter model 330 can be applied at the boundary of a trusted domain before the data leaves that trusted domain. For example, if the trusted domain covers only sensors, the sensor filter model 330 must be applied at the sensors before the data leaves the sensors. For example, if the trusted domain covers a network, such as a smart home network, the sensor filter model 330 can be applied at a network controller (for example, the protection management unit 32 in Figure 1) before the data leaves the network.

[0158] In one embodiment, the sensor filter model 330 can be implemented as a software or hardware process that requires all relevant sensor data as input and can be configured to output filtered data for all sensors within a single model, in which case it can be implemented in a central entity such as a hub that collects data from all sensors in a network. Alternatively, in another embodiment, the sensor filter model 330 can be implemented as a plurality of individual sensor filter models for sensors, each receiving sensor data from only one sensor, in which case they can be distributed to sensors to which it can be applied before communication, for example, when an access control system controls them to perform it. In yet another embodiment, these two filter model approaches can be mixed and used. Thus, depending on the configuration, for example, depending on the distance between sensors or groups of sensors, the protection system can be organized according to the optimal configuration and distribution of the protection process.

[0159] Optionally, further improvements to the filter model output can be achieved by passing the filtered sensor data to its usage applications and receiving data quality and / or utility measures or metrics (or the filter model can estimate these measures or metrics from the input stream), which can then be used to train the sensor filter model described above, for example, to ensure that filtering does not affect the utility of the transformed data.

[0160] Finally, instead of running sensor filter model 330 continuously (or at intervals indicated by the access control system), virtual sensor model 320 can be run continuously on the raw data of the confidentiality leak sensor, while filtering by sensor filter model 330 is performed only when its performance metric indicates confidentiality leakage, for example, when its performance metric exceeds a predefined threshold. The sensor can then switch whether or not to adopt the proposed confidentiality leakage prevention filtering (protective filtering).

[0161] However, if confidentiality leakage prevention filtering (protective filtering) is only applied when a potential confidentiality leakage is indicated, it is necessary to consider the risk that the protection system may not be able to react in time to prevent the initial confidentiality leakage. Such a scheme may not have significant computational advantages because the virtual sensor model 320 (which determines when confidentiality leakage begins) becomes active while the sensor filter model 330 is inactive. However, it is generally beneficial if the protection system (such as an access control system) is configured to use the sensor filter model at certain times and not at other times to configure the confidentiality leakage sensor.

[0162] In some embodiments, the access control system can be configured to provide different authorizations to entities / applications attempting to access raw data from sensitive leak sensors and / or filtered data from those same sensors. For example, some (trusted) applications may be able to use the raw data (if access permissions allow it), while less trusted applications may only be provided with filtered (protected) data that cannot be reconstructed from the data generated by the sensitive / protected sensors (i.e., access permissions do not permit access to the raw data of the identified sensitive leak sensors). For example, this can be achieved by changing the permissions required to access the raw data of a sensitive leak sensor to the permissions of the leaked sensor (or similar sensor). To achieve this, the permissions required to access data from the actual sensor that generates the raw data and / or a virtual sensor that generates the filtered data can be different. For example, a lower authorization value (e.g., priority) can be assigned to the filtered data than the value assigned to the raw data.

[0163] Therefore, a reliable receiver device or application may be given access to raw (actual) sensor data, while other less reliable receiver devices or applications may be given access only to filtered data.

[0164] In the embodiment, for each identified confidential data leak sensor, a kind of virtual sensor (intended to replicate the actual sensor) can be instantiated to generate filtered data. The virtual sensor is instantiated based on input settings that represent attribute values ​​stored in a configuration file. The configuration attribute values ​​can specify at least one piece of information necessary for operation (e.g., input source, output destination, read rate from input queue, fault handling policy, database address).

[0165] In some embodiments, the access control system can handle data at an abstract level without needing to deal with other underlying devices. A middleware security layer can be used to achieve this. Such a middleware security layer can perform access control on a per-function data basis (which may be a collection of data from different actual or virtual sensors) rather than on per-sensor data. At this abstract level, virtual sensors and real sensors are similar, and new virtual sensors can be added and data rights (authorities) can be assigned. The access control system can then handle which applications and / or receiving devices can access which sensor data, based on these authorizations. The middleware security layer is particularly important when combining sensors with application functionality. For example, in Matter, sensor data is aggregated at the destination (receiver) or within the application itself. Here, more access control can be granted to the user. For instance, the user can decide more precisely who accesses the data and how much.

[0166] In a related embodiment, when an application (for example, an application running on a mobile phone and downloadable from an app store, or an application used in a smart home system) requests access to a particular sensor, the middleware security layer may provide the application with either raw or filtered data.

[0167] In related embodiments, the middleware security layer can be configured to allow access to raw data by default, or to both raw and filtered data by default, or to filtered data by default.

[0168] In embodiments, a possible sensor can be abstracted into an "abstracted functional sensor" which includes actual sensors, virtual sensors, and filtered sensor data (which can also be abstracted as a virtual sensor). Some sensors (actual or virtual sensors) can be labeled by a protection process as either disclosing or not disclosing sensitive information with respect to the associated sensitive / protected (target) sensor. For example, obtaining data rights to an abstracted device involves at least one of the following approaches: ● In the case of an actual sensor without abstracted confidentiality leaks, the controller of the access control system (e.g., the protection management unit 32 in Figure 1) can be configured to convert the access control level of the actual sensor data on the device into usage rights (authorities) by reading access control data from the associated device. ● To provide an appropriate level of access control to an abstracted, confidential / protected sensor, the controller of the access control system can acquire user data regarding user settings for the use of sensor-generated data and combine / aggregate this data with access control data from specific function-related device controllers. ● In the case of abstracted confidentiality leak sensors, the abstracted raw sensor data is given the same access control level as the abstracted confidential / protected sensors that are leaked by them, and the access control level can be lowered according to the degree of confidentiality leak indicated, for example, by the virtual sensor performance metrics above, and can be aggregated with access control data from specific function-related device controllers. ● In the case of abstracted filtered data (e.g., processed by sensor filter model 330) generated by confidential data leak sensors, the controller of the access control system can treat these as if they were the original data, and translate the access control level to the actual sensor data within the device into usage rights (authorities) for these filtered virtual sensors.

[0169] As an example, the access control level assigned to virtual sensors that do not leak confidential information can be determined as a function (e.g., minimum value) of the access control level of their input devices.

[0170] In related embodiments that instantiate further options for access control, the middleware security layer may be configured to add usage rights or access rights to, for example, raw or filtered data. Such usage rights are sometimes referred to as “DRM rights.” It should be understood that the security layer may be configured to manage different aspects of usage rights (e.g., access or transfer) in independent manner, for example, they may be configured individually. An abstracted sensor requiring DRM rights may include at least one of actual sensors (including, for example, confidential / protected sensors, confidential leak sensors, and non-confidential leak sensors), virtual sensors downloaded and installed as apps, virtual sensors set up by an access control system (including the virtual sensors used here), and sensor filter model outputs registered as separate virtual sensors for each confidential leak sensor. This can be achieved, for example, by packing sensor data into a DRM-protected file so that when an app / device needs to access the data, the DRM-protected file outputs raw or filtered data according to the relevant DRM rights.

[0171] Figure 4 schematically shows a block diagram of a data protection system according to an embodiment, in which an intelligent data obfuscation or blurring system (obfuscation device) (IDO) 40 includes, or is integrated into, an access control system (ACS) 30 for sensors, which can include sensor data generated by actual (real) sensors, virtual sensors and / or protection measures, where the sensors can be abstracted to functional sensors.

[0172] Each abstraction or functional sensor may be provided with a set of associated metadata (MD) stored in database 36, the set of which may also be stored in the sensor itself, an access control list (ACL), a unique abstraction sensor identifier (ID), filtered data associated with the actual sensor, virtual sensor, or functional sensor represented as a virtual sensor (e.g., data that enables access to the device), a confidential / protected status flag that is assigned by default (e.g., all microphones) or assigned by user selection or confirmation, details about the period for which the sensor should be assigned a confidential / protected status, an indicator entry (e.g., a flag) indicating whether the sensor in question is leaking confidential information with respect to a particular confidential / protected sensor, and an indicator entry (e.g., a flag) indicating whether the sensor in question is leaking confidential information by other sensors.

[0173] Furthermore, the access control system 30 includes an access rights update (ARU) function 310 for assigning data access rights levels to abstracted sensors using metadata (and other acquired data), as detailed above.

[0174] Furthermore, a protection provision initialization (PPI) function 320 is provided, and the controller of the access control system 30 can determine when the protection provision system will be activated for a particular sensor and initiate this process.

[0175] Finally, a Protection Provided Usage Function (PPU) 330 is provided, and the controller of the access control system 30 can determine when the protection provided will be used to filter sensitive leak sensors and can start and stop the usage process when each application is consuming data provided by the identified sensitive leak sensor.

[0176] The access control function of the access control system can be executed (triggered), for example, via a user interface to the network hub provided on the network hub itself or on an app on a smartphone connected to the network hub (not shown in Figure 4).

[0177] The access control system 30 can also be implemented in smart devices such as smartphones that include multiple sensors and applications, where a first sensor (e.g., an accelerometer) may sample data that could potentially leak data / information collected by a second sensor (e.g., a microphone). For example, the accelerometer of a smartphone on a table may sample vibrations of the table caused by the voice of a conversation, thereby potentially leaking the conversation.

[0178] Furthermore, the integrated data obfuscation unit 40 includes a protection-providing system (PPS) 42 corresponding to the protection-control unit 40 in Figure 1, which includes a virtual sensor generation process (VSG) 410 that can be run (triggered) for each sensitive / protected sensor, thereby training a virtual sensor model to mimic the data output from that sensitive / protected sensor using data from (all) other sensors. The virtual sensor generation process 410 includes, or can access, a sensor data repository (SDR) 44 where sensor data is acquired over time and filtered to maximize its diversity, a training process (not shown in Figure 4), and a model cache (MC) 46 where the resulting virtual sensor model is stored.

[0179] Furthermore, the protection provision system (protection means) 42 includes a Sensor Confidentiality Leakage Metric (SCM) 420 configured as a process that accesses a virtual sensor model and some test input data and generates a measurement or metric of the virtual sensor model regarding the extent to which the relevant confidential / protected sensor may be confidential. This metric may be generated as part of the training process (as described, for example, in relation to Figure 3) or later as a specific function. A performance evaluation is performed using some of the information stored in the sensor data repository 44 (e.g., that not used in the training process) and a metric (e.g., a score) is obtained, for example, a numerical value from 0.0 (no leakage) to 1.0 (complete leakage).

[0180] This metric is associated with the virtual sensor model. If the metric is low, no protection is required, and all associated sensors are marked as not leaking confidential information with respect to the confidential / protected sensors.

[0181] Otherwise, further functions of the protective measures 430 to 450 are initiated, including updating the metadata of the confidential / protected sensor to indicate that it is being leaked, determining the sensor that is leaking confidential information, generating a sensor filter model, and applying a sensor filter model application.

[0182] The corresponding Confidentiality Disclosure Sensor Determination (CSD) function 430 includes a process for determining which sensors (or combinations of sensors) are involved in a potential confidentiality disclosure, in which a subset of all sensors may be considered. Here again, for each sensor used in the virtual sensor input, a metric (such as a score) is obtained from 0.0 (no disclosure) to 1.0 (disclosure). If the score is low, the sensor in question is labeled in the metadata database 36 as not having disclosed confidentiality with respect to the confidential / protected sensor. Otherwise, it is marked as having disclosed confidentiality with respect to that sensor.

[0183] Furthermore, the corresponding Sensor Filter Model Generation (SFMG) function 440 may include the training process of an autoencoder-like model using training data identical or similar to that of the virtual sensor model. This provides a function that can be applied to the data of a sensor that is leaking confidential information when protection should be performed. The Sensor Filter Model Generation function 440 can be applied to individual sensors alone or implemented to require input from multiple sensors. The sensor filter models can be stored in the model cache 46. A virtual sensor can be generated for each sensor that is leaking confidential information and added to the metadata associated with the filtered data obtained from that sensor.

[0184] Finally, the corresponding Sensor Filter Model Application (SFMA) function 450 may include processing the output data (raw data) of the sensitive leak sensor (at a specific time or for all time) by the sensor filter model to generate filtered data (e.g., one for each sensitive leak sensor) which is represented as the output of a virtual sensor.

[0185] Figure 5 schematically shows the data protection process flow diagram based on the system in Figure 4.

[0186] The user can use the user interface 34 to input protection control information (PCI), such as the IDs of confidential / protected sensors and / or confidential leakage sensors, and / or protection timing (intermittent or continuous protection), into the metadata database 36 of the access control system 30. The metadata is used by the access rights update function 310, protection provision initialization function 320, and protection provision usage function 330 of the access control system 30 of the intelligent data obfuscation device 40.

[0187] The virtual sensor generation process 410, the sensor confidentiality leak metric function 420, the confidentiality leak sensor determination function 430, and the sensor filter model generation function 440 perform the above-described functions based on information derived from the sensor data repository 44, which is supplied with external sensor data (SD).

[0188] More specifically, the protection provision initialization function 320, as described above, triggers protection provision initialization (PPI) in the virtual sensor generation process 410, which generates a virtual sensor model of the leaked sensor, stores it in the model cache 46, and supplies relevant information to the sensor leak metric function 420, which generates at least one sensor performance metric as described above. The output of the sensor leak metric function 420 is used by the leaked sensor determination function 430 to determine the leaked sensor, as described above. Furthermore, the leaked sensor metadata (CDS-MD) obtained by the sensor leak metric function 420 is stored in the metadata database 36. Furthermore, the leaked sensor metadata (CDS-MD) obtained by the leaked sensor determination function 430 is stored in the metadata database 36. The output of the leaked sensor determination function 430 is used by the sensor filter model generation function 440 to generate a new sensor filter model for a new virtual sensor, for example, using the virtual sensor model stored in the model cache 46 for the training process, as described above. The information about the new virtual sensor (NVS) obtained by the sensor filter model generation function 440 is stored in the metadata database 36. Finally, the virtual sensor model and the sensor filter model stored in the model cache 46, the protection provision and utilization (PPU) information (e.g., permissions, etc.) generated by the protection provision and utilization function 330, and the raw confidential leak sensor data (CGS-D) are stored in the metadata database 36. R Based on the above, the sensor filter model application function 450 applies the filtered non-confidential leaked virtual sensor data (NCGVS-D F ) is generated as protected (obfuscated) output data of the intelligent data obfuscation device 40.

[0189] Figure 6 schematically shows a flowchart of the data protection process according to the embodiment.

[0190] In the first step S601, at least some or all of the sensors present in the system to be protected (i.e., actual sensors, virtual sensors, etc.) are abstracted as functional sensors (abstract sensors (ABS)). Functional abstraction provides a mode in which a function performs its work or operation without showing how the operation is performed or how the function is implemented. Simply put, it hides the actual implementation of the sensor function's work and only shows the output that the sensor function provides.

[0191] Then, in step S602, an access level (AL) is assigned to the obtained abstract sensor according to the access control system and method, using the abstracted functional sensor metadata (and sensor-specific acquired data).

[0192] In step S603, a sensor is designated as a newly confidential / protected sensor, or a new sensor (NS) is added to the network, an existing sensor is marked as confidential / protected, and a protection measure for the confidential / protected sensor (e.g., the protection control unit 40 in Figure 1 or the protection provision system 42 in Figure 4) is invoked. This leads to the virtual sensor generation process in step S604a, where a virtual sensor model is obtained in the model cache 46 using training data from the sensor data repository 44, and in step S604b, a sensor confidentiality leak metric is calculated for that virtual sensor model.

[0193] If the calculated confidentiality leak metric falls below a defined threshold, the procedure branches to step S605, and no action is taken until, for example, the procedure is restarted.

[0194] Otherwise, if the calculated confidentiality leak metric is greater than or equal to a defined threshold, the procedure branches to steps S606-S609, where the confidential / protected sensors are marked as confidentially leaked sensors in the metadata (CDSM) in step S606, a confidentiality leak sensor determination (CSD) is performed in step S607, all sensors shown to be confidentially leaked with respect to the confidential / protected sensors are marked as such in the metadata, a sensor filter model generation (SFMG) is performed using training data from the sensor data repository and a virtual sensor model from the model cache 46 to obtain a sensor filter model placed in the model cache 46, and each filtered confidentiality leak sensor data (FSD) coming from the sensor filter model is represented as a new virtual sensor in step S609, marked as non-confidential with respect to the confidential / protected sensors, and this is shown in the metadata. The procedure then continues in step S610 on the main branch, where all modified abstract sensors are given a new access level (NAL) in accordance with the changes in functional sensor metadata using the access control system and method.

[0195] Finally, in step S611, the sensor filter model is applied at a timing indicated by the access control controller (for example, the protection management unit 32 in Figure 1) based on the information in the metadata.

[0196] Not all of the above steps are always necessary. If it is known that the first sensor (e.g., accelerometer) may leak information from the second sensor (e.g., microphone, voice information), it may not be necessary to create a virtual sensor for the first sensor first. Alternatively, the sensor filter model can be created based on the actual data from the first and second sensors.

[0197] In summary, the documentation describes devices / methods for enhanced data protection, determines the required level of data protection for the data source, identifies at least one data-generating device, and assigns protection scheme configurations and parameters to the data generated by at least one device. For example, a data stream from a data-generating device designated as confidential / protected (e.g., one that could potentially leak confidential information) is protected from reconstruction from unprotected data streams from one or more other data-generating devices that could be used to provide useful input to a virtual sensor of that confidential / protected sensor.

[0198] While the embodiments were described in the context of home networking and the metaverse, their applications are not limited to such types of operation. They can be applied to any device or network of devices where multiple sensors are present and some correlation may occur between sensor outputs, such as in healthcare, retail stores, shopping malls, and sports clubs. Therefore, they can be applied not only to single multi-sensor devices such as smartphones, but also to IoT and 5G networks, etc.

[0199] Furthermore, at least some of the embodiments include mobile phones, vital sign monitoring / telemetry devices, smartwatches, detectors, vehicles (vehicle-to-vehicle (V2V) communication, or more generally vehicle-to-everything (V2X) communication), V2X devices, IoT hubs, IoT devices including low-power medical sensors for health monitoring, medical (emergency) diagnostic and treatment equipment for hospitals or first responders, virtual reality (VR) headsets, and the like.

[0200] In additional or alternative embodiments, the proposed obfuscation of sensor data can be replaced or enhanced by source isolation, blind signal isolation (BSS), or blind source isolation, where a set of source signals is separated from a set of mixed signals without the help of any information (or very little information) about the source signals or the mixing process. The signal mixing is analyzed by digital signal processing with the aim of recovering the original component signals from the mixed signals. The recovered leaky component signals can be removed or suppressed from the leaky sensor output. Only a clean signal, i.e., after the leaky component signals have been removed, can be made public.

[0201] In additional or alternative embodiments, an access control layer (e.g., PA(32) in Figure 1) can analyze the correlations in each data source, e.g., S1(10) and S2(12), and derive appropriate parameters for blind source isolation (or data obfuscation).

[0202] In an additional or alternative embodiment, if the data generated by S2 contains any hidden components of the data generated by S1, the access control layer may provide, for example, S2(12) with the appropriate parameters for blind source isolation (or data obfuscation), S2 then applies the blind source isolation process to generate a clean signal.

[0203] In additional or alternative embodiments, the access control layer may apply the appropriate parameters for blind source isolation (or data obfuscation) to the S2(12) data (received from S2) if the data generated by S2 contains any hidden components of the data generated by S1.

[0204] Further use cases can consider personal smart wearable devices such as smartwatches and smart rings used by users. Users can use such personal smart wearable devices to facilitate tasks such as sports (e.g., running), work (e.g., receiving / sensing emails / calls), health (e.g., sleep tracking), and even health monitoring (e.g., arrhythmia monitoring). This is possible because personal smart wearable devices have multiple sensors / actuators (e.g., accelerometers and microphones) and applications that generate / consume data. For example, Applications such as fitness apps can use data from sensors in personal smart wearable devices to track user activity. Applications such as email clients can receive data (emails and notifications) and present them to the user.

[0205] These applications and data may be vulnerable to attacks such as the following: Sensors in personal smart wearable devices can be misused for user tracking and extraction of personal information. For example, when a user enters a password on a laptop or an unlock code on a smartphone, accelerometer readings from the personal smart wearable device can be used to extract / train / derive the password / unlock code. Therefore, applications on personal smart wearable devices can be misused to extract such information. Notifications / data received by personal smart wearable devices may be read by applications that could exploit them. For example, a personal smart wearable device may monitor email notifications and forward them to a third-party server.

[0206] Such personal smart wearable devices are specific examples illustrating where some embodiments of the present invention apply and how some of these embodiments can be further improved.

[0207] In embodiments, a personal smart wearable device is configured to restrict / limit access to sensing data (e.g., accelerometer) when certain privacy-sensitive actions are performed in a specific context. Common privacy-sensitive actions include entering credentials into a login function or entering other sensitive information into an information input application. In such cases, if the personal smart wearable device determines that the user is near a laptop and / or writing on a keyboard (context), the personal smart wearable device may restrict the quality of sensor (e.g., accelerometer) data provided to an application, such as a fitness application, restrict the exchange of such data, or block access to the sensing device (e.g., to a specific application). This is achieved by the personal smart wearable device being configured to receive notifications / messages from nearby personal information input devices (e.g., smartphones, tablets, laptops, Bluetooth-enabled keyboards) and / or monitoring the presence of those devices (e.g., by listening for presence announcement messages). This can be done by the personal information input device by sending notifications / messages to the personal smart wearable device. Therefore, the smart wearable device is configured to detect the proximity of a personal information input device, and the smart wearable device's security features prohibit (i.e., prevent) the transmission of data obtained from the smart wearable device's sensors to other devices or locations (such as a server). Advantageously, the prohibition of sensor data transmission can be time-limited. Alternatively, or additionally, this prohibition can be lifted by receiving instructions from the personal information input device or by detecting that the personal information input device is no longer in proximity. If the personal information input device is configured to indicate that it is not currently being used for inputting personal data, the smart wearable device can detect / monitor this and adapt its security controls accordingly.

[0208] In a further embodiment, a personal smart wearable device can be configured with a policy that restricts the disclosure / publication of private and sensitive data. This configuration can be performed by a controller, such as a smart home controller, a smartphone, or a personal smart wearable device that can configure device settings, such as a personal smart wearable device. This configuration may include restrictions on data quality in specific situations.

[0209] Smart devices such as smartphones (e.g., iPhones) may have applications such as email clients (e.g., Outlook) that handle the user's business / private emails. This information can be highly private and confidential. Often, users find it convenient to share notifications received on their smartphones with personal smart wearable devices such as smartwatches. These notifications may contain a large amount of information, and that information may be highly confidential. If this personal smart wearable device is hacked or compromised in any way, it could leak information about the main smart device's data, including the content of notifications. Therefore, in a further embodiment, the first device (smart device / phone) can be configured to apply protection to certain types of data (e.g., email notifications, data from a specific application) when operating with a second device (e.g., a personal smart wearable device), when sharing such data in notification messages sent to the smart wearable device. For example, all data from a specific application (e.g., an email application) may be limited to a simple announcement of an event within the application (receiving an email) without disclosing the content / subject. In another embodiment, a smart wearable device may be configured by the smart device to apply DRM-based protection to certain types of notifications, where the protection allows notifications to be displayed on the smart wearable device but prevents unprotected information from being copied from the smart wearable device to another location or accessed on another device. For this purpose, the smart wearable device may have secure storage for DRM encryption keys. For example, a smart device may be able to restrict notifications related to certain applications that may process personal data.

[0210] Similarly, DRM-based protection can be configured using the methods described above.

[0211] In summary, there is a wearable device worn by a user and configured to receive messages containing notification data from a second device, the wearable device having a processor, the processor being configured to use a security function that applies a security policy to the notification data, the security function being configured to prevent at least a portion of the notification data from being sent to a third device.

[0212] Furthermore, there is a wearable device that is worn by a user and configured to receive messages from a second device, the messages including notification data, the wearable device having a display and a processor, the processor being configured to use security functions to apply a security policy to the notification data, the security policy preventing at least a portion of the notification data from being displayed on the display.

[0213] The security policy is configured to selectively prohibit the transmission of at least some of the notification data based on its classification.

[0214] Furthermore, there is a wearable device configured to be worn by a user, having a sensor and a processor, wherein the sensor is configured to measure the user's physical parameters and output corresponding measurement data, and the processor is configured to use a security function to apply a security policy to the measurement data, the security function is configured to prevent at least a portion of the measurement data from being transmitted to a third device.

[0215] The wearable device is configured to receive messages from a second device, which in turn allows the second device to set security policies.

[0216] A wearable device according to any of the prior claims has secure storage, which is configured to store security information, and the security function is configured to use cryptographic keys as part of the enforcement of a security policy.

[0217] The security policy is based on detecting proximity between wearable devices and data input devices.

[0218] The data entry device is at least one of the following: a smartphone, tablet, laptop, or wireless keyboard.

[0219] The transmission prohibition is stopped upon receipt of a command from the data input device, or by an instruction from the data input device indicating that the data input device is not being used to input confidential data.

[0220] There is a device configured to set up a wearable device, and that setting up includes setting up the privacy policy for the wearable device.

[0221] Setting a privacy policy includes providing security information to smart wearable devices and identifying the types of data to which the privacy policy applies.

[0222] The device has a privacy policy, the device has an application program that generates data, the privacy policy can be configured to restrict the provision of data to a selected wearable device or a selected application program, and the selected application program is configured to run on the wearable device.

[0223] Furthermore, this can be summarized as a method to restrict the transmission of information from a wearable device, including setting a privacy policy for the wearable device by sending a message to the wearable device, the message including instructions for setting a privacy policy, the privacy policy being configured to selectively prohibit the wearable device from performing an action, the action being to display or transmit information received by the wearable device.

[0224] The method includes setting a privacy policy for a wearable device by sending a message to the wearable device, the message including instructions for setting the privacy policy, the privacy policy being configured to selectively prohibit the wearable device from performing an action, the action being the transmission of information generated by the wearable device from the wearable device.

[0225] The aforementioned configuration is performed by a second device and includes the receipt of security information from the second device by the wearable device, the security information being used by the wearable device to enable the execution of the aforementioned action.

[0226] A cellular system is a wireless communication system composed of three main components: user equipment (UE), radio access network (RAN), and core network (CN). These components work together to provide voice and data services to mobile users across a wide geographical area.

[0227] A user device (UE) is a device that a user can use to access a cellular system. UEs can take the form of devices such as smartphones, tablets, laptops, and wearable devices. UEs can include, in particular, the following components: A Universal Integrated Circuit Card (UICC) configured to store user identification and authentication information, such as subscription perpetual identifiers (SUPIs) and credentials; Transceiver; A processor configured to control the operation of the UE and run applications and services requested by the user; display; Microphone and speaker; Data input devices such as keyboards and touchscreens; Camera and / or video recorder; Memory configured to store data and programs.

[0228] If the UE is implemented in an IoT device, it may only include a subset of the elements mentioned above.

[0229] The Radio Access Network (RAN) is part of a cellular system that connects UEs (Users) to the Network Complex (CN) via an air interface. The RAN consists of two main components: base stations (BS) and the Radio Access Network Controller (RNC). A base station (BS) is a fixed or mobile transceiver that covers a specific geographical area called a cell. In 5G, BS are also called gNBs (next-generation node B). A BS can simultaneously serve multiple UEs within a cell using different frequencies, time slots, codes, or beams. BS also performs functions such as power control, handover control, channel allocation, and interference management. A base station can be divided into two units: a central unit (CU) and a distributed unit (DU). The CU performs higher-layer functions such as RLC, PDCP, and RRC. The DU performs lower-layer functions such as PHY and MAC. Depending on the network architecture and deployment, the CU and DU can be located in the same location or separated. A Radio Access Network Controller (RNC) is a device that controls and coordinates the operation of a BS group called a Radio Access Network (RAN). The RNC performs functions such as radio resource management, mobility management, call control, and signaling.

[0230] The main protocols used between the UE and RAN are as follows: - The physical layer (PHY) defines the characteristics of the air interface, such as frequency bandwidth, modulation scheme, coding rate, frame structure, and synchronization. - A media access control (MAC) layer that restricts UE access to shared radio channels using technologies such as orthogonal frequency division multiplexing (OFDMA), time division duplexing (TDD), and frequency division duplexing (FDD); - A radio link control (RLC) layer that provides reliable data transmission over a radio channel by employing techniques such as segmentation, reconstruction, error detection, error correction, and retransmission; - The Packet Data Convergence Protocol (PDCP) layer compresses and decompresses data packet headers, encrypts and decrypts data, and performs data integrity protection; - The Radio Resource Control (RRC) layer establishes, maintains, and releases radio bearers between the UE and RAN, and exchanges signaling messages for functions such as connection setup, handover, measurement reporting, and security activation.

[0231] The Core Network (CN) is part of the cellular system that connects the RAN to other networks such as the Internet and other cellular systems. The CN consists of two main (control / user) domains. The control domain is responsible for providing signaling and control functions to UEs, such as authentication, authorization, mobility management, and session management. The control plane consists of several network functions (NFs), including the Access and Mobility Management Function (AMF), Session Management Function (SMF), Unified Data Management (UDM), Policy Control Function (PCF), Network Exposure Function (NEF), and Authentication Server Function (AUSF). The Access and Mobility Management Function (AMF) is an NF that handles UE registration, deregistration, connection management, and mobility management. The AMF also communicates with the RNC to perform functions such as handover, authentication, and encryption. The Session Management Function (SMF) is an NF that handles the establishment, modification, and release of UE sessions. The SMF also communicates with user plane devices to perform functions such as IP address assignment, tunneling, and QoS. Unified Data Management (UDM) is an NF that stores and manages user data such as SUPIs, service profiles, and subscription status. Policy Control Function (PCF) is an NF that provides policy rules and billing information for the UE, such as access type, service level, data rate, and allocation. Network Exposure Function (NEF) is an NF that exposes network functions and services to external applications and devices such as IMS and IoT. Authentication Server Function (AUSF) is an NF that performs primary authentication using credentials and SUPIs. User domains are responsible for providing data and multimedia services to the UE using packets and IP addresses. The user plane consists of two main functions: User Plane Function (UPF) and Data Network (DN). The User Plane Function (UPF) is a device that forwards data packets between the UE and the DN and performs functions such as tunneling, firewalling, QoS, and billing.The data network (DN) is the network that provides access to the services and applications requested by the UE (Unified Entity), such as the Internet and IMS (Information Management System).

[0232] UEs can be used to sense and capture multiple types of data, including sensor data, image data, sound data, and location data. This data may need to be shared with other devices and UEs, both inside and outside the (cellular) network. For example, the information may refer to data related to the collection and publication of energy consumption data for energy conservation (TR 33.766), or data related to avatar-based IMS calls in real-time communication systems (TR 33.790) or 5G mobile metaverse services (TR 23.700-21, TR 33.721), but these are just examples.

[0233] In embodiments that can be used in combination with other embodiments or independently, the network function and / or application function can determine the required level of protection for certain types of data, identify UEs capable of providing / generating such data, register such UEs (first devices) and their data (metadata) as protection control information, and, based on the protection control information, assign the configuration of a protection scheme and related parameters to the target data generated or provided by the UE. This network or application function can be responsible for how data is shared and protected in a cellular system.

[0234] In embodiments that can be used in combination with other embodiments or independently, the UE can, once configured with protective control information, generate certain types of data and share them with an authorized party (second device). For example, energy consumption data can be shared with an authorized party, such as a network function responsible for data and statistics.

[0235] In embodiments that can be used independently or in combination with other embodiments, the NF determining the protection level may be an NF in the UE's home PLMN, and the NF may set the protection level for a given task using configuration messages such as NAS messages, UPU messages, or UCU messages. The UE can then protect the data accordingly and share the protected data with, for example, an NF in the domain of the PLMN being visited, or with operations, management, and maintenance systems. The UE may present or provide its protection capabilities as part of its functionality so that the network is aware of the types of data it can protect and how to protect them. The NF in the home PLMN may select appropriate protection parameters and provide them to the UE.

[0236] In embodiments used in combination with other embodiments or independently, a UE may be unable to implement a specific protection scheme required by the home PLMN, and such a UE may instead rely on, for example, an intermediate trusted node within the home PLMN to perform the specific protection. This may be indicated to the UE in a configuration message, for example, as in the previous embodiments.

[0237] In embodiments that can be used in combination with other embodiments or independently, a network function responsible for data and statistics (for example, in a serving PLMN) can receive protected data from the UE, process it securely using, for example, policies and / or key materials (e.g., evaluation keys), and share it.

[0238] Furthermore, the present invention can be applied to mobile phones, vital sign monitoring / telemetry devices, smartwatches, detectors, vehicles (vehicle-to-vehicle (V2V) communication or more generally vehicle-to-everything (V2X) communication), V2X devices, IoT hubs, low-power medical sensors for health monitoring, IoT devices including medical (emergency) diagnostic and treatment devices for hospitals or first responders, virtual reality (VR) headsets, and the like.

[0239] Other variations of the disclosed embodiments will be understood and implemented by those skilled in the art practicing the claims, from a study of the drawings, disclosures, and appended claims. In the claims, the word “has” does not preclude other elements or steps, and the indefinite article “a” or “an” does not preclude plurality. A single processor or other unit may fulfill the functions of several items enumerated in the claims. The mere fact that certain means are described in mutually different dependent claims does not imply that combinations of these means cannot be used advantageously. The foregoing description details certain embodiments. However, it should be understood that these embodiments are not limited to the disclosed embodiments, as they can be implemented in many ways, no matter how detailed the foregoing may be in the text. It should be noted that the use of certain terms when describing certain features or aspects should not be considered to mean that the terms are redefined herein and limited to including certain characteristics of the features or aspects to which the terms relate. Furthermore, the expression “at least one of A, B, and C” should be understood disjunctively, i.e., “A and / or B and / or C.”

[0240] A single unit or device may perform the functions of multiple items cited in a claim. The mere fact that certain means are described in different dependent claims does not imply that combinations of these means cannot be used advantageously.

[0241] The operations described in the embodiments above (for example, Figures 2, 3, 5, and 6) can be implemented as program code for a computer program and / or as dedicated hardware for an associated network device or function. The computer program may be stored and / or distributed on a suitable medium such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but it may also be delivered in other forms such as the Internet or other wired or wireless telecommunications systems.

Claims

1. A device for controlling data protection, said device, Determine the required level of protection for the target data. Determine at least a first device capable of generating or providing the aforementioned target data. The first device and metadata associated with the first device are registered as protective control information. Based on the protection control information and the policy determined by the user, the protection scheme and associated parameter settings are assigned to the target data generated or provided by the first device. A device configured in such a way.

2. The second device is assigned usage rights with respect to the data generated by the first device. The apparatus according to claim 1, configured to provide the usage rights, the protection scheme, and the settings of the parameters to the second device.

3. The apparatus according to claim 1 or 2, wherein the protection scheme is based on at least one of digital rights management, multi-party computation, homomorphic encryption, blind source isolation, and data obfuscation.

4. The apparatus according to claim 2, wherein the second device is located outside the network domain in which the first device is located.

5. An action that requires a device or application to provide its security capabilities and / or requirements; An action that requests a user, application, or service to provide input regarding the aforementioned target data; An action that prompts the user to provide input regarding the required level of protection for either individual devices or aggregated devices; Actions that expose data from one or more physical devices or one or more virtual devices protected in accordance with the settings of the aforementioned protection scheme; Actions to determine the protection level of a virtual device based on the aforementioned settings of the protection scheme and / or requirements based at least on the protection level requirements of the underlying device and the protection level of the consuming application; An action to instruct the first device to use a specific type of protection so that the data generated by the first device is protected along the way from the first device to the second device; An action to receive data from the first device and protect the received data stream from the first device, wherein protection from the access control layer to the second device is achieved by a given protection option; and Actions to receive feedback from the second device regarding the success and / or failure of applying a given type of protection; The apparatus according to any one of claims 1, 2, and 4, configured to perform one or more of the following:

6. The apparatus according to any one of claims 1, 2, and 4, wherein the protection scheme is a multi-party computing (MPC) protection scheme, and the protection key is shared in a key sharing scheme.

7. The apparatus according to any one of claims 1, 2, and 4, wherein the protection scheme is a homomorphic encryption scheme, the first device obtains a public key, the second device obtains an evaluation key, and the target data can be processed in an encryption domain.

8. The apparatus according to any one of claims 1, 2, and 4, wherein the protection scheme is an attribute-based encryption scheme, the first device obtains a public key derived from an access control attribute that grants access to the data, the first device encrypts the generated data using the public key, the second device obtains a private key that depends on its own attributes, and the second device accesses the data if its access control attribute matches the attributes of the public key used to encrypt the data.

9. The apparatus according to any one of claims 1, 2, and 4, wherein the protection scheme is based on a symmetric key, the first device obtains a symmetric key for protecting the generated data, the second device obtains the same symmetric key if permitted, and the second device can access the data if permitted.

10. The apparatus according to any one of claims 1, 2, and 4, wherein the required level of protection for the target data is determined based on the combined data protection requirements of a plurality of first devices.

11. The apparatus according to claim 10, wherein the required protection level corresponds to the maximum protection level of the first device, subject to performance.

12. A system for controlling data protection, A protection management unit including the device described in any one of claims 1, 2, and 4, The protection management unit includes an interface for setting the usage rights of the second device with respect to the target data generated by the first device, At least one of the first devices, It has, The first device is configured to protect the collected data and share the protected data using the protection scheme and the settings of the parameters. The protection management unit is configured to receive requests from the second device for the usage rights, the protection scheme, and the settings of the parameters. The protection management unit is a system that provides the second device with the usage rights and the settings of the protection scheme.

13. The system according to claim 12, wherein the protection management unit is controlled by a digital rights management (DRM) system and registers information on which sensors in the network are designated to generate DRM-protected data, and registers a corresponding access control list for the designated sensors based on that information, the usage rights define the conditions for the second device to access the sensor data, the protection management unit is configured to provide the designated sensors with the DRM protection keys in order to protect the sensor data by using the settings of the protection scheme and the DRM protection keys, and the protection management unit is configured to receive requests for the usage rights and the DRM protection keys from the second device and provide the usage rights and the DRM protection keys to the second device.

14. A method for controlling data protection, The steps include determining the required level of protection for the target data, The steps include determining at least a first device capable of generating the target data, The steps include registering the first device and associated metadata as protective control information, A method comprising the steps of assigning a protection scheme and associated parameter settings to the target data generated by the first device, based on the protection control information and a policy determined by the user.

15. The steps include assigning usage rights to the second device with respect to the data generated by the first device, The method according to claim 14, further comprising the step of providing the usage rights, the protection scheme, and the settings of the parameters to the second device.

16. The steps include: setting the right of the second device to use the target data generated by the first device; The steps include protecting the data collected by the first device by using the protection scheme and the settings of the parameters, The steps include sharing the protected data by the first device, The steps include receiving a request from the second device for the usage rights, the protection scheme, and the settings of the parameters, The steps include providing the second device with the usage rights and the settings of the protection scheme, The method according to claim 14 or 15, having the following characteristics:

17. A computer program that runs on a computer and causes the computer to perform the method described in any one of claims 14 to 16.

Images