Method and system for independent correct sampling proof of streaming data
By using a verifiable random function (VRF) at the data source to determine the sample selection of the data stream and sending inclusion proofs, the data stream owner provides the data stream correct sampling proof to potential users without compromising data privacy and confidentiality, realizing the representation of the data stream and user trust.
Patent Information
- Application Number
- CN202411384099.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-10-10
- Filing Date
- 2024-09-30
- Publication Date
- 2025-04-11
AI Technical Summary
In the field of decentralized data sharing and media streaming, it is difficult for data stream owners to provide a correct sampling proof of data flow to potential users without compromising data privacy and confidentiality, especially when processing large-capacity streaming data.
By adopting a verifiable random function (VRF) at each data source, each data source independently evaluates the VRF with its private key, determines whether it is selected as part of the sample, and sends a proof containing to ensure the correctness and representation of the sample.
It realizes that the correct sampling proof of the data flow is provided without transmitting data to the central location, ensuring the representativeness of the data flow and the trust of the user, while avoiding the risk of the data flow owner manipulating the sampling process.
Smart Images

Figure CN119815393A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to streaming technology, and in particular to correct sampling proofs for streaming data. Background Art
[0002] Any discussion of the background art throughout the specification should in no way be considered as an admission that such art is widely known or forms part of the common general knowledge in the field.
[0003] In the field of decentralized data sharing or media streaming, a data stream owner may own or coordinate multiple data sources that generate data. In many use case scenarios, the data stream may be useful to other users. For example, in one scenario, an equipment supplier may want to receive data on the performance of their equipment in a communication network. However, due to the sensitivity of the customer's (e.g., telecommunications supplier) own customer data, customers are very reluctant to share this information. In addition, customers may also be concerned that their data is used by equipment suppliers to help their competitors (e.g., telecommunications supplier "A" may be reluctant to share data because the data may be used to improve the network of telecommunications supplier "B"). Therefore, they have an incentive to interfere with the data shared with the equipment supplier (e.g., withhold it) and / or exploit the intelligence generated by the equipment supplier and other suppliers' data.
[0004] One difficulty is that in many cases the set of useful properties of a data stream is unknown. An example of such a useful property is the validity or internal consistency of the data, whether it is meaningful and / or feasible, such as, for personal health data, age less than 120 years or height less than 3 meters. Another example may be information properties of a data stream, such as a statistical mean, median, percentile and / or distribution. However, many of these properties may not be easily known and exhaustively enumerated, because the utility of a data stream often depends on the application use case as well as the properties of the data stream itself.
[0005] Another difficulty is processing large volumes of streaming data: data streams often require near real-time processing, requiring practical considerations about performance and cost. Performance is important because the timeliness of a data stream may change its value (e.g., a recent data stream may be more useful than an older one). In addition, there may be many data sources contributing to a high-rate stream, the data sources or the receivers may have limited network bandwidth, and the size of the data may be large. Similarly, cost comes into play because generating a proof of a data stream should not be more expensive than the subscription price of the stream; otherwise, there is no point in generating a proof.
[0006] Any solution to the problem of providing streaming data to others will also have to take into account the fact that the streaming data owner may or may not control the data sources that contribute to the data stream. Furthermore, the solution needs to ensure that the data stream owner cannot skew the provided streaming data to his own benefit by manipulating the data collection process. Such manipulation may be accomplished by adding / hiding data sources and / or discarding the collected data (i.e., omission of data).
[0007] It is an open problem to solve this particular aspect at the same time. In this relationship, it is also an open problem to prove to potential data users by the data stream owner that the provided samples were correctly created. Therefore, in view of at least some or all of the problems discussed above, there appears to be a general need to provide methods and systems for allowing data stream owners / data stream agents to provide data streams to potential data users in a scalable, private and confidential manner. Summary of the invention
[0008] A method of providing proof of correct sampling for streaming data for a scenario may be provided, wherein a data stream owner (or seller) attempts to provide (or even monetize) a streaming data set by providing (or even selling) the streaming data set to other users who may be in a data network and / or market or exchange. Prior to purchase, a potential purchaser desires certain guarantees about the data stream (e.g., statistical properties of the data). Since it is not feasible to enumerate all possible properties that are useful for the use cases of potential purchasers of the data and the large amounts of data generated in the stream, the stream owner agrees to supply potential purchasers with a set of data items from a variety of randomly selected (i.e., sampled) data source sets so that they can examine the data items according to their needs. The data items constitute a data sample.
[0009] Because the stream owner owns the data that can be supplied to potential buyers, some buyers may be reluctant to trust the data stream owner not to interfere with the sampling process of the data source that will produce the data. It assures potential buyers that the samples of the data source (and therefore, the data they produce) are created correctly.
[0010] According to a first aspect of the present disclosure, there is provided a first network element in a network for providing at least a part of a data stream and / or a data item as a data source based on a selection of the first network element to be included in a sample of the data stream, wherein the first network element comprises:
[0011] at least one processor; and
[0012] at least one memory storing instructions that, when executed by at least one processor, cause the first network element to at least:
[0013] employing a first function having a secret key (SK) specific to the first network element to determine whether the first network element is selected to be included in the sample; and / or
[0014] Therein, if it is determined that the first network element is selected to be included in the sample, at least a portion of the data flow and / or data item is provided to one or more second network elements as part of the sample, and an inclusion proof indicating that the data to be part of the data sample was correctly calculated (and / or also as a proof of the property of the data flow and / or data item) is provided.
[0015] In some examples, the first function is a verifiable random function VRF employed at the network element to independently evaluate the VRF using a secret key (SK) specific to the first network element.
[0016] In addition, the first network element or method can employ a verifiable random function (VRF) at each data source, so that each data source independently evaluates the VRF using their own secret key and determines whether they are selected in the sample. If so, they not only send their data as part of the stream, but also send proof that the selection in the sample is correctly calculated. Therefore, the receiver of the sampled data and the receiver of the proof (i.e., the potential purchaser) check the proof using the corresponding verification key of the source, so that they can obtain a guarantee that the sample is correctly calculated and is representative of the data stream. Given such a data sample, they can check the sample and evaluate whether it meets the requirements of their use case or application. Similarly, each data source that is not selected to be included in the sample sends a proof that they are not selected (instead of data). In other words, in the present disclosure, one goal is to define how to collect data "from" the source correctly and in a scalable manner. Here, this property means: scalability = using a subset of all sources to make it scale, correctness = honestly and fairly selecting the sources we collect from.
[0017] Having this verifiable sampling process distributed among the data sources avoids the additional cost of transmitting data to a central location and processing it there. A stream owner may still want to interfere with the sampling process to skew the sample data toward her preferences to convince a potential purchaser (e.g., discarding selective data packets). Such potential interference can be avoided, and potential data purchasers can be protected, because 1) each data source evaluates the VRF privately without involving the stream owner, 2) each data source sends proof of their VRF evaluation (even when not selected in the sample and not providing data), and 3) verifiers (i.e., potential purchasers) can keep track of such proofs to determine if the seller is untrustworthy or the stream is too unreliable, allowing them to avoid purchasing the data stream in either case.
[0018] In some examples, if the first network element determines that it is not selected to be included in the sample, the first network element is caused to:
[0019] The second network element is provided with an exclusion certificate that the first network element was not selected in the sample, but preferably without providing data as part of the sample.
[0020] In some examples, the first network element is a randomly selected data source from the set of data sources;
[0021] The first network element may be configured to: generate data forming part of a data flow to be provided to the second network element;
[0022] The sample may be a subset of data set items of the data stream determined via at least one statistical sampling method;
[0023] The sample comprises a plurality of data items of the first network element; and / or
[0024] The inclusion certificate is used to verify that the selections in the sample are calculated correctly.
[0025] In some examples, the first network element is further caused to:
[0026] A first function having a secret key (SK) specific to the first network element and a public key (PK) is evaluated.
[0027] In some examples, the first network element is further configured to sign the data it generates using its private key, which can be checked using a corresponding public key (PK) and / or a verification key to ensure that the data is indeed generated by a source selected in the sample via the described method.
[0028] In some examples, the first network element is further caused to: receive a control signal from the second network element; generate a public key (PK) and / or a secret key (SK); and provide the public key (PK) to the second network element.
[0029] In some examples, a data stream is divided into time slots; a first network element at each time slot generates a portion of data of the data stream; each time slot of the data stream is configured with a time slot representation value; and wherein the time slot representation value is used as a public key (PK). In another implementation, the time slot representation value (i.e., alpha) is used as part of the input of proof generation and verification, as follows: proof = VRF_prove (SK, alpha), and is_verified = VRF_verify (PK, alpha, proof). The public key is actually a public key corresponding to the VRF secret key.
[0030] In some examples, the slot representation value is obtained by: updating the slot representation value at a predetermined point in time, where the slot representation value changes incrementally at the predetermined point in time; or if the first network element is part of a decentralized blockchain, reaching a consensus for a next value of the slot representation value; or receiving the next value of the slot representation value from another network element configured to announce the next value of the slot representation value; or obtaining the next value of the slot representation value by requesting it from another network element.
[0031] In some examples, the next value of the time slot representative value corresponds to a hash of a previous sample, and / or other arbitrary value determined by another network element.
[0032] In some examples, the first network element also includes: at each time slot, the first network element is configured to use a public key (PK) and a secret key (SK) for a first function evaluation, and obtain pi as a VRF proof, pi being calculated based on the public key (PK) as a proof of a property of a data flow and / or data item.
[0033] In some examples, the first network element is configured to: obtain a value beta as an output of the VRF hash, and use beta to evaluate whether itself has been selected to be included in the sample for the time slot based on a sampling probability algorithm; and / or wherein beta is calculated based on pi; and / or the sampling probability is used to calculate the expected number of first network elements in the sample by multiplying the total number of first network elements by the sampling probability.
[0034] In some examples, wherein the first network element is further caused to: release the secret key (SK) at a predetermined time period, and create a new secret key (SK) for the first function evaluation.
[0035] In some examples, where excluding proofs is based on at least one of: the number of first network elements, the size of the proof, the size of the data, the size of the public key (PK), the size of the secret key (SK), the sampling probability, and the key release period.
[0036] In some examples, the periodic key release depends on at least one of: the number of first network elements, the size of the certificate, the size of the data, the size of the public key (PK), the size of the secret key (SK), the sampling probability, and the key release period.
[0037] According to a second aspect of the present disclosure, a second network element is provided for receiving at least a portion of a data stream and / or a data item of a selected data source, wherein the second network element comprises: at least one processor; and at least one memory storing instructions, which, when executed by the at least one processor, causes the second network element to at least:
[0038] receiving a sample of a data stream, a list of claims of public keys, and a set of inclusion proofs and / or a list of inclusion proofs;
[0039] wherein the sample is a set of selected portions of a data flow from the at least one first network element, and the set of inclusion proofs and / or the list of inclusion proofs comprises inclusion proofs for corresponding selected portions of the data flow; and
[0040] In order to determine whether the sampled data sources are selected correctly, in particular randomly, and to determine that the data sources are not added or removed at runtime, the sample is verified to determine whether the corresponding inclusion proof as a proof of the property of the data flow is included in the inclusion proof set and / or the inclusion proof list.
[0041] In some examples, the step of verifying further includes: receiving a signature list of a public key; checking whether a corresponding inclusion certificate is included in the signature list of the public key; or, receiving a membership certificate of the public key; checking whether the corresponding inclusion certificate is subject to the membership certificate.
[0042] In some examples, if the corresponding inclusion proof is not included in the inclusion proof set and / or the inclusion proof list, the second network element is further caused to: ignore the selected portion of the data flow and the corresponding first network element; otherwise, check the validity of the corresponding inclusion proof for the current time slot and whether the data flow is selected in the sample.
[0043] In some examples, the second network element is further caused to: receive an exclusion proof set and / or an exclusion proof list, wherein the exclusion proof set and / or the exclusion proof list includes exclusion proofs for first network elements that were not selected as part of the sample.
[0044] In some examples, the second network element is also caused to: if the first network element should be in the sample but does not send data, the second network element records the event; the second network element keeps track of how often each data source is missing exclusion proofs and the total number of missing exclusion proofs; if the first network element does not continuously send its inclusion proofs and / or exclusion proofs, the second network element determines that the activity is unreliable and / or the data flow is unreliable.
[0045] In some examples, the second network element is further caused to: obtain a revealed secret key of one or each previous time slot from the first network element; determine whether the first network element should send data in a specific time slot of the previous time slots; wherein, if the first network element should send data in the specific time slot, and / or check whether data is received from the first network element in the specific time slot, and if no data is received, identify the data flow as unreliable or insecure.
[0046] In some examples, the second network element is further caused to identify the first network element as untrustworthy if the first network element does not reveal its secret key after a predetermined time period.
[0047] According to another aspect of the present disclosure, a third network element is provided that controls or owns at least one first network element, the third network element comprising: at least one processor; and at least one memory storing instructions, which, when executed by the at least one processor, causes the third network node to at least:
[0048] receiving from the first network element a selected portion of the data production as part of the sample, and correctly computing a proof of inclusion of the selected portion of the data production in the sample;
[0049] adding the selected portion of the data production to the sample and adding the inclusion proof to the inclusion proof set and / or inclusion proof list; and / or
[0050] The sample and the inclusion proof set and / or inclusion proof list are sent to the second network element.
[0051] In some examples, the third network element is further caused to: receive exclusion proofs that were not selected in the sample from the first network element; add the exclusion proofs to the exclusion proof set and / or the exclusion proof list; and send the exclusion proof set and / or the exclusion proof list to the second network element.
[0052] In some examples, the third network element is further caused to: send the published number of the first network element to the second network element; and receive the request stream with the sampling probability from the second network element.
[0053] In some examples, after receiving the request stream with the sampling probability from the second network element, the third network element is further caused to: send an Ack to the second network element.
[0054] In some examples, the third network element is further caused to: collect public keys (PK) in a list from all first network elements owned by the third network element; and update the list when new public keys (PK) are collected.
[0055] In some examples, the third network element is further caused to: commit to utilizing the sampling probability and the sampling logic of the public key update mechanism; and sign the list, the sampling logic, and the update mechanism using its own private key.
[0056] According to another aspect of the present disclosure, a network system for providing a data flow is provided, the system comprising one or more first network elements, one or more second network elements and / or one or more third network elements.
[0057] According to another aspect of the present disclosure, there is provided a method for providing correct sampling proof to flow data in a network having at least a first network element and a second network element, wherein the method comprises:
[0058] employing the first function at each first network element such that each first network element independently evaluates the first function using their own secret key and determines whether they are selected in the sample;
[0059] If the first network element determines that it is selected in the sample, at least a portion of the data flow and / or data item is provided to one or more second network elements as part of the sample and a certificate containing the certificate as a property of the data flow and / or data item is provided.
[0060] According to another aspect of the present disclosure, a method for determining a correct sampling proof for flow data in a network having at least a first network element and a second network element is provided, wherein the method comprises:
[0061] Receiving, by a second network element, the selected portion of the data stream as a sample and the inclusion proof set and / or the inclusion proof list; and
[0062] An inclusion proof with a corresponding verification key of the first network element is determined to determine whether the sample is correctly computed and represents the data flow.
[0063] According to another aspect of the present disclosure, a computer readable medium having instructions stored thereon is provided, which, when executed by at least one processing unit of the machine, causes the machine to perform a method according to the present disclosure.
[0064] In addition, according to some other example embodiments, a computer program product is provided, for example, for a wireless communication device including at least one processor, which includes software code portions for performing the corresponding steps disclosed in the present disclosure when the product is run on the device. The computer program product may include a computer-readable medium on which the software code portions are stored. In addition, the computer program product may be directly loadable into the internal memory of the computer, and / or may be transmittable via a network by means of at least one of an upload program, a download program, and a push program.
[0065] Although some example embodiments will be described herein with particular reference to the above-mentioned applications, it will be appreciated that the present disclosure is not limited to such fields of use and is applicable in a wider context.
[0066] It is worth noting that it should be understood that the method according to the present disclosure is related to the method of operating the device according to the above-mentioned example embodiment and its variants, and the corresponding statements made about the device are equally applicable to the corresponding method, and vice versa, so that similar descriptions can be omitted for the sake of brevity. In addition, even if not explicitly disclosed, the above aspects can be combined in various ways. Those skilled in the art will understand that this combination of aspects and features / steps is possible unless it creates a contradiction that is explicitly excluded.
[0067] The implementation of the disclosed apparatus may include, but is not limited to, the use of one or more processors, one or more application specific integrated circuits (ASICs), and / or one or more field programmable gate arrays (FPGAs). The implementation of the apparatus may also include the use of other conventional and / or custom hardware, such as a software programmable processor, such as a graphics processing unit (GPU) processor.
[0068] Other and further example embodiments of the present disclosure will become apparent during the course of the following discussion and by reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0069] Figure 1 schematically illustrates an example use case in which a data stream owner monetizes his data at a data marketplace;
[0070] Figure 2 schematically illustrates an example use case in which a manufacturer desires to provide predictive maintenance on its customers' equipment;
[0071] Figure 3 schematically illustrates an example use case in which an energy supplier desires to obtain household electricity usage data to make informed decisions about its investments;
[0072] Figure 4 Schematically illustrates a workflow whereby a data stream owner generates a proof of a property and sends the proof to another party, who then verifies the proof;
[0073] Figure 5 An example scenario with 5 data sources (sensors) and epochs (time slots) as sensing periods is schematically illustrated. The common seed in the VRF evaluation corresponds to the alpha value and is common to all data sources in each epoch;
[0074] Fig. 6A and 6B Schematically illustrates a proposed setup between a stream owner / stream proxy and a stream receiver / stream user;
[0075] Fig. 7A and 7B The setup is schematically illustrated;
[0076] 8A and 8B schematically illustrate stream usage for each sampling period. DETAILED DESCRIPTION
[0077] In the following, different exemplary embodiments will be described using a communication network architecture based on a 3GPP standard for a communication network (such as 5G / NR) as an example of a communication network to which the examples of the embodiments can be applied, without limiting the embodiments to such an architecture. It is obvious to a person skilled in the art that the embodiments can also be applied to other kinds of communication networks in which the mobile communication principles are integrated with a D2D (device to device) configuration or a V2X (vehicle to everything) configuration, such as SL (side link), for example, Wi-Fi, Worldwide Interoperability for Microwave Access (WiMAX), Personal Communications Service (PCS), Wideband Code Division Multiple Access (WCDMA), systems using ultra-wideband (UWB) technology, mobile ad hoc networks (MANET), wired access, etc. In addition, without loss of generality, the description of some examples of the embodiments is related to mobile communication networks, but the principles of the present disclosure can be extended to and applied to any other type of communication network, such as a wired communication network.
[0078] The following examples and embodiments should be understood as illustrative examples only. Although this specification may refer to "one", "a kind of", or "some" (multiple) examples or (multiple) embodiments in multiple locations, this does not necessarily mean that each such reference is related to the same (multiple) examples or (multiple) embodiments, or that the feature only applies to a single example or embodiment. Individual features of different embodiments may also be combined to provide other embodiments. In addition, terms such as "including" and "comprising" should be understood as not limiting the described embodiments to only those features that have been mentioned; such examples and embodiments may also include features, structures, units, modules, etc. that are not specifically mentioned.
[0079] The basic system architecture of a (long-range) communication network comprising a mobile communication system may include the architecture of one or more communication networks comprising (multiple) wireless access network subsystems and (multiple) core networks, and some examples of embodiments are applicable in the mobile communication system. Such an architecture may include one or more communication network control elements or functions, access network elements, radio access network elements, access service network gateways or base transceiver stations (such as base stations (BS), access points (AP), NodeBs (NBs), eNBs or gNBs, distributed units (DUs) or centralized units / central units (CUs)), which control the corresponding coverage area or (multiple) cells and one or more communication stations (such as communication elements or functions, such as user equipment or terminal equipment, such as user equipment (UE), or another device with similar functions (such as modem chipsets, chips, modules, etc.)) can communicate with it via one or more channels, via one or more communication beams for sending multiple types of data in multiple access domains, and the one or more communication stations may also be parts of stations, elements, functions or applications capable of communication, such as UEs, elements or functions that can be used in machine-to-machine communication architectures, or as separate elements attached to such elements, functions or applications capable of communication, etc. In addition, core network elements or network functions may be included, such as gateway network elements / functions, mobility management entities, mobile switching centers, servers, databases, etc.
[0080] The following description may provide further details of alternatives, modifications and variations: The gNB includes, for example, a node that provides NR user plane and control plane protocol terminals to the UE, and the gNB is connected to the 5GC via an NG interface, for example in accordance with Section 3.2 of 3GPP TS 38.300 V16.6.0 (2021-06) incorporated by reference.
[0081] The gNB Central Unit (gNB-CU) includes, for example, a logical node that hosts the RRC protocol, SDAP protocol, and PDCP protocol of the gNB or the RRC protocol and PDCP protocol of the en-gNB, which controls the operation of one or more gNB-DUs. The gNB-CU terminates the F1 interface connected to the gNB-DU.
[0082] The gNB distributed unit (gNB-DU) includes, for example, logical nodes that host the RLC layer, MAC layer, and PHY layer of the gNB or en-gNB, and the operation of the gNB-DU is partially controlled by the gNB-CU. One gNB-DU supports one or more cells. One cell is supported by only one gNB-DU. The gNB-DU terminates the F1 interface connected to the gNB-CU.
[0083] The gNB-CU-Control Plane (gNB-CU-CP) includes, for example, a logical node hosting the RRC and control plane parts of the PDCP protocol, e.g., for an en-gNB or gNB-CU of a gNB. The gNB-CU-CP terminates the E1 interface with the gNB-CU-UP and the F1-C interface with the gNB-DU.
[0084] The gNB-CU-User Plane (gNB-CU-UP) includes, for example, a logical node hosting, for example, the user plane portion of the PDCP protocol for the gNB-CU of the en-gNB, and the user plane portions of the PDCP protocol and the SDAP protocol for the gNB-CU of the gNB. The gNB-CU-UP terminates the E1 interface connected to the gNB-CU-CP and the F1-U interface connected to the gNB-DU, for example, in accordance with Section 3.1 of 3GPP TS 38.401 V16.6.0 (2021-07), which is incorporated herein by reference.
[0085] Different functional splits between central and distributed units are possible, for example, known as options:
[0086] Option 1 (Type 1A Split):
[0087] The functional split in this option is similar to the 1A architecture in DC. RRC is in the central unit. PDCP, RLC, MAC, physical layer and RF are in the distributed unit.
[0088] Option 2 (3C-type split):
[0089] The functional split in this option is similar to the 3C architecture in DC. RRC and PDCP are in the central unit. RLC, MAC, physical layer and RF are in the distributed unit.
[0090] Option 3 (Intra-RLC Split):
[0091] Low RLC (partial functions of RLC), MAC, physical layer and RF are in the distributed unit. PDCP and High RLC (other partial functions of RLC) are in the central unit.
[0092] Option 4 (RLC-MAC Split):
[0093] MAC, physical layer and RF are in the distributed unit. PDCP and RLC are in the central unit.
[0094] Or, for example, according to section 11 of 3GPP TR 38.801 V14.0.0 (2017-03), incorporated by reference.
[0095] gNB supports different protocol layers, for example, Layer 1 (L1) - physical layer.
[0096] NR's Layer 2 (L2) is split into the following sublayers: Medium Access Control (MAC), Radio Link Control (RLC), Packet Data Convergence Protocol (PDCP), and Service Data Adaptation Protocol (SDAP), where for example:
[0097] The physical layer provides transport channels to the MAC sublayer;
[0098] The MAC sublayer provides logical channels to the RLC sublayer;
[0099] The RLC sublayer provides RLC channels to the PDCP sublayer;
[0100] The PDCP sublayer provides radio bearers to the SDAP sublayer;
[0101] The SDAP sublayer provides QoS flows to the 5GC;
[0102] Comp. refers to header compression and Segm. refers to segmentation;
[0103] Control channels include (BCCH, PCCH).
[0104] Layer 3 (L3) includes, for example, Radio Resource Control (RRC), for example, according to Section 6 of 3GPP TS 38.300 V16.6.0 (2021-06), incorporated by reference.
[0105] A RAN (Radio Access Network) node or a network node (e.g., a gNB, a base station, a gNB CU or a gNB DU or a portion thereof) may be implemented using, for example, a device having at least one processor and / or at least one memory having computer-readable instructions (computer program) configured to support and / or provide and / or process functions and / or features related to the CU and / or DU, and / or at least one protocol (sub) layer (e.g., Layer 2 and / or Layer 3) of the RAN (Radio Access Network).
[0106] The gNB CU part and the gNB DU part may, for example, be co-located or physically separated. The gNB DU may even be further divided into, for example, two parts, for example, one including a processing device and one including an antenna. The central unit (CU) may also be referred to as BBU / REC / RCC / C-RAN / V-RAN, O-RAN, or a part thereof. The distributed unit (DU) may also be referred to as RRH / RRU / RE / RU, or a part thereof. Hereinafter, in various example embodiments of the present disclosure, the CU-CP (or more generally, CU) may also be referred to as a (first) network node that supports at least one of the central unit control plane functions or the layer 3 protocols of the radio access network; and similarly, the DU may be referred to as a (second) network node that supports at least one of the distributed unit functions or the layer 2 protocols of the radio access network.
[0107] A gNB-DU supports one or more cells and can therefore act as, for example, a serving cell for a user equipment (UE).
[0108] User equipment (UE) may include a wireless device or mobile device, a device having a radio interface that interacts with a RAN (radio access network), a smart phone, a vehicle-mounted device, an IoT device, an M2M device, etc. Such a UE or device may include: at least one processor; and at least one memory including a computer program code; wherein the at least one memory and the computer program code are configured to, together with the at least one processor, cause the device to at least perform certain operations, such as an RRC connection to the RAN. The UE is, for example, configured to generate a message (e.g., including a cell ID) to be sent to the RAN via radio (e.g., to reach a serving cell and communicate with a serving cell). The UE may generate, send, and receive an RRC message containing one or more RRC PDUs (packet data units).
[0109] The UE may have different states (e.g., according to Sections 42.1 and 4.4 of 3GPP TS 38.331 V16.5.0 (2021-06) incorporated by reference).
[0110] When the RRC connection has been established, the UE is, for example, in the RRC_CONNECTED state or in the RRC_INACTIVE state.
[0111] In the RRC_CONNECTED state, the UE may:
[0112] Store AS context;
[0113] ·Transmitting unicast data to / from UE;
[0114] · monitoring a control channel associated with a shared data channel to determine whether data is scheduled for the data channel;
[0115] Provide channel quality information and feedback information;
[0116] Perform neighboring cell measurements and measurement reporting.
[0117] The RRC protocol includes the following main functions:
[0118] RRC connection control;
[0119] Measurement configuration and reporting;
[0120] Measurement configuration (e.g., setup / modify / release of intra-frequency measurements, inter-frequency measurements, and inter-RAT measurements);
[0121] · Setting and releasing of measuring gaps;
[0122] Measurement report.
[0123] The general functionality and interconnection of the described elements and functions, which also depend on the actual network type, are known to those skilled in the art and are described in the corresponding specifications, so for the sake of brevity, its specific implementation may be omitted herein. However, it should be noted that a plurality of additional network elements and signaling links may be employed for communication to or from an element, function or application, such as a communication endpoint, a communication network control element (such as a server, a gateway, a radio network controller, and other elements of the same communication network or other communication networks in addition to those described in detail below).
[0124] The communication network architecture considered in the examples of embodiments is also capable of communicating with other networks such as a public switched telephone network or the Internet. The communication network is also capable of supporting the use of cloud services for virtual network elements or their functions, wherein it should be noted that the virtual network portion of the telecommunications network may also be provided by non-cloud resources, such as an internal network, etc. It should be understood that the network elements and / or corresponding functions of the access system, core network, etc. may be implemented using any node, host, server, access node or entity, etc. suitable for such use. In general, the network functions may be implemented as network elements on dedicated hardware, as software instances running on dedicated hardware, or as virtualized functions instantiated on an appropriate platform (e.g., a cloud infrastructure).
[0125] In addition, network elements (such as communication elements, for example, UE, terminal equipment), control elements or functions (such as access network elements, for example, base stations / BS, gNB, radio network controllers), core network control elements or functions (such as gateway elements, or other network elements or functions as described herein), and any other elements, functions or applications can be implemented by software (for example, by a computer program product for a computer) and / or hardware. In order to perform their corresponding processing, the correspondingly used devices, nodes, functions or network elements may include multiple components, modules, units, components, etc. (not shown) required for control, processing and / or communication / signaling functions. Such components, modules, units and components may include, for example, one or more processors or processor units including one or more processing parts for executing instructions and / or programs and / or for processing data, storage or memory units or components for storing instructions, programs and / or data, storage or memory units or components for acting as working areas for processors or processing parts, etc. (e.g., ROM, RAM, EEPROM, etc.), input components or interface components for inputting data and instructions by software (e.g., floppy disks, CD-ROMs, EEPROMs, etc.), user interfaces for providing users with monitoring and manipulation possibilities (e.g., screens, keyboards, etc.), other interfaces or components for establishing links and / or connections under the control of a processor unit or processor part (e.g., wired and wireless interface components, radio interface components including, for example, antenna units, etc., components for forming radio communication parts, etc.), etc., wherein the corresponding components forming the interface (such as, the radio communication part) may also be located at a remote site (e.g., a radio head or radio station, etc.). It should be noted that in this specification, a processing portion should not be considered only as a physical portion representing one or more processors, but can also be considered as a logical division of the referred processing tasks performed by one or more processors. It should be understood that according to some examples, a so-called "liquid" or flexible network concept can be adopted, in which the operations and functions of a network element, a network function, or another entity of the network can be performed in a flexible manner in different entities or functions, such as in a node, a host or a server. In other words, the "division of labor" between the network elements, functions or entities involved can vary from case to case.
[0126] The concepts referred to in the present disclosure and the concepts to which the present disclosure can be applied can be summarized as follows:
[0127] One particular application of the present disclosure may be with reference to machine learning. Successfully enabling large-scale machine learning (ML) requires a framework in which large data sets can be used. To increase the amount of data, different entities may collaborate to bring together large data sets. Such collaboration requires different entities to ensure that their confidential information (i.e., data) will not be publicly revealed.
[0128] Another related aspect of the present disclosure is related to data privacy. Data privacy refers to keeping records used as part of training confidential. Confidential data may include, but is not limited to, medical records, credit card information, or personal emails and addresses, and measurements from sensors and industrial processes. For a variety of reasons, this record may have to remain private / confidential: The economic benefits discussed herein may be understood as the process of data collection and collation may be expensive and represent an expensive investment, and leaking the information may represent a financial loss. The contractual requirements discussed herein may be understood as companies that process data of client companies or private customers may have to fulfill contractual obligations to keep the data private. The regulatory requirements discussed herein may be understood as regulations that require protection of the privacy of consumer data (e.g., GDPR and CCPA).
[0129] Federated Learning. Federated learning (FL) discussed in this article can be understood as an ML technique for collaboratively training algorithms between multiple entities, where multiple entities (e.g., devices or servers) each hold part of the training data but do not exchange the data. In other words, FL enables collaborative learning among multiple data owners without exchanging data. In this setting, the model is sent to the data owner, who computes its local update and sends the local update back to the model owner. Existing technologies for securely exchanging model updates are secure multi-party computation, homomorphic encryption, and secure aggregation. FL is used to avoid both expensive data transfers between edge devices and the cloud and to protect the privacy of training data.
[0130] The trusted execution environment discussed in this article can be understood as a trusted execution environment (TEE) as a secure area of the main processor. It ensures that the code and data loaded into the TEE are protected with respect to confidentiality and integrity. TEE provides isolated execution and the integrity of the application and the integrity and confidentiality of its assets to the application. Generally speaking, TEE provides an execution space that provides a higher level of security than a rich operating system (OS) for trusted applications and provides more functions than a "secure element" (SE).
[0131] TEEs are integrated on multiple processor platforms from chip manufacturers. The most popular commercial TEE solution is Secure Guard Extensions (SGX). SGX is a set of security-related instruction codes built into modern CPUs. They allow user-level and operating system code to define private areas of memory (called enclaves) whose contents are protected and cannot be read or saved by any process outside the enclave itself, but can be programmatically verified by a remote party.
[0132] There are two remote attestation schemes: 1) Elliptic Curve Digital Signature Algorithm (ECDSA)-based attestation and 2) Enhanced Privacy ID (EPID)-based attestation.
[0133] ECDSA-based remote attestation enables third-party attestation via the SGX Datacenter Attestation Primitive (DCAP). ECDSA-based attestation with SGX DCAP allows vendors to build and deliver their own attestation services instead of using remote attestation services. This is useful for enterprises, data centers, and cloud service providers that need to use the large enclave sizes available in the Xeon Scalable processor family, run large parts of their networks in environments that are not reachable by Internet-based services, keep attestation decisions internally, and deliver applications that work in a distributed manner (e.g., peer-to-peer networks) that benefit from not relying on a single verification point.
[0134] Remote attestation based on EPID enables a relying party to attest to an enclave without knowing the specific processor on which the enclave is running. Using this technology requires the platform and requires the relying party to have internet access. An online attestation service was created to minimize the complexity of handling multiple security versions for platforms with TCBs for SGX.
[0135] Collaborative learning on SGX proposes a scenario where multiple data owners want to train a model together. The trained model is shared among all data owners. SGX is utilized because data owners do not trust the cloud provider. During the training process, data owners first identify each other and agree on a model design and training code. The code is then uploaded to the cloud enclave. Each data owner encrypts their data and uploads the data to the enclave. The key to the data is only shared after the enclave passes the verification.
[0136] Trusted Execution Environment (TEE) can not only preserve the privacy of data, but also keep the confidentiality of models during training. In this scheme, both data and models are encrypted with hardware keys to prevent information leakage. The calculation of both local model updates and global model updates is performed using trusted (verifiable) code that does not expose the private data of any data owner.
[0137] It may also be possible to sell the dataset to other parties who may want to use it for their own private training and / or application use cases, and / or sell their trained models at a later time. In such a scenario, the data owner may want to advertise the properties of their dataset to increase the confidence of potential users who may purchase the data. Similarly, potential users will want guarantees that the dataset they purchase will be useful for their purposes. The challenge here is to provide this guarantee without exposing private / confidential data to others. Doing so may trivially provide a solution to the guarantee requirement, but will defeat the purpose of monetizing the dataset. Ideally, participants in the market (including the market operator) may not necessarily be trusted entities, which requires providing guarantees via cryptographic means.
[0138] The merkle tree discussed herein can be a hash tree in which each "leaf" node is a cryptographic hash of a data item, and the "internal" node is a cryptographic hash of its child nodes. The tree is built by recursively pairing nodes to obtain their parent nodes and continuing until a single root is reached. Showing that a data item is part of a Merkle tree (i.e., providing proof of membership) requires showing that the corresponding leaf node is present in the tree by calculating a hash proportional to the height of the tree (i.e., the logarithm of the number of data items), which makes it an effective cryptographic commitment scheme. The member description includes the necessary sibling node values in the tree so that the hash root can be calculated. The sibling node is determined based on the Merkle path of the leaf (i.e., the path from a given leaf node to the root).
[0139] Furthermore, the Verifiable Random Function (VRF) discussed herein refers to a cryptographic primitive that enables the generation of a deterministic output from a given input (called alpha) and a secret key. The output is indistinguishable from a random output to anyone who does not know the secret key. The concept is similar to a keyed cryptographic hash; however, a VRF is a public key primitive. Most importantly, the output hash can be verified using the public key corresponding to the secret key.
[0140] VRF is usually composed of four algorithms: (1) a key generation algorithm that generates a secret key and a public key, (2) a VRF evaluation (or hashing) algorithm that deterministically generates a hash value given a secret key and an input (alpha), (3) a VRF proof generator algorithm that generates a proof based on the secret key and the input, and (4) a verification algorithm that ensures the correctness of the VRF hash of the input alpha given the public key, proof, input, and output hash. It is worth mentioning that the verification algorithm does not require a secret key.
[0141] Reference is now made to the drawings. In particular, it should be noted that, unless otherwise indicated, the same or similar reference numerals used in the drawings of the present disclosure may indicate the same or similar elements, so that repeated descriptions thereof may be omitted for the sake of brevity.
[0142] Figure 1 Example use cases are schematically illustrated in which data stream owners monetize their data in a marketplace or data network. Thus, this first scenario is related to monetization in a marketplace, where the data source is related to data such as sensor data of mining equipment. For example, a mining equipment supplier may want to obtain properly sampled data from their mining equipment owned and used by one or more mining companies. Data stream owners want to provide and advertise guarantees about their data streams without losing the privacy / confidentiality of their data. Similarly, potential data users want to have guarantees about a data stream before they purchase or subscribe to it.
[0143] In particular, in order to convince potential users of data streams (e.g., data buyers in a data market), the data stream owner must show the value of the data stream he owns. However, naively handing the data stream to other parties by selling it to other entities defeats the purpose of monetization. Therefore, there is a need to show the value of the data stream without exposing it.
[0144] For example, before a user subscribes to a data stream to use data with a private application goal (e.g., private model training), the user should be able to tell whether the data stream will be useful for his own purposes. There is also an incentive for the data stream owner to skew the data so that more users will find it useful and subscribe to the stream. However, this should be avoided or prohibited by the described methods and systems.
[0145] Figure 2 Schematically illustrated is a scenario in which a manufacturer desires to provide predictive maintenance on its customers' equipment, and therefore wants to receive data streams. In this case, i.e., predictive maintenance for an equipment supplier, the equipment supplier may want to receive data on the performance of its equipment in the real world. However, the manufacturer's customers (e.g., telecommunications providers) are very reluctant to share this information due to the sensitivity of their own customer data. In addition, they may also be concerned that the manufacturer is using their data to help their competitors (e.g., telecommunications provider A is reluctant to share data because the data will be used to improve telecommunications provider B's network). Therefore, they have an incentive to interfere with the data shared with the manufacturer (e.g., withhold it) and / or exploit the intelligence generated by the manufacturer's data with other suppliers.
[0146] Figure 3Yet another example use case in which an energy supplier desires to obtain household electricity usage data to make informed decisions about its investments is schematically illustrated. In this case, i.e., energy usage and investment, energy suppliers and grid operators want to observe the energy consumption of a household (e.g., which devices are used at what time and with what frequency) and how much sustainable energy the user generates (e.g., through solar panels). This information is important for capacity planning and energy investment (e.g., if daily usage seems high, an investment in solar energy may make more sense). However, home owners are concerned about the privacy of their activities. Moreover, the number of data sources (e.g., households, devices) and the frequency of their data generation will certainly overwhelm the data collector (i.e., energy supplier).
[0147] The above use cases all suffer from the common problems of scalability, confidentiality and privacy, and the present disclosure generally seeks to provide a method that helps data stream owners provide data streams to potential users in a scalable, confidential and private manner.
[0148] Ideally, the data stream owner would generate a proof of the properties of the data stream, send the proof to a potential data user, who would then verify the proof to be confident of the properties of the data stream, such as Figure 4 However, there are two aspects to consider:
[0149] 1) Unknown set of useful properties of the data stream: An example of such a useful property is the validity or internal consistency of the data, whether it is meaningful and / or feasible, such as, for personal health data, age less than 120 years or height less than 3 meters. Another example may be informative properties of the data stream, such as statistical means, medians, percentiles and / or distributions. However, many of these properties may not be easily known and exhaustively enumerated, because the utility of a data stream often depends on the application use case as well as the properties of the data stream itself.
[0150] 2) Processing high-volume streaming data: Data streams often require near real-time processing, requiring practical considerations about performance and cost. Performance is important because the timeliness of a data stream may change its value (e.g., a recent data stream may be more useful than an older data stream). In addition, there may be many data sources contributing to a high-rate stream, the data sources or the receivers may have limited network bandwidth, and the size of the data may be large. Similarly, cost comes into play because generating a proof of a data stream should not be more expensive than the subscription price of the stream; otherwise, there is no point in generating a proof.
[0151] Solving this aspect at the same time is an open problem. As will be described later, sampling can provide a solution with necessary considerations. The natural question that arises is then: how can the data stream owner prove to potential data users that the provided samples were created correctly.
[0152] Figure 4 Schematically illustrates a workflow whereby a data stream owner generates a proof of a property and sends the proof to another party, who then verifies the proof;
[0153] Any solution to the problem of correct sampling will also have to account for the fact that the stream data owner may or may not control the data sources that contribute to the data stream. As such, the solution will need to ensure that the data stream owner cannot skew the sample toward his or her own interests by manipulating the data collection process. Such manipulation may be accomplished by adding / hiding data sources and / or discarding collected data (i.e., omission of data).
[0154] Today, there is not a lot of relevant work on providing "correct sampling proofs for streaming data". Using zero-knowledge proofs (ZKPs), a "prover" party can prove to another "verifier" party that a statement is true, without revealing any other information about the statement. ZKPs have received increasing attention in the past few years due to the popularity of cryptocurrencies and their use in scalability solutions. As a result, technical advances have been made to improve their performance and usability. Technically, ZKPs can be used to prove the completeness and correctness of computations including the sampling process. Similarly, if a data owner can enumerate the useful properties of her dataset, she can create a separate ZKP proving that each of the properties holds for her dataset without revealing the contents of the dataset to anyone. However, despite the interest and recent advances, the performance of ZKPs for large datasets and arbitrary computations is still very low, which in turn makes them difficult for streaming data. ZKPs can also be orthogonal to "correct sampling proofs" so that ZKPs can still be completed in a feasible way for some additional properties, since the sample will be much smaller than the original dataset.
[0155] There may also be a data exchange protocol (DEFS) with free samples. In DEFS, the dataset owner first creates a series of encryption keys from a seed, each encryption key corresponding to each data item in the dataset, and builds a Merkle tree from the encryption key. The dataset owner then encrypts the dataset item using its corresponding encryption key and creates another Merkle tree from the encrypted dataset item. The encrypted dataset item is then uploaded to a storage location (e.g., IPFS, web server) where a potential buyer can download the encrypted dataset. The dataset owner also employs smart contracts on a decentralized blockchain with two Merkle tree roots (i.e., encryption keys and encrypted dataset items). Potential dataset buyers download the encrypted dataset item and trigger a smart contract to request a sample by requiring a decryption key at certain locations in the Merkle tree of the encrypted dataset. At this point, the dataset owner reveals the encryption key at that location to the potential buyer along with its Merkle path so that the buyer can ensure that the key belongs to the Merkle tree of the encryption key. If they check out, the buyer decrypts the requested dataset item to get a sample of the dataset and checks the quality of the sample according to her own application logic. If the sample is deemed acceptable, the buyer initiates an agreement to purchase the dataset by locking up sufficient funds in the contract. The dataset owner then reveals the seed used to generate the encrypted key to get paid for the dataset. The main purpose of this line of work is to provide dataset samples to potential buyers within a decentralized data exchange protocol. As such, it only works for static datasets and not for streaming data. Additionally, its focus is on supporting protocols for transactions for dataset exchange.
[0156] There may also be a system NIZK, where they focus on creating non-interactive ZKPs within a TEE. Their system is based on an interpretable language (Lua), whereby the interpreter is compiled into a binary and shared by all users. The user then supplies a script along with public and private inputs so that the script runs within the TEE and produces outputs that are included in a verification report. The verification report is then checked by the user using EPID remote verification. The verification report, which includes the script execution output and the signed corresponding verification result, is then used as a non-interactive proof for other users to verify the correctness of the computation.
[0157] NIZK uses TEE and remote attestation for attestation. Their solution focuses on blockchain scalability and privacy-preserving smart contracts. Therefore, their attestation process is done as scripts that are executed on demand, making it unsuitable for streaming data. Furthermore, it is unclear how NIZK can support large datasets due to the size limitations of the Lua interpreter and the fact that EPID-based attestation can only be used on servers with small EPC sizes.
[0158] There may also be a dataset management system for collaboratively creating, reviewing, updating, and evolving ML datasets. Such datasets may be provided to others, along with a fee and a license. The system provides a mechanism to propagate changes and updates made to a parent dataset to its child datasets (i.e., other datasets that use it). The system may present dataset items in various views so that users can identify problems in the data. The dataset items may have attached metadata that may reflect certain properties of the item that have been reviewed by the user (e.g., images associated with various class labels).
[0159] Although known schemes may use the term "sample", it sometimes refers to a "dataset item". The term "validation" sometimes refers to the "review and verification" of the attached metadata. For example, a dataset used to train an ML model to recognize objects may contain images of various animals. The images are then labeled with the corresponding reviewed and verified animal names (e.g., a dog picture is labeled "dog").
[0160] In contrast, the present disclosure refers to the term "sample" as "a subset of dataset items from a larger dataset" that is produced via a "statistical sampling" method. Verification is then a check of the proof that shows that the subset of dataset items was created correctly and honestly. In other words, the present disclosure is about "proving the correct selection of dataset items to form a subset of the dataset" instead of the content or metadata of the individual dataset items.
[0161] Sampling is a standard statistical tool used in many fields, including big data analysis, stream processing, and approximate computing. By using representative samples, the goal of the approach is to reduce the resources required to process large amounts of data, so that more data can be processed with a fixed number of resources or in a shorter time, without losing much of the accuracy of the results obtained by processing all the data.
[0162] Sampling is therefore a very applicable tool to alleviate the concerns that need to be considered for streaming data: representative samples can address the described challenges, where scalability issues may arise due to limited network bandwidth, increased data size, high frequency of data generation, and a large number of data sources. Furthermore, providing representative samples of data streams can allow data users to evaluate the samples according to their application requirements without violating the confidentiality and privacy of the data stream owners. Enabling such customized evaluations is particularly useful due to the infeasibility of generating proofs for unknown sets of useful properties.
[0163] Because the stream owner owns the data stream that can be supplied to potential users, some users may be reluctant to trust the data stream owner not to interfere with the sampling process of the data source that will produce the data. The present disclosure proposes a method to assure potential users that the samples of the data source (and therefore, the data they produce) are created correctly.
[0164] Note that another solution to the sampling problem is to collect all necessary data at a central location and perform the sampling there. However, this is not always possible due to the large volume of streaming data, the distributed deployment of data sources, and the cost of transmitting the data to a central location simply to discard most of the data (i.e., because the sample will be much smaller). Therefore, it may be expensive to pursue this alternative in some cases. In addition, this solution would require a central party to generate proof so that potential users can be sure that the central sampling process is not biased. In other words, circumventing the proposed solution may be expensive and would have to address issues regarding the correctness of the sampling and regarding the confidentiality and privacy of the data sources targeted in the present disclosure.
[0165] At a high level, the present disclosure employs a verifiable random function (VRF) at each data source, so that each data source can independently evaluate the VRF using their own secret key and determine whether they are selected in the sample. If so, they not only send their data as part of the stream, but also send proof that the selection in the sample is correctly calculated. Therefore, the recipient of the sampled data and the recipient of the proof (i.e., potential users) can check the proof using the corresponding verification key of the source, so that they can obtain a guarantee that the sample is correctly calculated and is representative of the data stream. Given such a data sample, they can check the sample and evaluate whether it meets the requirements of their use case or application. Similarly, each data source that is not selected to be included in the sample sends proof that they are not selected (instead of data). Distributing this verifiable sampling process among the data sources avoids the additional cost of transmitting data to a central location and processing the data there.
[0166] The stream owner may still want to interfere with the sampling process to skew the sample data toward her preferences to convince potential users (e.g., selectively drop data packets). Such potential interference can be avoided and potential data users can be protected because of at least one of the following features: 1) each data source evaluates the VRF privately without involving the stream owner, 2) each data source sends proof of their VRF evaluation (even when not selected in the sample and not providing data), and 3) the verifier (i.e., potential user) can keep track of such proofs to determine whether the data stream owner is untrustworthy or the stream is too unreliable, allowing potential users to avoid purchasing the data stream in either case.
[0167] Note that the sampling process according to the present disclosure is not a one-time event that helps potential users in some cases decide to receive a data stream and use it. Even after the decision to use the data stream is made, the sampling process can continue to help 1) the scalability of the data source and receiver, and 2) the detection of any changes in the reliability and utility of the data stream. For example, in a data market scenario, subscribers to the data stream may only want a subset of the data for scalability and / or pricing reasons and / or may still want to continuously assess the quality of the data stream (and make further decisions to stop their subscription). Similarly, a supplier may want to continuously assess the reliability of a data stream received from a customer in order to further evaluate and adjust its predictive maintenance operations. As a third example, a research organization using a data stream from an air quality monitor provided by a municipality can detect a drop in the received data and certification, which can indicate a network infrastructure problem in the area (e.g., a radio tower is down) and / or a sensor hardware problem (e.g., a battery needs to be replaced), which in turn can be used to notify the data stream owner to take corrective action.
[0168] In an exemplary network according to the present disclosure, the following actors may exist:
[0169] Data Source: A data source produces data that becomes part of a data stream. There can be multiple data sources. Some examples are sensors in a factory or individual users in a crowdsourcing system.
[0170] Data stream owner: A data stream owner may wish to provide his data to other users. However, providing the entire data stream may not always be possible due to scalability concerns (e.g., low network resources, large number of data sources) or acceptability (e.g., confidentiality would be violated, providing all data defeats the purpose of monetization). Therefore, portions of the data will be provided that can be used as explorations about the properties of the data stream. The provided data will be generated by a randomly selected sample of the data sources, which produces a data sample. This data sample can be used for further decisions (e.g., fully subscribing to the stream, continuing to use such a sample for scalability or pricing).
[0171] Data User: A data user may wish to obtain data samples from the data source of the data stream owner. He needs a guarantee that the sampled data source is correctly selected (i.e., randomly selected) and that data sources are not added or removed (i.e., skewing or manipulating the properties of the sampled data) at runtime.
[0172] For some implementations, one or more of the following assumptions about the scene and actors may be considered:
[0173] 1) Each data source (via trusted hardware and / or other cryptographic schemes) generates data correctly (e.g., the sensing is correct). This means that even if the data stream owner controls the data source, he cannot interfere with the correctness of data generation. He can only deny service (e.g., by dropping packets).
[0174] 2) Each data source signs the data it generates using its private key, which can be checked using the corresponding public key.
[0175] Given the above actors and assumptions, the goal is to obtain a sample of streaming data generated by multiple data sources, along with a proof that the sources contributing to the sample were correctly and honestly chosen (ie, randomly selected).
[0176] The protocol according to this implementation is described below. Each data source has a secret key (SK) and a corresponding verification key (PK) to be used for VRF evaluation of the data source for sampling. It may be the case that the key is the same key used to sign the data (public key == verification key, private key == secret key), but these keys are not necessarily the same. In the following, the protocol is described as if the key (as used to sign the data) is the same.
[0177] When a data stream owner wants to provide data to potential users, he declares his intention. This declaration can take place on a data marketplace / data exchange, on a decentralized system with smart contracts and transactions via news releases, via customer relationships, and through other communication means. The exact nature of this declaration depends on the use case and is outside the scope of this invention. There may be additional supporting agreements to handle the actual purchase, subscription, use, and delivery of the stream data. This supporting agreement is also outside the scope of this disclosure.
[0178] The data stream owner asserts one or more of the following information and commits to that information via its own signature (e.g., via a data exchange protocol):
[0179] 1) The number of data sources and their public keys (either as a hash of a list of public keys, or the root of a Merkle tree built using that public key);
[0180] 2) Sampling probabilities and algorithms (also known as “lotteries”);
[0181] 3) An update mechanism for the public seed (ie, alpha).
[0182] Data generation is divided into epochs (also called time slots or time slots). Each data source at each epoch will generate some data. The common input to the VRF is the epoch representation value alpha.
[0183] Depending on the implementation and use case, there may be many ways to obtain the epoch representation in various embodiments. The epoch change may be incremented by a fixed and predetermined amount. For example, the epoch change may be updated every x seconds, such as 10:30:00, 10:30:10, 10:30:20, etc., where x is 10. Note that there are other ways for data sources to synchronize on the alpha value for the new epoch. For example, the data sources may be part of a decentralized blockchain and reach a consensus on the next value. It may also be the case that a separate authority (e.g., a random oracle) announces the next value to all data sources, or that each data source contacts the authority to obtain an updated epoch value. The value may correspond to a hash of a previous data sample, and / or other arbitrary value determined by the authority. Depending on the use case, there may be other ways to obtain and synchronize the value among the data sources.
[0184] At each epoch, each data source may use the same deterministic input (ie, alpha) and their own secret key (SK) for VRF evaluation and obtain a proof pi such that pi = VRF_prove(SK, alpha).
[0185] Each data source will then obtain a random value beta such that beta = VRF_proof_to_hash(pi). The beta value will be used by the data source to evaluate whether the data source has been selected to be included in the sampling for that period based on the sampling probability and algorithm declared by the data stream owner. This evaluation can also be called a "lottery".
[0186] The sampling probability can be adjusted according to the number of data sources, the rate at which the data sources generate data, and the use case. When interpreted as a large number, the lottery algorithm requires that beta can be less than or greater than a certain value. Alternatively, the number of zeros in its binary representation can meet some criteria. This conditional check in the lottery algorithm is adjusted and set by the data stream owner according to the range of random numbers (i.e., beta) and the sampling probability, so that the expected number of data sources in the sample can be calculated by multiplying the total number of data sources by the sampling probability.
[0187] If a data source determines via the lottery algorithm that it is to be included in the sample, then the data source sends its data, its public verification key (or an index into a list of keys), and a proof (i.e., pi) showing that it should be included in the sample. This proof may be referred to as a "proof of inclusion" because it corresponds to including the data source (and its data) in the sample.
[0188] The data user (i.e., the recipient of the data and proof) acts as a verifier and checks one or more of the following:
[0189] 1) For each data source received in the sample, the receiver checks whether the public key is included in the list of public keys declared by the data stream owner. They do this by simply checking the signature list of the public key or by obtaining a membership certificate (i.e., Merkle path) for the corresponding public key from the data stream owner. If the public key is not included in the declared list of data source public keys, the data source and its data are ignored.
[0190] 2) For each data source received in the sample that has a public key that is valid and included in the sample, the receiver calculates the validity of the proof is_valid = VRF_verify(PK, alpha, pi) for the current epoch (i.e., with the corresponding alpha value). If the proof is valid, the receiver calculates beta_prime = VRF_proof_to_hash(pi) and evaluates the same lottery algorithm used by the data source to see if the data source was indeed selected in the sample. In other words, the receiver checks that beta_prime satisfies the same conditions as the beta value (of the data source). Any data source and its data that should not be in the sample due to invalid proof or unsatisfactory beta_prime value is ignored.
[0191] If the dataflow owner uses other means to update the common input (i.e., alpha) to be used in the VRF evaluation in the next epoch, then the dataflow owner does so before the next epoch begins. If not, then the alpha value is updated by each data source according to a predefined update mechanism (e.g., if time-dependent, it advances to the next predefined value).
[0192] Also note that preferably the proof is verified by the recipient using the same public input alpha. The recipient obtains a public alpha value similar to the data source, and only uses alpha values that belong to the current epoch. In other words, if a valid proof is reached that does not belong to the current epoch (i.e., it may still be valid because the proof is using an earlier / later version of the alpha value), then the proof can be considered an invalid proof.
[0193] Alternatively or additionally, each data source may send the alpha value they use along with the proof in a tamper-proof manner (i.e., signing the value). The recipient would then check that the alpha value has not been tampered with by checking the signature, and then use the alpha to verify the proof. Note, however, that this additional signature may create additional computational and network overhead at the data source. It should also be noted that the use case may require timely processing of streaming data; therefore, any delayed packets belonging to an earlier validity period may still be considered invalid.
[0194] Figure 5 An example of how to generate data samples using 5 data sources (here, sensors) along 3 periods (here, sensing cycles) is shown. Here, for illustration purposes, the sampling probability and lottery algorithm are stated to be approximately 40%. In each sensing cycle, each sensor will evaluate the VRF using a given public seed (e.g., sensing cycle number) and its own secret key. The result will determine whether the sensor's data is included in the sample. In the first sensing cycle, sensor 1 and sensor 3 are selected by the lottery algorithm, and therefore the data of sensor 1 and sensor 3 are sent together with their proof of VRF evaluation on the lottery. Similarly, in the second sensing cycle, sensor 1 and sensor 2 are selected. Finally, in the third period, sensor 4 and sensor 5 are selected in the sample.
[0195] Note that due to the probabilistic nature of the selection process, there may be periods with more or fewer data sources selected in the sample for that period. However, in the long run, the average number of data sources in the sample will converge to the expected number of data sources (i.e., the total number of data sources multiplied by the sampling probability).
[0196] Note that it is also possible to evaluate a lottery once and use its results for multiple periods. For example, if the data stream includes a data source that generates data at a high rate (e.g., 10KHz), the lottery can be completed every 5 seconds, so that the data source will send its data until the next lottery evaluation, and the certificate will be considered valid until then.
[0197] An alternative or additional feature is to omit detection. It is possible that some data sources may not be able to send data because they have unreliable data generation (e.g., overloaded hardware) and / or unreliable network connectivity (e.g., intermittent coverage of wireless signals). Some data sources may also be malicious, such that they may intentionally not send their data when the data source should send their data (i.e., if the VRF evaluation has provisions with their secret keys and lottery algorithms). It is also possible that the data stream owner may want to interfere with the sampling process to skew the data sample to make it look "better". In all of these cases, some data and corresponding proofs may be missing and therefore omitted.
[0198] Detecting such omissions may be important because it will enable potential data users to assess the quality of the stream (and not just the quality of the data supplied in the sample). As a result, potential users may not continue to receive data (e.g., users in a market may not subscribe to a data stream, a supplier may exclude a customer's data source from its own use).
[0199] This omission can be detected and handled via two solutions:
[0200] 1) Exclusion Proof: Regardless of whether the VRF evaluation results in inclusion in the sample, each data source sends their proof (i.e., both inclusion proof and exclusion proof are sent). The verifier (i.e., data user) checks the proof and evaluates the lottery algorithm and only accepts data from sources that should be in the sample (i.e., inclusion proof). If a data source should be in the sample but it did not send data, the verifier can generate a flag. In addition to this, the verifier also keeps track of how often each data source is missing an exclusion proof and the total number of missing exclusion proofs. If a data source does not continuously send its proof (inclusion or exclusion), the verifier may become suspicious of malicious activity (e.g., deliberate omission) or unreliable activity (e.g., unreliable data source and / or network connectivity).
[0201] 2) Periodic Timed Release: On a time scale coarser than data generation (e.g., every 1000 epochs), each data source reveals its secret key. For each previous epoch, the verifier evaluates the VRF of each data source using the corresponding secret key and checks the lottery algorithm. If the data source should send data in that epoch, the verifier checks whether it has received the data from the data source within that epoch. If not, the verifier may become suspicious. Similarly, if the data source does not reveal its secret key, the verifier may become suspicious. After revealing the secret key of the data source, the data source creates a new secret key for their VRF evaluation (e.g., via key evolution signatures) for future epochs. The data stream owner also updates the declared public key list (or Merkle tree) with the new public key.
[0202] Fig. 6A and Figure 6B The proposed setup between a stream owner / stream proxy and a plurality of stream receivers / stream users (stream receivers 1, 2, ... m) is schematically illustrated.
[0203] In step 60, the stream owner / stream proxy may publish the n data sources (and additional metadata) for the data stream.
[0204] In step 61, the stream receiver / user may request a stream with a sampling frequency f (i.e., affecting the public seed update mechanism). After the stream receiver / stream user receives and / or obtains the number of published data sources from the stream owner / stream proxy, the stream receiver / stream user may send a request to the stream owner / stream proxy at the sampling frequency.
[0205] In step 62, the stream owner / stream agent may confirm and move to "Setup" in the next step. Also, after the stream owner / stream agent receives a stream having a sampling frequency from at least one stream receiver / stream user, the stream owner / stream agent may send an ACK (acknowledgement character) to the stream receiver / stream user, respectively, and go to "Setup".
[0206] Fig. 7A and Figure 7B The setup is schematically illustrated. In step 70, the stream owner / agent may configure the data source to utilize: 1) sampling logic with sampling probability; 2) a common seed update mechanism (e.g., every 10 minutes). In this step, the stream owner / stream agent may respectively send a configuration signal to at least one data source, wherein the configuration signal includes: 1) sampling logic with sampling probability; 2) a common seed update mechanism (e.g., every 10 minutes).
[0207] In step 71, the data source may generate two public / private key pairs: 1) to sign the generated data; 2) to evaluate the VRF for sampling. In this step, each data source may generate two public / private key pairs: 1) to sign the generated data; 2) to evaluate the VRF for sampling upon receiving the configuration signal from the owner / agent. Thereafter, each data source may send the generated public key to the stream owner / stream agent.
[0208] In step 72, the owner / agent may collect public keys and may put them in a list. In this step, the owner / agent may at least receive a public key from a data source and may put the public key in a list.
[0209] In step 73, the owner / agent may publish with her own signature: 1) Data source public key; 2) Sampling logic; 3) Public seed update mechanism. In this step, the owner / agent may commit to the sampling logic and / or public seed update mechanism with sampling probability, and may sign the public key list, sampling logic and / or update mechanism with its own private key. The public key list (PKL), sampling logic (SL) and update mechanism (PSUM) signed with the owner / agent's own private key may be included in a signed configuration signal. The signed configuration signal may then be sent to each stream user / stream receiver.
[0210] In step 74, the stream user / stream receiver may verify the signature of the stream owner / stream proxy and store: 1) the public key list; 2) the sampling logic; 3) the public seed update mechanism.
[0211] FIG8A(1), FIG8A(2) and Figure 8BThe flow usage for each sampling period is schematically illustrated. In step 80, for each sampling period: 1) the data source can evaluate the VRF using its private key and sampling logic; 2) if in the sample, the data source signs the data they produce and sends the (inclusion) proof of the data source; 3) optionally, if not in the sample, the data source sends the (exclusion) proof.
[0212] In step 81, the stream owner / stream proxy may collect the received data and proof packets and send the data and proof packets to the stream user / stream receiver.
[0213] In step 82, for each packet, the stream user / stream receiver may: 1) verify that the public key is in the stored list; 2) verify the VRF certificate using the VRF public key; 3) verify the signature of the data.
[0214] As a result, providing proof of correct sampling for streaming data as proposed herein increases confidence among data users, as well as enables data stream owners to increase their utilization of their data streams (e.g., monetization, creation of predictive maintenance intelligence, optimization of internal operations).
[0215] In one practical implementation of the described system, multiple data sources and a single data sink are provided as well as sanity checks on sampling probability (lottery) evaluation (i.e., by keeping track of the sample size and comparing it to the sampling probability and number of data sources). The implementation illustratively uses elliptic curve cryptography as the underlying public / private key scheme with the Edwards25519 curve (EDWARDS25519) with SHA512. The proof size (i.e., pi) is 80 bytes, while the secret key size (i.e., SK) and verification key (i.e., PK) sizes are each 32 bytes.
[0216] In one implementation of 100s of VRF proof generation and verification, the default VRF hashing using SHA512 takes about, for example, 0.85ms. The VRF proof computation takes about 320ms. Note that this proof computation is performed in parallel by each data source in each data generation epoch. On the other hand, the VRF proof verification performed by the data sink (i.e., the recipient of the data sample and the proof) will take about, for example, 230ms per proof (i.e., per data source in the epoch). Even though such a verification process may seem expensive, it is an awkwardly parallel process and can be easily scaled horizontally to multiple servers (e.g., running on the cloud). Also note that the data sink is actually a potential data user and therefore may have the computational power necessary to process the sample data (and potentially the entire data stream).
[0217] A comparison of solutions for omitted detection is described below.
[0218] The criteria for using exclusion evidence or periodic timed release depends on at least one (and preferably all) of the following factors:
[0219] 1) N: Number of data sources
[0220] 2) S_proof: the size of the proof
[0221] 3) S_data: the size of the data
[0222] 4) S_pk: verification key size
[0223] 5) S_sk: size of the secret key
[0224] 6) P: sampling probability
[0225] 7) T_sk: key release period
[0226] Data sent with only inclusion proof: N*P*(S_data+S_pk+S_proof);
[0227] The data sent with inclusion proof and exclusion proof: N*P*S_data+N*(S_pk+S_proof);
[0228] Data sent with timed release including proof and key: N*P*(S_data+S_pk+S_proof)+N*S_sk / T_sk;
[0229] If the sample size is relatively small (e.g., 1%), the overhead of sending the exclusion proof may be too much. If the amount of data sent by each data source is large, the overhead of the exclusion proof may become negligible. Note that instead of the actual authentication key (PK), the data source can also send the index of the authentication key in the list, which will limit S_pk to 4 bytes or 8 bytes (instead of 32 bytes).
[0230] The periodic timed release frequency is relatively low; therefore, the overhead of sending secret keys from all data sources may be low. On the other hand, all verifiers will have to re-evaluate the VRF for all data sources with the released secret keys for the entire earlier period (i.e., T_sk period), which can create a non-negligible computational overhead.
[0231] Note that the verification process is awkwardly parallel and can be parallelized via multiple threads and / or multiple servers (e.g., in a cluster and / or cloud environment). Note also that the verification process can be outsourced to a trusted third party (e.g., a cloud provider) or can be performed within a Verifiable Computing as a Service (VCaaS), whereby the service checks the proof on behalf of the recipient and produces a single proof that all data source proofs are correctly checked. The VCaaS can be based on zero-knowledge proofs or on guarantees provided by trusted hardware such as SGX or a trusted execution environment (TEE) (see NC327989: Verifiable Computing as a Service with a Trusted Execution Environment for Static and Streaming Data).
[0232] Regarding inclusion proofs, it is conceivable that if data sources are sharing data, it can be implicitly assumed that their VRF and lottery evaluations indicate that they should contribute to the data sample. However, such an assumption is not always valid for at least two reasons, requiring the use of inclusion proofs. First, the data stream owner may want to ensure that only randomly selected data sources contribute to the data sample, so that no more than required data is exposed to potential data users (e.g., compromised / manipulated data sources). Second, the data stream owner may attempt to manipulate the data sample. Recall that we assume that the data generation at the source is correct. However, the data stream owner may still attempt to manipulate the data sample by having more data sources supply data (e.g., injecting more data to skew the sample), and the recipient will not be able to tell the difference. Inclusion proofs ensure that only sources that win the lottery supply data for the sample, addressing this concern.
[0233] Regarding the choice of common input alpha, note that the choice of common input (i.e., alpha) for VRF evaluation is important because it affects how each data source will determine whether they will be included in the sample and whether potential data users can correctly verify the proof. We have described the protocol such that the alpha value is updated at each epoch using a predetermined procedure (e.g., time-dependent data stream owner synchronization). However, such a scheme requires additional analysis regarding the ability of data stream owners to interfere with the sampling process (see the "Omission Detection" section). Note that in some cases it is assumed that the data generation process is correct via trusted hardware and / or various cryptographic schemes (such as commitments); therefore, the only thing that a data stream owner can do to interfere with the sampling process is to prune certain data sources and / or discard certain data.
[0234] Here, there are two cases to consider:
[0235] Case 1: The data stream owner cannot have access to the VRF secret keys of the individual data sources. An example of this case is a crowdsourced data generation system, where the data stream owner acts as a coordinator / agent, but does not necessarily control the individual data sources and their keys. Therefore, the data stream owner cannot pre-evaluate the lottery algorithm to determine in advance which data sources will be part of the sample in a given period, and cannot target them individually (i.e., drop their packets from the sample to skew the sample).
[0236] On the other hand, each individual data source may perform this pre-computation when they will be part of the sample, and may intentionally not send data (and proofs) to interfere with the data in the sample. First, it is noted that such data sources may be incentivized to participate and send their data (e.g., token rewards). Second, a single individual data source may not be able to influence the resulting sample too much.
[0237] In any case, the omission problem can be solved by requiring either exclusion proofs or periodic timed release of the key: when data sources are not part of the sample, they still need to send exclusion proofs without any actual data. If a certain data source does not provide the required proof, it can be detected by the potential user, who can keep track of the data source via its public verification key. The potential user can then set and adjust their threshold of how many such problems they will allow, and if they consider it unreliable due to the presence of such a data source, they will have the privilege of not continuing to use the data stream.
[0238] Case 2: The data stream owner has access to the VRF secret keys of the various data sources. An example of this case is a factory owner providing data generated by its sensors on the factory floor. As a result of this possession, the data stream owner can access the secret keys of the data sources, allowing to pre-compute when the data sources will be part of the sample in the future. This knowledge opens up the possibility for the data stream owner to skew the sample by targeting the data sources individually (e.g., pruning / dropping their packets).
[0239] This problem can be alleviated if the alpha value is synchronized among the data source instead of among the data stream owner. However, this is not feasible because the data source is owned by the data stream owner.
[0240] A potential user could introduce some secret into the updating process of the alpha value for the upcoming epoch (e.g., by sending the same secret nonce to the data source encrypted with its public key), such that alpha = epoch number + nonce. In this way, the potential user could use the same alpha as the data source and verify the proof. However, given that the data stream owner owns the data source (and has access to its key), this solution also does not solve the problem.
[0241] On the other hand, employing an omission detection solution as described above would enable potential data users to again keep track of how often such problems occur and exploit the data stream based on their decisions accordingly.
[0242] In summary, in any of the cases in this case, even when the public input alpha value for VRF evaluation can be known and used by a potential malicious entity (e.g., data flow owner, individual data source), the adoption of the omission detection method will help potential data users make decisions. In addition, the omission detection method also helps in cases where there is no malicious entity but the data flow is unreliable due to network connectivity issues.
[0243] In summary, compared to conventional technologies, there may be one or more of the following new aspects: first, when a stream owner / stream user needs to collect stream data from its client, in order to address scalability and confidentiality issues, sampling from different data sources is used; second, proof of correct sampling is used for verification by sample users / sample receivers; third, sample users / sample receivers can make fair decisions based on the verification of samples.
[0244] It should be noted that although in the above-mentioned example embodiments (refer to the accompanying drawings), the messages communicated / exchanged between network components / elements may appear to have specific / explicit names, depending on various implementations (e.g., underlining techniques), the messages may have different names and / or be communicated / exchanged in different forms / formats, as can be understood and appreciated by those skilled in the art.
[0245] According to some example embodiments, corresponding methods suitable for being performed by an apparatus (network element / component) as described above (such as a UE, CU, DU, etc.) are also provided.
[0246] It should be noted, however, that the device (equipment) features described above correspond to corresponding method features, which, however, for reasons of simplicity, may not be explicitly described. The disclosure of this document is considered to also extend to such method features. In particular, the disclosure is understood to be relevant to methods of operating the device described above, and / or to providing and / or arranging corresponding elements of the device.
[0247] In addition, according to some other example embodiments, a corresponding device (for example, implementing the UE, CU, DU, etc. described above) is also provided, which includes at least one processing circuit system and at least one memory for storing instructions to be executed by the processing circuit, wherein the at least one memory and the instructions are configured to be together with the at least one processing circuit so that the corresponding device performs at least the corresponding steps described above.
[0248] In some other example embodiments, a corresponding apparatus (eg, implementing the UE, CU, DU, etc. described above) is provided, the apparatus comprising corresponding components configured to at least perform the corresponding steps described above.
[0249] It should be noted that the examples of embodiments of the present disclosure are applicable to a variety of different network configurations. In other words, the examples shown in the figures described above, which are used as the basis for the examples discussed above, are merely illustrative and do not limit the present disclosure in any way. That is, additional existing functions and proposed new functions available in the corresponding operating environment can be used in conjunction with the examples of embodiments of the present disclosure based on the defined principles.
[0250] It should also be noted that the disclosed example embodiments can be implemented in many ways using hardware and / or software configurations. For example, the disclosed embodiments can be implemented using dedicated hardware and / or hardware associated with software executable thereon. The components and / or elements in the accompanying drawings are merely examples and do not limit the scope of use or functionality of any hardware, software combined with hardware, firmware, embedded logic components, or a combination of two or more such components that implement a particular embodiment of the present disclosure.
[0251] It should also be noted that the description and the accompanying drawings only illustrate the principles of the present disclosure. Those skilled in the art will be able to implement various arrangements, which, although not explicitly described or shown herein, embody the principles of the present disclosure and are included in its spirit and scope. In addition, all examples and embodiments outlined in the present disclosure are primarily and explicitly intended to be used only for illustrative purposes to help the reader understand the principles of the proposed method. In addition, all statements of the principles, aspects, and embodiments of the present disclosure and their specific examples provided herein are intended to cover their equivalents.
[0252] List of abbreviations:
[0253] Trusted Execution Environment (TEE)
[0254] Security Guard Extensions (SGX)
[0255] Zero-Knowledge Proof (ZKP)
[0256] Machine Learning as a Service (MLaaS)
[0257] Elliptic Curve Digital Signature Algorithm (ECDSA)
[0258] Enhanced Privacy ID (EPID)
[0259] Trusted Computing Base (TCB)
[0260] Data Center Authentication Primitive (DCAP)
[0261] Verifiable Compute as a Service (VCaaS)
Claims
1. A first network element for providing at least a portion of a data stream and / or a data item as a data source based on a selection of the first network element to be included in a sample of the data stream, wherein the first network element comprises: at least one processor; as well as at least one memory storing instructions that, when executed by the at least one processor, cause the first network element to at least: determining whether the first network element is selected to be included in the sample using a first function having a secret key (SK) specific to the first network element; wherein if it is determined that the first network element is selected to be included in the sample, providing at least a portion of the data flow and / or data items to one or more second network elements as part of the sample, and providing an inclusion proof indicating that the data to be part of the data sample was correctly calculated.
2. The first network element of claim 1, wherein the first function is a verifiable random function (VRF) employed at the network element to independently evaluate the VRF using a secret key (SK) specific to the first network element.
3. A first network element according to claim 1 or 2, wherein if the first network element determines that it is not selected to be included in the sample, the first network element is caused to: The second network element is provided with an exclusion certificate that the first network element was not selected in the sample, but preferably without providing data as part of the sample.
4. The first network element according to claim 1 or 2, wherein the first network element is a data source randomly selected from a data source set; The first network element is configured to: generate data forming part of the data flow to be provided to the second network element; The sample is a subset of data set items of the data stream determined via at least one statistical sampling method; The sample includes a plurality of data items of the first network element; and / or The inclusion proof is used to verify that the selection in the sample was calculated correctly.
5. The first network element according to claim 1 or 2, wherein the first network element is further configured to: The first function is evaluated with the secret key (SK) specific to the first network element and a public key (PK).
6. The first network element according to claim 1 or 2, further configured to sign the data generated by it using its private key, which private key can be checked using the corresponding public key (PK) and / or verification key.
7. The first network element of claim 4, wherein the first network element is further caused to: receiving a control signal from the second network element; generating the public key (PK) and / or the secret key (SK); The public key (PK) is provided to the second network element.
8. The first network element of claim 7, wherein: The data stream is divided into time slots; the first network element at each time slot generating a portion of the data of the data flow; Each time slot of the data stream is configured with a time slot representative value; and wherein, The slot representation value is used as part of the input for the proof generation and verification.
9. The first network element of claim 7, wherein the time slot representative value is obtained by: updating the time slot representative value at a predetermined time point, wherein the time slot representative value changes incrementally at the predetermined time point; or If the first network element is part of a decentralized blockchain, reaching a consensus on a next value for the slot representation value; or receiving the next value of the time slot representative value from another network element configured to announce the next value of the time slot representative value; or The next value of the time slot representative value is obtained by requesting it from another network element.
10. The first network element of claim 7, wherein The next value of the time slot representative value corresponds to a hash of a previous sample, and / or another arbitrary value determined by another network element.
11. The first network element according to claim 6 or 7, further comprising: At each time slot, the first network element is configured to use the public key (PK) and the secret key (SK) for the first function evaluation and obtain pi as a VRF proof, wherein pi is calculated based on the public key (PK) as a proof of the properties of the data flow and / or the data item.
12. The first network element according to claim 11, wherein: The first network element is configured to obtain a value beta as a VRF hash output and use beta to evaluate whether it has been selected to be included in the sample of the time slot according to a sampling probability algorithm; wherein, Beta is calculated based on pi; where The sampling probability is used to calculate the expected number of the first network elements in the sample by multiplying the total number of the first network elements by the sampling probability.
13. The first network element according to claim 1 or 2, wherein the first network element is further configured to: releasing the secret key (SK) at a predetermined time period, and A new secret key (SK) is created for the first function evaluation.
14. The first network element of claim 1 or 2, wherein the exclusion proof is based on at least one of: the number of the first network elements, the size of the proof, the size of the data, the size of the public key (PK), the size of the secret key (SK), a sampling probability, and a key release period.
15. The first network element according to claim 1 or 2, wherein the periodic key release depends on at least one of the following: the number of the first network elements, the size of the certificate, the size of the data, the size of the public key (PK), the size of the secret key (SK), a sampling probability, and a key release period.
16. A method of providing proof of correct sampling for flow data in a network having at least a first network element and a second network element, wherein the method comprises: employing a first function at each first network element such that each first network element independently evaluates the first function using their own secret key and determines whether they are selected in the sample; If the first network element determines that it is selected in the sample, at least a portion of the data flow and / or data item is provided to one or more second network elements as part of the sample and a proof of inclusion is provided as proof of properties of the data flow and / or data item.
17. A computer readable medium having stored thereon instructions which, when executed by at least one processing unit of a machine, cause the machine to perform the method of claim 16.