Privacy-preserving video analytics

Abstract

Generally discussed herein are devices, systems, and methods for privacy-preserving video. A method may include identifying which classes of objects are present in video data, for each of the classes identified in the video data, generating a corresponding video stream that includes objects of the class and excludes objects that do not belong to the class, and providing each of the corresponding video streams to a content distribution network.

Description

Background Art

[0001] A smart city is an urban area that uses different types of electronic sensors (such as Internet of Things (IoT) sensors) to collect data and uses insights from the data to manage assets, resources and services. Data can be collected from citizens, devices and assets. Data is processed and analyzed. Data can be used to monitor and manage transportation systems, power plants, utilities, water supply networks, waste management, crime detection, information systems, schools, libraries, hospitals and other community services. Some smart city applications (from traffic management to public safety alerts) rely on real-time analysis of videos from surveillance cameras in public places. However, more and more government regulations stipulate how data collected from these cameras must be handled to protect the privacy of citizens. Blacklisting objects in video streams (i.e., defining invisible content, attempting to delete invisible content, and displaying everything else) is error-prone and often leads to violations of privacy. Summary of the invention

[0002] This summary is provided to introduce aspects of the embodiments in a simplified form, and the embodiments are further explained in the detailed description below. The summary is not intended to identify necessary or essential features of the claimed subject matter, and the combination and order of elements listed in the summary is not intended to provide limitations on the elements of the claimed subject matter.

[0003] A system may include object detector circuitry to identify which classes of objects are present in video data. The system may include processing circuitry to generate, for each of the classes identified in the video data, a corresponding video stream that includes objects of the class and excludes objects that do not belong to the class. The processing circuitry may provide each of the corresponding video streams to a content distribution network.

[0004] The object detector circuit system may also be configured to, for each of the categories, identify pixels of each frame of the video data corresponding to an object of the category. The corresponding video stream may include pixel data within a bounding box, ellipse, or outline of the object of the category from the video data, and pixel data corresponding to pixels outside the edited bounding box, ellipse, or outline. The object detector circuit system may also be configured to provide metadata to each frame of the video data, the metadata indicating the identified category and the corresponding confidence value of the category for each identified object.

[0005] The device may also include encryption circuitry to encrypt each of the corresponding video streams using a different corresponding encryption key before providing each of the corresponding video streams to the content distribution network. The corresponding video stream may also include pixel data determined to correspond to a background from which editing is performed.

[0006] The device may also include a compression circuit system to compress each of the corresponding video streams before providing each of the corresponding video streams to the content distribution network. The device may also include a communication circuit system to receive a request for the video stream of the category and generate a request to the object detector circuit system to identify the object of the category in the video data. The object detector circuit system may also be configured to generate the video stream of the category in response to receiving the request.

[0007] The request for the video stream comes from a policy engine separate from the content distribution network. The policy engine can verify that the user is authorized to view the video stream of the category before transmitting the request. The object detector circuit system can also be configured to implement a convolutional neural network model to identify which categories of objects are present in the video data.

[0008] A non-transitory machine-readable medium may include instructions that, when executed by a server, cause the server to perform operations. The operations may include identifying which classes of objects are present in video data. The operations may include, for each of the classes identified in the video data, generating a corresponding video stream that includes objects of the class and excludes objects that do not belong to the class. The operations may include providing each of the corresponding video streams to a content distribution network.

[0009] The operations also include, for each of the categories, identifying pixels of each frame of the video data corresponding to an object of the category. The corresponding video stream may include pixel data within a bounding box, ellipse, or outline of the object of the category from the video data, and pixel data corresponding to pixels outside the edited bounding box, ellipse, or outline. The operations may also include providing metadata to each frame of the video data, the metadata indicating, for each identified object, the identified category and a corresponding confidence value for the category.

[0010] A method for providing video data streams in a privacy-preserving manner may include performing operations of a machine-readable medium. The operations may include encrypting each of the corresponding video streams using a different corresponding encryption key before providing each of the corresponding video streams to a content distribution network. The corresponding video streams also include pixel data determined to correspond to a background from which editing is to be performed.

[0011] The operations may also include compressing each of the corresponding video streams before providing each of the corresponding video streams to the content distribution network. The operations may also include receiving a request for a video stream of a category. The operations may also include generating a video stream that includes only objects of the category. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A diagram comparing whitelist and blacklist approaches to image data is illustrated by way of example.

[0013] Figure 2 Graphs of precision and recall for three object detection techniques are illustrated by way of example.

[0014] Figure 3 A graph comparing the privacy loss between whitelists and blacklists for various object detection techniques is illustrated by way of example.

[0015] Figure 4 A diagram of an embodiment of a system for privacy protection of video streaming is illustrated by way of example.

[0016] Figure 5 A chart comparing the inference frame rates of some object detection techniques is illustrated by way of example.

[0017] Figure 6 A graph illustrating precision and recall for some object detection techniques by way of example.

[0018] Figure 7 Snapshots of five edited video streams created for each application satisfying Table I using the whitelist approach are illustrated by way of example.

[0019] Figure 8 Snapshots of five edited video streams created for each application satisfying Table I using the blacklist approach are illustrated by way of example.

[0020] Fig. 9 A flow chart of an embodiment of a method for protecting privacy in a video stream while still allowing access to authorized viewers of the video stream is illustrated by way of example.

[0021] Fig.10 A conceptual block diagram of an embodiment of a system for privacy-preserving video streaming is illustrated by way of example.

[0022] Fig.11 A diagram of an embodiment of another method for streaming video data in a privacy-preserving manner is illustrated by way of example.

[0023] Fig.12A block diagram of an embodiment of a machine (eg, a computer system) for implementing one or more embodiments is illustrated by way of example. DETAILED DESCRIPTION

[0024] In the following description, reference is made to the accompanying drawings which form a part thereof, and in the accompanying drawings, specific embodiments that can be put into practice are shown by way of illustration. These embodiments are described in sufficient detail to enable those skilled in the art to practice them. It should be understood that other embodiments may be utilized, and structural, logical and / or electrical changes may be made without departing from the scope of the embodiments. Therefore, the following description of the embodiments should not be construed as limiting, and the scope of the embodiments is defined by the appended claims.

[0025] In some embodiments, the operations, functions or techniques described herein can be implemented in software. The software may include computer executable instructions stored on a computer or other machine-readable medium or storage device, such as one or more non-transient memories (e.g., non-transient machine-readable media) or other types of hardware-based storage devices (local or network). In addition, such functions may correspond to a subsystem, which may be software, hardware, firmware or a combination thereof. Multiple functions may be performed in one or more subsystems as needed, and the described embodiments are only examples. The software may be performed on a digital signal processor, an application specific integrated circuit system (ASIC), a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), or other types of processors operating on a computer system (such as, a personal computer, a server or other computer system), for turning such a computer system into a specially programmed machine. Functions or algorithms may be implemented using processing circuit systems, such as may include electrical and / or electronic components (e.g., one or more transistors, resistors, capacitors, inductors, amplifiers, modulators, demodulators, antennas, radios, regulators, diodes, oscillators, multiplexers, logic gates, buffers, caches, memories, GPUs, CPUs, FPGAs, ASICs, etc.).

[0026] Embodiments can advantageously improve the privacy of smart city or other video analysis applications. Embodiments can help balance the privacy and utility of video analysis by editing videos of smart city and other applications (e.g., near real-time or real-time). By whitelisting objects or blocking by default, scalable video analysis with improved performance characteristics of privacy protection compared to blacklisting can be achieved. Whitelisting reduces (e.g., minimizes) the risk of object detection errors that lead to privacy violations. Whitelisting also helps embodiments to expand to a large and diverse collection of applications. In particular, whitelisting reduces the bandwidth and computing resources required to generate object-specific video streams. Experiments show that whitelisting can significantly improve privacy performance (privacy performance is about 5,000 times higher than blacklisting) and bandwidth savings (up to about 5.5 times) compared to blacklisting objects, and the utility loss is negligible. Any utility loss can be mitigated, such as by including metadata in the video stream.

[0027] Surveillance cameras are ubiquitous in public places. To leverage these cameras, several video analytics frameworks have been proposed that can ingest video data and perform object detection and tracking on behalf of video analytics applications. Examples of such frameworks include traffic management, pedestrian detection, and public safety alerts. Cameras are an attractive data source for smart city initiatives because they can support a wide range of applications and are relatively easy to install. In contrast, dedicated sensing infrastructure such as road induction loops or radio frequency identification (RFID) tags and readers support a much smaller set of use cases and are more difficult to deploy.

[0028] The convenience of surveillance cameras comes at the cost of eroding citizen privacy. Cameras collect data indiscriminately, so the information they capture is not necessary for the application and can be easily misused. Governments around the world have responded to the privacy threats posed by surveillance technology. For example, the European Union passed the General Data Protection Regulations (GDPR). Preserving the utility of video-based smart city applications while maintaining compliance with privacy protection regulations requires technical solutions for controlling how applications extract information from video data.

[0029] Smartphones and other computers can control access to video data, but they typically support only coarse-grained permissions, such as by allowing full access or denying all access to data generated by the camera. Previous work has explored applying the principle of least privilege to video using fine-grained access control mechanisms. These systems rely on computer technology to detect objects in real time or near real time. These systems then use the detected objects to transform the raw video data before sharing it with the application. One way to transform video data is blacklisting or default sharing, which edits objects that the video consumer is not allowed to view (e.g., placing a black box on a person's face). A different improvement is whitelisting or default blocking, which only shares objects that the video consumer is explicitly allowed to view (e.g., blacking out all other objects except for objects of authorized detected categories).

[0030] The whitelisting techniques in this paper provide a solid foundation for making smart city infrastructure compliant with privacy regulations. Previous work was designed for smartphones and other computers, where a trusted kernel can transform video data for a small number of local applications (usually just one). In contrast, surveillance camera data circulates through a distributed system consisting of a diverse and large collection of resource-limited edge devices and potentially remote video-consuming applications.

[0031] As described above, the embodiment provides a framework for privacy protection of surveillance camera videos. The three challenges faced by the embodiment may include edge device computing limitations, edge device bandwidth limitations, and a large and diverse set of video consumers for smart city applications. The default blocking method of the embodiment helps to protect privacy in the face of these challenges.

[0032] To limit the size of the trusted computing base (the base of entities, whether users or devices authorized to view videos), embodiments can edit videos from devices on or near the edge (e.g., servers), such as by using off-the-shelf object detection models. However, performing object detection in near real time on resource-limited edge devices typically means using weaker models. Weaker models are more likely to have false negatives (e.g., missing the current object) than false positives (e.g., misclassifying the current object), and these errors may lead to privacy violations. Therefore, embodiments block video content by default or block whitelist objects to reduce the likelihood of model errors that lead to privacy violations. Default blocking (whitelisting) also helps embodiments scale to a large and diverse collection of applications, as further discussed elsewhere herein.

[0033] Modern object detection models such as YOLO (detailed in the paper "You only look once: Unified, real-time object detection" by Joseph Redmon, Santosh Kumar Divvala, Ross B. Girshick, and Ali Farhadi (CoRR, abs / 1506.02640, 2015)) can detect dozens of object categories, such as people, cars, and traffic lights. Under default sharing (blacklisting), embodiments must create uniquely edited streams for each category combination. Creating all these streams requires a lot of computation and bandwidth on the edge nodes. Note that even if the number of permission combinations for active near-real-time applications is small, the diversity of authorized video consumers may grow over time. For example, at the time the camera captures the video, there may be no active applications that need to see bicycles. However, later, a city's road planning committee may want to use archived video to characterize bicycle traffic.

[0034] To help alleviate this problem, an embodiment can create a separate video stream for each object category. Each video stream can only include objects of its corresponding category (e.g., a bicycle stream only shows bicycles). This approach helps the embodiment to expand in at least three ways. First, the maximum number of streams that can be generated is equal to the number of detected object categories (not the number of category combinations) (plus one to account for background streams that include all objects or parts of the scene that are not part of the detected category). Second, because most streams are mostly (or completely) blacked out, individual streams can be well compressed with minimal effort. Third, by dividing the object stream into segments and encrypting the segments with unique keys, an embodiment can decouple the service encrypted video data from authorization decisions (e.g., distributing decryption keys). This allows video delivery to be handled by conventional scalable network technologies such as content distribution networks (CDNs) without expanding the trusted computing base or compromising privacy. Applications authorized to view multiple objects can simply retrieve the appropriate keys and encrypted video clips. Then, the application can overlay the decrypted video to compose a coherent multi-object stream.

[0035] Now, the technical solution of the technical problem provided by the embodiment will be further described in detail with reference to the accompanying drawings. The description continues with the analysis of video analysis, which provides an understanding of the Figure 4 An improved basis for a system for privacy-preserving object streaming is presented.

[0036] Figure 1 A diagram illustrating a comparison of whitelist and blacklist approaches to image data is provided by way of example. Figure 1, an original image 102 is provided in the upper left corner. Below is a table 104 comparing the false positives and false negatives for the blacklist approach and the whitelist approach for car and truck objects.

[0037] At the top left of the table is an image 106 of a false positive in the whitelist approach. The whitelist specifies objects to be displayed (not edited) in the image and all other objects to be edited. Thus, a false positive in the whitelist approach means that an object that is not on the whitelist is shown in the image. Figure 1 In the example, whitelist false positives include people that the object detector mistakenly identifies as cars (see Figure 4 ).

[0038] In the upper right corner of the table is an image 108 of a false positive in the blacklist approach. The blacklist specifies the object to be edited and all other objects to be displayed in the image. Therefore, a false positive in the blacklist approach means that the object not to be edited is not shown in the image. Figure 1 In the example above, a person is mistaken for a car and is edited accordingly.

[0039] At the bottom left of the table is an image 110 of a false negative in the whitelist approach. Since the whitelist specifies the object to be shown, and all other objects in the image are to be edited, a false negative in the whitelist approach means that the object to be shown is edited. Figure 1 In the example of , a whitelist false negative includes a car not being detected as a car and therefore being edited.

[0040] At the bottom right of the table is an image 112 of a false negative in the blacklist approach. Since the blacklist specifies the object to be edited and all other objects to be displayed in the image, a false negative in the blacklist approach means that the object to be edited is displayed. Figure 1 In the example of , the truck is not detected as a truck and is therefore displayed.

[0041] Consider the whitelist, where detecting a person and misclassifying her as a car constitutes a false positive (upper left corner of the table), resulting in a loss of privacy for that person. Consider further that not detecting a car in the scene constitutes a false negative, which results in a loss of utility for the vehicle counting application (lower left corner of the table). For the blacklist, detecting a car as a person constitutes a false positive, which results in a loss of utility for the vehicle counting application. Alternatively, not detecting a person in the scene constitutes a false negative for the blacklist approach, which results in a loss of privacy.

[0042] Therefore, the relative prevalence of false positives and false negatives determines whether a whitelist or blacklist will result in a greater or lesser loss of privacy. This is typically measured using precision and recall, which can be defined as follows. Precision measures the accuracy of a detector (e.g., the proportion of correct detections). Precision can be determined using the following equation: Precision = TP / (TP+FP), where TP represents true positives and FP represents false positives. Recall measures the proportion of basic fact objects that are correctly detected and labeled by the detector. Recall can be determined using the following equation: Recall = TP / (TP+FN), where FN represents false negatives. Improving precision involves reducing the number of false positives (FP). On the other hand, in order to improve recall, the number of false negatives (FN) needs to be reduced.

[0043] Figure 2 A graph of the precision and recall of three object detection techniques for three videos generated from training (bdd-training), validation (bdd-validation), and tracking (bdd-tracking) data from the Berkeley Driving Dataset (BDD) is illustrated: Faster-RCNN (recurrent convolutional neural network), YOLO ("You Only Look Once") version 3 (v3), and Tiny-YOLO v3. The three videos contain a total of 1,100 hours of labeled video data, including 100,000 video sequences recorded from multiple moving vehicles at 1 frame per second (FPS). The videos contain labeled objects corresponding to 10 categories. Faster-RCNN and YOLO v3 are relatively heavy full-size models, while Tiny-YOLO is a lightweight approximation of the full-size YOLO model. The three models were not retrained and used their default weights obtained by training on common objects in the context (COCO) dataset with 80 object categories.

[0044] While all three detectors produce very high precision (about 0.93-0.96), the measured recall is significantly lower (about 0.07-0.6). For all three videos, it is also observed that the most complex model, Faster-RCNN, produces the highest recall, followed by YOLO and Tiny-YOLO.

[0045] Figure 2 The figure shows that prior art object detectors tend to have much higher precision than recall. This means that for prior art object detection techniques, false negatives are more common than false positives. This means that object detectors tend to miss more objects in the scene than they misclassify detected objects. Figure 2As shown in Figure 2, this problem is more serious for low-cost detectors such as Tiny-YOLO, which has a significantly lower recall (about 3 times lower). Tiny-YOLO is often used in resource-scarce environments such as cameras and edge computing platforms.

[0046] Reducing false positives is a much easier problem to solve than reducing false negatives, as it involves correctly classifying detected objects. Figure 2 This is supported by the results in , which show that lightweight detectors such as Tiny-YOLOv3 have similar precision to computationally expensive detectors such as YOLO and Faster-RCNN. On the other hand, reducing false negatives is more difficult because it involves both detecting missed objects and classifying them. While precision and recall are useful metrics to capture the prevalence of detection errors, not all such errors lead to privacy or utility loss.

[0047] False positive FPs that mislabel sensitive classes S∈β as related classes R∈ω S→R This leads to a privacy loss of whitelisting, where β is the set of all sensitive classes and ω is the set of all related classes. The false positives of mislabeling related classes R∈ω as sensitive classes S∈β

[0048] Positive FP R→S This will lead to a loss in the utility of the blacklist. On the other hand, the false negative FN that does not detect the sensitive class S∈β S This will lead to the loss of privacy of the blacklist. Alternatively, the false negative FN that does not detect the relevant class R∈ω R This results in a utility loss for the whitelist. The privacy loss P can be defined as the fraction of objects belonging to the sensitive class S∈β that are disclosed to the consumer. The utility loss U can be defined as the fraction of objects belonging to the relevant class R∈ω that are hidden from the consumer.

[0049] Using these definitions, P and U can be determined for both whitelists (WL) and blacklists (BL). The privacy loss of WL is denoted as P WL , the privacy loss of BL is denoted as P BL , the utility loss of WL is marked as U WL , the utility loss of BL is marked as U BL .

[0050]

[0051] U BL =∑ S∈β，R∈ω FP R→S / ∑ R∈ω TP R +FN R

[0052] Based on the above, false positives are relatively rare because modern object detectors have high precision. When using a whitelist, there is only a small subset of false positives, which cause privacy loss by mislabeling sensitive classes S∈β as related classes R∈ω. False negatives are more common than false positives because modern object detectors have relatively low recall. Although recall can be improved by training on specialized data, it is much more difficult than improving precision. When using a blacklist, not detecting all false negatives of sensitive classes S∈β causes privacy loss.

[0053] Figure 3 A graph comparing the privacy loss between whitelists and blacklists for various object detection techniques is illustrated by way of example. This graph, together with the previous insights, shows that when modern object detection methods are used at ingestion, whitelists can produce significantly reduced privacy loss compared to blacklists. Figure 3 It is shown that the percentage of privacy loss incurred by whitelists is significantly lower than that of blacklists. However, whitelists may also suffer higher utility losses (e.g., about 38-87%) compared to blacklists (e.g., about 0.02-0.09%).

[0054] However, consider that privacy loss is permanent while utility loss can be recovered. Consider an example where an object belonging to a sensitive category S is accidentally made public or unredacted. In this case, the privacy loss caused is permanent and cannot be recovered. On the other hand, if the detection of low-confidence objects belonging to the relevant category R is concealed from the consumer, it can be recovered by post-processing the frame with a more accurate object detector. This can be achieved, for example, by providing metadata indicating low-confidence relevant objects to the consumer. The consumer can then request post-processing to recover the lost utility.

[0055] Therefore, by using a whitelist instead of a blacklist to favor privacy, it helps to create a privacy-preserving video stream. Although embodiments support blacklists, whitelists, or a combination thereof, embodiments tend to favor whitelists. In subsequent sections, the additional benefits of whitelists in managing and distributing privacy-preserving video streams to multiple consumers and encoding video streams will be discussed.

[0056] Figure 4 A block diagram of an embodiment of a system 400 for privacy protection of video streaming is illustrated by way of example. The system 400 may be compatible with existing applications and CDNs. The system 400 as illustrated includes a camera 408, a content delivery network (CDN) 403, a server 401, a management party 414, and a video consumer 416.

[0057] Camera 408 is a device for recording visual images in the form of photographs, films, or other video signals. In some embodiments, camera 408 can be mounted separately to a building, street corner, traffic light, or other location. In other embodiments, camera 408 can be part of a device with more functions, such as a smart phone. In still other embodiments, camera 408 can be part of an Internet of Things (IoT) device, etc.

[0058] The camera 408 provides a raw video frame 410 to the system 400. The raw video frame 410 includes data indicating a view of a scene. Typically, the raw video frame includes a two-dimensional matrix of pixel values ​​(e.g., grayscale, color, multispectral, etc.). The raw video frame 410 may include metadata indicating one or more of the following: the name of the video including the raw video frame 410, the format of the raw video frame (e.g., Audio Video Interleaved (AVI), Flash Video Format (FLV), Windows Media Video (WMV), Apple QuickTime Movie (MOV), Moving Pictures Expert Group 4 (MP4), etc.), the codec type, the file size, the duration of the associated video, the bit rate, the date the video was recorded, the device that generated the video, the location where the video was captured, etc.

[0059] As shown, the server 401 includes an object detector 404 and a privacy protector 406. The object detector 404 can ingest a video frame 410. Ingestion can include decoding, decompression, or a combination thereof of the original video frame 410. The object detector 404 detects objects in the original video frame 410. There are many types of object detectors and many are based on neural networks (NN) or machine learning (ML). Examples of ML object detection techniques include Viola-Jones detection based on Haar features, scale-invariant feature transforms (SIFT), histograms of oriented gradients (HOG) features, etc. Examples of NN object detection techniques include Faster-RCNN, You Only Look Once (YOLO) and Tiny-YOLO, Circular Convolutional NN (R-CNN), Fast R-CNN, Single-shot Multi-box Detector (SSD), etc. In some embodiments, the object detector 404 supports frame skipping (detection is performed only once every n frames). In some embodiments, the server 401 can be tightly integrated with the camera 408. Camera 408 may include an onboard computer and processing circuitry (e.g., an application specific integrated circuit or other processing circuitry) for performing hardware accelerated object detection and video encoding (or other operations performed by object detector 404, privacy protector 406, or other operations performed by server 401).

[0060] In some embodiments, the object detector 404 can perform object tracking after object detection. Examples of object tracking techniques include point tracking, kernel tracking, and contour tracking. In point tracking, tracking is performed based on the position and motion of the object. Tracking is performed by associating points across frames. Kernel tracking is similar to point tracking, but kernel tracking uses primitive geometric shapes to represent and track objects instead of using points. Contour tracking is sometimes referred to as region tracking or region-based tracking. In contour tracking, objects are modeled in the form of color histograms, object edges, or object contours. Examples of object tracking techniques include Simple Online Real-Time Tracking (SORT), Multi-Domain Network (MDNET), GOTURN, Cycle YOLO (ROLO), etc.

[0061] The object detector 404 may generate a frame identification, detected objects, object boundaries (e.g., in the form of a bounding box, outline, ellipse, etc.), object category, prediction confidence score, or a combination thereof at data 412. The data 412 may be generated per analyzed frame and sent to the privacy protector 406.

[0062] The privacy protector 406 is responsible for implementing the video streaming mechanism and generating a combinable (e.g., encrypted) object-specific video stream 418. These streams 418 are created based on the object detection data 412 received from the object detector 404. By accessing the appropriate set of encryption keys, the object stream 418 can be decrypted and combined to reconstruct the video (e.g., in an edited form) including the original video frame 410. The types of object categories encountered in the scene usually follow the Zipfian distribution. Instead of generating streams for all object categories that the detector can detect, the privacy protector 406 relies on the policy engine 402 to dynamically specify the object categories of interest to the consumer (or, the blacklist management party). All pixels that are not detected as belonging to any category specified by the policy engine 402 can be encoded into the background stream.

[0063] The privacy protector 406 can support two or more types of outputs for each object stream: (i) original frames used locally by applications deployed on the edge, and (ii) encoded (e.g., H264-encoded) HTTP Live Streams (HLS) that can be used by external consumers for distribution to a content delivery network (CDN) 403.

[0064] HLS encodes videos into files with fixed configurable durations and uses the HTTP protocol to transfer these encoded files. The file-based nature of HLS makes it particularly useful from a content distribution perspective. Fast Forward MPEG (ffmpeg) can be used to generate H264-encoded HLS. All output object streams can be encrypted using AES-128 or other encryption techniques. The encryption keys generated by the policy engine 402 (e.g., a 128-bit stream for each object) can be provided to the server 401. To provide HLS files, nginx can be used in conjunction with the nginx-rtmp module, which provides HLS support.

[0065] As shown, CDN 403 includes a policy engine 402. Policy engine 402 exposes one or more of a consumer-oriented or manager-oriented application programming interface (API) (referred to as a consumer API and a manager API, respectively). The API may be a representational state transfer (REST) ​​API. An API is a collection of rules that allow programs to communicate with each other. Developers create APIs on a server and allow clients to talk to applications through the server. REST APIs typically use hypertext transfer protocol (HTTP) requests to get, put, publish, and delete data. The rule of a REST API is to provide a link to a uniform resource locator (URL) for a resource (data).

[0066] The Admin API allows an admin to create or remove consumer profiles. The Admin API allows an admin to specify object categories that a consumer is allowed to view (whitelisted) or is not allowed to view (blacklisted). The system 400 can support multiple consumer profiles. Three examples of consumer profiles include: (i) Whitelisted: only a specified set of object categories are exposed to the consumer, (ii) Blacklisted: only a specified set of object categories are editable from the consumer, and (iii) Unedited: viewers are able to view the stream in its entirety.

[0067] The Consumer API allows consumers to request to view videos. Consumers can specify a list of object categories. If any requested category is: (i) not in the whitelist specified through the Admin API, or (ii) in the blacklist specified through the Admin API, the policy engine can deny the request.

[0068] The policy engine 402 may be responsible for generating, revoking, and distributing cryptographic keys used to protect the object stream 418. The policy engine 402 may expose another API for consumers to download keys and implement authentication to verify whether the consumer has the right to access a particular object stream.

[0069] Along with streaming per-object video streams 418, the system 400 can provide encrypted metadata corresponding to each object stream. The metadata can be encrypted (using the same key as used to encrypt the corresponding stream). The metadata can include a list of detected objects found in frames corresponding to object categories, object boundaries, and confidence scores. This is useful because (i) providing a metadata stream for real-time analysis may be sufficient to enable many applications, and (ii) low-confidence object detections can be concealed from users to prevent privacy loss. Applications can then use the metadata to detect the presence of such objects. Thus, the application can restore utility by requesting that the frame be reanalyzed by a specified object detector. Metadata can be provided using a publish-subscribe channel (e.g., a NATS stream publish-subscribe channel), and a separate channel can be provided for object streams (e.g., each object stream).

[0070] In operation, camera 408 may operate to generate raw video frames 410. Raw video frames 410 may be provided to server 401. Object detector 404 may identify objects, object categories, object boundaries, and corresponding confidence that the object is part of the category. The identified objects, object categories, object boundaries, or confidences may be provided as data 412 to privacy protector 406.

[0071] The privacy protector 406 may encode, encrypt, or a combination thereof, the video stream 418 that includes only the specified class of objects. The rest of the video stream 418 (including the background) may be redacted.

[0072] A video consumer 416 (application, device, etc.) may issue a request 422 for an object-specific video stream 418 (sometimes referred to as an "object stream"). The CDN 403 may receive the request 422. The policy engine 402 may verify that the video consumer 416 is authorized to view the object stream 418. This verification may include looking up a unique identifier of the video consumer 416 and identifying that the video consumer 416 is associated with an object class corresponding to the object stream 418, requesting and verifying a password from the video consumer 416, and the like.

[0073] If the video consumer is authorized to view the object stream 418, the policy engine 402 can issue a request 424 to the server to generate or provide the object stream 418. The server 401 can enforce it. Alternatively, if the object stream 418 has been created and provided to the CDN 403, the CDN 403 can provide the stream 418 to the consumer 416.

[0074] Stream manager 414 is responsible for managing access permissions to object stream 418. Stream manager 414 may provide data 420 to policy engine 402 indicating a unique identifier of video consumer 416 and corresponding whitelist / blacklist(s) of object categories that video consumer 416 is or is not authorized to view. Policy engine 402 is responsible for enforcing the access permissions indicated by data 420.

[0075] The motivation for system 400 is now provided, focusing on the trade-offs of using a whitelist approach to video editing. It is assumed that system 400 performs object detection using onboard computing of a surveillance camera or a directly connected edge server. In either case, system 400 has a trusted computing base (TCB) that extends to all software and hardware that can access plain text video. The size of the TCB can be reduced by performing object detection and video encoding in a secure execution environment, such as Advanced Reduced Instruction Set Computer (ARM) TrustZone or Intel Software Guard Extensions (SGX). Regardless, since edge platforms are typically more resource-constrained than high-end server machines, some lightweight techniques are explored to detect and expose / edit objects in video streams. Subsequently, it is described how system 400 can deliver video to multiple applications simultaneously while protecting privacy and reducing the bandwidth required at edge sites. Finally, it is described how whitelisting simplifies application authorization.

[0076] Most object detectors, such as those implemented by object detector 404, are computationally intensive and require powerful hardware accelerators such as graphics processing units (GPUs) to support the inference frame rates required for real-time streaming and analysis. Such powerful accelerators may not typically be available on resource-constrained edge platforms.

[0077] Given that an object typically remains in the field of view of camera 408 for at least several frames, inference may be performed every n frames. Object tracking may be performed between detected objects between two consecutive inference steps (every n frames). In particular, the following is an analysis of the reduction in inference time and impact on privacy loss for various object detection techniques.

[0078] Figure 5The inference frame rates of the following are compared: (i) state-of-the-art object detectors, such as Faster-RCNN and YOLO, (ii) low-cost object detectors, such as Tiny-YOLO, and (iii) performing object detection every n frames using Faster-RCNN or YOLO followed by object tracking (e.g., using the SORT framework and an open computer vision (CV) based tracker), n = 5 and n = 10. Four videos are used, each with a high-definition (HD) resolution of 1280×720. On average, RCNN produces the lowest frame rate (about 2.7FPS), while tiny-yolo produces the fastest frame rate (about 42FPS). In addition, YOLO-skip-10 can provide similar or faster frame rates than TinyYOLO (about 36-46FPS). Note that while RCNN, YOLO, and TinyYOLO produce frame rates that are independent of the video content, the frame rates of RCNN-skip-n and YOLO-skip-n depend on the video since the tracking speed depends on the number of objects in the frame.

[0079] Figure 6 By way of example, a graph of precision and recall for YOLO-skip-5, YOLO-skip-10, and tiny YOLO is shown. Figure 6 , using labels generated by YOLO v3 as ground truth. This is suitable for this comparison because all object detection techniques compared are based on YOLO v3. Figure 6 In the graph of , YOLO-skip-n with n=5 and n=10 produces higher precision and significantly higher recall compared to TinyYOLO. This makes it a viable alternative for resource-constrained edge platforms. This is because it can provide higher precision and recall than TinyYOLO, which means lower privacy and utility loss, while providing similar or higher frame rates.

[0080] It may be desirable to remove sensitive information from the video as close to the source as possible. Therefore, it may be desirable to perform whitelisting or blacklisting at the edge. Bandwidth is an important constraint when streaming from edge devices. The edge device providing the video stream may be required to stream the video to human viewers or applications located in different locations. In order to distribute video to multiple consumers at scale, the following distribution models can be used:

[0081] 1. Video is streamed in real time from a server on or near the edge to a content delivery network (CDN) 403. CDN 403 is a geographically distributed network of proxy servers and their corresponding data centers. One goal of CDN 403 is to provide high availability and high performance by distributing services spatially relative to end users.

[0082] 2. Consumers 416 connect to CDN 403 to stream the video. If privacy-preserving transformations such as whitelisting or blacklisting are not performed, the video stream can be easily distributed to multiple consumers through CDN 403. Alternatively, if all consumers 416 are homogeneous in terms of their viewing permissions and targets, a single whitelisted or blacklisted stream can also be distributed through CDN 403. However, a video stream will typically have multiple consumers 416 with different targets. Consider a smart city scenario where multiple consumer applications use the same camera feed. Table 1 shows five smart city applications, each responsible for counting different types of objects in the video stream, and object categories that need to be made public or edited depending on whether whitelisting or blacklisting is used. In this case, a simple approach is to create consumer-specific video streams to meet the needs of each consumer while protecting privacy.

[0083] application Whitelisted objects Blacklisted objects Vehicle Counting vehicle people Pedestrian Counting people vehicle Bike Count bike Vehicles, people Two-wheeler count Bicycles, motorcycles Vehicles, people monitor All objects No object

[0084] Table I: Example multi-consumer smart city scenarios

[0085] Even the small scenario presented in Table I will cause the bandwidth requirements at the edge to increase rapidly as multiple consumers are added.

[0086] Figure 7 Snapshots of five edited video streams created for each application satisfying Table I using the whitelist approach are illustrated by way of example. Figure 8 Snapshots of five edited video streams created for each application satisfying Table I using the blacklist approach are illustrated by way of example. Figure 8 Each snapshot of the blacklist approach contains a common background, which is streamed five times from the edge in the example of Table I. In an unfavorable setting, when no object is detected belonging to any of the blacklisted categories, each consumer application gets the exact same frame. This will use up to 5 times the bandwidth of a single unedited video stream.

[0087] In contrast, if a whitelist approach is used, such as to protect privacy, some object categories may be in two or more streams. Figure 7 As shown, the bicycle and two-wheeler counting application and the monitoring application use objects belonging to the "bicycle" category. Therefore, these objects are transmitted three times from the edge. However, this is still more efficient than blacklisting because the background can be blacked out and not disclosed to the viewing party.

[0088] Video frames are usually encoded and transmitted in a compressed format, such as the Joint Photographic Experts Group (JPEG) format or the H264 format. These encoding formats encode information in the frequency domain. Therefore, the blackened areas do not contain any information or contribute to the streaming bandwidth.

[0089] Set the pixels of frame t It is regarded as the smallest information unit in the video stream with a resolution of (W, H), where 0<i<W, 0<j<H, t, W, H∈Z + By using an object detector, if a pixel is within the range of an object (e.g., an ellipse, a bounding box, a contour, etc.), it can be classified as: (i) belonging to the object class, otherwise (ii) belonging to the background. Therefore, in order to distribute video streams in a privacy-preserving and bandwidth-efficient manner, the method can be performed based on the following assumption: each pixel in frame t of the video stream contains useful information. Therefore, in order to transmit all useful information while minimizing bandwidth, each pixel in the frame can be encoded only once; and each pixel in frame t It can only be accessed by consumers who are authorized to view it (for example, for consumer C k , if it belongs to a whitelisted class R∈w k Or does not belong to the blacklisted category S∈β k ).

[0090] Fig. 9 A flow chart of an embodiment of a method 900 for protecting privacy in a video stream while still allowing access to authorized viewers of the video stream is illustrated by way of example. As illustrated, the method 900 includes: at operation 902, each pixel in frame t is Assign to the set Γ c , where c∈{Ω, background}, and Ω represents all object categories that can be detected by the object detector 404; at operation 904, a unique key k is used c Encrypt each pixel set Γ c ; and at operation 906, all encrypted pixels in each pixel set Encoded as a corresponding single frame.

[0091] Using the method 900, each encrypted video stream can be sent to multiple consumers, such as through a CDN. c Each set of c The encryption is performed so that only a consumer with the correct key can decrypt (and correctly decode) the corresponding pixels to recover the original video stream. Using method 900, viewers can only see what they are authorized to see.

[0092] Method 900 effectively converts the video stream distribution problem into one of key management. Combining this with knowledge of each consumer's whitelist categories, each consumer can be authenticated and provided with the set of keys required to view their relevant set of pixels. Method 900 (i) preserves privacy because each viewer can only see objects belonging to their whitelist categories, (ii) conserves bandwidth because each pixel is transmitted fewer times from the edge, and (iii) reduces computational cost because each pixel is only encrypted and encoded based on the number of categories that the pixel is mapped to.

[0093] Note how method 900 is agnostic to blacklists or whitelists. For all classes not on the blacklist Method 900 can be used to provide a k Provide all keys k c to generate the blacklisted flow.

[0094] Unlike method 900, typical video streaming techniques first perform encoding and then encrypt the encoded data. This is at least in part because performing encryption on the original frames destroys redundant information in the video frames. Redundant information allows for more efficient compression and encoding of the video. Therefore, embodiments may perform encoding followed by encryption.

[0095] Embodiments may generate N+1 encrypted video streams, where N represents the number of object categories in Ω. A stream may be generated for each object category c, thereby creating N video streams, and an additional stream may be generated for the background (any pixels determined not to be part of a category). Each object stream may include all pixels corresponding to objects detected as belonging to a particular category c. Each object stream can use an object class specific key k c N can be changed dynamically based on the object class to be detected, such as based on application white / black list requirements. Embodiments can also create a residual background stream that contains all pixels that are not assigned to any object class. This background stream can be encrypted with a unique key k background Encryption is performed.

[0096] Fig.10A conceptual block diagram of an embodiment of a system 1000 for privacy-preserving video streaming is illustrated by way of example. The system 1000 can be used to provide video streaming with privacy protection, such as in a multi-purpose smart city application. Each pixel of each raw video frame from one of the video capture devices 1110A, 1110B, 1110C is assigned to one or more of the N+1 created object streams. The assignment can be based on whether the object detector 404 determines that the pixel corresponds to a detected category. If the detector 404 determines that the pixel corresponds to a certain category of object, then the pixel can be associated with the category. As described in detail in relation to Figure 8 As discussed, an object can be a member of multiple categories, and thus a pixel can be assigned to more than one category. All pixels in the video stream that do not correspond to pixels associated with the category can be blacked out (e.g., set to a value of 0, etc.). Any pixel of the video frame that is not assigned to a video stream 1112A, 1112B, 1112C corresponding to an object can be assigned to a video stream 1112D corresponding to the background.

[0097] The video capture devices 1110A-1110C may include a smartphone, a dedicated camera, etc. The video capture devices 11110A-1110C may include cameras 408A, 408B, 408C, respectively, and communication circuit systems 1114A, 1114B, and 1114C, respectively. The communication circuit systems 1114A-1114C may include a transmit radio, a receive radio, a transceiver radio, an Ethernet port or another communication port, an antenna, a network interface card (NIC), a modulator, a demodulator, an amplifier, etc. The privacy protector 406 may encode and encrypt the video stream data corresponding to the video streams 1112A-1112D.

[0098] Embodiments that encode and then encrypt still meet the key expected properties provided by the ideal scheme, where encryption occurs before encoding. For example, each object stream can be encrypted using a different key. Therefore, although each user can consume all object streams through the CDN, they can only decrypt the streams they have the right to view (white list). In addition, each pixel in the frame is only assigned to a limited number of object streams. Therefore, for each object stream to which they are assigned, each pixel can be encoded, encrypted and transmitted to the CDN once. Embodiments can therefore provide privacy-preserving video streams in a computationally and bandwidth-efficient manner. Although embodiments have bandwidth efficiency at the edge, they can also save downstream bandwidth at the video consumer. This is at least in part because most consumers can now only consume related object streams instead of entire video streams.

[0099] Embodiments may generate new encryption keys at periodic intervals, such as to help ensure improved security while allowing consumers to see that permissions have changed over time. Another benefit among the benefits may include that, despite having all encryption keys k c , but each consumer can still stream and store the complete encrypted video stream locally. This opens up the possibility that consumers will get permission from the management to process previously inaccessible object streams in the future. Surveillance video is a good example. Security personnel are usually allowed to view all aspects of the video frame (for example, except for the face). However, if a crime is committed, access to the encryption key k required to view the face for a specific period of time can be requested. faces The process is similar to a legal search warrant, where a judge needs to grant permission to conduct an investigative search.

[0100] Fig.11 A diagram of an embodiment of a method 1100 for streaming video data in a privacy-preserving manner is illustrated by way of example. The method 1100 as illustrated includes identifying which categories of objects are present in the video data at operation 1102; generating, for each of the categories identified in the video data, a corresponding video stream including objects of the category and excluding objects not belonging to the category at operation 1104; and providing each of the corresponding video streams to a content distribution network at operation 1106.

[0101] The method 1100 may include encrypting each video stream in the corresponding video stream using a different corresponding encryption key before providing each video stream in the corresponding video stream to the content distribution network. The method 1100 may include, wherein the corresponding video stream also includes pixel data determined to correspond to the background from which editing is performed. The method 1100 may also include compressing each video stream in the corresponding video stream before providing each video stream in the corresponding video stream to the content distribution network. The method 1100 may also include receiving a request for a video stream of the category. The method 1100 may also include generating a video stream that includes only objects of the category.

[0102] Fig.12 A block diagram of an embodiment of a machine 1200 (e.g., a computer system) for implementing one or more embodiments is illustrated by way of example. The machine 1200 includes components that may be part of a server 401, a policy engine 402, an edge device (e.g., video capture devices 1110A-1110C), or another device. The machine 1200 may be used to implement Fig. 9 , Fig.111200, or other methods discussed herein. An example machine 1200 (in the form of a computer) may include a processing unit 1202, a memory 1203, a removable storage device 1210, and a non-removable storage device 1212. Although an example computing device is illustrated and described as a machine 1200, the computing device may have different forms in different embodiments. For example, the computing device may alternatively be a smartphone, a tablet computer, a smart watch, or include a computer program product related to the present invention. Fig.12 Other computing devices that may include the same or similar elements as illustrated and described. Devices such as smartphones, tablet computers, smart watches, etc. are generally referred to as mobile devices. In addition, although various data storage elements are illustrated as part of the machine 1200, the storage device may additionally or alternatively include a cloud-based storage device accessible via a network such as the Internet.

[0103] The memory 1203 may include volatile memory 1214 and nonvolatile memory 1208. The machine 1200 may include, or have access to, a computing environment that includes, various computer-readable media, such as the volatile memory 1214 and nonvolatile memory 1208, removable storage 1210, and non-removable storage 1212. Computer storage includes random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) and electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices capable of storing computer-readable instructions for performing the functions described herein.

[0104] The machine 1200 may include or may access a computing environment including input 1206, output 1204, and communication connection 1216. The output 1204 may include a display device that may also be used as an input device, such as a touch screen. The input 1206 may include one or more of a touch screen, a touch pad, a mouse, a keyboard, a camera, one or more device-specific buttons, one or more sensors integrated into the machine 1200 or coupled to the machine 1200 via a wired or wireless data connection, and other input devices. The computer may operate in a network environment using a communication connection to connect to one or more remote computers, such as a database server, including cloud-based servers and storage devices. The remote computer may include a personal computer (PC), a server, a router, a network PC, a peer device or other public network node, etc. The communication connection may include a local area network (LAN), a wide area network (WAN), a cellular, an Institute of Electrical and Electronics Engineers (IEEE) 802.11 (Wi-Fi), Bluetooth, or other network.

[0105] Computer readable instructions stored on a computer readable storage device are executable by a processing unit 1202 of the machine 1200. Hard drives, CD-ROMs, and RAM are some examples of items that include non-transitory computer readable media (such as storage devices). For example, the computer program 1218 can be used to cause the processing unit 1202 to perform one or more methods or algorithms described herein.

[0106] Additional notes and examples:

[0107] Example 1 includes a device comprising: an object detector circuit system for identifying which categories of objects are present in video data; a processing circuit system for generating, for each of the categories identified in the video data, a corresponding video stream that includes objects of the category and excludes objects that do not belong to the category; and wherein the processing circuit system provides each of the corresponding video streams to a content distribution network.

[0108] In Example 2, Example 1 may further include, wherein the object detector circuitry is further configured to identify, for each of the categories, pixels of each frame of the video data that correspond to an object of the category.

[0109] In Example 3, Example 2 may also include, wherein the corresponding video stream includes pixel data within a bounding box, ellipse, or outline of the category of objects from the video data, and the pixel data corresponding to pixels outside the edited bounding box, ellipse, or outline.

[0110] In Example 4, at least one of Examples 1-3 may also include, wherein the object detector circuit system is further configured to provide metadata for each frame of the video data, the metadata indicating, for each identified object, an identified category and a corresponding confidence value for the category.

[0111] In Example 5, at least one of Examples 1-4 may further include encryption circuitry to encrypt each of the corresponding video streams using a different corresponding encryption key before providing each of the corresponding video streams to the content distribution network.

[0112] In Example 6, at least one of Examples 1-5 may further include, wherein the corresponding video stream further includes pixel data determined to correspond to a background from which editing is performed.

[0113] In Example 7, at least one of Examples 1-6 may further include compression circuitry to compress each of the corresponding video streams before providing each of the corresponding video streams to the content distribution network.

[0114] In Example 8, at least one of Examples 1-7 may also include a communication circuit system, which is used to receive a request for the video stream of the category and generate a request to the object detector circuit system to identify objects of the category in the video data; wherein the object detector circuit system is also configured to generate the video stream of the category in response to receiving the request.

[0115] In Example 9, Example 8 may further include, wherein the request for the video stream is from a policy engine separate from the content distribution network.

[0116] In Example 10, Example 9 may further include, wherein the policy engine verifies that a user is authorized to view the video stream of the category before transmitting the request.

[0117] In Example 11, at least one of Examples 1-10 may further include, wherein the object detector circuit system is configured to implement a convolutional neural network model to identify which categories of objects are present in the video data.

[0118] Example 12 includes a non-transitory machine-readable medium, which includes instructions that, when executed by a server, cause the server to perform operations, the operations including: identifying which categories of objects are present in video data; for each of the categories identified in the video data, generating a corresponding video stream that includes objects of the category and excludes objects that do not belong to the category; and providing each of the corresponding video streams to a content distribution network.

[0119] In Example 13, Example 12 may further include, wherein the operations further include: for each of the categories, identifying pixels of each frame of the video data corresponding to an object of the category.

[0120] In Example 14, Example 13 may also include, wherein the corresponding video stream includes pixel data within a bounding box, ellipse, or outline of the category of objects from the video data, and the pixel data corresponding to pixels outside the edited bounding box, ellipse, or outline.

[0121] In Example 15, at least one of Examples 12-14 may further include, wherein the operations further include providing metadata for each frame of the video data, the metadata indicating, for each identified object, the identified category and a corresponding confidence value for the category.

[0122] Example 16 includes a method for providing privacy protection for a video data stream, the method comprising an operation according to one of Examples 12-15.

[0123] Example 17 includes a method for providing privacy protection for a video data stream, the method comprising identifying which categories of objects are present in the video data, for each of the categories identified in the video data, generating a corresponding video stream that includes objects of the category and excludes objects that do not belong to the category, and providing each of the corresponding video streams to a content distribution network.

[0124] In Example 18, Example 17 may further include encrypting each of the corresponding video streams using a different corresponding encryption key before providing each of the corresponding video streams to the content distribution network.

[0125] In Example 19, at least one of Examples 17-18 may further include, wherein the corresponding video stream further includes pixel data determined to correspond to a background from which editing is performed.

[0126] In Example 20, at least one of Examples 17-19 may further include compressing each of the corresponding video streams before providing each of the corresponding video streams to the content distribution network.

[0127] In Example 21, at least one of Examples 17-20 may further include receiving a request for the video stream of the category and generating the video stream including only objects of the category.

[0128] Although several embodiments have been described in detail above, other modifications are possible. For example, the logic flows depicted in the figures do not require the order shown or sequential order to obtain the desired results. Other steps can be provided or steps can be deleted from the described flows, and other components can be added to or deleted from the described systems. Other embodiments may be within the scope of the following claims.

Claims

1. A device comprising: object detector circuitry to identify what classes of objects are present in the video data; processing circuitry to generate, for each of the categories identified in the video data, a respective video stream including objects of the category and excluding objects not belonging to the category, wherein the category is whitelisted for display of the objects of the category, and the processing circuitry to provide each of the respective video streams to a content delivery network; as well as Encryption circuitry is configured to encrypt each of the corresponding video streams using a different corresponding encryption key before providing each of the corresponding video streams to the content distribution network. 2 . The apparatus of claim 1 , wherein the object detector circuitry is further configured to, for each of the categories, identify pixels of each frame of the video data that correspond to objects of the category.

3. The apparatus of claim 1 , wherein the object detector circuitry is further configured to provide metadata for each frame of the video data, the metadata indicating, for each identified object, the identified category and a corresponding confidence value for the category. 4 . The apparatus of claim 1 , wherein the corresponding video stream further includes pixel data determined to correspond to a background from which editing is to be performed.

5. The apparatus of claim 1, further comprising compression circuitry to compress each of the corresponding video streams prior to providing each of the corresponding video streams to the content distribution network.

6. The device according to claim 1, further comprising: communications circuitry to receive a request for the video stream of the category and to generate a request to the object detector circuitry to identify objects of the category in the video data; and The object detector circuitry is further configured to generate the video stream of the category in response to receiving the request.

7. The apparatus of claim 1 , wherein the object detector circuitry is configured to implement a convolutional neural network model for identifying which categories of objects are present in the video data.

8. The apparatus of claim 2, wherein the corresponding video stream includes pixel data within a bounding box, ellipse, or outline of the category of objects from the video data, and the pixel data corresponding to pixels outside the edited bounding box, ellipse, or outline.

9. The apparatus of claim 6, wherein the request for the video stream comes from a policy engine separate from the content distribution network.

10. The apparatus of claim 9, wherein the policy engine verifies that a user is authorized to view the video stream of the category before transmitting the request.

11. A non-transitory machine-readable medium comprising instructions that, when executed by a server, cause the server to perform operations comprising: Identify which categories of objects exist in the video data; generating, for each of the categories identified in the video data, a corresponding video stream including objects of the category and excluding objects not belonging to the category, wherein the category is whitelisted for display of the objects of the category; encrypting each of the corresponding video streams using a different corresponding encryption key before providing each of the corresponding video streams to a content distribution network; as well as Each of the corresponding video streams is provided to the content distribution network.

12. The non-transitory machine-readable medium of claim 11, wherein the operations further comprise: For each of the categories, pixels of each frame of the video data corresponding to objects of the category are identified.

13. The non-transitory machine-readable medium of claim 11, wherein the operations further comprise: Metadata is provided for each frame of the video data, the metadata indicating, for each identified object, the identified class and a corresponding confidence value for the class.

14. The non-transitory machine-readable medium of claim 12, wherein the corresponding video stream includes pixel data within a bounding box, ellipse, or outline of the category of objects from the video data, and the pixel data corresponding to pixels outside the edited bounding box, ellipse, or outline.

15. A method for providing privacy protection for a video data stream, the method comprising: Identify which categories of objects exist in the video data; generating, for each of the categories identified in the video data, a corresponding video stream including objects of the category and excluding objects not belonging to the category, wherein the category is whitelisted for display of the objects of the category; encrypting each of the corresponding video streams using a different corresponding encryption key before providing each of the corresponding video streams to a content distribution network; as well as Each of the corresponding video streams is provided to the content distribution network.

16. The method of claim 15, wherein the corresponding video stream further includes pixel data determined to correspond to a background from which editing is performed.

17. The method according to claim 15, further comprising: Each of the corresponding video streams is compressed before each of the corresponding video streams is provided to the content distribution network.

18. The method according to claim 15, further comprising: receiving a request for the video stream of the category; as well as A video stream is generated that includes only objects of the class.

Images