Methods and systems to edit a media content stream to preserve privacy

Abstract

Embodiments include methods, electronic device, storage medium, and computer program for editing a media content stream. In one embodiment, a method comprises: detecting (502) one or more objects within a region of interest (ROI) of a scene to be edited in a media stream; labeling (504) the one or more objects detected within the ROI with one or more object categories of the ROI; providing (506) the object categories of the ROI to an application through a secure channel, the application to obtain generic media content corresponding to the object categories, wherein the generic media content represents the object categories but with no information identifiable with the media stream; receiving (508) the generic media content corresponding to the object categories from the application; and replacing (510) media content corresponding to the one or more objects within the ROI by the generic media content received from the application.

Description

METHODS AND SYSTEMS TO EDIT A MEDIA CONTENT STREAM TO PRESERVEPRIVACYTECHNICAL FIELD

[0001] Embodiments of the invention relate to the field of networking; and more specifically, to editing a media content stream to preserve privacy.BACKGROUND

[0002] With readily available cameras of a variety of types and explosion of augmented reality (AR) and virtual reality (VR) applications, the sheer volume of media content is staggering. The media content can be represented in two-dimensional (2D) format and / or three-dimensional (3D) format. 2D content includes images and graphics, while 3D content includes red green blue (RGB) and depth images, point clouds, meshes, or similar formats. Depth images may be used for many different applications including 3D object reconstruction and immersive video conferencing such as holographic communication. Depth images may be computer generated or captured, e.g., by a depth image camera, either as a stand-alone camera or a camera producing texture + depth output, e.g., red green blue depth (RGBD). Depth images can be for example acquired by Light Detection and Ranging (Lidar) sensors (e.g., Microsoft Kinect, Intel real- sense), true-depth sensors (e.g., iPhones), or other devices.

[0003] With such a large volume of media content, there is an urgent need to protect user privacy in processing media content. In fact, privacy laws and regulations are in place to protect user privacy in various jurisdictions (e.g., European Union or the United States) based on data types (e.g., health data or financial data), user characteristics (e.g., minors or adults), data usage (commercial purpose or private collection), and other factors. For example, Freedom of Information Act (FOIA) laws require government agencies to release video upon request while it must maintain certain degrees of privacy. Video redaction includes the obfuscation or removal of personal information in videos for privacy protection.

[0004] Yet the currently available privacy-preserving solutions often only provide a method for redacting / obfuscating the privacy-sensitive content but not for compensating for the redacted portion by some non-privacy sensitive objects. When they do fill / compensate for privacysensitive objects, they put the onus of privacy-preserving processes on camera system entity.

[0005] Additionally, some solutions for preserving privacy of media content rely on decrypting / decoding the image blocks in video stream frame as per the protection level or policy about a user / customer. These solutions don't specify how the encrypted content would be shown to users and how it would affect the quality of experience for the users.SUMMARY OF THE INVENTION

[0006] Embodiments include methods, electronic device, storage medium, and computer program for editing a media content stream. In one embodiment, a method comprises: detecting (502) one or more objects within a region of interest (ROI) of a scene to be edited in a media stream; labeling (504) the one or more objects detected within the ROI with one or more object categories of the ROI; providing (506) the one or more object categories of the ROI to an application through a secure channel, the application to obtain generic media content corresponding to the one or more object categories, wherein the generic media content exemplifies the one or more object categories but with no information identifiable with the media stream; receiving (508) the generic media content corresponding to the object categories from the application; and replacing (510) media content corresponding to the one or more objects within the ROI by the generic media content received from the application.

[0007] Embodiments include electronic devices for editing a media content stream. In one embodiment, an electronic device is disclosed to comprise a processor and non-transitory machine-readable storage medium that provides instructions that, when executed by the processor, are capable of causing the processor to perform: detecting (502) one or more objects within a region of interest (ROI) of a scene to be edited in a media stream; labeling (504) the one or more objects detected within the ROI with one or more object categories of the ROI; providing (506) the one or more object categories of the ROI to an application through a secure channel, the application to obtain generic media content corresponding to the one or more object categories, wherein the generic media content exemplifies the one or more object categories but with no information identifiable with the media stream; receiving (508) the generic media content corresponding to the object categories from the application; and replacing (510) media content corresponding to the one or more objects within the ROI by the generic media content received from the application.

[0008] Embodiments include machine-readable storage media for editing a media content stream. In one embodiment, a machine-readable storage medium is disclosed, and it provides instructions that, when executed by a processor, are capable of causing the processor to perform: detecting (502) one or more objects within a region of interest (ROI) of a scene to be edited in a media stream; labeling (504) the one or more objects detected within the ROI with one or more object categories of the ROI; providing (506) the one or more object categories of the ROI to an application through a secure channel, the application to obtain generic media content corresponding to the one or more object categories, wherein the generic media content exemplifies the one or more object categories but with no information identifiable with the media stream; receiving (508) the generic media content corresponding to the object categoriesfrom the application; and replacing (510) media content corresponding to the one or more objects within the ROI by the generic media content received from the application.

[0009] By implementing embodiments as described, a media stream may be edited efficiently to replace media content that contains information that is identifiable with the media stream so that the media stream includes the replacement media content.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The invention may best be understood by referring to the following description and accompanying drawings that are used to illustrate embodiments of the invention. In the drawings:

[0011] Figure 1 illustrates operations of editing a media content stream to preserve privacy per some embodiments.

[0012] Figure 2 illustrates operations of editing a media content stream to preserve privacy per additional embodiments.

[0013] Figures 3A and3B illustrate a hierarchical structure of object categories and examples of the hierarchical structure per some embodiments.

[0014] Figure 4 illustrates data transmission in a secure channel per some embodiments.

[0015] Figure 5 is a flow diagram illustrating operations of editing a media content stream to preserve privacy per some embodiments.

[0016] Figure 6 illustrates an electronic device to edit a media content stream to preserve privacy per some embodiments.

[0017] Figure 7 illustrates an example of a communication system per some embodiments.

[0018] Figure 8 illustrates a user equipment (UE) per some embodiments.

[0019] Figure 9 illustrates a network node per some embodiments.

[0020] Figure 10 is a block diagram of a host, which may be an embodiment of the host of Figure 7, per various aspects described herein.

[0021] Figure 11 is a block diagram illustrating a virtualization environment in which functions implemented by some embodiments may be virtualized.

[0022] Figure 12 illustrates a communication diagram of a host communicating via a network node with a user equipment (UE) over a partially wireless connection per some embodiments.DETAILED DESCRIPTION

[0023] Generally, all terms used herein are to be interpreted according to their ordinary meaning in the relevant technical field, unless a different meaning is clearly given and / or is implied from the context in which it is used. All references to a / an / the element, apparatus,component, means, step, etc. are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step and / or where it is implicit that a step must follow or precede another step. Any feature of any of the embodiments disclosed herein may be applied to any other embodiment, wherever appropriate. Likewise, any advantage of any of the embodiments may apply to any other embodiments, and vice versa. Other objectives, features, and advantages of the enclosed embodiments will be apparent from the following description.

[0024] Privacy -preserving solutions in media streaming should respect the integrity and overall aesthetic of a user environment. Additionally, they should not be limited to the limited local resources but leverage remote resources such as cloud computing that are scalable and efficient, where machine learning (ML) and / or artificial intelligence (Al) techniques may be deployed. A media stream refers to a continuous flow of audio, video, and / or other types of multimedia content. The media stream may include two-dimensional (2D) and / or three- dimensional (3D) video content with the corresponding audio and metadata. The embodiments disclosed herein can be applied for use-cases such as healthcare applications between therapist and patients, remote learning or work, live entertainment or any other use-cases for holographic communications.

[0025] These embodiments provide approaches for editing a media content stream with privacy-preserving content replacement. The content replacement swaps out content that reveals captured features in the media content stream that are considered to be private, by either the generator of the media content stream and / or the recipient, based on preference or applicable laws / regulations. By integrating the privacy-preserving content with the rest of the media content stream, the updated media content stream after the editing preserves privacy while maintaining user experience without any conspicuous, revelatory, or abruptly added poor-quality regions in the media content stream.Operations of Content Replacement

[0026] Figure 1 illustrates operations of editing a media content stream to preserve privacy per some embodiments. System 100 includes an editing preference module 102, a media processing module 104, and a repository of generic media content 106. An electronic device 150 (e.g., electronic device 602 or UE 800) may include editing preference module 102 and media processing module 104 thus the electronic device may process a media stream based on corresponding editing preference. A cloud application 152 may implement or interact with repository of generic media content 106. Cloud application 152 may be referred to as beingremote while electronic device 150 being local to indicate that the former may be far away geographically from the media processing at the latter. Cloud application 152 may be implemented in an electronic device as well in some embodiments.

[0027] Media processing module 104 may processing media content obtained in a variety of ways, e.g., through a camera, a display and an input device (e.g., touch screen, mouse, trackpad, keyboard, microphone or camera with gesture detection) that is implemented within or coupled to an electronic device (e.g., electronic device 150). The media processing module 104 may be an application implemented on the electronic device.

[0028] Editing preference module 102 stores editing preference and provides a user interface for an editor to interact with two-dimensional (2D) and / or three-dimensional (3D) scene capture. A 3D scene can include a person captured in a scene and any objects in the environment. Capturing can be performed by a stand-alone camera, a mobile phone, or another device (e.g., any of the UEs discussed herein relating to Figure 8). The editor also provides input to select regions of interest (ROIs) and tags for objects as discussed in more detail herein.

[0029] Repository of generic media content 106 contains generic media content to be used to replace content in a media stream, where the content contains aspects or features that are deemed private thus requiring or recommending removal from the media stream. Generic media content may be also referred to privacy-preserving content, obfuscated media content, common media content, standardized media content, generic content or similar terms, and it do not include information that is identifiable with a media stream to which it is applied to replace certain existing content of the media stream. Generic media content includes images, videos, or other visual content that has no specific or unique features that reveal personal identifying information, contact information, familial information, professional / educational information, financial information, health information, geolocation data, biometric data, and so on. The generic media content exemplifies an object without revealing privacy specific to individuals and their environments. Generic media content corresponds to corresponding object categories and, each content may be tagged to or indexed by an object category.

[0030] The generic media content within the repository may be generated through ML / Al, and it may be generated prior to the media processing by media processing module or at run time based on the certain existing content of the media stream as provided. Repository of generic media content 106 may include a structured data storage system. The structured data storage system is intended to store data (which may be voluminous) for possible consumption by multiple clients. The structured data storage system includes those which typically store persist data (e.g., an on-disk database) and those which typically store non-persist data (e.g., an inmemory database, a streaming platform, a key-value datastore, a document store, etc.).

[0031] The generation of the generic media content may be performed by a set of servers that may host ML / Al models and process messages / packets from media processing module 104 (e.g., through executing cloud application 152), as further discussed relating to Figure 2.

[0032] At reference 112, media processing module 104 initiates a 2D / 3D scene capture process to generate a media stream. The scene capture process may be initiated and controlled by an editor through a user interface (e.g., provided by editing preference module 102). In some embodiments, the media stream is generated somewhere else, and it is provided to media processing module 104 for media stream editing. The 2D / 3D scene may be captured in an immersive environment such as metaverse, which includes a digital or virtual space where a user may interact with a three-dimensional, computer-generated world through an augmented reality (AR) and / or virtual reality (VR) application. In some embodiments, the media stream corresponds to a 3D stream used for real-time conversational services such as holographic communication.

[0033] In these and similar embodiments, the media editing of a media stream to replace media content with generic media content is performed in real-time, as the media stream is being captured, the privacy revealing content within the media stream is being replaced with the generic media content.

[0034] For a 2D scene capture, the geometry of scenes may be inferred using any available 2D-to-3D depth approximation method (e.g., 3D or 2D + 1 projection). For 3D scene capture, scenes of geometric mesh or point cloud may be generated using information such as Red Green Blue (RGB) and measured depth information obtained using, e.g., Light Detection and Ranging (LiDAR) input data, true depth sensors, stereo cameras.

[0035] The inferred / generated scenes may be presented (e.g., through a display on electronic device 150) to an editor through a user interface (e.g., one at editing preference module 102). The editor may then interact with the user interface to select one or more regions of interest (ROIs) within the scenes at reference 114. The user interface may be one of a Command Line Interface (CLI), a web-based interface, an Application Programming Interface (API) based interface, and a graphical user interface (GUI) based interface.

[0036] A region of interest (RO I) indicates an area that includes features in the scene that are identifiable with the media stream and that are deemed not suitable / legal to be shown without redaction / replacement. For example, a bystander in a video stream, unless the bystander gives consent to be included in the video stream, needs to be redacted or replaced before publication of the video stream in some jurisdictions. The editor then may select the ROI that includes the bystander from the scenes so media processing module 104 may replace the content containing bystander’s personal information with generic media content indicating a generic bystander (notthe particular bystander captured by the media stream). An ROI may be in a variety of geometric shapes, including a triangle, a circle, an oval, or a free form. For example, the editor may select the ROI around the contour of the bystander or a group of bystanders.

[0037] Alternatively, the ROIs may be selected automatically through machine learning. In the example of bystander in the video stream, media processing module 104 may detect, by a machine learning model (e.g., one implemented at media processing module 104, editing preference module 102, or another module), the content creator of the video stream, and any regions with a detected human body other than the content creator may be set to be an ROI for replacement.

[0038] Then at reference 116, objects within the ROIs are detected and the objects are labelled with applicable object categories. An object within an ROI may be detected using any 2D or 3D object detection models, including Region-based Convolutional Neural Network (R-CNN), You Only Look Once (YOLO), Single Shot MultiBox Detector (SSD), Point R-CNN (PointRCNN), VoxelNet, Sparsely Embedded Convolutional Detection (SECOND), PointPillars, and CenterNet.

[0039] The detected objects are then labelled with applicable object categories, where a detected object may be labelled with multiple object categories. The labelling may be based on a mapping data structure that contains entries to map one object to one or more object categories. An object may be labelled with multiple applicable object categories with different granularity of object categories to which the object belongs. For example, a bystander detected in a ROI may be categorized with increasing specificity as a generic bystander, a male bystander, or a male bystander wearing a suit.

[0040] The applicable object categories for one object with increasing specificity is advantageous for scene integration and provide better user experience with the media stream after the content replacement. For example, in a scene of professional setting, a more specific content based on a male bystander wearing a suit as a replacement content will likely integrate with the scene better than a more abstract content based on a male bystander who happens to wear a vacation shirt. The mapping data structure is discussed in further detail relating to Figure 2 and 3 below.

[0041] At reference 118, when a detected object in the ROIs is labelled with multiple applicable object categories, one object category is selected from the multiple applicable object categories to be the sole label for the object. The multiple applicable object categories may be presented (e.g., through a display on electronic device 150) to an editor through a user interface (e.g., one at editing preference module 102). The editor may then interact with the user interfaceto select the single object category to label the object. Similar to the operations at reference 114, the object category may be selected automatically through machine learning as well.

[0042] At reference 120, media processing module 104 provides the object categories of the detected objects in the RO Is to the repository of generic media content 106 and / or the corresponding application 152, which returns generic media content without information identifiable with the media stream at reference 122. The content within repository of generic media content 106 may be indexed on or indicated with object categories, so that based on a provided object category, the corresponding generic media content is retrieved from the repository of generic media content 106. When no corresponding generic media content is found in the repository of generic media content 106, new generic media content may be generated for the provided object category.

[0043] At reference 126, the generic media content for the ROIs replaces the existing media content for the detected objects in the ROIs and is integrated with the rest of the media content in the scene. The integration may be performed by media integration tool such as OpenShot, FFmpeg, Avidemux, DaVinci Resolve, Unity, and Unreal Engine in some embodiments.

[0044] With the integration, the new media stream can offer a user immersive experience with privacy-preserving content. The process may be repeated as the media stream is provided to the media processing module 104, which removes identifiable information based on preference or applicable laws / regulations continuously until the completion of the media stream.

[0045] Figure 2 illustrates operations of editing a media content stream to preserve privacy per additional embodiments. Figure 2 shows more specific embodiments in steps shown in Figure 1. System 200 is similar to system 100 and the same or similar references indicate elements or components having the same or similar operations, and the difference / additional operations are discussed herein below.

[0046] At reference 212, a user interface is provided for user selection of ROIs after 2D / 3D scene capture (e.g., operation at reference 112). The selection of the ROIs may be manually done by an editor, or through applying machine learning as shown at reference 252. The selected ROIs are then provided to media processing module 104 at reference 222.

[0047] Once the ROIs are selected, the coordinates of each ROI may be recorded. These coordinates are often referred to as the "bounding box" or "ROI coordinates." For example, for a rectangle ROI in a 2D image, the boundary of the ROI may be represented by the coordinates of four values in the top-left and bottom-right comers: X-coordinate of the top-left comer (XI), Y- coordinate of the top-left corner (Yl), X-coordinate of the bottom -right corner (X2), and Y- coordinate of the bottom-right comer (Y2).

[0048] In some embodiments, the coordinates of an ROI may be revised based on input of an editor to a user interface or by machine learning (e.g., through editing preference module 102). For example, the revision may be a buffer value entered by the editor to guard against shifting of moving objects or accidental revelation of content that contain information to be preserved. The buffer value offers margin allowance thus enlarges the ROI for content replacement.

[0049] On the other hand, in some embodiments, some content within the ROI may be exempted from content replacement, and such content may be selected to be in a safelist so that the content will not be replaced despite being within the ROI. The selection to the safelist may be based on input of an editor to a user interface or by machine learning (e.g., through editing preference module 102) as well, and the coordinates of the content on the safelist are stored.

[0050] Additionally, the ROI may include objects that are partially within the ROI and partially outside, and input of an editor or input based on machine learning (e.g., through editing preference module 102) may indicate whether an object that is partially in the ROI will be replaced with generic media content. Instead of the binary selection of either replacing or not replacing, one or more rules may be applied, e.g., if more than half of a detected object is within the ROI, the object is replaced with generic media content, otherwise the object is not replaced.

[0051] At reference 116, objects within the ROIs are detected and the objects are labelled with applicable object categories. A container may be initiated for a selected ROI, thus multiple containers may be initiated for the ROIs of a scene. Each container may include a data structure to store content of the ROI, where the ROI may be processed by buffer value revision, safelist selection, and / or partial object processing setting in some embodiments as shown at reference 215.

[0052] One or more objects within an ROI container are then detected, and such detection may be offloaded to another local application or remote application (e.g., cloud application) that has more resources / expertise for object detection, as shown at reference 254. The content within the ROI container is provided to the object detection application, which provides the object detection results, including the detected object information. The detected object information includes the object label and the corresponding coordinates of the object. In some embodiments, the detected object information within the ROI container may be displayed, along with the detected object and / or object label (see Figures 3A-3B).

[0053] In some embodiments, the detected objects are then labelled with applicable object categories based on a mapping data structure at reference 256. The mapping data structure may be a table, a map, a dictionary, a list, an array, a file, or another data structure that includes a mapping between an object and one or more object categories. The mapping data structure may be stored locally (e.g., at media processing module 104, editing preference module 102, oranother module within or accessible by electronic device 150) or remotely (e.g., at cloud application 152 or another remote application), and it may be stored in a database.

[0054] The mapping data structure may maintain multiple hierarchical structures of object categories, one object is mapped to one hierarchical structure of object categories, and the object can be label to either a more generic or more specific object category in a corresponding hierarchical structure. The selection of the object category may be based on input to a user interface in some embodiments. Alternatively, the selection may be based on a machine learning model, which may be trained to select the object category that provides the generic media content that integrates with the rest of the media content to offer the best user experience.

[0055] Figures 3A-3B illustrate a hierarchical structure of object categories and examples of the hierarchical structure per some embodiments. As shown in Figure 3 A, the hierarchical structure of object categories includes multiple layers, where the top node (root) is the object category of a group of bystanders at reference 302. The next level below includes the nodes of object subcategory level 1, ones for either a group of bystanders without showing individuals at reference 312, or a group of bystanders showing individuals at reference 314. For the latter, the next level below in the hierarchy includes the nodes of object subcategory level 2, ones for either each bystander displayed without showing individuals at reference 322, or each bystander displayed with facial features shown at reference 324.

[0056] When an object is detected with the resulting object label, the label (e.g., object 350) may be used to select the object category from multiple applicable object categories (e.g., through mapping 354). For example, a human is detected in an ROI, and unless the human is on the safelist, the human is determined to be a bystander to be replaced so no personal information of the human may be revealed through the media stream. An object category in any level of the hierarchy may be selected based on input of an editor or machine learning. The selected object category for the object exemplifies the object without revealing one or more features that are captured specifically in the scene of the media stream.

[0057] When the selection is based on editor input, the hierarchical structure of object categories, along with the content associated with the object categories in some embodiments, is shown to the editor, so the editor may provide the appropriate object category for the object.

[0058] Figure 3B shows examples of the hierarchical structure of object categories for editor to select. In this example, a decorative plant is detected in a scene, and it is given an object label of decorative object at reference 362, and the object label corresponds to a hierarchical structure of decorative plant with at least three level of categories. The top node is decorative plant (Cat: decorative plant), and its subcategory level 1 is potted plant (subcat: pottedplant), which has two subcategories level 2, pothos (subcat2: pothos) and ceramic pot (subcat2: ceramicpot), and thelatter of which has a subcategory level 3, drainage pot (subcat3: drainpot). The hierarchical structure of object categories itself, or the hierarchical structure of object categories along with the scene of the detected object may be presented to the editor (e.g., through a user interface) so the editor may provide input and select one of the category / subcategories in the hierarchy to be the selected object category.

[0059] Other examples of relevant object categories may be “framed images,” e.g., a framed photograph on a wall depicted in the media stream. A subcategory level 1 may be photographs. Another subcategory level 1 may be diplomas / certificates. A subcategory level 2, depending from the “photographs” subcategory level 1, may be “family photographs.” Because a family photograph can contain identifying information by depicting a person’s family members, it may be desired to remove the actual family photograph to avoid such disclosure and ensure familial privacy.

[0060] Referring back to Figure 2, the object category of the objects is selected at reference 118 when multiple object categories are applicable. At reference 258, the selected object categories for the detected objects are then used to swap the detected objects in the ROI containers. That is, in each ROI container, the selected object categories and corresponding metadata for each of the one or more objects within the ROI container are stored in the ROI container, while the corresponding objects of the object categories are removed from the ROI container.

[0061] At reference 260, one or more data packets are formed to include the selected object categories and corresponding metadata for a ROI container. A stream of data packets may be formed for all the ROI containers corresponding to the selected ROIs. The data packets may carry payloads in one or more structures such as JavaScript Object Notation (JSON), extensible Markup Language (XML), YAML Ain't Markup Language (YAML), Concise Binary Object Representation (CBOR), Binary JSON (BSON), and Hierarchical Data Format 5 (HDF5).

[0062] At reference 220, these data packets are transmitted through a secure channel. The data packets are transmitted to obtain the corresponding generic media content at cloud application 152. The transmission channel may be secured in a variety of ways. Figure 4 illustrates data transmission in a secure channel per some embodiments. As shown, multiple electronic devices are communicated with cloud application 152 through a communication network 490 (e.g., telecommunication network 702). The electronic devices include electronic device 150 for media processing as discussed herein above, and they also include electronic device 452 and / or 454, and other electronic devices for media processing or other operations. Each of the electronic devices may communicate with cloud application 152 through a secure channel.

[0063] The secure channel, as shown at reference 442, may implement one or more of protocols such as Quick UDP Internet Connections (QUIC), WebSocket, Stream Control Transmission Protocol (SCTP), Message Queuing Telemetry Transport (MQTT), Web Real- Time Communication (WebRTC), and Hypertext Transfer Protocol Version three (HTTP / 3).

[0064] In some embodiments, additional security is added, where privacy preserving aggregation from multiple parties is added at reference 444, so that the recipient of the data packets is unaware of the sources of the received data packets. To mask the sources of the data packets for the object categories and corresponding metadata, the privacy preserving aggregation techniques include federated learning, secure multi-party computation (SMPC), differential privacy, and private set intersection. The privacy preserving aggregation adds an additional lay of security so that private data, even just object categories of objects in an edited media stream is not exposed to another party, including cloud application 152 that provides the generic media content replacement. In these embodiments, the provided generic media content replacement is broadcasted to the electronic devices in aggregate, which then select their required generic media content.

[0065] Reference back to Figure 2, cloud application 152 decrypts the data packets and extracts the payload that indicates the object categories and corresponding metadata to retrieve an applicable generic media content from the repository of generic media content 106. As discussed herein above, the content within repository 106 may be indexed on or indicated with object categories, so that based on a provided object category, the corresponding generic media content within repository 106 is retrieved promptly. When corresponding generic media content is found within repository 106, new generic media content is generated. The generation of the new generic media content based on the provided object category and corresponding metadata may be performed by machine learning models 206.

[0066] Machine learning models 206 may include one or more discriminative and / or generative machine learning models. A discriminative machine learning model is used to classify or differentiate data into predefined categories based on input data. Cloud application 152 may provide an object category and corresponding metadata to a discriminative machine learning model as input data, so the discriminative machine learning model may search other repository to identify generic media content that matches the object category and corresponding metadata. The discriminative machine learning models in machine learning models 206 may include one or more of logistic regression, Support Vector Machines (SVM), neural networks, decision trees, random forests, Naive Bayes, and K-Nearest Neighbors (K-NN).

[0067] A generative machine learning model generates new data that is similar to the data it was trained on, where the produced novel, synthetic data samples share characteristics with theoriginal dataset. Cloud application 152 may train one or more generative machine learning models with various object categories and corresponding generic media content. The trained models are then used to provide new generic media content based on the provided object category and corresponding metadata. The generative machine learning models in machine learning models 206 may include one or more of Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Recurrent Neural Networks (RNNs), transformers (e.g., Generative Pre-trained Transformer), and Long Short-Term Memory (LSTM) Networks.

[0068] Note that in some embodiments, the generative machine learning model may also generate entries within the mapping data structure to identify object categories of a detected object when no applicable object categories is identified in the mapping data structure for the object.

[0069] At reference 252, machine learning models 206 are applied to identify the generic media content corresponding to the object categories and corresponding metadata. The packets including the generic media content are then transmitted back to media processing module 104 at reference 222. The transmission of the packets is through a secure channel as well in some embodiments.

[0070] Media processing module 104 then extracts the payloads of the packets that include the generic media content, and at reference 126, integrates the generic media content with the rest of the media content in the scene, replacing the content for the detected objects in the ROIs.Operations per some embodiments

[0071] Figure 5 is a flow diagram illustrating operations of editing a media content stream to preserve privacy per some embodiments. The operations of method 500 may be implemented in an electronic device (e.g., electronic device 150 or 602) that includes media processing module 104.

[0072] At reference 502, one or more objects are detected within a region of interest (ROI) of a scene to be edited in a media stream. The ROI may be one of multiple ROIs selected from the scene as discussed herein above. The objects may be detected through 2D or 3D object detection models discussed herein above. As an example, a framed family photograph may be included in the scene and be detected. See Figures 3 A-3B and related discussion herein above.

[0073] At reference 504, the one or more objects detected within the ROI are then labelled with one or more corresponding object categories of the ROI. For example, the framed family photography may be labelled, tagged, or otherwise associated with a category and / or subcategory identifying it as potentially including familial information or other information that the streamer may desire not to disclose.

[0074] At reference 506, the object categories are provided to an application through a secure channel, the application to obtain generic media content corresponding to the one or more object categories, wherein the generic media content exemplifies the one or more object categories but with no information identifiable with the media stream. As in the example for the framed family photograph, the application may obtain a stock photograph of a family or a landscape or other image that does not have the potential to disclose private information.

[0075] At reference 508, the generic media content corresponding to the one or more object categories is received from the application, and at reference 510, media content corresponding to the one or more objects within the ROI is replaced by the generic media content received from the application.

[0076] In some embodiments, a plurality of object categories is determined to be associated with an object of the one or more objects, and one of the plurality of object categories is selected for the object to be provided to the application based on input to a user interface. In some embodiment, the ROI to be replaced is selected based on input to a user interface. The user interface may be one provided through editing preference module 102 in some embodiments as discussed herein.

[0077] In some embodiments, a set of objects within the ROI is selected to be a safelist and not labeled and replaced.

[0078] In some embodiments, data of the ROI is stored in a container based on coordinates of the ROI, wherein the one or more objects are detected within the container, and the one or more objects are labelled with object categories by looking up a mapping data structure to identify object categories of the objects, the mapping data structure including mappings between objects and object categories.

[0079] In some embodiments, boundaries of the ROI are refined prior to detecting the one or more objects, wherein the boundaries are refined by adding one or more offsets of the coordinates of the ROI.

[0080] In some embodiments, the mapping data structure maintains a plurality of hierarchical structures of object categories, one object is mapped to one hierarchical structure of object categories, and the object can be labelled to either a more generic or more specific object category in a corresponding hierarchical structure based on input to a user interface. An exemplary hierarchical structure of object categories is shown in Figure 3.

[0081] In some embodiments, an entry to map an object of the one or more objects to a corresponding object category is generated through a generative machine learning model.

[0082] In some embodiments, wherein whether an object overlapping with boundaries of the ROI is to be replaced is set based on input to a user interface.

[0083] In some embodiments, the corresponding object categories of the ROI are provided to the application with the source information of the corresponding object categories being masked. The masking is performed through the privacy preserving aggregation techniques discussed herein above.

[0084] In some embodiments, through a generative machine learning model, the application provides the generic media content corresponding to the object category.

[0085] In some embodiments, the generative machine learning model is executed upon a determination that the media content corresponding to the object categories is not stored in an existing media content a repository of generic media content.

[0086] Through the embodiments, a 2D or 3D media stream may be edited efficiently to replace media content that contains information that is identifiable with the media stream so that the media stream preserves privacy with generic media content. Efficiency herein is achieved through the application to obtain the generic media content separated from the media content editing. By offloading the generic media content obtained to the application, the application can be executed more efficiently (e.g., in a resource rich cloud environment). The generic media content is generated based on the object categories and can be used to replace objects within the ROIs and integrate with the rest of the media stream seamlessly.

[0087] Additionally, the settings for the media stream editing, from the regions of interest selection, object category selection to label an object, to overlapping object replacement setting, can be based on input to a user interface or machine learning, and the flexibility allowed these embodiments to be deployed in different scenarios.

[0088] Furthermore, the generative machine learning model allows these embodiments to generate generic media content that is not available in the existing repository.

[0089] These features of the embodiments offer unique advantages over prior approaches, so that the embodiments may be used to edit a media stream in real-time or near real-time and with the seamless integration, the editing offers user immersive experience with the privacypreserving content.Devices and Environments for Implementing Embodiments of the Invention

[0090] Figure 6 illustrates an electronic device to edit a media content stream to preserve privacy per some embodiments. The electronic device 602 may be a host in a cloud system, or a network node or UE in a wireless / wireline network, and the operating environment and further embodiments the host and the network node are discussed in more details discussed relating to Figures 7 to 12. The electronic device 602 may be implemented using custom applicationspecific integrated-circuits (ASICs) as processors and a special-purpose operating system (OS),or common off-the-shelf (COTS) processors and a standard OS. In some embodiments, electronic device 602 implements the operations discussed herein relating to Figures 1 to 5.

[0091] The electronic device 602 includes hardware 640 comprising a set of one or more processors 642 (which are typically COTS processors or processor cores or ASICs) and physical NIs 646, as well as non-transitory machine-readable storage media 649 having stored therein software 650. During operation, the one or more processors 642 may execute the software 650 to instantiate one or more sets of one or more applications 664A-R. While one embodiment does not implement virtualization, alternative embodiments may use different forms of virtualization. For example, in one such alternative embodiment, the virtualization layer 654 represents the kernel of an operating system (or a shim executing on a base operating system) that allows for the creation of multiple instances 662A-R called software containers that may each be used to execute one (or more) of the sets of applications 664A-R. The multiple software containers (also called virtualization engines, virtual private servers, or jails) are user spaces (typically a virtual memory space) that are separate from each other and separate from the kernel space in which the operating system is run. The set of applications running in a given user space, unless explicitly allowed, cannot access the memory of the other processes. In another such alternative embodiment, the virtualization layer 654 represents a hypervisor (sometimes referred to as a virtual machine monitor (VMM)) or a hypervisor executing on top of a host operating system, and each of the sets of applications 664A-R run on top of a guest operating system within an instance 662A-R called a virtual machine (which may in some cases be considered a tightly isolated form of software container) that run on top of the hypervisor - the guest operating system and application may not know that they are running on a virtual machine as opposed to running on a “bare metal” host electronic device, or through para-virtualization the operating system and / or application may be aware of the presence of virtualization for optimization purposes. In yet other alternative embodiments, one, some, or all of the applications are implemented as unikernel(s), which can be generated by compiling directly with an application only a limited set of libraries (e.g., from a library operating system (LibOS) including drivers / libraries of OS services) that provide the particul r OS sendees needed by the application. As a unikernel can be implemented to run directly on hardware 640, directly on a hypervisor (in which case the unikernel is sometimes described as running within a LibOS virtual machine), or in a software container, embodiments can be implemented fully with unikemels running directly on a hypervisor represented by virtualization layer 654, unikernels running within software containers represented by instances 662A-R, or as a combination of unikemels and the above-described techniques (e.g., unikemels and virtual machines both rundirectly on a hypervisor, unikemels, and sets of applications that are run in different software containers).

[0092] The software 650 includes media processing module 104 that performs operations described with reference to operations as discussed relating to Figures 1 to 5. The content replacement management module 655 may perform operations of media processing module 104. The content replacement management module 655 be instantiated within the applications 664 A- R. The instantiation of the one or more sets of one or more applications 664A-R, as well as virtualization if implemented, are collectively referred to as software instance(s) 652. Each set of applications 664 A-R, corresponding virtualization construct (e.g., instance 662 A-R) if implemented, and that part of the hardware 640 that executes them (be it hardware dedicated to that execution and / or time slices of hardware temporally shared), forms a separate virtual electronic device 660A-R.

[0093] A network interface (NI) may be physical or virtual. In the context of Internet Protocol (IP), an interface address is an IP address assigned to an NI, be it a physical NI or virtual NI. A virtual NI may be associated with a physical NI, with another virtual interface, or stand on its own (e.g., a loopback interface, a point-to-point protocol interface). A NI (physical or virtual) may be numbered (a NI with an IP address) or unnumbered (a NI without an IP address). The NI is shown as network interface card (NIC) 644. The physical network interface 646 may include one or more antenna of the electronic device 602. An antenna port may or may not correspond to a physical antenna. The antenna comprises one or more radio interfaces.A Wireless Network per Some Embodiments

[0094] Figure 7 illustrates an example of a communication system 700 per some embodiments. In the example, the communication system 700 includes a telecommunication network 702 that includes an access network 704, such as a radio access network (RAN), and a core network 706, which includes one or more core network nodes 708. The access network 704 includes one or more access network nodes, such as network nodes 710 A and 710B (one or more of which may be generally referred to as network nodes 710), or any other similar 3rdGeneration Partnership Project (3GPP) access nodes or non-3GPP access points. Moreover, as will be appreciated by those of skill in the art, a network node is not necessarily limited to an implementation in which a radio portion and a baseband portion are supplied and integrated by a single vendor. Thus, it will be understood that network nodes include disaggregated implementations or portions thereof. For example, in some embodiments, the telecommunication network 702 includes one or more Open-RAN (ORAN) network nodes. An ORAN network node is a node in the telecommunication network 702 that supports an ORAN specification (e.g., a specification published by the O-RAN Alliance, or any similarorganization) and may operate alone or together with other nodes to implement one or more functionalities of any node in the telecommunication network 702, including one or more network nodes 710 and / or core network nodes 708. A network can correspond to a 3GPP network (4G / 5G / 6G), Wi-Fi, or any other standard-based or proprietary network.

[0095] Examples of an ORAN network node include an open radio unit (O-RU), an open distributed unit (O-DU), an open central unit (O-CU), including an O-CU control plane (O-CU- CP) or an O-CU user plane (O-CU-UP), a RAN intelligent controller (near-real time or non-real time) hosting software or software plug-ins, such as a near-real time control application (e.g., xApp) or a non-real time control application (e.g., rApp), or any combination thereof (the adjective “open” designating support of an ORAN specification). The network node may support a specification by, for example, supporting an interface defined by the ORAN specification, such as an Al, Fl, Wl, El, E2, X2, Xn interface, an open fronthaul user plane interface, or an open fronthaul management plane interface. Moreover, an ORAN access node may be a logical node in a physical node. Furthermore, an ORAN network node may be implemented in a virtualization environment (described further below) in which one or more network functions are virtualized. For example, the virtualization environment may include an O-Cloud computing platform orchestrated by a Service Management and Orchestration Framework via an O-2 interface defined by the 0-RAN Alliance or comparable technologies. The network nodes 710 facilitate direct or indirect connection of user equipment (UE), such as by connecting UEs 712A, 712B, 712C, and 712D (one or more of which may be generally referred to as UEs 712) to the core network 706 over one or more wireless connections.

[0096] Example wireless communications over a wireless connection include transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the communication system 700 may include any number of wired or wireless networks, network nodes, UEs, and / or any other components or systems that may facilitate or participate in the communication of data and / or signals whether via wired or wireless connections. The communication system 700 may include and / or interface with any type of communication, telecommunication, data, cellular, radio network, and / or other similar type of system.

[0097] The UEs 712 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and / or operable to communicate wirelessly with the network nodes 710 and other communication devices. Similarly, the network nodes 710 are arranged, capable, configured, and / or operable to communicate directly or indirectly with the UEs 712 and / or with other network nodes or equipment in the telecommunication network 702to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as administration in the telecommunication network 702.

[0098] In the depicted example, the core network 706 connects the network nodes 710 to one or more hosts, such as host 716. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. The core network 706 includes one more core network nodes (e.g., core network node 708) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and / or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 708. Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0099] The host 716 may be under the ownership or control of a service provider other than an operator or provider of the access network 704 and / or the telecommunication network 702, and may be operated by the service provider or on behalf of the service provider. The host 716 may host a variety of applications to provide one or more services. Examples of such applications include live and pre-recorded audio / video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.

[0100] As a whole, the communication system 700 of Figure 7 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and / or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and / or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.

[0101] In some examples, the telecommunication network 702 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunication network 702 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network 702. For example, the telecommunication network 702 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and / or Massive Machine Type Communication (mMTC) / Massive loT services to yet further UEs.

[0102] In some examples, the UEs 712 are configured to transmit and / or receive information without direct human interaction. For instance, a UE may be designed to transmit information to the access network 704 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 704. Additionally, a UE may be configured for operating in single- or multiple radio access technology (multi-RAT) or multistandard mode. For example, a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e., being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC).

[0103] In the example, the hub 714 communicates with the access network 704 to facilitate indirect communication between one or more UEs (e.g., UE 712C and / or 712D) and network nodes (e.g., network node 710B). In some examples, the hub 714 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, the hub 714 may be a broadband router enabling access to the core network 706 for the UEs. As another example, the hub 714 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 710, or by executable code, script, process, or other instructions in the hub 714. As another example, the hub 714 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, the hub 714 may be a content source. For example, for a UE that is a virtual reality (VR) headset, display, loudspeaker or other media delivery device, the hub 714 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 714 then provides to the UE either directly, after performing local processing, and / or after adding additional local content. In still another example, the hub 714 acts as a proxy server or orchestrator for the UEs, in particular if one or more of the UEs are low energy loT devices.

[0104] The hub 714 may have a constant / persistent or intermittent connection to the network node 710b. The hub 714 may also allow for a different communication scheme and / or schedulebetween the hub 714 and UEs (e.g., UE 712C and / or 712D), and between the hub 714 and the core network 706. In other examples, the hub 714 is connected to the core network 706 and / or one or more UEs via a wired connection. Moreover, the hub 714 may be configured to connect to a machine-to-machine (M2M) service provider over the access network 704 and / or to another UE over a direct connection. In some scenarios, UEs may establish a wireless connection with the network nodes 710 while still connected via the hub 714 via a wired or wireless connection. In some embodiments, the hub 714 may be a dedicated hub - that is, a hub whose primary function is to route communications to / from the UEs from / to the network node 71 OB. In other embodiments, the hub 714 may be a non-dedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 71 OB, but which is additionally capable of operating as a communication start and / or end point for certain data channels. In some embodiments, electronic device 150 that implements media processing module 104 may be, comprise, or be coupled to one of network nodes 708, 710A-B, or host 717.UE per Some Embodiments

[0105] Figure 8 illustrates a UE 800 per some embodiments. As used herein, a UE refers to a device capable, configured, arranged and / or operable to communicate wirelessly with network nodes and / or other UEs. Examples of a UE include, but are not limited to, a smart phone, mobile phone, cell phone, voice over IP (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless cameras, gaming console or device, music storage device, playback appliance, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), smart device, wireless customer-premise equipment (CPE), vehicle, vehicle-mounted or vehicle embedded / integrated wireless device, etc. Other examples include any UE identified by the 3rd Generation Partnership Project (3GPP), including a narrow band internet of things (NB-IoT) UE, a machine type communication (MTC) UE, and / or an enhanced MTC (eMTC) UE. Electronic device 150 that implements media processing module 104 may be or be coupled to UE 800.

[0106] A UE may support device-to-device (D2D) communication, for example by implementing a 3 GPP standard for sidelink communication, Dedicated Short-Range Communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle- to-everything (V2X). In other examples, a UE may not necessarily have a user in the sense of a human user who owns and / or operates the relevant device. Instead, a UE may represent a device that is intended for sale to, or operation by, a human user but which may not, or which may not initially, be associated with a specific human user (e.g., a smart sprinkler controller).Alternatively, a UE may represent a device that is not intended for sale to, or operation by, auser but which may be associated with or operated for the benefit of a user (e.g., a smart power meter).

[0107] The UE 800 includes processing circuitry 802 that is operatively coupled via a bus 804 to an input / output interface 806, a power source 808, a memory 810, a communication interface 812, and / or any other component, or any combination thereof. Certain UEs may utilize all or a subset of the components shown in Figure 8. The level of integration between the components may vary from one UE to another UE. Further, certain UEs may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0108] The processing circuitry 802 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine-readable computer programs in the memory 810. The processing circuitry 802 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, field- programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general -purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, the processing circuitry 802 may include multiple central processing units (CPUs) or graphics processing units (GPUs).

[0109] In the example, the input / output interface 806 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and / or output devices. Examples of an output device include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smartcard, another output device, or any combination thereof. An input device may allow a user to capture information into the UE 800. Examples of an input device include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a web camera, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smartcard, and the like. The presence-sensitive display may include a capacitive or resistive touch sensor to sense input from a user. A sensor may be, for instance, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, an optical sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.

[0110] In some embodiments, the power source 808 is structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricity outlet), photovoltaic device, or power cell, may be used. The power source 808 may further includepower circuitry for delivering power from the power source 808 itself, and / or an external power source, to the various parts of the UE 800 via input circuitry or an interface such as an electrical power cable. Delivering power may be, for example, for charging of the power source 808. Power circuitry may perform any formatting, converting, or other modification to the power from the power source 808 to make the power suitable for the respective components of the UE 800 to which power is supplied.[oni] The memory 810 may be or be configured to include memory such as random-access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable readonly memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, the memory 810 includes one or more application programs 814, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data 816. The memory 810 may store, for use by the UE 800, any of a variety of various operating systems or combinations of operating systems.

[0112] The memory 810 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD-DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini -dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a Universal Subscriber Identity Module (USIM) and / or IP Multimedia Services Identity Module (ISIM), other memory, or any combination thereof. The UICC may for example be an embedded UICC (eUICC), integrated UICC (iUICC) or a removable UICC commonly known as ‘SIM card.’ The memory 810 may allow the UE 800 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data. An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 810, which may be or comprise a device-readable storage medium.

[0113] The processing circuitry 802 may be configured to communicate with an access network or other network using the communication interface 812. The communication interface 812 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 822. The communication interface 812 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or a networknode in an access network). Each transceiver may include a transmitter 818 and / or a receiver 820 appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth). Moreover, the transmitter 818 and receiver 820 may be coupled to one or more antennas (e.g., antenna 822) and may share circuit components, software or firmware, or alternatively be implemented separately.

[0114] In the illustrated embodiment, communication functions of the communication interface 812 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short- range communications such as Bluetooth, near-field communication, location-based communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof. Communications may be implemented in according to one or more communication protocols and / or standards, such as IEEE 802.11, Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol / internet protocol (TCP / IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), Quick UDP Internet Connections (QUIC), Hypertext Transfer Protocol (HTTP), WebRTC (Web Real-Time Communication), and so forth.

[0115] Regardless of the type of sensor, a UE may provide an output of data captured by its sensors, through its communication interface 812, via a wireless connection to a network node. Data captured by sensors of a UE can be communicated through a wireless connection to a network node via another UE. The output may be periodic (e.g., once every 15 minutes if it reports the sensed temperature), random (e.g., to even out the load from reporting from several sensors), in response to a triggering event (e.g., when moisture is detected an alert is sent), in response to a request (e.g., a user initiated request), or a continuous stream (e.g., a live video feed of a patient).

[0116] As another example, a UE comprises an actuator, a motor, or a switch, related to a communication interface configured to receive wireless input from a network node via a wireless connection. In response to the received wireless input the states of the actuator, the motor, or the switch may change. For example, the UE may comprise a motor that adjusts the control surfaces or rotors of a drone in flight according to the received input or to a robotic arm performing a medical procedure according to the received input.

[0117] A UE, when in the form of an Internet of Things (loT) device, may be a device for use in one or more application domains, these domains comprising, but not limited to, city wearable technology, extended industrial application and healthcare. Non-limiting examples of such an loT device are a device which is or which is embedded in: a connected refrigerator or freezer, aTV, a connected lighting device, an electricity meter, a robot vacuum cleaner, a voice controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door / window sensor, a flood / moisture sensor, an electrical door lock, a connected doorbell, an air conditioning system like a heat pump, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smart watch, a fitness tracker, a head-mounted display, or glasses for Augmented Reality (AR) or Virtual Reality (VR), a wearable for tactile augmentation or sensory enhancement, a water sprinkler, an animal- or item-tracking device, a sensor for monitoring a plant or animal, an industrial robot, an Unmanned Aerial Vehicle (UAV), and any kind of medical device, like a heart rate monitor or a remote controlled surgical robot. A UE in the form of an loT device comprises circuitry and / or software in dependence of the intended application of the loT device in addition to other components as described in relation to the UE 800 shown in Figure 8.

[0118] As yet another specific example, in an loT scenario, a UE may represent a machine or other device that performs monitoring and / or measurements, and transmits the results of such monitoring and / or measurements to another UE and / or a network node. The UE may in this case be an M2M device, which may in a 3GPP context be referred to as an MTC device. As one particular example, the UE may implement the 3GPP NB-IoT standard. In other scenarios, a UE may represent a vehicle, such as a car, a bus, a truck, a ship and an airplane, or other equipment that is capable of monitoring and / or reporting on its operational status or other functions associated with its operation.

[0119] In practice, any number of UEs may be used together with respect to a single use case. For example, a first UE might be or be integrated in a drone and provide the drone’s speed information (obtained through a speed sensor) to a second UE that is a remote controller operating the drone. When the user makes changes from the remote controller, the first UE may adjust the throttle on the drone (e.g., by controlling an actuator) to increase or decrease the drone’s speed. The first and / or the second UE can also include more than one of the functionalities described above. For example, a UE might comprise the sensor and the actuator, and handle communication of data for both the speed sensor and the actuators.Network Node per Some Embodiments

[0120] Figure 9 illustrates a network node 900 per some embodiments. As used herein, network node refers to equipment capable, configured, arranged and / or operable to communicate directly or indirectly with a UE and / or with other network nodes or equipment, in a telecommunication network. Examples of network nodes include, but are not limited to, access points (APs) (e.g., radio access points), base stations (BSs) (e.g., radio base stations, Node Bs,evolved Node Bs (eNBs) and NR NodeBs (gNBs)), O-RAN nodes or components of an O-RAN node (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs) and NR NodeBs (gNBs)).

[0121] Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations. A base station may be a relay node or a relay donor node controlling a relay. A network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units, distributed units (e.g., in an O-RAN access node) and / or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[0122] Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, multi -standard radio (MSR) equipment such as MSR BSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi -cell / multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, Self-Organizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and / or Minimization of Drive Tests (MDTs).

[0123] The network node 900 includes a processing circuitry 902, a memory 904, a communication interface 906, and a power source 908. The network node 900 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In certain scenarios in which the network node 900 comprises multiple separate components (e.g., BTS and BSC components), one or more of the separate components may be shared among several network nodes. For example, a single RNC may control multiple NodeBs. In such a scenario, each unique NodeB and RNC pair may in some instances be considered a single separate network node. In some embodiments, the network node 900 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 904 for different RATs) and some components may be reused (e.g., a same antenna 910 may be shared by different RATs). The network node 900 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 900, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wirelesstechnologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 900.

[0124] The processing circuitry 902 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and / or encoded logic operable to provide, either alone or in conjunction with other network node 900 components, such as the memory 904, to provide network node 900 functionality.

[0125] In some embodiments, the processing circuitry 902 includes a system on a chip (SOC). In some embodiments, the processing circuitry 902 includes one or more of radio frequency (RF) transceiver circuitry 912 and baseband processing circuitry 914. In some embodiments, the radio frequency (RF) transceiver circuitry 912 and the baseband processing circuitry 914 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 912 and baseband processing circuitry 914 may be on the same chip or set of chips, boards, or units.

[0126] The memory 904 may comprise any form of volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and / or any other volatile or non-volatile, non-transitory device-readable and / or computer-executable memory devices that store information, data, and / or instructions that may be used by the processing circuitry 902. The memory 904 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and / or other instructions capable of being executed by the processing circuitry 902 and utilized by the network node 900. The memory 904 may be used to store any calculations made by the processing circuitry 902 and / or any data received via the communication interface 906. In some embodiments, the processing circuitry 902 and memory 904 are integrated.

[0127] The communication interface 906 is used in wired or wireless communication of signaling and / or data between a network node, access network, and / or UE. As illustrated, the communication interface 906 comprises port(s) / terminal(s) 916 to send and receive data, for example to and from a network over a wired connection. The communication interface 906 also includes radio front-end circuitry 918 that may be coupled to, or in certain embodiments a part of, the antenna 910. Radio front-end circuitry 918 comprises filters 920 and amplifiers 922. The radio front-end circuitry 918 may be connected to an antenna 910 and processing circuitry 902.The radio front-end circuitry may be configured to condition signals communicated between antenna 910 and processing circuitry 902. The radio front-end circuitry 918 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection. The radio front-end circuitry 918 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 920 and / or amplifiers 922. The radio signal may then be transmitted via the antenna 910. Similarly, when receiving data, the antenna 910 may collect radio signals which are then converted into digital data by the radio front-end circuitry 918. The digital data may be passed to the processing circuitry 902. In other embodiments, the communication interface may comprise different components and / or different combinations of components.

[0128] In certain alternative embodiments, the network node 900 does not include separate radio front-end circuitry 918, instead, the processing circuitry 902 includes radio front-end circuitry and is connected to the antenna 910. Similarly, in some embodiments, all or some of the RF transceiver circuitry 912 is part of the communication interface 906. In still other embodiments, the communication interface 906 includes one or more ports or terminals 916, the radio front-end circuitry 918, and the RF transceiver circuitry 912, as part of a radio unit (not shown), and the communication interface 906 communicates with the baseband processing circuitry 914, which is part of a digital unit (not shown).

[0129] The antenna 910 may include one or more antennas, or antenna arrays, configured to send and / or receive wireless signals. The antenna 910 may be coupled to the radio front-end circuitry 918 and may be any type of antenna capable of transmitting and receiving data and / or signals wirelessly. In certain embodiments, the antenna 910 is separate from the network node 900 and connectable to the network node 900 through an interface or port.

[0130] The antenna 910, communication interface 906, and / or the processing circuitry 902 may be configured to perform any receiving operations and / or certain obtaining operations described herein as being performed by the network node. Any information, data and / or signals may be received from a UE, another network node and / or any other network equipment. Similarly, the antenna 910, the communication interface 906, and / or the processing circuitry 902 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and / or signals may be transmitted to a UE, another network node and / or any other network equipment.

[0131] The power source 908 provides power to the various components of network node 900 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component). The power source 908 may further comprise, or be coupled to, power management circuitry to supply the components of the network node 900 with power forperforming the functionality described herein. For example, the network node 900 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of the power source 908. As a further example, the power source 908 may comprise a source of power in the form of a battery or battery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.

[0132] Embodiments of the network node 900 may include additional components beyond those shown in Figure 9 for providing certain aspects of the network node’s functionality, including any of the functionality described herein and / or any functionality necessary to support the subject matter described herein. For example, the network node 900 may include user interface equipment to allow input of information into the network node 900 and to allow output of information from the network node 900. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the network node 900. In some embodiments, the network node 900 may implement media processing module 104 and perform operations described herein.Host per Some Embodiments

[0133] Figure 10 is a block diagram of a host 1000, which may be an embodiment of the host 716 of Figure 7, per various aspects described herein. As used herein, the host 1000 may be or comprise various combinations hardware and / or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, container, or processing resources in a server farm. The host 1000 may provide one or more services to one or more UEs. In some embodiments, the host 1000 may implement media processing module 104 and perform operations described herein. Alternatively, the host 1000 may implement cloud application 152 and / or repository of generic media content 106, while a network node / UE implements implement media processing module 104.

[0134] The host 1000 includes processing circuitry 1002 that is operatively coupled via a bus 1004 to an input / output interface 1006, a network interface 1008, a power source 1010, and a memory 1012. Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as Figures 8 and 9, such that the descriptions thereof are generally applicable to the corresponding components of host 1000.

[0135] The memory 1012 may include one or more computer programs including one or more host application programs 1014 and data 1016, which may include user data, e.g., data generated by a UE for the host 1000 or data generated by the host 1000 for a UE. Embodiments of the host1000 may utilize only a subset or all of the components shown. The host application programs 1014 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), Moving Picture Experts Group (MPEG), VP9) and audio codecs (e.g., Free Lossless Audio Codec (FLAC), Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for multiple different classes, types, or implementations of UEs (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). The host application programs 1014 may also provide for user authentication and licensing checks and may periodically report health, routes, and content availability to a central node, such as a device in or on the edge of a core network. Accordingly, the host 1000 may select and / or indicate a different host for over-the-top services for a UE. The host application programs 1014 may support various protocols, such as the HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.Virtualization Environment per Some Embodiments

[0136] Figure 11 is a block diagram illustrating a virtualization environment ' 1100 in which functions implemented by some embodiments may be virtualized. In the present context, virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources. As used herein, virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1100 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a network node, UE, core network node, or host. Further, in embodiments in which the virtual node does not require radio connectivity (e.g., a core network node or host), then the node may be entirely virtualized. In some embodiments, the virtualization environment 1100 includes components defined by the 0-RAN Alliance, such as an O-Cloud environment orchestrated by a Service Management and Orchestration Framework via an O-2 interface. The virtualization environment may also implement pods and containers as shown in Figure 3.

[0137] Applications 1102 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment 1150 to implement some of the features, functions, and / or benefits ofsome of the embodiments disclosed herein. In some embodiments, Applications 1102 may implement media processing module 104 and perform operations described herein.

[0138] Hardware 1104 includes processing circuitry, memory that stores software and / or instructions executable by hardware processing circuitry, and / or other hardware devices as described herein, such as a network interface, input / output interface, and so forth. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1106 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 1108 A and 1108B (one or more of which may be generally referred to as VMs 1108), and / or perform any of the functions, features and / or benefits described in relation with some embodiments described herein. The virtualization layer 1106 may present a virtual operating platform that appears like networking hardware to the VMs 1108.

[0139] The VMs 1108 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 1106. Different embodiments of the instance of a virtual appliance 1102 may be implemented on one or more of VMs 1108, and the implementations may be made in different ways. Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.

[0140] In the context of NFV, a VM 1108 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, non-virtualized machine. Each of the VMs 1108, and that part of hardware 1104 that executes that VM, be it hardware dedicated to that VM and / or hardware shared by that VM with others of the VMs, forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 1108 on top of the hardware 1104 and corresponds to the application 1102.

[0141] Hardware 1104 may be implemented in a standalone network node with generic or specific components. Hardware 1104 may implement some functions via virtualization. Alternatively, hardware 1104 may be part of a larger cluster of hardware (e.g., such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 1110, which, among others, oversees lifecycle management of applications 1102. In some embodiments, hardware 1104 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components toprovide a virtual node with radio capabilities, such as a radio access node or a base station. In some embodiments, some signaling can be provided with the use of a control system 1112 which may alternatively be used for communication between hardware nodes and radio units.Communication among host network node, and UE per Some Embodiments

[0142] Figure 12 illustrates a communication diagram of a host 1202 communicating via a network node 1204 with a UE 1206 over a partially wireless per some embodiments. Example implementations, in accordance with various embodiments, of the UE (such as a UE 712A of Figure 7 and / or UE 800 of Figure 8), network node (such as a network node 710 of Figure 7 and / or network node 900 of Figure 9), and host (such as host 716 of Figure 7 and / or host 1000 of Figure 10) discussed in the preceding paragraphs will now be described with reference to Figure 12. Either host 1202, network node 1204, or UE 1206 may implement media processing module 104 and perform operations described herein.

[0143] Like host 1000, embodiments of host 1202 include hardware, such as a communication interface, processing circuitry, and memory. The host 1202 also includes software, which is stored in or accessible by the host 1202 and executable by the processing circuitry. The software includes a host application that may be operable to provide a service to a remote user, such as the UE 1206 connecting via an over-the-top (OTT) connection 1250 extending between the UE 1206 and host 1202. In providing the service to the remote user, a host application may provide user data which is transmitted using the OTT connection 1250.

[0144] The network node 1204 includes hardware enabling it to communicate with the host 1202 and UE 1206. The connection 1260 may be direct or pass through a core network (like core network 706 of Figure 7) and / or one or more other intermediate networks, such as one or more public, private, or hosted networks. For example, an intermediate network may be a backbone network or the Internet.

[0145] The UE 1206 includes hardware and software, which is stored in or accessible by UE 1206 and executable by the UE’s processing circuitry. The software includes a client application, such as a web browser or operator-specific “app” that may be operable to provide a service to a human or non-human user via UE 1206 with the support of the host 1202. In the host 1202, an executing host application may communicate with the executing client application via the OTT connection 1250 terminating at the UE 1206 and host 1202. In providing the service to the user, the UE's client application may receive request data from the host's host application and provide user data in response to the request data. The OTT connection 1250 may transfer both the request data and the user data. The UE's client application may interact with the user to generate the user data that it provides to the host application through the OTT connection 1250.

[0146] The OTT connection 1250 may extend via a connection 1260 between the host 1202 and the network node 1204 and via a wireless connection 1270 between the network node 1204 and the UE 1206 to provide the connection between the host 1202 and the UE 1206. The connection 1260 and wireless connection 1270, over which the OTT connection 1250 may be provided, have been drawn abstractly to illustrate the communication between the host 1202 and the UE 1206 via the network node 1204, without explicit reference to any intermediary devices and the precise routing of messages via these devices.

[0147] As an example of transmitting data via the OTT connection 1250, in step 1208, the host 1202 provides user data, which may be performed by executing a host application. In some embodiments, the user data is associated with a particular human user interacting with the UE 1206. In other embodiments, the user data is associated with a UE 1206 that shares data with the host 1202 without explicit human interaction. In step 1210, the host 1202 initiates a transmission carrying the user data towards the UE 1206. The host 1202 may initiate the transmission responsive to a request transmitted by the UE 1206. The request may be caused by human interaction with the UE 1206 or by operation of the client application executing on the UE 1206. The transmission may pass via the network node 1204, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 1212, the network node 1204 transmits to the UE 1206 the user data that was carried in the transmission that the host 1202 initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 1214, the UE 1206 receives the user data carried in the transmission, which may be performed by a client application executed on the UE 1206 associated with the host application executed by the host 1202.

[0148] In some examples, the UE 1206 executes a client application which provides user data to the host 1202. The user data may be provided in reaction or response to the data received from the host 1202. Accordingly, in step 1216, the UE 1206 may provide user data, which may be performed by executing the client application. In providing the user data, the client application may further consider user input received from the user via an input / output interface of the UE 1206. Regardless of the specific manner in which the user data was provided, the UE 1206 initiates, in step 1218, transmission of the user data towards the host 1202 via the network node 1204. In step 1220, in accordance with the teachings of the embodiments described throughout this disclosure, the network node 1204 receives user data from the UE 1206 and initiates transmission of the received user data towards the host 1202. In step 1222, the host 1202 receives the user data carried in the transmission initiated by the UE 1206.

[0149] In an example scenario, factory status information may be collected and analyzed by the host 1202. As another example, the host 1202 may process audio and video data which mayhave been retrieved from a UE for use in creating maps. As another example, the host 1202 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, the host 1202 may store surveillance video uploaded by a UE. As another example, the host 1202 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs. As other examples, the host 1202 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and / or transmitting data.

[0150] In some examples, a measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring the OTT connection 1250 between the host 1202 and UE 1206, in response to variations in the measurement results. The measurement procedure and / or the network functionality for reconfiguring the OTT connection may be implemented in software and hardware of the host 1202 and / or UE 1206. In some embodiments, sensors (not shown) may be deployed in or in association with other devices through which the OTT connection 1250 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software may compute or estimate the monitored quantities. The reconfiguring of the OTT connection 1250 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not directly alter the operation of the network node 1204. Such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary UE signaling that facilitates measurements of throughput, propagation times, latency and the like, by the host 1202. The measurements may be implemented in that software causes messages to be transmitted, in particular empty or ‘dummy’ messages, using the OTT connection 1250 while monitoring propagation times, errors, etc.

[0151] Although the computing devices described herein (e.g., UEs, network nodes, hosts) may include the illustrated combination of hardware components, other embodiments may comprise computing devices with different combinations of components. It is to be understood that these computing devices may comprise any suitable combination of hardware and / or software needed to perform the tasks, features, functions and methods disclosed herein. Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or convertedinformation to information stored in the network node, and / or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination. Moreover, while components are depicted as single boxes located within a larger box, or nested within multiple boxes, in practice, computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components. For example, a communication interface may be configured to include any of the components described herein, and / or the functionality of the components may be partitioned between the processing circuitry and the communication interface. In another example, non-computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.

[0152] In certain embodiments, some or all of the functionality described herein may be provided by processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer- readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner. In any of those particular embodiments, whether executing instructions stored on a non-transitory computer- readable storage medium or not, the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device, but are enjoyed by the computing device as a whole, and / or by end users and a wireless network generally.Terms

[0153] References in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” and so forth, indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

[0154] The description and claims may use the terms “coupled” and “connected,” along with their derivatives. These terms are not intended as synonyms for each other. “Coupled” is used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, co-operate or interact with each other. “Connected” is used to indicatethe establishment of wireless or wireline communication between two or more elements that are coupled with each other. A “set,” as used herein can refer to any whole number of items including one item.

[0155] An electronic device (such as the electronic device 150 or 602) stores and transmits (internally and / or with other electronic devices over a network) code (which is composed of software instructions and which is sometimes referred to as a computer program code or a computer program) and / or data using machine-readable media (also called computer-readable media), such as machine-readable storage media (e.g., magnetic disks, optical disks, solid state drives, read only memory (ROM), flash memory devices, phase change memory) and machine- readable transmission media (also called a carrier) (e.g., electrical, optical, radio, acoustical, or other form of propagated signals - such as carrier waves, infrared signals). Thus, an electronic device (e.g., a computer) includes hardware and software, such as a set of one or more processors (e.g., of which a processor is a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), other electronic circuitry, or a combination of one or more of the preceding) coupled to one or more machine-readable storage media to store code for execution on the set of processors and / or to store data. For instance, an electronic device may include non-volatile memory containing the code since the non-volatile memory can persist code / data even when the electronic device is turned off (when power is removed). When the electronic device is turned on, that part of the code that is to be executed by the processor(s) of the electronic device is typically copied from the slower non-volatile memory into volatile memory (e.g., dynamic random-access memory (DRAM), static random-access memory (SRAM)) of the electronic device. Typical electronic devices also include a set of one or more physical network interface(s) (NI(s)) to establish network connections (to transmit and / or receive code and / or data using propagating signals) with other electronic devices. For example, the set of physical NIs (or the set of physical NI(s) in combination with the set of processors executing code) may perform any formatting, coding, or translating to allow the electronic device to send and receive data whether over a wired and / or a wireless connection. In some embodiments, a physical NI may comprise radio circuitry capable of (1) receiving data from other electronic devices over a wireless connection and / or (2) sending data out to other devices through a wireless connection. This radio circuitry may include transmitter(s), receiver(s), and / or transceiver(s) suitable for radio frequency communication. The radio circuitry may convert digital data into a radio signal having the proper parameters (e.g., frequency, timing, channel, bandwidth, and so forth). The radio signal may then be transmitted through antennas to the appropriate recipient(s). In some embodiments, the set of physical NI(s) may comprisenetwork interface controlled s) (NICs), also known as a network interface card, network adapter, or local area network (LAN) adapter. The NIC(s) may facilitate connecting the electronic device to other electronic devices allowing them to communicate with wire through plugging in a cable to a physical port connected to an NIC. One or more parts of an embodiment of the invention may be implemented using different combinations of software, firmware, and / or hardware.

[0156] The terms “module,” “logic,” and “unit” used in the present application, may refer to a circuit for performing the function specified. In some embodiments, the function specified may be performed by a circuit in combination with software such as by software executed by a general -purpose processor.

[0157] Any appropriate steps, methods, features, functions, or benefits disclosed herein may be performed through one or more functional units or modules of one or more virtual apparatuses. Each virtual apparatus may comprise a number of these functional units. These functional units may be implemented via processing circuitry, which may include one or more microprocessor or microcontrollers, as well as other digital hardware, which may include digital signal processors (DSPs), special-purpose digital logic, and the like. The processing circuitry may be configured to execute program code stored in memory, which may include one or several types of memory such as read-only memory (ROM), random-access memory (RAM), cache memory, flash memory devices, optical storage devices, etc. Program code stored in memory includes program instructions for executing one or more telecommunications and / or data communications protocols as well as instructions for carrying out one or more of the techniques described herein. In some implementations, the processing circuitry may be used to cause the respective functional unit to perform corresponding functions according to one or more embodiments of the present disclosure.

[0158] The term unit may have conventional meaning in the field of electronics, electrical devices, and / or electronic devices and may include, for example, electrical and / or electronic circuitry, devices, modules, processors, memories, logic solid state and / or discrete devices, computer programs or instructions for carrying out respective tasks, procedures, computations, outputs, and / or displaying functions, and so on, as such as those that are described herein.

Claims

CLAIMSWhat is claimed is:

1. A method to replace media content in a media stream, comprising: detecting (502) one or more objects within a region of interest (ROI) of a scene to be edited in a media stream; labeling (504) the one or more objects detected within the ROI with one or more object categories of the ROI; providing (506) the one or more object categories of the ROI to an application through a secure channel, the application to obtain generic media content corresponding to the one or more object categories, wherein the generic media content exemplifies the one or more object categories but with no information identifiable with the media stream; receiving (508) the generic media content corresponding to the one or more object categories from the application; and replacing (510) media content corresponding to the one or more objects within the ROI by the generic media content received from the application.

2. The method of claim 1, wherein a plurality of object categories is determined to be associated with an object of the one or more objects, and one of the plurality of object categories is selected for the object to be provided to the application based on input to a user interface.

3. The method of claim 1 or 2, wherein the ROI to be edited is selected based on input to a user interface.

4. The method of any of claims 1 to 3, wherein a set of objects within the ROI is selected to be a safelist and not labeled and replaced.

5. The method of any of claims 1 to 4, wherein data of the ROI is stored in a container based on coordinates of the ROI, wherein the one or more objects are detected within the container, and the one or more objects are labelled with object categories by looking up a mapping data structure to identify object categories of the objects, the mapping data structure including entries to map objects and object categories.

6. The method of claim 5, wherein boundaries of the ROI are refined prior to detecting the one or more objects, wherein the boundaries are refined by adding one or more offsets of the coordinates of the ROI.

7. The method of claim 5, wherein the mapping data structure maintains a plurality of hierarchical structures of object categories, one object is mapped to one hierarchical structure of object categories, and the object can be labelled to either a more generic or more specific object category in a corresponding hierarchical structure based on input to a user interface.

8. The method of claim 5, wherein an entry to map an object of the one or more objects is to a corresponding object category is generated through a generative machine learning model.

9. The method of any of claims 1 to 8, wherein whether an object overlapping with boundaries of the ROI is to be replaced is set based on input to a user interface.

10. The method of any of claims 1 to 9, wherein corresponding object categories of the ROI are provided to the application with source information of the corresponding object categories being masked.

11. The method of any of claims 1 to 10, wherein through a generative machine learning model, the application provides the generic media content corresponding to the object categories.

12. The method of claim 11, wherein the generative machine learning model is executed upon a determination that the media content corresponding to the object categories is not stored in an existing media content a repository of generic media content.

13. An electronic device (602) to replace media content in a media stream, comprising: a processor (642) and non-transitory machine-readable storage medium (649) that provides instructions that, when executed by the processor (642), are capable of causing the processor to perform: detecting (502) one or more objects within a region of interest (ROI) of a scene to be edited in a media stream with corresponding categories of the ROI; labeling (504) the one or more objects detected within the ROI with one or more object categories of the ROI; providing (506) the one or more object categories of the ROI to an application through a secure channel, the application to obtain generic media content corresponding to the one or more object categories, wherein the generic media content exemplifies the one or more object categories but with no information identifiable with the media stream;receiving (508) the generic media content corresponding to the one or more object categories; and replacing (510) media content corresponding to the one or more objects within the ROI by the generic media content received from the application.

14. The electronic device of claim 13, wherein a plurality of object categories is determined to be associated with an object of the one or more objects, and one of the plurality of object categories is selected for the object to be provided to the application based on input to a user interface.

15. The electronic device of claim 13 or 14, wherein the ROI to be edited is selected based on input to a user interface.

16. The electronic device of any of claims 13 to 15, wherein a set of objects within the ROI is selected to be a safelist and not labeled and replaced.

17. The electronic device of any of claims 13 to 16, wherein data of the ROI is stored in a container based on coordinates of the ROI, wherein the one or more objects are detected within the container, and the one or more objects are labelled with object categories by looking up a mapping data structure to identify object categories of the objects, the mapping data structure including entries to map objects and object categories.

18. The electronic device of claim 17, wherein boundaries of the ROI are refined prior to detecting the one or more objects, wherein the boundaries are refined by adding one or more offsets of the coordinates of the ROI.

19. The electronic device of claim 17, wherein the mapping data structure maintains a plurality of hierarchical structures of object categories, one object is mapped to one hierarchical structure of object categories, and the object can be labelled to either a more generic or more specific object category in a corresponding hierarchical structure based on input to a user interface.

20. The electronic device of claim 17, wherein an entry to map an object of the one or more objects is to a corresponding object category is generated through a generative machine learning model.

21. The electronic device of any of claims 13 to 20, wherein whether an object overlapping with boundaries of the ROI is to be replaced is set based on input to a user interface.

22. The electronic device of any of claims 13 to 21, wherein corresponding object categories of the ROI are provided to the application with source information of the corresponding object categories being masked.

23. The electronic device of any of claims 13 to 22, wherein through a generative machine learning model, the application provides the generic media content corresponding to the object categories.

24. The electronic device of claim 23, wherein the generative machine learning model is executed upon a determination that the media content corresponding to the object categories is not stored in an existing media content a repository of generic media content.

25. A non-transitory machine-readable storage medium (649) that provides instructions that, when executed by a processor (642), are capable of causing the processor to perform any of methods 1 to 12.

Images