Providing limitations in computer-generated reality recordings

Abstract

Implementations of the subject technology provide for analysis of a recording of content within a field of view of a device, the analysis including identification of a set of objects included in the content. The subject technology identifies a subset of the set of objects that are indicated as corresponding to protected content. The subject technology generates a modified version of the recording that obfuscates or filters the subset of the set of objects. Additionally, the subject technology provides the modified version of the recording to a host application program for playback.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 897,907, filed September 9, 2019, entitled “Providing Restrictions in Computer-Generated Reality Recordings,” the entire disclosure of which is incorporated herein by reference. Technical Field

[0003] This instruction manual deals with computer-generated real-world environments. Background Technology

[0004] Augmented reality (AR) technology aims to bridge the gap between virtual and physical environments by providing an augmented physical environment that utilizes electronic information. Therefore, the electronic information appears to be part of the user's perceived physical environment. In this example, AR further provides a user interface for interacting with the electronic information overlaid on the augmented physical environment. Attached Figure Description

[0005] Some features of this subject matter are shown in the appended claims. However, for illustrative purposes, several embodiments of this subject matter are illustrated in the following figures.

[0006] Figure 1 An exemplary system architecture is shown, comprising various electronic devices that can implement the system of this subject, according to one or more specific implementations.

[0007] Figure 2 An exemplary software architecture that can be implemented on an electronic device and / or server according to one or more specific implementations of the subject matter is shown.

[0008] Figure 3A and Figure 3B An example is shown of analyzing computer-generated reality records according to a specific implementation of the techniques in this subject matter and generating restricted computer-generated reality records based on user profiles containing privacy information.

[0009] Figure 4 An example is shown of analyzing computer-generated reality records according to a specific implementation of the subject matter technology and generating restricted computer-generated reality records based on user profiles with limiting information.

[0010] Figure 5 A flowchart is shown as an exemplary process for analyzing computer-generated records of real-world content, according to a specific implementation of the art of this subject.

[0011] Figure 6An electronic system is shown that can be used to implement one or more specific embodiments of the subject matter technology. Detailed Implementation

[0012] The specific embodiments shown below are intended to describe various configurations of the subject matter and are not intended to represent the only configuration in which the subject matter can be practiced. The accompanying drawings are incorporated herein and form part of the specific embodiments. The specific embodiments include particular details intended to provide a thorough understanding of the subject matter. However, the subject matter is not limited to the specific details set forth herein, but may be practiced in one or more other specific embodiments. In one or more specific embodiments, structures and components are shown in block diagram form to avoid obscuring the concepts of the subject matter.

[0013] Computer-Generated Reality (CGR) systems enable the combination of physical and virtual environments to varying degrees, thereby facilitating real-time user interaction. Therefore, as described herein, such CGR systems can include a variety of possible combinations of physical and virtual environments, including augmented reality that primarily comprises physical elements and is closer to the physical environment than a virtual environment (e.g., without physical elements). In this way, the physical environment can be connected to the virtual environment via the CGR system. Users immersed in a CGR environment can navigate within it, and the CGR system can track the user's viewpoint to provide visualizations based on the user's location within the environment.

[0014] The physical environment refers to the physical world that people can sense and / or interact with without the aid of electronic systems. Physical environments, such as physical parks, include physical objects such as physical trees, physical buildings, and physical people. People can directly sense and / or interact with the physical environment through senses such as sight, touch, hearing, taste, and smell.

[0015] Conversely, a computer-generated reality (CGR) environment refers to a fully or partially simulated environment that people sense and / or interact with via electronic systems. In CGR, a subset of a person's physical motion, or a representation thereof, is tracked, and in response, one or more characteristics of one or more virtual objects simulated in the CGR environment are adjusted in a manner consistent with at least one physical law. For example, a CGR system may detect a person's body and / or head rotations and, in response, adjust the graphical content and sound field presented to the person in a manner similar to how such field of vision and sound change in a physical environment. In some cases (e.g., for accessibility reasons), the adjustment of characteristics of virtual objects in a CGR environment may be done in response to a representation of physical motion (e.g., a voice command).

[0016] Humans can use any of their senses to sense and / or interact with CGR objects, including sight, hearing, touch, taste, and smell. For example, a person can sense and / or interact with audio objects that create a 3D or spatial audio environment that provides the perception of a point audio source in 3D space. As another example, audio objects can enable audio transparency, which selectively introduces ambient sound from the physical environment, with or without computer-generated audio. In some CGR environments, a person can sense and / or interact only with audio objects.

[0017] Examples of CGR include virtual reality and mixed reality.

[0018] A virtual reality (VR) environment is a simulated environment designed to provide one or more senses entirely based on computer-generated sensory input. A VR environment includes multiple virtual objects that a person can sense and / or interact with. Examples of virtual objects include trees, buildings, and computer-generated images representing human avatars. A person can sense and / or interact with virtual objects in a VR environment through the simulation of their presence within the computer-generated environment and / or through the simulation of a subgroup of physical movements of a person within the computer-generated environment.

[0019] Compared to VR environments, which are designed to be entirely based on computer-generated sensory input, mixed reality (MR) environments are simulated environments designed to incorporate sensory input from the physical environment, or representations thereof, in addition to computer-generated sensory input (e.g., virtual objects). On the virtual continuum, a mixed reality environment is any state between a purely physical environment as one end and a virtual reality environment as the other end, but not including either end.

[0020] In some MR environments, computer-generated sensory input can respond to changes in sensory input from the physical environment. Additionally, some electronic systems used to present the MR environment can track position and / or orientation relative to the physical environment, enabling virtual objects to interact with real objects (i.e., physical objects or representations of them from the physical environment). For example, the system can cause movement so that virtual trees appear stationary relative to the physical ground.

[0021] Augmented reality (AR) environments are simulated environments in which one or more virtual objects are overlaid on a physical environment or a representation thereof. For example, an electronic system for presenting an AR environment may have a transparent or semi-transparent display through which a person can directly view the physical environment. The system can be configured to present virtual objects on the transparent or semi-transparent display, allowing a person to perceive virtual objects overlaid on a portion of the physical environment. Alternatively, the system may have an opaque display and one or more imaging sensors that capture images or videos of the physical environment, which are representations of the physical environment. The system combines the images or videos with virtual objects and presents the combination on the opaque display. A person uses the system to indirectly view the physical environment via images or videos of the physical environment and perceives virtual objects overlaid on and / or behind a portion of the physical environment. As used herein, video of the physical environment displayed on an opaque display is referred to as “pass-through video,” meaning that the system uses one or more image sensors to capture images of the physical environment and uses those images when presenting the AR environment on the opaque display. Alternatively, the system may have a projection system that projects virtual objects onto a physical environment, such as as a hologram or on a physical surface, so that a person can use the system to perceive the virtual objects superimposed on the physical environment.

[0022] Augmented reality environments also refer to simulated environments where the representation of the physical environment is transformed by computer-generated sensory information. For example, in providing pass-through video, a system can transform images from one or more sensors to apply a selected viewpoint (e.g., viewpoint) different from the viewpoint captured by the imaging sensor. Alternatively, the representation of the physical environment can be transformed by graphically modifying (e.g., magnifying) portions of it, such that the modified portion is a representative but not realistic version of the original captured image. Furthermore, the representation of the physical environment can be transformed by graphically removing or blurring portions of it.

[0023] Augmented virtual (AV) environments are simulated environments that combine virtual or computer-generated environments with one or more sensory inputs from a physical environment. Sensory inputs can be representations of one or more characteristics of the physical environment. For example, an AV park could have virtual trees and buildings, but a person's face could be realistically reproduced from an image taken of a physical person. Similarly, virtual objects could adopt the shape or color of a physical object imaged by one or more imaging sensors. Furthermore, virtual objects could use shadows that correspond to the sun's position within the physical environment.

[0024] Many different types of electronic systems enable people to sense and / or interact with a variety of CGR environments. Examples include mobile devices, tablet devices, projection-based systems, head-up displays (HUDs), head-mounted systems, vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays shaped like lenses designed to be placed on the human eye (e.g., similar to contact lenses), head-mounted receivers / headsets, speaker arrays, input systems (e.g., wearable or handheld controllers with or without haptic feedback), smartphones, tablets or tablet devices, and desktop / laptop computers. For example, a head-mounted system may have one or more speakers and an integrated opaque display. Alternatively, a head-mounted system may be configured to receive an external opaque display (e.g., a smartphone). A head-mounted system may incorporate one or more imaging sensors for capturing images or video of the physical environment, and / or one or more microphones for capturing audio of the physical environment. A head-mounted system may have a transparent or semi-transparent display instead of an opaque display. The transparent or semi-transparent display may have a medium through which light representing the image is directed to the human eye. The display can utilize digital light projection, OLED, LED, uLED, liquid crystal on silicon, laser scanning light source, or any combination of these technologies. The medium can be an optical waveguide, holographic medium, optical combiner, optical reflector, or any combination thereof. In one embodiment, a transparent or translucent display can be configured to selectively become opaque. Projection-based systems can employ retinal projection technology, which projects graphic images onto the human retina. Projection systems can also be configured to project virtual objects onto a physical environment, such as as holograms or on a physical surface.

[0025] The specific implementation of the subject matter technology described herein provides a CGR system that provides restrictions on content in a given CGR record based at least in part on identifying objects corresponding to protected content and / or user profile information including information indicating privacy preferences. In this manner, the subject matter system enables the segmentation of specific parts of a scene within the restricted CGR record. Such parts include physical and / or virtual objects corresponding to restricted parts in the scene. The subject matter system can modify the CGR record to filter or mask restricted parts and provides a CGR record for playback on a given host application.

[0026] Figure 1An exemplary system architecture 100 comprising various electronic devices capable of implementing the system of this subject matter, according to one or more embodiments, is shown. However, not all depicted components are usable in all embodiments, and one or more embodiments may include additional or different components compared to those shown in the figures. Variations in the arrangement and type of these components may be made without departing from the spirit or scope of the claims set forth herein. Additional components, different components, or fewer components may be provided.

[0027] System architecture 100 includes electronic device 105, handheld electronic device 104, electronic device 110, electronic device 115, and server 120. For illustrative purposes, system architecture 100 is described in... Figure 1 The system is shown as including electronic device 105, handheld electronic device 104, electronic device 110, electronic device 115, and server 120; however, system architecture 100 may include any number of electronic devices and any number of servers or a data center including multiple servers.

[0028] Electronic device 105 may be implemented, for example, as a tablet device, a handheld and / or mobile device, or (e.g., a head-mounted portable system worn by user 101). Electronic device 105 includes a display system capable of presenting a visualization of a computer-generated realistic environment to the user. Electronic device 105 may be powered by a battery and / or another power source. In the example, the display system of electronic device 105 provides the user with a stereoscopic representation of the computer-generated realistic environment, enabling the provision of a three-dimensional visual display rendering a specific scene. In one or more embodiments, instead of utilizing electronic device 105 to access the computer-generated realistic environment, or otherwise, the user may use a handheld electronic device 104, such as a tablet, watch, mobile device, etc.

[0029] Electronic device 105 may include one or more cameras, such as camera 150 (e.g., visible light camera, infrared camera, etc.). Furthermore, electronic device 105 may include various sensors 152, including but not limited to cameras, image sensors, touch sensors, microphones, inertial measurement units (IMUs), heart rate sensors, temperature sensors, depth sensors (e.g., lidar sensors, radar sensors, sonar sensors, time-of-flight sensors, etc.), GPS sensors, Wi-Fi sensors, near-field communication sensors, radio frequency sensors, etc. Additionally, electronic device 105 may include hardware elements capable of receiving user input, such as hardware buttons or switches. User input detected by such sensors and / or hardware elements corresponds to various input modalities, such as those used to initiate coexistence sessions within an application. For example, such input modalities may include, but are not limited to, face tracking, eye tracking (e.g., gaze direction), hand tracking, posture tracking, biometric readings (e.g., heart rate, pulse, pupil dilation, respiration, temperature, electroencephalogram, olfaction), speech or audio recognition (e.g., specific hot words), and activation of buttons or switches, etc.

[0030] In one or more embodiments, electronic device 105 may be communicatively coupled to infrastructure devices, such as electronic device 110 and / or electronic device 115. Generally, such infrastructure devices may include more computing resources and / or available power compared to electronic device 105. In one example, electronic device 105 may operate in various modes. For example, electronic device 105 may operate in an independent mode, independent of any infrastructure device. When electronic device 105 operates in independent mode, the number of input modes may be constrained by the power and / or processing limitations of electronic device 105, such as the device's available battery power. In response to power limitations, electronic device 105 may deactivate certain sensors within the device itself to conserve battery power and / or free up processing resources.

[0031] Electronic device 105 can also operate in a wireless connection mode (e.g., connected to a base device via a wireless connection), thereby working in conjunction with a given base device. Electronic device 105 can also operate in a connection mode where electronic device 105 is physically connected to a base device (e.g., via a cable or some other physical connector), and can utilize power resources provided by the base device (e.g., when the base device is charging electronic device 105 while physically connected).

[0032] When electronic device 105 operates in wireless or connected mode, at least a portion of processing user input and / or rendering a computer-generated real-world environment can be offloaded to the underlying device, thereby reducing the processing burden on electronic device 105. For example, in a specific implementation, electronic device 105 works in conjunction with electronic device 110 or 115 to generate a computer-generated real-world environment, which includes physical and / or virtual objects that enable various forms of interaction (e.g., visual, auditory, and / or physical or tactile interaction) between the user and the generated computer-generated real-world environment in real time. In the example, electronic device 105 provides a rendering of a scene corresponding to the computer-generated real-world environment, which can be perceived and interacted with by the user in real time, such as a host environment for coexisting with another user. Additionally, as part of presenting the rendered scene, electronic device 105 may provide the user with auditory and / or tactile or haptic feedback. The content of a given rendered scene may depend on available processing power, network availability and capacity, available battery power, and the current system workload.

[0033] Electronic device 105 can also detect events that have occurred within a scene of a computer-generated real-world environment. Examples of such events include detecting the presence of a specific person, entity, or object in the scene. In response to a detected event, electronic device 105 can provide annotations (e.g., in the form of metadata) in the computer-generated real-world environment corresponding to the detected event.

[0034] In one embodiment, electronic device 105 generates a record of the physical environment and generates a computer-generated realistic environment including two-dimensional (2D) video for sharing and playback. In another embodiment, electronic device 105 generates a record and subsequently generates a three-dimensional (3D) representation that incorporates information from all sensors and / or combines the record with records from other users (e.g., different point-of-view (POV)). It should also be understood that electronic devices 110 and / or 115 may also generate such computer-generated realistic environments, either in conjunction with or independently of electronic device 105.

[0035] Network 106 may communicatively (directly or indirectly) couple electronic devices 104, 105, 110, and / or 115 to each other device and / or server 120. In one or more embodiments, network 106 may be an interconnection network of devices that may include the Internet or be communicatively coupled to the Internet.

[0036] Electronic device 110 may include a touchscreen and may be, for example, a smartphone including a touchscreen, a portable computing device such as a laptop computer including a touchscreen, an accessory including a touchscreen (e.g., a digital camera, headphones), a tablet device including a touchscreen, a wearable device including a touchscreen (e.g., a watch, wristband, etc.), any other suitable device including, for example, a touchscreen, or any electronic device with a touchpad. In one or more embodiments, electronic device 110 may not include a touchscreen but may support touchscreen-like gestures, such as in a computer-generated real-world environment. In one or more embodiments, electronic device 110 may include a touchpad. Figure 1 In this embodiment, by way of example, electronic device 110 is depicted as a mobile smartphone device with a touchscreen. In one or more embodiments, electronic device 110, handheld electronic device 104, and / or electronic device 105 may be and / or may include all or part of the electronic device described below with respect to an electronic system, which is described below with respect to an electronic system. Figure 6 As described above. In one or more embodiments, electronic device 110 may be another device, such as an Internet Protocol (IP) camera, a tablet computer, or an accessory such as an electronic stylus.

[0037] Electronic device 115 may be, for example, a desktop computer, portable computing devices such as laptops, smartphones, accessories (e.g., digital cameras, headphones), tablet devices, wearable devices such as watches, wristbands, etc. Figure 1 In this context, by way of example, electronic device 115 is described as a desktop computer. Electronic device 115 may be and / or may include, as described below in relation to... Figure 6 The electronic system under discussion, in whole or in part.

[0038] Server 120 may form all or part of a computer network or server group 130, such as in a cloud computing or data center implementation. For example, server 120 stores data and software and includes specific hardware (e.g., processors, graphics processors, and other dedicated or custom processors) for rendering and generating computer-generated reality content such as graphics, images, videos, audio, and multimedia files. In a specific implementation, server 120 may function as a cloud storage server that stores any of the aforementioned computer-generated reality content generated by the aforementioned devices and / or server 120.

[0039] In specific implementation, server 120 can provide a security environment corresponding to the secure cloud environment for analyzing CGR content records, as discussed further below. Such records can be provided by at least one of the following: electronic device 105, electronic device 110, and / or electronic device 115. Similarly, electronic device 105, electronic device 110, and / or electronic device can also provide the following described below. Figure 2 The security environment used for analyzing CGR content records is discussed further in this paper.

[0040] Figure 2 An exemplary software architecture 200, which may be implemented on electronic device 105, electronic device 115, and server 120 according to one or more specific embodiments of the subject matter, is illustrated. For illustrative purposes, the software architecture 200 is described as being composed of... Figure 1 The software architecture 200 is implemented by electronic devices 105, 115, and 120, such as through the processors and / or memory of electronic devices 105, 115, and 120; however, suitable portions of the software architecture 200 may be implemented by any other electronic device, including electronic device 110. However, not all depicted components are usable in all embodiments, and one or more embodiments may include additional or different components compared to those shown in the figures. Variations in the arrangement and type of these components may be made without departing from the spirit or scope of the claims set forth herein. Additional components, different components, or fewer components may be provided.

[0041] Software architecture 200 implemented on electronic device 105 includes a framework. As used herein, a framework can refer to a software environment that provides specific functionality as part of a larger software platform to facilitate the development of software applications, and can provide one or more application programming interfaces (APIs) that developers can use to programmatically design computer-generated real-world environments and handle operations for such computer-generated real-world environments.

[0042] As shown in the figure, a recording frame 230 is provided. The recording frame 230 provides functionality for recording a computer-generated realistic environment provided by the input modalities as described above. An event detector 220 is provided, which receives information corresponding to inputs from various input modalities. A system manager 210 is provided to monitor resources from the electronic device 105 and determine quality of service metrics based on available resources. The system manager 210 can make decisions to select specific hardware components corresponding to a given input modality to activate and / or deactivate based on the quality of service metrics. For example, a camera used for tracking facial expressions may be turned off, or another camera used for tracking gestures may be turned off. When specific hardware is deactivated, the electronic device 105 can provide a notification to warn the user that the specific input modality is unavailable. Similarly, the electronic device 105 can provide a notification to warn the user that a specific input modality is available when the specific hardware is activated.

[0043] In practical implementation, software architecture 200 enables the management of CGR records by providing a repository or storage location for uploading such records. For example... Figure 2As shown, the electronic device 105 includes a database 240 for storing CGR records 242, which may be provided by a record frame 230 and / or a secure record analyzer 260, as described further below. In this example, the record frame 230 may generate CGR records and encrypt such records for storage in the database 240.

[0044] The specific implementation of software architecture 200 and its components enables automatic segmentation of CGR records based on content and annotations, which also enable record searching. Furthermore, software architecture 200 and its components enable recommendations for other records, at least in part, based on topics and / or user interests, which can be provided from user preferences stored in associated user profiles. Software architecture 200 and its components enable the generation of multiple records based on recommendations or user preferences, and also enable random browsing of records (pause, playback, drag, etc.).

[0045] In the example, security record analyzer 260 may receive a given CGR record from record frame 230, or receive a request from record frame 230 to retrieve a CGR record stored in database 240. The security record analyzer 260 analyzes the record and generates metadata and / or annotations based on the analysis. In a specific implementation, security record analyzer 260 may execute a specific classifier for object detection and determine the motion vector of the object if such an object moved during recording. When an event is detected or a person is detected and / or identified, security record analyzer 260 provides the record with annotations (e.g., tagged with metadata information) for the object. Furthermore, the analysis can be customized based on the record type (e.g., movement, specific location such as a beach, etc.) to target specific events or objects that may exist in records of that type.

[0046] In practice, the security record analyzer 260 analyzes the record to identify protected items for determining restrictions on CGR content. Alternatively, such protected items can be pre-marked as protected based on whether a user viewing the CGR record has sufficient access to a given item (e.g., paid content, age-restricted content, violent content, etc.), or whether a person detected in the CGR record has an enabled privacy flag. The security record analyzer 260 can return information based on the analysis of the record frame 230 (or alternatively, an editing tool) to facilitate the generation of modified CGR records. In the example, using this information, the record frame 230 determines which virtual items are not included in the scene and / or which physical objects will be obfuscated and / or excluded from the scene in order to generate modified CGR records. As used herein, physical objects are representations of objects captured by cameras within the scene.

[0047] Furthermore, in practical implementation, different user profiles can be provided based on access levels. For example, users can have various profiles for different access levels, including public profiles, private profiles, profiles for friends / family, work profiles, etc. Using profile information, user avatars can be rendered based on user profiles suitable for the scenario and based on which other users are viewing the CGR record. Additionally, during playback, the CGR record can be filtered or adjusted (e.g., slowed down or lowered in pitch) based on the pulse or other inputs of the user viewing the CGR record.

[0048] In practical implementation, people and / or objects in a given CGR record can be segmented based on the operating mode of the electronic device 105 (e.g., plugged in, wireless, standalone). Therefore, when the original record is limited during recording due to hardware or resource constraints, post-processing (e.g., analyzing the CGR record as described above) can be performed, and this can also occur continuously in the cloud (e.g., utilizing deeper neural networks and / or other machine learning algorithms). In particular, as user profiles change, the aforementioned cloud algorithms can continue to learn user preferences. In another example, when a user is moving, the techniques of this subject enable the rendering of a given CGR record to be adjusted to further adapt to domain permissions that may have changed due to different geographical locations.

[0049] As also shown in the figure, server 120 provides cloud environment 270. Cloud environment 270 can use its own set of processors (e.g., CPU and / or GPU). In a specific implementation, server 120 includes a secure cloud record analyzer 280 that performs the same or similar functions as the secure record analyzer 260 discussed above regarding a given CGR record. In the example, server 120 may receive a request from electronics 105 to perform analysis on a given CGR record. The results of any analysis performed by secure record analyzer 260 (e.g., generated metadata information or recommendations) may be returned to electronics 105 for further processing.

[0050] In a specific implementation, electronic device 115 includes a security record analyzer 265, which can perform the same functions as described above for security record analyzer 260 in conjunction with electronic device 105. For example, security record analyzer 260 can analyze a CGR record from one of the CGR records 247 stored in database 246.

[0051] As shown in the figure, the electronic device 115 includes a set of tools 245, corresponding to various editing tools (e.g., specific applications) that provide editing functions to modify CGR records. Each editing tool is capable of receiving a CGR record (e.g., corresponding to a CGR record 247 stored in a database 246) for editing. Examples of editing tools include image editors, tools for 3D composition, and / or multimedia editors capable of organizing CGR content along a timeline. A given editing tool can also enable the insertion of new objects or content into the scene, including new virtual content.

[0052] Despite Figure 2 Although not shown, it should be understood that electronic device 110 may also include similar components as discussed above in conjunction with electronic device 115.

[0053] In practice, the algorithms discussed herein are embedded in silicon (e.g., face detection); additionally, there are software-based algorithms. Furthermore, specific implementations of the subject matter (including any of the aforementioned electronic devices) can utilize dedicated hardware (e.g., encoding hardware, GPUs, specific processors for accelerating neural networks and machine learning operations, etc.).

[0054] The following discussion involves analyzing various examples of computer-generated real-world recordings to determine the limitations of generating modified records for playback.

[0055] The following section discusses examples of analyzing computer-generated reality recordings and generating restricted computer-generated reality recordings based on specific implementations of the techniques in this subject matter. The following discussion involves... Figure 2 The software architecture 200 components.

[0056] In the example, the scene depicts a meeting taking place in a conference room, which is included in a computer-generated reality (CGR) record. In this scene, a person is giving a presentation in an area corresponding to a whiteboard in the conference room. In this example, the presentation in the area could be virtual content rendered as a visual presentation on the whiteboard in the CGR record.

[0057] In this example, a security record analyzer (e.g., security record analyzer 260 or 265, or security cloud record analyzer 280) analyzes the scenario and determines that the virtual content on the whiteboard is protected content. Based on the user's current access level, it determines that the user who has requested to view the CGR record does not have sufficient access permissions to view the content on the whiteboard. The security record analyzer can provide analysis-related information as metadata to the record frame 230 or editing tools for further processing. Using the provided information, the record frame 230 or editing tools can generate a modified version of the CGR record.

[0058] In the example, a second scene, corresponding to a modified version of the scene and part of the modified CGR record, may have been generated, at least in part, based on information generated from the aforementioned analysis of the CGR record (e.g., via a specific editing tool or host application or record frame 230). In the second scene 320, virtual content on a whiteboard has been rendered to obscure the protected content previously shown in the scene. In this way, the protected content is not displayed to the user when viewed by a user without sufficient access privileges.

[0059] The following discussion relates to an example of analyzing computer-generated reality recordings and generating restricted computer-generated reality recordings based on specific implementations of the techniques in this subject matter. Figure 2 The software architecture 200 components.

[0060] In the example, the scene depicts a specific indoor environment (e.g., a concert featuring a rock band) included in a computer-generated reality (CGR) recording. In this scene, the rock band is performing in an area corresponding to the space above the stage. In the example, this area can be considered protected content, requiring specific access permissions (e.g., users must pay a fee to view the area) to watch the rock band.

[0061] In this example, a security record analyzer (e.g., security record analyzer 260 or 265, or security cloud record analyzer 280) analyzes the scenario and determines that the content in that area is protected, and determines, based on the user's current access level, that the user who has requested to view the CGR record does not have sufficient access permissions to view the content in that area. In one or more implementations, the determination of protected content in the scenario is based on detecting tags, performing object identification, and / or detecting protected objects (which may be marked as protected). The security record analyzer may provide analysis-related information in the form of metadata to the record frame 230 or editing tools for further processing. Using the provided information, the record frame 230 or editing tools may generate a modified version of the CGR record.

[0062] Using the associated metadata information, the record frame 230 or editing tools can generate a modified version of the CGR record. In the example, such metadata information may include information indicating the location of objects in the scene, information about the object type (e.g., classification), information about the object size, and / or any other appropriate information that helps identify or locate objects in the scene, including object descriptors that the record frame 230 uses to distinguish objects. Figure 4In B, a second scene corresponding to a modified version of the scene may have been generated, at least in part, based on information generated from the aforementioned analysis of the CGR record (e.g., via a specific editing tool, host application, or recording frame 230). In the second scene, virtual content 422 has been rendered to obfuscate the protected content (e.g., a rock band) previously shown in the scene. In this way, the protected content is not displayed to the user when viewed by a user without sufficient access rights. In this example, the user can interact with the system and pay the necessary fee to watch the rock band. Once the fee payment transaction is verified, the rendering of the computer-generated realistic environment during playback resembles this scene, thus enabling the user to watch the rock band performing. In specific implementations, the aforementioned restrictions (e.g., obfuscation or removal of content) are applied in a reversible manner so that the restrictions can be removed in the future.

[0063] Figure 3A and Figure 3B An example is shown illustrating the analysis of computer-generated reality records according to a specific implementation of the techniques described in this subject matter, and the generation of restricted computer-generated reality records based on user profiles containing privacy information. The following discussion relates to... Figure 2 The software architecture 200 components.

[0064] exist Figure 3A In the example, a security record analyzer (e.g., security record analyzer 260 or 265, or secure cloud record analyzer 280) analyzes scenario 310 and determines privacy attributes related to the people identified in scenario 310. In scenario 310, a person 312 corresponding to an adult male, a person 314 corresponding to an adult female, and a person 316 corresponding to a male child exist and have been successfully identified. In the example, the security record analyzer is able to determine the respective identities of the three people in scenario 310 based at least in part on the corresponding associated devices that each person may carry and / or use.

[0065] When each of the identified individuals is included in a specific scene for viewing by different users, that individual may have an associated user profile for display with a specific avatar. For example, a person may have a profile for contacts, a profile for work, a profile for friends and family, a public profile, a personal profile, etc., whose type of avatar used to render in the scene is indicated relative to the viewing user of the scene. Alternatively, if the identification of one or more individuals in a given scene is unsuccessful, the scene may be rendered to obfuscate faces (e.g., by blurring a person's face) and indicate that the person is unknown or unrecognized.

[0066] Each of the users corresponding to persons 312, 314, and 316 can also set a privacy filter in their user profile information. When such a privacy filter is set, the user's representation rendered in the scene can be anonymized, making the user difficult to identify visually. In the example, when the privacy filter is set in the user profile, the rendering of the user avatar as described above (e.g., based on the profile type of the user being viewed) is not applicable.

[0067] The security record analyzer can provide analysis-related information as metadata to the record frame 230 or editing tools for further processing. Using the provided information, the record frame 230 or editing tools can generate a modified version of the CGR record.

[0068] Using the associated metadata information, record frame 230 or editing tools can generate a modified version of the CGR record. Figure 3B In this process, scene 320, corresponding to a modified version of scene 310, may have been generated (e.g., through a specific editing tool or host application or record frame 230) based at least in part on information generated from the aforementioned analysis of the CGR record. In scene 320, avatars 322, 324, and 326 have been rendered for display within scene 320. Based on associated user profile information corresponding to persons 312, 314, and 316 previously identified in scene 310, Figure 3B The avatar in the example can be selected for display to users viewing scene 320. In this way, the appropriate avatar can be included in the rendering of the modified CGR record, at least in part, based on user profile information and / or privacy settings.

[0069] Figure 4 An example is shown illustrating the analysis of computer-generated reality recordings according to a specific implementation of the techniques described in this subject matter, and the generation of restricted computer-generated reality recordings based on user profiles with limiting information. The following discussion relates to... Figure 2 The software architecture 200 components.

[0070] In the example, a security record analyzer (e.g., security record analyzer 260 or 265, or security cloud record analyzer 280) analyzes the scenario and determines that the content in that area is restricted (e.g., based on the age of the viewing user), and determines, based on the user's profile information, that the user who has requested to view the CGR record does not have sufficient access permissions to view the content in that area. In the example, age restrictions may be determined based on a label indicating the minimum age for viewing the content. The security record analyzer may provide analysis-related information in the form of metadata to the record frame 230 or an editing tool for further processing. Using the provided information, the record frame 230 or the editing tool may generate a modified version of the CGR record.

[0071] Using the associated metadata information, record frame 230 or editing tools can generate a modified version of the CGR record. Figure 4 Scene 420, corresponding to a modified version of another scene, may have been generated, at least in part, based on information generated from the aforementioned analysis of the CGR record (e.g., via a specific editing tool or host application or recording frame 230). In scene 420, virtual content 422 has been rendered to obscure previously displayed restricted content (e.g., two men fighting). In this way, restricted content is not displayed to the user when viewed by a user without sufficient access rights. In this example, the user is not shown restricted content corresponding to the violent event because the user's profile information indicates that the user is not of sufficient age to view such violent content. This restriction can be set by the system or by another user with security permissions for that user's account (such as the user's parent or guardian).

[0072] Figure 5 A flowchart is shown of an exemplary process 500 for analyzing computer-generated real-world content records according to a specific implementation of the subject matter. For illustrative purposes, this document primarily refers to... Figure 1 and Figure 2 The process 500 is described using electronic device 105. However, process 500 is not limited to... Figure 1 and Figure 2 The electronic device 105, and one or more blocks (or operations) of process 500 may be performed by one or more other components of other suitable devices (including electronic device 105, electronic device 110, and / or server 120). Further for illustrative purposes, the blocks of process 500 are described herein as occurring sequentially or linearly. However, multiple blocks of process 500 may occur in parallel. Furthermore, the blocks of process 500 need not be performed in the order shown, and / or one or more blocks of process 500 need not be performed and / or may be replaced by other operations.

[0073] like Figure 5 As shown, electronic device 105 analyzes a recording of content within the device's field of view, the analysis including identifying a set of objects included in the content (510). In this example, the field of view corresponds to the extent of the observable world visible to the user in a CGR environment. The field of view can be measured in degrees, and the higher the device's field of view (e.g., the greater the degree), the farther the virtual world will extend to the edge of the user's field of vision. Electronic device 105 identifies a subset of objects indicated as corresponding to the protected content (512). Electronic device 105 generates a modified version of the recording that obfuscates or filters the subset of objects (514). Electronic device 105 provides the modified version of the recording to a host application for playback (516).

[0074] As described above, one aspect of this technology involves collecting and using data from various sources. This disclosure anticipates that, in some instances, such collected data may include personal information that uniquely identifies or can be used to contact or locate specific individuals. This personal information may include demographic data, location-based data, telephone numbers, email addresses, social network identifiers, home addresses, data or records related to a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying or personal information.

[0075] This disclosure recognizes that the use of such personal information data in the techniques of this invention can be used to benefit users. This disclosure also contemplates uses of personal information data that are beneficial to users. For example, health and fitness data can be used to provide insights into a user's overall health status or as positive feedback for individuals using the technology to pursue health goals.

[0076] This disclosure assumes that entities responsible for collecting, analyzing, disclosing, transmitting, storing, or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and practices that are generally recognized as meeting or exceeding industry or governmental requirements for maintaining the privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as data collection and / or use change. Personal information from users should be collected for the entity's lawful and reasonable purposes and not shared or sold outside of these lawful uses. Furthermore, such collection / sharing should be conducted only after obtaining informed consent from users. In addition, such entities should consider taking any necessary steps to protect and safeguard access to such personal information data and ensure that others with access to such personal information data comply with their privacy policies and processes. Additionally, such entities may be subject to third-party evaluations to demonstrate their compliance with widely accepted privacy policies and practices. Furthermore, policies and practices should be adapted to the specific types of personal information data collected and / or accessed, and to applicable laws and standards, including specific considerations regarding jurisdiction. For example, in the United States, the collection or acquisition of certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while in other countries, health data may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy practices should be maintained for different types of personal data in each country.

[0077] Regardless of the foregoing, this disclosure also anticipates implementation schemes for users to selectively block the use or access to their personal information data. That is, this disclosure anticipates providing hardware and / or software components to prevent or block access to such personal information data. For example, this technology can be configured to allow users to opt-in or opt-out at any time during or after registering for the service. In addition to providing opt-in and opt-out options, this disclosure envisions providing notifications related to access to or use of personal information. For example, users may be notified when downloading an application that their personal information data will be accessed, and then reminded again just before the application accesses the personal information data.

[0078] Furthermore, the purpose of this disclosure is to manage and process personal information data to minimize the risk of unintentional or unauthorized access or use. Once data is no longer needed, this risk can be minimized by limiting data collection and deleting data. Additionally, and where applicable, including in certain health-related applications, data deidentification can be used to protect user privacy. Deidentification can be facilitated, where appropriate, by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than the address level), controlling how data is stored (e.g., aggregating data among users), and / or other methods.

[0079] Therefore, while this disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, it is also contemplated that various embodiments can be implemented without access to such personal information data. That is, various embodiments of the present invention will not be rendered inoperable due to the absence of all or part of such personal information data. For example, content can be selected and delivered to the user based on non-personal information data or a small amount of personal information, such as content requested by a device associated with the user, other non-personal information, or publicly available information.

[0080] Figure 6 An electronic system 600 is shown that can be used to implement one or more specific embodiments of the subject matter technology. The electronic system 600 may be... Figure 1 The illustrated electronic devices 105, 104, 110, 115, and / or server 120 are shown and / or may be part of them. Electronic system 600 may include various types of computer-readable media and interfaces for various other types of computer-readable media. Electronic system 600 includes a bus 608, one or more processing units 612, system memory 604 (and / or cache), ROM 610, persistent storage device 602, input device interface 614, output device interface 606, and one or more network interfaces 616, or subsets and variations thereof.

[0081] Bus 608 generally represents the entire system bus, peripheral bus, and chipset bus that communicatively connect the numerous internal devices of electronic system 600. In one or more embodiments, bus 608 communicatively connects one or more processing units 612 to ROM 610, system memory 604, and permanent storage device 602. The one or more processing units 612 retrieve instructions to be executed and data to be processed from these various memory units in order to perform the processes disclosed in this subject matter. In different embodiments, the one or more processing units 612 may be a single processor or a multi-core processor.

[0082] ROM 610 stores static data and instructions required by one or more processing units 612 and other modules of electronic system 600. On the other hand, persistent storage device 602 can be a read-write memory device. Persistent storage device 602 can be a non-volatile memory cell that stores instructions and data even when electronic system 600 is powered off. In one or more embodiments, mass storage devices (such as magnetic disks or optical disks and their corresponding disk drives) can be used as persistent storage device 602.

[0083] In one or more embodiments, a removable storage device (such as a floppy disk, flash drive, and its corresponding disk drive) may be used as persistent storage device 602. Like persistent storage device 602, system memory 604 may be a read-write memory device. However, unlike persistent storage device 602, system memory 604 may be volatile read-write memory, such as random access memory. System memory 604 may store any instructions and data that one or more processing units 612 may need during operation. In one or more embodiments, the processes disclosed in this subject matter are stored in system memory 604, persistent storage device 602, and / or ROM 610. One or more processing units 612 retrieve instructions to be executed and data to be processed from these various memory units to execute the processes of one or more embodiments.

[0084] Bus 608 is also connected to input device interface 614 and output device interface 606. Input device interface 614 enables a user to transmit information and select commands to electronic system 600. Input devices that can be used with input device interface 614 may include, for example, an alphanumeric keypad and pointing devices (also known as "cursor control devices"). Output device interface 606 may, for example, enable the display of images generated by electronic system 600. Output devices that can be used with output device interface 606 may include, for example, printers and display devices such as liquid crystal displays (LCDs), light-emitting diode (LED) displays, organic light-emitting diode (OLED) displays, flexible displays, flat panel displays, solid-state displays, projectors, or any other device for outputting information. One or more embodiments may include devices that act as both input and output devices, such as touchscreens. In these embodiments, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, voice, or tactile input.

[0085] Finally, as Figure 6 As shown, bus 608 also couples electronic system 600 to one or more networks and / or to one or more network nodes via said one or more network interfaces 616, such as Figure 1 The electronic device 110 shown is an example. In this way, the electronic system 600 can be part of a computer network (such as a LAN, wide area network (“WAN”), or intranet), or a network of networks (such as the Internet). Any or all components of the electronic system 600 can be used with respect to the subject matter disclosed herein.

[0086] These functions can be implemented in computer software, firmware, or hardware. The technology can be implemented using one or more computer program products. Programmable processors and computers can be included in or packaged as mobile devices. The process and logic flow can be executed by one or more programmable processors and one or more programmable logic circuits. General-purpose and special-purpose computing devices, as well as storage devices, can be interconnected via communication networks.

[0087] Some specific implementations include electronic components such as microprocessors, storage devices, and memories that store computer program instructions in machine-readable or computer-readable media (also known as computer-readable storage media, machine-readable media, or machine-readable storage media). Examples of such computer-readable media include RAM, ROM, read-only optical discs (CD-ROM), recordable optical discs (CD-R), rewritable optical discs (CD-RW), read-only digital versatile optical discs (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and / or solid-state hard disk drives, read-only and recordable... Disks, high-density optical discs, any other optical or magnetic media, and floppy disks. Computer-readable media may store computer programs that can be executed by at least one processing unit and include a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as machine code generated by a compiler, and files that include higher-level code that can be executed by a computer, electronic components, or microprocessor using an interpreter.

[0088] While the above discussion primarily concerns microprocessors or multi-core processors that execute software, some specific implementations utilize one or more integrated circuits such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some implementations, such integrated circuits execute instructions stored on the circuit itself.

[0089] As used in this specification and any claim of this patent application, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms exclude persons or groups of persons. For the purposes of this specification, the terms "displayed" or "being displayed" mean displayed on an electronic device. As used in this specification and any claim of this patent application, the terms "computer-readable medium" and "computer-readable media" are entirely limited to tangible, touchable objects that store information in a form readable by a computer. These terms do not include any wireless signals, wired download signals, or any other transient signals.

[0090] To provide interaction with the user, specific embodiments of the subject matter described in this specification can be implemented on a computer having a display device for displaying information to the user and a keyboard and pointing device that the user can use to provide input to the computer. The display device is, for example, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, and the pointing device is, for example, a mouse or trackball. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, voice, or tactile input. Furthermore, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser in response to a request received from a web browser on the user's client device.

[0091] Implementations of the subject matter described in this specification can be implemented in a computing system that includes back-end components, such as a data server, or middleware components, such as an application server, or front-end components, such as a client computer with a graphical user interface or web browser through which a user can interact with a specific implementation of the subject matter described in this specification, or any combination of one or more such back-end components, middleware components, or front-end components. The components of the system can be interconnected via digital data communication of any form or medium, such as a communication network. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), interconnected networks (e.g., the Internet) and peer-to-peer networks (e.g., self-organizing peer-to-peer networks).

[0092] A computing system may include clients and servers. Clients and servers are generally geographically separated but can interact via a communication network. The client-server relationship is established by computer programs running on respective computers and having a client-server relationship with each other. In some specific implementations, the server transmits data (e.g., HTML pages) to the client device (e.g., to display data to a user interacting with that client device and to receive user input from that user). Data generated at the client device (e.g., the result of user interaction) can be received from the client device at the server.

[0093] Those skilled in the art will recognize that the various exemplary blocks, modules, elements, components, methods, and algorithms described herein can be implemented as electronic hardware, computer software, or a combination of both. To illustrate this interchangeability between hardware and software, the various exemplary blocks, modules, elements, components, methods, and algorithms have been generally described above in terms of functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The functionality can be implemented differently for each specific application. Various components and blocks can be arranged differently (e.g., in different orders or divided in different ways) without departing from the scope of the subject matter.

[0094] It should be understood that the specific order or hierarchical structure of the steps in the process disclosed in this invention is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchical structure of the steps in the process can be rearranged. Some steps in the process can be performed simultaneously. The appended method claims present elements of various steps in the sample order and are not intended to limit one to the specific order or hierarchical structure presented.

[0095] The preceding description is provided to enable those skilled in the art to practice the various aspects described herein. The preceding description provides various examples of the subject matter, and the subject matter is not limited to these examples. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, this claim is not intended to be limited to the aspects shown herein, but rather to be consistent with the language of the claim, wherein reference to elements in the singular form is not intended to mean “only one,” but rather “one or more,” unless specifically indicated. Unless otherwise specifically stated, the term “some” means one or more. Male pronouns (e.g., his) include female and neutral (e.g., her and its), and vice versa. Titles and subtitles (if any) are for convenience only and do not limit the invention described herein.

[0096] As used herein, the term "website" can include any aspect of a website, including one or more web pages, one or more servers used to host or store web-related content, etc. Therefore, the term "website" is used interchangeably with the terms "web page" and "server." The predicates "configured to," "capable of operating," and "programmed to" do not imply any specific tangible or intangible modification to a particular subject but are intended to be used interchangeably. For example, a component or a processor configured to monitor and control operations may also mean that the processor is programmed to monitor and control operations or that the processor is capable of operating to monitor and control operations. Similarly, a processor configured to execute code can be interpreted as a processor programmed to execute code or capable of operating to execute code.

[0097] As used herein, the term "automatic" can include execution by a computer or machine without user intervention; for example, by instructions in response to a predicate action of a computer or machine or other initiating mechanism. The word "example" is used herein to mean "serving as an example or illustration." No aspect or design described herein as an "example" is necessarily to be construed as preferred or advantageous over other aspects or designs.

[0098] Phrases such as "aspect" do not imply that this aspect is essential to the subject matter or that this aspect applies to all configurations of the subject matter. Disclosures relating to an aspect may apply to all configurations, or one or more configurations. An aspect may provide one or more examples. Phrases such as "aspect" may refer to one or more aspects, or vice versa. Phrases such as "implementation" do not imply that this implementation is essential to the subject matter or that this implementation applies to all configurations of the subject matter. Disclosures relating to an implementation may apply to all implementations, or one or more implementations. An implementation may provide one or more examples. Phrases such as "implementation" may refer to one or more implementations, or vice versa. Phrases such as "configuration" do not imply that this configuration is essential to the subject matter or that this configuration applies to all configurations of the subject matter. Disclosures relating to a configuration may apply to all configurations or one or more configurations. A configuration may provide one or more examples. Phrases such as "configuration" may refer to one or more configurations, or vice versa.

Claims

1. A method comprising: Retrieve records of the physical environment content corresponding to the device's field of view from the database, the content including composite video, the composite video including at least one virtual object combined with a video comprising at least one physical object; The analysis of the recording of the content within the field of view of the device includes: Identify a set of objects included in the content. Determine the corresponding motion vector for each corresponding object that moved during the recording period, and Generate one or more annotations corresponding to the identified set of objects; Based on one or more generated annotations, identify a subset of the set of objects indicated to correspond to protected content, wherein the subset of the set of objects includes one or more virtual objects and one or more physical objects; Generate a modified version of the record, the modified version obfuscating or filtering the subset of the set of objects; and The modified version of the record is provided to the host application for playback.

2. The method of claim 1, wherein the device comprises a computer-generated reality display device configured to display content to a user.

3. The method of claim 1, wherein identifying the subset of objects indicated to correspond to protected content further comprises: Each object from the subset of the set of objects is determined to include content for which the user needs to pay additional monetary fees to view.

4. The method according to claim 1, further comprising: Identify the area in the scene of the record that corresponds to the protected content.

5. The method of claim 1, wherein identifying the subset of objects indicated to correspond to protected content further comprises: Determine the specific access level corresponding to the user for viewing protected content; as well as It was determined that the specific access level was insufficient to view the subset of objects indicated as corresponding to protected content.

6. The method of claim 5, wherein the specific access level is based on an age restriction for viewing protected content.

7. The method of claim 1, wherein identifying the subset of objects indicated to correspond to protected content further comprises: The representation of a person included in the record is determined to correspond to a specific user, which is associated with an indication of a privacy filter to blur the rendering of the face of the person's representation.

8. The method of claim 1, wherein the modified version of generating the record of the subset of the set of objects that obfuscates or filters further comprises: The record generates a rendering of a scene where the subset of the set of objects is missing.

9. The method of claim 1, wherein the modified version of generating the record of the subset of the set of objects that obfuscates or filters further comprises: The record generates a rendering of a scene that provides a blurred visual representation of the subset of the set of objects.

10. A system comprising: processor; A memory device comprising instructions that, when executed by the processor, cause the processor to perform operations including: Retrieve records of the physical environment content corresponding to the device's field of view from the database, the content including composite video, the composite video including at least one virtual object combined with a video comprising at least one physical object; Analyzing the recording of the content within the field of view of the device, the analysis includes: Identify a set of objects included in the content. Determine the corresponding motion vector for each corresponding object that moved during the recording period, and Generate one or more annotations corresponding to the identified set of objects; Based on one or more generated annotations, identify a subset of the set of objects indicated to correspond to protected content, wherein the subset of the set of objects includes one or more virtual objects and one or more physical objects; Generate a modified version of the record, the modified version obfuscating or filtering the subset of the set of objects; and The modified version of the record is provided to the host application for playback.

11. The system of claim 10, wherein the device includes a computer-generated reality display device configured to display content to a user.

12. The system of claim 10, wherein identifying the subset of objects indicated to correspond to protected content causes the processor to perform additional operations, the additional operations further comprising: Each object from the subset of the set of objects is determined to include content for which the user needs to pay additional monetary fees to view.

13. The system of claim 12, wherein the memory device includes additional instructions that, when executed by the processor, cause the processor to perform additional operations, the additional operations further comprising: Identify the area in the scene of the record that corresponds to the protected content.

14. The system of claim 10, wherein identifying the subset of objects indicated to correspond to protected content causes the processor to perform additional operations, the additional operations further comprising: Determine the specific access level corresponding to the user for viewing protected content; as well as It was determined that the specific access level was insufficient to view the subset of objects indicated as corresponding to protected content.

15. The system of claim 14, wherein the specific access level is based on an age restriction for viewing protected content.

16. The system of claim 10, wherein identifying the subset of objects indicated to correspond to protected content causes the processor to perform additional operations, the additional operations further comprising: The representation of a person included in the record is determined to correspond to a specific user, which is associated with an indication of a privacy filter to blur the rendering of the face of the person's representation.

17. The system of claim 10, wherein generating a modified version of the records that obfuscates or filters the subset of the set of objects causes the processor to perform additional operations, the additional operations further comprising: The record generates a rendering of a scene where the subset of the set of objects is missing.

18. A non-transitory computer-readable medium comprising instructions, which, when executed by a computing device, cause the computing device to perform operations including: Retrieve records of the physical environment content corresponding to the device's field of view from the database, the content including composite video, the composite video including at least one virtual object combined with a video comprising at least one physical object; Analyzing the recording of the content within the field of view of the device, the analysis includes: Identify a set of objects included in the content. Determine the corresponding motion vector for each corresponding object that moved during the recording period, and Generate one or more annotations corresponding to the identified set of objects; Based on one or more generated annotations, identify a subset of the set of objects indicated to correspond to protected content, wherein the subset of the set of objects includes one or more virtual and one or more physical objects; Generate a modified version of the record, the modified version obfuscating or filtering the subset of the set of objects; and The modified version of the record is provided to the host application for playback.

Images