Computer-generated reality recorder

Abstract

This technical subject provides for the analysis of content records. This technical subject generates metadata information based at least in part on this analysis. This technical subject identifies regions of interest or objects of interest in the content records based at least in part on at least one of user preferences or detected events. Based at least in part on the identified regions of interest or objects of interest, this technical subject generates a modified version of the content record. Furthermore, this technical subject stores the modified version of the content record for subsequent playback on electronic devices.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 897,912, filed on September 9, 2019, entitled “Computer-Generated RealityRecorder,” the entire disclosure of which is incorporated herein by reference. Technical Field

[0003] This instruction manual deals with computer-generated real-world environments. Background Technology

[0004] Augmented reality (AR) technology aims to bridge the gap between virtual and physical environments by providing an augmented physical environment that utilizes electronic information. Therefore, the electronic information appears to be part of the user's perceived physical environment. In this example, AR further provides a user interface for interacting with the electronic information overlaid on the augmented physical environment. Attached Figure Description

[0005] Some features of this subject matter are shown in the appended claims. However, for illustrative purposes, several embodiments of this subject matter are illustrated in the following figures.

[0006] Figure 1 An exemplary system architecture is shown, comprising various electronic devices that can implement the system of this subject, according to one or more specific implementations.

[0007] Figure 2 An exemplary software architecture that can be implemented on an electronic device and / or server according to one or more specific implementations of the subject matter is shown.

[0008] Figure 3A , Figure 3B and Figure 3C An example of analyzing computer-generated real-world records according to a specific implementation of the techniques in this subject matter is shown.

[0009] Figure 4A , Figure 4B and Figure 4C An example of analyzing computer-generated real-world records according to a specific implementation of the techniques in this subject matter is shown.

[0010] Figure 5A , Figure 5B and Figure 5C An example is shown of analyzing computer-generated reality recordings according to a specific implementation of the subject matter technique and providing playback of the computer-generated reality recordings based on the analysis.

[0011] Figure 6A flowchart is shown as an exemplary process for analyzing computer-generated records of real-world content, according to a specific implementation of the subject matter.

[0012] Figure 7 An electronic system is shown that can be used to implement one or more specific embodiments of the subject matter technology. Detailed Implementation

[0013] The specific embodiments shown below are intended to describe various configurations of the subject matter and are not intended to represent the only configuration in which the subject matter can be practiced. The accompanying drawings are incorporated herein and form part of the specific embodiments. The specific embodiments include particular details intended to provide a thorough understanding of the subject matter. However, the subject matter is not limited to the specific details set forth herein, but may be practiced in one or more other specific embodiments. In one or more specific embodiments, structures and components are shown in block diagram form to avoid obscuring the concepts of the subject matter.

[0014] Computer-Generated Reality (CGR) systems enable the combination of physical and virtual environments to varying degrees, thereby facilitating real-time user interaction. Therefore, as described herein, such CGR systems can include a variety of possible combinations of physical and virtual environments, including augmented reality that primarily comprises physical elements and is closer to the physical environment than a virtual environment (e.g., without physical elements). In this way, the physical environment can be connected to the virtual environment via the CGR system. Users immersed in a CGR environment can navigate within it, and the CGR system can track the user's viewpoint to provide visualizations based on the user's location within the environment.

[0015] The physical environment refers to the physical world that people can sense and / or interact with without the aid of electronic systems. Physical environments, such as physical parks, include physical objects such as physical trees, physical buildings, and physical people. People can directly sense and / or interact with the physical environment through senses such as sight, touch, hearing, taste, and smell.

[0016] Conversely, a computer-generated reality (CGR) environment refers to a fully or partially simulated environment that people sense and / or interact with via electronic systems. In CGR, a subset of a person's physical motion, or a representation thereof, is tracked, and in response, one or more characteristics of one or more virtual objects simulated in the CGR environment are adjusted in a manner consistent with at least one physical law. For example, a CGR system may detect a person's body and / or head rotations and, in response, adjust the graphical content and sound field presented to the person in a manner similar to how such field of vision and sound change in a physical environment. In some cases (e.g., for accessibility reasons), the adjustment of characteristics of virtual objects in a CGR environment may be done in response to a representation of physical motion (e.g., a voice command).

[0017] Humans can use any of their senses to sense and / or interact with CGR objects, including sight, hearing, touch, taste, and smell. For example, a person can sense and / or interact with audio objects that create a 3D or spatial audio environment that provides the perception of a point audio source in 3D space. As another example, audio objects can enable audio transparency, which selectively introduces ambient sound from the physical environment, with or without computer-generated audio. In some CGR environments, a person can sense and / or interact only with audio objects.

[0018] Examples of CGR include virtual reality and mixed reality.

[0019] A virtual reality (VR) environment is a simulated environment designed to provide one or more senses entirely based on computer-generated sensory input. A VR environment includes multiple virtual objects that a person can sense and / or interact with. Examples of virtual objects include trees, buildings, and computer-generated images representing human avatars. A person can sense and / or interact with virtual objects in a VR environment through the simulation of their presence within the computer-generated environment and / or through the simulation of a subgroup of physical movements of a person within the computer-generated environment.

[0020] Compared to VR environments, which are designed to be entirely based on computer-generated sensory input, mixed reality (MR) environments are simulated environments designed to incorporate sensory input from the physical environment, or representations thereof, in addition to computer-generated sensory input (e.g., virtual objects). On the virtual continuum, a mixed reality environment is any state between a purely physical environment as one end and a virtual reality environment as the other end, but not including either end.

[0021] In some MR environments, computer-generated sensory input can respond to changes in sensory input from the physical environment. Additionally, some electronic systems used to present the MR environment can track position and / or orientation relative to the physical environment, enabling virtual objects to interact with real objects (i.e., physical objects or representations of them from the physical environment). For example, the system can cause movement so that virtual trees appear stationary relative to the physical ground.

[0022] Augmented reality (AR) environments are simulated environments in which one or more virtual objects are overlaid on a physical environment or a representation thereof. For example, an electronic system for presenting an AR environment may have a transparent or semi-transparent display through which a person can directly view the physical environment. The system can be configured to present virtual objects on the transparent or semi-transparent display, allowing a person to perceive virtual objects overlaid on a portion of the physical environment. Alternatively, the system may have an opaque display and one or more imaging sensors that capture images or videos of the physical environment, which are representations of the physical environment. The system combines the images or videos with virtual objects and presents the combination on the opaque display. A person uses the system to indirectly view the physical environment via images or videos of the physical environment and perceives virtual objects overlaid on and / or behind a portion of the physical environment. As used herein, video of the physical environment displayed on an opaque display is referred to as “pass-through video,” meaning that the system uses one or more image sensors to capture images of the physical environment and uses those images when presenting the AR environment on the opaque display. Alternatively, the system may have a projection system that projects virtual objects onto a physical environment, such as as a hologram or on a physical surface, so that a person can use the system to perceive the virtual objects superimposed on the physical environment.

[0023] Augmented reality environments also refer to simulated environments where the representation of the physical environment is transformed by computer-generated sensory information. For example, in providing pass-through video, a system can transform images from one or more sensors to apply a selected viewpoint (e.g., viewpoint) different from the viewpoint captured by the imaging sensor. Alternatively, the representation of the physical environment can be transformed by graphically modifying (e.g., magnifying) portions of it, such that the modified portion is a representative but not realistic version of the original captured image. Furthermore, the representation of the physical environment can be transformed by graphically removing or blurring portions of it.

[0024] Augmented virtual (AV) environments are simulated environments that combine virtual or computer-generated environments with one or more sensory inputs from a physical environment. Sensory inputs can be representations of one or more characteristics of the physical environment. For example, an AV park could have virtual trees and buildings, but a person's face could be realistically reproduced from an image taken of a physical person. Similarly, virtual objects could adopt the shape or color of a physical object imaged by one or more imaging sensors. Furthermore, virtual objects could use shadows that correspond to the sun's position within the physical environment.

[0025] Many different types of electronic systems enable people to sense and / or interact with a variety of CGR environments. Examples include mobile devices, tablet devices, projection-based systems, head-up displays (HUDs), head-mounted systems, vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays shaped like lenses designed to be placed on the human eye (e.g., similar to contact lenses), head-mounted receivers / headsets, speaker arrays, input systems (e.g., wearable or handheld controllers with or without haptic feedback), smartphones, tablets or tablet devices, and desktop / laptop computers. For example, a head-mounted system may have one or more speakers and an integrated opaque display. Alternatively, a head-mounted system may be configured to receive an external opaque display (e.g., a smartphone). A head-mounted system may incorporate one or more imaging sensors for capturing images or video of the physical environment, and / or one or more microphones for capturing audio of the physical environment. A head-mounted system may have a transparent or semi-transparent display instead of an opaque display. The transparent or semi-transparent display may have a medium through which light representing the image is directed to the human eye. The display can utilize digital light projection, OLED, LED, uLED, liquid crystal on silicon, laser scanning light source, or any combination of these technologies. The medium can be an optical waveguide, holographic medium, optical combiner, optical reflector, or any combination thereof. In one embodiment, a transparent or translucent display can be configured to selectively become opaque. Projection-based systems can employ retinal projection technology, which projects graphic images onto the human retina. Projection systems can also be configured to project virtual objects onto a physical environment, such as as holograms or on a physical surface.

[0026] The specific implementation of the subject matter technology described herein provides a CGR system that offers storage and management of computer-generated reality (CGR) records to facilitate the editing and sharing of CGR content. In the example, CGR content records can be analyzed to detect people and / or objects of interest to the user. Based on the detected people and / or objects, the subject matter system can generate information for creating modified versions of the CGR records, which can also be generated based on user preferences. The subject matter system can also provide playback of CGR content (e.g., modified CGR records) and enable user input to determine playback behavior. Furthermore, playback of CGR content can be determined based on user preferences.

[0027] Figure 1An exemplary system architecture 100 comprising various electronic devices capable of implementing the system of this subject matter, according to one or more embodiments, is shown. However, not all depicted components are usable in all embodiments, and one or more embodiments may include additional or different components compared to those shown in the figures. Variations in the arrangement and type of these components may be made without departing from the spirit or scope of the claims set forth herein. Additional components, different components, or fewer components may be provided.

[0028] System architecture 100 includes electronic device 105, handheld electronic device 104, electronic device 110, electronic device 115, and server 120. For illustrative purposes, system architecture 100 is described in... Figure 1 The system is shown as including electronic device 105, handheld electronic device 104, electronic device 110, electronic device 115, and server 120; however, system architecture 100 may include any number of electronic devices and any number of servers or a data center including multiple servers.

[0029] Electronic device 105 may be implemented, for example, as a tablet device, a handheld and / or mobile device, or (e.g., a head-mounted portable system worn by user 101). Electronic device 105 includes a display system capable of presenting a visualization of a computer-generated realistic environment to the user. Electronic device 105 may be powered by a battery and / or another power source. In the example, the display system of electronic device 105 provides the user with a stereoscopic representation of the computer-generated realistic environment, enabling the provision of a three-dimensional visual display rendering a specific scene. In one or more embodiments, instead of utilizing electronic device 105 to access the computer-generated realistic environment, or otherwise, the user may use a handheld electronic device 104, such as a tablet, watch, mobile device, etc.

[0030] Electronic device 105 may include one or more cameras, such as camera 150 (e.g., visible light camera, infrared camera, etc.). Furthermore, electronic device 105 may include various sensors 152, including but not limited to cameras, image sensors, touch sensors, microphones, inertial measurement units (IMUs), heart rate sensors, temperature sensors, depth sensors (e.g., lidar sensors, radar sensors, sonar sensors, time-of-flight sensors, etc.), GPS sensors, Wi-Fi sensors, near-field communication sensors, radio frequency sensors, etc. Additionally, electronic device 105 may include hardware elements capable of receiving user input, such as hardware buttons or switches. User input detected by such sensors and / or hardware elements corresponds to various input modalities, such as those used to initiate coexistence sessions within an application. For example, such input modalities may include, but are not limited to, face tracking, eye tracking (e.g., gaze direction), hand tracking, posture tracking, biometric readings (e.g., heart rate, pulse, pupil dilation, respiration, temperature, electroencephalogram, olfaction), speech or audio recognition (e.g., specific hot words), and activation of buttons or switches, etc.

[0031] In one or more embodiments, electronic device 105 may be communicatively coupled to a base device, such as electronic device 110 and / or electronic device 115. Generally, such a base device may include more computing resources and / or available power compared to electronic device 105. In the example, electronic device 105 may operate in various modes. For example, electronic device 105 may operate in an independent mode, independent of any base device. When electronic device 105 operates in independent mode, the number of input modes may be constrained by the power and / or processing limitations of electronic device 105, such as the device's available battery power. In response to power limitations, electronic device 105 may deactivate certain sensors within the device itself to conserve battery power and / or free up processing resources.

[0032] Electronic device 105 can also operate in a wireless connection mode (e.g., connected to a base device via a wireless connection), thereby working in conjunction with a given base device. Electronic device 105 can also operate in a connection mode where electronic device 105 is physically connected to a base device (e.g., via a cable or some other physical connector), and can utilize power resources provided by the base device (e.g., when the base device is charging electronic device 105 while physically connected).

[0033] When electronic device 105 operates in wireless or connected mode, at least a portion of processing user input and / or rendering a computer-generated real-world environment can be offloaded to the underlying device, thereby reducing the processing burden on electronic device 105. For example, in a specific implementation, electronic device 105 works in conjunction with electronic device 110 or 115 to generate a computer-generated real-world environment, which includes physical and / or virtual objects that enable various forms of interaction (e.g., visual, auditory, and / or physical or tactile interaction) between the user and the generated computer-generated real-world environment in real time. In the example, electronic device 105 provides a rendering of a scene corresponding to the computer-generated real-world environment, which can be perceived and interacted with by the user in real time, such as a host environment for coexisting with another user. Additionally, as part of presenting the rendered scene, electronic device 105 may provide the user with auditory and / or tactile or haptic feedback. The content of a given rendered scene may depend on available processing power, network availability and capacity, available battery power, and the current system workload.

[0034] Electronic device 105 can also detect events that have occurred within a scene of a computer-generated real-world environment. Examples of such events include detecting the presence of a specific person, entity, or object in the scene. In response to a detected event, electronic device 105 can provide annotations (e.g., in the form of metadata) in the computer-generated real-world environment corresponding to the detected event.

[0035] In one embodiment, electronic device 105 generates a record of the physical environment and generates a computer-generated realistic environment including two-dimensional (2D) video for sharing and playback. In another embodiment, electronic device 105 generates a record and subsequently generates a three-dimensional (3D) representation that incorporates information from all sensors and / or combines the record with records from other users (e.g., different point of view (POV) and / or different field of view (FOV)). It should also be understood that electronic devices 110 and / or 115 may also generate such computer-generated realistic environments, either in conjunction with or independently of electronic device 105.

[0036] Network 106 may communicatively (directly or indirectly) couple electronic devices 104, 105, 110, and / or 115 to each other device and / or server 120. In one or more embodiments, network 106 may be an interconnection network of devices that may include the Internet or be communicatively coupled to the Internet.

[0037] Electronic device 110 may include a touchscreen and may be, for example, a smartphone including a touchscreen, a portable computing device such as a laptop computer including a touchscreen, an accessory including a touchscreen (e.g., a digital camera, headphones), a tablet device including a touchscreen, a wearable device including a touchscreen (e.g., a watch, wristband, etc.), any other suitable device including, for example, a touchscreen, or any electronic device with a touchpad. In one or more embodiments, electronic device 110 may not include a touchscreen but may support touchscreen-like gestures, such as in a computer-generated real-world environment. In one or more embodiments, electronic device 110 may include a touchpad. Figure 1 In this embodiment, by way of example, electronic device 110 is depicted as a mobile smartphone device with a touchscreen. In one or more embodiments, electronic device 110, handheld electronic device 104, and / or electronic device 105 may be and / or may include all or part of the electronic device described below with respect to an electronic system, which is described below with respect to an electronic system. Figure 7 As described above. In one or more embodiments, electronic device 110 may be another device, such as an Internet Protocol (IP) camera, a tablet computer, or an accessory such as an electronic stylus.

[0038] Electronic device 115 may be, for example, a desktop computer, portable computing devices such as laptops, smartphones, accessories (e.g., digital cameras, headphones), tablet devices, wearable devices such as watches, wristbands, etc. Figure 1 In this context, by way of example, electronic device 115 is described as a desktop computer. Electronic device 115 may be and / or may include, as described below in relation to... Figure 7 All or part of the aforementioned electronic system.

[0039] Server 120 may form all or part of a computer network or server group 130, such as in a cloud computing or data center implementation. For example, server 120 stores data and software and includes specific hardware (e.g., processors, graphics processors, and other dedicated or custom processors) for rendering and generating computer-generated reality content such as graphics, images, videos, audio, and multimedia files. In a specific implementation, server 120 may function as a cloud storage server that stores any of the aforementioned computer-generated reality content generated by the aforementioned devices and / or server 120.

[0040] In specific implementation, server 120 can provide a security environment corresponding to the secure cloud environment for analyzing CGR content records, as discussed further below. Such records can be provided by at least one of the following: electronic device 105, electronic device 110, and / or electronic device 115. Similarly, electronic device 105, electronic device 110, and / or electronic device can also provide the following described below. Figure 2 The security environment used for analyzing CGR content records is discussed further in this paper.

[0041] Figure 2 An exemplary software architecture 200, which may be implemented on electronic device 105, electronic device 115, and / or server 120 according to one or more specific embodiments of the subject matter, is illustrated. For illustrative purposes, the software architecture 200 is described as being composed of... Figure 1 The software architecture 200 is implemented by electronic devices 105, 115, and 120, such as through the processors and / or memory of electronic devices 105, 115, and 120; however, suitable portions of the software architecture 200 may be implemented by any other electronic device, including electronic device 110. However, not all depicted components are usable in all embodiments, and one or more embodiments may include additional or different components compared to those shown in the figures. Variations in the arrangement and type of these components may be made without departing from the spirit or scope of the claims set forth herein. Additional components, different components, or fewer components may be provided.

[0042] Software architecture 200 implemented on electronic device 105 includes a framework. As used herein, a framework can refer to a software environment that provides specific functionality as part of a larger software platform to facilitate the development of software applications, and can provide one or more application programming interfaces (APIs) that developers can use to programmatically design computer-generated real-world environments and handle operations for such computer-generated real-world environments.

[0043] As shown in the figure, a recording frame 230 is provided. The recording frame 230 provides functionality for recording the computer-generated real-world environment provided by the input modalities as described above. An event detector 220 is provided, which receives information corresponding to inputs from various input modalities. A system manager 210 is provided to monitor resources from the electronic device 105 and determine quality of service metrics based on available resources. The system manager 210 can make decisions to select specific hardware components corresponding to a given input modality to activate and / or deactivate based on the quality of service metrics. For example, a camera tracking facial expressions may be turned off, or another camera tracking gestures may be turned off. When specific hardware is deactivated, the electronic device 105 can provide a notification to warn the user that the specific input modality is unavailable. Similarly, the electronic device 105 can provide a notification to warn the user that a specific input modality is available when the specific hardware is activated.

[0044] In practical implementation, software architecture 200 enables the management of CGR records by providing a repository or storage location for uploading such records. For example... Figure 2As shown, the electronic device 105 includes a database 240 for storing CGR records 242, which may be provided by a recording frame 230 and / or a secure environment 250, as further described below. In this example, the recording frame 230 may generate CGR records and encrypt such records for storage in the database 240.

[0045] As further shown, electronic device 105 includes a secure environment 250. As used herein, the secure environment provides a secure computing environment isolated from other processes executing on a given processor of a particular device (e.g., electronic device 105), wherein the secure environment is inaccessible to both the processor executing the other processes and the other processes themselves. Furthermore, the secure environment encrypts any portion of the memory it utilizes and can utilize a separate processor (e.g., a custom security-oriented processor) compared to the processor executing such other processes (e.g., on electronic device 105). The secure environment can also use homomorphic encryption when processing a given CGR record. Additionally, the secure environment can be enabled to store... Figure 1 Decrypt the encrypted CGR record in any of the electronic devices shown.

[0046] The specific implementation of software architecture 200 and its components enables automatic segmentation of CGR records based on content and annotations, which also enable record searching. Furthermore, software architecture 200 and its components enable recommendations for other records, at least in part, based on topics and / or user interests, which can be provided from user preferences stored in associated user profiles. Software architecture 200 and its components enable the generation of multiple records based on recommendations or user preferences, and also enable random browsing of records (pause, playback, drag, etc.).

[0047] In the example, the secure environment 250 may receive a given CGR record from the recording frame 230, or receive a request from the recording frame 230 to retrieve a CGR record stored in the database 240. As shown, the secure environment 250 includes a secure record analyzer 260 that analyzes the record and generates metadata and / or annotations based on the analysis. In a specific implementation, the secure record analyzer 260 may perform a specific classifier for object detection and determine the motion vector of the object if such an object moves during recording. In the example, such motion vectors may be used to track objects and / or predict where the object may move. The secure record analyzer 260 provides annotations (e.g., tagged with metadata information) to the object and detected events and / or identified people. In another example, the secure record analyzer 260 determines acoustic information about a person or object from speech, sound, and / or audio, which can be used for movement or location tracking and / or identification of the person or object. Furthermore, the analysis can be customized based on the recording type (e.g., motion, specific location such as a beach, etc.) to target specific events or objects that may exist in records of that type.

[0048] As also shown in the figure, server 120 provides a secure cloud environment 270, which corresponds to a specific type of secure environment in a particular implementation. For example, secure cloud environment 270 supports a set of processor instruction codes that enable user-level code to allocate private regions of protected and isolated memory (e.g., environment) from other processes. Secure cloud environment 270 can use its own set of processors (e.g., CPU and / or GPU). In a specific implementation, server 120 includes a secure cloud record analyzer 280 that performs the same or similar functions as the secure record analyzer 260 discussed above with respect to a given CGR record. In the example, server 120 may receive a request from electronics 105 to perform analysis on a given CGR record. The results of any analysis performed by secure record analyzer 260 (e.g., generated metadata information or recommendations) may be returned to electronics 105 for further processing.

[0049] In a specific implementation, electronic device 115 includes a security environment 255 and a security record analyzer 265, each of which can perform the same functions as described above in conjunction with electronic device 105: security environment 250 and security record analyzer 260. For example, security record analyzer 260 can analyze a CGR record from one of the CGR records 247 stored in database 246.

[0050] As shown in the figure, the electronic device 115 includes a set of tools 245, corresponding to various editing tools (e.g., specific applications) that provide editing functions to modify CGR records. Each editing tool is capable of receiving a CGR record (e.g., corresponding to a CGR record 247 stored in a database 246) for editing. Examples of editing tools include image editors, tools for 3D composition, and / or multimedia editors capable of organizing CGR content along a timeline. A given editing tool can also enable the insertion of new objects or content into the scene, including new virtual content.

[0051] Despite Figure 2 Although not shown, it should be understood that electronic device 110 may also include similar components as discussed above in conjunction with electronic device 115.

[0052] Figure 3A , Figure 3B , Figure 3C , Figure 4A , Figure 4B , Figure 4C , Figure 5A , Figure 5B and Figure 5C The following discussion involves examples of analyzing various computer-generated real-world records, including detecting objects and people and generating annotations to incorporate into modified records for playback.

[0053] Figure 3A , Figure 3B and Figure 3C An example of analyzing computer-generated real-world records according to a specific implementation of the techniques described in this subject matter is shown. The following discussion relates to... Figure 2 The software architecture 200 components.

[0054] like Figure 3A As shown, scene 310 illustrates a sporting event (e.g., a football match) included in a computer-generated reality (CGR) record. In scene 310, individual objects corresponding to three football players on the field appear together with a group of spectators in the background.

[0055] exist Figure 3B In this context, a security record analyzer (e.g., security record analyzer 260 or 265, or security cloud record analyzer 280) analyzes scenario 310 and detects the presence of people 312 (e.g., a football player without a football), people 314 (e.g., a football player near a football), a football 316, and people 318 (e.g., a goalkeeper). In response, the security record analyzer can generate associated metadata information corresponding to the detected people.

[0056] In its implementation, the security record analyzer detects objects in scenario 310 at least in part by running an object classifier on the CGR record, and identifies objects using at least an object database, which may include information for identifying objects (e.g., in the case of people or other entities, or for providing further contextual information related to the object). The security record analyzer can identify objects using multiple attributes, including available metadata information, shape, physical properties, audio / sound, etc. In this example, person 314 can be identified as a specific star football player, and information can be generated to indicate that specific star football player. In this way, associated metadata information for the object is generated at least in part based on object identification. As mentioned above, when scenario 310 involves a sporting event, the analysis of the security record analyzer can be customized to better analyze the sporting event (e.g., detecting athletes, detecting the path of the ball, detecting specific events that may occur during the match, etc.). Using the associated metadata information, the record frame 230 or editing tools can generate a modified version of the CGR record.

[0057] exist Figure 3C In this context, a modified version of scenario 310, corresponding to scenario 320, may have been generated (e.g., via a specific editing tool or host application or record frame 230) based at least in part on information generated from the aforementioned analysis of the CGR record. In scenario 320, annotations 330 and 332 are provided for the identified star football player (e.g., person 314) and person 312. Such annotations can be presented to the user during playback of the modified version of the CGR record to further enhance the CGR environment experience. In one or more embodiments, the annotations may be stored in association with corresponding objects and can subsequently be used to identify the corresponding objects in the CGR record.

[0058] Figure 4A , Figure 4B and Figure 4C An example of analyzing computer-generated real-world records according to a specific implementation of the techniques described in this subject matter is shown. The following discussion relates to... Figure 2 The software architecture 200 components.

[0059] like Figure 4A As shown, scene 410 illustrates a specific outdoor environment (e.g., a beach) included in a computer-generated reality (CGR) record. In scene 410, various objects corresponding to specific items on the beach appear together with other objects in the background corresponding to specific bodies of water (e.g., the ocean).

[0060] exist Figure 4BScene 420, included in the computer-generated reality (CGR) record and located temporally after scene 410, shows the presence of a person not present in scene 410. A security record analyzer (e.g., security record analyzer 260 or 265, or security cloud record analyzer 280) analyzes scene 420 and detects events corresponding to the newly present person 422 (e.g., a child). In response, the security record analyzer can generate associated metadata information corresponding to the detected event (e.g., the presence of person 422). In this example, person 422 can be further identified as a specific person of interest (e.g., a family member) based on user preferences (e.g., annotated when a family member enters the CGR record), and information can be generated to indicate events related to the family member. In another example, the security record analyzer can detect anomalies related to people and / or events in the scene and generate corresponding metadata as annotations for the scene. Such anomalies can also be detected based on user preferences (e.g., annotated when a specific event occurs, such as detecting a baby starting to cry).

[0061] Additionally, the security record analyzer analyzes scene 420 and detects object 424 corresponding to a paraglider in the background of scene 420. In the example, user preferences may indicate that the user is interested in paragliders (e.g., the user wants any object detected as a paraglider to be highlighted in a beach outdoor environment). The security record analyzer can generate associated metadata information corresponding to the detected objects (e.g., the presence of a paraglider).

[0062] Using the associated metadata information, record frame 230 or editing tools can generate a modified version of the CGR record. Figure 4C In the process, a modified version of scene 420 corresponding to scene 430 may have been generated, at least in part, based on information generated from the analysis of the CGR record described above (e.g., via a specific editing tool or host application or recording frame 230). In scene 430, annotations 432 and 434 are provided corresponding to the identified family member (e.g., person 422) and the detected object 424 (e.g., paraglider). Such annotations may be presented to the user during playback of the modified version of the CGR record to further enhance the CGR environment experience and indicate the detected events and / or objects.

[0063] Figure 5A , Figure 5B and Figure 5C An example is shown of analyzing computer-generated reality recordings according to a specific implementation of the techniques described in this subject matter, and providing playback of the computer-generated reality recordings based on this analysis. The following discussion relates to... Figure 2 The software architecture 200 components.

[0064] like Figure 5AAs shown, scene 510 illustrates a sporting event (e.g., a hockey match) included in a computer-generated real-world recording. Spectators are positioned in the background of scene 510, and hockey players are shown on an ice rink in the foreground of scene 510. A secure recording analyzer (e.g., secure recording analyzer 260 or 265, or secure cloud recording analyzer 280) analyzes scene 510 and detects object 504 corresponding to the hockey puck. Furthermore, in this example, the user has selected a region of interest corresponding to object 504, which can be used to focus on object 504 during playback, as discussed further below.

[0065] The security record analyzer has detected and identified Figure 5A The existence of a specific person 506 (e.g., a star hockey player) and a person 502 (a hockey player's competitor) is identified. The security record analyzer can generate associated metadata information corresponding to the aforementioned objects or people. In this example, the security record analyzer detects the presence of a moving person (e.g., a hockey player) or a moving object (e.g., an ice hockey puck) in the record and determines the motion vector of the moving person or object. Using the associated metadata information, the record frame 230 or an editing tool can generate a modified version of the CGR record. In this example, the record frame 230 or the editing tool can generate a three-dimensional (3D) mesh of at least one of the identified objects in the record, wherein the modified CGR record is based on a 3D representation of the record, which includes a rendering of the 3D mesh of at least one of the identified objects. In another example, an acoustic mesh can be generated based on the analysis of audio content (e.g., speech, sound, audio) from the scene, which can be used to identify objects or people in the CGR record and / or determine the location of such objects or people and any movement within the scene from the record.

[0066] In some implementations, during playback of a given modified CGR record, navigation to certain events and interaction with objects are enabled based on generated metadata / annotations. Playback from different viewpoints can also be provided if multiple records of the same environment are available. Annotations can be presented during playback and based on events occurring in the scene. Furthermore, users can identify objects of interest during playback, and the type of playback representation can be determined based on user preferences (e.g., the path of a ball, focusing on specific people of interest, etc.). During playback, users can indicate or infer areas of interest (e.g., people entering the scene). It should also be understood that playback of modified CGR records is not time-limited (e.g., as in typical playback of records without CGR components) and can be based on user playback and order. Based on user preferences, playback can focus on an individual and / or on things deemed important in the scene. Playback can further prioritize a type of modality based on certain events (e.g., broken glass in the scene, specific people), and / or prioritize playback on devices with better capabilities.

[0067] Figure 5B and Figure 5C This involves different scenarios during the playback of modified CGR records. Figure 5B Scene 520 shows a modified presentation of the same hockey match from Scene 510. For example, based on user preferences, Scene 520 focuses on specific individuals 506, 502, and regions of interest corresponding to the hockey puck (e.g., object 504). In this example, a 3D mesh-based representation of the hockey puck can be presented to the user during playback to better highlight object 504 and enhance the user experience of watching the modified CGR recording. Furthermore, the spectators in the background of Scene 510, as well as other hockey players present in Scene 510, have been erased (or removed) in Scene 520. Additionally, specific individual 506 is presented differently in Scene 520 to better highlight the actions of star hockey players. New digital content corresponding to digital objects 524 and 526 has also been inserted into Scene 520 to provide contextual information related to the hockey match. Scene 520 includes annotations 522 describing the events occurring within Scene 520.

[0068] exist Figure 5C In scene 530, a specific person 506 moves to a different location than in scene 520 and is completely within the view frame of scene 530. Scene 530 includes annotation 532 describing the detected events occurring in scene 530.

[0069] Figure 6A flowchart illustrating an exemplary process 600 for analyzing computer-generated real-world content records according to a specific implementation of the subject matter is shown. For illustrative purposes, this document primarily refers to... Figure 1 and Figure 2 The process 600 is described using electronic device 105. However, process 600 is not limited to... Figure 1 and Figure 2 The electronic device 105, and one or more blocks (or operations) of process 600 may be performed by one or more other components of other suitable devices (including electronic device 105, electronic device 110, and / or server 120). Further for illustrative purposes, the blocks of process 600 are described herein as occurring sequentially or linearly. However, multiple blocks of process 600 may occur in parallel. Furthermore, the blocks of process 600 need not be performed in the order shown, and / or one or more blocks of process 600 need not be performed and / or may be replaced by other operations.

[0070] like Figure 6 As shown, electronic device 105 receives a content record (610). Electronic device 105 analyzes the content record, wherein the analysis includes identifying objects in the content record and detecting events occurring in the content record (612). Electronic device 105 generates metadata information based at least in part on the analysis, which identifies the identified objects and detected events in the record (614). Electronic device 105 identifies regions of interest or objects of interest in the content record based at least in part on at least one of user preferences or detected events (616). Electronic device 105 generates a modified version of the content record based at least in part on the identified regions of interest or objects of interest, which incorporates at least the generated metadata information (618). Electronic device 105 stores the modified version of the content record for subsequent playback, for example, in conjunction with the generated metadata (620). Electronic device 105 provides the modified version of the record for playback (622).

[0071] As described above, one aspect of this technology involves collecting and using data from various sources. This disclosure anticipates that, in some instances, such collected data may include personal information that uniquely identifies or can be used to contact or locate specific individuals. This personal information may include demographic data, location-based data, telephone numbers, email addresses, social network identifiers, home addresses, data or records related to a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying or personal information.

[0072] This disclosure recognizes that the use of such personal information data in the techniques of this invention can be used to benefit users. This disclosure also contemplates uses of personal information data that are beneficial to users. For example, health and fitness data can be used to provide insights into a user's overall health status or as positive feedback for individuals using the technology to pursue health goals.

[0073] This disclosure assumes that entities responsible for collecting, analyzing, disclosing, transmitting, storing, or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and practices that are generally recognized as meeting or exceeding industry or governmental requirements for maintaining the privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as data collection and / or use change. Personal information from users should be collected for the entity's lawful and reasonable purposes and not shared or sold outside of these lawful uses. Furthermore, such collection / sharing should be conducted only after obtaining informed consent from users. In addition, such entities should consider taking any necessary steps to protect and safeguard access to such personal information data and ensure that others with access to such personal information data comply with their privacy policies and processes. Additionally, such entities may be subject to third-party evaluations to demonstrate their compliance with widely accepted privacy policies and practices. Furthermore, policies and practices should be adapted to the specific types of personal information data collected and / or accessed, and to applicable laws and standards, including specific considerations regarding jurisdiction. For example, in the United States, the collection or acquisition of certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while in other countries, health data may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy practices should be maintained for different types of personal data in each country.

[0074] Regardless of the foregoing, this disclosure also anticipates implementation schemes for users to selectively block the use or access to their personal information data. That is, this disclosure anticipates providing hardware and / or software components to prevent or block access to such personal information data. For example, this technology can be configured to allow users to opt-in or opt-out at any time during or after registering for the service. In addition to providing opt-in and opt-out options, this disclosure envisions providing notifications related to access to or use of personal information. For example, users may be notified when downloading an application that their personal information data will be accessed, and then reminded again just before the application accesses the personal information data.

[0075] Furthermore, the purpose of this disclosure is to manage and process personal information data to minimize the risk of unintentional or unauthorized access or use. Once data is no longer needed, this risk can be minimized by limiting data collection and deleting data. Additionally, and where applicable, including in certain health-related applications, data deidentification can be used to protect user privacy. Deidentification can be facilitated, where appropriate, by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than the address level), controlling how data is stored (e.g., aggregating data among users), and / or other methods.

[0076] Therefore, while this disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, it is also contemplated that various embodiments can be implemented without access to such personal information data. That is, various embodiments of the present invention will not be rendered inoperable due to the absence of all or part of such personal information data. For example, content can be selected and delivered to the user based on non-personal information data or a small amount of personal information, such as content requested by a device associated with the user, other non-personal information, or publicly available information.

[0077] Figure 7 An electronic system 700 is shown that can be used to implement one or more specific embodiments of the subject matter technology. The electronic system 700 may be... Figure 1 The illustrated electronic devices 105, 104, 110, 115, and / or server 120 are shown and / or may be part of them. Electronic system 700 may include various types of computer-readable media and interfaces for various other types of computer-readable media. Electronic system 700 includes a bus 708, one or more processing units 712, system memory 704 (and / or cache), ROM 710, persistent storage device 702, input device interface 714, output device interface 706, and one or more network interfaces 716, or subsets and variations thereof.

[0078] Bus 708 generally represents all system buses, peripheral bus, and chipset bus that communicatively connect numerous internal devices of electronic system 700. In one or more embodiments, bus 708 communicatively connects one or more processing units 712 to ROM 710, system memory 704, and permanent storage device 702. One or more processing units 712 retrieve instructions to be executed and data to be processed from these various memory units in order to perform the processes disclosed in this subject matter. In different embodiments, one or more processing units 712 may be a single processor or a multi-core processor.

[0079] ROM 710 stores static data and instructions required by one or more processing units 712 and other modules of electronic system 700. On the other hand, persistent storage device 702 can be a read-write memory device. Persistent storage device 702 can be a non-volatile memory cell that stores instructions and data even when electronic system 700 is powered off. In one or more embodiments, mass storage devices (such as magnetic disks or optical disks and their corresponding disk drives) can be used as persistent storage device 702.

[0080] In one or more embodiments, a removable storage device (such as a floppy disk, flash drive, and its corresponding disk drive) may be used as persistent storage device 702. Like persistent storage device 702, system memory 704 may be a read-write memory device. However, unlike persistent storage device 702, system memory 704 may be volatile read-write memory, such as random access memory. System memory 704 may store any instructions and data that one or more processing units 712 may need during operation. In one or more embodiments, the processes disclosed in this subject matter are stored in system memory 704, persistent storage device 702, and / or ROM 710. One or more processing units 712 retrieve instructions to be executed and data to be processed from these various memory units to execute the processes of one or more embodiments.

[0081] Bus 708 is also connected to input device interface 714 and output device interface 706. Input device interface 714 enables a user to transmit information and select commands to electronic system 700. Input devices that can be used with input device interface 714 may include, for example, an alphanumeric keypad and pointing devices (also known as "cursor control devices"). Output device interface 706 may, for example, enable the display of images generated by electronic system 700. Output devices that can be used with output device interface 706 may include, for example, printers and display devices such as liquid crystal displays (LCDs), light-emitting diode (LED) displays, organic light-emitting diode (OLED) displays, flexible displays, flat panel displays, solid-state displays, projectors, or any other device for outputting information. One or more embodiments may include devices that act as both input and output devices, such as touchscreens. In these embodiments, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, voice, or tactile input.

[0082] Finally, as Figure 7 As shown, bus 708 also couples electronic system 700 to one or more networks and / or to one or more network nodes via one or more network interfaces 716, such as Figure 1The electronic device 110 is shown. In this way, the electronic system 700 may be part of a computer network (such as a LAN, a wide area network (“WAN”), or an intranet), or may be part of a network of networks (such as the Internet). Any or all components of the electronic system 700 may be used in conjunction with the subject matter disclosed herein.

[0083] These functions can be implemented in computer software, firmware, or hardware. The technology can be implemented using one or more computer program products. Programmable processors and computers can be included in or packaged as mobile devices. The process and logic flow can be executed by one or more programmable processors and one or more programmable logic circuits. General-purpose and special-purpose computing devices, as well as storage devices, can be interconnected via communication networks.

[0084] Some specific implementations include electronic components such as microprocessors, storage devices, and memories that store computer program instructions in machine-readable or computer-readable media (also known as computer-readable storage media, machine-readable media, or machine-readable storage media). Examples of such computer-readable media include RAM, ROM, read-only optical discs (CD-ROM), recordable optical discs (CD-R), rewritable optical discs (CD-RW), read-only digital versatile optical discs (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and / or solid-state hard disk drives, read-only and recordable... Disks, high-density optical discs, any other optical or magnetic media, and floppy disks. Computer-readable media may store computer programs that can be executed by at least one processing unit and include a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as machine code generated by a compiler, and files that include higher-level code that can be executed by a computer, electronic components, or microprocessor using an interpreter.

[0085] While the above discussion primarily concerns microprocessors or multi-core processors that execute software, some implementations are performed by one or more integrated circuits such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some implementations, such integrated circuits execute instructions stored on the circuit itself.

[0086] As used in this specification and any claim of this patent application, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms exclude persons or groups of persons. For the purposes of this specification, the terms "displayed" or "being displayed" mean displayed on an electronic device. As used in this specification and any claim of this patent application, the terms "computer-readable medium" and "computer-readable media" are entirely limited to tangible, touchable objects that store information in a form readable by a computer. These terms do not include any wireless signals, wired download signals, or any other transient signals.

[0087] To provide interaction with the user, specific embodiments of the subject matter described in this specification can be implemented on a computer having a display device for displaying information to the user and a keyboard and pointing device that the user can use to provide input to the computer. The display device is, for example, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, and the pointing device is, for example, a mouse or trackball. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, voice, or tactile input. Furthermore, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser in response to a request received from a web browser on the user's client device.

[0088] Implementations of the subject matter described in this specification can be implemented in a computing system that includes back-end components, such as a data server, or middleware components, such as an application server, or front-end components, such as a client computer with a graphical user interface or web browser through which a user can interact with a specific implementation of the subject matter described in this specification, or any combination of one or more such back-end components, middleware components, or front-end components. The components of the system can be interconnected via digital data communication of any form or medium, such as a communication network. Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), interconnected networks (e.g., the Internet) and peer-to-peer networks (e.g., self-organizing peer-to-peer networks).

[0089] A computing system may include clients and servers. Clients and servers are generally geographically separated but can interact via a communication network. The client-server relationship is established by computer programs running on respective computers and having a client-server relationship with each other. In some specific implementations, the server transmits data (e.g., HTML pages) to the client device (e.g., to display data to a user interacting with that client device and to receive user input from that user). Data generated at the client device (e.g., the result of user interaction) can be received from the client device at the server.

[0090] Those skilled in the art will recognize that the various exemplary blocks, modules, elements, components, methods, and algorithms described herein can be implemented as electronic hardware, computer software, or a combination of both. To illustrate this interchangeability between hardware and software, the various exemplary blocks, modules, elements, components, methods, and algorithms have been generally described above in terms of functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The functionality can be implemented differently for each specific application. Various components and blocks can be arranged differently (e.g., in different orders or divided in different ways) without departing from the scope of the subject matter.

[0091] It should be understood that the specific order or hierarchical structure of the steps in the process disclosed in this invention is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchical structure of the steps in the process can be rearranged. Some steps in the process can be performed simultaneously. The appended method claims present elements of various steps in the sample order and are not intended to limit one to the specific order or hierarchical structure presented.

[0092] The preceding description is provided to enable those skilled in the art to practice the various aspects described herein. The preceding description provides various examples of the subject matter, and the subject matter is not limited to these examples. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, this claim is not intended to be limited to the aspects shown herein, but rather to be consistent with the language of the claim, wherein reference to elements in the singular form is not intended to mean “only one,” but rather “one or more,” unless specifically indicated. Unless otherwise specifically stated, the term “some” means one or more. Male pronouns (e.g., his) include female and neutral (e.g., her and its), and vice versa. Titles and subtitles (if any) are for convenience only and do not limit the invention described herein.

[0093] As used herein, the term "website" can include any aspect of a website, including one or more web pages, one or more servers used to host or store web-related content, etc. Therefore, the term "website" is used interchangeably with the terms "web page" and "server." The predicates "configured to," "capable of operating," and "programmed to" do not imply any specific tangible or intangible modification to a particular subject but are intended to be used interchangeably. For example, a component or a processor configured to monitor and control operations may also mean that the processor is programmed to monitor and control operations or that the processor is capable of operating to monitor and control operations. Similarly, a processor configured to execute code can be interpreted as a processor programmed to execute code or capable of operating to execute code.

[0094] As used herein, the term "automatic" can include execution by a computer or machine without user intervention; for example, by instructions in response to a predicate action of a computer or machine or other initiating mechanism. The word "example" is used herein to mean "serving as an example or illustration." No aspect or design described herein as an "example" is necessarily to be construed as preferred or advantageous over other aspects or designs.

[0095] Phrases such as "aspect" do not imply that this aspect is essential to the subject matter or that this aspect applies to all configurations of the subject matter. Disclosures relating to an aspect may apply to all configurations, or one or more configurations. An aspect may provide one or more examples. Phrases such as "aspect" may refer to one or more aspects, or vice versa. Phrases such as "implementation" do not imply that this implementation is essential to the subject matter or that this implementation applies to all configurations of the subject matter. Disclosures relating to an implementation may apply to all implementations, or one or more implementations. An implementation may provide one or more examples. Phrases such as "implementation" may refer to one or more implementations, or vice versa. Phrases such as "configuration" do not imply that this configuration is essential to the subject matter or that this configuration applies to all configurations of the subject matter. Disclosures relating to a configuration may apply to all configurations or one or more configurations. A configuration may provide one or more examples. Phrases such as "configuration" may refer to one or more configurations, or vice versa.

Claims

1. A method for analyzing computer-generated reality (CGR) content records, the method comprising: Receive the CGR content record, which has been captured by an electronic device and includes one or more virtual objects; Analyzing the CGR content records, the analysis includes identifying objects within the CGR content records; The region of interest or object of interest in the CGR content record is identified, at least in part, based on the detected events; At least in part based on the identified region of interest or object of interest, a modified version of the CGR content record for subsequent playback is generated, the generation including: At least a portion of the recorded data, excluding the identified region of interest or object of interest, is blurred, wherein in the modified version used for subsequent playback, the blurred portion is not visually highlighted relative to the identified region of interest or object of interest; and The modified version of the CGR content record is stored for subsequent playback.

2. The method according to claim 1, further comprising: The modified version of the record is provided to the electronic device for playback.

3. The method according to claim 1, wherein analyzing the CGR content record further comprises: The detection of objects is based at least in part on running an object classifier on the record; as well as At least an object database is used to identify the object, the object database including information for identifying the object using multiple attributes of the object.

4. The method according to claim 3, further comprising: The associated metadata information of the object is generated, at least in part, based on the identification of the object.

5. The method according to claim 1, wherein analyzing the CGR content record further comprises: The records show the presence of moving people or moving objects. Determine the motion vector of the moving person or the moving object; as well as Determine acoustic information corresponding to the moving person or the moving object from at least one of speech, sound, or audio.

6. The method of claim 1, wherein generating the modified version of the CGR content record further comprises: Generate a three-dimensional (3D) mesh of at least one of the objects identified in the record; Generate a 3D representation of the CGR content record, the 3D representation including rendering the 3D mesh of the at least one of the identified objects; and Acoustic meshes for generating scenarios based on acoustic information.

7. The method of claim 1, wherein identifying the region of interest based at least in part on the detected event further comprises: The records show the presence of a specific person. The specific person being identified as an interested party is determined at least in part based on user preferences; Detect anomalies related to the specific person or the detected event in the record; as well as Generate metadata corresponding to the detected anomalies.

8. The method of claim 1, wherein identifying the region of interest based at least in part on the detected event further comprises: The record contains a specific object; as well as The specific object is identified as an object of interest based at least in part on user preferences.

9. The method of claim 2, wherein the playback of the modified version of the record is at least partially based on focusing on the region of interest.

10. The method of claim 1, wherein the analysis of the CGR content records is provided in a secure environment isolated from other execution processes.

11. A system for analyzing computer-generated reality (CGR) content records, the system comprising: processor; A memory device comprising instructions that, when executed by the processor, cause the processor to perform operations including: Receive the CGR content record, which has been captured by an electronic device and includes one or more virtual objects; Analyzing the CGR content records, the analysis includes identifying objects within the CGR content records; The region of interest or object of interest in the CGR content record is identified, at least in part, based on the detected events; At least in part based on the identified region of interest or object of interest, a modified version of the CGR content record for subsequent playback is generated, the generation including: At least a portion of the recorded data, excluding the identified region of interest or object of interest, is blurred, wherein in the modified version used for subsequent playback, the blurred portion is not visually highlighted relative to the identified region of interest or object of interest; and The modified version of the CGR content record is stored for subsequent playback.

12. The system of claim 11, wherein the memory device further comprises instructions, which, when executed by the processor, cause the processor to perform further operations, the further operations including: The modified version of the record is provided to the electronic device for playback.

13. The system of claim 11, wherein analyzing the CGR content record causes the processor to perform further operations, the further operations including: The detection of objects is based at least in part on running an object classifier on the record; At least an object database is used to identify the object, the object database including information for identifying the object using multiple attributes of the object.

14. The system of claim 13, wherein the memory device further comprises instructions that, when executed by the processor, cause the processor to perform further operations, the further operations including: The associated metadata information of the object is generated, at least in part, based on the identification of the object.

15. The system of claim 11, wherein analyzing the CGR content record causes the processor to perform further operations, the further operations including: The records show the presence of moving people or moving objects. as well as Determine the motion vector of the moving person or the moving object.

16. The system of claim 11, wherein generating the modified version of the CGR content record causes the processor to perform further operations, the further operations including: Generate a three-dimensional (3D) mesh of at least one of the objects identified in the record; Generate a 3D representation of the CGR content record, the 3D representation including rendering of the 3D mesh of the at least one of the identified objects; Acoustic information is determined from the recorded speech, sound, or audio. as well as An acoustic mesh of the recorded scene is generated, at least in part, based on the acoustic information.

17. The system of claim 11, wherein identifying the region of interest based at least in part on the detected event causes the processor to perform further operations, the further operations further comprising: The records show the presence of a specific person. The specific person being identified as an interested party is determined at least in part based on user preferences; Detect anomalies related to the specific person or detected event in the record; as well as Generate metadata corresponding to the detected anomalies.

18. The system of claim 11, wherein identifying the region of interest based at least in part on the detected event causes the processor to perform further operations, the further operations further comprising: The record contains a specific object; as well as The specific object is identified as an object of interest based at least in part on user preferences.

19. The system of claim 12, wherein playback of the modified version of the record is based at least in part on focusing on the region of interest.

20. A non-transitory computer-readable medium including instructions, which, when executed by a computing device, cause the computing device to perform a method for analyzing a computer-generated reality (CGR) content record, the method comprising: Receive the CGR content record, which has been captured by an electronic device and includes one or more virtual objects; Analyzing the CGR content records, the analysis includes identifying objects within the CGR content records; The region of interest or object of interest in the CGR content record is identified, at least in part, based on the detected events; At least in part based on the identified region of interest or object of interest, a modified version of the CGR content record for subsequent playback is generated, the generation including: At least a portion of the recorded data, excluding the identified region of interest or object of interest, is blurred, wherein in the modified version used for subsequent playback, the blurred portion is not visually highlighted relative to the identified region of interest or object of interest; and The modified version of the CGR content record is stored for subsequent playback.

Images